Skip to content

feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0) - #299

Merged
xjacka merged 18 commits into
mainfrom
claude/escaped-defects-followup
Nov 6, 2026
Merged

xjacka merged 18 commits into
mainfrom
claude/escaped-defects-followup

Conversation

@xjacka

@xjacka xjacka commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

This PR follows #298, which is merged. #298 teaches the review the class of each escaped defect. This PR shows whether the classes work, and where defects escaped.

It adds three things: review hits per class, escapes per directory in the profile, and an offer to move a class into the tailored skill when the class failed again.

Reviewer guide

What changes in substance

  • findings-json gets an optional escape field with the class that a finding matched. escape.sh stats counts these hits per class, and audit task 39 reports them. Old lines without the field stay valid.
  • The ledger row gets paths, the repository paths of the failing code. The profile's ## History table gets an escapes column per directory, counted for all time and not for the 90-day review window. A directory that has only escapes gets its own row. Escapes rank first in the table.
  • A class that is bound to this repository and failed again goes to Action needed as a tailored-skill rule. The operator decides. The skill's Change round pre-selects the class with the source escape <date>.

Where to look closely

  • The profile ranking weighs one escape like one critical finding (open). A directory with old escapes can push out a directory with current findings when there are more than 40 rows.
  • A hit is counted only when the reviewer writes escape in findings-json. Nothing checks this field, so the count is a floor (open).
  • A recurred class goes to the tailored skill only through the operator, never on its own (open). The tailored skill is a PR on the target repository.
  • Benchmark fixtures from real escapes are left out (open): see Scope.

Scope

  • scripts/escape.sh: paths validation in record, hits in stats. Tests in test_escape.sh.
  • scripts/profile.sh: escapes per directory in ## History and in history_slice. Test in test_profile.sh.
  • Docs: escapes.md (ledger paths, tailored-skill candidate), review.md (the hit names its class), review-mechanics.md (escape field), profile.md (History row), audit.md (task 39 and its report line), tailored-skill.md (inputs, rule source, Change).
  • VERSION 14.8.0 and its CHANGELOG.md entry, after 14.7.0 of feat: an error report from a running environment teaches the review its defect class (14.7.0) #298 on main.

Not in this PR: benchmark fixtures built from real escapes. A benchmark fixture holds 10–20 planted defects and scores precision. One real defect per fixture breaks that scoring, so this needs its own design.

Issue

No issue tracks this change.

Testing

  • scripts/tests/docker.sh passes on test_escape, test_profile, test_review_worklist, test_audit_stats, test_check_definition, test_survey and test_review_pr. The result is ALL TESTS PASSED.
  • The CI steps "Internal doc links resolve" and "Cross-file section references resolve" pass locally.
  • Not run: the full suite (CI runs it), and a live report on a pod.

…ts defect class (14.4.0)

A channel message that reports an error seen in production, DEV or any
other running environment is an escaped defect: a defect the review let
through. The agent finds the introducing PR (a PR the report names is
checked; otherwise scripts/escape.sh blames the stack frames and maps the
commits to merged PRs, as leads only), compares it with its own review of
that PR (raised / weak / missed / not_reviewed / not_visible / unknown),
and learns the defect class: one line under `## Escaped Defect Classes` in
work/MEMORY.md or the area file, evidence in the archive, checked against
the diff. Every review applies the classes. The reply goes to the report's
thread; nothing is posted on GitHub; the fix stays with others.

- docs/escapes.md: the procedure, the ledger work/ESCAPES.jsonl, bounds.
- scripts/escape.sh: leads, seen, record (validates), stats (audit).
- review.md criteria and self-check, runbook trust boundary and map,
  preferences routing and consolidation (classes merge, never age out),
  audit task 39 and report line.
- preflight memory budget: escape classes <= 20 in MEMORY.md.
- work-backup protects ESCAPES.jsonl; MEMORY.md template gets the section.
…rofile and offered to the tailored skill (14.5.0)

- findings-json gets an optional `escape` field: the class a finding
  matched. `escape.sh stats` counts these review hits per class, all time
  and this week; the audit reports them.
- The ledger row gets `paths` (the failing code). The profile's
  `## History` counts escapes per directory, all time, so a review of that
  area sees where defects escaped before.
- A class bound to this repository that recurred is offered in the audit
  as a tailored-skill rule; the skill's Change round pre-selects it with
  source `escape <date>`.
…s; the audit files the definition issue

- The blobless clone reads blobs on demand. A token-less origin made those
  reads fail on a private repository. The clone now uses gh as its
  credential helper, with no token in a URL, argv or on disk.
- An absent path and a failed read now differ: ls-tree checks the path, and
  a failed blame or history read is a frame error counted in reads_failed.
- The channel route posts nothing on GitHub. A generic class that no
  criterion or skill covers is filed by the weekly audit (task 39).
…scaped-defects-followup

# Conflicts:
#	docs/audit.md
#	docs/escapes.md

@dam-code-protector dam-code-protector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Code Guardian - Code guardian repo — ⚠️ Code Review @ 6367715

PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.5.0)

Author: xjacka | Branch: claude/escaped-defects-followup → claude/production-error-reporting-1176a5 | Changes: +99 −26 (12 files)

Summary

This PR adds an escape class to the review findings line and paths to the escape ledger row. It also adds an escapes column to the profile history and a tailored-skill offer to audit task 39.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.

Findings

  • 🟡 Warning: the hit count reads the pruned history file, not the shared parser (scripts/escape.sh:88)

Issue Fit

No linked issue — scope checked against the PR description only.

✅ Delivers the three goals the PR body states; no undeclared scope.

Human Review Guide

📊 Visual guide for the human reviewer — judge the new fields, the history ranking and who owns the skill rule (about 10 min)

Documentation Check (doc-drift)

✅ No findings.

Agent Engineering

✅ No findings.

Verdict

COMMENT — one 🟡 holds approval: the hit count must survive a prune.


Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi

Comment thread scripts/escape.sh Resolved
…pruned PR keeps its count

The ledger row and the history parser carry a finding's `escape` class, and
`escape.sh stats` counts hits through review_records: the ledger row and the
history files, one record per (pr, ts). Hits of the same count sort by class.
…or-reporting-1176a5

# Conflicts:
#	CHANGELOG.md
#	VERSION
…or-reporting-1176a5

# Conflicts:
#	CHANGELOG.md
#	docs/review.md
…scaped-defects-followup

# Conflicts:
#	CHANGELOG.md
#	docs/review.md
@xjacka xjacka changed the title feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.5.0) feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.6.0) Nov 6, 2026
…r in the inventories

- runbook.md and escapes.md name the one definition-repo write an escape
  can lead to: the weekly audit's anonymous issue for a generic class.
- verify-onboarding knows ESCAPES.jsonl; logging.md exempts it from
  retention; preferences.md lists the [escape …] tag.
- test_work_backup: a persist that deletes ESCAPES.jsonl is refused.
- escapes.md step 2 says what `outcome: error` of leads means; the audit
  report line counts `unknown`.
…scaped-defects-followup

# Conflicts:
#	docs/audit.md
…or-reporting-1176a5

# Conflicts:
#	CHANGELOG.md
…scaped-defects-followup

# Conflicts:
#	CHANGELOG.md
@xjacka xjacka changed the title feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.6.0) feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.7.0) Nov 6, 2026
…or-reporting-1176a5

# Conflicts:
#	CHANGELOG.md
#	README.md
#	docs/runbook.md
…scaped-defects-followup

# Conflicts:
#	CHANGELOG.md
@xjacka xjacka changed the title feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.7.0) feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0) Nov 6, 2026
Base automatically changed from claude/production-error-reporting-1176a5 to main November 6, 2026 15:54
…s-followup

# Conflicts:
#	CHANGELOG.md
#	VERSION
#	docs/audit.md
#	docs/escapes.md
#	docs/review.md
#	scripts/escape.sh
#	scripts/tests/test_escape.sh
@xjacka xjacka added the code-guardian:rereview Request a code-guardian delta re-review label Nov 6, 2026

@dam-code-protector dam-code-protector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Code Guardian - Code guardian repo — ⚠️ Code Review @ 6029148

PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0)

Author: xjacka | Branch: claude/escaped-defects-followup → main | Changes: +112 −35 (15 files)

Summary

This PR adds review hits per escaped defect class and an escapes column in the profile history. It also adds an escape-sourced rule in the tailored-skill flow.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.
2 suggestion(s) dropped under the 🟢 budget.

Changes since last review

Previous HEAD: 6367715 (2026-11-06T14:35:45Z) — verdict COMMENT

  • ✅ Fixed: hit count reads pruned history, not the shared parser (scripts/escape.sh:88)
  • 🆕 New: the hits contract omits its filter and its report value (docs/audit.md:356)
  • 🆕 New: a partial ledger line silently zeroes the hits (scripts/escape.sh:96)
  • 🔎 Missed earlier: escape rule source is not in the doc's source list (docs/tailored-skill.md:181) — present at 6367715
  • 🔎 Missed earlier: the audit offer has no tailored-skill state gate (docs/audit.md:360) — present at 6367715

Findings

  • 🟡 Warning: The hits text says "findings that named a class", but the script counts only new findings (docs/audit.md:356). The report line leaves <week hits> undefined, so two audit runs can report different numbers. The per-class counts stay hidden.
    Fix: State the status == new filter and define <week hits> as the sum of week.
  • 🟡 Warning: Two reviews that post together can leave a partial line in work/REVIEW-LEDGER.jsonl. The new read fails whole on it (scripts/escape.sh:96). HITS then becomes [] with no warning, so the audit reports no review hits while hits exist.
    Fix: Read the ledger with the tolerant parse rows uses and skip a partial line.
  • 🟢 Suggestion (missed earlier): Present at 6367715; the previous review missed it. The template now allows escape <date>, but the intro at line 9 lists only two sources (docs/tailored-skill.md:181). It names an in-tree path and the operator's answer.
  • 🟢 Suggestion (missed earlier): Present at 6367715; the previous review missed it. The audit tells the run to offer a skill rule. The tailored-skill page allows an offer only when the state line is absent or open (docs/audit.md:360).

Issue Fit

✅ No findings.

Human Review Guide

📊 Visual guide for the human reviewer — whether escapes weigh like critical findings and hits stay a floor (about 25 min)

Documentation Check (doc-drift)

✅ No findings.

Agent Engineering

  • 🟢 Suggestion: docs/escapes.md declares paths optional with the default [], while its procedure and self-check require it (docs/escapes.md:122).

Verdict

COMMENT — two 🟡 warnings hold approval: the hits contract and a silent zero on a partial ledger line.


Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi

@dam-code-protector dam-code-protector Bot removed the code-guardian:rereview Request a code-guardian delta re-review label Nov 6, 2026
…the audit defines hits and gates the skill offer

- escape.sh stats reads the review ledger through a tolerant copy, so a
  partial line from two concurrent posts no longer zeroes the hits. An
  unreadable record set is `hits: null`, never an empty list.
- audit task 39 states the `new` filter, defines the report value as the
  sum of `hits[].week`, and suggests a tailored-skill Change only when the
  skill is `active`.
- tailored-skill.md lists an escaped defect class among rule sources;
  escapes.md says when `paths` may stay empty.
@xjacka xjacka added the code-guardian:rereview Request a code-guardian delta re-review label Nov 6, 2026

@dam-code-protector dam-code-protector Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛡️ Code Guardian - Code guardian repo — ✅ Code Review @ 5002c58

PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0)

Author: xjacka | Branch: claude/escaped-defects-followup → main | Changes: +129 −37 (15 files)

Summary

This PR adds review hits per escaped defect class and an escapes column in the profile history. It also adds an escape-sourced rule in the tailored-skill flow.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.

Changes since last review

Previous HEAD: 6029148 (2026-11-06T16:22:30Z) — verdict COMMENT

  • ✅ Fixed: escape rule source is not in the doc's source list (docs/tailored-skill.md:181)
  • ✅ Fixed: the audit offer has no tailored-skill state gate (docs/audit.md:360)
  • ✅ Fixed: the hits contract omits its filter and its report value (docs/audit.md:356)
  • ✅ Fixed: a partial ledger line silently zeroes the hits (scripts/escape.sh:96)
  • 🆕 New: the tolerant ledger read covers one reader, so preflight still reports a zero week on a partial line (scripts/escape.sh:95)

Findings

  • 🟢 Suggestion: The tolerant ledger read is added in escape.sh alone. review_records still fails whole on a partial REVIEW-LEDGER.jsonl line. The other readers then report a zero week: the preflight.sh audit aggregates and audit-trend.sh. (scripts/escape.sh:95)

Issue Fit

No linked issue — scope checked against the PR description only.

✅ The delta stays inside the scope the PR body declares; no undeclared change.

Human Review Guide

📊 Visual guide for the human reviewer — at 6029148: same open points; the two blocking findings are fixed

Documentation Check (doc-drift)

✅ No findings.

Agent Engineering

✅ No findings.

Verdict

APPROVE — all four findings of the last review are fixed at this HEAD. One suggestion does not block.


Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi

@dam-code-protector dam-code-protector Bot removed the code-guardian:rereview Request a code-guardian delta re-review label Nov 6, 2026
@xjacka
xjacka merged commit 5e2cee0 into main Nov 6, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant