Repository navigation
feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0) - #299
Conversation
…ts defect class (14.4.0) A channel message that reports an error seen in production, DEV or any other running environment is an escaped defect: a defect the review let through. The agent finds the introducing PR (a PR the report names is checked; otherwise scripts/escape.sh blames the stack frames and maps the commits to merged PRs, as leads only), compares it with its own review of that PR (raised / weak / missed / not_reviewed / not_visible / unknown), and learns the defect class: one line under `## Escaped Defect Classes` in work/MEMORY.md or the area file, evidence in the archive, checked against the diff. Every review applies the classes. The reply goes to the report's thread; nothing is posted on GitHub; the fix stays with others. - docs/escapes.md: the procedure, the ledger work/ESCAPES.jsonl, bounds. - scripts/escape.sh: leads, seen, record (validates), stats (audit). - review.md criteria and self-check, runbook trust boundary and map, preferences routing and consolidation (classes merge, never age out), audit task 39 and report line. - preflight memory budget: escape classes <= 20 in MEMORY.md. - work-backup protects ESCAPES.jsonl; MEMORY.md template gets the section.
…book lists the escape clone
…rofile and offered to the tailored skill (14.5.0) - findings-json gets an optional `escape` field: the class a finding matched. `escape.sh stats` counts these review hits per class, all time and this week; the audit reports them. - The ledger row gets `paths` (the failing code). The profile's `## History` counts escapes per directory, all time, so a review of that area sees where defects escaped before. - A class bound to this repository that recurred is offered in the audit as a tailored-skill rule; the skill's Change round pre-selects it with source `escape <date>`.
…s; the audit files the definition issue - The blobless clone reads blobs on demand. A token-less origin made those reads fail on a private repository. The clone now uses gh as its credential helper, with no token in a URL, argv or on disk. - An absent path and a failed read now differ: ls-tree checks the path, and a failed blame or history read is a frame error counted in reads_failed. - The channel route posts nothing on GitHub. A generic class that no criterion or skill covers is filed by the weekly audit (task 39).
…scaped-defects-followup # Conflicts: # docs/audit.md # docs/escapes.md
There was a problem hiding this comment.
🛡️ Code Guardian - Code guardian repo — 6367715
PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.5.0)
Author: xjacka | Branch: claude/escaped-defects-followup → claude/production-error-reporting-1176a5 | Changes: +99 −26 (12 files)
Summary
This PR adds an escape class to the review findings line and paths to the escape ledger row. It also adds an escapes column to the profile history and a tailored-skill offer to audit task 39.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.
Findings
- 🟡 Warning: the hit count reads the pruned history file, not the shared parser (
scripts/escape.sh:88)
Issue Fit
No linked issue — scope checked against the PR description only.
✅ Delivers the three goals the PR body states; no undeclared scope.
Human Review Guide
📊 Visual guide for the human reviewer — judge the new fields, the history ranking and who owns the skill rule (about 10 min)
Documentation Check (doc-drift)
✅ No findings.
Agent Engineering
✅ No findings.
Verdict
COMMENT — one 🟡 holds approval: the hit count must survive a prune.
Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi
…pruned PR keeps its count The ledger row and the history parser carry a finding's `escape` class, and `escape.sh stats` counts hits through review_records: the ledger row and the history files, one record per (pr, ts). Hits of the same count sort by class.
…or-reporting-1176a5 # Conflicts: # CHANGELOG.md # VERSION
…scaped-defects-followup
…or-reporting-1176a5 # Conflicts: # CHANGELOG.md # docs/review.md
…scaped-defects-followup # Conflicts: # CHANGELOG.md # docs/review.md
…r in the inventories - runbook.md and escapes.md name the one definition-repo write an escape can lead to: the weekly audit's anonymous issue for a generic class. - verify-onboarding knows ESCAPES.jsonl; logging.md exempts it from retention; preferences.md lists the [escape …] tag. - test_work_backup: a persist that deletes ESCAPES.jsonl is refused. - escapes.md step 2 says what `outcome: error` of leads means; the audit report line counts `unknown`.
…scaped-defects-followup # Conflicts: # docs/audit.md
…or-reporting-1176a5 # Conflicts: # CHANGELOG.md
…scaped-defects-followup # Conflicts: # CHANGELOG.md
…or-reporting-1176a5 # Conflicts: # CHANGELOG.md # README.md # docs/runbook.md
…scaped-defects-followup # Conflicts: # CHANGELOG.md
…s-followup # Conflicts: # CHANGELOG.md # VERSION # docs/audit.md # docs/escapes.md # docs/review.md # scripts/escape.sh # scripts/tests/test_escape.sh
There was a problem hiding this comment.
🛡️ Code Guardian - Code guardian repo — 6029148
PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0)
Author: xjacka | Branch: claude/escaped-defects-followup → main | Changes: +112 −35 (15 files)
Summary
This PR adds review hits per escaped defect class and an escapes column in the profile history. It also adds an escape-sourced rule in the tailored-skill flow.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.
2 suggestion(s) dropped under the 🟢 budget.
Changes since last review
Previous HEAD: 6367715 (2026-11-06T14:35:45Z) — verdict COMMENT
- ✅ Fixed: hit count reads pruned history, not the shared parser (
scripts/escape.sh:88) - 🆕 New: the hits contract omits its filter and its report value (
docs/audit.md:356) - 🆕 New: a partial ledger line silently zeroes the hits (
scripts/escape.sh:96) - 🔎 Missed earlier: escape rule source is not in the doc's source list (
docs/tailored-skill.md:181) — present at6367715 - 🔎 Missed earlier: the audit offer has no tailored-skill state gate (
docs/audit.md:360) — present at6367715
Findings
- 🟡 Warning: The
hitstext says "findings that named a class", but the script counts onlynewfindings (docs/audit.md:356). The report line leaves<week hits>undefined, so two audit runs can report different numbers. The per-class counts stay hidden.
Fix: State thestatus == newfilter and define<week hits>as the sum ofweek. - 🟡 Warning: Two reviews that post together can leave a partial line in
work/REVIEW-LEDGER.jsonl. The new read fails whole on it (scripts/escape.sh:96).HITSthen becomes[]with no warning, so the audit reports no review hits while hits exist.
Fix: Read the ledger with the tolerant parserowsuses and skip a partial line. - 🟢 Suggestion (missed earlier): Present at
6367715; the previous review missed it. The template now allowsescape <date>, but the intro at line 9 lists only two sources (docs/tailored-skill.md:181). It names an in-tree path and the operator's answer. - 🟢 Suggestion (missed earlier): Present at
6367715; the previous review missed it. The audit tells the run to offer a skill rule. The tailored-skill page allows an offer only when the state line is absent oropen(docs/audit.md:360).
Issue Fit
✅ No findings.
Human Review Guide
📊 Visual guide for the human reviewer — whether escapes weigh like critical findings and hits stay a floor (about 25 min)
Documentation Check (doc-drift)
✅ No findings.
Agent Engineering
- 🟢 Suggestion:
docs/escapes.mddeclarespathsoptional with the default[], while its procedure and self-check require it (docs/escapes.md:122).
Verdict
COMMENT — two 🟡 warnings hold approval: the hits contract and a silent zero on a partial ledger line.
Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi
…the audit defines hits and gates the skill offer - escape.sh stats reads the review ledger through a tolerant copy, so a partial line from two concurrent posts no longer zeroes the hits. An unreadable record set is `hits: null`, never an empty list. - audit task 39 states the `new` filter, defines the report value as the sum of `hits[].week`, and suggests a tailored-skill Change only when the skill is `active`. - tailored-skill.md lists an escaped defect class among rule sources; escapes.md says when `paths` may stay empty.
There was a problem hiding this comment.
🛡️ Code Guardian - Code guardian repo — ✅ Code Review @ 5002c58
PR #299: feat: escaped defect classes are measured in reviews, mapped in the profile and offered to the tailored skill (14.8.0)
Author: xjacka | Branch: claude/escaped-defects-followup → main | Changes: +129 −37 (15 files)
Summary
This PR adds review hits per escaped defect class and an escapes column in the profile history. It also adds an escape-sourced rule in the tailored-skill flow.
Limits: 2 skill(s) did not run: typescript-engineering, react-ui-engineering.
Changes since last review
Previous HEAD: 6029148 (2026-11-06T16:22:30Z) — verdict COMMENT
- ✅ Fixed: escape rule source is not in the doc's source list (
docs/tailored-skill.md:181) - ✅ Fixed: the audit offer has no tailored-skill state gate (
docs/audit.md:360) - ✅ Fixed: the hits contract omits its filter and its report value (
docs/audit.md:356) - ✅ Fixed: a partial ledger line silently zeroes the hits (
scripts/escape.sh:96) - 🆕 New: the tolerant ledger read covers one reader, so preflight still reports a zero week on a partial line (
scripts/escape.sh:95)
Findings
- 🟢 Suggestion: The tolerant ledger read is added in
escape.shalone.review_recordsstill fails whole on a partialREVIEW-LEDGER.jsonlline. The other readers then report a zero week: thepreflight.shaudit aggregates andaudit-trend.sh. (scripts/escape.sh:95)
Issue Fit
No linked issue — scope checked against the PR description only.
✅ The delta stays inside the scope the PR body declares; no undeclared change.
Human Review Guide
📊 Visual guide for the human reviewer — at 6029148: same open points; the two blocking findings are fixed
Documentation Check (doc-drift)
✅ No findings.
Agent Engineering
✅ No findings.
Verdict
APPROVE — all four findings of the last review are fixed at this HEAD. One suggestion does not block.
Review by Code Guardian - Code guardian repo · automated code guardian · cb/rits/deepseek-ai/DeepSeek-V4.1-Flash-1017 on pi
What and why
This PR follows #298, which is merged. #298 teaches the review the class of each escaped defect. This PR shows whether the classes work, and where defects escaped.
It adds three things: review hits per class, escapes per directory in the profile, and an offer to move a class into the tailored skill when the class failed again.
Reviewer guide
What changes in substance
findings-jsongets an optionalescapefield with the class that a finding matched.escape.sh statscounts these hits per class, and audit task 39 reports them. Old lines without the field stay valid.paths, the repository paths of the failing code. The profile's## Historytable gets anescapescolumn per directory, counted for all time and not for the 90-day review window. A directory that has only escapes gets its own row. Escapes rank first in the table.escape <date>.Where to look closely
escapeinfindings-json. Nothing checks this field, so the count is a floor (open).Scope
scripts/escape.sh:pathsvalidation inrecord,hitsinstats. Tests intest_escape.sh.scripts/profile.sh: escapes per directory in## Historyand inhistory_slice. Test intest_profile.sh.escapes.md(ledgerpaths, tailored-skill candidate),review.md(the hit names its class),review-mechanics.md(escapefield),profile.md(History row),audit.md(task 39 and its report line),tailored-skill.md(inputs, rule source, Change).VERSION14.8.0 and itsCHANGELOG.mdentry, after 14.7.0 of feat: an error report from a running environment teaches the review its defect class (14.7.0) #298 onmain.Not in this PR: benchmark fixtures built from real escapes. A benchmark fixture holds 10–20 planted defects and scores precision. One real defect per fixture breaks that scoring, so this needs its own design.
Issue
No issue tracks this change.
Testing
scripts/tests/docker.shpasses ontest_escape,test_profile,test_review_worklist,test_audit_stats,test_check_definition,test_surveyandtest_review_pr. The result isALL TESTS PASSED.