Repository navigation
Detect and refresh stale scaffolded agents on upgrade - #43
Merged
Merged
Conversation
An upgrade replaced the CLAUDE.md block but left agents scaffolded by an earlier release in place, and check_install passed them: an advisor from before 3.3.0 has no Source/Open questions lines that policy-rev 6+ relies on. - check_install compares each agent body with assets/<role>.md, renames applied, and names the first differing line. model: and name: stay free. - diagnose reports Stackwich agents that differ from the assets as stale, ignoring agent names so collision renames still match. - SKILL.md step 3 treats a credited agent as an earlier install and offers an in-place refresh, never a silent overwrite; step 4 checks it. - evals case 2 and docs/evals.md cover the new behavior. The policy block is unchanged; policy-rev stays 7. Closes #32, #33. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k
This was referenced Nov 2, 2026
ovhirup
marked this pull request as ready for review
November 2, 2026 08:00
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dfc13482e3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
diagnose.py normalizes agent names from a set(), so rename() built its regex alternatives in hash order. With a collision rename that extends a role name (executor -> executor-stackwich), `executor` could match the start of `executor-stackwich` at the hyphen's word boundary, and a correctly scaffolded install was reported stale under some PYTHONHASHSEED values (0, 3, 5 among 0-5). Alternatives are now sorted longest first. Tests: rename() with the prefix listed first, and diagnose under 8 hash seeds with the extending rename. Reported by the Codex review on #43. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k
# Conflicts: # evals/evals.json
The release workflow publishes the 3.5.0 section as the release notes, and it only described the policy-rev 8 changes. Stale-agent detection (#31), the self-ignoring .stackwich/ (#34) and tools/validate.py (#35) ship in the same release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #31. Closes #32 and #33.
Problem
Re-running
/stackwich:stackwichreplaced theCLAUDE.mdblock but left the agents that an earlier release had scaffolded. Nothing flagged them. Anadvisor.mdfrom before 3.3.0 has noSource:orOpen questions:lines, which policy-rev 6 and later rely on, andtools/check_install.pystill reportedINSTALL: PASSfor it.Changes
tools/check_install.pycompares each agent's body withassets/<role>.md, with any rename applied, and names the first line that differs. Changes tomodel:andname:in the frontmatter still pass, since the README invites model swaps.tools/diagnose.pyreports a Stackwich agent that differs from the current assets asStackwich, staleand adds a problem line. Agent names are left out of the comparison, so collision renames still match. It still prints only names, never file contents.SKILL.mdstep 3 treats an existing agent with the Stackwich credit as an earlier install, not a collision. If its body differs, the installer offers to replace it in place, keeping itsname:andmodel:, and never overwrites without asking. Step 4 gains a matching check. Both steps sit outside the policy block, sopolicy-revstays at 7.evals/evals.jsoncase 2 anddocs/evals.mdcover the new behavior.scaffold()now applies a rename to references inside agent bodies, as SKILL step 3 already required. Before, it renamed only thename:line.Verification
cd tools && python3 -m unittest discover -v: 50 passed, up from 45, with none skipped.cd measured && python3 -m unittest -v test_measured: 12 passed, andapprove.py --self-checkreportsSOD: PASS.validate.yml, run locally, printed OK.git show e632345^:…/advisor.md).main'scheck_install.pyreportsINSTALL: PASS.FAIL: agent advisor — body differs from assets/advisor.md (stale or edited); first difference at line 25….diagnose.pyexits 1 and reports the advisor as stale.check_install.pyreportsINSTALL: PASS.Not included
Generated by Claude Code