From be35b26925320605993124bb7ae9f4348e23e66e Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 08:36:19 +0000 Subject: [PATCH 1/2] policy-rev 9: research retries thin results, then names what it didn't check (#30) A forked research scan took its first result as complete and handed it on, so plans could rest on unchecked context with no record of the gap. - Policy (Context): a research fork states its question and done condition; on thin results it rephrases or switches source (code, docs, history, issues, web), at most twice more, then returns Sources:, NOT FOUND: and UNCHECKED: lines. UNCHECKED: items stay open. - advisor.md lists NOT FOUND:/UNCHECKED: items under Open questions and never builds a step on them. - The block is 734 of its 735-word budget; "exploration" was dropped from the fork bullet, since "research" covers it. - Codex rev 7 and Grok rev 6 carry the rule; Codex's advisor template gets the open-questions line. - evals case 10; docs/evals.md lists cases 8-10 (8 and 9 were missing); enforcement.md marks the rule as instructed. - Release 3.6.0: plugin.json, SKILL metadata and step 4, CHANGELOG, README Versions and layer tables, port READMEs, evals cases 1-2. Closes #30. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k --- CHANGELOG.md | 20 +++++++++++++++++++ README.md | 14 ++++++------- codex/README.md | 2 +- codex/references/templates.md | 8 +++++++- docs/enforcement.md | 5 +++-- docs/evals.md | 5 ++++- evals/evals.json | 20 ++++++++++++++++--- grok/README.md | 2 +- grok/SKILL.md | 6 +++++- plugins/stackwich/.claude-plugin/plugin.json | 2 +- plugins/stackwich/skills/stackwich/SKILL.md | 17 +++++++++------- .../skills/stackwich/assets/advisor.md | 3 +++ 12 files changed, 79 insertions(+), 25 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19ed4c2..bb86134 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,26 @@ Headings are `## — `. Ports are named in the body with their own rev counters: a port tracks the parent's layer structure, not its revision number. +## 3.6.0 — policy-rev 9: research that retries, then says what it didn't check + +The Claude Code policy changes, so `policy-rev` moves from 8 to 9: re-run `/stackwich:stackwich` +to apply it in place, and accept the agent refresh it offers, because `advisor.md` changed. The +`` marker is unchanged. + +### Policy +- A research fork states its question and done condition. On thin results it rephrases or + switches source (code, docs, history, issues, web), at most twice more, then returns + `Sources:`, `NOT FOUND:` and `UNCHECKED:` lines; `UNCHECKED:` items stay open. (#30) +- `advisor.md` lists `NOT FOUND:` and `UNCHECKED:` items under `Open questions` and never builds a + step on them. +- The block stays inside its 735-word budget (734 words): the Context fork bullet drops + "exploration", which "research" already covers. +- `evals/evals.json` case 10 describes the expected behavior; `docs/evals.md` now lists cases 8–10. + +### Ports +- Codex moves to `stackwich-codex-rev 7` and Grok to `stackwich-grok-rev 6` with the same rule; + Codex's `advisor` template gets the open-questions line. + ## 3.5.0 — policy-rev 8: a leaner policy with one failure ladder The Claude Code policy changes, so `policy-rev` moves from 7 to 8: re-run `/stackwich:stackwich` diff --git a/README.md b/README.md index f2df942..16d6a58 100644 --- a/README.md +++ b/README.md @@ -19,7 +19,7 @@ Stackwich is 15 years of delivery coaching — plan → build → test → revie | Layer | Habit | |---|---| | **Prompt** | Meta-prompt every delegation. Context-full, with a stated done condition. Understanding never gets delegated. | -| **Context** | Fork research and read-heavy scans off the main thread, never to the writer. Interrupted work resumes from a handoff. | +| **Context** | Fork research and read-heavy scans off the main thread, never to the writer. Research retries thin results twice at most, then names what it couldn't confirm. Interrupted work resumes from a handoff. | | **Harness** | One writer. Read-only roles stay read-only in practice, not just in the tool list. Nothing "done" until observed. | | **Loop** | Recurring work runs on `/loop`. Failures climb one ladder — retry, re-plan, then stop — and are never silent. | | **Graph** | `advisor` plans → `executor` implements → `verifier` gates → `advisor` reviews, with explicit hand-off contracts. | @@ -83,12 +83,12 @@ Four numbers move independently. Only the policy revisions change what an instal | Number | Current | Where it lives | When you act | |---|---|---|---| -| Repository and plugin version | 3.5.0 | `plugin.json`, git tags `vX.Y.Z`, [CHANGELOG](CHANGELOG.md) headings | Update the plugin. | -| Claude Code policy revision | `policy-rev 8` | The comment inside the `` block in your `CLAUDE.md` | Re-run `/stackwich:stackwich` when it changes. | -| Grok port revision | `stackwich-grok-rev 5` | Inside the block the Grok port writes | Re-apply the Grok port when it changes. | -| Codex port revision | `stackwich-codex-rev 6` | Inside the `` block in `AGENTS.md` | Re-apply the Codex port when it changes. | +| Repository and plugin version | 3.6.0 | `plugin.json`, git tags `vX.Y.Z`, [CHANGELOG](CHANGELOG.md) headings | Update the plugin. | +| Claude Code policy revision | `policy-rev 9` | The comment inside the `` block in your `CLAUDE.md` | Re-run `/stackwich:stackwich` when it changes. | +| Grok port revision | `stackwich-grok-rev 6` | Inside the block the Grok port writes | Re-apply the Grok port when it changes. | +| Codex port revision | `stackwich-codex-rev 7` | Inside the `` block in `AGENTS.md` | Re-apply the Codex port when it changes. | -A repository release can leave every policy revision unchanged; 3.2.0 did, while 3.3.0, 3.4.0 and 3.5.0 each moved all three. Each CHANGELOG entry says whether there is anything to re-run. The markers themselves never change. +A repository release can leave every policy revision unchanged; 3.2.0 did, while 3.3.0 through 3.6.0 each moved all three. Each CHANGELOG entry says whether there is anything to re-run. The markers themselves never change. ### Older copies @@ -132,7 +132,7 @@ stackwich/ │ ├── executor.md │ ├── verifier.md │ └── rebound.md # cost/rebound playbook, read only for cost or loop questions -├── evals/evals.json # five install paths + two policy-behavior cases +├── evals/evals.json # five install paths + five policy-behavior cases ├── tools/ # check_install, diagnose, uninstall, verify_gate, handoff, and validate (CI's checks) ├── examples/first-task/ # the demo project for docs/first-task.md ├── recipes/ # API, browser-flow, responsive, and accessibility checks diff --git a/codex/README.md b/codex/README.md index cb30e23..d742b35 100644 --- a/codex/README.md +++ b/codex/README.md @@ -69,7 +69,7 @@ scaffold `advisor`, `executor`, and `verifier` as TOML agents. > insert a duplicate block. > **Port status:** at parity with the Claude Code parent's five-layer policy, as -> `stackwich-codex-rev 6`. The port tracks the parent's layer structure, not its revision +> `stackwich-codex-rev 7`. The port tracks the parent's layer structure, not its revision > number — a parent bump does not automatically oblige a port bump. --- diff --git a/codex/references/templates.md b/codex/references/templates.md index d4823b4..e5c2b5c 100644 --- a/codex/references/templates.md +++ b/codex/references/templates.md @@ -7,7 +7,7 @@ credit in every custom-agent file. ```markdown - + ## Working Architecture (Stackwich Codex) ### Prompt @@ -21,6 +21,11 @@ credit in every custom-agent file. ### Context - Use subagents for independent research, exploration, read-heavy scans, tests, or log analysis when this keeps noisy output out of the main thread. +- A research delegation states its question and done condition. On thin results the subagent + rephrases (synonyms, symbol names, file patterns) or switches source (code, docs, git history, + issues, web), at most twice more, then returns `Sources:`, `NOT FOUND:` and `UNCHECKED:` + lines. `NOT FOUND:` and `UNCHECKED:` items are open questions for `advisor` and the user, + never facts. - Prefer a context-carrying subagent when prior task context is relevant. Use a fresh, minimally briefed agent only when independence is valuable or the work is genuinely unrelated. @@ -122,6 +127,7 @@ When planning: - Mark hard-to-reverse or shared-system touchpoints as requiring user confirmation. - Leave no avoidable design decisions for the executor. - If you were given an existing plan, spec, or ticket, it is the plan: name it as the source, keep its goal, scope, and decisions, add only what the executor needs, and list gaps or conflicts as open questions for the user instead of deciding them. +- Research that reports `NOT FOUND:` or `UNCHECKED:` lines gives you open questions, not facts: list each as an open question and never build a step on it. When reviewing: 1. Check the change against the approved plan and stated intent; reject scope creep. diff --git a/docs/enforcement.md b/docs/enforcement.md index c7af604..8da44fd 100644 --- a/docs/enforcement.md +++ b/docs/enforcement.md @@ -25,6 +25,7 @@ Three kinds of claim: | Verdict lines (`GATE: PASS`, `APPROVE`, `REVISE:`) | Instructed | They are fixed strings so routing does not depend on reading prose, but the main session does the routing. | | Failure ladder: retry, then `advisor` re-plans, then stop at the third failure | Instructed | No counter exists outside the conversation. | | Read-heavy scans go to a fork, not `executor` | Instructed | The main session chooses where to send a scan. | +| Research retries thin results at most twice, then reports `Sources:`, `NOT FOUND:`, `UNCHECKED:` | Instructed | Nothing counts the rounds or checks that the listed sources were searched. `advisor` is told to plan on unconfirmed items only as open questions. | | Nothing "done" until verified | Instructed | The policy requires reporting the command and its output. No tool checks that a claim matches a run. | ## Codex port @@ -33,7 +34,7 @@ Three kinds of claim: |---|---|---| | `advisor` and `verifier` cannot write files | Mechanical, where Codex honors the setting | Their templates set `sandbox_mode = "read-only"`, so Codex runs their commands in its read-only sandbox. | | `executor` stays in scope and does not push, deploy, or publish | Instructed | `executor` has no `sandbox_mode` in its template and inherits the session's sandbox and approval settings. | -| Sandwich, verdict routing, failure ladder, scan routing | Instructed | Same as the Claude Code plugin. | +| Sandwich, verdict routing, failure ladder, scan routing, research retries | Instructed | Same as the Claude Code plugin. | | Model choice | Not pinned | The templates set reasoning effort and sandbox only, so they stay portable across accounts. | A read-only sandbox also blocks legitimate writes. A test, build, or coverage run that needs to @@ -45,7 +46,7 @@ verifier's instructions say to label environment problems as such. | Behavior | Kind | Detail | |---|---|---| | Single writer, PLAN/VERIFY/REVIEW never edit | Instructed | The port runs the sandwich as phases inside one agent. There is no separate role with fewer tools. | -| Sandwich, failure ladder (EXECUTE, then PLAN, then the user), verification before "done" | Instructed | Same policy as the parent, carried by `grok/SKILL.md`. | +| Sandwich, failure ladder (EXECUTE, then PLAN, then the user), research retries, verification before "done" | Instructed | Same policy as the parent, carried by `grok/SKILL.md`. | ## Measured approval gate (`measured/approve.py`) diff --git a/docs/evals.md b/docs/evals.md index 05bd930..b2cd788 100644 --- a/docs/evals.md +++ b/docs/evals.md @@ -109,7 +109,7 @@ and the approval gate's self-check. ## Behavior evals: `evals/evals.json` -Seven scenarios describe what an agent should do. Their assertions are about conversation and +Ten scenarios describe what an agent should do. Their assertions are about conversation and behavior, so a model has to be run and its transcript read. | Case | Scenario | What `check_install.py` can confirm | What needs the transcript | @@ -121,6 +121,9 @@ behavior, so a model has to be run and its transcript read. | 5 | Missing closing marker | The broken state is detected | Region shown; no replacement before confirmation | | 6 | Unbounded loop request | Nothing | Stop condition, run cap, and no-change stop stated before starting | | 7 | Large parallel batch | Nothing | Shard count and reason; batches fit one verifier pass; run counts reported | +| 8 | Work from an existing spec | Nothing | Spec named as the plan's source; gaps raised as open questions; review against the spec | +| 9 | Resume after an interruption | `tools/handoff.py resume` names what changed | Handoff read first; earlier PASS not trusted until the gate re-runs | +| 10 | Research that needs a second try | Nothing | Question and done condition stated; rephrase or new source, at most twice more; `Sources:`, `NOT FOUND:`, `UNCHECKED:` returned; unconfirmed items kept as open questions | ### How to run one diff --git a/evals/evals.json b/evals/evals.json index 0d8d049..e3eaf15 100644 --- a/evals/evals.json +++ b/evals/evals.json @@ -5,7 +5,7 @@ "id": 1, "name": "fresh-user-level-install", "prompt": "Set up the Stackwich architecture on this machine at the user level, including the subagents.", - "expected_output": "~/.claude/CLAUDE.md created with exactly one stackwich:v1 marker pair and policy-rev 8; advisor.md, executor.md, verifier.md written to ~/.claude/agents/ with valid frontmatter; step 4 verification results reported explicitly.", + "expected_output": "~/.claude/CLAUDE.md created with exactly one stackwich:v1 marker pair and policy-rev 9; advisor.md, executor.md, verifier.md written to ~/.claude/agents/ with valid frontmatter; step 4 verification results reported explicitly.", "assertions": [ "CLAUDE.md contains exactly one opening and one closing marker", "policy-rev line is present inside the block", @@ -19,12 +19,12 @@ "id": 2, "name": "update-existing-v1-install", "prompt": "I already have Stackwich installed from a while back — can you bring it up to date?", - "expected_output": "Existing block located via the opening marker, shown to the user with its declared policy-rev, replaced in place after confirmation with the policy-rev 8 block. No duplicate section appended. Scaffolded Stackwich agents that differ from the current assets are named as out of date and offered for in-place refresh, keeping their name and model; none is overwritten without asking.", + "expected_output": "Existing block located via the opening marker, shown to the user with its declared policy-rev, replaced in place after confirmation with the policy-rev 9 block. No duplicate section appended. Scaffolded Stackwich agents that differ from the current assets are named as out of date and offered for in-place refresh, keeping their name and model; none is overwritten without asking.", "assertions": [ "existing block is shown to the user before modification", "user is asked before replacing", "final file contains exactly one marker pair", - "final block declares policy-rev 8 and no older policy-rev line remains", + "final block declares policy-rev 9 and no older policy-rev line remains", "stale Stackwich agents are reported and refreshed only after the user agrees" ], "files": [] @@ -119,6 +119,20 @@ "an earlier PASS is not relied on until the gate is re-run", "with no handoff, the agent says so instead of guessing where the work stood" ] + }, + { + "id": 10, + "name": "research-retrieval-loop", + "prompt": "Before we change the retry logic, find out where the HTTP client's timeout is configured and what it defaults to.", + "files": [], + "expected_output": "The research is forked with its question and a done condition (the setting's location and its default value). When the first search comes back empty or thin, the agent rephrases (synonyms, symbol names, file patterns) or switches source (code, docs, git history, issues, web), at most twice more. It returns `Sources:` naming where it looked, and `NOT FOUND:` or `UNCHECKED:` lines for anything it could not confirm. Any plan built on it lists those lines as open questions rather than treating them as facts.", + "assertions": [ + "the research states its question and what would count as a sufficient answer", + "a weak or empty first result is followed by a rephrased query or a different source, not accepted as complete", + "no more than 2 extra retrieval rounds are run", + "the result names its sources and includes NOT FOUND or UNCHECKED lines for anything unconfirmed", + "unconfirmed items appear as open questions in any plan, not as facts" + ] } ] } diff --git a/grok/README.md b/grok/README.md index c62a6da..c395dae 100644 --- a/grok/README.md +++ b/grok/README.md @@ -60,7 +60,7 @@ cp SKILL.md LICENSE ~/.grok/skills/stackwich-grok/ ``` > **Port status:** at parity with the Claude Code parent's five-layer policy, as -> `stackwich-grok-rev 5`. The port tracks the parent's layer structure, not its revision +> `stackwich-grok-rev 6`. The port tracks the parent's layer structure, not its revision > number — a parent bump does not automatically oblige a port bump. The Graph layer is > expressed as explicit phases inside one agent rather than three subagents: a deliberate > design choice for this runtime, not a shortfall. diff --git a/grok/SKILL.md b/grok/SKILL.md index ada2720..c7e0667 100644 --- a/grok/SKILL.md +++ b/grok/SKILL.md @@ -103,7 +103,7 @@ When the user asks to install or activate Stackwich: ```markdown - + ## Working Architecture (Stackwich) ### Prompt @@ -116,6 +116,10 @@ When the user asks to install or activate Stackwich: ### Context - Keep raw research, long tool dumps and exploratory scans out of the main reasoning thread. Summarize, or write intermediate results to files, rather than carrying them forward. +- Research starts from its question and done condition. On thin results, rephrase (synonyms, + symbol names, file patterns) or switch source (code, docs, git history, issues, web), at most + twice more, then record `Sources:`, `NOT FOUND:` and `UNCHECKED:` lines. PLAN treats + `NOT FOUND:` and `UNCHECKED:` items as open questions, never as facts. - Durable memory is for facts that outlive the task; a plan is for implementation alignment; task tracking is for current progress. These are not interchangeable. - When work may outlive the session (a cycle in flight, a multi-step plan), keep diff --git a/plugins/stackwich/.claude-plugin/plugin.json b/plugins/stackwich/.claude-plugin/plugin.json index 8b4d238..4cc9d24 100644 --- a/plugins/stackwich/.claude-plugin/plugin.json +++ b/plugins/stackwich/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "stackwich", - "version": "3.5.0", + "version": "3.6.0", "description": "Delivery discipline for AI coding agents. Writes the five-layer Stackwich policy (Prompt / Context / Harness / Loop / Graph) into CLAUDE.md and scaffolds advisor, executor and verifier subagents. An optional local measured layer can record sandwich steps and delivery signals without changing that policy.", "author": { "name": "Abhirup Banerjee" }, "homepage": "https://github.com/ovhirup/stackwich", diff --git a/plugins/stackwich/skills/stackwich/SKILL.md b/plugins/stackwich/skills/stackwich/SKILL.md index 8823f4a..e6b4bd4 100644 --- a/plugins/stackwich/skills/stackwich/SKILL.md +++ b/plugins/stackwich/skills/stackwich/SKILL.md @@ -3,8 +3,8 @@ name: stackwich description: Installs the Stackwich architecture — meta-prompt delegation, a plan/execute/verify/review sandwich, loop-by-default for recurring work, and fork-based context engineering — into any Claude Code setup. Writes the operating policy into CLAUDE.md and optionally scaffolds three companion subagents (advisor/executor/verifier). Use this whenever the user wants to adopt, install, bootstrap, upgrade, or re-apply this delegation architecture on a new machine, project, or someone else's setup — and also when they ask more loosely for help making their Claude Code setup more disciplined, setting up planner/executor/reviewer subagents, adding a review gate before risky changes, enforcing cheap-model tiering, or fixing agents that make sloppy or out-of-scope edits. Also use it when the user asks why agent usage or cost keeps climbing despite cheaper models, or whether a loop or more parallel agents will actually save effort. metadata: author: Abhirup Banerjee (@ovhirup) - version: 3.5.0 - policy-rev: 8 + version: 3.6.0 + policy-rev: 9 --- # Stackwich — install the Stackwich architecture @@ -59,7 +59,7 @@ substitute consistently everywhere they appear. ```markdown - + ## Working Architecture (Stackwich) ### Prompt @@ -68,9 +68,12 @@ substitute consistently everywhere they appear. - Every delegation states its done condition: the verification command and expected output. ### Context -- Fork research, exploration, and read-heavy scans so raw tool output stays out of the main - conversation; prefer a fork over a fresh agent unless the work is unrelated. Never send - scans to `executor`. +- Fork research and read-heavy scans so raw tool output stays out of the main conversation; + prefer a fork over a fresh agent unless the work is unrelated. Never send scans to + `executor`. +- A research fork states its question and done condition. On thin results, rephrase or switch + source (code, docs, history, issues, web), at most twice more, then return `Sources:`, + `NOT FOUND:`, `UNCHECKED:`. `UNCHECKED:` stays open. - When work may outlive the session, keep `.stackwich/handoff.md` current at each stage change, with `.stackwich/` kept out of git: the plan and where it lives, the stage, the next step, the last gate verdict with its command, and the commit plus `git status --short`. On resuming, @@ -180,7 +183,7 @@ This skill installs a verify-before-done policy, so it holds itself to it. After confirm and report actual observed results — not intentions: 1. Both markers present in `/CLAUDE.md`, exactly once each. -2. `policy-rev: 8` is present inside the block (an upgrade from an older revision must +2. `policy-rev: 9` is present inside the block (an upgrade from an older revision must leave no trace of the previous one). 3. If agents were scaffolded: all three files exist in `/agents/`, each starts with valid YAML frontmatter containing `name`, `description`, `model`, and `tools`. diff --git a/plugins/stackwich/skills/stackwich/assets/advisor.md b/plugins/stackwich/skills/stackwich/assets/advisor.md index b3d7e5e..c2cfd24 100644 --- a/plugins/stackwich/skills/stackwich/assets/advisor.md +++ b/plugins/stackwich/skills/stackwich/assets/advisor.md @@ -29,6 +29,9 @@ decisions; add only what an executor needs (exact files, edits, a verification c step). Where it is ambiguous, contradicts the code, or leaves a decision open, list that under `Open questions` for the user instead of deciding it yourself. +If research behind the plan reports `NOT FOUND:` or `UNCHECKED:` lines, those are open +questions, not facts: list each under `Open questions` and never build a step on it. + ``` Source: Goal: From 8b17660d7dc8d71528b0e2e64713603908902e85 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 2 Nov 2026 08:37:50 +0000 Subject: [PATCH 2/2] Count policy words the same way locally and in CI The word-budget step ran `wc -w` in the runner's UTF-8 locale, which counts the block's em dashes as extra words: CI measured 741 where a C-locale run measured 734, so this branch failed in CI only (main itself measured 707 in CI against 700 locally). The step now pins LC_ALL=C, so the count is whitespace-separated words everywhere. Reproduced: under LC_ALL=C.UTF-8 the step failed at 741; with the fix it reports 734 of 735 under both C.UTF-8 and C. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k --- .github/workflows/validate.yml | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/.github/workflows/validate.yml b/.github/workflows/validate.yml index 7b66c26..4a05cdf 100644 --- a/.github/workflows/validate.yml +++ b/.github/workflows/validate.yml @@ -29,7 +29,9 @@ jobs: - name: Policy block within its word budget run: | BUDGET=735 - W=$(awk '/^$/,/^$/' plugins/stackwich/skills/stackwich/SKILL.md | wc -w) + # LC_ALL=C: count whitespace-separated words the same way everywhere; a UTF-8 locale counts + # the block's em dashes as extra words, so CI and a local run disagreed. + W=$(awk '/^$/,/^$/' plugins/stackwich/skills/stackwich/SKILL.md | LC_ALL=C wc -w) [ "$W" -gt 0 ] || { echo "::error::no policy block found to measure"; exit 1; } [ "$W" -le "$BUDGET" ] || { echo "::error::policy block is $W words, over its $BUDGET-word budget — it is read on every turn, so cut before adding"; exit 1; } echo "policy block: $W of $BUDGET words OK"