Skip to content

policy-rev 9: research retries thin results, then names what it didn't check (3.6.0) - #47

Merged
ovhirup merged 2 commits into
mainfrom
issue-30-research-retrieval
Nov 2, 2026
Merged

ovhirup merged 2 commits into
mainfrom
issue-30-research-retrieval

Conversation

@ovhirup

@ovhirup ovhirup commented Nov 2, 2026

Copy link
Copy Markdown
Owner

Closes #30.

Problem

The Context layer says to fork research, but not how that research should retrieve. A forked scan runs one grep or search, takes what comes back, and hands it on as if it were complete. The plan is then built on thin context, and nothing records which source went unchecked.

Policy change (policy-rev 8 → 9)

One Context-layer bullet:

A research fork states its question and done condition. On thin results, rephrase or switch source (code, docs, history, issues, web), at most twice more, then return Sources:, NOT FOUND:, UNCHECKED:. UNCHECKED: stays open.

  • Word budget: the block is now 734 of its 735-word budget. To fit, the fork bullet drops "exploration", which "research" already covers. I didn't raise the budget. Since this leaves one word of headroom, the next policy change will have to cut something before it can add anything.
  • Advisor: advisor.md now lists NOT FOUND: and UNCHECKED: items under Open questions and never builds a step on them. That part lives in the agent file because only the advisor plans.
  • Retry cap: "at most twice more" matches the existing loop cap, as Agentic retrieval loop for forked research: rewrite, route, check, retry #30 asked.

How this meets #30's acceptance criteria

  • The policy block and both ports state the rule, and every revision is bumped. Codex rev 7 adds the rule to the Context layer and the open-questions line to the advisor template. Grok rev 6 adds the rule, and its PLAN phase treats unconfirmed items as open questions. The CHANGELOG 3.6.0 entry tells users to re-run the installer and accept the agent refresh.
  • The research output includes Sources:, NOT FOUND: and UNCHECKED: lines.
  • The rule stays short, with no taxonomy prose: one bullet of 36 words.
  • docs/enforcement.md lists the rule as instructed, not mechanical, for all three: Claude Code, Codex and Grok.
  • CI and the install checks pass with the new revision (see Verification).
  • A behavior scenario is added: evals/evals.json case 10, where the answer is only found after a rephrase or a second source.

Also fixed

  • docs/evals.md: it said "Seven scenarios" and stopped at case 7, though cases 8 and 9 shipped in 3.3.0 and 3.4.0. Adding case 10 changed the count anyway, so it now says ten and lists cases 8–10.
  • README layout comment: the stale eval count there is corrected too.

Verification

  • python3 tools/validate.py: all 9 categories OK. The budget step from validate.yml reports policy block: 734 of 735 words OK.
  • tools/: 63 tests pass. measured/: 12 tests pass. None skipped.
  • No policy-rev 8, grok-rev 5, codex-rev 6 or 3.5.0 references remain outside the CHANGELOG history.
  • Upgrade from a real rev-8 install (the block and advisor.md from main):

Limits


Generated by Claude Code

…t check (#30)

A forked research scan took its first result as complete and handed it
on, so plans could rest on unchecked context with no record of the gap.

- Policy (Context): a research fork states its question and done
  condition; on thin results it rephrases or switches source (code, docs,
  history, issues, web), at most twice more, then returns Sources:,
  NOT FOUND: and UNCHECKED: lines. UNCHECKED: items stay open.
- advisor.md lists NOT FOUND:/UNCHECKED: items under Open questions and
  never builds a step on them.
- The block is 734 of its 735-word budget; "exploration" was dropped from
  the fork bullet, since "research" covers it.
- Codex rev 7 and Grok rev 6 carry the rule; Codex's advisor template
  gets the open-questions line.
- evals case 10; docs/evals.md lists cases 8-10 (8 and 9 were missing);
  enforcement.md marks the rule as instructed.
- Release 3.6.0: plugin.json, SKILL metadata and step 4, CHANGELOG,
  README Versions and layer tables, port READMEs, evals cases 1-2.

Closes #30.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k
@ovhirup ovhirup mentioned this pull request Nov 2, 2026
15 of 16 tasks
The word-budget step ran `wc -w` in the runner's UTF-8 locale, which
counts the block's em dashes as extra words: CI measured 741 where a
C-locale run measured 734, so this branch failed in CI only (main itself
measured 707 in CI against 700 locally). The step now pins LC_ALL=C, so
the count is whitespace-separated words everywhere.

Reproduced: under LC_ALL=C.UTF-8 the step failed at 741; with the fix it
reports 734 of 735 under both C.UTF-8 and C.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01TgCNfwCj3YG9xP3pMB6z2k
@ovhirup
ovhirup marked this pull request as ready for review November 2, 2026 08:38
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T08:40:30.746153Z 8b17660 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ovhirup
ovhirup merged commit ea60b0e into main Nov 2, 2026
2 checks passed
@ovhirup
ovhirup deleted the issue-30-research-retrieval branch November 2, 2026 09:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Agentic retrieval loop for forked research: rewrite, route, check, retry

2 participants