Skip to content

PR 01: attention evidence foundation and gated baseline campaign - #905

Draft
kalebbroo wants to merge 16 commits into
mainfrom
codex/attention-evidence-pr01
Draft

kalebbroo wants to merge 16 commits into
mainfrom
codex/attention-evidence-pr01

Conversation

@kalebbroo

@kalebbroo kalebbroo commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

PR 01 only: add observation and measurement infrastructure without changing attention arithmetic, admission, memory budgets, execution ordering or fallback selection. This PR remains incomplete and draft. The user has separately authorized starting draft PR 02 while its baseline and exit-validation work remains outstanding.

Three review groups:

  • A — Execution evidence: bounded default-off metadata, actual candidate/nested-row observations, preparation/cache/layer attribution, backend graph replay, and the repaired shared-4090 quiet window. The stop-signal race reported by review is fixed and covered by a reproducing test.
  • B — Comparison foundation: exact serialized fixtures shared by C#, Python and OpParity; independently computed oracle checks; pristine-production-root builds; isolated pinned Kitchen source/environment checks; distinct timing boundaries; fixed bootstrap/confirmation analysis; ordinary Qwen and Krea request tools.
  • C — Evidence and handoff: hashed local results, dependency lock, partial K0 runtime index, explicit matrix amendment, remaining exit gates, 4090 continuation instructions and the proposed later queue.

Validation:

  • CPU projects: 3,207 passed, 2 failed, 1 skipped. Both failures reproduce on pristine B0: the API mesh-operation inventory omits remesh; the Engine MiniMax-H3 path assertion differs in Windows VAE/vae casing.
  • GPU suites ran sequentially across Core, LLM, Diffusion and Engine. Resource skips and resource-guarded reported passes remain unvalidated; this is not a full regression pass.
  • All 167 Core CPU tests pass after the per-sequence-length review fix; the seven new fixture cases include the generated mixed-length failure. Sixteen Python tests pass. Relevant .NET 10 tools and the .NET 8 Engine build pass.
  • Pilot complete output hashes and top-level routes agree across pristine B0, diagnostics disabled and enabled on CUDA and Vulkan. Final tool-provenance smokes also match B0. Memory-pressure, nested-route and real-generation equivalence remain outstanding.
  • GenCapture execution/reporting share the shipped protocol warmup count; build and publish copies are byte-identical. The Kitchen runner rechecks imported Python source hashes; a changed-source probe is rejected before timing and the unchanged pinned fallback still passes.
  • Exact K0 source runs in isolated PyTorch 2.9.1+cu130. GPU fallback probes cover BF16 decode/lengths/LSE, speculative merges and partial-block Sol. Sage rejects the Python-only cohort as expected. Native K0 remains unverified after the CUDA 13/MSVC compiler failure. Diagnostic profiling and timing runs are separate.

No primary speed, matched-quality non-inferiority or complete Kitchen parity claim is made. The Qwen candidate renders exactly 8,192 tokens; B0 rejects placement on the local 3060 (11.0 GB free versus 11.1 GB required), so natural 128-token completion and paged-path proof need the 4090. Krea assets, quality gates and complete generation remain pending. Finite prequantized reuse-count cells are explicitly ineligible until their state accounting is implemented and validated.

Pristine B0: a6c465bc8aa9121aadcb9d088ddc72e815ae4288. Kitchen K0: 7d6a22ae8281cd1053a8449530fa0e764a2e51cb.

Local evidence status · 4090 handoff · Proposed queue

Raw campaigns, models, binaries and tensor bytes remain outside tracked source, with hashes and locations in the report. Draft PR 02 (#910) is now started under separate user authorization. Complete the outstanding prerequisite evidence and correctness reviews before accepting its changes or freezing Bfix.

@kalebbroo

Copy link
Copy Markdown
Contributor Author

@codex review pr

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-15T07:23:58.056230Z 4ce2c22 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5fdd8cfc66

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread benchmarks/attention/quiet_window.py Resolved
@kalebbroo kalebbroo changed the title Establish attention evidence and pinned Kitchen comparison PR 01: attention evidence foundation and gated baseline campaign Nov 15, 2026
@kalebbroo

Copy link
Copy Markdown
Contributor Author

@codex Review PR

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b314bbea3c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@kalebbroo

Copy link
Copy Markdown
Contributor Author

@codex Review again.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96fa97d70a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Contributor Author

All four inline findings are fixed and resolved. Please review the current PR 01 head 3ee783e, including the mixed per-sequence-length rejection added after the prior review. PR 01 remains draft: baseline/workload/hardware exit evidence is incomplete. The user is handing the pinned 4090 test assignment to a GPU agent; the returned report will be reviewed before any merge. The stacked PR 02 handoff includes both PRs and immutable source pins.

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3ee783e769

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Copy link
Copy Markdown
Contributor Author

The fixture-API finding is fixed and resolved in 4ce2c22. This is an evidence-tool change only; production source and GPU test pins are unchanged. All 187 Core CPU tests pass on net8.0 and net10.0; benchmark and OpParity build on both targets. Both actual consumers reject generated tree/step merge and Sol fixtures before result creation; unsupported API cells remain unvalidated. The separately pinned harness and invalidation rule are recorded in the updated GPU handoff on #910. PR 01 remains draft pending its hardware/workload exit evidence.

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 4ce2c223c6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant