Skip to content

test(goal): strengthen source-session successor isolation - #5227

Merged
huangruiteng merged 3 commits into
loopx-project:mainfrom
Duang777:codex/goal-instance-end-to-end-isolation
Sep 28, 2026
Merged

huangruiteng merged 3 commits into
loopx-project:mainfrom
Duang777:codex/goal-instance-end-to-end-isolation

Conversation

@Duang777

@Duang777 Duang777 commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Goal

Extend the existing source-session lifetime integration test with the missing successor-isolation relations identified in review. This PR does not claim Todo, claim, plan, constraint-recovery, or late-result coverage.

Changes

  • Removed the duplicate 444-line standalone test file.
  • Extended test_recreation_retires_bindings_fences_stale_operations_and_allows_successor in the existing CLI lifetime suite.
  • Retired two bindings in one recreation and verified both appear in the retirement receipt.
  • Verified stale bind and stale unbind attempts leave the complete registry bytes unchanged and create no receipts.
  • Verified the successor instance can bind a new session afterward without changing its Goal record.

Validation

  • 22 passed for tests/cli_commands/test_source_session_lifetime.py
  • Focused successor-isolation test: 1 passed
  • Ruff: passed
  • git diff --check: passed
  • Mutation probe after stale bind: flipping successor execution_authority made the full-byte assertion fail.
  • Mutation probe after stale unbind: the same mutation made the second full-byte assertion fail.

No production behavior or persisted schema changes.

Verify that Goal recreation retires the old instance identity, rejects
stale session bindings/unbindings, and allows new bindings through the
real source-session lifecycle entrypoints. 10 test cases cover:

- Old instance resolution as "retired" after recreation
- New instance auto-generation and "current" resolution
- Stale session bind/unbind rejection with stale_goal_instance error
- New session binding success and current resolution
- Old session ids recorded in retired_session_ids
- Recreation idempotency (replayed=true on replay)
- Registration idempotency (replayed=true on replay)
- Session binding idempotency
- operation_id conflict detection for different sessions

Design owner: source-session lifetime (source_session_lifetime.ts).
Qualification: goal-immutability-coherence-defense-v0.md.

Signed-off-by: Duang777 <duang777@gmail.com>
Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed exact head: 3c66bc57e6e588bb57eb299a62facfedc50bd195
Comparison base: bb0b2baa81bffbef3d96d3b1cc7ae29f915b1bbb

动机

旧 Goal A 的延迟操作不能改写同名后继 B,而且 B 的合法操作仍应继续,这是值得验证的真实边界。不过 continuity design note 明确是非规范性后续设计,不是已经交付的端到端保证。当前 PR 声称验证“不泄漏 mutation 或 execution authority”,我以这个声明和既有 source-session 契约审查,不要求它顺带实现整个 Todo/claim/plan 或 R5。

主干已有真实 CLI → Python service → TypeScript decision → registry transaction 覆盖,并非只有纯函数测试。这个新增文件没有证明足够的独立增量,且关键负向 oracle 对实际持久化错误不敏感。当前结论是 REQUEST_CHANGES,原因是覆盖质量和重复维护成本,不是远程 CI 状态。

改动思路

归属是合理的:测试调用 register_fresh_source_session_project、recreate_goal_instance、bind/unbind/resolve 服务;实例匹配仍由 source_session_lifetime.ts 决定,Python 负责文件锁和格式保真的 registry 事务。没有新增生产模块、权限或第二个状态 owner。

正向流程为登记 A、绑定会话、重建 B、绑定新会话并读回 current;负向流程为携带 A 的 instance 发起新 bind/unbind,等待 stale rejection。但异常成立不等于 B 没有发生写入。最小有益方案是扩展现有 lifetime suite 中缺少的 stale-unbind、多绑定退休及 B 合法继续场景,复用其 fixture,并将完整无副作用断言串进同一条恢复路径;无需再建一个同形测试文件或新的测试框架。

具体改动

整份 diff 只有 tests/control_plane/test_goal_instance_qualified_continuity.py,新增 444 行、10 项测试。五项覆盖替换、旧 bind、新 bind、两会话退休、旧 unbind;另外五项覆盖 recreation 重放/旧实例拒绝、registration 重放、binding 重放及同键不同会话冲突。所有新增行均为测试、fixture 和说明,生产行为及持久化格式没有改变。

关键代码讲解

  • _fresh_registration 创建一次性 synthetic registry/runtime;_registration_and_ids、_load_registry 仅组织登记结果和读回,没有实现新的 continuity 规则。
  • test_stale_session_bind_rejected_after_recreation 与 test_stale_unbind_rejected_after_recreation 仅检查 ValueError 的 stale 文本。两者都没有在拒绝后核对 B 的完整 Goal、配额、bindings、receipts 或 execution-authority 状态。
  • test_new_session_can_bind_after_recreation 和 test_recreation_retires_old_session_bindings 分别检查 B 可绑定及两个旧 session 不再 current;这些是可保留的有用场景,但应和延迟操作后的持久化读回合并,而不是只在不同 fixture 中分别成功。
  • 三类 idempotency 测试只比较部分响应或 replay 标记。现有 CLI suite 已检查原 receipt、完整 registry 字节和后续状态不被重放改写,覆盖更强。

对主干的风险

[P2,阻塞] 负向测试没有验证其承诺的“B 不被修改”。 在一次性测试空间对真实 _commit_project_session_operation 注入单一故障:原函数正常拒绝 stale instance 后,通过真实 source_session_registry_transaction 将 B 的 quota.compute 改为 99,再原样抛出异常。整个新增文件仍是 10 passed,实际发生 2 次 successor quota 写入。相同故障运行现有 test_recreation_retires_bindings_and_fences_stale_bind,其完整 registry 字节断言按预期失败。这是 oracle 盲点的实测,不是在声称当前生产代码已经有该漏洞。

请为 stale bind/unbind 在操作前后比较完整权威状态、原/新增 receipts 和执行权限,确认合法 B 操作随后继续成功;相同 mutation 应让新增断言失败,去掉 fault 后通过。位置是本文件 stale bind 的 162 行附近及 stale unbind 的 302 行附近。

[P2,阻塞] 把重复覆盖收敛到现有 owner,保留真正缺失的场景。 tests/cli_commands/test_source_session_lifetime.py 已有 test_bind_and_unbind_commit_exact_receipts、test_recreation_retires_bindings_and_fences_stale_bind、test_resolve_classifies_current_and_retired_exact_refs,以及 registration/recreation 的重放、损失响应和进程中断恢复测试;它们走的是同一组真实服务。tests/control_plane_ts/source_session_lifetime.test.ts 另外覆盖 typed stale matching、重放及容量边界。请不要用“新增 end-to-end”标签把这些既有集成路径重写一遍;将少量未覆盖的关系直接补入现有 suite,保留更强的无副作用 oracle。

验证记录:不可变 base 的相关 Python suite 32 passed;head 同一 suite 加本文件 42 passed;base/head typed lifetime suite 各 6 passed;新增文件的配置内 Ruff、diff whitespace、公开边界扫描通过。故障注入中新文件全绿、旧 oracle 失败是特意执行的灵敏度对照,不能算作正常产品测试失败。未查询、轮询或等待 GitHub CI;未改 PR 分支、合并、操作 active Goal 或真实外部系统。

我的整体评价

REQUEST_CHANGES。测试可以有价值,但“正常运行”不足以支持这里的无 mutation 声明。long_horizon 上,新测试对旧输入写坏后继状态这一持续运行风险仍未给出敏感证据;user_experience 上,生产入口没有变化,不能把测试数增加描述为恢复旅程已经交付。Future-facing pass 的有界建议是复用现有 lifetime fixture、合并重复重放测试,并新增真正敏感的同一路径拒绝后读回;不是新增生产抽象或要求全面迁移。

同作者在约六分钟内提交 #5227/#5228/#5229 三份相似 continuity 测试,需按共同 owner 整合和削薄。我没有仅凭时间或作者身份判负;上述反例及已有覆盖才是本次依据。此为明确的低价值重复提交警告:若在 REQUEST_CHANGES 后继续提交未处理这些问题的同形批次,将建议项目 owner 限制该账号继续提交 PR;本评审本身不执行账号限制。

复审所需证据:收敛后的独立增量、stale bind/unbind 的完整不变性读回、故障被 oracle 捕获、B 合法继续,以及原有 suite/Ruff 通过。测试边界之外的 Todo、claim、plan、跨后端/长期 qualification 仍未认证,不能从本次结果外推。

English verdict: REQUEST_CHANGES - 3c66bc5. The new denial tests miss real successor-state mutation, and much of the 444-line suite duplicates stronger existing CLI coverage. Native tests pass (42 Python, 6 typed); a disposable mutation survives all 10 new tests but is caught by the existing oracle. Consolidate and strengthen the exact boundary; no remote CI or merge action was used.

@huangruiteng

Copy link
Copy Markdown
Collaborator

Frame-aligned conclusion at 3c66bc5: REQUEST_CHANGES. The continuity note is a deferred non-normative design, while the existing source-session / Goal instance owner distinguishes identity from execution authority. This review does not require full Todo/claim/plan implementation. The declared no-successor-mutation assertion is not qualified: a disposable real-transaction mutation after stale denial survives all10 added tests, but existing CLI full-byte readback detects it. Consolidate duplicated registration/replay/resolve paths and add the missing same-path successor invariants and legitimate continuation.

本轮亲自验证:immutable base32/head42 Python 与 base/head6 typed 通过;Ruff/hygiene 通过。相似批次 #5227/#5228/#5229 需有实质独立增量,不能用 PR 数量或绿测试证明 acceptance。完整评审含具体重复提交警告,尚未执行账号限制。当前生产漏洞、完整 R5、远程 CI 和 merge readiness 均未认证;没有改分支或合并。

Full exact-head review.

Address the exact-head review by removing the duplicate 444-line suite and extending the existing CLI lifetime owner. The focused path now proves multiple binding retirement, byte-for-byte successor invariance after stale bind and unbind attempts, absence of stale receipts, and a valid successor bind after both denials.

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>
@Duang777 Duang777 changed the title test(goal): add Instance-qualified continuity end-to-end coverage test(goal): strengthen source-session successor isolation Sep 28, 2026
@Duang777

Copy link
Copy Markdown
Collaborator Author

Addressed the exact-head review in 7e2b8f5de.

I removed the duplicate 444-line suite and moved the independent coverage into the existing CLI lifetime owner. The revised path now checks two-binding retirement, byte-for-byte registry invariance after stale bind and stale unbind, absence of stale receipts, and a successful bind against successor B.

Validation: the lifetime suite is 22/22, the focused test passes, Ruff and git diff --check pass. Two disposable mutation probes flipped B's execution_authority after stale bind and stale unbind respectively; each probe failed at the corresponding full-byte assertion. The repository files were not modified by those probes.

I also narrowed the PR description and removed claims about Todo/claim/plan, constraint recovery, and late-result disposition.

…nd-to-end-isolation

Signed-off-by: duanjialing.777 <duanjialing.777@bytedance.com>

@huangruiteng huangruiteng left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

English verdict: APPROVE - reviewed exact head 5dfc2220fee37d36302be6db74221a8d4cb77bd9 against immutable base 440a19aab2b69dc23748bb734fa58f2a53e2872f. No blocking finding. The previous duplicate-suite and denial-oracle concerns are resolved; this is focused regression coverage, not full Goal continuity qualification.

动机

旧 review 指出的两件事确实解决了:不再维护重复的 444 行独立测试;也不再只用 stale 异常证明“没有副作用”。当前增量守住真实 CLI 的后继隔离,同时确认新实例仍能正常绑定,不把更大的 Todo/claim/plan 连续性目标假装成已完成。

改动思路

沿用现有 lifetime 场景,让 A 的两个 session 都被 recreation 退休,再对旧 A 的 bind、unbind 比较完整 registry 字节。最后用 B 的合法操作作为正例,避免测试只奖励“拒绝更多请求”。没有新增生产规则、schema 或默认行为。

具体改动

关键内容讲解

在 重建与后继场景 中,两个旧 binding 都要出现在退休 receipt。重复 recreation 保持同一个 B 和相同 registry;stale bind 与 stale unbind 的失败都必须不改 registry、不生成该操作 receipt。随后 B 的 successor-session 成功绑定,且 B 的 Goal 行不变。当前整份 diff 只有这个测试文件的 +51/-2;相对上次 review,独立重复文件已删除。

对主干的风险

生产实现没有变化,主要风险是测试出现“异常正确、状态却写坏”的假阳性。我分别注入真实 stale bind/unbind 拒绝后仍写入 B 的故障,两次都在新增的完整字节比较处失败;旧 baseline 场景检测不到 stale unbind 这一边。这不是把故障注入的红测试算作正常失败,而是证明新 oracle 对错误写入敏感。

在 source-checkout 环境执行:

uv run --extra test python -m pytest tests/cli_commands/test_source_session_lifetime.py tests/architecture/test_source_session_registry_denial.py -q
node --no-warnings --experimental-strip-types --test tests/control_plane_ts/source_session_lifetime.test.ts
uv run --extra test python -m ruff check tests/cli_commands/test_source_session_lifetime.py

不可变 base 与 head 均为 24 个 Python、6 个 typed 测试通过;Ruff、diff hygiene 和最终 premerge 的 4 个 direct checks 通过。exact-scope 质量凭证 cqr_2d48297c4575658dfdbd 已验证有效。没有读取或等待远端 CI。

我的整体评价

APPROVE,无阻塞发现。长期维护价值来自删除重复、增加能失败的独立 oracle;现有 CLI 用户体验保持不变,B 的合法继续路径也受到保护。相关的 future-facing pass 已做:复用并收紧现有场景即可,没有必要再加一层 helper。#5228/#5229 已关闭,本批不接受重复拆分。更广的 Goal 连续性及默认激活仍在原有边界之外;本次不执行合并。

@huangruiteng
huangruiteng merged commit 514a602 into loopx-project:main Sep 28, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants