Repository navigation
feat(knowledge): add full-catalog selection, applicability checks and bounded reads - #534
amy-why-3459 wants to merge 7 commits into
Conversation
Add full evidence cards, explicit Agent selection, condition checks, source-bound assessments and reference expansion with atomic required resources. Agent-assisted-by: Codex (implementation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Retain fixed inputs, Agent selections, qualified read results and source-quote assessments. Report observed omissions and unverified task benefit. Agent-assisted-by: Codex (implementation and evaluation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Dong1017
left a comment
There was a problem hiding this comment.
PR review verdict — DO NOT PASS
Reviewed base 1b826b5f9666a8b734000e1eb2990e582cdef876 and head 3f6d5d32ff42e5fbab1644988d28f02fd0d499c0 (first review on opened; pr/prior-reviews.md carries no formal review on this head, so there is no prior verdict to weigh either way). I read governing Issue #529, including its five 完成条件.
Only docs are visible in the seeded patch, and the patch is truncated. pr/meta.json has diffComplete: false, and pr/diff.patch stops mid-array inside docs/reviews/catalog-selection-529/queries.json. Visible: docs/catalog-selection.md (new), docs/operations.md (+1 table row), and docs/reviews/catalog-selection-529/{README.md, assessments.json, comparison.json, protocol.json, queries.json (partial)}. I cannot PASS on a truncated diff, and the missing portion is exactly the part that carries #529's的实现 acceptance.
Blockers
-
The
feat(knowledge)surface is not in what I could see. The title anddocs/catalog-selection.mdname a CLI (inferrsi catalog-select),inferrsi.apps.knowledge.select_catalog_evidence,confirm_catalog_evidence,inferrsi.knowledge.catalog_selection.CatalogSelection, andtools/compare_catalog_selection.py. Nosrc/module, no CLI wiring, no test file appears in the visible patch. That leaves #529 完成条件 2 (one real end-to-end chain, "交付不能仅停留在导出目录或提示词") and 条件 5 (端到端测试) unverifiable here. Either republish the complete diff or point me at the exact paths and revisions so the next pass can read them. -
The recorded run is scoped to a different revision than this head.
docs/reviews/catalog-selection-529/README.mdpins the code candidate asc457c85, while the reviewed head is3f6d5d32. The README says it explicitly: after fixing the reference closure only deterministic reads were replayed, reusing the same baseline and the pre-fix selections ("修正引用闭包后只重放确定性读取", "候选决定在代码修正前保留"). Soassessments.jsonandcomparison.jsoncharacterizec457c85, not this head. Nothing in the visible patch mapsc457c85onto3f6d5d3. State that relationship (ancestor? equivalent for these artifacts?) or re-run. Good news on the dependency: the baseline1b826b5is this PR's own base and the README records #516 as merged there, which clears the unmerged-dependency blocker from the #529 triage. -
Delivered packages and context-limit omissions are indistinguishable in the trace. In
comparison.json, entryid: "version-alignment"listspackage:inferrsi.ascend.skills.vllm_ascendincandidate_delivered(expansion path,status: "delivered", 86,668 bytes) and also incandidate_omissionswithreason: "context_limit",required_bytes: 21756,available_bytes: 9828. Entryid: "day-0"does the same forpackage:vllm_omni.common.omni.stage_ownership_and_search(delivered 3,044 bytes; also omitted at 18,293/5,223). Those omission records carry noresource/pathand no scope discriminator, so a reader cannot tell which declared resource was dropped. That collides withdocs/catalog-selection.md, "Reads, expansion and trace": "Required skill procedures/resources, source attribution and license closure are atomic: insufficient space produces an omission, never a partial procedure marked complete", and with #529 条件 4 (终止原因、缺失材料与预算不足均可观察). Add the specificresource/path(or an explicitscope: package|resource) to omission records, and confirm the 86,668-byte delivery did not include a partial required procedure. -
Byte accounting does not say whether the typed reader metadata is inside the budget.
docs/catalog-selection.mdstates "read_bytesis the complete bundle size, not a promise about budgeted context size", and the CLI "counts that exact transport, including its accounting envelope". #529's dependency note records that #516'sread --materialpayloads grew ~45% because themetadata.inferrsiblock is delivered inside content. Ifcontext_bytes/expansion_bytesaccounting excludes that block,required_bytes/available_bytesundercount and the over-budget omissions above would be wrong. Please state one way or the other in the doc; this is a question, not a proven defect.
Non-blocking
- In all 9
assessments.jsonentries,claimandreasonare byte-identical. The doc describes them as distinct fields ("the specificclaim,verdict,reason"). Confirm the interface populates them from separate inputs rather than one variable. README.mdreports 168 passed / 2 skipped and a full-repo attempt stopped at 1,143 passed / 12 failed / 12 skipped with all three representative failures reproduced on the unmodified baseline. That reporting style is right, but I ran nothing — no test file is visible, and CI state is not seeded.
Credited from what I could read: cards are capped per field with explicit truncation flags while the candidate list is never cut (that answers the uncapped-limitations/card-size concern from the #529 triage); over-budget input returns catalog_over_budget with ordinary Find results and no selector call; unqualified, outside-snapshot and duplicate refs fail rather than resolve silently; match/analogue/unknown/conflict is kept separate from source verification level; and benefit is stated as INCONCLUSIVE rather than claimed. The review artifacts also disclose their own degradations per query honestly, including the two queries where the selected Record was never delivered.
Not verified here: the implementation itself, its tests, CI status, and the ten-query run. This review does not approve the change or alter the scope of #529. I will re-read the full head after the author pushes or once the complete diff and the c457c85 → 3f6d5d3 relationship are in the seed.
Posted by Dong-bot, an automated agent acting on behalf of @Dong1017 (always-on-bot, model claude-sonnet-5-5).
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
初次独立检视 1b826b5f → 3f6d5d32,按 #529 的完整目录和可恢复引用契约检查。批准前需修两项:
catalog_selection.py:222–249:可选 Markdown URL 直接交给urlsplit(),其解析异常会中止整个 snapshot,预算超限时也不能正常回退到 Find。应只把坏引用记为不可解析,继续生成其余候选;加测试证明同集合的有效候选和回退均仍可用。catalog_selection.py:70–83:词法检查把1.4.x识别为精确版本,和实际1.4.2比较得到 conflict(方法用途下 analogue),与文档承诺的非精确版本归 unknown 不符。请区分明确版本与通配/范围,覆盖对照用例。
已静态核对实现、文档和测试定义;未执行 CLI、测试或查询 CI。#529 的设计结论不变。
There was a problem hiding this comment.
Independent third-party tip COMMENT (cursor[bot]; not Approve / not Request changes).
Head 3f6d5d32ff42e5fbab1644988d28f02fd0d499c0 still carries tip DO NOT PASS from Dong1017 (always-on) and NO PASS from zengchuang-hw. I am not clearing those.
Blockers I independently treat as still open before any cursor formal vote:
- Dong: complete
feat(knowledge)surface must be reviewable (seed/diff completeness; #529 end-to-end acceptance), recorded-run revision vs tip head relationship, omission records that cannot distinguish delivered vs context-limit drops, and explicit byte-accounting scope for typed reader metadata. - Zeng:
catalog_selection.pyURL parse failures must not abort the whole snapshot (isolate bad refs; keep Find fallback + tests), and lexical version checks must not treat1.4.x-style ranges as exact versions.
Ready checks are green on this tip, but that does not override the review blockers. Waiting on author push; will re-verify after tip moves.
Technical verify only — collaborator permission none; not write+ / dual / merge-ready.
… digests Retain full source-content freshness checks, invalidate changed catalogs, and fall back to full rebuilds for links or redirects. Share immutable resource digests within readers while validating metadata and dependencies on every manifest. Add cache mutation regressions and a reproducible retained-choice timing tool. Agent-assisted-by: Codex / unknown [effort=unknown] (implementation and evaluation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Keep Find response normalization separate from the structured catalog expansion trace. The first measurement attempt stopped during result serialization; retained-choice search decisions are unchanged. Agent-assisted-by: Codex (evaluation tooling) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Retain fixed-version three-way interface timings for the Qwen query and ten existing queries, exact semantic comparison digests, all latency samples, validation scope and the failed timing adapter attempt. Model choice and device inference remain outside this measurement. Agent-assisted-by: Codex (evaluation and documentation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
增量 3f6d5d32 → 05fa349f,上轮检视。缓存与计时已加入,但两个批准前必修项未变:catalog_selection.py:314 仍会因一个格式错误的可选链接中止整份目录和 Find 回退;:77 仍把 1.4.x 当作精确版本,导致约定应为 unknown 的比较变为 conflict。请按上轮补修与回归。
新增缓存路径还在构建前遍历并读取整树(:134–165,224)。一个正常读取不使用的文件不可读,也会在候选级遗漏处理前中止。建议失败时跳过缓存、回到原读取路径;实际选中材料变化时明确报告 snapshot 过期,不能复用旧结果。
已静态核对增量、当前实现和测试定义;未运行代码、计时或查询 CI。
There was a problem hiding this comment.
Independent third-party tip COMMENT (cursor[bot]; not Approve / not Request changes).
Head moved 3f6d5d32ff42… → 05fa349f197259482a0978e61753eb81c04cb1e4. Prior tip COMMENT on the old head does not cover this revision. I am not clearing the open NO PASS from zengchuang-hw on this tip, and Dong1017 always-on DO NOT PASS on the prior tip still stands until an always-on / write+ re-review clears it.
Independently confirmed on this tip:
- zeng blockers remain: malformed optional Markdown URL can still abort catalog/Find fallback; lexical check still treats
1.4.xas an exact version (conflict vs unknown). - Ready lane green does not clear merge eligibility: tip
declarations/knowledge-model-policyare noncompliant —Exception requires explicit independent owner approval;Material AI author 1: model is unapproved or suspended(run 44674018831).
Defer formal cursor vote until author resolves the NO PASS items and declarations/KM are compliant. Technical/third-party verify only (collaborator permission none) — not a write+ eligible Approver under docs/agent-collaboration.md.
…ailures Track required checkout provenance and notices in the pinned bundle. Compare declared backends and exact versions consistently, preserve Find fallback for malformed references and skip cache reuse when optional scans fail. Identify omitted entries and linked resources separately. Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (implementation and evaluation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
Record the source revisions, per-query delivery digests and cited omissions from the fresh-snapshot replay. Link the original evaluation to its implementation and preserve the frozen selection inputs. Agent-assisted-by: Codex / gpt-6.2-astra [effort=xhigh] (evaluation and documentation) Signed-off-by: amy-why-3459 <wuhaiyan17@huawei.com>
zengchuang-hw
left a comment
There was a problem hiding this comment.
NO PASS
增量 05fa349f → 27a8ba59,上轮检视。此前三项已对应修复:坏链接只记为无效,通配版本保持 unknown,预扫描失败不再拦截正常读取。对应回归定义已补。
新的兼容性阻塞:knowledge_operations.py:666–674 将祖先 LICENSE 注入共享 checkout bundle,而非仅用于新目录选择模式。audit-vllm-kv-lifetimes 和 diagnose-vllm-admission-pressure 的原 source 未变、也未声明该文件,却会新增 systems/engines/vllm/LICENSE,改变 bundle revision;现有显式 read/resource --revision 会在 :560 拒绝旧 pin。这违反 #529 对未选新模式调用方的读取兼容要求。请把新增闭包限定在新模式,或提供明确的旧 pin 兼容迁移,并补一个旧 pin 在同一来源下仍能读的回归。
仅静态核对源码、文件树、需求和测试定义;未执行测试、CLI 或查询 CI。
|
Posted by an agent on behalf of @Dong1017 (incremental review only; points already raised in this thread or addressed by the author are not repeated). PR review verdict — PASS (incremental, non-voting comment) Reviewed base Earlier blockers, re-checked on this head
New findings (non-blocking)
Nothing here changes the verdict: both items concern card/trace observability, not selection correctness, atomicity or the read-only boundary. What I verified positively
检查范围 / Scope checked: read PR body, the |
There was a problem hiding this comment.
Independent third-party tip COMMENT (cursor[bot]; not Approve / not Request changes).
Technical verify only — collaborator permission none; not write+ / dual-tip / merge-ready.
Tip moved since prior cursor COMMENT (05fa349f… → 27a8ba59b4999155b4e22bf62819fb38c82ba505). Ready tip green (full-check, declarations, installed-package). Independently re-checked against zengchuang-hw tip NO PASS on this head.
Confirmed tip blocker
- LICENSE injection escapes the new selection mode. In
knowledge_operations.py_skill(~L666–674), ancestor LICENSE/notice files are folded into every skill checkout bundle, not gated to the new full-catalog selection path. Skills whose declared sources/resources never included those notices (e.g. audit/diagnose vLLM skills) gainsystems/engines/vllm/LICENSE(and similar), changingbundle.revision. Existingread/resource --revisionpins then hit the revision mismatch reject at_read(~L560–563). That breaks #529's compatibility requirement for callers that did not opt into the new mode. Prior three blockers look addressed on this tip; this compatibility regression remains blocking.
Not Approve
Wait author: scope LICENSE/notice closure to the new mode (or ship an explicit old-pin migration) and add a same-source old-pin read regression. Not Request changes from cursor.
Governing Issue and delivery slice
Refs #529
The requester authorized implementation, a PR, and a ten-query knowledge retrieval comparison (not ten hardware optimization rounds), with selection performed in the current conversation.
This adds an opt-in complete-catalog retrieval path over the shared Skill/Record/Case corpus. The current Agent selects snapshot-bound qualified references; Knowledge checks conditions, expands explicit references within budgets, and delivers original resources with pinned revisions. Find remains the default. Flat checkout Skills now expose an explicitly declared backend during discovery. The seven primary task routes retain their existing behavior. Over-budget catalogs fall back explicitly to Find.
catalog-selectand the Apps Python selector seam share snapshot generation, identity validation and read assembly.本次检视与修复
代码修复为
15b3459,27a8ba5只增加复验报告。首次评估版本c457c85到首次被检视的3f6d5d3也只有报告变化。评审种子截断可以解释当时看不到实现。可从固定版本直接进入 CLI、Apps 选材与引文确认、目录与读取、回归测试 和历史版本差异。vllm-ascend首次递送的 86,668 字节包含全部 4 份必需材料,遗漏为后续model-performance.md展开(21,756 / 9,828 字节)。Day-0 的 3,044 字节 Record 已完整递送,后续omni-architecture/SKILL.md展开超限(18,293 / 5,223 字节)。metadata.inferrsi从 procedure 投影剥离,预算按投影后字节计;另行递送的 JSON 计入全部字节。选择输入另外计算整个序列化 envelope。历史 9 条评估各自输入了相同的 claim/reason,独立字段回归已覆盖不同文本。验证 相关测试 244 项通过,2 项跳过,另有 2 个子测试通过。Ruff、
git diff --check及独立内部代码检视通过。固定查询在新快照重放前逐个核对了全部 135 对 ref/revision。10 题使用 25 个原选择、递送 19 项、9 条引文校验通过。另一个 Qwen 1.35x 查询选 5 项、递送 4 项、4 条引文校验通过。两组的条目级 verdict、递送顺序、版本、资源路径和原文字节均与先前记录一致。本轮没有重新进行 Agent 选材、设备推理或性能计时。逐题复验结果 · 复现方法上轮评论引用的声明检查曾经失败。同一旧提交
05fa349随后已通过声明检查。下方声明绑定本次 PR head,正式审批由独立 reviewer 给出。Changed artifacts and evidence
Usage and contracts · ten-query report · qualified results · source-quote assessments
The original comparison pins baseline
1b826b5f9666a8b734000e1eb2990e582cdef876and implementationc457c85. Subsequent commits optimize catalog construction and read validation; the fresh timing comparison below pins both implementations and verifies identical source delivery. Knowledge bytes are unchanged: 80 supporting Skills, 48 Records and 7 Cases. Queries reuse the repository's seven task fixtures plus three labelled Qwen/quality-boundary questions.Find returned 50 entries and delivered 39 owners. The Agent selected 25 entries and the final implementation delivered 19 owners; six selected owners could not fit their required resources. Each query's IDs and omissions are retained. The confirmed full-image quality failure question deliberately selects no extra evidence. Full transport size is 77,086–77,113
o200k_basetokens per query under the declared 128,000 budget and 4,096 reserve.These are observational outputs, not accuracy or speedup claims. The same current conversation was used; the Agent inspected all 135 ID/summary entries, then relevant source excerpts. Actual model usage and decision latency are unavailable. Fixed Agent decisions were retained through a correction to deterministic reference reads; the final replay reused the same baseline and selections. No outcome-driven model reruns were used. Task benefit remains INCONCLUSIVE.
改动前后对比(10 个固定查询)
本次对照是旧版 Find 与显式启用的新模式。合入本 PR 后,未启用新模式的调用方仍沿用原有 Find。知识内容、题面与任务入口保持一致。
上述数量按题累加,同一材料跨题重复计算;“成功递送”表示 reader 实际返回原文字节,不表示模型已完整阅读或采用。返回数量不同不能作为精确率、召回率或效果提升的依据。
逐题全部命中与选择
技能采用稳定短 ID;Record/Case 保留原 ID。未递送表示新模式选中了材料,但受读取预算限制没有返回完整必需资源。最后一列只计实际成功递送。
ascend-model-adaptationvllm_omni.ascend.diffusion.qwen_image_21_eager_t2isglang_omni.common.audio.serving_and_streaming_qualityvllm_omni.common.audio.streaming_serving_measurementascend-video-diffusion-optimizationascend-model-adaptationvllm_omni.common.omni.stage_ownership_and_searchvllm_omni.common.diffusion.production_integration_gatesascend-video-diffusion-optimizationvllm_omni.common.diffusion.measurement_and_profilinginference-benchmark-designvideo-diffusion-optimizationhost-overhead-optimizationvllm_omni.common.diffusion.measurement_and_profilingperformance-ceiling-analysis(未递送)host-overhead-profilingcase.qwen-b300-rejected-directionsascend-graph-modepytorch.cuda.compiler.cudagraph_shapes_and_lifetimesflashinfer.cuda.llm.paged_decode_dispatchascend-aclgraph-capture-replaypytorchpytorch.cuda.compiler.cudagraph_shapes_and_lifetimes(未递送)inference-benchmark-designqwen3_tts_engine.ascend.audio.fixed_customvoice_promptqwen3_tts_engine.ascend.audio.codec_constants_and_stateqwen3_tts_engine.ascend.audio.graph_batching_state_ownershiptorch-npuinference-benchmark-design(未递送)sglang_omni.common.audio.serving_and_streaming_qualityvllm_omni.common.audio.streaming_serving_measurementvllm-ascendvllm-metalvllm_omni.ascend.diffusion.minimax_h3_reference_kv_reuseascend-model-adaptationaudit-afd-p2p-exchangesrepository-evolution-analysisvllm-ascendtorch-npu(未递送)deepinfer.common.llm.zero_overhead_deferred_schedulingdeepinfer.common.llm.bounded_windowed_replayvllm-metalhost-overhead-optimizationdeepinfer.common.llm.speculative_verification_budgetdeepinfer.common.llm.zero_overhead_deferred_schedulingvllm.common.llm.runtime_and_kv_ownershipvllm.common.llm.extension_correctness_gatesrepository-evolution-analysisinference-benchmark-designproject-mayaweight-loading-optimizationstrata.common.llm.engine_validation_contractcann-attribution-overlapinference-benchmark-design(未递送)kernel-writing-guideclaim-contractcase.qwen-b300-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-ascend-rejected-directionsvllm_omni.cuda.diffusion.minimax_h3_qk_norm_ropeinference-benchmark-designcase.qwen-b300-rejected-directionsinference-benchmark-design(未递送)vllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-ascend-rejected-directionscase.qwen-b300-rejected-directionsascend-graph-modeascend-model-adaptationcase.qwen-ascend-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2icase.qwen-b300-rejected-directionsinference-benchmark-designcase.drama3-rejected-directionsvllm_omni.ascend.diffusion.qwen_image_21_eager_t2iascend-operator-tuning逐题解释、退化与结论边界
o200k_base预算计数不能替代实际会话用量。完整材料:对比报告 · 逐题机器结果 · 原文确认 · 固定题面。
检索性能优化与复测(2026-11-20)
The performance follow-up reuses the prepared catalog, reference graph and reader discovery while full source-content checks still run before and after delivery. Shared resource validation/digests are reused by immutable bytes; metadata and dependency versions are validated each time. Same-size edits with restored timestamps, unselected-resource changes, store updates and source mutations during construction/delivery invalidate the snapshot. Caller edits cannot poison the retained catalog. Symlinks/redirects disable prepared reuse and use fresh full builds.
固定原 Find
1b826b5f9666a8b734000e1eb2990e582cdef876、优化前新模式3f6d5d32ff42e5fbab1644988d28f02fd0d499c0、优化后fc6c144a60c56781a04eadf92a530ec9a557defb。表中优化后实现到05fa349的提交仅记录报告。本表计时止于该版本。各次使用新进程、交替顺序、相同知识和读取预算,重放同一份 Agent 选择。qwen2.1-image 优化到1.35x加速,各 5 次Qwen 的目录准备为 2.238 → 1.284 s,校验/读取为 4.685 → 0.314 s。所有查询的 snapshot ID、选中 refs、实际递送原文字节和版本、条件判断、遗漏与引用展开路径摘要均一致。没有缩减 135 条目录,也没有删掉新鲜度检查或依赖闭包。
计时仅覆盖接口构建至原文递送,排除进程启动、import、tokenizer 初始化、序列化及 Agent 选材;操作系统页缓存未控制。模型和外部知识调用均为 0。此处 1.35x 是查询目标,没有运行 Qwen 设备优化;任务收益保持 INCONCLUSIVE。
全部计时样本、版本、结果摘要及保留选择 · 复现工具
Independent internal review passed after fixing two demonstrated cache freshness gaps (explicit resources named
.pyc/incidental names, and authored Record symlink targets). The reviewer checked the four implementation/test files at diff SHA-256fd61754686866ae7cadd45be7da2cbc79b4f8c36acaf99d1ed56d7818881743aagainst3f6d5d3, with 13 independent tests passing. This is not an eligible formal GitHub approval. The first timing attempt stopped in result serialization for the legacy Find assembly label; its failure was retained, and the corrected runner completed without new Agent selections.Checks and dependencies
ruff check src tests tools: passed.git diff --check: passed.Remaining work and Issue closure
Formal independent PR review and CI remain separate. The report explicitly retains large-resource omissions, missing target-native evidence, absent independent scoring and conversation-isolation limits. Goal acceptance/closure remains with the requester and maintainers; this PR uses a non-closing issue reference.
Knowledge authorship declaration
The requester identifies the Codex author of the shared reader optimization as GPT-6.2 Astra (
gpt-6.2-astra) with extra-high reasoning (xhigh).