Repository navigation
Conversation
Align patch, LayerNorm, rotary, head-64 attention and vision GEMM boundaries with the pinned BF16 reference. Keep legacy/JEMM behavior and optional ABI compatibility. The original 280-answer whole-model precision gate still fails; this is the bounded vision stage for InferMatrixLab#114.
4 tasks
hsliuustc0106
left a comment
Contributor
There was a problem hiding this comment.
Review draft — #211 "omnijev: add eager unmerged reference Score readout"
Verdict: COMMENTED. The PR is OPEN and explicitly DRAFT; this is a first review, not an approval or completed validation. The contributor's human checklist is deliberately unchecked and stays untouched.
Provenance and pins
- Base:
main@f1e674f2d324f5bb4e1ba2162551d0b633f20be0(matches GitHubbaseRefOidand body). - Head:
72af54fbca76fe34fc525cc4e5d49a7bbe89bd36(matchesorigin/pr/211); tree17fc7ebb41ef696b7e0324e3679e467f194f288f— verified viagit rev-parse 72af54fbc^{tree}. - Diff vs base: 25 files, +1389/−41 (GitHub and local agree). Incremental vs #209: 12 files, +548/−23 = 571 changed lines — matches the body exactly; "483 authored-code" is plausible from the numstat mix (code files dominate; docs are the remainder) and was not separately re-derived.
- Author
psv666, head repopsv666/system1-omni, branchcodex/omnijev114-language-20261120.
Position in the series (cross-reference)
Second commit of the verified linear stack 209 ⊂ 211 ⊂ 212 (first-parent chain 6410e763 → 72af54fb from base f1e674f2; ancestry re-checked with git merge-base --is-ancestor). Not contained in #215 — see the stack note in the #209 body and the #215 review. Review order applied: #209 → #211 → #212 → #215.
Template gates
- The four required contributor answers and the necessity answer are restated inline with a link to #209's section; the draft decision is explicit. Acceptable for a stacked successor; the answers are restated, not just referenced.
- Checklist: four boxes, all unchecked — correct for draft.
- Evidence pins: all verified (base, head, tree, #209 head pin
6410e7634…, incremental compare link, per-stage artifact hashes are author-reported pins consistent with the #209 set).
Findings
No P0/P1/P2 findings.
- Score finishing path:
Executor::score_one_shot(src/models/omnijev/native/src/executor.rs:265-307at head) runs the normal full-row reference path per row (vision forward once, then one unpadded forward per option row via the shared row builder, which routes toforward_multimodal_reference_readout), gathers zq/u, appliesHeads::ordinal(FP32 sequential theta accumulation, sigmoid CDF, differences,max(1e-8)floor — pre-existing production finishing, unchanged), thencontract::answerwith calibration and elapsed time includingpreparation_seconds. Vision and language are synchronized on both success and error paths before results are released. - Unmerged semantics: a separate artifact format
omnijev-unmerged-reference/1is loaded only byExecutor::load_one_shot_reference→export::load_reference, which requires the manifest LoRA profile{rank 32, alpha 64, scale 2, float32, 400 tensors}and validates the complete per-layer module mapping and shapes before any device allocation (export.rs::validate_reference_adapter). The loaders are dual-strict: the merged loader rejects an export that containsadapter_model.safetensors; the reference loader rejects a merged manifest. The language model is loaded withModel::load_with_lora(..., 2.0)plusenable_reference_text_numerics()and requiresfp32_gdn— FP32 LoRA/unmerged arithmetic as described. - Fail-closed surface: full-request
execute/execute_rows, prefix reuse, Graphs (env checked before artifact/library access;reference_graph_is_rejected_before_loadingproves rejection before loading anything) and LM targets (targets.is_empty()guard inforward_multimodal_reference_readout) are all rejected. Negative paths are tested (tests/omnijev/worker.rs::unmerged_reference_original_q2_gateasserts execute/execute_rows/out-of-range/nonscore rejections on the real request). - float_repr / CPython parity discipline (compensated-sum lesson): no new float-reduction claim is made beyond measurement. The body reports the head boundary measured against an official same-hidden control (max
5.960464477539063e-8) and raw finishing independently replayed to round back to the production result; the normative original branch remains the oracle and the plain-row control is explicitly labeled diagnostic. The CPU FP64 log-softmax approximation disclaimer appears in the successor (#212), and this PR's API returns no logprobs (logprobs: Vec::new()). Tolerances: the original unchanged 0.01 gate is hard-asserted in the test; the measured max returned-field delta is 0.0027 against official 0.5613 — measured-then-declared, no tolerance was relaxed. - No CUDA/ABI changes: the delta contains no
.cu,ops.hor Cargo/lockfile changes (verified from the file list), matching the body claim. - Informational:
score_one_shotis a one-Score-question library API by design (documented in body andrecipe/omnijev/README.md); repetition determinism is asserted (zq/u/probabilities equal across two calls). The evidence (original request, oracle) is deliberately private and hash-pinned (a34c2d13…in the test), so these numbers cannot be re-derived by a reviewer without the contributor's archive — recorded as author-reported.
Evidence provenance
- Source inspected: full incremental diff
6410e7634..72af54fbc(all 12 files);executor.rs,export.rs,model.rs(qwen) in depth;heads.rs::ordinaland contract finishing read at the head. - Commands run (personally): ancestry/tree pins; incremental diff stat; numstat reproduction; binary-file scan (none); docs/body cross-check of the measured numbers (0.5640 vs 0.5613, 0.0027 delta — consistent across body, recipe README, and test thresholds).
- Observed CI (both repo hostnames, exact head_sha
72af54fbc…):CIsuccess (44683034925) andDocssuccess (44683034946). Fork repo: 0 runs. - Author-reported: all L20 measurements (freshly-red baseline, same-hidden zq/u bitwise match vs plain-row control, 426/400/25 tensor byte checks, PEFT GEMM equality, 62 asset hashes, export manifest and library SHA256s).
Unverified checks / evidence gaps
- GPU suites (
unmerged_reference_original_q2_gate,reference_selected_rows_preserve_full_row_and_recover, and the existingtext_readout_equals_forward_fixed/ bit-for-bit prefix reuse tests) are author-reported; no GPU here. The Rust CPU weight-free suites could not run locally (no cargo); observed CI covers the workspace checks at this exact head. - All-type serving, full-280, JPEG, HTTP are explicitly unrun in this stage — the PR says so and stays draft. Nothing here should be read as closing #114.
4 tasks
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
OmniJev's fixed selected-readout path rejects unmerged adapters and FP32 GDN. Add an explicit eager one-shot Score library API that runs the existing normal full-row reference path and gathers selected final-norm hidden rows. The original full-context PNG q2 now meets the unchanged 0.01 answer gate. This is not an all-type worker or completion of #114.
Stacked on draft #209 at
6410e7634fd5e077000810ae07dc2f251b4f044d; main remainsf1e674f2d324f5bb4e1ba2162551d0b633f20be0. The new commit is72af54fbca76fe34fc525cc4e5d49a7bbe89bd36, tree17fc7ebb41ef696b7e0324e3679e467f194f288f. The incremental diff against #209 is 12 files / 571 changed lines, including 483 authored-code lines. The main-based PR also contains the unchanged #209 prerequisite; its authorship is retained.The old merged export/worker/defaults remain. The new profile rejects Graphs, prefix reuse, whole-request execution, Choice/Noul and LM targets. The existing fixed/window guards remain intact. No CUDA implementation, ABI, ops.h or Cargo/lockfile changes.
Required human contributor answers
The contributor's explicit shared intent and necessary-draft decision are recorded in the preceding stage's human-answer section. They remain applicable to this bounded successor of RFC #114:
PR needed: Yes; the contributor authorized necessary implementation, L20 verification, self-review and a reviewable draft for each bounded stage. This PR is kept draft while all-type/full-corpus acceptance remains outstanding.
Evidence
Actual observed failure and result, preserving all 12 original questions, prefix 581, usage 1214, image and checkpoint identity:
The baseline is freshly red twice; the candidate is repeated exactly at hidden/head boundaries. Maximum returned-field delta is 0.0027. The protected Score decision remains
harmless(reference raw top-two margin 0.5176341654616531). Original 0.01 and protected-margin gates are unchanged.The new selected zq/u match every BF16 value in a freshly run official unmerged plain-row control. That control is diagnostic; the normative original branch remains the gate oracle. Actual loaded 426 base, 400 FP32 adapter and 25 FP32 head tensors all match the new export's dtype/shape/bytes. With identical
[581,2560]input, the existing reference base GEMM, LoRA A/B GEMMs and final scaled/cast projection match actual PEFT exactly, including[581,8192]output. The production ordinal head versus actual official same-hidden head differs by at most5.960464477539063e-8; raw finishing is independently replayed and rounds back to the production result.Frozen artifacts: official source
14dbec4f71e194852c8d7b88ab36ef639493f400, checkpointffe5f436eaf22e20e2f041f8e74e121fd057a6cb, base851bf6e806efd8d0a36b00ddf55e13ccb7b8cd0a. Python 3.10.12 / Torch 2.14.0+cu130 / Transformers 5.17.0 / PEFT 0.19.1, no FLA; 64-package hash lock is unchanged. One L20 GPU0, Rust 1.100 / native CUDA 12.8. This is numerical validation, with no performance claim.Test plan and result
cargo fmt --all --check; workspace locked Clippy/all-targets/omni-laya/serve/-D warnings; ordinary workspace tests; locked release workspace build; strict MkDocs; test discovery; exporter AST: pass. Ordinary tests: 428 passed / 0 failed / 125 ignored. Ignored checks are not counted as device passes.reference_selected_rows_preserve_full_row_and_recover, newunmerged_reference_original_q2_gate, existingtext_readout_equals_forward_fixedand existing OmniJevprefix_reuse_matches_plain_passes_bit_for_bit: pass. Selected rows cover order, duplicates, last-row equivalence, invalid rows/LM-target rejection, scratch growth, restoration and existing prefix/fixed guards.reference_graph_is_rejected_before_loadingwith Graph requested: pass, before artifact/CUDA loading. A real old worker process rejects the unmerged format at startup with expected exit 1, before readiness.f5ffba7e53e3289b0019a90bcb487483f7bc33e122c2d57ceaafafe2cf5e33bf; immutable CUDA library SHA256:b1f08a2177ffe5aa5701c710eec3a9dd71c0efe43b1b6b12bc6182f2c0e061e5.Unsupported/unrun: new-profile all-type/full-280 replay and HTTP, reference Graph/reuse, JPEG repair, other GPUs/Metal, whole real-weight Cua/Open-Jev models, and performance. The previous #209 full-280 gate remains failed; this stage does not relabel its old results as a fresh candidate run.
Demo / evidence and successor
Maintained reproduction and mode boundaries are in
recipe/omnijev/README.md; all tests/helpers are under roottests/. No image/video demo is applicable to this library-only numerical slice. Original requests, model tensors, raw logs, exact source/library hashes and all failed controls are retained in the contributor's private evidence archive; they are not committed or redistributed here.LM features need a separate slice: pinned
mso/branch.py::branch_forwardbatches the original request's 35 rows in one chunk, producing[35,33,2560]suffix hidden. Its option-token projection input is[112,2560], then zq input[31,2560], with vocabulary 248079 and BF16 logits cast to FP32 before log-softmax. These shapes/order are derived from the exact original compiled rows and pinned source, not presented as an LM same-hidden measurement. CurrentModel::vocabulary_logprobsgathers up to 64 targets per call under the fixed-row-independence contract. Reusing that grouping with a reference handle has not been validated; the new API therefore rejects LM targets. The successor needs explicit grouping/ownership plus actual same-hidden projection/log-softmax and six-feature validation before all-type/full-280/HTTP acceptance. No new CUDA need is asserted by this stage.Merge Danger
Door: two-way. The new artifact and library entry points are explicit, and existing worker defaults remain merged.
Blast Radius: bounded. Shared Qwen gains a guarded full-row readout API; OmniJev owns artifact/profile/Score orchestration. Existing default consumer regressions pass, but complete #114 acceptance and new-mode serving remain outstanding. Keep draft; do not close #114 or advance media/performance gates.
Self-review
Agent-assisted full incremental diff/architecture/numerical/lifetime/artifact review completed; it does not replace human or maintainer review. Human checklist remains for the contributor: