Repository navigation
Conversation
This was referenced Nov 20, 2026
4 tasks
hsliuustc0106
left a comment
Contributor
There was a problem hiding this comment.
Draft review — PR #208 "fix(vela): enforce the 16384-token native row limit"
Verdict: COMMENTED (first-pass review of a DRAFT; draft state must not change and no approval, merge or completed validation is implied). Nothing was posted; this is a local draft body.
Review state and provenance
- Draft state: OPEN,
isDraft: true(verified twice viagh pr view, start and end of review). Head unchanged at1d460564165310498a1c3d8afddbdc202b4d0dfe. Head repoInferMatrixLab/system1-omni, branchcodex/vela175-row-cap-20261120; basecodex/vela175-processing-base@cdff2999b9e082e246f933235b608d7a7e5978b2(verified in git: MrDongsls's #175 head, single parentb49f7c69). Authorpsv666(first-time contributor on my record). - Template: body contains all four post-#138 answers, the necessity decision, bug-fix failure evidence (before: debug 5 passed/4 failed, release 6/3, with admitted 16,385-token rows, 16,443-token windows and "attempt to subtract with overflow (render.rs:674)"),
## Summary,## Purpose,## Test Planwith the "Version / commit:" field,## Test Resulttable,## Demo / evidence(pinned modelvllm-sr/Vela-2.0-0.8B@326b01d8..., tool versions, off-repo evidence archive with SHA-256),## Merge Danger,## Self-reviewchecklist present and unchecked (left untouched). - Gate status: intent answers attributed to contributor confirmations 2026-11-19/20 retained in an off-repo task record — not independently verifiable from the repository; recorded as author-reported. Draft → COMMENTED regardless.
- Skill load record: same as sibling reviews — local unpublished skill
/Users/hsliu/thinkflowlab/system1-omni/.agents/skills/system1-omni-review/SKILL.mdSHA-25658be3bc60806c4c5af7038a0c6a80fadb94f3360d714556a3e965c8903dde108;references/repository-contracts.mdSHA-256d6ae9650e4e18e8986926a790339412f52750fd1eeb19d358e5d91cd547f1202.
Base/head and merge-base
- Merge-base with mirror
origin/main(5fa5179d5):5fa5179d5(mirror head). Against canonical main (614b0abc, fork): merge-base is285c7910— canonical main is 3 commits ahead of this PR's effective base point (#196, #204, #214). The base branch predates #196/#204, so retargeting onto current main is required after the parent stack lands. gh pr view 175→ OPEN, unmerged.
Diff (focused: cdff2999..1d460564)
5 files, +287/−4 — matches PR metadata (--numstat: recipe/vela/native.md +24/−2; src/models/vela/README.md +15/−1; src/models/vela/native/src/processing.rs +4/−0; src/models/vela/native/src/render.rs +7/−1; tests/vela/native_row_limits.rs +237/−0). One commit, authored by psv666. Total changed text ≈ 291 lines; the body's "249 changed Rust lines" is consistent with render+processing+tests numstat. Far below large-change thresholds.
src/models/vela/native/src/render.rs: (a) line 674:64.max(self.max_length - budget)→64.max(self.max_length.saturating_sub(budget)); (b) aftern_tokensis computed for a completed row (lines 771-776 at head):if n_tokens > self.max_length { return Err(too_long(self.max_length)); }.src/models/vela/native/src/processing.rs(+4):#[cfg(test)] #[path = "../../../../../tests/vela/native_row_limits.rs"] mod row_limit_tests;— test bodies stay in the repository-leveltests/tree per CONTRIBUTING;src/holds only registration.tests/vela/native_row_limits.rs(+237): 9 weight-free regressions.recipe/vela/native.mdandsrc/models/vela/README.md: document the per-row 16,384 cap, its deliberate narrowing of the pinned Python input domain (20,085-token reference row at the model-call boundary), that summedusage.input_tokensis not capped, and the retained 422 multi-part-span / 400 non-finite-JSON wording differences; README's stale "26 tests" count replaced.
Review questions
- Where is the limit enforced: in the renderer,
RenderContext::render(pub fn render, render.rs:561; gate at render.rs:773), which is called fromProcessor::plan(processing.rs:96) at processing.rs:158 for the ordinary row and processing.rs:174 for each span-window row. That is the processing/planning stage — the last step before any batching/device execution (at this base the crate is a library plus a parity example; serving runs through the reference Python worker). It is not an admission-time check (contract.rs::admit_requestis unchanged) and not a loader check — the body's "enforce at the renderer shared by ordinary/window paths" matches the code. - Fail-closed: yes.
context.render(..)?propagates theRequestErrorout ofplan(), so a single over-limit ordinary or window row rejects the whole request with the existingtoo_longerror — status 413, body{"detail":[{"loc":["body","questions"],"msg":"the questions alone do not fit the 16384-token window","type":"too_long"}]}(render.rs:385-392). The saturating_sub change additionally removes the debug overflow panic and, in release, the wrap-around that previously produced a huge assemble budget and silently admitted oversized rows; the final gate now bounds both modes. - Test pinned: yes — 9 weight-free regressions through the real
Processorwith a tinyWordLevel/WhitespaceSplittokenizer (no checkpoint, no GPU): cap−1 and cap admitted; cap+1 rejected; 20,000-word span header → 413 without panic; non-span kinds (noul/choice/score/set) → 413; mixed rows whose sum exceeds the cap but each row ≤ cap, with question order pinned; an oversized later row rejecting the whole request; window rows each ≤ cap with total > cap; raw-state clipping unchanged; an oversized window rejected while the primary row fits. Rejections assert both status 413 and the exact body JSON. - Documented contract: consistent.
bundle.rs:106pins the manifest identitymax_length == 16384andbundle.rs:122-123defaults to 16,384; the gate uses exactly this value. The docs updated in this PR state the cap is per completed row (parts + question blocks), that the request's summed usage may exceed it, and explicitly frame it as a restriction relative to the pinned Python runtime — with the 20,085-token reference observation disclosed as not establishing successful inference. The author-reported comparator treats that single out-of-domain observation as an approved exception; all in-domain mismatches still fail (author-reported).
Findings
No actionable findings (P0–P3) in the focused diff.
Informational notes:
- N1: The 64-token floor in
64.max(max_length.saturating_sub(budget))is intentionally retained; the new post-assembly gate is the backstop for the oversized span header/repeated target it can produce. The in-code comment and the docs describe this accurately. - N2: The tests construct
Processor { .. }by struct literal rather thanProcessor::load, coupling them to the struct's field set (they compile inside the crate viacfg(test), so this is visible breakage on field changes, not silent drift). - N3: The cap is per row, not per request — requests with many valid rows/windows can still sum far beyond 16,384 input tokens. This is documented and tested (
mixed_rows_can_exceed_the_cap_in_total_and_keep_question_order,window_rows_can_exceed_the_cap_in_total), not a defect, but reviewers/consumers should not mistake it for a request-level budget.
Evidence separation
- Source inspected: full focused diff including the 237-line test file;
render.rsat head (render entry, budget assembly, gate,too_long);processing.rsat head (plan, render call sites,RenderContextconstruction);bundle.rsmanifest pin;contract.rs(RequestError statuses); both docs files at head. - Commands personally run: ref/SHA and ancestry checks (
git merge-base --is-ancestorused for cross-PR work below); worktree/tmp/pr-review-pass194/wt208@1d460564;PYTHONPATH=src /Users/hsliu/miniconda3/bin/python -m pytest tests/vela -q→ 138 passed (9.3 s);PYTHONPATH=. ... pytest tests/benchmarks -q→ 133 passed (28.2 s);mkdocs build --strict(mkdocs 1.6.1) → exit 0 (docs touched, so required). Nocargoon this host — the Rust regressions were not runnable here. - Observed CI: none.
gh pr checks 208→ "no checks reported on the branch". Actions APIhead_sha=1d4605641...→total_count: 0in bothThinkFlowLab/system1-omniandInferMatrixLab/system1-omni. - Author-reported (not verified here): all cargo debug/release regression runs (9/9 each), ordinary Vela Rust (46 passed, 2 ignored), fmt/clippy/release build, workspace tests (450/103), strict MkDocs, the five byte-equal token plans, real-bundle processing tests, 16 admission probes against the reference, and the off-repo evidence archive contents/hash.
Material unverified checks
- The Rust regression suite and production gate behavior as executed by cargo (no Rust toolchain locally). The Python-side suites and docs build were verified here, but they do not execute
render.rs. - Model forward / native HTTP / GPU behavior — explicitly not claimed by the PR; none run.
- Comparator state (approved out-of-domain exception) — author-reported only.
Cross-PR and integration
- #210 is stacked directly on this PR:
1d460564is a verified ancestor of #210's base4792f8c7. Overlap is docs-only (recipe/vela/native.md,src/models/vela/README.md— both normalize the same trailing "Checks:" sentence; no conflict in stacked order). No Rust file overlap with #210 (render.rs/processing.rsvsheads.rs/executor tests) or with #195's executor code. - #195 is OPEN/unmerged (verified via
gh pr view 195;mergedAt: null) — contrary to any premise that #195 is merged; canonical main has no native Vela crate at all. The whole chain #175 → #208 → (#195) → #210 must land in order, with retargeting onto current main (3 commits ahead of this PR's base point) and fresh checks at each hop.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Reject an entire native Vela 0.8B request when any completed ordinary or span-window row exceeds 16,384 tokens. The previous renderer could admit a 16,385-token row or 16,443-token windows, and a sufficiently long span header panicked in debug during unsigned budget subtraction.
This is a stacked supplement to #175. Base
codex/vela175-processing-baseis pinned to MrDongsls'scdff2999b9e082e246f933235b608d7a7e5978b2; head1d460564165310498a1c3d8afddbdc202b4d0dfepreserves that history. Five files, +287/-4, including 249 changed Rust lines. Retarget/integrate after the author stack lands and recheck. The recipe manifest-anchor fix already in #205 is not duplicated.Required human contributor answers
The contributor explicitly confirmed the common scope on 2026-11-19 and the per-row 16,384-token cap with explicit 413 on 2026-11-20; the detailed confirmation is retained in the contributor's task record.
Is a PR needed? Yes. The contributor authorized necessary stage draft PRs and independent supplements preserving other authors' commits.
Bug evidence: The new root regressions fail against the original #175 implementation before the production change; concrete output is recorded below.
Purpose
Use safe budget arithmetic, enforce a final row bound at the renderer shared by ordinary/window paths, and document the native input subset. State clipping, windowing, question order and usage accounting remain intact; valid requests may sum more than 16,384 input tokens across rows. The manifest remains the pinned 0.8B
max_length=16384identity.This is an intentional restriction relative to the pinned Python runtime: the captured long-span request reaches its model-call boundary with a 20,085-token row. Native returns 413. The reference was intercepted before forward; successful inference and HTTP behavior were not established. Existing non-finite-JSON wording and multi-part-span differences remain explicit.
Evidence
Test Plan
Version / commit:
1d460564165310498a1c3d8afddbdc202b4d0dfe, tree060b317326271fe62286e2a6c0887715417dde9c; tested source tree matched on local and L20 checkouts.From repository root:
Ordinary tests use a tiny local tokenizer at the real processor seam and need no checkpoint, download or GPU. Test bodies stay under root tests; src contains only cfg/path registration.
Test Result
Demo / evidence
Model/runtime/tokenizer:
vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490. Existing source shard SHA256c70237ecd579c60289c177fa9806e71c24840c2fc557b5c76af1395dc3081fa6; unchanged flat-bundle manifest anchora41d87ddfc561551f24632e03e1b3fa7617004b05785f024f4b364df131a010f. Rust1.100.0, Python3.12.13, torch2.11.0+cpu, transformers5.19.0, Python tokenizers0.23.2, numpy2.5.3 and safetensors0.8.0.The comparator still records the reference's 20,085-token row and exact native 413. Its cap exception is limited to that approved out-of-domain observation; all in-domain mismatches fail. The five frozen fixtures, field/order/usage assertions and numerical gates are unchanged. Example usage remains1041/0.
Full raw stdout, baseline failures, source/binary/fixture hashes and scripts are retained off-repo in
vela175-cap-evidence.tar.gz, SHA256caa8e30577e605bcb102155ad6cd6414d6ef6ad32a9957f20ae5eac2bcfaad5e. Earlier blocked-stage evidence is preserved separately. No generated logs/archive are added to product Git.Unrun: model forward, native HTTP, GPU, #176 head numerics and later executor stages. No accuracy/performance claim; a video or GPU campaign is not needed for this input-contract slice.
Merge Danger
Door: two-way.
Blast Radius: contract.
Native Vela's admitted row domain is now explicit. The original author branch is untouched; later slices must incorporate this supplement before relying on the accepted input prerequisite.
Self-review
Agent precheck covered the complete five-file diff, ownership, root-test discovery, red/green evidence, exact real-package replay, documentation and artifact hygiene. Keep draft; contributor/maintainer review remains separate.