Skip to content

fix(vela): enforce the 16384-token native row limit - #208

Draft
psv666 wants to merge 1 commit into
codex/vela175-processing-basefrom
codex/vela175-row-cap-20261120
Draft

psv666 wants to merge 1 commit into
codex/vela175-processing-basefrom
codex/vela175-row-cap-20261120

Conversation

@psv666

@psv666 psv666 commented Nov 20, 2026

Copy link
Copy Markdown
Collaborator

Summary

Reject an entire native Vela 0.8B request when any completed ordinary or span-window row exceeds 16,384 tokens. The previous renderer could admit a 16,385-token row or 16,443-token windows, and a sufficiently long span header panicked in debug during unsigned budget subtraction.

render ordinary/window row
  remaining budget = max(64, max_length.saturating_sub(overhead))
  completed n_tokens > 16384 → whole-request 413 / too_long

This is a stacked supplement to #175. Base codex/vela175-processing-base is pinned to MrDongsls's cdff2999b9e082e246f933235b608d7a7e5978b2; head 1d460564165310498a1c3d8afddbdc202b4d0dfe preserves that history. Five files, +287/-4, including 249 changed Rust lines. Retarget/integrate after the author stack lands and recheck. The recipe manifest-anchor fix already in #205 is not duplicated.

Required human contributor answers

The contributor explicitly confirmed the common scope on 2026-11-19 and the per-row 16,384-token cap with explicit 413 on 2026-11-20; the detailed confirmation is retained in the contributor's task record.

  1. Outcome: Bring assigned model work to reproducible, reviewable acceptance so supported models can be used reliably.
  2. Highest-leverage variable: Reliable model coverage and numerical consistency.
  3. How this moves it: Reuse existing implementations and resolve contract/evidence gaps with reproducible validation and meaningful regressions.
  4. User and need: Model, application and agent developers needing stable interfaces, reliable outputs and reproducible deployment.

Is a PR needed? Yes. The contributor authorized necessary stage draft PRs and independent supplements preserving other authors' commits.

Bug evidence: The new root regressions fail against the original #175 implementation before the production change; concrete output is recorded below.

Purpose

Use safe budget arithmetic, enforce a final row bound at the renderer shared by ordinary/window paths, and document the native input subset. State clipping, windowing, question order and usage accounting remain intact; valid requests may sum more than 16,384 input tokens across rows. The manifest remains the pinned 0.8B max_length=16384 identity.

This is an intentional restriction relative to the pinned Python runtime: the captured long-span request reaches its model-call boundary with a 20,085-token row. Native returns 413. The reference was intercepted before forward; successful inference and HTTP behavior were not established. Existing non-finite-JSON wording and multi-part-span differences remain explicit.

Evidence

  • Before: debug root regressions: 5 passed / 4 failed; release: 6 passed / 3 failed.
    over-limit request was admitted: ordinary=[16385], windows=[]
    over-limit request was admitted: ordinary=[35, 16385], windows=[]
    over-limit request was admitted: ordinary=[16383], windows=[16443, 16443]
    debug: attempt to subtract with overflow (render.rs:674)
    
  • After: the same nine regressions pass in debug and release, including cap−1/cap/cap+1, long span headers, non-span kinds, mixed/later rows, window-only rejection, aggregate usage over the cap and original state clipping.

Test Plan

Version / commit: 1d460564165310498a1c3d8afddbdc202b4d0dfe, tree 060b317326271fe62286e2a6c0887715417dde9c; tested source tree matched on local and L20 checkouts.

From repository root:

cargo test --locked -p omni-vela-native --lib row_limit_tests -- --nocapture
cargo test --release --locked -p omni-vela-native --lib row_limit_tests -- --nocapture
cargo test --locked -p omni-vela-native -- --list

Ordinary tests use a tiny local tokenizer at the real processor seam and need no checkpoint, download or GPU. Test bodies stay under root tests; src contains only cfg/path registration.

Test Result

Personally executed on L20 Result
New root regressions, debug and release 9/9 each
Ordinary Vela Rust 46 passed, 2 opt-in ignored
Ordinary Vela Python 138 passed
fmt, workspace clippy, release build PASS
Workspace tests 450 passed, 103 ignored
Strict MkDocs PASS
Five freshly recorded real-package token plans Byte-equal to committed fixtures
Release real-bundle processing tests 2 passed; all five cases and structured-state IDs
16 actual reference-entry admission probes 13 equal, 2 exactly characterized existing differences, 1 approved out-of-domain cap difference; no unexpected mismatch

Demo / evidence

Model/runtime/tokenizer: vllm-sr/Vela-2.0-0.8B@326b01d81f61fff7fd4ae570e54b87a49119e490. Existing source shard SHA256 c70237ecd579c60289c177fa9806e71c24840c2fc557b5c76af1395dc3081fa6; unchanged flat-bundle manifest anchor a41d87ddfc561551f24632e03e1b3fa7617004b05785f024f4b364df131a010f. Rust1.100.0, Python3.12.13, torch2.11.0+cpu, transformers5.19.0, Python tokenizers0.23.2, numpy2.5.3 and safetensors0.8.0.

The comparator still records the reference's 20,085-token row and exact native 413. Its cap exception is limited to that approved out-of-domain observation; all in-domain mismatches fail. The five frozen fixtures, field/order/usage assertions and numerical gates are unchanged. Example usage remains1041/0.

Full raw stdout, baseline failures, source/binary/fixture hashes and scripts are retained off-repo in vela175-cap-evidence.tar.gz, SHA256 caa8e30577e605bcb102155ad6cd6414d6ef6ad32a9957f20ae5eac2bcfaad5e. Earlier blocked-stage evidence is preserved separately. No generated logs/archive are added to product Git.

Unrun: model forward, native HTTP, GPU, #176 head numerics and later executor stages. No accuracy/performance claim; a video or GPU campaign is not needed for this input-contract slice.

Merge Danger

Door: two-way.

Blast Radius: contract.

Native Vela's admitted row domain is now explicit. The original author branch is untouched; later slices must incorporate this supplement before relying on the accepted input prerequisite.

Self-review

Agent precheck covered the complete five-file diff, ownership, root-test discovery, red/green evidence, exact real-package replay, documentation and artifact hygiene. Keep draft; contributor/maintainer review remains separate.

  • I have reviewed the full diff and addressed the issues I found.
  • I have checked that the change follows the project's architecture and stays focused on its purpose.
  • I have run the appropriate checks and reported results and unverified checks above.
  • I have checked that the description and claims match implementation and evidence.

@hsliuustc0106 hsliuustc0106 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Draft review — PR #208 "fix(vela): enforce the 16384-token native row limit"

Verdict: COMMENTED (first-pass review of a DRAFT; draft state must not change and no approval, merge or completed validation is implied). Nothing was posted; this is a local draft body.

Review state and provenance

  • Draft state: OPEN, isDraft: true (verified twice via gh pr view, start and end of review). Head unchanged at 1d460564165310498a1c3d8afddbdc202b4d0dfe. Head repo InferMatrixLab/system1-omni, branch codex/vela175-row-cap-20261120; base codex/vela175-processing-base @ cdff2999b9e082e246f933235b608d7a7e5978b2 (verified in git: MrDongsls's #175 head, single parent b49f7c69). Author psv666 (first-time contributor on my record).
  • Template: body contains all four post-#138 answers, the necessity decision, bug-fix failure evidence (before: debug 5 passed/4 failed, release 6/3, with admitted 16,385-token rows, 16,443-token windows and "attempt to subtract with overflow (render.rs:674)"), ## Summary, ## Purpose, ## Test Plan with the "Version / commit:" field, ## Test Result table, ## Demo / evidence (pinned model vllm-sr/Vela-2.0-0.8B@326b01d8..., tool versions, off-repo evidence archive with SHA-256), ## Merge Danger, ## Self-review checklist present and unchecked (left untouched).
  • Gate status: intent answers attributed to contributor confirmations 2026-11-19/20 retained in an off-repo task record — not independently verifiable from the repository; recorded as author-reported. Draft → COMMENTED regardless.
  • Skill load record: same as sibling reviews — local unpublished skill /Users/hsliu/thinkflowlab/system1-omni/.agents/skills/system1-omni-review/SKILL.md SHA-256 58be3bc60806c4c5af7038a0c6a80fadb94f3360d714556a3e965c8903dde108; references/repository-contracts.md SHA-256 d6ae9650e4e18e8986926a790339412f52750fd1eeb19d358e5d91cd547f1202.

Base/head and merge-base

  • Merge-base with mirror origin/main (5fa5179d5): 5fa5179d5 (mirror head). Against canonical main (614b0abc, fork): merge-base is 285c7910 — canonical main is 3 commits ahead of this PR's effective base point (#196, #204, #214). The base branch predates #196/#204, so retargeting onto current main is required after the parent stack lands.
  • gh pr view 175 → OPEN, unmerged.

Diff (focused: cdff2999..1d460564)

5 files, +287/−4 — matches PR metadata (--numstat: recipe/vela/native.md +24/−2; src/models/vela/README.md +15/−1; src/models/vela/native/src/processing.rs +4/−0; src/models/vela/native/src/render.rs +7/−1; tests/vela/native_row_limits.rs +237/−0). One commit, authored by psv666. Total changed text ≈ 291 lines; the body's "249 changed Rust lines" is consistent with render+processing+tests numstat. Far below large-change thresholds.

  • src/models/vela/native/src/render.rs: (a) line 674: 64.max(self.max_length - budget) → 64.max(self.max_length.saturating_sub(budget)); (b) after n_tokens is computed for a completed row (lines 771-776 at head): if n_tokens > self.max_length { return Err(too_long(self.max_length)); }.
  • src/models/vela/native/src/processing.rs (+4): #[cfg(test)] #[path = "../../../../../tests/vela/native_row_limits.rs"] mod row_limit_tests; — test bodies stay in the repository-level tests/ tree per CONTRIBUTING; src/ holds only registration.
  • tests/vela/native_row_limits.rs (+237): 9 weight-free regressions.
  • recipe/vela/native.md and src/models/vela/README.md: document the per-row 16,384 cap, its deliberate narrowing of the pinned Python input domain (20,085-token reference row at the model-call boundary), that summed usage.input_tokens is not capped, and the retained 422 multi-part-span / 400 non-finite-JSON wording differences; README's stale "26 tests" count replaced.

Review questions

  • Where is the limit enforced: in the renderer, RenderContext::render (pub fn render, render.rs:561; gate at render.rs:773), which is called from Processor::plan (processing.rs:96) at processing.rs:158 for the ordinary row and processing.rs:174 for each span-window row. That is the processing/planning stage — the last step before any batching/device execution (at this base the crate is a library plus a parity example; serving runs through the reference Python worker). It is not an admission-time check (contract.rs::admit_request is unchanged) and not a loader check — the body's "enforce at the renderer shared by ordinary/window paths" matches the code.
  • Fail-closed: yes. context.render(..)? propagates the RequestError out of plan(), so a single over-limit ordinary or window row rejects the whole request with the existing too_long error — status 413, body {"detail":[{"loc":["body","questions"],"msg":"the questions alone do not fit the 16384-token window","type":"too_long"}]} (render.rs:385-392). The saturating_sub change additionally removes the debug overflow panic and, in release, the wrap-around that previously produced a huge assemble budget and silently admitted oversized rows; the final gate now bounds both modes.
  • Test pinned: yes — 9 weight-free regressions through the real Processor with a tiny WordLevel/WhitespaceSplit tokenizer (no checkpoint, no GPU): cap−1 and cap admitted; cap+1 rejected; 20,000-word span header → 413 without panic; non-span kinds (noul/choice/score/set) → 413; mixed rows whose sum exceeds the cap but each row ≤ cap, with question order pinned; an oversized later row rejecting the whole request; window rows each ≤ cap with total > cap; raw-state clipping unchanged; an oversized window rejected while the primary row fits. Rejections assert both status 413 and the exact body JSON.
  • Documented contract: consistent. bundle.rs:106 pins the manifest identity max_length == 16384 and bundle.rs:122-123 defaults to 16,384; the gate uses exactly this value. The docs updated in this PR state the cap is per completed row (parts + question blocks), that the request's summed usage may exceed it, and explicitly frame it as a restriction relative to the pinned Python runtime — with the 20,085-token reference observation disclosed as not establishing successful inference. The author-reported comparator treats that single out-of-domain observation as an approved exception; all in-domain mismatches still fail (author-reported).

Findings

No actionable findings (P0–P3) in the focused diff.

Informational notes:

  • N1: The 64-token floor in 64.max(max_length.saturating_sub(budget)) is intentionally retained; the new post-assembly gate is the backstop for the oversized span header/repeated target it can produce. The in-code comment and the docs describe this accurately.
  • N2: The tests construct Processor { .. } by struct literal rather than Processor::load, coupling them to the struct's field set (they compile inside the crate via cfg(test), so this is visible breakage on field changes, not silent drift).
  • N3: The cap is per row, not per request — requests with many valid rows/windows can still sum far beyond 16,384 input tokens. This is documented and tested (mixed_rows_can_exceed_the_cap_in_total_and_keep_question_order, window_rows_can_exceed_the_cap_in_total), not a defect, but reviewers/consumers should not mistake it for a request-level budget.

Evidence separation

  • Source inspected: full focused diff including the 237-line test file; render.rs at head (render entry, budget assembly, gate, too_long); processing.rs at head (plan, render call sites, RenderContext construction); bundle.rs manifest pin; contract.rs (RequestError statuses); both docs files at head.
  • Commands personally run: ref/SHA and ancestry checks (git merge-base --is-ancestor used for cross-PR work below); worktree /tmp/pr-review-pass194/wt208 @ 1d460564; PYTHONPATH=src /Users/hsliu/miniconda3/bin/python -m pytest tests/vela -q → 138 passed (9.3 s); PYTHONPATH=. ... pytest tests/benchmarks -q → 133 passed (28.2 s); mkdocs build --strict (mkdocs 1.6.1) → exit 0 (docs touched, so required). No cargo on this host — the Rust regressions were not runnable here.
  • Observed CI: none. gh pr checks 208 → "no checks reported on the branch". Actions API head_sha=1d4605641... → total_count: 0 in both ThinkFlowLab/system1-omni and InferMatrixLab/system1-omni.
  • Author-reported (not verified here): all cargo debug/release regression runs (9/9 each), ordinary Vela Rust (46 passed, 2 ignored), fmt/clippy/release build, workspace tests (450/103), strict MkDocs, the five byte-equal token plans, real-bundle processing tests, 16 admission probes against the reference, and the off-repo evidence archive contents/hash.

Material unverified checks

  • The Rust regression suite and production gate behavior as executed by cargo (no Rust toolchain locally). The Python-side suites and docs build were verified here, but they do not execute render.rs.
  • Model forward / native HTTP / GPU behavior — explicitly not claimed by the PR; none run.
  • Comparator state (approved out-of-domain exception) — author-reported only.

Cross-PR and integration

  • #210 is stacked directly on this PR: 1d460564 is a verified ancestor of #210's base 4792f8c7. Overlap is docs-only (recipe/vela/native.md, src/models/vela/README.md — both normalize the same trailing "Checks:" sentence; no conflict in stacked order). No Rust file overlap with #210 (render.rs/processing.rs vs heads.rs/executor tests) or with #195's executor code.
  • #195 is OPEN/unmerged (verified via gh pr view 195; mergedAt: null) — contrary to any premise that #195 is merged; canonical main has no native Vela crate at all. The whole chain #175 → #208 → (#195) → #210 must land in order, with retargeting onto current main (3 commits ahead of this PR's base point) and fresh checks at each hop.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants