Repository navigation
fix(ENG-2966): stackone-defender - fall back to Tier 1 and warn when the classifier can't load - #52
Conversation
…the classifier can't load
When the Tier 2 model or ONNX runtime failed to load, @stackone/defender allowed
every result and only warned on the console, which the daemon's ignored stdio
dropped. With Tier 1 off in the shipped config the plugin scanned nothing while
its log read the same as a healthy install.
- The daemon checks isTier2Ready() after warmup. If the classifier is not ready
it rebuilds the defense with Tier 1 on and every field scanned (Tier 1 only
reads named "risky" fields, and plain tool output is wrapped as { output }).
- tier2Ready is logged, written to the state file, shown by --status, and sent
with each scan result.
- The hook tells Claude once per session that Defender is degraded and how to
check, alongside any HIGH RISK note.
- README: replace the nonexistent "MLP head" fallback with the real behaviour.
- Pin transformers, fasttext.wasm and onnxruntime-node exactly to the versions
the lockfile already resolves.
Verification: node --test 18/18; end-to-end hook runs with a missing model
(notice once, injection flagged, benign silent) and with the real model (no
notice, Tier 2 flags the injection).
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
|
Running ultrareview automatically — This changes Defender's security-critical fail-open path into a Tier 1 fallback across the daemon, hook, and state, so a subtle bug could silently miss prompt injections or misjudge degraded daemons — worth a slower multi-pass review.. I'll post findings when complete. |
![aikido-pr-checks[bot]](https://avatars.githubusercontent.com/in/898896?s=60&v=4){kind=small}
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
Ultrareview ran automatically — This changes Defender's security-critical fail-open path into a Tier 1 fallback across the daemon, hook, and state, so a subtle bug could silently miss prompt injections or misjudge degraded daemons — worth a slower multi-pass review.
All reported issues were addressed across 9 files
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved fallback coverage and degraded-state handling issues need correction.
3 open findings
What changed in this PR
Adds a Tier 1 fallback so Defender retains pattern-based protection when its Tier 2 classifier cannot load.
Changes:
- Reports degraded readiness through daemon state, scan results, and session notices.
- Adds fallback tests and updates troubleshooting documentation.
- Pins classifier dependencies to exact versions.
| File | Description |
|---|---|
plugins/security/stackone-defender/tests/degraded-tier2.test.mjs |
Tests fallback scanning and notice claims. |
plugins/security/stackone-defender/scripts/scan-tool-result.mjs |
Emits degraded notices alongside risk warnings. |
plugins/security/stackone-defender/scripts/degraded-notice.mjs |
Tracks session notification claims. |
plugins/security/stackone-defender/scripts/defender-daemon.mjs |
Integrates fallback and exposes readiness. |
plugins/security/stackone-defender/scripts/daemon-paths.mjs |
Defines the notice marker path. |
plugins/security/stackone-defender/scripts/build-defense.mjs |
Builds the Tier 1 fallback. |
plugins/security/stackone-defender/README.md |
Corrects fallback and troubleshooting guidance. |
plugins/security/stackone-defender/package.json |
Pins classifier dependencies. |
plugins/security/stackone-defender/package-lock.json |
Aligns dependency declarations with exact pins. |
Files not reviewed (1)
- plugins/security/stackone-defender/package-lock.json: Generated file
🧠 Review effort: Balanced
Give feedback about Copilot approvals in this survey to enter a drawing for a $150 gift card.
…ad failures - One marker file per session, created with wx, so concurrent sessions and parallel hooks each get the notice once (the single "last session" marker re-notified on every call when two sessions alternated). No session id, no notice. - The notice says pattern-only mode can flag files that only discuss injection. - A degraded daemon restarts after 10 minutes so a transient load failure (e.g. mid-install) retries Tier 2 instead of lasting until idle. - Relabel the startup fatal: it now covers building either defense. Verification: node --test 22/22; end-to-end with a missing model and two alternating sessions, notice once per session. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ment the marker - Capture the library's warmup console warnings into the daemon log, so the notice and README point at a log that holds the actual load error. - Fallback field pattern /^/ instead of /./, so an empty key is scanned too. - README file table lists the per-session degraded marker; hook header comment covers the degraded notice; drop an unused test variable. Verification: node --test 23/23. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
All reported issues were addressed across 7 files (changes from recent commits).
Reply with feedback, questions, or to request a fix.
View guided diff | Turn on auto-fix | Re-trigger cubic
…rkers - Log captured warmup warnings in finally, so they survive a warmupTier2 throw, and render Error arguments by message rather than [object Object]. - Prune degraded-notice markers older than a week on each new claim, so a machine that stays degraded does not gain a file per conversation forever. Verification: node --test 25/25. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
0 issues found across 3 files (changes from recent commits).
Auto-approved: Fixes the plugin's silent fail-open when the Tier 2 classifier can't load by falling back to Tier 1 over every field, surfacing the degraded state once per session, pinning deps, with tests and README updates showing the corrected behavior is bounded and clearly beneficial.
View guided diff | Turn on auto-fix | Re-trigger cubic
Summary
When the Tier 2 model or ONNX runtime failed to load, the plugin scanned nothing and reported healthy (found in the SOL-423 audit, tracked as ENG-2966):
@stackone/defenderallows every result in that case and only warns on the console, which the daemon'sstdio: "ignore"dropped.enableTier1: falseleft no fallback, and the hook only writes output for blocked results.This PR:
isTier2Ready()after warmup; if not ready it rebuilds the defense with Tier 1 on and every field scanned. (Tier 1 only reads named "risky" fields likedescription/body, and plain tool output is wrapped as{ output }, so enabling Tier 1 alone would still have scanned nothing.)tier2Readyis logged, written to the state file, shown by--status, and sent with each scan result. The hook tells Claude once per session that Defender is degraded and how to check, alongside any HIGH RISK note. A daemon from before this change omitstier2Readyand is treated as ready.@stackone/defender; it now describes the real behaviour.@huggingface/transformers3.8.1,fasttext.wasm1.0.1,onnxruntime-node1.27.0), matching what the lockfile already resolves. A newer transformers mangles the tokenizer and makes Tier 2 skip silently, the same failure mode.Decisions on the ticket's questions: degraded state is visible (1); Tier 1 is enabled only when Tier 2 is unavailable (2);
requireTier2is not used, since it makes every tool call wait out the 6 s spawn deadline and then fail open (3); README updated (4).Test plan
node --test: 18/18, including the newtests/degraded-tier2.test.mjs(missing model blocks an overt injection via the fallback, benign prose allowed, notice once per session)HOME: first scan returns the degraded notice plus the HIGH RISK note, benign output is silent, the second injection gets only the HIGH RISK note;--statusshowstier2Ready: falseNot covered here:
stackone-defender-antigravitylikely has the same issue.🤖 Generated with Claude Code
Summary by cubic
Fixes the
@stackone/defenderplugin so a failed Tier 2 model load no longer silently scans nothing and reports healthy. The daemon now falls back to Tier 1 pattern scanning over every field and surfaces the degraded state instead of dropping the load error.Bug Fixes
tier2Ready, writes the latter to the state file, shows it in--status, sends it with each scan result, and tells Claude once per session when degraded, warning that pattern-only mode can flag files that merely discuss injection.@huggingface/transformers,fasttext.wasm, andonnxruntime-nodeexactly to the versions the lockfile resolves, so a future compatible-range upgrade can't silently disable Tier 2.Written for commit a516c00. Summary will update on new commits.