Per-rule actions for every guard and content filtering of answer text

[fylorn]

What this changes

Every rule of every guard now has its own action, and the content filter can check the text of answers as well as requests.

• Content filter: custom rules gain scope (request and/or response), case_sensitive, and a replace action with a per-rule replacement (default ***). Built-in rules can be set to replace and stay request-only. Rules that check answers filter the text the client sees in streamed answers (Anthropic, Chat, Responses including custom_tool_call_input, Gemini SSE and JSON array, Bedrock), whole answers and the Responses WebSocket. Answers cannot be blocked and are not matched by code points; both are config errors.
• Outbound redaction: each rule is placeholder (default) or record. Record-only rules are found and reported, but the value is sent as it is and never enters the ledger.
• Control API 48: RuleAction gains placeholder and replace; rule views carry scope, replacement, the real case_sensitive of content matchers, and redaction actions; CustomRuleSave gains scope and replacement; TestSecurity gains scope (test as request or answer) and replacement; new PUT /security/{guard}/rules/enabled (SetRulesEnabled) switches several built-in and custom rules with one config write. ContentMatched gains direction, SecretItem gains replaced, OutcomeDetail::Replaced gains optional segments/replacement, SecurityCounts gains content_replaced. New message codes are in the manifest; security.content_action_unknown became security.content_action_bad (its sentence changed) and security.no_action_of_its_own is gone.

Why

Users need to pick, per rule, between changing the text and only recording a match, and need a way to keep specific words out of what the model says back, not only out of what is sent.

How it was verified

• tw-guard: the answer engine (content::answer) is tested for contains/regex, Unicode case variants, matches split at every position and across SSE chunks cut inside a UTF-8 character, several overlapping rules, strip vs replace vs record, observe mode (no hold-back, still reported), flush at lane end, stream end and on errors, and the regex cap. A randomized test (deterministic seed, 400 rounds) streams random text with random rules and random cuts and asserts the output equals whole-text application and equals a direct find_iter reference; it was also run with other seeds at 30,000 rounds. Per-dialect frame tests cover Anthropic, Chat, Responses (*.done, content_part.done, output_item.done and response.completed repeat exactly what was sent, deltas-free lanes counted once), Gemini SSE/JSON array and Bedrock; whole answers in all five formats. Request-side replace, replacement text not re-matched, case-sensitive matching, record-only redaction (found, reported, not replaced, not numbered), scope validation errors, and TestSecurity with scope.
• tw-gateway: end-to-end tests stream an answer in 7-byte chunks through passthrough, Chat and Gemini (JSON array) conversions and whole answers, check the client never receives a matched word, that the hit is reported with direction: response before the request ends, and that the stored client answer is the filtered one; observe and off; one rule on both sides; record-only redaction; the Responses WebSocket per answer.
• tw-control: bulk enable endpoint (one write, factory values not written, unknown ids/names refused with nothing written), content rules with scope/replacement written and tried as an answer, redaction record action.
• tw-store: answer hits logged with direction response, the sent model, outcome replaced and the new counter; record-only secrets logged as recorded.
• cargo test --workspace: 3528 passed, 0 failed, twice in a row (after the seven review follow-ups below). cargo clippy --workspace --all-targets -- -D warnings and cargo fmt --all -- --check are clean. Message-code manifest and configuration manual regenerated.
• Leaks: the answer filter only removes or replaces text; excerpts and locations of answer hits are masked with the request's redaction rules and ledger before they leave the gateway, as tool-call hits already are.

Notes for review

• No version bump; no release notes yet.
• Hold-back is a fixed window per lane (longest keyword − 1; for a regex its maximum length in characters, one more when it looks at the next character, capped at 256). Observe mode does not hold anything back. A regex match longer than the cap may have its start sent; this is documented in the manual.
• Overlapping matches of different rules merge into one edit; the first replace rule's text wins, otherwise the region is deleted. Requests and answers share this logic.
• Enterprise (consumes tw-guard) needs these adaptations: add action to CustomRedactRule literals and actions where RedactPolicy is built field by field; add scope, case_sensitive, replacement to CustomContentRule literals (and case_sensitive, sides, replacement to content::RuleInput, or use ..RuleInput::default()); handle content::Outcome::Replaced and ContentAction::Replace in exhaustive matches; new PolicyError variants (EmptyScope, BlockInAnswer, CodepointsInAnswer) and TrialError variants (BlockInAnswer, CodepointsInAnswer); RuleAction gained two variants. flow::look/flow::replace already skip record-only rules; callers that replace from scan_text hits themselves should pass them through RuleSet::to_replace, and audit events should use RuleSet::records to tell recorded from replaced values. Answer filtering is available as content::answer::{Answer, Body, FrameFilter, whole} when enterprise wires it in.

Review follow-up (second commit)

An adversarial review and fuzzing pass found no panics; these gaps are fixed with regression tests:

• Redaction overlap: a record-only hit could win the overlap resolution and hide a placeholder rule's hit, sending the value upstream in clear (e.g. a record-only custom OPENAI_API_KEY=\S+ over the built-in key, or built-in email set to record over a custom placeholder rule). Placeholder hits are now resolved first (see the second follow-up for how overlaps merge). Record-only hits hidden behind a placeholder hit are still reported (flow::reported, used by find, look_hits and the WebSocket path). A randomized test asserts that adding record-only rules never changes what is replaced.
• Request-side case-insensitive contains now uses the same case-insensitive regex as answers. The previous lowercase-and-map-back fast path cut the wrong range when a character's lowercase had a different byte length (Ⱥ, Kelvin sign).
• Gemini: text parts separated by a non-text part (function call, code, image; thought parts do not separate) are separate lanes, so held text no longer moves past a call and matches do not span it. A whole answer whose text did not change is returned byte-identical. The property tests now cover mixed parts.
• More text channels: Anthropic content_block_start text, and Chat / Responses refusals (deltas, refusal.done, content parts, whole answers). Logprobs repeat the text token by token: while rules can delete or replace, they are removed from streamed answers, and from whole answers whose text changed (documented in the manual).
• Broken input: an SSE frame or whole answer with invalid UTF-8 is read lossily and rewritten when it changes instead of passing raw. An unparseable Gemini JSON-array element is dropped while rules can change text (passed through in observe mode).
• WebSocket: a late frame of an answer that already ended (response.failed after error) now goes through the filter. When the upstream goes away mid-answer, held text is delivered and hits are reported on their turn.
• Memory: a lane keeps per-rule counts and at most 50 located matches per rule (the security log's limit) instead of every match. Lane::take() returns Matches { found, counts }; Lane::keeping_all() keeps every position for the admin "test" endpoint only.

API notes for consumers of tw-guard: Lane::take_found() became Lane::take() (Matches), LaneKey gained Refusal, and FrameFilter::edits(), flow::reported and RuleSet::recording_only are new. FrameFilter::{new, frame, drain, take_hits}, answer::whole, Body and AnswerHit are unchanged.

Second review follow-up (third commit)

• WebSocket, late frame of a finished answer: it was filtered with the next answer's filter, and its response.failed closed that answer's lanes, so the next answer's held text was released early and a stripped word could be assembled across frames. Late frames now use their own answer's filter (kept with its response id) or a standalone one, and never touch the answer in progress. Regression test: answer 1 errors, answer 2 starts, answer 1's late response.failed arrives mid-word, answer 2 finishes.
• Overlapping placeholder hits: they now merge into one placeholder over their union. Before, the losing hit's extra tail went upstream in clear (aaa<<TW_SECRET_1>>ccc).
• Storage masking with record-only rules: a record-only hit that overlaps a placeholder hit keeps its pieces outside the placeholder range. Stored bodies and logs mask the whole record-only match (e.g. a record-only postgres://… around the built-in connection-string password), while what goes upstream is still decided by placeholder hits. Reporting counts the record-only match once.
• Error frames: any frame with a top-level error (Chat, Gemini, others) ends every lane, so held text is sent before the error frame. A per-format test checks the ordering.
• The redaction property test now also asserts that every byte any single rule matches is covered by the merged hits (masked when stored), and every byte a placeholder rule matches is replaced.

No API changes in this commit: scan/flow::hits still return sorted, disjoint hits. They may now contain record-only fragments next to a placeholder hit, so consumers that mask from them (enterprise pii::redact_blob, Redaction::mask) cover the union without code changes.

Third review follow-up (fourth commit)

• Merge order: redaction hits that do not count (on one of our existing placeholders, in base64 payloads, custom hits in protocol ids) are now dropped before overlapping hits merge. Before, a value to be replaced could be merged with such a hit and dropped with it, e.g. alice@corp.example.org<<TW_SECRET_1>> with a custom example\.org\S* rule, or a built-in key in a tool-call id overlapping a custom hit.
• JSON structure: custom redaction regexes run one JSON string at a time, so \S+ and similar patterns can no longer start on {/:/,, swallow quotes, or make the request body invalid JSON.
• Request-side strip/replace repeat until nothing more changes, rescanning only segments changed in the last round. A segment still changing after 64 rounds (deliberately nested input) is emptied, so no match is ever sent.
• Stored copies of restored text: new public helper tw_guard::redact::replace::hide_values(text: &str, ledger: &Ledger, form: Form) -> Cow<'_, str> puts the placeholder back wherever a ledger value appears literally (raw, JSON-escaped once or twice), never inside an existing placeholder. Stored bodies and security-log excerpts use it, so a value that needs context to be recognised (a connection-string password, a custom OPENAI_API_KEY=… value) echoed alone in an answer is not stored in clear.
• Error frames: only a top-level error object with a message or code ends the answer lanes. null, false, "" and {} no longer do, so a word split around such a frame is still caught.
• WebSocket: hits from a late frame of an answer older than the current one are not reported on the turn in progress. That text repeats what the older answer already said, and its hits were reported then. The text itself is still filtered.
• The redaction property test now also covers greedy patterns, existing placeholders and tool-call ids, and checks that the sent body stays valid JSON and restores exactly.

Enterprise-facing API: hide_values (above) is new. scan/flow::hits keep their signatures and still return sorted, disjoint hits.

Fourth review follow-up (fifth commit)

• Nested strips in answers: the answer filter made a single pass, so stripping sesecretcret re-formed secret and sent it. Answers now get the request side's guarantee. Text a pass deleted or replaced goes to a further pass that tries every start position whose match could read across a seam (up to the rule's hold before it), merges and edits, until nothing new turns up. Replacement text is never matched again. Record-only rules keep seeing the model's original text only. After 64 further passes the rest of the lane is dropped, as the request side empties the segment.
• Whole answers (Lane::whole, whole bodies, the trial endpoint) are processed in one go and are always clean. A whole body whose lane spans several fields is filtered as one text and split back like the streamed fields.
• Streams: the last pass holds back as many characters again as the longest edit rule needs (reach; 0 when no edit can leave a seam), so a re-formed match one level beyond what has been seen never reaches text already sent. Such text gives the same result as the whole answer at every split. A match nested deeper than that may have its start sent already: the unsent rest is deleted, so the client never receives a whole match. The one exception is a look-around regex (\b, $) whose match lies entirely in text already sent. The manual describes the extra hold and this limit.
• Tests: nested strips in streamed and whole answers split at every position (including via another rule and via \b), nesting past the bound, and a new nesting fuzz. Both property tests now compare against a reference that rescans every position until a fixed point, and assert that no edit-rule match outside replacement text remains. Deeper-than-held streams are counted and must stay rare. Stress-run at 4 seeds × 100k cases.

Fifth review follow-up (sixth commit)

• Security detail cost (request side): later-round positions map back through one merged table, where deletions at the same spot collapse into one entry, instead of round by round. Recorded places are deduplicated with a set. places() on a 64-deep nested 128 KiB request and on aabb×100k is linear again; a new content_scale test guards it with a size-ratio check.
• Value masking for storage (hide_values, now built on value_spans):
  • Values are found in one Aho-Corasick pass. aho-corasick 1.x was already in the lock file via regex; it is now a direct workspace dependency, and no new crates were added.
  • In JSON form, values are only replaced inside string contents, decoded at every escape depth (now up to six; see below) (\n, \", \uXXXX incl. surrogate pairs, \/). Numbers, literals and structure are never touched, so stored bodies stay valid JSON.
  • Text that is not one JSON document (SSE, plain errors) is also searched outside strings, skipping whole numbers and literals.
  • Overlapping values merge into one region. Placeholder overlap uses binary search.
  • Redaction::piece takes value spans on the original text and lets rule hits win.
  • A scale test covers 10k vs 40k values with as many placeholders.
• JSON vs decoded text: JSON-ness is decided by walking the JSON structure, not by the first character. Decoded text starting with ", { or [ is scanned whole again. Answer context in the security log is masked as decoded text (Form::Text).
• Whole-answer re-scan: each seam re-check searches only the window a match can span. Whole bodies filter a multi-field lane once (Lane::whole_split) and split the result back by original position.
• Iterate-until-stable edges:
  • Regexes anchored to text or line boundaries (^, $, \A, \z) are checked once in answers.
  • Deletions at the very start or end of a lane are not seams. As a result, (?s).\z, \d$ and .\b no longer erode a whole answer, and whole and streamed agree.
  • Past the round bound, only the part still changing is dropped; the head is kept.
• Error frames: a bare error frame ends the lanes. That covers a string error, or an error object without message/code, on a frame with no choices/candidates/type depending on the dialect. Normal frames with nested or false error fields still do not.
• Late WebSocket frames: the filters of the last eight answers are kept. Hits in text the answer never streamed are reported on that answer's own turn, and repeats are not counted again. Frames of unknown or older answers are still filtered but not reported.
• Regression tests for each item. The reviewer fuzzers pass, apart from the documented deeper-than-held streaming cases.

Sixth review follow-up (seventh commit)

• Detail cost: the ledger's values are found once per text (ValueSpans, new in tw-guard). Every excerpt takes its slice by binary search, instead of each piece searching the whole body again. A new scale test runs the reported-items path with values and body growing together.
• Values split across excerpt parts:
  • An answer match that falls inside a ledger value widens to the whole value, and excerpt edges never cut a value. No fragment shows in before/matched/after.
  • A value cut by a piece edge, or partly covered by a rule hit, is masked in full.
• Nested values:
  • The search keeps, at each position, only the longest value ending there, and unions the intervals. It walks the Aho-Corasick automaton and memoizes the longest pattern per state.
  • Runs matched by \d{6,} (k=600) now take 6 ms with no memory blow-up. An adversarial scale test guards this, and a randomized test checks against brute force.
• Non-JSON text: text that is not strictly valid JSON is masked everywhere, numbers included. Valid JSON stays strings-only, decoded up to six escape levels; each further level of decoding runs only while the text still contains backslashes. is_json is now a strict parse, so [555-1234] is no longer JSON.
• Replacements: only deletions leave seams. A non-empty replacement next to a word boundary is no longer re-matched and eaten further (\d{4}\b, \w\b), and whole and streamed agree.
• Leading deletions: several deletions at the very start of a lane no longer count as a seam for the hold-back, so a stream no longer emits early.
• Late WebSocket frames: a lifecycle frame carrying another answer's id is filtered by that answer's filter, or a standalone one, even after the turn window has forgotten it. It never closes the current answer's lanes. (Refined in the next section: the decision is now made by connection state, not id inequality alone.)
• Regression tests for each item. The reviewer reproducers pass, apart from the documented deeper-than-held streaming cases.

Seventh review follow-up (eighth commit)

• Which answer a WebSocket frame belongs to is decided by connection state, not by id inequality alone (relays may rewrite ids):
  • A closing frame that arrives while an answer is in progress belongs to it, unless its id belongs to an ended answer. The last 1024 ended ids are kept per connection.
  • Unknown ids follow what the upstream has shown. If it has closed an answer with a rewritten id, unknown ids count as the current answer's; if its ids have always matched, unknown ids are strays.
  • Before any closing frame has been seen, an unknown response.completed counts as the current answer's, and response.failed/incomplete do not.
  • Frames judged to be another answer's touch nothing of the current one: not the turn bookkeeping (upstream, finish_front), not the reply hooks, not the answer filter.
  • A new test covers rewritten ids, rewritten ids with done frames, and a stray mid-answer failure.
• Lean decoding: escape levels are decoded one at a time. Decoding stops at the first level that decodes nothing, and position maps are composed into one u32 table. 4 MiB of \q now peaks at about 28 MB RSS.
• Placeholders: a value region that overlaps an existing placeholder is masked outside it rather than skipped whole.
• Outbound JSON: outbound redaction treats a body as JSON when it parses with raw control characters allowed in strings, so numbers outside strings are never replaced. Storage masking keeps its strict rule; numeric masking in stored SSE stays as accepted.
• Scale tests: these now compare interleaved medians over an eightfold size step, with generous ratios.
• The full suite ran twice in a row with no failures.

🤖 Generated with Claude Code