Skip to content

Ask: optional command suggestions from the local model, Copy/Insert only (#432) - #440

Merged
raiseCatError merged 3 commits into
devfrom
feature/v022-ask-command-suggestions
Oct 21, 2026
Merged

raiseCatError merged 3 commits into
devfrom
feature/v022-ask-command-suggestions

Conversation

@raiseCatError

@raiseCatError raiseCatError commented Oct 18, 2026 •

Copy link
Copy Markdown
Owner

Draft, do not merge before v0.21 ships. #430 has carried dev into release/v0.21.0; a fix v0.21's physical QA needs would reach the release the same way, so anything merged into dev before the release would ride along. Part of v0.22 (#431); approved as decision D2. First slice: the local model only. Remote BYOK providers are a separate PR after a security and credential-handling review.

What it does

With Settings → Ask → Command suggestions → Local model (Off by default), a request Ask's own resolver cannot place may get one suggested shell command from the person's local model (the one set up in /llm):

› /btw convert every png here to webp
Ask  A suggested command, for you to check before running it yourself.
     for f in *.png; do cwebp "$f" -o "${f%.png}.webp"; done
     NMSh's check: changes things. Your local model (Qwen3 0.6B) says: “Each PNG to WebP.” Check it before you run it.
     Copy command · Insert into the composer, unsent

Boundaries (each tested)

  • Explicit requests only, through /btw//ask. Composer text is never interpreted as a request. (A composer Ask mode can follow after the keybinding and input-ownership review.)
  • The deterministic resolver stays first and the default. Only unclear or unsupported outcomes reach the model; a refusal (unsafe) stands and is never routed around; a closed Ask never calls the model.
  • Nothing runs. The suggestion is a literal Ask command block: Copy or Insert only, never Run. Insert parks Ask and puts the text in the composer unsent; only the person's Enter runs it.
  • NMSh labels the risk itself from the paste-review classifier: destructive, privileged, or piped into an interpreter → destructive; installs → installs software; writes, network, project scripts or unknown → changes things; reading only → reads only. A risk the model claims is an extra field and rejects its answer.
  • Data minimization: the model receives the request text (control characters removed, at most 300 characters), the shell and the OS family. No files, environment, history, transcript or output.
  • Strict validation: exactly command and explanation; one plain line of at most 1 KiB; no control, escape or invisible format characters (bidi overrides included); explanation at most 300 characters; or command: null when the model declines.
  • Clear states: a declined request keeps Ask's own answer. Local understanding Off, no model chosen, an older model service that lacks the task, a timeout, or an invalid answer each say why (with Ask's suggestions still offered).
  • Older services: the shared model service advertises command in its welcome; the client never sends the task to one that does not.
  • Lightweight: nothing loads or starts unless suggestions are on and Ask cannot place a request; it reuses the one shared model service (no new process, dependency or network access).

Verification

  • tests/askCommandSuggestions.test.ts: validation (valid, declined, and 15 rejected shapes including a second line, escape sequences, a bidi override, oversize, an extra risk field); prompt minimization; NMSh's labels including a model that calls rm -rf ~/Downloads "read-only"; the setting's default and normalization; the service path with a fake runtime (command prompt and schema); an older service is never sent the task.
  • tests/askCommandSuggestionsApp.test.ts through the real TerminalApp with a stub model: off never calls; on calls only for what Ask cannot place, with exactly {text, shell, os}; refusals stand; the block is Copy/Insert only and labelled; Insert puts it in the composer and nothing is submitted or written; declined and unavailable states; a closed Ask never calls.
  • Ask, Settings, understanding, Setup Cat and column-gutter suites: 188/188 (the Ask coverage audit included). Full canonical npm run verify: running; recorded here when done.

Review follow-up (103b1e9c)

A security review found two problems, both fixed at the cause with regressions written first (they failed before the fix):

  • echo $(rm -rf ~) was labelled "reads only". "Reads only" now needs every part to be plainly read-only: no substitution ($(…), backticks, <(…)), no wrapper that runs another command (env, xargs, nice…), no find action (-exec, -delete…), and nothing the stricter /track gate (repeatRisk) questions, such as git -c or --output=. Otherwise "changes things".
  • The model's explanation could pose as NMSh's check. The note now opens with NMSh's verdict, then quotes the model's words and names the model; an explanation that mentions NMSh is rejected.

Ask suites after the fix: 117/117.

The Setup coverage audit also found the new row missing from Setup; fixed in c1a30017 (Setup → Ask & local understanding → Command suggestions, the same row, Off until chosen; tested). Setup suites and the suggestion tests: 45/45.

Not in this PR

Remote providers (Anthropic, OpenAI-compatible) with environment-variable credentials; a composer Ask mode; refinement turns. Design and decisions: #432 and docs/design/v022-natural-language-commands.md (#435).

Physical QA (not done)

With a local model set up in /llm and Settings → Ask → Command suggestions → Local model: /btw convert every png here to webp shows a suggestion with NMSh's label; Insert puts it in the composer, nothing runs; Enter runs it. /btw delete all my files stays a refusal. With suggestions Off, Ask behaves as before. With local understanding Off, the suggestion attempt says how to turn it on.

Refs #432

#432 (decision D2), first slice: the local model only. When the person turns
command suggestions on, a request Ask's own resolver (and the intent model)
cannot place may get one suggested shell command.

- Settings → Ask → Command suggestions (Off by default, or Local model):
  askCommandSuggestions; remote providers are not part of this slice.
- understanding/tasks: a `command` task. The prompt carries the request text,
  the shell and the OS family only (never files, environment, history, the
  transcript or output). The answer is validated strictly: exactly command
  and explanation, one plain line of at most 1 KiB, no control, escape or
  invisible format characters, an explanation of at most 300 characters, or
  command: null when the model declines. A claimed risk is an extra field
  and rejects the answer.
- The shared model service runs it with its own schema and token budget and
  advertises 'command'; the client sends it only to a service that does
  (an older one is told apart by its welcome and the person is told why).
- ask/commandSuggestions: NMSh labels the risk itself from the paste-review
  classifier (destructive, privileged or piped into an interpreter is
  destructive; installs; anything that writes, reaches the network, runs
  project scripts or is unknown changes things; only reading reads). The
  block is literal, so Ask offers Copy and Insert only, never Run; Insert
  places it in the composer unsent.
- TerminalApp.resolveAsk: only unclear or unsupported outcomes reach the
  model, never a refusal and never a closed Ask. A declined request keeps
  Ask's own answer; local understanding Off, no model, an older service or
  an invalid answer each say why.

Tests: validation (including adversarial output), prompt minimization,
NMSh's labels overriding the model's wording, the service path with a fake
runtime, an older service never sent the task, and the whole Ask path
through TerminalApp with a stub model (off, on, refusal, declined,
unavailable, closed Ask, Insert never runs).

Refs #432
…h's verdict first

A security review found that echo $(rm -rf ~) was labelled 'reads only'.
The read verdict now also requires no substitution, no wrapper command,
no find action and nothing the /track repeat gate questions. The note
states NMSh's check first and quotes the model's words, attributed; an
explanation that names NMSh is rejected.
The Setup coverage audit requires every Settings row in a Setup section.
The same row, Off until chosen, with a note on what each choice does.
raiseCatError added a commit that referenced this pull request Oct 18, 2026
…QA passes

The ledger lists every slice and PR with its verification and open
risks; the physical QA checklist gains the held-input states and passes
for #437, #438/#441, #439 and #440; the NL design records the decisions
and the first slice's scope.
@raiseCatError
raiseCatError merged commit 5657871 into dev Oct 21, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant