v0.22 design, ledger and natural-language proposal

docs/design/v022-daily-use.md

@@ -0,0 +1,138 @@
+# v0.22 design: daily-use reliability and workflow
+
+Planning and acceptance record for the v0.22 candidate ([#431](https://github.com/raiseCatError/notMyShell/issues/431)). Durable progress lives in the
+[v0.22 ledger](../development/v022-ledger.md). Automated results are not physical-terminal validation.
+
+## Goal
+
+Make NMSh better for everyday terminal work without turning it into a shell, a terminal emulator or an IDE. The
+release starts from reliability: no typed key may be lost, duplicated, misrouted, shown when it should be hidden, or
+run unexpectedly. On that foundation it adds a small number of workflow improvements that reuse what NMSh already has
+(`/view`, `/edit`, `/open`, the editor bridge, Ask, the queue). Everything else stays where the issue tracker puts it.
+
+## Baseline and the v0.21 release boundary
+
+- Baseline: `dev` at `a8b383c2` (v0.21 integrated: #411–#424, then #425, #427 and #428).
+- `release/v0.21.0` is now `86bb3a6c`: PR #430 merged `dev` (`a8b383c2`) into it on 2026-10-18. Draft PR #429 is the
+  release itself; its physical QA has not run.
+- **Rule:** v0.22 PRs target `dev` but stay **draft and unmerged until v0.21 has shipped**. A fix that v0.21's physical
+  QA needs lands on `dev` and reaches the release branch by another `dev` → `release/v0.21.0` merge, as #430 did, so
+  anything v0.22 merged into `dev` before the release would ride along. v0.22 never touches `release/v0.21.0`, its QA
+  evidence, `master` or the publishing workflow.
+- After v0.21 ships (release PR merged into `master`, tag published), reconcile: bring `master` back into `dev` if the
+  runbook does, rebase the open v0.22 branches, and move v0.22 changelog entries into the new `[Unreleased]` section
+  (they are written into `dev`'s `[Unreleased]` today, beside v0.21's).
+
+## Scope
+
+### Required reliability fixes (adopted)
+
+| Slice | Issue | Summary | State |
+| --- | --- | --- | --- |
+| R1 | #426 | Keys typed before NMSh knows who reads them wait for the look instead of an 800 ms deadline; a runnable first sample is not "busy" | Implemented, draft PR |
+| R2 | #356 | Fish key loss after a handed-over read: reproduce under load on the current harness; fix only what reproduces | Investigation |
+
+### High-value user-facing improvements (adopted, from accepted issues)
+
+| Slice | Issue | Summary | State |
+| --- | --- | --- | --- |
+| U1 | #366 | `/view` → open the file at the line on screen in the person's editor through the existing `/open` bridge; refusal messages offer the same | Draft PR #434 |
+
+### Optional, decided by the maintainer (2026-10-18)
+
+| Item | Issue | Decision and state |
+| --- | --- | --- |
+| D1 | #389 | A recalled history entry browses as one entry (Up/Down step entries until it is edited). Reproduced on `dev`; changes a behavior the maintainer triaged as acceptable and a test codifies. **Approved.** Draft PR #438; the `History n/N` indicator is a separate slice on an existing composer edge, draft PR #441 (based on #438) |
+| D2 | #432 | Natural-language command suggestions from the composer ([design](v022-natural-language-commands.md)). **Approved as optional:** Off by default, local model first (draft PR #440); remote bring-your-own-key providers later, in a separate PR after a security review |
+| D3 | #366 | Choose a terminal editor for "Open with" when `$VISUAL`/`$EDITOR` is unset (detected installed editors). **Approved.** Draft PR #439 |
+
+### Explicitly deferred (status unchanged)
+
+#330–#332 (workspace sidebar/panel program, paused), #337 and #338 (worktree and GitHub write workflows, paused), #339
+and #340 (managed-agent adapters, Remote Control research), #333–#335 (tmux Studio, Setup Cat profiles, keymap
+remapping), #341 (executable mods), #398 (Learn, deferred by decision), #399 (host config writes), #304 (Theme Bridge
+expansion), #296 (hosted demo), #15. The ongoing indices #14, #20 and #305 stay ongoing. Nothing here is reactivated by
+v0.22 starting; none is a dependency of the adopted slices.
+
+## R1 — #426: input ownership under load
+
+**Behavior.** While a command runs and NMSh has no evidence yet that it reads input, the first key of a draft and an
+Enter that would queue are held while the session looks (`input-look`). The hold now ends only when the look answers
+or evidence arrives (bounded at 5 s), never at a fixed 800 ms. Until then nothing typed is drawn, sent or queued. A
+look still running after 800 ms says so in the composer: `Checking whether <program> is waiting for input · your typing
+is kept`. Ctrl+C is never held. In the session, a first sample that sees a process on a CPU with nothing earlier to
+compare is "runnable", not computing; when the shell holds its terminal, the follow-up sample decides.
+
+**Why not the issue's example (route text at 800 ms, hold only the Enter).** It fixes the queueing but still draws a
+hidden reply in the composer whenever the look is slow, for a prompt-less `read -s` and for a prompted `sudo` or `ssh`
+password alike. The unfixed baseline failed the live test's "the hidden answer is never drawn" assertion this way under
+load. Holding until the owner is known keeps the privacy property independent of machine load. The cost is confined to
+a loaded machine: type-ahead appears when the look answers instead of at 800 ms, with the note above.
+
+**Boundaries.** `TerminalApp` hold policy and `InputWatch` sampling only. No protocol change; older services are never
+asked and behave as before. No change to queue semantics, Fish's reader handover or the direct-key path.
+
+**Tests.** Deterministic (mocked timers, a look the test settles): prompt-less line, hidden and prompted (`sudo`)
+replies typed at once are never queued, drawn or run; keys after a held Enter stay behind it; the bound releases a
+look that never answers; the command ending mid-hold runs the line once; Ctrl+C releases; quick looks add no delay. A
+watch test for the runnable first sample (line and key reads) and for a busy shell staying not-waiting. Live: the
+input-ownership, input-awareness, typing-after-read, query-order and queue PTY suites; the ownership file 4× in
+parallel.
+
+**Performance.** No added work per key. A look is the existing `stty` + `ps` probe (one or two samples). Normal-load
+latency is unchanged; the bound is 5 s.
+
+**Physical QA.** Ghostty and Terminal.app, zsh, Bash and Fish: `read -r reply; echo got=$reply` and `read -s` typed
+at once and after a pause; `sudo -k true` typed at once; type-ahead during `sleep 5`; a heavy build in another window
+(load) while repeating the reads: the note appears, nothing is queued or shown.
+
+## R2 — #356: Fish keys after a handed-over read
+
+v0.21 stabilization ran the reproducer 40/40 (macOS, Fish 4.9.3, 8× load) and 22/22 (Linux, Fish 3.6). v0.22 repeats
+it on the v0.22 baseline at higher parallel load and with the #426 changes. The issue is not declared fixed without a
+reproduction and a verified fix; a clean run is recorded on the issue and the status stays with the maintainer.
+
+## U1 — #366: from `/view` to the person's editor
+
+**Behavior.** In `/view` of a file, `o` opens that file at the first line on screen with the configured "Open with"
+target: the surrounding editor (Zed, VS Code) is launched with argv; a terminal editor (`$VISUAL`/`$EDITOR`) gets a
+reviewed command placed in the composer for the person to run, never executed silently. With no editor known, the
+viewer says how to set one. `/view`'s refusals (too large, binary, not a regular file) name `/open <path>` as the way
+on, where it applies. `e` (native `/edit`) is unchanged.
+
+**Boundaries.** Reuses `resolveHostActions`, `editorArguments` and the `/open` dispatch; no new launcher, no editor
+registry, no new PTY ownership. Paths stay single argv elements. Viewing output (not a file) offers nothing new.
+
+**Tests.** Viewer key → action (`o` only for files, with the top visible line); dispatch for Zed/VS Code (spawn argv),
+`$EDITOR` (composer text, nothing executed), none (message); narrow width and `NO_COLOR` footer.
+
+**Physical QA.** Ghostty and Terminal.app with `$EDITOR=nvim` and with Zed/VS Code integrated terminals.
+
+## D1 — #389 multiline history browsing (approved; #438, #441)
+
+Reproduction on `dev` (`a8b383c2`): recall a two-line entry with Up, press Up again: the caret moves to the entry's
+first line; a third Up recalls the older entry (`tests/composerHistory.test.ts` "a recalled multiline command navigates
+its own lines before older history" codifies it). An N-line entry costs N extra presses. Proposed minimal rule: while
+the composer shows an unchanged recalled entry, Up/Down step entries; any edit, paste, caret movement other than Up/Down,
+or selection ends browsing. Draft multiline editing is unchanged. Implementation would sit in `TerminalApp`'s Up/Down
+routing and `ComposerHistory`; `CommandEditor` stays as is. The `History n/N` indicator is a separate step.
+
+## D2 — natural-language command suggestions (approved as optional; #440)
+
+See [the design](v022-natural-language-commands.md). Summary: one optional, off-by-default `CommandSuggester` behind
+Ask, with a local provider (the person's own OpenAI-compatible local runtime) and an optional bring-your-own-key remote
+provider; a request is explicit (`/btw` or a `?? ` prefix), the suggestion is shown for review and inserted into the
+composer, and only the person's Enter runs it. Nothing about files, environment or transcripts is sent unless the
+person attaches it.
+
+As decided, the first slice (#440) is the local model only, reached through `/btw`/Ask (no `?? ` prefix and no
+composer mode yet), with Copy and Insert and no refinement turns. NMSh's own risk label comes first and the model's
+words are quoted and attributed.
+
+## Release gates
+
+Per slice: regressions that fail on the old code, focused suites, `verify:fast`, and the full `npm run verify` before
+the PR leaves draft. At integration: `verify:release` on the exact head, CI on the exact head (Ubuntu and macOS shards,
+Node 22, Fedora subset, timing smoke), and physical QA per slice in Ghostty and Terminal.app. Local runs use
+`NMSH_TEST_TMPDIR=/Volumes/T7Tmp/ai-dev` (Unix socket path limit). No release, tag or deployment without the
+maintainer.

docs/design/v022-natural-language-commands.md

@@ -0,0 +1,138 @@
+# Natural-language command suggestions
+
+**Status: approved as optional (decision D2, 2026-10-18); first slice in draft PR #440.** Part of the
+[v0.22 design](v022-daily-use.md); discussed in [#432](https://github.com/raiseCatError/notMyShell/issues/432). The first
+slice is the local model only. Remote providers are a separate, later PR after a security and credential-handling
+review. The decisions are at the end.
+
+## The need
+
+Some terminals let a person switch fluidly between typing commands and describing what they want ("find files over
+100 MB changed this week"), then go straight back to commands. NMSh already has most of the pieces: Ask (`/btw`,
+`/ask`, and `/btw <request>` straight from the composer), a deterministic resolver for NMSh's own capabilities, an
+optional local model that may only choose among those capabilities, and command classifiers used by paste review and
+`/track`. What it cannot do is propose an arbitrary shell command for an arbitrary terminal task. Ask's model contract
+deliberately forbids that today ("a model cannot introduce a file, session, worktree or command").
+
+## Principles
+
+1. **Nothing runs by itself.** A suggestion is text shown for review. The only way it runs is the person inserting it
+   into the composer and pressing Enter, exactly like a command they typed (visible, highlighted, history rules apply).
+2. **A request is never shell syntax.** It is entered through an NMSh surface (`/btw`, or an explicit Ask mode in the
+   composer with its own marker), so it can never reach the shell by accident, even with the feature off.
+3. **Send the minimum.** By default only the request text, the shell (zsh, Bash, Fish) and the OS family. No files,
+   environment, history, transcript, working directory or hostname. Anything more is attached per request, shown first.
+4. **One Ask.** This is a new outcome inside the existing Ask, after its deterministic resolver, not a second assistant.
+5. **Off by default; useful offline.** With no provider, Ask works as today and says how to enable suggestions.
+6. **NMSh's own safety check wins.** The command is classified by NMSh's existing classifier, whatever the model says.
+
+## Flow
+
+```
+› /btw find files over 100 MB changed this week
+  (or: Ask mode in the composer, marked and hinted "Ask NMSh · Enter asks · Esc returns to commands")
+
+Ask NMSh
+  1. Deterministic resolver (unchanged). An NMSh action or answer wins.
+  2. Unsupported or unclear, and command suggestions are on → CommandSuggester.
+
+  Suggested command · zsh · Claude Haiku 5.5 with your API key
+    find . -type f -size +100M -mtime -7
+    Lists regular files under this directory over 100 MB, modified in the last 7 days.
+    Read-only (NMSh check)
+  Enter Insert into the composer · c Copy · r Refine · Esc Back
+
+› find . -type f -size +100M -mtime -7          ← the composer, back in command mode; Enter runs it
+```
+
+Refine sends one follow-up with the previous suggestion (no longer conversation). A suggestion NMSh classifies as
+destructive, privileged or writing over the network is shown with that label and needs an explicit second key to
+insert; it is never offered as a one-key action.
+
+## Architecture
+
+- `src/ask/commandSuggest.ts`: the `CommandSuggester` interface, the fixed prompt, the result schema and validation.
+
+  ```ts
+  interface SuggestRequest { request: string; shell: 'zsh' | 'bash' | 'fish'; os: 'macos' | 'linux'; attachments?: Attachment[] }
+  interface CommandSuggestion { command: string; explanation: string }
+  type SuggestResult = {kind: 'suggestion'; suggestion: CommandSuggestion; usage?: {input: number; output: number}}
+    | {kind: 'none'; reason: string} | {kind: 'unavailable'; reason: string};
+  interface CommandSuggester { id: string; label: string; suggest(request: SuggestRequest, signal: AbortSignal): Promise<SuggestResult> }
+  ```
+
+- Providers, each optional and lazily loaded:
+  - **Local**: the existing user-global model service (`src/understanding/ModelService.ts`) gains a `command` task type
+    next to intent and folding, so every window shares one runtime (llama.cpp, or an already-running Ollama or LM
+    Studio). The recommended 0.6B model is fit for choosing NMSh actions, not for writing shell commands: suggestions
+    need a larger model the person chose. NMSh never downloads a model for this feature.
+  - **Anthropic** (bring your own key): the Messages API through the official `@anthropic-ai/sdk`, default model
+    `claude-haiku-5-5`, low effort, the result constrained by a JSON schema (`output_config.format`) and validated again
+    locally.
+  - **OpenAI-compatible endpoint** (optional third): a URL plus an optional key, which covers other hosted services
+    without per-vendor code.
+- Ask gains a typed `suggestion` outcome; `AskPanel` renders it with the existing action controls.
+- Settings → Ask → Command suggestions: provider (Off, Local, Anthropic, OpenAI-compatible), model, endpoint, and the
+  **name** of the environment variable that holds the key (NMSh stores the name, never the key).
+- Validation before display: one line unless the person asked for a multi-line script, at most 1 KiB, no control
+  characters or escape sequences (rendered as visible marks if present, then refused), explanation at most 300
+  characters. Anything else is a "could not suggest" result, never a partial command.
+- Classification: the same classifier paste review and `/track` use (`src/tasks/repeatSafety.ts` and the shared command
+  classifier). The model returns no risk field: risk is NMSh's judgment only.
+
+## Security and privacy
+
+- **Credentials.** API keys from a provider's developer platform only, read from an environment variable the person
+  names, at request time, from NMSh's own process (not the managed shell). Never written to configuration,
+  transcripts, journals, logs or errors. NMSh does not read browser cookies, consumer app sessions or chat
+  subscriptions, and does not assume a consumer subscription includes API access.
+- **What leaves the machine.** Request text, shell and OS family. Attachments (the last command and its exit status, or
+  one selected output block, bounded and passed through the existing report redaction) are opt-in per request and shown
+  before sending. The first use of a remote provider shows exactly what will be sent and to which host.
+- **Prompt injection.** The request is the person's own text; an attached output block is untrusted. Because nothing is
+  executed and NMSh classifies the result itself, an injected instruction can at worst produce a suggestion the person
+  sees, with NMSh's label.
+- **Recording.** Ask's existing "Record Ask in transcript" setting applies to the visible request and suggestion. The
+  prompt sent to the provider and raw responses are never stored. Ask text never enters shell history.
+- **Network.** HTTPS to the configured host only; a 15 s timeout; Esc cancels; no background or speculative requests;
+  no telemetry.
+
+## Cost and tokens
+
+A request is a fixed instruction of roughly 300–400 tokens plus the request (20–60 tokens); the answer is 50–150
+tokens. On `claude-haiku-5-5` at $0.10 per million input and $0.50 per million output tokens (current list price for
+prompts up to 100K tokens), that is about $0.0001–0.0003 per suggestion, or $0.10–0.30 per thousand, including some
+thinking at low effort. Prompt caching does not apply (the fixed prefix is below the minimum cacheable size) and is not
+needed. Ask can show a suggestion's token use. Local providers cost nothing beyond the machine's own resources; on an 8
+GB Mac a model large enough to write commands competes with everything else, which is a reason the local path is never
+started implicitly.
+
+## Not in scope
+
+No agent loop, tool use, file reading, multi-step plans or automatic execution; no conversation memory beyond one
+refinement; no new assistant surface; no replacement of the deterministic resolver; no change to how commands run.
+
+## Tests
+
+Fake providers only (no network in tests): schema validation; adversarial outputs (escape sequences, control
+characters, multi-line, oversize, `rm -rf ~`, `curl … | sh`, a command hidden after a newline); NMSh's label overriding
+any model wording; insert puts the exact text in the composer and runs nothing; Esc and timeout cancel; missing key,
+unreachable endpoint and model refusal each produce a clear message and the deterministic result; nothing secret in
+transcripts, journals or settings; narrow width, `NO_COLOR` and Safe glyphs.
+
+## Decisions (2026-10-18)
+
+1. **Remote providers:** allowed as an optional, Off-by-default bring-your-own-key choice, in a separate PR after a
+   security review. Consumer subscriptions (ChatGPT, Claude) are never used as API credentials; no browser cookies or
+   session tokens.
+2. **Order:** the local model first (#440); Anthropic and OpenAI-compatible providers later.
+3. **Entry point:** explicit requests through `/btw` and Ask only. A composer Ask mode can follow after the keybinding
+   and input-ownership review.
+4. **Credentials:** an environment-variable name only, for the remote slice.
+5. **Dependency:** decided with the remote slice.
+
+Behavior in every slice: display only; Insert is deliberate and only Enter runs; NMSh owns the risk label; nothing is
+sent silently (files, transcripts, environment, secrets, output); lightweight on an 8 GB Mac.
+
+**The first slice (#440):** the `suggested` outcome, the local provider through the shared model service, Copy and
+Insert, NMSh's label and the validation rules above. Not yet: refinement turns, a composer mode, remote providers.