diff --git a/CHANGELOG.md b/CHANGELOG.md index 7adfa9c6..3fc3b600 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/). ## [Unreleased] +### Fixed + +- Multi-line progress displays that redraw their own rows (cursor up and erase line, as concurrent downloads and container pulls do) now update in place instead of leaving a copy of every frame in the transcript ([#388](https://github.com/raiseCatError/notMyShell/issues/388)). A program can redraw only its own recent rows, at most a screen height up, never NMSh's lines or earlier commands; `/copy` copies the final rows once. Absolute cursor positioning keeps the previous behavior. +- A queued command whose command word was still being classified when the next prompt reported new shell knowledge no longer keeps the unclassified color ([#408](https://github.com/raiseCatError/notMyShell/issues/408)). + ## [0.20.0] - 2026-10-18 **Terminal workflow, first-run setup, background tasks & native modules.** The first stable release since v0.18.0. A new first run lets you pick a starting point under a live aurora and review it before anything applies; queued commands read as a chronological timeline with saved summaries; completed output can be selected, copied and acted on from the keyboard; optional tools are discovered, reviewed and installed through one flow; long-running commands can run as background tasks beside the shell; and the native module catalog covers twelve more runtimes and build tools. Every new control is reached through an explicit command, key or review, and the persistent shell, the queue and program input keep their behavior. diff --git a/docs/development/v021-ledger.md b/docs/development/v021-ledger.md index 36142b49..9b179e5c 100644 --- a/docs/development/v021-ledger.md +++ b/docs/development/v021-ledger.md @@ -14,8 +14,8 @@ Automated results here are not physical-terminal validation. | # | Slice | Issues | Branch / PR | Merged commit | State | | --- | --- | --- | --- | --- | --- | -| 0 | Design and ledger | #410 | | | in progress | -| 1 | Reliability gates | #403, #408, #388 | | | | +| 0 | Design and ledger | #410 | `docs/v021-design` / #411 | | PR open, CI green; awaiting merge | +| 1 | Reliability gates | #403, #408, #388 | `fix/v021-reliability` (stacked on #411) | | PR open | | 2 | `/view` | #397, #402 | | | | | 3 | `/edit` | #402 | | | | | 4 | Semantic Output | #386 | | | | @@ -33,8 +33,21 @@ Automated results here are not physical-terminal validation. ## Physical QA still required -Recorded per slice when it merges. +Checklist: [v0.21 physical QA](../testing/v021-physical-qa.md). Slice 1: section 1 (multi-row progress in Ghostty +and Terminal.app, queued highlight colors). + +## Verification notes + +- Slice 1 evidence: each new regression fails on the unfixed code and passes on the fix (Fish generator present; + superseded lookup settled `unknown` and the stored line never recolored; 9 of 10 redraw tests fail on the old parser, + the tenth being the documented unchanged absolute-positioning case). #408 tests passed 16/16 in two concurrent loops. +- Local full `npm run verify` on this 8 GB Mac: run with `NMSH_TEST_TMPDIR` at the T7 storage root + (`/Volumes/T7Tmp/ai-dev`), never a deeper directory: the runner requires test roots under 32 characters for Unix + sockets, and a longer root fails the runner self-test, tmux and session-socket tests for environmental reasons. + Live PTY tests can time out under full-suite load here and pass alone; exact-head CI is the independent gate. ## Open risks -None recorded yet. +- Merging into `dev` needs the maintainer (the agent's merge was refused by its permission policy). Slices are + stacked PRs; merge them in order, each squashed. +- #388: programs that position rows absolutely inline (rare) still append; recorded in the design. diff --git a/docs/testing/v021-physical-qa.md b/docs/testing/v021-physical-qa.md new file mode 100644 index 00000000..133614c2 --- /dev/null +++ b/docs/testing/v021-physical-qa.md @@ -0,0 +1,31 @@ +# v0.21 physical QA + +## Status + +- **Candidate, not released.** Items are added as each v0.21 slice lands ([ledger](../development/v021-ledger.md)). + Nothing below has been run yet. +- **What does not count:** headless PTY fixtures, synthetic keys, captures of a window driven that way, and green CI. + Each item needs a person at a real terminal with a real keyboard and pointer. +- **Build under test:** record the exact commit (`node dist/index.js --version`) built from a clean checkout + (`npm ci && npm run build`). +- **Isolation:** a throwaway `HOME` (or `XDG_CONFIG_HOME`) and a temporary working directory. + +## Results + +| Pass | Host and version | OS | Shell and version | Commit | Result | Notes | +|---|---|---|---|---|---|---| +| 1 Multi-row progress (#388) | Ghostty | macOS | zsh | | Not run | | +| 1 Multi-row progress (#388) | Terminal.app | macOS | Bash or Fish | | Not run | | + +## 1. Multi-row progress redraws (#388) + +- [ ] 1.1 `node -e 'for (let i=0;i<=10;i++){process.stdout.write((i?"\x1b[1A\x1b[2K":"")+"Downloading "+i*10+"%\n")}'` + leaves one `Downloading 100%` line, at 80 columns and at 40 columns. +- [ ] 1.2 A two-row display (for example `docker pull` of a multi-layer image, or `brew install` of a formula with + concurrent downloads) updates its rows in place; log lines printed between frames all stay. Record the command. +- [ ] 1.3 Resize the window narrower and wider while 1.2 runs: no earlier command, prompt or NMSh notice is erased. +- [ ] 1.4 `/copy` of that block copies the final rows once, not every frame. +- [ ] 1.5 `printf '10%%\r20%%\r100%%\n'` still shows one `100%` line; `clear` still clears. +- [ ] 1.6 `vim`, `less` and `htop` still take the full screen and return to the composer. +- [ ] 1.7 Queue two quick commands (`ls` then `pwd`) with Ctrl+Q while a short command runs: both stored command + lines end with the known-command color (#408). diff --git a/scripts/test-shell-home.mjs b/scripts/test-shell-home.mjs new file mode 100644 index 00000000..be75a11c --- /dev/null +++ b/scripts/test-shell-home.mjs @@ -0,0 +1,17 @@ +import {mkdirSync} from 'node:fs'; +import {join} from 'node:path'; + +/** + * Marks Fish's man-page completion generation as already done in a test home (#403). + * + * On a home without generated completions, an interactive Fish starts `fish_update_completions` detached + * (Fish 4 checks `$__fish_cache_dir/generated_completions`, Fish 3 `$__fish_user_data_dir/generated_completions`). + * That orphaned process keeps writing about 1,400 files into the home after the shell exits, so removing the home + * races it. Real installations keep Fish's normal behavior; only test homes are marked, in both places. + * + * @param {string} home the HOME the test's shells run with + * @param {{dataHome?: string, cacheHome?: string}} [dirs] XDG_DATA_HOME / XDG_CACHE_HOME when the test sets them + */ +export function markFishCompletionsGenerated(home, {dataHome = join(home, '.local', 'share'), cacheHome = join(home, '.cache')} = {}) { + for (const base of [dataHome, cacheHome]) mkdirSync(join(base, 'fish', 'generated_completions'), {recursive: true}); +} diff --git a/scripts/test.mjs b/scripts/test.mjs index 482eb2b1..36f564a2 100644 --- a/scripts/test.mjs +++ b/scripts/test.mjs @@ -4,6 +4,7 @@ import {availableParallelism, tmpdir} from 'node:os'; import {join} from 'node:path'; import {pathToFileURL} from 'node:url'; import {discoverTestFiles, parseTestOptions, selectTestGroups, testConcurrency} from './test-selection.mjs'; +import {markFishCompletionsGenerated} from './test-shell-home.mjs'; /** Short private temp roots keep Unix sockets below their path length limit. */ export async function runTestFiles(files, args = [], {cwd = process.cwd(), stdio = 'inherit', report = message => console.error(message)} = {}) { @@ -12,11 +13,17 @@ export async function runTestFiles(files, args = [], {cwd = process.cwd(), stdio try { const testHome = join(root, 'home'); mkdirSync(testHome, {mode: 0o700}); + markFishCompletionsGenerated(testHome); // Existing presentation snapshots pin truecolor; baseline fixtures override this explicitly. // Tests never see the user's real NMSh settings: configuration lives under this run's private root. const env = {...process.env, HOME: testHome, COLORTERM: process.env.COLORTERM ?? 'truecolor', TMPDIR: root, TMP: root, TEMP: root, XDG_CONFIG_HOME: join(root, 'config'), NMSH_DISABLE_UPDATES: '1'}; // A nested runner must not impersonate its parent's test worker. delete env.NODE_TEST_CONTEXT; + // Ambient XDG data, cache and state directories (a developer's, or a wrapper's shared cache) are not the tests': + // shells and tools fall back to the private HOME, as they do on CI where these are unset. + delete env.XDG_DATA_HOME; + delete env.XDG_CACHE_HOME; + delete env.XDG_STATE_HOME; // PTY workers also own shell/helper processes. Bound fan-out on larger // development hosts; explicit Node --test-concurrency options still win. const concurrency = testConcurrency(process.env.NMSH_TEST_CONCURRENCY, availableParallelism()); diff --git a/src/app/TerminalApp.ts b/src/app/TerminalApp.ts index e727d766..07f2d810 100644 --- a/src/app/TerminalApp.ts +++ b/src/app/TerminalApp.ts @@ -10352,6 +10352,7 @@ export class TerminalApp { this.lastPtyRows = plan.ptyRows; this.lastPtyColumns = columns; this.session.resize(columns, plan.ptyRows); + this.output.setScreenRows(plan.ptyRows); } const wrapped = this.output.wrapped(columns); diff --git a/src/output/AnsiOutputParser.ts b/src/output/AnsiOutputParser.ts index be7d481e..7e0863dc 100644 --- a/src/output/AnsiOutputParser.ts +++ b/src/output/AnsiOutputParser.ts @@ -54,9 +54,29 @@ export type SerializedLine = SerializedCell[]; */ export const MAX_CONTROL_SEQUENCE = 8192; +/** The screen height assumed until the owner reports the PTY's rows (a terminal's classic default). */ +const DEFAULT_SCREEN_ROWS = 24; + +/** + * The live region (#388). A program may redraw its own recent rows in place, as on a terminal: cursor up/down + * (`CSI A/B/E/F`, `ESC M/D/E`), save/restore (`ESC 7/8`, `CSI s/u`) and erasing (`CSI J` 0/1, `CSI K` 0/1/2) act on + * the row the cursor is on. The cursor never moves above the fence (the line after the newest one NMSh wrote: a + * command header, a notice, an earlier command) nor more than a screen height above the newest line, because a + * terminal cannot address its scrollback either. So a multi-row progress display updates its rows instead of adding + * a copy of them per frame, and nothing outside the running program's own rows can be changed. Absolute row + * positioning (`CSI H/f/d` rows) is not mapped: an inline program cannot know where its rows are, so it keeps the + * previous behavior. The stored lines are what a terminal would show at the end; intermediate frames are not kept. + */ export class AnsiOutputParser { readonly lines: StyledLine[] = []; private current: StyledLine = []; + /** The cursor's row: an index into `lines`, or `lines.length` for the current (newest) line. */ + private row = 0; + /** The lowest row the program may address (see the live region). */ + private fence = 0; + private screenRows = DEFAULT_SCREEN_ROWS; + /** DECSC / SCOSC saved cursor (row, column, and for ESC 7 the style). */ + private saved?: {row: number; column: number; style?: string}; private column = 0; private style = ''; private pending = ''; @@ -86,6 +106,50 @@ export class AnsiOutputParser { constructor(private readonly onClear?: () => void) {} + /** The PTY's rows: how far above the newest line a program's cursor can reach. */ + setScreenRows(rows: number): void { + this.screenRows = Math.max(1, Math.min(1000, Math.floor(rows) || DEFAULT_SCREEN_ROWS)); + } + + /** + * Program output from here on may redraw only rows from `index` (clamped to the stored lines); NMSh-written rows + * and earlier commands stay out of reach. Used when a running command is resumed from a journal. + */ + fenceAt(index: number): void { + this.fence = Math.max(0, Math.min(index, this.lines.length)); + this.row = this.lines.length; + this.saved = undefined; + } + + /** The line the cursor is on. */ + private get target(): StyledLine { + return this.row < this.lines.length ? this.lines[this.row]! : this.current; + } + + /** The highest row the cursor may move to: the fence, or a screen height above the newest line. */ + private get topRow(): number { + return Math.max(this.fence, this.lines.length - (this.screenRows - 1)); + } + + private moveRow(delta: number): void { + this.row = Math.max(this.topRow, Math.min(this.lines.length, this.row + delta)); + } + + /** Line feed: down a row, or a new line at the bottom. */ + private lineFeed(): void { + if (this.row < this.lines.length) { this.row += 1; return; } + this.lines.push(this.current); + this.current = []; + this.row = this.lines.length; + } + + /** NMSh wrote a line: the cursor returns below it and nothing at or above it can be redrawn. */ + private settle(): void { + this.row = this.lines.length; + this.fence = this.lines.length; + this.saved = undefined; + } + write(chunk: string): void { const input = this.pending + chunk; this.pending = ''; @@ -128,8 +192,7 @@ export class AnsiOutputParser { continue; } if (character === '\n') { - this.lines.push(this.current); - this.current = []; + this.lineFeed(); this.column = 0; this.ending = true; index += 1; @@ -149,7 +212,7 @@ export class AnsiOutputParser { const spaces = 8 - (this.column % 8); for (let count = 0; count < spaces; count += 1) { this.put(' ', 1); - const cell = this.current[this.column - 1]!; + const cell = this.target[this.column - 1]!; cell.tab = true; if (count === 0) cell.tabSpan = spaces; } @@ -182,11 +245,14 @@ export class AnsiOutputParser { this.hyperlink = undefined; this.style = style; this.write(text); + // Text with its own line breaks or row movement still ends on the newest line. + this.row = this.lines.length; this.lines.push(this.current); this.current = []; this.column = 0; this.style = ''; this.hyperlink = undefined; + this.settle(); } ensureLineBoundary(): void { @@ -195,6 +261,7 @@ export class AnsiOutputParser { this.current = []; this.column = 0; } + this.settle(); } replaceLine(index: number, text: string): void { @@ -205,6 +272,10 @@ export class AnsiOutputParser { const oldPending = this.pending; const oldHyperlink = this.hyperlink; const oldDiscarding = this.discarding; + const oldRow = this.row; + const oldFence = this.fence; + const oldSaved = this.saved; + const oldLength = this.lines.length; this.current = []; this.column = 0; @@ -212,13 +283,15 @@ export class AnsiOutputParser { this.pending = ''; this.discarding = false; this.hyperlink = undefined; - + // Replacement text is one line of NMSh's own: it is written as the newest line and cannot reach other rows. + this.row = this.lines.length; + this.fence = this.lines.length; + this.saved = undefined; + this.write(text); - if (this.pending.length > 0) { - // Flush any remaining characters if write left pending state (e.g. incomplete escape) - // For simple replacement lines, this shouldn't happen unless malformed, but just in case. - } - this.lines[index] = this.current; + // A line break in replacement text would add lines; only the first one replaces the stored line. + const replaced = this.lines.length > oldLength ? this.lines.splice(oldLength)[0]! : this.current; + this.lines[index] = replaced; this.current = oldCurrent; this.column = oldColumn; @@ -226,6 +299,9 @@ export class AnsiOutputParser { this.pending = oldPending; this.hyperlink = oldHyperlink; this.discarding = oldDiscarding; + this.row = oldRow; + this.fence = oldFence; + this.saved = oldSaved; } completedCount(): number { @@ -284,12 +360,17 @@ export class AnsiOutputParser { const restored: StyledLine = line.map(cell => cell === null ? null : 'empty' in cell ? undefined : {...cell}); this.lines.push(restored); } + // A restored transcript is history; a resumed command reopens its own rows with fenceAt. + this.settle(); } trim(maxLines: number): number { if (this.lines.length <= maxLines) return 0; const removed = this.lines.length - maxLines; this.lines.splice(0, removed); + this.row = Math.max(0, this.row - removed); + this.fence = Math.max(0, this.fence - removed); + if (this.saved) this.saved = {...this.saved, row: Math.max(0, this.saved.row - removed)}; return removed; } @@ -305,7 +386,7 @@ export class AnsiOutputParser { } const params = match[1] ?? ''; const final = match[3] ?? ''; - this.applyCsi(params, final); + this.applyCsi(params, final, match[2] ?? ''); return {complete: true, next: start + match[0].length}; } if (kind === ']') { @@ -338,9 +419,22 @@ export class AnsiOutputParser { if (start + 1 + intermediates >= input.length || intermediates >= MAX_CONTROL_SEQUENCE - 1) return {complete: false, next: start}; return {complete: true, next: start + 1 + intermediates}; } + // Two-byte escapes that move within the live region; others (keypad modes, RIS…) are dropped as before. + if (kind === '7') this.saved = {row: this.row, column: this.column, style: this.style}; + else if (kind === '8') this.restoreCursor(true); + else if (kind === 'M') this.moveRow(-1); + else if (kind === 'D') { this.lineFeed(); this.ending = true; } + else if (kind === 'E') { this.lineFeed(); this.column = 0; this.ending = true; } return {complete: true, next: Math.min(input.length, start + 2)}; } + private restoreCursor(withStyle: boolean): void { + if (!this.saved) return; + this.row = Math.max(this.topRow, Math.min(this.lines.length, this.saved.row)); + this.column = this.saved.column; + if (withStyle && this.saved.style !== undefined) this.style = this.saved.style; + } + /** * SGR in order: 0 resets where it appears, and the arguments of extended * colours (38/48/58 with 5;n or 2;r;g;b) are never read as attributes, so a @@ -368,17 +462,32 @@ export class AnsiOutputParser { if (kept.length > 0) this.style += `\u001B[${kept.join(';')}m`; } - private applyCsi(params: string, final: string): void { + private applyCsi(params: string, final: string, intermediates = ''): void { this.touch(); const values = params.replace(/^\?/u, '').split(';').map(value => Number(value || '0')); const amount = values[0] || 1; + // Row movement, save/restore and partial erases: plain forms only (a private marker or intermediate is another request). + const plain = !intermediates && !/^[<=>?]/u.test(params); if (final === 'm') { // A private marker makes it another request (`CSI > 4 ; 2 m` is xterm's modifyOtherKeys), not SGR. if (/^[<=>?]/u.test(params)) return; this.applySgr(params); } else if (final === 'K') { - if ((values[0] ?? 0) === 2) this.current = []; - else this.current.splice(this.column); + const line = this.target; + const mode = values[0] ?? 0; + if (mode === 2) line.length = 0; + else if (mode === 1) { for (let position = 0; position <= this.column && position < line.length; position += 1) line[position] = undefined; } + else line.splice(this.column); + this.touch(line); + } else if (plain && (final === 'A' || final === 'B' || final === 'E' || final === 'F')) { + this.moveRow(final === 'A' || final === 'F' ? -amount : amount); + if (final === 'E' || final === 'F') this.column = 0; + } else if (plain && params === '' && final === 's') { + this.saved = {row: this.row, column: this.column}; + } else if (plain && params === '' && final === 'u') { + this.restoreCursor(false); + } else if (plain && final === 'J' && (values[0] ?? 0) <= 1) { + this.eraseDisplay(values[0] ?? 0); } else if (final === 'G') { this.column = Math.max(0, amount - 1); } else if (final === 'C') { @@ -390,6 +499,9 @@ export class AnsiOutputParser { this.lines.length = 0; this.current = []; this.column = 0; + this.row = 0; + this.fence = 0; + this.saved = undefined; this.onClear?.(); } } @@ -397,22 +509,45 @@ export class AnsiOutputParser { private put(text: string, width: number): void { this.ending = false; - for (let position = 0; position < width; position += 1) this.current[this.column + position] = undefined; - this.current[this.column] = {text, width, style: this.style, ...(this.hyperlink ? {hyperlink: this.hyperlink, ...(this.authoring ? {authored: true as const} : {})} : {})}; + const line = this.target; + for (let position = 0; position < width; position += 1) line[this.column + position] = undefined; + line[this.column] = {text, width, style: this.style, ...(this.hyperlink ? {hyperlink: this.hyperlink, ...(this.authoring ? {authored: true as const} : {})} : {})}; this.touch(); - for (let position = 1; position < width; position += 1) this.current[this.column + position] = null; + for (let position = 1; position < width; position += 1) line[this.column + position] = null; this.column += width; } - private touch(): void { revisions.set(this.current, lineRevision(this.current) + 1); } + private touch(line: StyledLine = this.target): void { revisions.set(line, lineRevision(line) + 1); } private findPreviousCell(): StyledCell | undefined { - for (let index = Math.min(this.column - 1, this.current.length - 1); index >= 0; index -= 1) { - const cell = this.current[index]; + const line = this.target; + for (let index = Math.min(this.column - 1, line.length - 1); index >= 0; index -= 1) { + const cell = line[index]; if (cell) return cell; } return undefined; } + + /** Erase in display within the live region: 0 below the cursor, 1 above it. */ + private eraseDisplay(mode: number): void { + const line = this.target; + if (mode === 0) { + line.splice(this.column); + if (this.row < this.lines.length) { + // The cursor's row becomes the newest line; the program's rows below it are gone, as on the screen. + this.current = line; + this.lines.length = this.row; + } else this.touch(line); + this.touch(this.current); + return; + } + for (let row = this.topRow; row < this.row; row += 1) { + this.lines[row]!.length = 0; + this.touch(this.lines[row]!); + } + for (let position = 0; position <= this.column && position < line.length; position += 1) line[position] = undefined; + this.touch(line); + } } /** diff --git a/src/output/OutputBuffer.ts b/src/output/OutputBuffer.ts index 1f3cff68..5dff1188 100644 --- a/src/output/OutputBuffer.ts +++ b/src/output/OutputBuffer.ts @@ -352,9 +352,18 @@ export class OutputBuffer { const placed = startId >= after && outputStartId <= this.parser.completedCount() && (header || (startId === 0 && outputStartId === 0)); if (!placed) this.retireClearedBlocks(); else if (!header) this.active.cleared = true; + // The resumed command may go on redrawing its own rows (not NMSh's lines written into its block meanwhile). + let ownRows = placed ? outputStartId : 0; + for (let line = ownRows; line < this.parser.completedCount(); line += 1) if (this.lineTypes.has(line)) ownRows = line + 1; + this.parser.fenceAt(ownRows); return {cleared: Boolean(this.active.cleared)}; } + /** The PTY's rows, which bound how far a program's output can redraw rows above its newest line. */ + setScreenRows(rows: number): void { + this.parser.setScreenRows(rows); + } + updateCommandHighlight(startId: number, formattedLines: string[]): void { for (let i = 0; i < formattedLines.length; i++) { this.parser.replaceLine(startId + i, formattedLines[i] ?? ''); diff --git a/src/shell/SemanticService.ts b/src/shell/SemanticService.ts index e094f215..6ffb9efa 100644 --- a/src/shell/SemanticService.ts +++ b/src/shell/SemanticService.ts @@ -46,6 +46,9 @@ export function parseCompletionFacts(line: string): CompletionFacts | undefined return {zshCompletions: zc === '1', fzfTab: ft === '1', completionSystem: cs === '1'}; } +/** How many times a lookup superseded by newer shell knowledge is asked again before it settles as unknown. */ +const REASKS = 3; + export class SemanticService { private child: ReturnType; private pending = new Map void>(); @@ -65,8 +68,10 @@ export class SemanticService { this.shellNamesComplete = text.split('\n').includes('complete'); this.cache.clear(); for (const [name, type] of this.shellNames) this.cache.set(name, type); - for (const resolve of this.pending.values()) resolve('unknown'); + // Lookups in flight were asked against the old knowledge; each one asks again against this one (see lookup). + const superseded = [...this.pending.values()]; this.pending.clear(); + for (const resolve of superseded) resolve('unknown'); for (const resolve of this.rawPending.values()) resolve(''); this.rawPending.clear(); } @@ -207,6 +212,15 @@ done\n`); private isDead = false; async classifyCommand(cmd: string): Promise { + return this.lookup(cmd, 0); + } + + /** + * A lookup that newer shell knowledge superseded while it was in flight never caches its answer (it may be stale), + * and is asked again against the new knowledge, at most REASKS times. Without that, a stored command classified + * just as the next prompt arrived (a fast queue) would keep the unclassified color for good (#408). + */ + private async lookup(cmd: string, reasked: number): Promise { if (!cmd || cmd.trim().length === 0) return 'unknown'; // Only classify the first word if it has spaces cmd = cmd.split(' ')[0]; @@ -227,7 +241,7 @@ done\n`); }, 1500); this.pending.set(id, (res) => { clearTimeout(timer); - if (generation !== this.generation) { resolve('unknown'); return; } + if (generation !== this.generation) { resolve(reasked < REASKS ? this.lookup(cmd, reasked + 1) : 'unknown'); return; } if (this.cache.size >= 8192) this.cache.delete(this.cache.keys().next().value!); this.cache.set(cmd, res); resolve(res); diff --git a/tests/ansiLiveRegion.test.ts b/tests/ansiLiveRegion.test.ts new file mode 100644 index 00000000..08281714 --- /dev/null +++ b/tests/ansiLiveRegion.test.ts @@ -0,0 +1,141 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import {once} from 'node:events'; +import {mkdtempSync, realpathSync, rmSync} from 'node:fs'; +import {tmpdir} from 'node:os'; +import {join} from 'node:path'; +import {AnsiOutputParser, plainLine} from '../src/output/AnsiOutputParser.js'; +import {OutputBuffer} from '../src/output/OutputBuffer.js'; +import {ShellSession} from '../src/shell/ShellSession.js'; + +/** + * #388: programs that redraw several of their own rows in place (multi-line progress) update those rows instead of + * leaving a copy per frame, and can never reach rows outside their live region. + */ +const plain = (parser: AnsiOutputParser) => parser.allLines().map(line => plainLine(line)); +const ESC = '\u001b'; +const up = (rows: number) => `${ESC}[${rows}A`; +const clearLine = `${ESC}[2K`; + +test('the #388 fixture: a one-row redraw with cursor-up and erase-line leaves one line, not one per frame', () => { + const parser = new AnsiOutputParser(); + for (let step = 0; step <= 10; step += 1) parser.write(`${step ? `${up(1)}${clearLine}` : ''}Downloading ${step * 10}%\r\n`); + assert.deepEqual(plain(parser), ['Downloading 100%']); + // Carriage-return rewriting, the existing control, still works. + parser.write('10%\r20%\r100%\r\n'); + assert.deepEqual(plain(parser), ['Downloading 100%', '100%']); +}); + +test('a two-row progress display interleaved with log lines keeps every log line and only the final progress rows', () => { + const parser = new AnsiOutputParser(); + const frame = (a: number, b: number) => `${clearLine}layer a ${a}%\r\n${clearLine}layer b ${b}%\r\n`; + parser.write(frame(0, 0)); + for (let step = 1; step <= 5; step += 1) { + if (step === 3) parser.write(`${up(2)}${ESC}[Jpulled base image\r\n`); + else parser.write(up(2)); + parser.write(frame(step * 20, step * 10)); + } + assert.deepEqual(plain(parser), ['pulled base image', 'layer a 100%', 'layer b 50%']); +}); + +test('the cursor never leaves the live region: NMSh lines, earlier commands and rows beyond a screen height stay untouched', () => { + const parser = new AnsiOutputParser(); + parser.addLine('$ previous'); + parser.write('previous output\r\n'); + parser.addLine('$ current'); + parser.write('own 1\r\nown 2\r\n'); + parser.write(`${up(50)}X`); + assert.deepEqual(plain(parser), ['$ previous', 'previous output', '$ current', 'Xwn 1', 'own 2'], 'clamped at the first own row'); + + const tall = new AnsiOutputParser(); + tall.setScreenRows(3); + for (let line = 0; line < 10; line += 1) tall.write(`line ${line}\r\n`); + tall.write(`${up(9)}X\r\n`); + assert.deepEqual(plain(tall).slice(6), ['line 6', 'line 7', 'Xine 8', 'line 9'], 'no further up than a screen height'); +}); + +test('an NMSh line written mid-redraw fences the rows above it: later frames continue below it', () => { + const parser = new AnsiOutputParser(); + parser.write('progress 1\r\n'); + parser.addLine('· copied'); + for (let step = 2; step <= 4; step += 1) parser.write(`${up(1)}${clearLine}progress ${step}\r\n`); + assert.deepEqual(plain(parser), ['progress 1', '· copied', 'progress 4']); +}); + +test('save/restore, next/previous line, reverse index and partial erases act within the region', () => { + const parser = new AnsiOutputParser(); + parser.write(`${ESC}7first\r\nsecond\r\n${ESC}8${ESC}[1mF${ESC}[m\r\n`); + assert.deepEqual(plain(parser), ['First', 'second']); + parser.write(`${ESC}[2Fx${ESC}[1Ey\r\n`); + assert.deepEqual(plain(parser), ['xirst', 'yecond']); + parser.write(`${ESC}Mz`); + assert.deepEqual(plain(parser), ['xirst', 'zecond']); + parser.write(`${ESC}[s${up(1)}${ESC}[3G${ESC}[1K${ESC}[u!`); + assert.deepEqual(plain(parser), [' st', 'z!cond']); + parser.write(`${up(1)}\r${ESC}[J`); + assert.deepEqual(plain(parser), [], 'erase below removes the program\'s rows under the cursor'); +}); + +test('sequences split across reads still redraw in place', () => { + const parser = new AnsiOutputParser(); + parser.write('one\r\n'); + for (const chunk of [`${ESC}`, '[', '1', 'A', `${ESC}[2`, 'Ktwo\r', '\n']) parser.write(chunk); + assert.deepEqual(plain(parser), ['two']); +}); + +test('absolute row positioning is not mapped: it appends as before instead of guessing which row is meant', () => { + const parser = new AnsiOutputParser(); + parser.write('a\r\nb\r\n'); + parser.write(`${ESC}[1;1Hc\r\n`); + assert.deepEqual(plain(parser), ['a', 'b', 'c']); +}); + +test('a resumed command keeps redrawing its own rows, and a command-line recolor never moves its cursor', () => { + const buffer = new OutputBuffer(); + const startId = buffer.beginCommand('pull', ['$ pull']); + buffer.write('layer 1%\r\n'); + buffer.updateCommandHighlight(startId, ['$ pull']); + buffer.write(`${up(1)}${clearLine}layer 50%\r\n`); + const transcript = buffer.transcript(); + const restored = new OutputBuffer(); + restored.restoreTranscript(transcript); + restored.resumeActive('pull', startId, startId + 1); + restored.write(`${up(1)}${clearLine}layer 100%\r\n`); + const record = restored.complete(0)!; + assert.equal(record.output, 'layer 100%'); +}); + +test('ten thousand frames of a three-row display stay three rows and parse in bounded time', () => { + const buffer = new OutputBuffer(); + buffer.beginCommand('build', ['$ build']); + const started = performance.now(); + for (let step = 0; step < 10_000; step += 1) { + buffer.write(`${step ? up(3) : ''}${clearLine}a ${step}\r\n${clearLine}b ${step}\r\n${clearLine}c ${step}\r\n`); + } + const elapsed = performance.now() - started; + const record = buffer.complete(0)!; + assert.equal(record.output, 'a 9999\nb 9999\nc 9999'); + assert.equal(record.endId - record.outputStartId, 3); + assert.ok(elapsed < 5000, `took ${Math.round(elapsed)}ms`); +}); + +test('a real PTY: the #388 fixture run by zsh leaves one progress line', {timeout: 60_000}, async () => { + const home = realpathSync(mkdtempSync(join(tmpdir(), 'nmsh-redraw-'))); + const shell = new ShellSession(home, 100, 30, home, {...process.env, HOME: home}); + let output = ''; + try { + await once(shell, 'prompt'); + shell.on('data', data => { output += data; }); + const done = once(shell, 'prompt'); + const script = 'for (let i=0;i<=10;i++){process.stdout.write((i?"\\x1b[1A\\x1b[2K":"")+"Downloading "+i*10+"%\\n")}'; + shell.submit(`${JSON.stringify(process.execPath)} -e '${script}'`); + await done; + const parser = new AnsiOutputParser(); + parser.setScreenRows(30); + parser.write(output); + assert.deepEqual(plain(parser).filter(line => line.includes('Downloading')), ['Downloading 100%']); + } finally { + shell.kill(); + rmSync(home, {recursive: true, force: true, maxRetries: 5, retryDelay: 100}); + } +}); diff --git a/tests/helpers/liveFrontend.ts b/tests/helpers/liveFrontend.ts index e5061ead..b620a957 100644 --- a/tests/helpers/liveFrontend.ts +++ b/tests/helpers/liveFrontend.ts @@ -8,6 +8,7 @@ import nodePty, {type IPty} from 'node-pty'; import {listLiveSessions} from '../../src/session/connectSession.js'; import {parseSlashCommand} from '../../src/commands/slashCommands.js'; import {TranscriptStore} from '../../src/sessions/TranscriptStore.js'; +import {markFishCompletionsGenerated} from '../../scripts/test-shell-home.mjs'; export const TSX = import.meta.resolve('tsx'); export const ENTRY = fileURLToPath(new URL('../../src/index.ts', import.meta.url)); @@ -37,6 +38,8 @@ export class LiveSandbox { constructor(config: Record = {}, private readonly extraEnv: Record = {}) { mkdirSync(this.home); + // Fish's detached man-page generator would otherwise outlive the sandbox's shells and race disposal (#403). + markFishCompletionsGenerated(this.home); mkdirSync(this.temp); mkdirSync(join(this.config, 'nmsh'), {recursive: true}); mkdirSync(this.runtime, {mode: 0o700}); diff --git a/tests/postV020Folding.test.ts b/tests/postV020Folding.test.ts index ff61d3be..ed9abb14 100644 --- a/tests/postV020Folding.test.ts +++ b/tests/postV020Folding.test.ts @@ -15,12 +15,14 @@ import {beginSelection} from '../src/output/TranscriptSelection.js'; import {executionFrameFixture} from './support/executionTimelineFrames.js'; import {DEFAULT_PROMPT_CONFIGURATION, loadPromptConfiguration} from '../src/prompt/configuration.js'; import {SETTINGS_ROWS} from '../src/ui/SettingsPanel.js'; +import {markFishCompletionsGenerated} from '../scripts/test-shell-home.mjs'; const sequence = (n: number) => Array.from({length: n}, (_, i) => `${i + 1}`).join('\n') + '\n'; async function realApp(shell: ShellId) { const root = realpathSync(mkdtempSync(join(tmpdir(), `nmsh-fold-${shell}-`))); const cwd = join(root, 'project'); mkdirSync(cwd); + markFishCompletionsGenerated(root, {dataHome: join(root, 'data')}); const keys = ['HOME', 'XDG_CONFIG_HOME', 'XDG_DATA_HOME', 'XDG_STATE_HOME', 'NMSH_RETURN_SESSION', 'NMSH_HANDOFF_SHELL', 'TERM_PROGRAM']; const saved = new Map(keys.map(key => [key, process.env[key]])); const previousCwd = process.cwd(); diff --git a/tests/queryOrder.test.ts b/tests/queryOrder.test.ts index fbd0f12e..e3b04c3d 100644 --- a/tests/queryOrder.test.ts +++ b/tests/queryOrder.test.ts @@ -6,6 +6,7 @@ import {join} from 'node:path'; import {tmpdir} from 'node:os'; import {QueryOrder} from '../src/shell/QueryOrder.js'; import {ShellSession} from '../src/shell/ShellSession.js'; +import {markFishCompletionsGenerated} from '../scripts/test-shell-home.mjs'; const DA1 = '\u001b[?62;22c'; @@ -114,6 +115,7 @@ const fish = ['/opt/homebrew/bin/fish', '/usr/local/bin/fish', '/usr/bin/fish']. test('fish\'s read is answered like its prompt: its queries never reach the host, and it reads at once', {skip: fish ? false : 'fish not installed', timeout: 30_000}, async () => { const home = mkdtempSync(join(tmpdir(), 'nt-qo-')); + markFishCompletionsGenerated(home, {dataHome: join(home, 'data')}); const shell = new ShellSession(home, 100, 30, home, {...process.env, HOME: home, XDG_CONFIG_HOME: join(home, '.config'), XDG_DATA_HOME: join(home, 'data'), PATH: `${fish!.replace(/\/fish$/u, '')}:${process.env.PATH}`, TERM: 'xterm-256color'}, 'fish'); let output = ''; diff --git a/tests/queuedHighlight.test.ts b/tests/queuedHighlight.test.ts index aea05775..165c1ba1 100644 --- a/tests/queuedHighlight.test.ts +++ b/tests/queuedHighlight.test.ts @@ -1,18 +1,46 @@ import test from 'node:test'; import assert from 'node:assert/strict'; +import {once} from 'node:events'; import {isolateConfig} from './support/isolatedConfig.js'; import {TerminalApp} from '../src/app/TerminalApp.js'; -/** A semantic service that classifies on request, as the real one does, without starting a shell. */ -function classifyingApp() { - const app = new TerminalApp(); - app['render'] = () => {}; +/** + * A semantic service that classifies on request, as the real one does. The app's real shell reports its startup + * knowledge first: that replaces the classification cache, so it must land before the test's own classifications + * rather than in the middle of them (#408). Every lookup the app starts is recorded, so a test waits for exactly + * those (and the recoloring chained on them) instead of guessing at event-loop turns. + */ +async function classifyingApp() { + const app = await readyApp(); const semantic = app['semanticService']; semantic.cache.clear(); - semantic.classifyCommand = async (word: string) => { semantic.cache.set(word.split(' ')[0]!, 'executable'); return 'executable' as const; }; + const lookups: Promise[] = []; + semantic.classifyCommand = (word: string) => { + const lookup = (async () => { semantic.cache.set(word.split(' ')[0]!, 'executable'); return 'executable' as const; })(); + lookups.push(lookup); + return lookup; + }; + // The recoloring callbacks were attached to each lookup before this waits on them, so they have run when it resolves. + const settle = async () => { await Promise.all(lookups); }; + return {app, settle}; +} + +/** An app whose real shell has reported its startup knowledge. */ +async function readyApp(): Promise { + const app = new TerminalApp(); + app['render'] = () => {}; + try { await once(app['session'], 'prompt', {signal: AbortSignal.timeout(20_000)}); } + catch (error) { app['stop'](0); app['session'].kill(); throw error; } return app; } -const settle = () => new Promise(resolve => setImmediate(resolve)); + +async function until(check: () => boolean, what: string, timeoutMs = 10_000): Promise { + const deadline = Date.now() + timeoutMs; + while (!check()) { + if (Date.now() > deadline) throw new Error(`timed out waiting for ${what}`); + await new Promise(resolve => setTimeout(resolve, 20)); + } +} type Cell = {text: string; style: string}; const cells = (app: TerminalApp, startId: number): Cell[] => app['output'].transcript().lines[startId] as Cell[]; const text = (line: Cell[]) => line.map(cell => cell.text).join(''); @@ -21,7 +49,7 @@ const styleOf = (line: Cell[], word: string) => line[text(line).indexOf(word)]!. test('a queued command is highlighted like a composer command: its command word is classified and the stored line recolored', async () => { const isolation = isolateConfig(); - const app = classifyingApp(); + const {app, settle} = await classifyingApp(); try { const sgr = app['syntaxSgr']; // The queue runs an entry the composer never saw, so its command word is not classified yet. @@ -41,7 +69,7 @@ test('a queued command is highlighted like a composer command: its command word test('a command the composer submitted and a queued one end with the same colors', async () => { const isolation = isolateConfig(); - const app = classifyingApp(); + const {app, settle} = await classifyingApp(); try { app['onShellExec']('cargo build --release', Date.now(), undefined, 'cargo build --release'); const queuedId = app['output'].activeStartId!; @@ -53,3 +81,20 @@ test('a command the composer submitted and a queued one end with the same colors assert.equal(styleOf(queued, 'cargo'), app['syntaxSgr'].KnownCommand, 'and so does the queued block'); } finally { app['stop'](0); app['session'].kill(); isolation.restore(); } }); + +test('a stored command whose classification newer shell knowledge superseded still ends classified (#408)', {skip: process.platform === 'win32'}, async () => { + const isolation = isolateConfig(); + const app = await readyApp(); + try { + const sgr = app['syntaxSgr']; + app['semanticService'].cache.clear(); + // A fast queue: the entry starts, and the previous entry's prompt knowledge lands while its lookup is in flight. + app['onShellExec']('ls -la', Date.now(), undefined, 'ls -la'); + const startId = app['output'].activeStartId!; + assert.equal(styleOf(cells(app, startId), 'ls'), sgr.Command, 'unclassified at first'); + app['semanticService'].applyShellKnowledge('complete\n'); + await until(() => styleOf(cells(app, startId), 'ls') === sgr.KnownCommand, 'the stored line to be recolored from the new knowledge'); + assert.equal(styleOf(cells(app, startId), '-la'), sgr.Flag); + assert.ok(app['formatCommandAnsi']('ls -la', null)[0]!.includes(`${sgr.KnownCommand}l`), 'the composer path agrees'); + } finally { app['stop'](0); app['session'].kill(); isolation.restore(); } +}); diff --git a/tests/shellAdapters.test.ts b/tests/shellAdapters.test.ts index dd95fd6c..136bc775 100644 --- a/tests/shellAdapters.test.ts +++ b/tests/shellAdapters.test.ts @@ -1,7 +1,8 @@ import test from 'node:test'; import assert from 'node:assert/strict'; +import {spawnSync} from 'node:child_process'; import {once} from 'node:events'; -import {chmodSync, mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, writeFileSync} from 'node:fs'; +import {chmodSync, mkdirSync, mkdtempSync, readdirSync, readFileSync, realpathSync, rmSync, writeFileSync} from 'node:fs'; import {connect, type Socket} from 'node:net'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; @@ -19,6 +20,7 @@ import {parseSlashCommand} from '../src/commands/slashCommands.js'; import {normalizePromptConfiguration} from '../src/prompt/configuration.js'; import {createShellPanel, shellPanelKey} from '../src/shell/ShellPanel.js'; import type {ShellMarker} from '../src/shell/ShellProtocol.js'; +import {markFishCompletionsGenerated} from '../scripts/test-shell-home.mjs'; const available = (id: ShellId) => Boolean(shellAdapter(id).resolveExecutable(process.env)); const scratch = (prefix: string) => realpathSync(mkdtempSync(join(tmpdir(), prefix))); @@ -89,8 +91,9 @@ interface Lifecycle { markers: ShellMarker[]; execs: Array<[string, number | und async function startShell(id: ShellId): Promise { const home = scratch(`nmsh-adapter-${id}-`); mkdirSync(join(home, 'work')); + markFishCompletionsGenerated(home); const shell = new ShellSession(home, 100, 30, home, {...process.env, HOME: home, XDG_CONFIG_HOME: join(home, '.config'), XDG_DATA_HOME: join(home, '.local', 'share'), - LANG: 'C.UTF-8', LC_ALL: 'C.UTF-8'}, id); + XDG_CACHE_HOME: join(home, '.cache'), LANG: 'C.UTF-8', LC_ALL: 'C.UTF-8'}, id); const state: Lifecycle = {markers: [], execs: [], output: '', shell, home}; shell.on('prompt', marker => state.markers.push(marker)); shell.on('exec', (command, allowed) => state.execs.push([command, allowed])); @@ -151,6 +154,19 @@ async function stopShell(state: Lifecycle): Promise { rmSync(state.home, {recursive: true, force: true, maxRetries: 5, retryDelay: 100}); } +test('fish: a test home marked as generated starts no detached man-page completion generator (#403)', {skip: !available('fish') && 'fish is not installed', timeout: 60_000}, async () => { + const state = await startShell('fish'); + try { + // Fish starts the generator during interactive startup, before its first prompt, and it runs for seconds. + await run(state, 'true'); + const generators = (spawnSync('ps', ['-axo', 'command='], {encoding: 'utf8'}).stdout ?? '').split('\n') + .filter(line => line.includes('--manpath') && line.includes(state.home)); + assert.deepEqual(generators, [], 'nothing outlives the shell and writes into its home'); + assert.deepEqual(readdirSync(join(state.home, '.cache', 'fish', 'generated_completions')), []); + assert.deepEqual(readdirSync(join(state.home, '.local', 'share', 'fish', 'generated_completions')), []); + } finally { await stopShell(state); } +}); + test('fish: a leading space is private, and Fish completion feeds structured candidates', {skip: !available('fish') && 'fish is not installed', timeout: 60_000}, async () => { const state = await startShell('fish'); try { @@ -224,6 +240,7 @@ test('hot swap in the service: zsh → fish → bash → zsh keeps session ident const runtimeDir = scratch('nmsh-swap-'); chmodSync(runtimeDir, 0o700); const home = scratch('nmsh-swaph-'); mkdirSync(join(home, 'project')); + markFishCompletionsGenerated(home, {dataHome: join(home, '.d')}); const service = new SessionService({runtimeDir, startupIdleMs: 60_000}); await service.start(); const keeper = new Peer(service.socketPath); diff --git a/tests/shellContextEnvironment.test.ts b/tests/shellContextEnvironment.test.ts index a24c8c24..cf573cd3 100644 --- a/tests/shellContextEnvironment.test.ts +++ b/tests/shellContextEnvironment.test.ts @@ -5,6 +5,7 @@ import {mkdtempSync, realpathSync, rmSync} from 'node:fs'; import {tmpdir} from 'node:os'; import {join} from 'node:path'; import {ShellSession} from '../src/shell/ShellSession.js'; +import {markFishCompletionsGenerated} from '../scripts/test-shell-home.mjs'; import {shellAdapter} from '../src/shell/adapters/registry.js'; import {SHELL_IDS, knowledgeJobCount, type ShellId} from '../src/shell/adapters/ShellAdapter.js'; import {parseShellKnowledge} from '../src/shell/ShellKnowledge.js'; @@ -15,6 +16,7 @@ const available = (id: ShellId) => Boolean(shellAdapter(id).resolveExecutable(pr async function start(id: ShellId): Promise<{shell: ShellSession; home: string}> { const home = realpathSync(mkdtempSync(join(tmpdir(), `nmsh-envsnap-${id}-`))); + markFishCompletionsGenerated(home); // The launch environment carries a secret and a frontend-only value: neither may be reported as a value. const shell = new ShellSession(home, 100, 30, home, {...process.env, HOME: home, XDG_CONFIG_HOME: join(home, '.config'), XDG_DATA_HOME: join(home, '.local', 'share'), LANG: 'C.UTF-8', LC_ALL: 'C.UTF-8', AWS_SESSION_TOKEN: 'LAUNCH_TOKEN_VALUE', AWS_PROFILE: 'launch-profile'}, id); diff --git a/tests/shellIntelligence.test.ts b/tests/shellIntelligence.test.ts index 86fbe9a8..fb1e9616 100644 --- a/tests/shellIntelligence.test.ts +++ b/tests/shellIntelligence.test.ts @@ -102,11 +102,17 @@ test('complete snapshots reveal unshadowed commands, partial snapshots preserve service.applyShellKnowledge('function positive\npartial\n'); assert.equal(await service.classifyCommand('positive'), 'function'); assert.equal(await service.classifyCommand('configured_only'), 'alias'); + // A lookup in flight when newer knowledge arrives is asked again against that knowledge (#408): the answer its + // old knowledge implied (plain lookup: unknown) is never cached, and the new one (configured: the rc alias) is. + service.applyShellKnowledge('complete\n'); + const superseded = service.classifyCommand('configured_only'); + service.applyShellKnowledge('partial\n'); + assert.equal(await superseded, 'alias'); + assert.equal(service.cache.get('configured_only'), 'alias'); service.applyShellKnowledge('alias live_alias\ncomplete\n'); - const stale = service.classifyCommand('stale_result'); + const named = service.classifyCommand('latest'); service.applyShellKnowledge('function latest\ncomplete\n'); - assert.equal(await stale, 'unknown'); - assert.equal(service.cache.has('stale_result'), false); + assert.equal(await named, 'function', 'answered from the knowledge that superseded it, not settled as unknown'); service.kill(); assert.equal(await service.classifyCommand('latest'), 'function'); } finally {