diff --git a/CHANGELOG.md b/CHANGELOG.md index fb6a6cb3e..a9a8e1b8e 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,36 @@ it shipped into a `## [x.y.z] - date` section of their own; the tag's ### Added +- The MCP server speaks the stateless protocol revision 2026-07-28 over + stdio and HTTP, per request, beside the `initialize`-based ones: `server/discover`, + `resultType` and caching hints on results, the `-32020`–`-32022` error + codes, `subscriptions/listen` for tool-list changes and resource updates, + and destructive-tool confirmation as a multi round-trip + (`input_required` with a signed `requestState`). Over HTTP a stateless + request needs the `Mcp-Method` / `Mcp-Name` headers and is served without + a session. Existing clients are served as before. +- The MCP server negotiates protocol version 2025-11-25 and sends its + `description` in `serverInfo` to clients of that revision. +- Computer use with `computer_toolset_20260801` answers `zoom` with a + full-resolution crop of the region. +- `WorkQueueError` and `CheckpointStoreError` (both + `AutoControlException`) for a database that cannot be opened or used. +- `stop_event=` on `AgentLoop`, `run_computer_use` and `run_dag`; the + Computer Use and DAG Runner tabs have a Stop action, and closing the + window asks a running job to stop. +- `parse_multipart` files carry `content_base64` (the exact bytes). +- Computer use speaks the GA `computer_toolset_20260801`, used + automatically for `claude-opus-5-5` (which rejects the beta tool); pass + `tool_type="computer_toolset_20260801"` to use it with other models. +- REST `POST /execute` accepts `"raise_on_error": true`: the run stops at + the first failing action and answers `{"ok": false, "error": ...}`. + `AdminConsoleClient.broadcast_execute(raise_on_error=True)` reports such a + host as `ok: false`. +- `HistoryStore.list_runs(script_path=...)`, and `environ=` on + `validate_config` / `ConfigSchema.validate`. +- **`AC_idempotency_release`** / MCP `ac_idempotency_release` / Script + Builder *Idempotency: Release*: free an in-progress idempotency key whose + work failed so a retry runs it. - `cua_action.resolve_key_name` / `split_key_combo`, and `compile_postcondition(before=...)`. - `pii_text.luhn_valid` and `normalize_text(strip_format=...)`. @@ -54,6 +84,101 @@ it shipped into a `## [x.y.z] - date` section of their own; the tag's ### Changed +- An MCP HTTP request whose `MCP-Protocol-Version` header names an + unsupported version is still a 400, now with a JSON-RPC + `UnsupportedProtocolVersion` (`-32022`) body listing the supported + versions instead of `{"error": ...}`. +- Quick Connect verifies the host certificate for `wss://` targets (use the + Advanced viewer with *Skip cert verification* for self-signed hosts). +- The signaling client and the USB browser's device fetch go through + `http_client`: http(s) only, the egress policy applies, and credentials + do not follow a redirect to another host. +- `match_theme`, `propose_elements` and `read_barcodes` return screen + coordinates for a screen grab (they were relative to `region`). +- `plan_open` / `open_path` refuse opaque URL schemes off the allow list; + webhook transports are case-insensitive and unknown ones are refused. +- `handle_file_dialog` waits for a window titled exactly like the dialog and + types only after bringing it to the front; watchdog key rules press their + key only in the popup. +- Empty process names and window-title patterns are refused by the waits + and process assertions. +- `assert_http` and config sync send their requests through `http_client`: + the egress policy and the body cap apply, and config sync no longer + follows redirects. +- S3 store failures raise `ArtifactStoreError`; unreadable Office files raise + `AutoControlActionException`. +- VLM locate / click raise `VLMRequestError` when the request fails, + instead of reporting the element as not found. +- Agent tool schemas carry resolved parameter types and omit private and + callback parameters; the OpenAI agent backend refuses more than 128 tools. +- Generated tests and scripts replay actions with + `ac.executor.execute_action(..., raise_on_error=True)`, so they fail at + the first failed action; actions holding `${...}` go through the executor. +- Several `je_auto_control_mcp --list-*` flags print one JSON object. +- Accessibility matches with an on-screen rectangle come first, and + `click_accessibility_element` returns `False` for a match without one. +- Anchor locate raises when its OCR, accessibility or VLM backend is not + set up, instead of reporting the anchor as not found. +- `AC_shell_command` fails when its program cannot start (it reported + success) and logs only the program, not its arguments. +- `AC_read_file_to_var` reads `utf-8-sig` by default. +- The Live HUD samples only while it is on screen; its log tail keeps + collecting while it is hidden. +- MCP `tools/call` arguments that fail the tool's input schema are + answered as a tool execution error (`isError: true`), not a `-32602` + JSON-RPC error, so the model can correct them. +- The MCP HTTP transport answers a wrong bearer token 401 (was 403), + as the MCP authorization spec requires; every 401 from it and the REST + API carries a `WWW-Authenticate: Bearer` challenge. +- The REST API answers a known path asked with the other method 405 with + `Allow` (was 404). +- `AdminConsoleClient` treats `labels=[]` as no host (was every host). +- Config sync breaks timestamp ties the same way on every client and + reports them as conflicts. +- `AC_call_macro` restores the caller's variables of the parameters' + names after the call. +- A plugin command named like a block command is refused. +- The LLM cost table carries current Claude list prices (Opus 4.7 is + $5/$25, not $15/$75) and resolves dated or provider-prefixed ids. +- `vex_statement` takes `action_statement=` and requires it for + `affected` (OpenVEX). +- The SBOM prefers PEP 639 `License-Expression`; licence evaluation + reads every `licenses` entry. +- `perceptual_diff` discounts anti-aliasing with pixelmatch's test + instead of a morphological open, so thin real changes (small text, + 1 px rules) now count. +- `image_histogram` channels are L1-normalised; histogram intersection + divides by the larger mass. +- `ssim_compare` scales its constants to the images' dynamic range and + reports -1..1. +- `confusable_skeleton` follows UTS #39 (NFKD, map, NFD); `×` and `÷` + count as Common script. +- Without rapidfuzz, fuzzy scores are the symmetric Indel ratio (the + same as the rapidfuzz backend). +- Toolset screenshots are fitted into the high-resolution tier (2576 px + long edge, 4784 visual tokens) instead of 1568 px / 1.15 MP. +- `parse_dotenv` decodes `\'` and `\\` inside single-quoted values, as + python-dotenv does. +- `format_message` keeps an apostrophe before `#` outside a plural and + before `|` (ICU); French has the CLDR `many` category. +- `parse_rrule` raises `AutoControlException` for RRULE parts it does + not support (`BYHOUR`, `BYWEEKNO`, `BYYEARDAY`…) and for `COUNT` with + `UNTIL`, instead of silently ignoring them. +- `CookieJar.update` keeps a cookie with an empty value (only + `Max-Age<=0` or a past `Expires` deletes one), and `parse_traceparent` + accepts a newer version (read as `00`); only `ff` is rejected. +- `parse_problem` ignores `title` / `status` / `detail` / `instance` of + the wrong JSON type instead of keeping or coercing them. +- `format_annotation` / `emit_annotations` / `AC_ci_annotations` raise + `ValueError` for an unknown level instead of emitting `error`. + `generate_sop` raises `ValueError` for a step that is neither a list nor a + command string. +- `compare_field_value` / `verify_field_value` / `fill_and_verify` raise + `ValueError` for an unknown `mode` and accept any case. Profiles of numeric + columns carry a `non_finite` count. +- **Webhook methods**: `PATCH` webhooks are served; verbs the server cannot + answer (e.g. `HEAD`) are refused when the webhook is added instead of + returning 501 on every request. - **Golden-image capture (`take_golden` / `compare_to_golden`) reads its region in mouse coordinates**, like every other capture; region goldens taken on a scaled display need re-taking. @@ -198,6 +323,13 @@ it shipped into a `## [x.y.z] - date` section of their own; the tag's ### Security +- The egress policy matches hosts by their IDNA encoding, so soft + hyphens, fullwidth characters and ideographic full stops no longer + slip past a deny list. +- `GPLv3+`, `GPL-3.0 License` and `LGPL-2.1-or-later` are caught by the + copyleft deny list. +- The secrets scan skips only values that are a single placeholder. +- Secret redaction no longer stops at an escaped quote. - **HTTP cassettes no longer record credential headers**, and a `match_on` field they cannot compare raises instead of matching every request. - **JWT decoding rejects characters outside base64url** (which made tokens @@ -295,6 +427,323 @@ it shipped into a `## [x.y.z] - date` section of their own; the tag's ### Fixed +- The Flow Editor opens action files saved with a BOM, keeps a wrapped + file's other keys on save, and writes atomically. +- The region selector (template cropping, OCR / screenshot / WebRTC regions) + covers every screen and returns native pixels: it was offset by the + virtual desktop's origin and did not cover screens with display scaling. +- Numeric fields in the Image Detect, Auto Click and Screenshot tabs accept + a decimal point and refuse grouped digits under any locale. +- Script Builder: shown defaults equal the executor's (an edit no longer + writes `detect_threshold: 0.8` into an exact image match, nor a file path + for agent-card / SBOM / MCP manifest steps); grid and annotation fields are + required; positional arguments and the other keys of a wrapped file + survive load and save; decimals can be typed under comma locales. +- WebRTC host annotations from a viewer are validated and bounded. +- The tray icon keeps the application running only while a host runs. +- Stopping a host while its approval dialog is open no longer raises. +- Trust-list and address-book imports report files of the wrong shape. +- Stop cancels a pending WebRTC auto-reconnect; a new remote window keeps + the pen mode; an empty recording is no longer reported as saved. +- Quick Connect handles ws:// sessions on close and in its status badge, + ends the session on an error, and closes a timed-out approval box. +- The Quick Connect popup forwards mouse and keyboard input. +- Closing a remote screen window no longer aborts the process when its + panel is deleted first. +- Remote desktop connect errors (such as an invalid host name) are reported. +- Audio follows its checkbox while the Advanced section is collapsed. +- The host share text quotes the settings the host started with. +- Address-book, known-hosts and remote-inbox data of the wrong shape no + longer raise in the GUI. +- The voice router is safe under concurrent use; `match_ensemble` votes on + one frame. +- Failure signatures group failures that differ only in timings. +- `AC_delta_observation` counts match its summary, and removed elements + carry no stale index. +- The package imports without a home directory; the path guard reports a + missing home as `PathNotAllowedError`. +- Adaptive timeouts, checkbox reads and contrast checks handle infinite + samples, clipped boxes and anti-aliased edges. +- MCP `ac_kill_process` and `wait_until_window_title` no longer let psutil + or regex errors escape. +- File associations report `None` for an unregistered type; file drops and + clipboard file lists carry absolute paths. +- The Window Manager tab focuses and closes the selected window, not the + first title match. +- Secrets nested inside action arguments, and webhook URLs, are masked in + logs and run records. +- `X-Api-Key` and `X-Auth-Token` are dropped on redirects to another origin. +- A truncated HTTP error body or a deeply nested reply no longer aborts a + script. +- The IMAP trigger fires once per message, honours UIDVALIDITY and accepts + non-ASCII mailbox names. +- HTTP cassettes mask credentials in query strings and bodies. +- SQLite data sources that cannot be opened raise an action error. +- Agent turns cut short by `max_tokens` or a refusal no longer run their + tool calls; OpenAI refusals and filtered replies are not final answers. +- Computer-use scrolls keep their direction, the cursor position is in + screenshot pixels, and `ctrl++` keeps its plus key. +- The Anthropic VLM backend reads replies in the pixels of the image the + model saw, and replies such as `x=512, y=300` or decimals are read. +- A reused agent backend starts each run afresh; the agent loop records + more command errors as step errors instead of ending the run. +- Bedrock `global.` ids and dated OpenAI ids are priced. +- `je_auto_control_mcp --read-only` restricts the server's tools, not + only the listings. +- The MCP stdio server and the CLIs write UTF-8 whatever the console code + page. +- `--port` outside 0-65535 is a usage error instead of a traceback. +- Codegen refuses action shapes the executor refuses, keeps dict key order, + and takes lone surrogates and sigil runs in Robot output. +- SARIF and SOP writers take lone surrogates; SOP creates its folder and + writes atomically; Allure results carry start and stop times. +- A closed window's `COMError` no longer escapes UIA state reads; `find_text` + no longer finds absent text. +- macOS accessibility elements report their real bounds. +- Accessibility audits and `describe_screen` work on Windows. +- The accessibility recorder keeps running after a backend error; the + Linux control search is bounded. +- Mixed text formatting reads as unknown; Tesseract errors other than a + missing binary say what failed; integer roles match. +- `AC_shell_to_var` refuses cmd metacharacters in a batch file's + arguments, and its timeout, like MCP `shell_command`'s, ends everything + the command started. +- MCP `shell_command` keeps output the code page cannot decode strictly. +- An empty Linux clipboard reads as empty; clipboard tool failures are + `ClipboardError`. +- adb errors no longer carry the command's arguments; `no permissions` + devices get that state. +- USB/IP: device lists on Windows and with alternate settings, speed codes, + idle attached devices, and isochronous URBs. +- `import je_auto_control` no longer emits a DeprecationWarning from + defusedxml, so it works in test suites that turn warnings into errors. +- SARIF export no longer calls `PurePath.as_uri()`, deprecated in Python + 3.14. +- USB sharing, its hotplug watcher and the passthrough flag it turned on + are released when the USB Sharing panel is destroyed. +- GUI slots show missing or malformed files, images not on screen, bad + regions, unknown commands and closed windows instead of raising. +- Hidden tabs, the Live HUD's log tail, the main window's language + listener and USB prompt dialogs no longer outlive their window. +- A GUI worker's unexpected exception reaches its failure callback. +- The Script Builder keeps a choice value it does not list. +- REST and MCP replies holding a lone surrogate, REST replies that cannot + be serialised, huge `/history` limits and JSON nested too deeply no + longer drop the connection without a response. +- Requests with conflicting `Content-Length` headers are refused. +- Access-log lines of the REST, MCP HTTP and webhook servers escape + control characters. +- A histogram given a `+Inf` bucket renders it once. +- `ResourceProfiler.is_running` is right without psutil. +- Scheduled, triggered, hotkey, webhook and e-mail runs in which an + action failed are recorded as errors, with an error snapshot. +- `*/15`-style cron jobs keep their pace through the repeated DST hour. +- Re-enabled scheduler jobs wait for their next slot; interval jobs no + longer drift. +- A trigger replaced under the same id is not charged for the old run. +- One popup-watchdog rule's error no longer stops the others. +- Concurrent hotkey daemon start/stop no longer leaves a loop running. +- `AC_retry` backoff is capped at 300 s. +- A remote-desktop upload aborted while it was starting no longer leaves + its `.part` file and open handle behind. +- The action JSON Schema lists every command, block commands included, + and types parameters from their annotations instead of "string". +- The linter reports a block command's missing required arguments + (e.g. `AC_sleep` without `seconds`). +- `${webhook.body}`, `${email.subject}` and other dotted trigger + variables resolve in scripts. +- Step repair repeats a no-op action whatever form the verdict takes. +- The self-healing log survives a torn multi-byte line. +- A/B locator reports see other stores' records; a strategy that never + succeeded is not recommended. +- Time-travel replay shows actions logged before the first frame. +- Agent memory recalls CJK keywords. +- Variable files with a UTF-8 BOM load. +- `AC_trace_reset` starts a new trace id. +- Exiting while a WebRTC signaling poll is running, or closing the + remote-desktop viewer during a file transfer, no longer aborts the + process. +- `merge_results` no longer doubles errors and cases when merging merged + reports. +- Time-series buckets place edge points correctly at Unix-time + magnitudes. +- `assert_text(regex=True)` honours `ignore_case`. +- `wait_until_screen_stable` measures the quiet time from the first + matching frame. +- `summarise_llm_costs()` works without arguments. +- `validate_rows` reports huge integers instead of raising. +- Welch confidence intervals at tiny alpha and df near 1. +- MCP `initialize` answers with a protocol version the server supports + (not whatever the client sent) and declares only server capabilities; + an unsupported `MCP-Protocol-Version` header gets 400; + `request_sampling` needs the client's sampling capability. +- `AC_run_agent` with the Anthropic backend fits screenshots into the + model's image tier and maps tool-call `x` / `y` back to the screen. +- Screenshot fitting follows the documented resize rule exactly. +- SLSA provenance omits empty metadata timestamps; verification reports + a subject without a name instead of raising. +- PEP 440 ordering of `.postN.devM` and of omitted numbers. +- Redaction boxes merge until none overlap. +- OSV ranges with several introduced/fixed pairs match every pair. +- In-memory approval gates and the credential broker are thread-safe. +- Print-format IBANs are detected (mod-97 checked). +- SARIF findings without a severity are warnings; the name "Dan" is + not a jailbreak marker. +- Computer use on the beta tool fits screenshots into the model's image + tier, declares that size and maps coordinates back, so clicks land + correctly on screens above the model's image limits (4K, or 1080p on + standard-tier models). +- `ssim_compare` accepts single-channel HxWx1 arrays. +- Heading detection uses the true median line height. +- Element matching never pairs boxes that do not overlap. +- `average_hash` / `dhash` accept NumPy arrays. +- `apply_unified`, `unified_diff` and `three_way_merge` split lines at + line feeds only, so form feeds and U+2028 inside a line survive and + CRLF text keeps its endings. +- Readability counts only sentences that hold a word. +- `normalize_text(casefold=True)` stays in the requested form. +- `is_balanced` checks bidi controls per paragraph. +- Closing the window while a GUI job is inside a long step (an LLM + request) no longer aborts the process. +- `format_message` accepts `offset: 1` and reports an infinite count + as not a number. +- `read_mo` decodes a catalogue in the charset its header declares. +- JSON Schema `$ref` tokens follow RFC 6901 (ASCII indexes, + percent-decoded fragment). +- A `;` inside an SQL string literal no longer counts as a second + statement. +- `parse_number("Infinity")` raises `ValueError`. +- The secret vault no longer loses a secret when two managers write + it at once. +- JSON stores read a file with a UTF-8 BOM, and a write waits for a + Windows reader instead of failing. +- HTTP cassettes redact `set_cookie`. +- A truncated deflate body raises instead of returning a prefix; + `x-gzip` is accepted. +- JSONPath `!=` keeps nodes that lack the member; a filter string may + contain `)]`. +- `parse_multipart` keeps a backslash in a filename. +- `LatencyDigest` percentiles over negative values. +- `decode_jwt` raises `JwtError` for a non-string `alg` (was + `TypeError`) and rejects any `crit` header. +- The SSE parser no longer drops an event when a CRLF is split so the + `\n` arrives alone. +- An escaped quote inside a quoted Link or Cache-Control parameter no + longer ends the string (a Link header's `rel` could be lost). +- `urls_equal` / URL normalisation decode escaped unreserved characters + (`%7E` is `~`). +- Closing a GUI tab or the main window while its background job + (Admin Console poll, USB browser, computer use, DAG, LLM planner) is + running no longer aborts the process. +- Closing the LAN browse dialog with Use, Cancel or Esc stops its mDNS + browser, and a closed presence tab no longer stays registered with the + presence registry. +- Computer use: + - drags end at the model's `coordinate`; + - `key` honours `repeat`; + - modifier keys on clicks, drags and scrolls are held. +- A DAG remote node whose actions failed on the host no longer counts as + succeeded, and the Admin Console broadcast shows a remote action failure + as a failed host. +- **GUI threads**: + - Admin Console refresh and thumbnails, and the USB Browser and passthrough actions, now run; their workers were collected before starting. + - Worker results are applied on the GUI thread. + - The Quick Connect viewer no longer repaints or opens dialogs from its network thread. + - Stopping or restarting a WebRTC signaling session no longer aborts the application. +- **Traces and reports**: + - Replay traces with Unicode line separators read back. + - `match_persistence` requires every frame to agree. + - `hold_modifiers("shift")` presses Shift, not five letters. + - Change boxes off the frame score nothing. + - OTLP output is strict JSON with typed array and map values. + - `generate_sop` reads the wrapped action-file form. +- **Utilities**: + - Dropping files onto a window no longer leaks memory on failure and no longer reports success for window 0. + - A zero-weight grounding candidate no longer divides by zero. + - Table cells and borderless rows read in reading order. + - A data profile survives `inf` / `nan`. + - An unknown verify mode is refused. + - Collation distinguishes accent position and handles decomposed text. + - `parse_cf_html` applies header offsets to `str` input. + - A superscript digit no longer crashes role parsing. + - Out-of-range HSV bounds are clamped or wrapped. +- **macOS media keys**: pressing a media key no longer raises + `AttributeError`; the PyObjC selector name was missing its trailing `_`. +- **Error family**: twenty-five errors that derived only from a builtin + exception (the Android and iOS clients, the remote-desktop wire errors, + ACME, the circuit breaker, egress, the Interception loader and others) now + also derive from `AutoControlException`, so family-only boundaries contain + them; `except RuntimeError` / `except ValueError` still catch them. +- **X11, uinput and macOS input**: the uinput backend types the intended + keys and scrolls with the same sign rules as XTest; sending keys and clicks + to an X window releases what it pressed; an unbound X key raises instead of + pretending; recorded wheel events no longer break replay; macOS media keys + press and release; a timed-out macOS recording tap is re-enabled. +- **Wayland**: libei input keeps working after a screen lock or VT switch + and releases each device reference once; screen size follows rotated and + scaled outputs; a malformed capture-command override is a screen error. +- **WebRTC media**: screen frames are stamped at the rate they are sent and + frame rates above 30 fps take effect; host voice plays at the right speed on + a mono output; an audio device that fails no longer aborts the connection or + wedges later starts; a viewer reused for a second session shows video; + mDNS and the asyncio bridge release their resources on failure and stop. +- **USB passthrough, agent loop and locators**: the second of two identical + USB devices opens by serial, a transfer whose direction contradicts the + endpoint is refused and closing gives the device back to the kernel; a + malformed agent decision no longer ends the run; a screenshot failure in + self-heal is a miss; USB/IP, self-heal and anchor errors are + `AutoControlException`s; the a11y audit no longer flags table cells. +- **Layout and data checks**: flow selection and sharding see a flow's own + history however many other runs followed it; column reading order survives + long runs of paragraphs; `ConfigField.env` is honoured and lossy int + coercion refused; single-column uniqueness ignores nulls; unit lists follow + CLDR outside English; A2A card modes are MIME types; `within` excludes the + pixel past its region; Windows accessibility reads edit and slider values. +- **MCP, USB passthrough and device helpers**: an overflowing or short + argument no longer leaves an MCP request unanswered; a failed drag releases + the button; sampling works over HTTP; waits look once at `timeout=0` and + survive an infinite poll; USB credits are never missed and transfers on one + claim no longer swap data; assertion failures propagate through callbacks; + gamepad, clipboard and volume errors stay in the `AutoControlException` + family. +- **Remote desktop**: WebSocket frames follow RFC 6455 masking and + control-frame limits and a malformed handshake key no longer kills the + handshake thread; a transfer to a path naming no file fails cleanly; the + encrypted recorder's frame count matches its entries and a tampered manifest + verifies as `False`; a mic whose device failed to start can start again; + turning off viewer audio keeps the host's voice. +- **Triggers and scheduler**: concurrent trigger-engine start / stop no + longer doubles the polling thread or raises; one malformed email no longer + stops a mailbox's polling; mailbox names with spaces or brackets work; a + string `max_runs` stops the job; a corrupt .xlsx data source is an ordinary + action error. +- **JSONPath, OCR structure and H.264**: quoted unions and mismatched quotes + raise, quoted names decode escapes, parenless filters no longer run into the + next filter, and ordering follows RFC 9535; OCR field values come from the + cell below the label; only hardware encoders that really open are listed and + each gets options it accepts (NVENC works); odd-sized frames encode and + `close()` always closes the container. +- **Text, config and registries**: `-or-later` licences are no longer split + (a denylist naming one now holds); REST USB booleans must be JSON booleans; + `.po` entries need no blank line between them and CRLF files parse; + negative numbers take CLDR's plural category and large counts keep their + digits; `file://` URIs naming another host are refused; coturn fields and + XML names cannot inject directives or markup; presence ids are matched as + registered and its errors are `AutoControlException`s. +- **Data utilities**: JWTs are rejected at their expiry second and only in + canonical base64url; malformed tokens raise `JwtError`; n-gram similarity + rejects `n < 1`; `DagDefinitionError` is an `AutoControlException`; + time-series samples sharing a timestamp keep their order and unknown fills + raise; NaN fails range rules; feature flags accept `{"variant": ...}` and + fall back on malformed serves; schema compatibility sees required-only + fields and rejects unknown modes; CSV test data takes rows with different keys. +- **Small utilities**: `use_ssl` mail defaults to port 465; the resource + profiler's report stops at `stop()` and its speedscope export loads; + `Content-Length` must be ASCII digits and `chunked` the final coding; + `web_screenshot` works against WebRunner; failed notifications report + `shown=False` and Windows toasts appear; a dead key no longer reads as its + US character; inverted and off-image regions are handled in annotate and + colour stats. - **Emergency stop on Linux / macOS**: the stop key wakes a sleeping or waiting main thread there too (SIGINT is sent to the main thread). - **Image analysis and packaging**: colour-vision simulation uses Machado diff --git a/CLAUDE.md b/CLAUDE.md index 90583ba24..7e8826f65 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -95,7 +95,7 @@ The map is only useful while it matches the tree, so **update it in the same cha ### README stays current and in sync across all three languages -`README.md` (English, the source), `README/README_zh-CN.md`, and `README/README_zh-TW.md` MUST stay current with the code. Any user-facing change (a feature or API surface, an `AC_*` command, a CLI flag, a GUI tab, install / setup, configuration, an env var, a requirement, or a quoted count) updates `README.md` **and both translations in the same commit**, structure and content aligned. Never update one language and leave the others stale. `test/unit_test/headless/test_doc_counts.py` guards the quoted figures across all three (see above), but everything else — new sections, changed commands, reworded setup — is on you: read the diff against all three before committing. +`README.md` (English, the source), `README/README_zh-CN.md`, and `README/README_zh-TW.md` MUST stay current with the code. Any user-facing change (a feature or API surface, an `AC_*` command, a CLI flag, a GUI tab, install / setup, configuration, an env var, a requirement, or a quoted count) updates `README.md`, **both translations, and the user-facing docs under `docs/` (the Sphinx tree in `docs/source/` and reference docs such as `docs/API_LIFECYCLE.md` and `docs/CAPABILITY_MATRIX.md`), all in the same commit**, structure and content aligned. Each translation must reflect the English README's actual content, not merely share its headings. Never update one language, or `README.md` alone, and leave the other language or the docs stale. `test/unit_test/headless/test_doc_counts.py` guards the quoted figures across all three READMEs (see above), but everything else — new sections, changed commands, reworded setup, the docs — is on you: read the diff against all of them before committing. ### Outstanding work goes in `Progress.md` @@ -109,7 +109,7 @@ Anything agreed but not done — deferred follow-ups, known gaps, half-delivered ### Project-specific rules -- **Exception hierarchy is flat by design** — every framework error derives from `AutoControlException` so containment boundaries (executor, background poll loops, request handlers, GUI slots) can catch the family in one `except`. Never add a sibling inheriting `Exception` directly; it silently escapes every boundary. Assertion failures (`AutoControlAssertionException`) must keep propagating through `raise_on_error=False`. +- **Exception hierarchy is flat by design** — every framework error derives from `AutoControlException` so containment boundaries (executor, background poll loops, request handlers, GUI slots) can catch the family in one `except`. Never add a sibling inheriting `Exception` directly, or only a builtin such as `RuntimeError` / `ValueError`; it silently escapes every boundary. To keep a builtin for existing callers, list both: `class XError(AutoControlException, RuntimeError)`. Assertion failures (`AutoControlAssertionException`) must keep propagating through `raise_on_error=False`. - **Fail fast** — raise the specific typed exception at the point of failure; do not swallow errors. - **Validate at boundaries** — user input, file content, network data, and JSON action commands. Reject unknown command names; `realpath` and bound user-supplied paths. - **Least privilege** — servers bind `127.0.0.1` by default; `0.0.0.0` needs an explicit, documented opt-in. @@ -172,6 +172,8 @@ Workspace rule shared by every repository under `D:\Codes` (full text: `D:\Codes - **Commit at every stage.** A stage is the smallest piece of work that leaves the repository consistent and passes this project's checks (definition of done, tests, lint): one finished `Progress.md` item, or one self-contained step of a larger one. Commit it before starting the next stage, before switching to another repository, and before the session ends. Do not leave work uncommitted across sessions; if a stage cannot be finished, commit the consistent part and record the rest in `Progress.md`. - Stage only the files that stage touched (`git add `, never `git add -A`), follow this file's commit-message rules, and never add AI attribution. - Committing is not pushing: push or open a PR only as this project's branch flow says or when asked. + - **Commit and push frequently.** After each big feature — a self-contained stage that passes this project's checks — commit and push to the remote; do not pile up a large batch of work before committing or pushing. Smaller batches collide less with other sessions, let CI catch problems earlier, and are easier to revert. Follow this project's normal branch flow (usually `dev`). + - **SonarCloud / Codacy findings.** When a PR or commit fails a SonarCloud or Codacy check, look the findings up through their APIs instead of guessing. The keys are in environment variables: `SonarCloudToken` (SonarCloud, e.g. `curl -s -u "$SonarCloudToken:" "https://sonarcloud.io/api/issues/search?componentKeys=&pullRequest=&resolved=false"`) and `CODACY_PROJECT_TOKEN` (a Codacy project token, valid only for its own project: any other repository answers "Bad credentials", so for a public repository query `https://app.codacy.com/api/v3/analysis/organizations/gh//repositories//pull-requests//issues?status=new` without a key). **Never reveal a key or any personal credential while doing so**: refer to the variables by name only, never echo or print their values, and never put them in files, commit messages, PR or issue text, logs, or any output that leaves the machine. - **`Progress.md`** (repository root, tracked) holds outstanding work only: no finished items, no history, no rules. - **`docs/updates/`** records finished work: one batch file per month (`YYYY-MM.md`), one entry per piece of work headed `## U-YYYYMMDD-NN · date · title · #tags`, and an index with query commands in `docs/updates/README.md`. When a `Progress.md` item is done, delete it and add a `#done` entry plus its index row in the same commit. - **`architecture.md`** (repository root) is the short architecture overview: layers, entry points, main flows, extension points, cross-project boundaries. Update it in the same commit whenever a change alters any of those. `architecture_explore.md` stays the detailed per-module map under its own rule in this file. diff --git a/Progress.md b/Progress.md index df4efa7e4..9aebbe833 100644 --- a/Progress.md +++ b/Progress.md @@ -26,8 +26,8 @@ | 檔案 | 行數 | 為何還沒拆 | | --- | ---: | --- | -| `utils/mcp_server/tools/_handlers_executor_bridge.py` | 1,448 | 2026-09-23 拆 `_handlers.py` 時新建。252 個純委派（中位數 3 行）：`from action_executor import _x` 再 `return _x(...)`,沒有分支。**不套用 flat data tables 條款**——那一條講的是「一個對照表或清單」,這裡是 252 個函式定義。再切下去只能照 MCP 工廠領域分（159 個領域）,那會把同一種委派散進十幾個檔,而它們之間沒有語意邊界。規則照舊:只准變短。 | -| `gui/remote_desktop/webrtc_panel.py` | 2,530 | 單一 Qt 面板,但已含連線、監視器選擇、頻寬自適應、麥克風、錄影五組互動狀態。應拆成 panel + 各控制器。 | +| `utils/mcp_server/tools/_handlers_executor_bridge.py` | 1,429 | 2026-09-23 拆 `_handlers.py` 時新建。253 個純委派（中位數 3 行）：`from action_executor import _x` 再 `return _x(...)`,沒有分支。**不套用 flat data tables 條款**——那一條講的是「一個對照表或清單」,這裡是 252 個函式定義。再切下去只能照 MCP 工廠領域分（159 個領域）,那會把同一種委派散進十幾個檔,而它們之間沒有語意邊界。規則照舊:只准變短。 | +| `gui/remote_desktop/webrtc_panel.py` | 2,527 | 單一 Qt 面板,但已含連線、監視器選擇、頻寬自適應、麥克風、錄影五組互動狀態。應拆成 panel + 各控制器。 | | `utils/accessibility/backends/windows_backend.py` | 805 | 已拆出 `windows_query.py`（193）、`windows_state.py`（98）與 `windows_reads.py`（142,2026-09-23;拆完 801,同日加焦點查詢的委派 +4）。剩下的是同一套 UIA COM 生命週期管理,再拆會把 `CoInitialize`／介面釋放的配對邏輯切散。 | **本質豁免（依 `CLAUDE.md` 的「flat data tables」條款,不算既有豁免）**: @@ -163,6 +163,12 @@ pip install --dry-run --only-binary=:all: --platform win_arm64 --python-version - **X11 預設滾動方向與 Windows／macOS 相反**：`wrapper/auto_control_mouse.py` `mouse_scroll(..., scroll_direction="scroll_down")`，正值在 X11 往下、其他平台往上，與 docstring「一份寫法各平台通用」不符。做法：預設改 `scroll_up`，或改 docstring 講清楚（重播路徑已在 U-20260924-14 明確傳 `scroll_up`）。 - **`mouse_scroll` 的 NaN 座標被悄悄夾到桌面邊緣**：`auto_control_mouse.py` 的夾限在 `_coordinate()` 驗證之前，`mouse_scroll(3, x=nan, y=100)` 移到 `(-1920, 100)` 才滾；`set_mouse_position(nan, …)` 則正確丟例外。做法：夾限前先過 `_coordinate()`。 - **座標截斷而非四捨五入**：`set_mouse_position(-0.6, 10.9)` 得到 `(0, 10)`，註解寫的是「rounded point」。做法：`int(round(value))`。 +- **`post_key` 打出三次同一字元**：`windows/window/windows_window_manage.py:347` 自己送 `WM_CHAR`，而目標的 `TranslateMessage` 又從 `WM_KEYDOWN` 與（`lParam=0` 被當成按下的）`WM_KEYUP` 各產生一次，`post_key_to_window(title, "a")` 打出 `aaa`。做法：可列印字元只送 `WM_CHAR`，其他鍵送 `WM_KEYDOWN`（`lParam = 1 | scan<<16`）與 `WM_KEYUP`（`0xC0000001 | scan<<16`）。`post_key_to_window(title, "enter")`／`"esc"` 在 Windows 丟 `unknown key name`（`wrapper/auto_control_window.py:170`），一併改走 `resolve_key_name`。 +- **焦點、顯示、z-order 失敗仍回報成功**：`windows_window_manage.py:191-229` 丟掉 `SetForegroundWindow`／`ShowWindow` 的回傳值，`window_zorder.py:50` 永遠回 `True`；Windows 的前景鎖常拒絕背景程序。做法：回傳 BOOL，`focus_window` 以 `GetForegroundWindow() == hwnd` 確認，否則丟 `AutoControlActionException`（`WindowManageBackend.bring_to_front` 已經這樣做）。 +- **列出看不見的視窗**：`windows_window_manage.py:88` 只看 `IsWindowVisible`，被 DWM cloak 的視窗（`Windows 輸入體驗`、背景的「設定」）與零面積視窗都算，`find_window` 可能選到它們。做法：略過 `DWMWA_CLOAKED` 非零與空矩形的視窗。 +- **視窗版面每次還原都偏移**：`utils/window_capture/window_capture.py:115` 存 DWM 可見框、還原時交給 `MoveWindow`（它定位的是含隱形邊框的完整矩形），每輪右移 7 px、縮小 14×7 px；最大化視窗與不同 DPI 的第二螢幕偏得更多。做法：存 `GetWindowRect`，或改用 `GetWindowPlacement`／`SetWindowPlacement`。同一檔的 snap／grid／cascade 用整個螢幕而非工作區，最底下 48 px 落在工作列下，一併改用 `SPI_GETWORKAREA`。 +- **`wait_for_window` 睡過逾時**：`wrapper/auto_control_window.py:79` 以 `poll` 整段睡，`poll=30` 就睡 30 秒，`poll=inf` 丟 `OverflowError`。做法：`clamp_poll_interval`，並只睡到截止時間。 +- **Windows 鍵表沒有標點鍵**：`plus`、`minus`、`comma`、`period`、`slash` 等沒有對應的 `VK_OEM_*`，computer use 的 `ctrl+minus` 在 Windows 失敗。做法：在 Windows 鍵表補上 `VK_OEM_PLUS`／`VK_OEM_MINUS`／`VK_OEM_COMMA`／`VK_OEM_PERIOD`／`VK_OEM_2` 等。 同一次稽核的影像與 OCR 部分也在它的路徑上（Discord bot 的 `!find_image`／`!find_text`），一併等： @@ -215,19 +221,6 @@ claim 標成需排空，丟掉下一個回覆——但 host 若根本沒回， --- -## Admin console 廣播的 `ok` 只代表 HTTP 200 - -`TODO` — 讓遠端 `/execute` 的動作失敗也能回報成失敗 - -`utils/admin/admin_client.py`（`_execute_one`）在 host 回 200 時一律 `ok: True`；遠端 `/execute` 以 -`raise_on_error=False` 執行，動作失敗只出現在結果內容裡（例如 `{"execute: [...]": "TypeError(...)"}`）。 -`utils/dag/runner.py:270` 的遠端節點因此把失敗的節點算成成功，本機路徑早已用 `raise_on_error=True` 修正過。 - -**做法**：REST `/execute` 接受並轉交 `raise_on_error`（失敗時回非 200 或 `ok: false`），admin client 與 DAG 遠端 -節點帶上它；同時更新 REST 的 OpenAPI 描述與 `architecture.md` §6（其他工具也會呼叫 `/execute`）。 - ---- - ## 全域 executor 的變數會留到下一次執行 `DECIDE` — 每次頂層執行要不要有自己的變數範圍（行為改動，維護者拍板） @@ -258,36 +251,19 @@ socket server 的執行也都用同一個 `executor`；`for_each` 的迴圈變 --- -## Computer use 改走 GA 的 `computer_toolset_20260801` +## Computer use 的預設還是 beta 的 `computer_20251124` -`TODO` — 換成新的工具形式需要改 agent 迴圈，不只是換一個 tool 型別 +`TODO` — 在 `claude-opus-5`（兩種形式都接受）上實測 GA toolset 後，把它設成所有模型的預設 -`utils/agent/backends/anthropic_computer_use.py` 現在以 beta 送 `computer_20251124`（2026-09-24 修正：原本沒帶 beta， -每個請求都被 API 拒絕）。GA 的 `computer_toolset_20260801` 不需要 beta，但每個動作是一個名稱為成員名的 `tool_use` -（`screenshot`、`left_click`…），可能一回合好幾個，每個 `tool_result` 都要帶回 `"toolset_name": "computer"`； -截圖要先縮到模型的影像上限內。Claude Opus 5.5 只接受這個形式。 - -**做法**：`_decision_from_computer_action` 改讀區塊的 `name`，一回合允許多個呼叫並逐一回覆，`_ingest_history` 帶上 -`toolset_name`；在 `claude-opus-5`（兩種都接受）上測過再換預設。 +`utils/agent/backends/anthropic_computer_use.py` 已支援 `computer_toolset_20260801`（`_computer_toolset.py`：成員名即動作、 +一回合多個呼叫逐一執行後一次回覆、每個 `tool_result` 帶 `toolset_name`、截圖縮到高解析度層級的 2576 px／4784 visual tokens 內並換算座標、`zoom` 以全解析度裁切回覆）， +`claude-opus-5-5` 自動使用它；其他模型仍預設 beta 形式，因為 toolset 只以假 client 測過、還沒對真的 API 跑過。 **附帶**：`AC_run_agent backend="openai"` 送出全部約 740 個工具，超過 OpenAI Chat Completions 的 128 個上限， 所以一定失敗——與「`AC_run_agent` 預設工具集」那一條 DECIDE 一起決定。 --- -## Idempotency 的 `release` 還沒有執行器指令 - -`TODO` — 只有 headless API，JSON 腳本與 MCP 還放不掉失敗的鍵 - -`utils/idempotency/idempotency.py` 的 `IdempotencyStore.release()` 讓工作失敗的 `in_progress` 鍵可以重跑，但 -`action_executor.py` 的 `_idempotency_begin`／`_idempotency_complete` 旁邊沒有對應的 `AC_idempotency_release`， -而執行器的具名儲存沒有 TTL，所以腳本裡工作失敗的鍵仍然永遠是 `in_progress`。 - -**做法**：加 `AC_idempotency_release`、`ac_idempotency_release` 與 Script Builder 的 **Flow** 指令，並重量指令數 -（`test_doc_counts.py` 會要求 README 三份與 `architecture_explore.md` 一起改）。 - ---- - ## MCP registry 的 server 名稱與專案網址還是舊組織 `DECIDE` — 要發布到 MCP registry 前得先定名稱，改名會影響已發布的項目 @@ -381,20 +357,6 @@ viewer 端的 `FileReceiver`（`utils/remote_desktop/file_transfer.py`）照單 --- -## Config sync 刪掉的項目會在下次同步時回來 - -`TODO` — 同步格式要加 tombstone，伺服器端與舊版客戶端的相容要一起想 - -`utils/config_sync/client.py` 的 `ConfigBucket.remove()` 直接把項目從本機 dict 拿掉；`merge_buckets` 把 -「只有遠端有」的項目照收，所以 `remove()` 之後 `sync()` 會把它從伺服器拿回來（2026-09-23 稽核重現）。 - -**做法**：`remove()` 留下 `{"deleted": True, "last_modified": now}`，merge 照一般 last-write-wins 比較， -合併完再把 tombstone 從對外的檢視濾掉；過了保留期（例如 30 天）才真正清掉。 - -**要先想清楚**：已經在跑的舊版客戶端看不懂 `deleted`，會把 tombstone 當成一般項目；伺服器是否要認得它。 - ---- - ## `AC_run_agent` 預設把每個 AC_* 指令都交給模型 `DECIDE` — 預設工具集要不要排除高風險指令 @@ -410,6 +372,42 @@ viewer 端的 `FileReceiver`（`utils/remote_desktop/file_transfer.py`）照單 **為什麼要拍板**：這會縮小既有的 agent 能力，依賴它跑 shell 的腳本會改變行為。 +實測數字（2026-09-25）：預設清單有 741 個指令，含 `AC_run_agent` 本身（模型可以遞迴開 agent）； +`backend="openai"` 超過 Chat Completions 的 128 個工具上限，現在建 backend 時就明確拒絕； +Anthropic 每一步送約 202 KB 的工具 schema、沒有 `cache_control`。拍板後一併決定上限與快取。 + + +--- + +## macOS 無法還原最小化的視窗 + +`TODO` — 需要在 macOS 上驗證，離線的 pyobjc 替身抓不到 + +`wrapper/window_backends/macos_backend.py` 的 `_info_for` 只搜「在螢幕上」的視窗，最小化的不在其中： +`minimize(77)` 成功後 `list_windows` 看不到它、`restore(77)` 丟「請授權 Accessibility」（即使已授權）。 +Windows 的 `list_windows` 則包含最小化視窗。 + +**做法**：`_info_for` 改用 `CGWindowListCopyWindowInfo(kCGWindowListOptionIncludingWindow, window_id)`； +在 macOS CI（TCC 已授權）加一個真的最小化再還原的測試。 + +--- + +## Agent 的截圖修剪會改寫較早的回合 + +`TODO` — 需要付費實機跑一次多步驟任務驗證，不能只靠離線測試 + +`utils/agent/backends/base.py` 的 `prune_old_screenshots` 每一步把較舊的截圖換成文字，改的是已送出過的訊息。 +Claude Fable 5.1 與 Opus 5.5 的 thinking 區塊綁定它之前的整段對話，2026-08-31 之後建立的帳號會直接回 400 +（"block is bound to a different conversation"），約在第 4 步中斷；其他模型則是每一步都讓 prompt cache 失效。 +不修剪也不行：每步重送全部截圖會超過 32 MB 的請求上限。 + +**做法（擇一，依 claude-api 文件的 append-only 對照表）**：用戶端「簡單壓縮」——截圖數超過上限時，以一則摘要 +（目標、已執行的動作）加最新截圖開新對話，不重播舊回合；或送 +`thinking.block_binding.prefix_mismatch_behavior: "drop_block"`（beta `thinking-binding-controls-2026-08-01`）， +讓被改到的 thinking 區塊被丟棄而不是 400。伺服器端 tool-result clearing 不會縮小請求本身，擋不住 32 MB。 + +**要動的地方**：`anthropic.py`、`anthropic_computer_use.py`（兩條路徑）呼叫 `prune_old_screenshots` 之處； +OpenAI 後端沒有這個綁定，照舊。 --- @@ -426,7 +424,23 @@ MCP 工具的檔案參數（`path`、`file_path`、`db`、`image_path`、`golden **做法**：在 `utils/mcp_server/tools/_factories.py` 的 schema 裡把真正是檔案路徑的屬性標上 `"format": "path"`（不能照名字判斷：`ac_json_query` 的 `path` 是 JSON 路徑，`template`／`source`／ `target` 有時是檔案有時不是），`server.py` 的 `_prepare_tool_call` 在設定了根目錄時先 `realpath` -再檢查是否落在根目錄內，不在就回 `-32602`。 +再檢查是否落在根目錄內，不在就回工具執行錯誤（`isError`，和其他參數驗證失敗一樣）。 **為什麼要拍板**：根目錄從哪來（新的環境變數、沿用 `roots/list`、或兩者），唯讀模式要不要預設開啟； 預設開啟會讓現有讀取工作區外檔案的用法失效。 + +--- + +## 遠端桌面的 viewer 槽位由各面板共用 + +`DECIDE` — 要改 `registry` 的擁有權模型 + +`utils/remote_desktop/registry.py` 的 TCP 與 WS viewer 各只有一個槽位，快速連線（`gui/remote_desktop/connection_screen.py`）、 +舊式 viewer 分頁（`viewer_panel.py`）與 `AC_remote_connect` 都寫同一格。每一方連線前先 `registry.disconnect_viewer()`， +於是在一邊連線會切斷另一邊的連線，被切斷的面板卻不知道：它的彈出視窗仍停在最後一格畫面， +「中斷」按鈕則會切斷別人的連線。快速連線的「開始被遠端」也一樣會停掉主機分頁開的 host。 + +**做法**：registry 記錄每個 viewer／host 由誰開的（owner token），`disconnect_*` 只在 owner 相符時動作； +被別人取代時通知原本的面板收掉自己的視窗。或是反過來讓每個面板持有自己的 viewer，不經 registry。 + +**為什麼要拍板**：`AC_remote_*` 指令與 MCP 工具依賴「registry 裡就是那一個 viewer」，改成多槽位要一起改它們的語意。 diff --git a/README.md b/README.md index d0b515788..fa7e40b3f 100644 --- a/README.md +++ b/README.md @@ -22,7 +22,7 @@ from JSON files / CLI / servers, and a **GUI tab**. Nothing is GUI-only. - **One API, seven platforms.** `wrapper/platform_wrapper.py` picks the backend at import time; your script does not change between Windows, macOS, X11, and Wayland. -- **Scriptable without Python.** 774 `AC_*` commands cover the whole feature set, so a +- **Scriptable without Python.** 775 `AC_*` commands cover the whole feature set, so a JSON file can do anything the library can — including loops, branches, try/catch, macros, and variables. - **Headless by default.** `import je_auto_control` never loads Qt. The GUI is an @@ -154,7 +154,7 @@ desktop app; tab commands live in the window's **Actions** menu. | Natural-language planner | `plan_actions`, `run_from_description` | `AC_llm_plan` | LLM Planner | | Computer-use agent | `AgentLoop`, `run_agent` | `AC_run_agent` | Computer Use | | Record & replay | `record`, `stop_record` | `AC_record`, `AC_stop_record` | Record | -| JSON scripting | `execute_action`, `execute_files` | all 774 commands | Script, Script Builder | +| JSON scripting | `execute_action`, `execute_files` | all 775 commands | Script, Script Builder | | Variables & flow control | `execute_action_with_vars` | `AC_set_var`, `AC_loop`, `AC_for_each`, `AC_try`, `AC_retry` | Variables | | Data-driven runs | — | `AC_for_each_row` (CSV / JSON / SQLite / Excel) | Data Sources | | Assertions | `assert_text`, `assert_image` | `AC_assert_text` + 20 more | Assertions | @@ -206,7 +206,7 @@ still goes on to the end), so a CI step fails with it. The legacy | Surface | Start it with | Notes | |---|---|---| -| **MCP server** | `je_auto_control_mcp` (stdio) or `AC_start_mcp_http_server` | 677 tools for Claude Desktop / Claude Code / custom tool loops. Bearer auth, TLS, audit log, rate limit, plugin hot-reload, CI fake backend. | +| **MCP server** | `je_auto_control_mcp` (stdio) or `AC_start_mcp_http_server` | 678 tools for Claude Desktop / Claude Code / custom tool loops. Speaks the stateless MCP 2026-07-28 beside the `initialize`-based revisions. Bearer auth, TLS, audit log, rate limit, plugin hot-reload, CI fake backend. | | **REST API** | `je_auto_control start-rest` | Bearer token, per-IP rate limit + lockout, SQLite audit hook, `/metrics`, `/openapi.json`, `/docs` Swagger UI, `/dashboard`. | | **TCP socket server** | `je_auto_control start-server` | Newline-framed JSON action lists. Binds `127.0.0.1` by default. | | **pytest plugin** | installed automatically | Fixtures plus a Gherkin step library for pytest-bdd / behave. | diff --git a/README/README_zh-CN.md b/README/README_zh-CN.md index 879b4a6cf..69e548699 100644 --- a/README/README_zh-CN.md +++ b/README/README_zh-CN.md @@ -20,7 +20,7 @@ - **一套 API，七个平台。** `wrapper/platform_wrapper.py` 在导入时挑选后端；同一份脚本在 Windows、macOS、X11 与 Wayland 上都不需要改写。 -- **不写 Python 也能脚本化。** 774 个 `AC_*` 命令覆盖全部功能，因此一个 JSON 文件能做到库 +- **不写 Python 也能脚本化。** 775 个 `AC_*` 命令覆盖全部功能，因此一个 JSON 文件能做到库 能做的任何事——包含循环、分支、try/catch、宏与变量。 - **默认无头运行。** `import je_auto_control` 绝不会加载 Qt。GUI 是可选包，包在同一个无头内核之外。 - **四种定位方式。** 模板匹配、OCR、无障碍树、视觉语言模型——可通过锚点定位器与自愈回退串接组合。 @@ -142,7 +142,7 @@ python -c "import je_auto_control; je_auto_control.start_autocontrol_gui()" | 自然语言规划 | `plan_actions`、`run_from_description` | `AC_llm_plan` | LLM Planner | | Computer-use agent | `AgentLoop`、`run_agent` | `AC_run_agent` | Computer Use | | 录制与回放 | `record`、`stop_record` | `AC_record`、`AC_stop_record` | Record | -| JSON 脚本 | `execute_action`、`execute_files` | 全部 774 个命令 | Script、Script Builder | +| JSON 脚本 | `execute_action`、`execute_files` | 全部 775 个命令 | Script、Script Builder | | 变量与流程控制 | `execute_action_with_vars` | `AC_set_var`、`AC_loop`、`AC_for_each`、`AC_try`、`AC_retry` | Variables | | 数据驱动执行 | — | `AC_for_each_row`（CSV／JSON／SQLite／Excel） | Data Sources | | 断言 | `assert_text`、`assert_image` | `AC_assert_text` 等 21 个 | Assertions | @@ -191,7 +191,7 @@ je_auto_control version | 接口 | 启动方式 | 说明 | |---|---|---| -| **MCP 服务器** | `je_auto_control_mcp`（stdio）或 `AC_start_mcp_http_server` | 677 个工具，供 Claude Desktop／Claude Code／自定义 tool loop 使用。Bearer 认证、TLS、审计日志、限流、插件热重载、CI 假后端。 | +| **MCP 服务器** | `je_auto_control_mcp`（stdio）或 `AC_start_mcp_http_server` | 678 个工具，供 Claude Desktop／Claude Code／自定义 tool loop 使用。除了以 `initialize` 握手的各版协议，也支持无状态的 MCP 2026-07-28。Bearer 认证、TLS、审计日志、限流、插件热重载、CI 假后端。 | | **REST API** | `je_auto_control start-rest` | Bearer token、按 IP 限流与锁定、SQLite 审计 hook、`/metrics`、`/openapi.json`、`/docs` Swagger UI、`/dashboard`。 | | **TCP socket 服务器** | `je_auto_control start-server` | 以换行分隔的 JSON 动作列表。默认绑定 `127.0.0.1`。 | | **pytest 插件** | 安装后自动生效 | 提供 fixture 与供 pytest-bdd／behave 使用的 Gherkin step library。 | diff --git a/README/README_zh-TW.md b/README/README_zh-TW.md index 4bd926718..61af44442 100644 --- a/README/README_zh-TW.md +++ b/README/README_zh-TW.md @@ -20,7 +20,7 @@ - **一套 API，七個平台。** `wrapper/platform_wrapper.py` 在匯入時挑選後端；同一份腳本在 Windows、macOS、X11 與 Wayland 上都不需要改寫。 -- **不寫 Python 也能腳本化。** 774 個 `AC_*` 指令涵蓋全部功能，因此一個 JSON 檔能做到函式庫 +- **不寫 Python 也能腳本化。** 775 個 `AC_*` 指令涵蓋全部功能，因此一個 JSON 檔能做到函式庫 能做的任何事——包含迴圈、分支、try/catch、巨集與變數。 - **預設無頭執行。** `import je_auto_control` 絕不會載入 Qt。GUI 是選用套件，包在同一個無頭核心之外。 - **四種定位方式。** 樣板比對、OCR、無障礙樹、視覺語言模型——可透過錨點定位器與自癒後備串接組合。 @@ -142,7 +142,7 @@ python -c "import je_auto_control; je_auto_control.start_autocontrol_gui()" | 自然語言規劃 | `plan_actions`、`run_from_description` | `AC_llm_plan` | LLM Planner | | Computer-use agent | `AgentLoop`、`run_agent` | `AC_run_agent` | Computer Use | | 錄製與重播 | `record`、`stop_record` | `AC_record`、`AC_stop_record` | Record | -| JSON 腳本 | `execute_action`、`execute_files` | 全部 774 個指令 | Script、Script Builder | +| JSON 腳本 | `execute_action`、`execute_files` | 全部 775 個指令 | Script、Script Builder | | 變數與流程控制 | `execute_action_with_vars` | `AC_set_var`、`AC_loop`、`AC_for_each`、`AC_try`、`AC_retry` | Variables | | 資料驅動執行 | — | `AC_for_each_row`（CSV／JSON／SQLite／Excel） | Data Sources | | 斷言 | `assert_text`、`assert_image` | `AC_assert_text` 等 21 個 | Assertions | @@ -191,7 +191,7 @@ je_auto_control version | 介面 | 啟動方式 | 說明 | |---|---|---| -| **MCP 伺服器** | `je_auto_control_mcp`（stdio）或 `AC_start_mcp_http_server` | 677 個工具，供 Claude Desktop／Claude Code／自訂 tool loop 使用。Bearer 驗證、TLS、稽核記錄、限流、外掛熱重載、CI 假後端。 | +| **MCP 伺服器** | `je_auto_control_mcp`（stdio）或 `AC_start_mcp_http_server` | 678 個工具，供 Claude Desktop／Claude Code／自訂 tool loop 使用。除了以 `initialize` 握手的各版協定，也支援無狀態的 MCP 2026-07-28。Bearer 驗證、TLS、稽核記錄、限流、外掛熱重載、CI 假後端。 | | **REST API** | `je_auto_control start-rest` | Bearer token、逐 IP 限流與鎖定、SQLite 稽核 hook、`/metrics`、`/openapi.json`、`/docs` Swagger UI、`/dashboard`。 | | **TCP socket 伺服器** | `je_auto_control start-server` | 以換行分隔的 JSON 動作清單。預設綁 `127.0.0.1`。 | | **pytest 外掛** | 安裝後自動生效 | 提供 fixture 與供 pytest-bdd／behave 使用的 Gherkin step library。 | diff --git a/architecture.md b/architecture.md index b97e7e460..fb56f425a 100644 --- a/architecture.md +++ b/architecture.md @@ -142,6 +142,11 @@ new, add it to both. `je_web_runner` is not a declared dependency; when it is missing the bridge raises `WebRunnerBridgeError`. Moving that WebRunner module breaks the bridge. +**Wire contract between AutoControl versions:** the Admin Console and DAG remote nodes drive other hosts through +REST `POST /execute` (`{"actions": [...], "raise_on_error": bool}`), and those hosts may run an older release. +A new body field must be optional and safe to ignore — an old host drops `raise_on_error` and answers the pre-flag +`{"result": ...}`, which the client still reads as `ok: true`. + **Import-time contracts** - `import je_auto_control` must not load PySide6; the GUI window is imported only inside `start_autocontrol_gui()`. diff --git a/architecture_explore.md b/architecture_explore.md index 268c83f8a..63987baf6 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -6,7 +6,7 @@ > 擷取每個模組的 docstring 與頂層公開名稱；統計數字取自實際檔案，非估算。 > 指令數與公開 API 數以 `executor.known_commands()` 與 `je_auto_control.__all__` 在工作樹上實測取得。 > -> **掃描時間**：2026-09-22　**版本**：`pyproject.toml` version `0.0.221`　**分支**：`feat/coverage-to-80` +> **掃描時間**：2026-09-25　**版本**：`pyproject.toml` version `0.0.221`　**分支**：`feat/coverage-to-80` --- @@ -19,13 +19,13 @@ iOS（WebDriverAgent）。核心能力是滑鼠／鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | -| Python 模組總數（含周邊子專案） | 1,049 | -| 程式碼總行數 | 148,925 | +| Python 模組總數（含周邊子專案） | 1,059 | +| 程式碼總行數 | 154,389 | | `je_auto_control/utils/` 子套件數 | 310 | -| `AC_*` 動作指令數（`known_commands()` 實測） | 774 | -| 套件門面 `__all__` 公開名稱數 | 1,241 | +| `AC_*` 動作指令數（`known_commands()` 實測） | 775 | +| 套件門面 `__all__` 公開名稱數 | 1,244 | | GUI 分頁數（`main_widget` 註冊） | 48 | -| MCP 工具數（`build_default_tool_registry()` 實測） | 677 | +| MCP 工具數（`build_default_tool_registry()` 實測） | 678 | | `test_*.py` 測試檔／測試函式 | 478 / 4,654 | | 範例腳本 | 27 | @@ -49,7 +49,7 @@ USB/IP 協定、Prometheus 指標），以維持這條輕相依基線。 │ 全部只呼叫下面這一層，不含業務邏輯 ┌───────────────────────────────▼──────────────────────────────────────────┐ │ 執行核心 Execution Core │ -│ utils/executor/action_executor.py ── Executor.event_dict（774 個 AC_*） │ +│ utils/executor/action_executor.py ── Executor.event_dict（775 個 AC_*） │ │ utils/executor/flow_control.py ── 34 個區塊指令（迴圈/分支/try/巨集） │ │ utils/script_vars ── ${var} 插值 │ utils/json ── action 檔 I/O │ └───────────────────────────────┬──────────────────────────────────────────┘ @@ -156,11 +156,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | --- | ---: | --- | | `je_auto_control/__init__.py` | 1,970 | **套件門面**。集中匯入並再匯出 1,200 個公開名稱，以功能區塊註解分段（callback／exception／executor／a11y／vision／clipboard…）。 | | `je_auto_control/__main__.py` | 87 | 舊版 argparse 進入點：`-e` 執行單檔、`-d` 執行整個目錄、`--execute_str` 執行 JSON 字串、`-c` 建立專案。 | -| `je_auto_control/cli.py` | 338 | **主 CLI**（`je_auto_control` console script）。子命令：`run`（含 `--var`／`--dry-run`）、`validate`／`lint`、`list-commands`、`fmt`、`record`、`codegen`、`failure-bundle`、`list-jobs`、`start-server`、`start-rest`、`version`。所有子命令延遲匯入，確保不碰 Qt。 | +| `je_auto_control/cli.py` | 353 | **主 CLI**（`je_auto_control` console script）。子命令：`run`（含 `--var`／`--dry-run`）、`validate`／`lint`、`list-commands`、`fmt`、`record`、`codegen`、`failure-bundle`、`list-jobs`、`start-server`、`start-rest`、`version`。所有子命令延遲匯入，確保不碰 Qt。 | | `je_auto_control/api/__init__.py` | 22 | 版本化整合進入點。 | | `je_auto_control/api/core.py` | 19 | **穩定無頭 API 門面**：只暴露 `execute_action`、`execute_action_with_vars`、`generate_code`、`run_diagnostics`、`create_failure_bundle`、`failure_bundle_on_error`、`FailureBundleOptions`。mypy 型別契約以此為起點，現已擴到整包（見「設定基線」）。 | +| `je_auto_control/utils/cli_output.py` | 44 | 本套件命令列工具與 stdio 伺服器的標準串流：不論碼頁一律 UTF-8。 | | `je_auto_control/utils/deprecation.py` | 35 | 公開 API 的一致性棄用警告。 | -| `je_auto_control/utils/http_headers.py` | 115 | 入站 HTTP 標頭與 chunked 內文的共用防禦式解析。 | +| `je_auto_control/utils/http_headers.py` | 189 | 本套件各伺服器共用的防禦式輔助：標頭、內文、回應與日誌。 | | `je_auto_control/utils/sqlite_support.py` | 112 | 選用標準函式庫 `sqlite3` 的取用點：`require_sqlite3()`／`sqlite3_available()`／`SQLITE_ERRORS`。十個以 SQLite 存放狀態的子系統都經由這裡，所以 FreeBSD 這種把 `sqlite3` 另外包成 `databases/py-sqlite3` 的 Python 仍然 import 得起門面。 | | `je_auto_control/utils/timeouts.py` | 33 | 把使用者給的逾時換成截止時間：`deadline_after()` 拒絕 NaN（`json` 接受它，而 `clock() >= NaN` 永遠不成立，輪詢迴圈會永遠跑下去），負值與無限大維持原意；`clamp_poll_interval()` 把背景迴圈的輪詢間隔夾在 0.05 秒到 1 小時之間（`Event.wait(inf)` 在 Windows 會丟 `OverflowError`）。 | @@ -182,11 +183,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `wrapper/auto_control_image.py` | 83 | 影像 API：`locate_all_image`、`locate_image_center`、`locate_and_click`。 | | `wrapper/auto_control_record.py` | 124 | 錄製 API：`record`／`stop_record`／`record_to_json`（支援 stop event 與逾時）。 | | `wrapper/auto_control_window.py` | 287 | 視窗管理門面：列舉、尋找、聚焦、等待、關閉、顯示狀態、幾何、所屬行程 PID、依行程列舉／最小化視窗、不搶焦點的投遞式輸入（目前僅 Windows 實作）。 | -| `wrapper/window_backends/` | 988 | 視窗管理的平台縫（`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`）。放在 `wrapper/` 而不是 `utils/`，因為它必須 import `windows/`、`linux_with_x11/`、`osx/`，而 `utils/` 在分層上在那三者之上。 | +| `wrapper/window_backends/` | 1,004 | 視窗管理的平台縫（`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`）。放在 `wrapper/` 而不是 `utils/`，因為它必須 import `windows/`、`linux_with_x11/`、`osx/`，而 `utils/` 在分層上在那三者之上。 | ### 5.3 平台後端 -#### Windows（`windows/`，23 檔／1,957 行） +#### Windows（`windows/`，23 檔／1,959 行） | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -200,39 +201,39 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `screen/win32_screen.py` | 95 | 螢幕尺寸與像素讀取。**每支 Win32 函式都明寫 argtypes/restype**（HDC 是指標寬度，走預設的 c_int 會截斷，錯誤會沉默地擴散到 GetPixel／ReleaseDC），並持有自己的 user32／gdi32 handle。import 時呼叫 `SetProcessDPIAware()`——**行程層級且不可還原**，實體↔邏輯座標換算請走 `utils/monitor_layout`。 | | `window/windows_window_manage.py` | 374 | 視窗列舉／聚焦／關閉／最小化／幾何／所屬行程 PID／投遞式輸入（`auto_control_window` 的實作）。**每支 Win32 函式都明寫 argtypes/restype**，並持有自己的 user32 handle，避免把原型外溢到別的模組；hwnd 一律是 int。 | | `message/window_message.py` | 97 | 直接對視窗送 `WM_*` 訊息（背景輸入）。 | -| `interception/_dll.py` | 230 | `interception.dll` 的延遲 ctypes 載入與結構定義。 | +| `interception/_dll.py` | 232 | `interception.dll` 的延遲 ctypes 載入與結構定義。 | | `interception/keyboard.py` | 70 | 經 Interception 驅動的鍵盤輸入（繞過部分反自動化偵測）。 | | `interception/mouse.py` | 160 | 經 Interception 驅動的滑鼠輸入。 | -#### macOS（`osx/`，17 檔／919 行） +#### macOS（`osx/`，17 檔／925 行） | 模組 | 行數 | 職責 | | --- | ---: | --- | | `core/utils/osx_vk.py` | 113 | macOS 虛擬鍵碼表。 | -| `mouse/osx_mouse.py` | 137 | Quartz `CGEvent` 滑鼠事件。 | -| `keyboard/osx_keyboard.py` | 137 | Quartz 鍵盤事件。 | +| `mouse/osx_mouse.py` | 143 | Quartz `CGEvent` 滑鼠事件。 | +| `keyboard/osx_keyboard.py` | 144 | Quartz 鍵盤事件。 | | `keyboard/osx_keyboard_check.py` | 24 | 按鍵狀態查詢。 | -| `listener/osx_listener.py` | 257 | 專屬執行緒上的 listen-only `CGEventTap`＋自己的 `CFRunLoopRunInMode` 切片；不在 import 時建 `NSApplication`，也不用會卡住呼叫緒的 `AppHelper.runEventLoop()`。修飾鍵由 `flagsChanged` 的旗標還原成 press／release，座標取 `CGEventGetLocation`（左上原點，與重播送出的座標同一空間）。 | +| `listener/osx_listener.py` | 261 | 專屬執行緒上的 listen-only `CGEventTap`＋自己的 `CFRunLoopRunInMode` 切片；不在 import 時建 `NSApplication`，也不用會卡住呼叫緒的 `AppHelper.runEventLoop()`。修飾鍵由 `flagsChanged` 的旗標還原成 press／release，座標取 `CGEventGetLocation`（左上原點，與重播送出的座標同一空間）。 | | `record/osx_record.py` | 41 | 錄製。捕捉後的整形（舊版按下事件 Queue、時間軸、只錄滑鼠／只錄鍵盤）走共用的 `utils/input_macro/recorder_base.py`。 | | `screen/osx_screen.py` | 143 | 螢幕擷取與尺寸（含 Retina 座標處理）。 | -| `pid/pid_control.py` | 64 | 以 PID 操作應用程式。 | +| `pid/pid_control.py` | 53 | 以 PID 操作應用程式。 | -#### Linux X11（`linux_with_x11/`，19 檔／1,236 行） +#### Linux X11（`linux_with_x11/`，19 檔／1,281 行） | 模組 | 行數 | 職責 | | --- | ---: | --- | | `core/utils/x11_linux_display.py` | 16 | 共用 `Xlib.display.Display` 實例。 | | `core/utils/x11_linux_vk.py` | 199 | X11 keysym 對照表。 | -| `mouse/x11_linux_mouse_control.py` | 155 | XTest 滑鼠事件。 | -| `keyboard/x11_linux_keyboard_control.py` | 88 | XTest 鍵盤事件。 | +| `mouse/x11_linux_mouse_control.py` | 158 | XTest 滑鼠事件。 | +| `keyboard/x11_linux_keyboard_control.py` | 99 | XTest 鍵盤事件。 | | `listener/x11_linux_listener.py` | 208 | XRecord 監聽。 | -| `record/x11_linux_record.py` | 76 | 錄製。 | +| `record/x11_linux_record.py` | 78 | 錄製。 | | `screen/x11_linux_screen.py` | 65 | 螢幕尺寸與擷取。 | -| `uinput/_device.py` | 244 | `/dev/uinput` 封裝（核心層輸入，選用）。 | -| `uinput/keyboard.py` | 32 | uinput 鍵盤後端，介面與 X11 版一致。 | -| `uinput/mouse.py` | 115 | uinput 滑鼠後端。 | +| `uinput/_device.py` | 246 | `/dev/uinput` 封裝（核心層輸入，選用）。 | +| `uinput/keyboard.py` | 44 | uinput 鍵盤後端，介面與 X11 版一致。 | +| `uinput/mouse.py` | 130 | uinput 滑鼠後端。 | -#### Linux Wayland（`linux_wayland/`，17 檔／2,870 行） +#### Linux Wayland（`linux_wayland/`，17 檔／2,921 行） | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -242,25 +243,25 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `_dbus_client.py` | 24 | 只用標準函式庫的 D-Bus session bus 客戶端（連線／認證／`Hello`／`AddMatch`／一次方法呼叫／等訊號）。portal 的回應是**指名送給發出呼叫的那條連線**,所以訂閱與呼叫必須同一條連線——這是 `gdbus monitor` + `gdbus call` 兩個行程做不到的事。 | | `_select_input.py` | 85 | 決定使用原生 libei 或 CLI shim；`active_backend()` 是 keyboard／mouse 的唯一入口，`emitted()` 讓被拒絕的單次發送退回 CLI。 | | `_layout.py` | 83 | 版面原點的共用查詢。擷取與輸入不是同一個座標空間,差的就是這個原點:libei 的 region offset 是 `uint32`（描述不了負原點）,`ydotool mousemove --absolute` 的原點是合成器夾取的那個角落——兩條路都要減掉它,所以放在這裡而不是各自複製。讀數快取一秒——擷取那一側刻意不快取,但 ydotool 每次絕對移動都會問,不快取等於每次移動多開一個 `wlr-randr` 行程。 | -| `oeffis.py` | 196 | liboeffis 綁定：跑完 RemoteDesktop portal 交握，交出 EIS fd。 | -| `libei.py` | 632 | libei 綁定與完整握手（seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame）。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕（libei 對這種移動是靜靜丟掉的）。 | +| `oeffis.py` | 197 | liboeffis 綁定：跑完 RemoteDesktop portal 交握，交出 EIS fd。 | +| `libei.py` | 655 | libei 綁定與完整握手（seat 綁定能力 → 由事件取得 device → start_emulating → 每次發送後 frame）。另負責絕對指標的座標空間:讀回裝置的 region,把版面座標映射進去,沒有任何 region 涵蓋就拒絕（libei 對這種移動是靜靜丟掉的）。 | | `mouse.py` | 384 | 滑鼠後端：移動、按鈕與捲動都 libei 優先，退回 ydotool；送往 libei 時垂直捲動軸取負（kernel `REL_WHEEL` 與 `wl_pointer` 正負號相反）。退到 ydotool 的絕對移動會先減掉版面原點（`--absolute` 是相對於版面左上角,不是版面座標的 `(0, 0)`),並依 `pointer_accel_mode()` 處理指標加速度——倍率讀不回來,只有操作者知道,所以由 `JE_AUTOCONTROL_WAYLAND_POINTER_ACCEL` 宣告:未設定＝每個行程警告一次後照送、`flat`＝已關掉加速度故靜靜送出、`strict`＝拒絕這次移動。 | | `keyboard.py` | 173 | 鍵盤後端：libei 優先，退回 ydotool／wtype。 | | `keymap.py` | 155 | 友善鍵名 → evdev key code。 | -| `capture.py` | 241 | 擷取分層：操作者自訂指令 → grim → gnome-screenshot → spectacle → portal。 | +| `capture.py` | 246 | 擷取分層：操作者自訂指令 → grim → gnome-screenshot → spectacle → portal。 | | `portal.py` | 207 | `org.freedesktop.portal.Screenshot` 最後備援,經 `_dbus_client` 直接講 D-Bus（不再需要安裝 `gdbus`,只要有 session bus）。 | -| `screen.py` | 274 | 螢幕後端；發布 `grab_image` 與 `layout_origin`（擷取畫面左上角的版面座標，有螢幕在主螢幕左側／上方時為負），全框架的擷取都經由它。 | +| `screen.py` | 296 | 螢幕後端；發布 `grab_image` 與 `layout_origin`（擷取畫面左上角的版面座標，有螢幕在主螢幕左側／上方時為負），全框架的擷取都經由它。 | | `listener.py` / `record.py` | 48 / 34 | 監聽與錄製 stub（Wayland 限制）。 | #### 行動裝置 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `android/adb_client.py` | 197 | `adb` CLI 的薄封裝。 | -| `android/client.py` | 127 | `uiautomator2.Device` 的延遲封裝。 | -| `android/find.py` | 107 | uiautomator2 widget 樹的元素查詢。 | -| `ios/client.py` | 122 | `facebook-wda`（WebDriverAgent）封裝。 | -| `ios/find.py` | 93 | XCUITest 無障礙查詢。 | +| `android/adb_client.py` | 213 | `adb` CLI 的薄封裝。 | +| `android/client.py` | 129 | `uiautomator2.Device` 的延遲封裝。 | +| `android/find.py` | 108 | uiautomator2 widget 樹的元素查詢。 | +| `ios/client.py` | 124 | `facebook-wda`（WebDriverAgent）封裝。 | +| `ios/find.py` | 94 | XCUITest 無障礙查詢。 | | `ios/input.py` | 51 | iOS 觸控與按鍵原語。 | | `ios/screen.py` | 34 | iOS 裝置螢幕擷取與尺寸。 | @@ -271,77 +272,77 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.1 執行引擎與腳本資產 -> 24 個套件、約 14,247 行。 +> 24 個套件、約 14,553 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/action_lint/` | 369 | action 檔 linter 與 JSON Schema 產生器（CI 用 `python -m` 進入點） | +| `utils/action_lint/` | 429 | action 檔 linter 與 JSON Schema 產生器（CI 用 `python -m` 進入點） | | `utils/action_signing/` | 380 | action 檔 HMAC-SHA256 簽章與 Fernet 加密，`execute_files` 會強制驗簽 | -| `utils/checkpoint/` | 120 | 流程檢查點與續跑，讓長 action list 具持久性 | -| `utils/codegen/` | 255 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | -| `utils/dag/` | 492 | 跨主機 DAG 編排器（圖模型 + runner） | +| `utils/checkpoint/` | 129 | 流程檢查點與續跑，讓長 action list 具持久性 | +| `utils/codegen/` | 294 | 由 action list 產生可執行的 pytest / python / robot 測試碼 | +| `utils/dag/` | 536 | 跨主機 DAG 編排器（圖模型 + runner） | | `utils/decision_table/` | 112 | DMN 風格決策表：規則 + 命中策略，把分支外部化 | | `utils/deterministic/` | 116 | 決定性執行控制：固定亂數種子 + 凍結時鐘 | -| `utils/executor/` | 9,412 | **核心**。`Executor` 指令分派表（774 個 `AC_*`）、參數插值、乾跑、逐步 callback；`flow_control` 提供 34 個區塊指令（迴圈／分支／try／巨集／變數） | +| `utils/executor/` | 9,504 | **核心**。`Executor` 指令分派表（775 個 `AC_*`）、參數插值、乾跑、逐步 callback；`flow_control` 提供 34 個區塊指令（迴圈／分支／try／巨集／變數） | | `utils/flow_debugger/` | 155 | action list 的單步除錯器與追蹤器 | | `utils/input_macro/` | 451 | 定時輸入事件：錄製結果的整形（`timeline`／`InputRecorder`，Windows 與 macOS 共用）、重播與宣告式輸入序列 DSL | | `utils/json/` | 99 | action JSON 檔讀寫與正規化格式化（`fmt --check` 的後端） | -| `utils/json_store/` | 241 | JSON 字典檔持久化的共用小工具（內部管線） | +| `utils/json_store/` | 271 | JSON 字典檔持久化的共用小工具（內部管線） | | `utils/loop_guard/` | 158 | 機械式卡死迴圈偵測（agent loop 用） | -| `utils/plugin_loader/` | 142 | 掃描外部 Python 外掛目錄並註冊其 `AC_` callable | +| `utils/plugin_loader/` | 147 | 掃描外部 Python 外掛目錄並註冊其 `AC_` callable | | `utils/plugin_sdk/` | 80 | 外掛 SDK：透過 entry points 發佈／載入第三方 `AC_*` 指令 | -| `utils/project/` | 186 | 專案腳手架：建立目錄結構與範本 action 檔 | +| `utils/project/` | 187 | 專案腳手架：建立目錄結構與範本 action 檔 | | `utils/recording_edit/` | 150 | 不重錄的前提下裁切／過濾／縮放已錄製的 action list | | `utils/saga/` | 100 | Saga 協調器：失敗時以 LIFO 補償動作回滾 | -| `utils/script_vars/` | 197 | 執行期變數作用域與 `${var}` / `${secrets.*}` 插值 | +| `utils/script_vars/` | 211 | 執行期變數作用域與 `${var}` / `${secrets.*}` 插值 | | `utils/skill_library/` | 115 | 具名可重用 action 序列（skill）的持久化倉庫 | | `utils/state_machine/` | 268 | 宣告式有限狀態機驅動 action JSON | | `utils/stubs/` | 311 | 為 `AC_*` 指令面產生型別 stub | | `utils/test_record/` | 70 | 全域測試紀錄單例，記錄每個動作的參數與例外 | -| `utils/work_queue/` | 268 | 交易式工作佇列（dispatcher／performer），支撐大量批次執行 | +| `utils/work_queue/` | 280 | 交易式工作佇列（dispatcher／performer），支撐大量批次執行 | ### 5.4.2 框架基礎設施 -> 14 個套件、約 2,922 行。 +> 14 個套件、約 3,014 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/callback/` | 204 | Observer 模式：`callback_executor` 以字串名觸發功能，執行後呼叫回呼 | +| `utils/callback/` | 209 | Observer 模式：`callback_executor` 以字串名觸發功能，執行後呼叫回呼 | | `utils/config_bundle/` | 424 | 使用者設定的單檔匯出／匯入 | | `utils/critical_exit/` | 132 | 監看緊急停止鍵的守護執行緒，用於中止失控腳本 | | `utils/diagnostics/` | 330 | 跨子系統的「一切正常嗎」健檢，附 `python -m` 進入點 | | `utils/dbus_client/` | 703 | 只用標準函式庫的 D-Bus session bus 客戶端。原本在 `linux_wayland/` 為 portal 交握而寫，AT-SPI 無障礙後端成為第二個使用者後搬到這裡（`utils/` 在分層上在各 OS 套件之上） | -| `utils/exception/` | 212 | **例外階層根**。所有錯誤繼承 `AutoControlException`，加上集中式錯誤訊息字串（`exception_tags`） | +| `utils/exception/` | 213 | **例外階層根**。所有錯誤繼承 `AutoControlException`，加上集中式錯誤訊息字串（`exception_tags`） | | `utils/failure_bundle/` | 219 | 可攜、已遮蔽的失敗診斷 ZIP（截圖 + 診斷 + log 尾段） | | `utils/file_process/` | 40 | 目錄檔案列舉（`execute_dir` 的後端） | -| `utils/logging/` | 161 | `autocontrol_logger` 單例 + 家目錄共用記錄檔 handler（`JE_AUTOCONTROL_LOG_FILE` 可改） | +| `utils/logging/` | 168 | `autocontrol_logger` 單例 + 家目錄共用記錄檔 handler（`JE_AUTOCONTROL_LOG_FILE` 可改） | | `utils/package_manager/` | 101 | 動態載入套件並把 executor 注入其中 | -| `utils/path_guard/` | 99 | 命令列傳入路徑的正規化與邊界檢查（防路徑穿越） | +| `utils/path_guard/` | 114 | 命令列傳入路徑的正規化與邊界檢查（防路徑穿越） | | `utils/platform_id/` | 62 | 作業系統家族的單一判定點。`sys.platform` 原本在一百多處跟字面清單比對，而那些清單都沒有 BSD；`is_x11_unix()` 問的是「這是不是 X11 unix」，這才是守衛一直想問的問題 | -| `utils/shell_process/` | 194 | `ShellManager`：以 argv list 執行外部命令（禁用 `shell=True`） | -| `utils/start_exe/` | 41 | 啟動另一個執行檔行程 | +| `utils/shell_process/` | 263 | `ShellManager`：以 argv list 執行外部命令（禁用 `shell=True`） | +| `utils/start_exe/` | 36 | 啟動另一個執行檔行程 | ### 5.4.3 排程、觸發與背景監看 -> 11 個套件、約 3,964 行。 +> 11 個套件、約 4,210 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/hotkey/` | 837 | 全域熱鍵守護行程，把 OS 層熱鍵綁到 action 檔（Win/macOS/X11 三後端） | +| `utils/hotkey/` | 846 | 全域熱鍵守護行程，把 OS 層熱鍵綁到 action 檔（Win/macOS/X11 三後端） | | `utils/idle_keepawake/` | 245 | 偵測使用者閒置時間並在無人值守執行期間阻止系統睡眠 | | `utils/lock_session/` | 166 | 鎖定工作站、等待解鎖並分類鎖定狀態轉換 | | `utils/observer/` | 234 | 反應式畫面觀察者，在出現／消失／變化時觸發 | -| `utils/recurrence/` | 388 | RFC 5545 重複規則解析與發生時間展開 | -| `utils/scheduler/` | 439 | 間隔式與 cron 式的 action JSON 排程器 | +| `utils/recurrence/` | 398 | RFC 5545 重複規則解析與發生時間展開 | +| `utils/scheduler/` | 500 | 間隔式與 cron 式的 action JSON 排程器 | | `utils/session_guard/` | 62 | 驅動輸入前先偵測工作階段是否已鎖定／非互動 | -| `utils/triggers/` | 1,241 | 事件驅動觸發引擎：影像／視窗／像素／檔案／webhook／IMAP 郵件 | -| `utils/voice/` | 87 | 語音指令路由：把辨識到的語句對應到 `AC_*` action list | -| `utils/watchdog/` | 183 | 背景彈窗／中斷看門狗，供無人值守自動化 | -| `utils/watcher/` | 82 | 無頭輪詢原語：滑鼠位置、像素顏色、log tail | +| `utils/triggers/` | 1,377 | 事件驅動觸發引擎：影像／視窗／像素／檔案／webhook／IMAP 郵件 | +| `utils/voice/` | 97 | 語音指令路由：把辨識到的語句對應到 `AC_*` action list | +| `utils/watchdog/` | 195 | 背景彈窗／中斷看門狗，供無人值守自動化 | +| `utils/watcher/` | 90 | 無頭輪詢原語：滑鼠位置、像素顏色、log tail | ### 5.4.4 輸入模擬與動作品質 -> 22 個套件、約 2,722 行。 +> 22 個套件、約 2,768 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -352,105 +353,105 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/actionability/` | 168 | 動作前就緒閘門（可見 + 穩定 + 啟用 + 未被遮擋） | | `utils/ensure_state/` | 74 | 冪等地把控制項／設定帶到期望狀態 | | `utils/field_entry/` | 76 | 清空再輸入的欄位填寫慣用法（Playwright `fill`） | -| `utils/gamepad/` | 324 | 虛擬遊戲手把後端（Windows ViGEmBus 驅動） | +| `utils/gamepad/` | 333 | 虛擬遊戲手把後端（Windows ViGEmBus 驅動） | | `utils/humanize/` | 191 | 擬人輸入：貝茲曲線滑鼠路徑 + 抖動打字節奏 | | `utils/ime_state/` | 146 | 讀取即時 IME 組字／轉換狀態，確保 CJK 輸入安全 | | `utils/key_hold/` | 109 | 按住按鍵一段時間，或以固定頻率自動重複 | -| `utils/modifier_state/` | 76 | 跨一組動作按住修飾鍵，並保證安全釋放 | +| `utils/modifier_state/` | 81 | 跨一組動作按住修飾鍵，並保證安全釋放 | | `utils/mouse_path/` | 106 | 多路徑點滑鼠手勢（沿折線移動或拖曳） | | `utils/mouse_relative/` | 59 | 相對位移滑鼠移動 | | `utils/postcondition/` | 146 | 宣告式的動作預期結果規格，對照畫面驗證 | -| `utils/step_repair/` | 134 | 失敗／無效動作的修復策略（自我修正迴圈） | -| `utils/table_grid_fill/` | 143 | 以 OCR 文字填滿格線表格，取得可定址的表格 | +| `utils/step_repair/` | 136 | 失敗／無效動作的修復策略（自我修正迴圈） | +| `utils/table_grid_fill/` | 163 | 以 OCR 文字填滿格線表格，取得可定址的表格 | | `utils/input_reach/` | 111 | 送出去的輸入到不到得了：桌面鎖定查詢（免費）＋ 實際送一個 F13 確認沒有被過濾（有副作用，只給診斷用） | -| `utils/keyboard_layout/` | 148 | 向系統問「這個鍵盤配置下每個鍵印出什麼字」（`ToUnicodeEx`），問不到退回 US 對照表 | +| `utils/keyboard_layout/` | 152 | 向系統問「這個鍵盤配置下每個鍵印出什麼字」（`ToUnicodeEx`），問不到退回 US 對照表 | | `utils/text_unicode/` | 151 | 輸入任意 Unicode（emoji／CJK／重音字）：優先送字元按鍵事件，不支援時退回剪貼簿貼上 | | `utils/tween_drag/` | 101 | 沿曲線的緩動插值拖曳 | -| `utils/verify_field/` | 112 | 打字後讀回欄位，確認內容確實落地 | +| `utils/verify_field/` | 118 | 打字後讀回欄位，確認內容確實落地 | ### 5.4.5 影像辨識與畫面分析 -> 37 個套件、約 5,612 行。 +> 37 個套件、約 5,782 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/annotate/` | 115 | 截圖標註：畫框、highlight、箭頭、標籤 | -| `utils/barcode/` | 53 | 一維條碼（EAN／UPC）解碼，解碼器可注入 | +| `utils/annotate/` | 121 | 截圖標註：畫框、highlight、箭頭、標籤 | +| `utils/barcode/` | 59 | 一維條碼（EAN／UPC）解碼，解碼器可注入 | | `utils/color_match/` | 127 | 在 HSV 通道上做顏色感知的樣板比對 | | `utils/color_region/` | 96 | 以顏色定位畫面區域（遮罩 + 連通元件） | -| `utils/color_stats/` | 98 | 區域顏色統計：平均色與主色 | +| `utils/color_stats/` | 103 | 區域顏色統計：平均色與主色 | | `utils/coordinate_space/` | 93 | 模型網格座標與實體像素之間的座標空間對映 | | `utils/cv2_utils/` | 798 | OpenCV 基礎層：擷取後端選擇（`screen_grabber`，Pillow／mss 或平台後端）、截圖、樣板比對（走 `grab_logical`，涵蓋所有螢幕）、螢幕錄影、影片錄製（兩者都經 `frame_clock` 依 fps 配速）、連通元件、影像堆疊的取用口（`optional`，Windows arm64 沒有 wheel 時語意報錯）、非 ASCII 路徑也讀寫得到的影像檔存取（`image_file`） | | `utils/edge_lines/` | 122 | 以 Hough 轉換偵測線條／格線／分隔線 | | `utils/edge_match/` | 115 | 邊緣形狀（Chamfer／距離轉換）樣板比對 | | `utils/feature_match/` | 143 | ORB 特徵比對：在旋轉／縮放／主題變更下定位樣板 | -| `utils/hsv_segment/` | 91 | HSV 色彩空間分割（抗光照的顏色遮罩 + blob 框） | +| `utils/hsv_segment/` | 104 | HSV 色彩空間分割（抗光照的顏色遮罩 + blob 框） | | `utils/icon_classify/` | 132 | 從像素形狀判斷一個框是哪一類元件 | -| `utils/image_dedup/` | 90 | 感知雜湊影像去重（Pillow aHash/dHash） | +| `utils/image_dedup/` | 100 | 感知雜湊影像去重（Pillow aHash/dHash） | | `utils/image_quality/` | 77 | 在 OCR／比對前評分影像品質（銳利度／對比／亮度） | -| `utils/img_histogram/` | 105 | 顏色直方圖指紋與變化偵測（抗光照） | +| `utils/img_histogram/` | 112 | 顏色直方圖指紋與變化偵測（抗光照） | | `utils/marks_layout/` | 149 | Set-of-Marks 標籤的不重疊排版與可讀配色 | | `utils/match_autothresh/` | 114 | Otsu 自動門檻，免去手動調 `min_score` | -| `utils/match_ensemble/` | 63 | 多樣板共識比對（多張參考圖投票到同一位置） | -| `utils/match_stability/` | 68 | 比對前的靜止閘門與跨影格的比對持續性 | +| `utils/match_ensemble/` | 67 | 多樣板共識比對（多張參考圖投票到同一位置） | +| `utils/match_stability/` | 70 | 比對前的靜止閘門與跨影格的比對持續性 | | `utils/match_trust/` | 144 | 樣板比對可信度評分（次峰比 + peak-to-sidelobe） | | `utils/monitor_layout/` | 320 | 多螢幕／虛擬桌面幾何（在哪個螢幕、位置、重映射）＋ `logical_frame` 以滑鼠座標空間擷取畫面 | | `utils/motion_regions/` | 73 | 兩影格間的局部變化／活動偵測（absdiff） | -| `utils/perceptual_diff/` | 100 | 感知式（YIQ）影像差異，抑制反鋸齒邊緣誤報 | +| `utils/perceptual_diff/` | 196 | 感知式（YIQ）影像差異，抑制反鋸齒邊緣誤報 | | `utils/preprocess/` | 219 | OCR／比對前的影像前處理（灰階、二值化、去傾斜…） | -| `utils/qr/` | 60 | 從影像或螢幕區域解碼 QR code（OpenCV） | +| `utils/qr/` | 59 | 從影像或螢幕區域解碼 QR code（OpenCV） | | `utils/rotated_match/` | 166 | 容忍旋轉與縮放的樣板比對（尺度空間 × 角度掃描） | | `utils/saliency/` | 114 | 頻譜殘差視覺顯著性：顯著圖與排序後的顯著區域 | | `utils/scale_detect/` | 84 | 偵測樣板實際渲染的顯示縮放／視覺 DPI | | `utils/screen_grid/` | 146 | 供 VLM 接地用的粗粒度標號網格（點 ↔ 格對映） | | `utils/set_of_marks/` | 154 | Set-of-Marks 疊圖：為畫面元素編號供 VLM 指認 | | `utils/shape_locator/` | 108 | 以邊緣／輪廓偵測定位元件（矩形／形狀，免樣板） | -| `utils/ssim/` | 143 | 結構相似度比較：感知分數 + 變化區域 | +| `utils/ssim/` | 163 | 結構相似度比較：感知分數 + 變化區域 | | `utils/subpixel_match/` | 103 | 以二次曲面擬合做次像素級比對精修 | -| `utils/theme_normalize/` | 92 | 主題無關的影像正規化，讓亮色樣板能配對深色模式 | +| `utils/theme_normalize/` | 94 | 主題無關的影像正規化，讓亮色樣板能配對深色模式 | | `utils/video_report/` | 171 | 影片步驟疊圖報告：把截圖加字幕串成操作導覽影片 | | `utils/visual_match/` | 515 | 會回傳信心值的樣板比對（分數、多尺度、find-all + NMS）；擷取走 `grab_logical`，命中座標已加回虛擬桌面原點，單色樣板直接拒收 | | `utils/visual_regression/` | 251 | 桌面 GUI 的視覺回歸測試（黃金圖比對） | ### 5.4.6 OCR 與文字理解 -> 19 個套件、約 3,336 行。 +> 19 個套件、約 3,469 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/bidi_check/` | 129 | 雙向文字 QA（bidi 控制碼、巢狀平衡、Trojan-source 掃描） | -| `utils/column_layout/` | 150 | 從垂直空白推斷欄位，處理無框線表格 | -| `utils/confusables/` | 139 | 易混淆／同形字偵測（Unicode 欺騙骨架） | -| `utils/form_fields/` | 128 | 多方向關聯表單標籤與值，並讀取核取方塊狀態 | -| `utils/fuzzy/` | 96 | 模糊字串比對與去重（預設 difflib，有 rapidfuzz 則優先） | +| `utils/bidi_check/` | 138 | 雙向文字 QA（bidi 控制碼、巢狀平衡、Trojan-source 掃描） | +| `utils/column_layout/` | 153 | 從垂直空白推斷欄位，處理無框線表格 | +| `utils/confusables/` | 146 | 易混淆／同形字偵測（Unicode 欺騙骨架） | +| `utils/form_fields/` | 134 | 多方向關聯表單標籤與值，並讀取核取方塊狀態 | +| `utils/fuzzy/` | 111 | 模糊字串比對與去重（預設 difflib，有 rapidfuzz 則優先） | | `utils/grid_locator/` | 71 | 以 (row, column) 從邊界框定址表格／網格儲存格 | -| `utils/guardrail/` | 116 | 針對畫面／OCR 文字的啟發式 prompt-injection 防護 | -| `utils/heading_segment/` | 69 | 判定 OCR 行是標題或內文，建出文件大綱 | +| `utils/guardrail/` | 117 | 針對畫面／OCR 文字的啟發式 prompt-injection 防護 | +| `utils/heading_segment/` | 71 | 判定 OCR 行是標題或內文，建出文件大綱 | | `utils/near_dup/` | 108 | 近似重複文字偵測（SimHash／MinHash） | -| `utils/ocr/` | 1,126 | OCR 引擎門面 + 三個後端（Tesseract／EasyOCR／PaddleOCR）、版面結構化與跨詞比對（`text_span`） | -| `utils/pii_text/` | 119 | 自由文字中的 PII 偵測與遮蔽（email／電話／SSN／卡號／IP／IBAN） | -| `utils/readability/` | 138 | 可讀性評分（Flesch、Flesch-Kincaid、Gunning Fog、SMOG、ARI） | -| `utils/reading_flow/` | 119 | 以遞迴 XY-cut 推導欄位感知的閱讀順序 | +| `utils/ocr/` | 1,140 | OCR 引擎門面 + 三個後端（Tesseract／EasyOCR／PaddleOCR）、版面結構化與跨詞比對（`text_span`） | +| `utils/pii_text/` | 141 | 自由文字中的 PII 偵測與遮蔽（email／電話／SSN／卡號／IP／IBAN） | +| `utils/readability/` | 140 | 可讀性評分（Flesch、Flesch-Kincaid、Gunning Fog、SMOG、ARI） | +| `utils/reading_flow/` | 145 | 以遞迴 XY-cut 推導欄位感知的閱讀順序 | | `utils/search_index/` | 145 | 記憶體內 BM25／TF-IDF 全文檢索 | | `utils/text_blocks/` | 88 | 把 OCR 行組成段落與項目符號／編號清單 | -| `utils/text_diff/` | 187 | unified diff 產生、套用與三方合併 | -| `utils/text_normalize/` | 82 | Unicode 正規化與 slug 產生 | -| `utils/text_regions/` | 161 | 免模型的畫面文字區域偵測（MSER）：區域與行 | -| `utils/text_similarity/` | 165 | 字串距離度量（文字比對用） | +| `utils/text_diff/` | 202 | unified diff 產生、套用與三方合併 | +| `utils/text_normalize/` | 84 | Unicode 正規化與 slug 產生 | +| `utils/text_regions/` | 163 | 免模型的畫面文字區域偵測（MSER）：區域與行 | +| `utils/text_similarity/` | 172 | 字串距離度量（文字比對用） | ### 5.4.7 無障礙樹與原生控制項 -> 16 個套件、約 4,516 行。 +> 16 個套件、約 4,619 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/a11y_audit/` | 355 | 以無障礙樹 + OCR 進行無障礙與 i18n 稽核 | -| `utils/accessibility/` | 3,032 | 跨平台無障礙樹定位與錄製；Windows UIA／macOS AX／null 三後端。支援限定視窗（換搜尋起點，不是過濾）、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值／勾選／選取／數值（密碼欄位不回內容） | +| `utils/a11y_audit/` | 362 | 以無障礙樹 + OCR 進行無障礙與 i18n 稽核 | +| `utils/accessibility/` | 3,117 | 跨平台無障礙樹定位與錄製；Windows UIA／macOS AX／null 三後端。支援限定視窗（換搜尋起點，不是過濾）、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值／勾選／選取／數值（密碼欄位不回內容） | | `utils/ax_events/` | 29 | 反應式 UIA 事件等待（focus-changed） | | `utils/ax_props/` | 44 | 讀取豐富 UIA 屬性（enabled／offscreen／help／status／快捷鍵） | | `utils/ax_text/` | 102 | 透過 UIA TextPattern 取得原生文字（讀取／尋找／選取／屬性） | -| `utils/ax_tree_walk/` | 118 | 可讀、可定址的無障礙樹後處理（角色名 + 節點路徑） | -| `utils/contrast_map/` | 120 | 取樣實際顏色以評定畫面文字的可讀性（WCAG） | +| `utils/ax_tree_walk/` | 119 | 可讀、可定址的無障礙樹後處理（角色名 + 節點路徑） | +| `utils/contrast_map/` | 130 | 取樣實際顏色以評定畫面文字的可讀性（WCAG） | | `utils/control_patterns/` | 88 | 延伸 UIA 控制項模式動作（Expand／Select／Range／Scroll） | | `utils/cvd_simulate/` | 140 | 模擬色覺缺陷並標示在該狀況下會撞色的顏色 | | `utils/element_repository/` | 113 | 原生 UI 元素的具名定位器倉庫（object repository） | @@ -463,228 +464,228 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.8 元素定位、自我修復與智慧等待 -> 23 個套件、約 4,205 行。 +> 23 個套件、約 4,316 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/ab_locator/` | 382 | A/B 定位器框架：同時競速 N 種策略並記錄各自勝率 | -| `utils/adaptive_timeout/` | 84 | 由觀測到的步驟耗時推導等待逾時，而非硬猜 | -| `utils/anchor_locator/` | 457 | 錨點定位器：以空間關係組合 影像／OCR／VLM／a11y 四種來源 | +| `utils/ab_locator/` | 386 | A/B 定位器框架：同時競速 N 種策略並記錄各自勝率 | +| `utils/adaptive_timeout/` | 92 | 由觀測到的步驟耗時推導等待逾時，而非硬猜 | +| `utils/anchor_locator/` | 476 | 錨點定位器：以空間關係組合 影像／OCR／VLM／a11y 四種來源 | | `utils/app_idle/` | 109 | 等應用程式不再忙碌，再驅動下一步 | -| `utils/change_localize/` | 80 | 把畫面變化歸因到實際改變的元素框 | +| `utils/change_localize/` | 83 | 把畫面變化歸因到實際改變的元素框 | | `utils/critic_features/` | 85 | 每步的 critic 特徵集合與規則式步驟評分 | -| `utils/element_diff/` | 93 | 跨影格的幾何感知元素比對（穩定 ID、移動追蹤） | +| `utils/element_diff/` | 94 | 跨影格的幾何感知元素比對（穩定 ID、移動追蹤） | | `utils/element_parse/` | 106 | 融合並排序畫面元素框（IoU、合併、多來源融合、閱讀順序） | -| `utils/element_proposal/` | 86 | 免樣板、免模型地從原始像素提出乾淨元素清單 | +| `utils/element_proposal/` | 92 | 免樣板、免模型地從原始像素提出乾淨元素清單 | | `utils/element_scoring/` | 105 | 加權候選評分（角色 + 名稱相似度 + 鄰近度 + 啟用狀態） | | `utils/expect_poll/` | 149 | 反覆取值直到符合條件（Playwright `expect.poll` 風格） | -| `utils/grounding_consensus/` | 127 | 對同一目標的多個接地提案做自我一致性投票 | +| `utils/grounding_consensus/` | 153 | 對同一目標的多個接地提案做自我一致性投票 | | `utils/heal_analytics/` | 77 | 自癒事件記錄的分析（治癒率、脆弱定位器） | | `utils/locator_chain/` | 112 | 可組合／可過濾的候選定位器（chained-locator 慣用法） | | `utils/locator_repair/` | 117 | 自癒回寫：把修正後的定位器持久化 | | `utils/observation/` | 92 | 供 VLM／agent 接地用的 token 預算內、帶索引的 a11y 文字觀察 | -| `utils/observation_delta/` | 103 | token 預算內的觀察差異：兩個 UI 影格之間變了什麼 | -| `utils/screen_state/` | 189 | 語義畫面狀態：快照／差異與結構化畫面描述 | +| `utils/observation_delta/` | 122 | token 預算內的觀察差異：兩個 UI 影格之間變了什麼 | +| `utils/screen_state/` | 191 | 語義畫面狀態：快照／差異與結構化畫面描述 | | `utils/scroll_find/` | 103 | 捲動直到目標影像／文字可見 | -| `utils/self_healing/` | 352 | 自癒定位器：先影像樣板、失敗改用 VLM，並留稽核記錄 | +| `utils/self_healing/` | 359 | 自癒定位器：先影像樣板、失敗改用 VLM，並留稽核記錄 | | `utils/semantic_recording/` | 460 | 為錄製內容加上語義錨點，支援換機重播與自癒重播 | | `utils/settle_detector/` | 79 | 以純函式介面判定 UI 是否已靜止 | -| `utils/smart_waits/` | 658 | 智慧等待：以影格差異取代 `time.sleep` | +| `utils/smart_waits/` | 674 | 智慧等待：以影格差異取代 `time.sleep` | ### 5.4.9 AI / Agent / LLM -> 13 個套件、約 21,391 行。 +> 13 個套件、約 23,127 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/a2a/` | 92 | A2A（agent-to-agent）agent card 產生 | -| `utils/agent/` | 1,446 | 閉環 Computer-Use Agent 主迴圈 + Anthropic／OpenAI／Computer-Use 三後端 | -| `utils/agent_memory/` | 154 | agent 的持久化情節記憶（goal → trajectory → outcome） | -| `utils/agent_replay/` | 63 | 可攜的 agent 軌跡追蹤（記錄 observation→action 並重播） | -| `utils/agent_trace/` | 168 | agent 可觀測性：OpenTelemetry GenAI 慣例的 LLM span | -| `utils/cost_telemetry/` | 307 | 每次呼叫的 LLM 成本遙測：token 數 + 估算美金 | +| `utils/agent/` | 1,975 | 閉環 Computer-Use Agent 主迴圈 + Anthropic／OpenAI／Computer-Use 三後端 | +| `utils/agent_memory/` | 166 | agent 的持久化情節記憶（goal → trajectory → outcome） | +| `utils/agent_replay/` | 67 | 可攜的 agent 軌跡追蹤（記錄 observation→action 並重播） | +| `utils/agent_trace/` | 172 | agent 可觀測性：OpenTelemetry GenAI 慣例的 LLM span | +| `utils/cost_telemetry/` | 345 | 每次呼叫的 LLM 成本遙測：token 數 + 估算美金 | | `utils/cua_action/` | 204 | 標準化 computer-use 動作結構（Anthropic／OpenAI → `AC_*`） | | `utils/llm/` | 365 | 自然語言 → action list 規劃器 + Anthropic／null 後端 | | `utils/mcp_registry/` | 97 | MCP registry `server.json` 資訊清單產生（可被發現） | -| `utils/mcp_server/` | 17,675 | **無頭 MCP 伺服器**（16K LOC，預設註冊 677 個工具＝658 個 `ac_*` + 19 個別名）：stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | -| `utils/tool_use_schema/` | 189 | 把 `AC_*` 指令匯出成 Claude／OpenAI 的 tool-use schema | +| `utils/mcp_server/` | 18,798 | **無頭 MCP 伺服器**（16K LOC，預設註冊 678 個工具＝659 個 `ac_*` + 19 個別名）：stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | +| `utils/tool_use_schema/` | 195 | 把 `AC_*` 指令匯出成 Claude／OpenAI 的 tool-use schema | | `utils/trajectory_eval/` | 113 | agent 軌跡評估：依評分規準為一次執行打分 | -| `utils/vision/` | 518 | VLM 元素定位器（依描述找元素）+ Anthropic／OpenAI／null 後端 | +| `utils/vision/` | 538 | VLM 元素定位器（依描述找元素）+ Anthropic／OpenAI／null 後端 | ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 18,835 行。 +> 6 個套件、約 19,332 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/admin/` | 396 | 多主機管理主控台：平行輪詢 N 個 AutoControl REST 端點 | -| `utils/config_sync/` | 323 | 透過訊令伺服器做跨機器設定同步 | +| `utils/admin/` | 418 | 多主機管理主控台：平行輪詢 N 個 AutoControl REST 端點 | +| `utils/config_sync/` | 332 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣：同一 action list 於多台裝置平行執行 | -| `utils/remote_desktop/` | 12,561 | **遠端桌面子系統**（56 檔／11.7K LOC）：TCP／WebSocket／WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN／中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | -| `utils/usb/` | 4,472 | 跨平台 USB 列舉／熱插拔／裝置直通（WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道） | -| `utils/usbip/` | 945 | USB/IP 線路協定主機端（協定封包、TCP 伺服器、libusb URB 後端） | +| `utils/remote_desktop/` | 12,912 | **遠端桌面子系統**（56 檔／11.7K LOC）：TCP／WebSocket／WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN／中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | +| `utils/usb/` | 4,524 | 跨平台 USB 列舉／熱插拔／裝置直通（WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道） | +| `utils/usbip/` | 1,008 | USB/IP 線路協定主機端（協定封包、TCP 伺服器、libusb URB 後端） | ### 5.4.11 伺服器、網路協定與外部整合 -> 24 個套件、約 6,472 行。 +> 24 個套件、約 6,721 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/acme_v2/` | 614 | 完整 ACME v2 用戶端（RFC 8555），不依賴 certbot | +| `utils/acme_v2/` | 617 | 完整 ACME v2 用戶端（RFC 8555），不依賴 certbot | | `utils/chatops/` | 667 | Chat-ops bot：接收 Slack／Discord／webhook 的 slash 指令並路由到動作 | -| `utils/cookie_jar/` | 121 | RFC 6265 cookie jar | -| `utils/email_send/` | 116 | SMTP 寄信（email 觸發器的發送端搭檔） | +| `utils/cookie_jar/` | 122 | RFC 6265 cookie jar | +| `utils/email_send/` | 118 | SMTP 寄信（email 觸發器的發送端搭檔） | | `utils/events/` | 106 | 對外 CloudEvents 發送（執行生命週期事件） | -| `utils/http_cassette/` | 153 | 錄製／重播 HTTP 互動，做離線決定性 API 測試 | -| `utils/http_client/` | 228 | 零依賴 HTTP(S) 用戶端，供 action 步驟呼叫 API | -| `utils/http_conditional/` | 108 | 條件式 HTTP 請求與快取驗證器 | -| `utils/http_content/` | 148 | HTTP 內容協商與回應解壓縮 | -| `utils/http_problem/` | 117 | RFC 9457 problem+json 解析 | -| `utils/jwt/` | 219 | JWT（HMAC 家族）編碼、解碼與 claim 驗證 | -| `utils/link_header/` | 146 | RFC 8288 Link header 解析與分頁 | -| `utils/multipart/` | 175 | multipart/form-data 建構與解析 | -| `utils/notify/` | 95 | 跨平台桌面通知 | -| `utils/notify_channels/` | 100 | 對外聊天／webhook 通知（Slack／Discord／Teams／raw） | +| `utils/http_cassette/` | 200 | 錄製／重播 HTTP 互動，做離線決定性 API 測試 | +| `utils/http_client/` | 245 | 零依賴 HTTP(S) 用戶端，供 action 步驟呼叫 API | +| `utils/http_conditional/` | 115 | 條件式 HTTP 請求與快取驗證器 | +| `utils/http_content/` | 158 | HTTP 內容協商與回應解壓縮 | +| `utils/http_problem/` | 118 | RFC 9457 problem+json 解析 | +| `utils/jwt/` | 240 | JWT（HMAC 家族）編碼、解碼與 claim 驗證 | +| `utils/link_header/` | 150 | RFC 8288 Link header 解析與分頁 | +| `utils/multipart/` | 181 | multipart/form-data 建構與解析 | +| `utils/notify/` | 106 | 跨平台桌面通知 | +| `utils/notify_channels/` | 105 | 對外聊天／webhook 通知（Slack／Discord／Teams／raw） | | `utils/otp/` | 37 | TOTP 一次性密碼產生（自動化 2FA 登入） | | `utils/outbox/` | 107 | 交易式 outbox，保證至少一次的事件投遞 | | `utils/pytest_plugin/` | 380 | pytest 外掛 + BDD step library（`pytest11` entry point） | -| `utils/rest_api/` | 1,793 | 純標準庫 REST 前端：路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | -| `utils/socket_server/` | 156 | 執行 action JSON 的執行緒式 TCP 指令伺服器（預設綁 127.0.0.1） | -| `utils/sse_client/` | 126 | Server-Sent Events 用戶端解析 | +| `utils/rest_api/` | 1,881 | 純標準庫 REST 前端：路由、Bearer 驗證、限流、Prometheus 指標、OpenAPI 3.1 產生 | +| `utils/socket_server/` | 160 | 執行 action JSON 的執行緒式 TCP 指令伺服器（預設綁 127.0.0.1） | +| `utils/sse_client/` | 128 | Server-Sent Events 用戶端解析 | | `utils/tls_acme/` | 455 | TLS 自動化：HTTP-01 挑戰伺服器、金鑰／CSR、自動續期 | -| `utils/url_canon/` | 144 | RFC 3986 URL 正規化與查詢字串工具 | -| `utils/webrunner_bridge/` | 161 | 把 action JSON 橋接到 WebRunner（`je_web_runner`） | +| `utils/url_canon/` | 156 | RFC 3986 URL 正規化與查詢字串工具 | +| `utils/webrunner_bridge/` | 169 | 把 action JSON 橋接到 WebRunner（`je_web_runner`） | ### 5.4.12 報表、可觀測性與測試治理 -> 34 個套件、約 7,281 行。 +> 34 個套件、約 7,506 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/anomaly/` | 114 | 單一序列異常偵測 | | `utils/approval/` | 118 | Approval testing：以核可基準線驗證產出物 | -| `utils/assertion/` | 881 | 斷言 DSL：畫面狀態驗證 + 組合子 | +| `utils/assertion/` | 887 | 斷言 DSL：畫面狀態驗證 + 組合子 | | `utils/baggage/` | 120 | W3C Baggage 傳遞 | | `utils/canonical_log/` | 96 | canonical log line 與結構化 JSON 日誌 | -| `utils/ci_annotations/` | 62 | 由執行結果輸出 CI 工作流程註記（GitHub Actions） | +| `utils/ci_annotations/` | 65 | 由執行結果輸出 CI 工作流程註記（GitHub Actions） | | `utils/compliance/` | 153 | 合規：把治理證據對應到 SOC2／ISO 27001 控制項 | | `utils/failure_hooks/` | 415 | 失敗 → 工單自動化：開 Jira／Linear／GitHub issue | -| `utils/failure_signature/` | 74 | 把錯誤訊息正規化成穩定的 SHA-256 失敗簽章並分群 | +| `utils/failure_signature/` | 76 | 把錯誤訊息正規化成穩定的 SHA-256 失敗簽章並分群 | | `utils/flake_cluster/` | 103 | 以共同失敗 Jaccard 相似度為易碎測試分群 | | `utils/flakiness/` | 150 | 以執行歷史分析不穩定測試 | | `utils/generate_report/` | 293 | HTML／JSON／XML 三種報表產生器（Template Method） | | `utils/media_assert/` | 242 | 媒體斷言：音訊活動與影片動態檢查 | -| `utils/observability/` | 697 | Prometheus 格式指標 + OpenTelemetry 相容 trace + `/metrics` 匯出伺服器 | -| `utils/otlp_export/` | 81 | OTLP/JSON span 匯出 | -| `utils/percentiles/` | 116 | 可合併的串流延遲摘要與精確百分位數 | -| `utils/process_doc/` | 85 | 由錄製的 action list 產生逐步 SOP 文件 | +| `utils/observability/` | 710 | Prometheus 格式指標 + OpenTelemetry 相容 trace + `/metrics` 匯出伺服器 | +| `utils/otlp_export/` | 109 | OTLP/JSON span 匯出 | +| `utils/percentiles/` | 119 | 可合併的串流延遲摘要與精確百分位數 | +| `utils/process_doc/` | 108 | 由錄製的 action list 產生逐步 SOP 文件 | | `utils/process_mining/` | 123 | 流程探勘：從動作日誌挖掘可自動化的候選 | -| `utils/profiler/` | 426 | 逐動作效能剖析器 + 資源剖析器 | +| `utils/profiler/` | 451 | 逐動作效能剖析器 + 資源剖析器 | | `utils/quarantine/` | 200 | 易碎測試隔離區，讓套件執行器跳過已知不穩定案例 | | `utils/run_diff/` | 123 | 兩次執行軌跡的差異（LCS 對齊：新增／移除／狀態翻轉／退化） | -| `utils/run_history/` | 410 | 執行歷史儲存與產出物管理 | -| `utils/sarif/` | 163 | 以 SARIF 2.1.0 匯出發現項，供 GitHub／Azure code scanning | +| `utils/run_history/` | 439 | 執行歷史儲存與產出物管理 | +| `utils/sarif/` | 187 | 以 SARIF 2.1.0 匯出發現項，供 GitHub／Azure code scanning | | `utils/slo/` | 115 | SLO 評估：SLI、錯誤預算與多視窗燃燒率告警 | | `utils/smoothing/` | 67 | 數列移動平均平滑 | | `utils/soft_assert/` | 79 | 軟斷言：累積檢查並在區塊結束時一次拋出 | -| `utils/stats/` | 223 | 描述統計與 A/B 顯著性檢定（純標準庫） | +| `utils/stats/` | 236 | 描述統計與 A/B 顯著性檢定（純標準庫） | | `utils/step_timeline/` | 81 | 每次執行的步驟瀑布圖與瓶頸（關鍵路徑）步驟排名 | -| `utils/test_select/` | 123 | 以執行歷史做風險導向的測試選取 | -| `utils/test_shard/` | 98 | 以耗時為權重的套件切分與分片結果合併 | -| `utils/test_suite/` | 527 | QA 套件編排：把扁平 action list 評分為測試案例 + CI 報表 | -| `utils/time_travel/` | 383 | 錄製 session 的時光回溯除錯（控制器 + 播放器） | -| `utils/timeseries/` | 163 | 時間序列轉換（rate／降採樣／重採樣） | -| `utils/trace_context/` | 177 | W3C Trace Context 傳遞 | +| `utils/test_select/` | 129 | 以執行歷史做風險導向的測試選取 | +| `utils/test_shard/` | 105 | 以耗時為權重的套件切分與分片結果合併 | +| `utils/test_suite/` | 547 | QA 套件編排：把扁平 action list 評分為測試案例 + CI 報表 | +| `utils/time_travel/` | 388 | 錄製 session 的時光回溯除錯（控制器 + 播放器） | +| `utils/timeseries/` | 175 | 時間序列轉換（rate／降採樣／重採樣） | +| `utils/trace_context/` | 183 | W3C Trace Context 傳遞 | ### 5.4.13 資料來源、結構驗證與 i18n -> 24 個套件、約 4,337 行。 +> 24 個套件、約 4,662 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/checksum/` | 138 | 檢查碼演算法：Luhn、Verhoeff、Damm、ISO 7064 MOD 97-10 | -| `utils/config_schema/` | 109 | 型別化設定結構驗證 | +| `utils/config_schema/` | 130 | 型別化設定結構驗證 | | `utils/data_drift/` | 128 | 分布漂移偵測 | -| `utils/data_profile/` | 121 | 資料剖析與結構推斷 | -| `utils/data_quality/` | 201 | 資料品質：列結構驗證、欄位擷取、遮蔽 | -| `utils/data_source/` | 192 | 資料驅動執行：從 CSV／JSON／SQLite／Excel 載入資料列 | +| `utils/data_profile/` | 129 | 資料剖析與結構推斷 | +| `utils/data_quality/` | 218 | 資料品質：列結構驗證、欄位擷取、遮蔽 | +| `utils/data_source/` | 235 | 資料驅動執行：從 CSV／JSON／SQLite／Excel 載入資料列 | | `utils/dataset_diff/` | 89 | 表格資料列差異比對（CDC 風格） | -| `utils/gettext_catalog/` | 322 | GNU gettext 目錄 I/O（解析 .po、編譯／讀取 .mo、訊息查詢） | +| `utils/gettext_catalog/` | 362 | GNU gettext 目錄 I/O（解析 .po、編譯／讀取 .mo、訊息查詢） | | `utils/i18n_test/` | 231 | 國際化／在地化測試輔助 | | `utils/json_contract/` | 145 | JSON 契約／快照比對：`match_json`、`diff_json`、`snapshot_json` | | `utils/json_patch/` | 352 | JSON Pointer（6901）、JSON Patch（6902）與 Merge Patch（7386） | -| `utils/json_schema/` | 419 | JSON Schema（Draft 2020-12 子集）驗證 | -| `utils/jsonpath/` | 242 | 精簡 JSONPath 查詢 | -| `utils/list_format/` | 72 | 地區感知清單格式化（CLDR 風格的「A、B 和 C」） | -| `utils/locale_collation/` | 128 | 地區感知字串排序（決定性多層排序鍵） | -| `utils/locale_parse/` | 79 | 地區感知數字／貨幣／日期解析與格式化（選用 babel） | -| `utils/message_format/` | 254 | ICU-lite MessageFormat（plural／select／selectordinal） | -| `utils/office/` | 180 | Office 文件無頭讀寫（Excel／Word／PowerPoint） | +| `utils/json_schema/` | 426 | JSON Schema（Draft 2020-12 子集）驗證 | +| `utils/jsonpath/` | 322 | 精簡 JSONPath 查詢 | +| `utils/list_format/` | 82 | 地區感知清單格式化（CLDR 風格的「A、B 和 C」） | +| `utils/locale_collation/` | 135 | 地區感知字串排序（決定性多層排序鍵） | +| `utils/locale_parse/` | 80 | 地區感知數字／貨幣／日期解析與格式化（選用 babel） | +| `utils/message_format/` | 288 | ICU-lite MessageFormat（plural／select／selectordinal） | +| `utils/office/` | 198 | Office 文件無頭讀寫（Excel／Word／PowerPoint） | | `utils/pdf/` | 117 | PDF 讀取與斷言（選用 pypdf 後端） | -| `utils/referential/` | 75 | 跨資料集的參照完整性檢查 | -| `utils/schema_compat/` | 172 | JSON Schema 相容性分級 | -| `utils/sql/` | 84 | 對 SQLite 的臨時唯讀 SQL 查詢 | -| `utils/test_data/` | 210 | 帶種子的合成測試資料產生（純標準庫） | -| `utils/xml/` | 277 | XML 檔讀寫與結構變更（`defusedxml`） | +| `utils/referential/` | 83 | 跨資料集的參照完整性檢查 | +| `utils/schema_compat/` | 177 | JSON Schema 相容性分級 | +| `utils/sql/` | 88 | 對 SQLite 的臨時唯讀 SQL 查詢 | +| `utils/test_data/` | 211 | 帶種子的合成測試資料產生（純標準庫） | +| `utils/xml/` | 298 | XML 檔讀寫與結構變更（`defusedxml`） | ### 5.4.14 安全、機密與合規 -> 13 個套件、約 2,793 行。 +> 13 個套件、約 2,913 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/config_redaction/` | 85 | 設定結構與 log 字串的機密遮蔽 | -| `utils/egress/` | 146 | 無頭 HTTP 用戶端的網路外連允許清單守衛 | -| `utils/governance/` | 237 | 治理：maker-checker 核准閘門與即時憑證租約 | -| `utils/license_policy/` | 220 | 以 SBOM 元件評估 SPDX 授權允許／拒絕政策 | -| `utils/provenance/` | 117 | SLSA 建置來源證明（in-toto v1） | +| `utils/config_redaction/` | 86 | 設定結構與 log 字串的機密遮蔽 | +| `utils/egress/` | 169 | 無頭 HTTP 用戶端的網路外連允許清單守衛 | +| `utils/governance/` | 242 | 治理：maker-checker 核准閘門與即時憑證租約 | +| `utils/license_policy/` | 240 | 以 SBOM 元件評估 SPDX 授權允許／拒絕政策 | +| `utils/provenance/` | 126 | SLSA 建置來源證明（in-toto v1） | | `utils/rbac/` | 299 | 角色型存取控制：使用者、角色與權杖驗證（尚未接到 REST／MCP） | -| `utils/redaction/` | 504 | 截圖遮蔽層：規則偵測 + 政策 + 協調器（上傳 VLM 前先遮） | -| `utils/sbom/` | 143 | SBOM（CycloneDX）產生 | +| `utils/redaction/` | 508 | 截圖遮蔽層：規則偵測 + 政策 + 協調器（上傳 VLM 前先遮） | +| `utils/sbom/` | 148 | SBOM（CycloneDX）產生 | | `utils/secret_ref/` | 143 | URI scheme 形式的值參照解析 | -| `utils/secrets/` | 340 | 加密機密儲存庫，供 `${secrets.NAME}` 解析 | -| `utils/secrets_scan/` | 133 | 掃描 action JSON／資料中應入庫卻硬編碼的機密 | -| `utils/vex/` | 167 | OpenVEX 陳述撰寫與漏洞分類處置 | -| `utils/vuln_scan/` | 259 | 以 OSV 比對 SBOM 元件的漏洞（純標準庫） | +| `utils/secrets/` | 360 | 加密機密儲存庫，供 `${secrets.NAME}` 解析 | +| `utils/secrets_scan/` | 138 | 掃描 action JSON／資料中應入庫卻硬編碼的機密 | +| `utils/vex/` | 178 | OpenVEX 陳述撰寫與漏洞分類處置 | +| `utils/vuln_scan/` | 276 | 以 OSV 比對 SBOM 元件的漏洞（純標準庫） | ### 5.4.15 韌性、流量控制與設定 -> 14 個套件、約 1,994 行。 +> 14 個套件、約 2,033 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/artifact_store/` | 128 | S3 相容產出物儲存（報表／截圖／錄影） | +| `utils/artifact_store/` | 148 | S3 相容產出物儲存（報表／截圖／錄影） | | `utils/assets/` | 178 | 環境範圍的型別化資產／設定儲存（UiPath Assets 風格） | | `utils/bulkhead/` | 141 | Bulkhead 併發隔離 + 伺服器限流標頭解析 | | `utils/chaos/` | 153 | 決定性混沌實驗（穩態假說 + 故障注入） | | `utils/dedup_window/` | 72 | 時間視窗內的訊息去重 | -| `utils/dotenv/` | 157 | `.env` 檔解析與序列化 | -| `utils/feature_flags/` | 182 | 功能旗標評估，含目標規則與決定性灰度 | +| `utils/dotenv/` | 165 | `.env` 檔解析與序列化 | +| `utils/feature_flags/` | 191 | 功能旗標評估，含目標規則與決定性灰度 | | `utils/idempotency/` | 142 | 冪等鍵儲存與已存回應重放 | | `utils/layered_config/` | 110 | 分層設定解析 | | `utils/optimistic/` | 135 | 樂觀併發的版本化儲存 | | `utils/rate_limit/` | 204 | 用戶端限流：token bucket、滑動視窗、throttle | -| `utils/resilience/` | 144 | 韌性原語：退避重試與斷路器 | +| `utils/resilience/` | 146 | 韌性原語：退避重試與斷路器 | | `utils/retry_budget/` | 158 | 重試預算：以牆鐘期限與 full jitter 約束重試 | | `utils/sequence_gap/` | 90 | 逐串流的序號缺口偵測 | ### 5.4.16 系統、視窗與剪貼簿 -> 16 個套件、約 2,523 行。 +> 16 個套件、約 2,610 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/clipboard/` | 446 | 跨平台無頭剪貼簿存取（文字 + 影像）＋ `win32_clipboard_api.py`：**所有剪貼簿格式共用的 Win32 原型與 open/alloc/lock 流程**（`open_clipboard()` 會等過短暫被別的行程佔住的剪貼簿——Win32 一次只允許一個行程開啟，別人正在複製就必然失敗）（`argtypes` 只宣告一半曾讓四支 writer 在 64 位元上必然丟 `OverflowError`，見 CHANGELOG）。`set_clipboard_image` 同時接受 PNG 位元組與檔案路徑——先前這個名字在本子套件裡有**兩份不同簽章的實作**（`clipboard.py` 吃 bytes、`clipboard_image.py` 吃路徑），匯錯來源只會在執行期才炸，已合併成一支 | -| `utils/clipboard_files/` | 112 | 剪貼簿檔案清單（CF_HDROP）：純 DROPFILES 封裝 + Win32 存取 | +| `utils/clipboard/` | 465 | 跨平台無頭剪貼簿存取（文字 + 影像）＋ `win32_clipboard_api.py`：**所有剪貼簿格式共用的 Win32 原型與 open/alloc/lock 流程**（`open_clipboard()` 會等過短暫被別的行程佔住的剪貼簿——Win32 一次只允許一個行程開啟，別人正在複製就必然失敗）（`argtypes` 只宣告一半曾讓四支 writer 在 64 位元上必然丟 `OverflowError`，見 CHANGELOG）。`set_clipboard_image` 同時接受 PNG 位元組與檔案路徑——先前這個名字在本子套件裡有**兩份不同簽章的實作**（`clipboard.py` 吃 bytes、`clipboard_image.py` 吃路徑），匯錯來源只會在執行期才炸，已合併成一支 | +| `utils/clipboard_files/` | 118 | 剪貼簿檔案清單（CF_HDROP）：純 DROPFILES 封裝 + Win32 存取 | | `utils/clipboard_formats/` | 151 | 檢視與分類剪貼簿可用格式（純分類／差異 + Win32 列舉） | | `utils/clipboard_history/` | 114 | 剪貼簿歷史：環形緩衝 + 背景輪詢器 | | `utils/clipboard_rich_formats/` | 328 | 豐富剪貼簿格式 — RTF 與 CSV/TSV 編解碼 + Windows 存取 | -| `utils/file_assoc/` | 92 | 解析哪個應用程式被註冊來開啟某副檔名 | -| `utils/file_dialog/` | 66 | 驅動原生檔案 開啟／儲存／資料夾選擇 對話框 | -| `utils/file_drop/` | 96 | 以 WM_DROPFILES 把檔案拖放到視窗 | -| `utils/rich_clipboard/` | 131 | 豐富剪貼簿格式 — HTML（CF_HTML）建構／解析／存取 | -| `utils/shell_open/` | 97 | 以預設應用開啟檔案，或以預設瀏覽器開啟 URL | -| `utils/system_volume/` | 199 | 讀取與控制系統主音量與靜音狀態 | +| `utils/file_assoc/` | 98 | 解析哪個應用程式被註冊來開啟某副檔名 | +| `utils/file_dialog/` | 77 | 驅動原生檔案 開啟／儲存／資料夾選擇 對話框 | +| `utils/file_drop/` | 124 | 以 WM_DROPFILES 把檔案拖放到視窗 | +| `utils/rich_clipboard/` | 133 | 豐富剪貼簿格式 — HTML（CF_HTML）建構／解析／存取 | +| `utils/shell_open/` | 99 | 以預設應用開啟檔案，或以預設瀏覽器開啟 URL | +| `utils/system_volume/` | 212 | 讀取與控制系統主音量與靜音狀態 | | `utils/trash/` | 93 | 把檔案移到系統資源回收筒（可復原刪除） | | `utils/window_capture/` | 304 | 逐視窗截圖、視窗版面儲存／還原、貼齊與排列 | | `utils/window_geometry/` | 81 | 視窗客戶區幾何（外框內縮、client→screen 對映） | @@ -695,38 +696,42 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 上表以子套件為單位；以下把行數最大的幾個子系統展開到檔案層。 -#### `utils/executor/`（9,412 行）— 執行核心 +#### `utils/executor/`（9,504 行）— 執行核心 | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `action_executor.py` | 8,289 | `Executor` 類別與 `event_dict` 分派表（774 個指令），另含數百個把 utils 能力接成指令的 adapter 函式；全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | -| `flow_control.py` | 622 | 真正的流程控制：`AC_loop`／`AC_for_each`／`AC_while_*`／`AC_if_*`／`AC_try`／`AC_retry`／`AC_parallel`／`AC_define_macro`／`AC_call_macro`／變數指令（`AC_set_var`／`AC_get_var`／`AC_inc_var`）。`LoopBreak`／`LoopContinue` 以例外實作。34 個區塊指令的分派表 `BLOCK_COMMANDS` 也在這裡，含下一列匯入的資料來源指令。 | -| `flow_data_commands.py` | 262 | `AC_*_to_var` 資料來源與轉換指令：shell、時鐘、亂數、PDF、TOTP、SQL、檔案、HTTP、OCR，加上 `AC_assert_var`／`AC_assert_db`／`AC_assert_duration`／`AC_transform_var`。都不執行巢狀 action list，所以沒有迴圈／分支語意。 | -| `action_schema.py` | 128 | action list 的結構驗證：形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式：`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字（REST `/execute` 用它回 400）。 | -| `action_redaction.py` | 72 | 記錄與紀錄鍵用的遮蔽：`AC_secret_*` 的參數（金庫通行碼、機密值）在寫進 log、當成結果紀錄的鍵之前換成 `***`，巢狀在區塊指令裡的也一樣。 | +| `action_executor.py` | 8,308 | `Executor` 類別與 `event_dict` 分派表（775 個指令），另含數百個把 utils 能力接成指令的 adapter 函式；全域單例 `executor` 與 `add_command_to_executor()` 擴充點。 | +| `flow_control.py` | 643 | 真正的流程控制：`AC_loop`／`AC_for_each`／`AC_while_*`／`AC_if_*`／`AC_try`／`AC_retry`／`AC_parallel`／`AC_define_macro`／`AC_call_macro`／變數指令（`AC_set_var`／`AC_get_var`／`AC_inc_var`）。`LoopBreak`／`LoopContinue` 以例外實作。34 個區塊指令的分派表 `BLOCK_COMMANDS` 也在這裡，含下一列匯入的資料來源指令。 | +| `flow_data_commands.py` | 272 | `AC_*_to_var` 資料來源與轉換指令：shell、時鐘、亂數、PDF、TOTP、SQL、檔案、HTTP、OCR，加上 `AC_assert_var`／`AC_assert_db`／`AC_assert_duration`／`AC_transform_var`。都不執行巢狀 action list，所以沒有迴圈／分支語意。 | +| `action_schema.py` | 159 | action list 的結構驗證：形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式：`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字（REST `/execute` 用它回 400）。 | +| `action_redaction.py` | 83 | 記錄與紀錄鍵用的遮蔽：`AC_secret_*` 的參數（金庫通行碼、機密值）在寫進 log、當成結果紀錄的鍵之前換成 `***`，巢狀在區塊指令裡的也一樣。 | | `mouse_aliases.py` | 39 | 單鍵點擊別名（`AC_click_left` 等），executor 與 callback executor 共用。 | -#### `utils/mcp_server/`（17,675 行，677 個工具）— 最大子系統 +#### `utils/mcp_server/`（18,798 行，678 個工具）— 最大子系統 | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `tools/_factories.py` | 9,005 | 工具工廠：每個函式回傳一個領域的 `MCPTool` 清單（把 `AC_*` 能力包成 MCP 工具）。 | +| `tools/_factories.py` | 9,023 | 工具工廠：每個函式回傳一個領域的 `MCPTool` 清單（把 `AC_*` 能力包成 MCP 工具）。 | | `tools/_handlers.py` | 545 | 把 MCP 工具呼叫橋接到 AutoControl 無頭 API 的 adapter；主題模組拆完之後這裡留的是資料／文字／HTTP 那一類與 WebRunner 橋接。 | -| `tools/_handlers_qa.py` | 414 | 同一種 adapter，QA 主題：斷言 DSL、資料驅動、SQL／PDF／郵件／HTTP 步驟、codegen、視覺回歸、狀態機、flaky 偵測與隔離、suite runner、無障礙稽核、裝置矩陣、媒體斷言。從 `_handlers.py` 依主題拆出的第一塊（750 行上限）；兩者互不引用。 | -| `tools/_handlers_input.py` | 212 | 同一種 adapter，輸入主題：滑鼠、鍵盤、虛擬手把（ViGEm）。 | -| `tools/_handlers_screen.py` | 305 | 同一種 adapter，螢幕主題：擷取、像素、影像與文字搜尋、螢幕錄影。 | -| `tools/_handlers_system.py` | 566 | 同一種 adapter，桌面工作階段：視窗、行程與 shell、開檔、閒置與睡眠、音量、鎖定、輸入法狀態、欄位驗證與重試、色彩對比、變更排序、元件分類、剪貼簿。 | +| `tools/_handlers_qa.py` | 419 | 同一種 adapter，QA 主題：斷言 DSL、資料驅動、SQL／PDF／郵件／HTTP 步驟、codegen、視覺回歸、狀態機、flaky 偵測與隔離、suite runner、無障礙稽核、裝置矩陣、媒體斷言。從 `_handlers.py` 依主題拆出的第一塊（750 行上限）；兩者互不引用。 | +| `tools/_handlers_input.py` | 218 | 同一種 adapter，輸入主題：滑鼠、鍵盤、虛擬手把（ViGEm）。 | +| `tools/_handlers_screen.py` | 327 | 同一種 adapter，螢幕主題：擷取、像素、影像與文字搜尋、螢幕錄影。 | +| `tools/_handlers_system.py` | 575 | 同一種 adapter，桌面工作階段：視窗、行程與 shell、開檔、閒置與睡眠、音量、鎖定、輸入法狀態、欄位驗證與重試、色彩對比、變更排序、元件分類、剪貼簿。 | | `tools/_handlers_runs.py` | 110 | 同一種 adapter，執行主題：executor、執行歷史、錄製、動作檔。 | | `tools/_handlers_scheduling.py` | 200 | 同一種 adapter，排程主題：排程器、觸發器、熱鍵常駐。 | | `tools/_handlers_remote.py` | 66 | 同一種 adapter，遠端桌面的 host 與 viewer。 | -| `tools/_handlers_executor_bridge.py` | 1,448 | 252 個純委派（中位數 3 行，最長的 16 行全是參數簽章）：每個都是 `from action_executor import _x` 再 `return _x(...)`，沒有分支邏輯。超過 750 行,理由記在 `Progress.md` 的豁免表（再切只能照 MCP 工廠領域分,會把同一種委派散進十幾個沒有語意邊界的檔）。 | -| `tools/_handlers_locators.py` | 423 | 同一種 adapter，定位主題：無障礙樹、智慧等待、自我修復、螢幕觀察、座標空間、視覺與 OCR、影像去重、元件倉庫、A/B 定位。 | +| `tools/_handlers_executor_bridge.py` | 1,429 | 252 個純委派（中位數 3 行，最長的 16 行全是參數簽章）：每個都是 `from action_executor import _x` 再 `return _x(...)`，沒有分支邏輯。超過 750 行,理由記在 `Progress.md` 的豁免表（再切只能照 MCP 工廠領域分,會把同一種委派散進十幾個沒有語意邊界的檔）。 | +| `tools/_handlers_locators.py` | 436 | 同一種 adapter，定位主題：無障礙樹、智慧等待、自我修復、螢幕觀察、座標空間、視覺與 OCR、影像去重、元件倉庫、A/B 定位。 | | `tools/_handlers_operations.py` | 647 | 同一種 adapter，營運主題：agent 與其記憶／追蹤、治理與合規、成本與遙測、失敗掛鉤、看門狗、速率限制、檢查點、核可、產物與資產、測試選擇與分片、佇列與 saga。 | -| `server.py` | 718 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器：連線範圍狀態、行內／併發分派、工具與 resource／prompt 處理器。 | -| `http_transport.py` | 585 | MCP 的 HTTP 傳輸。 | +| `server.py` | 700 | JSON-RPC 2.0 over stdio 的最小 MCP 伺服器：連線範圍狀態、行內／併發分派、工具與 resource／prompt 處理器；握手時代的方法表（`_run_method`），兩個協定時代的逐請求分派在 `_stateless.py`。 | +| `http_transport.py` | 715 | MCP 的 HTTP 傳輸；宣告 2026-07-28 的請求走 `_http_stateless.py` 的標頭規則，不發 session。 | +| `_http_stateless.py` | 185 | MCP 2026-07-28 在 Streamable HTTP 上的規則：`MCP-Protocol-Version`／`Mcp-Method`／`Mcp-Name` 必須與 body 相符（`=?base64?…?=` 先解碼），不符是 400＋`HeaderMismatch`；版本與中繼資料錯誤 400、未知方法 404。純函式，由 `http_transport.py` 回覆。 | | `http_sessions.py` | 247 | MCP 的 HTTP 傳輸用的 session 身分:`Mcp-Session-Id` 註冊表,以及每個 session 那條常駐的 server→client SSE 串流。 | -| `_client_requests.py` | 249 | 伺服器主動送出的請求：`roots/list`／`elicitation/create`／`sampling/createMessage`,對應表與回應路由,以及破壞性工具的確認交握。 | -| `_protocol.py` | 167 | JSON-RPC 線路格式：版本與識別常數、`_MCPError`、決定失敗工具行為的錯誤 tuple、envelope 產生器、工具回傳值轉 `content` 區塊。不碰伺服器狀態。 | +| `_client_requests.py` | 254 | 伺服器主動送出的請求：`roots/list`／`elicitation/create`／`sampling/createMessage`,對應表與回應路由,以及破壞性工具的確認交握。只屬於握手時代：無狀態請求裡送出會丟例外。 | +| `_stateless.py` | 267 | MCP 2026-07-28 無狀態版本，與以 `initialize` 握手的版本並存：逐請求的 `_meta`（版本、client 能力、`logLevel`）、`server/discover`、結果的 `resultType`／`serverInfo`／快取提示、`-32020`～`-32022` 錯誤碼，以及兩個時代逐請求分派的 mixin。 | +| `_input_required.py` | 160 | 多輪往返請求（MRTR）：`input_required` 結果，與 HMAC 簽章、會過期、只兌換一次的 `requestState`；破壞性工具確認在無狀態請求裡的形式。 | +| `_subscriptions.py` | 256 | 變更通知：握手時代的 `resources/subscribe`／`unsubscribe` 與未經訂閱的 `resources/updated`、`tools/list_changed`（不送給無狀態的對端）；2026-07-28 的 `subscriptions/listen`：確認、以訂閱 id 標記的通知、取消與伺服器結束時的完成回覆。 | +| `_protocol.py` | 219 | JSON-RPC 線路格式：版本與識別常數、`_MCPError`、決定失敗工具行為的錯誤 tuple、envelope 產生器、工具回傳值轉 `content` 區塊。不碰伺服器狀態。 | | `resources.py` | 307 | MCP resource 提供者。 | | `prompts.py` | 220 | MCP prompt 目錄。 | | `fake_backend.py` | 184 | CI／無頭測試用的記憶體內假後端。 | @@ -734,93 +739,93 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `tools/_base.py` | 146 | 工具註冊表的共用型別與輔助。 | | `tools/_validation.py` | 122 | MCP 工具用到的 JSON Schema 子集驗證器。 | | `tools/plugin_tools.py` | 89 | 把外掛載入的 `AC_*` callable 包成 `MCPTool`。 | -| `log_bridge.py` | 90 | 把 Python logging 記錄橋接成 MCP `notifications/message`。 | +| `log_bridge.py` | 118 | 把 Python logging 記錄橋接成 MCP `notifications/message`；2026-07-28 的請求只收到自己設了 `logLevel` 時產生的記錄。 | | `audit.py` | 87 | MCP 工具呼叫稽核記錄。 | | `context.py` | 71 | 傳給 opt-in 工具處理器的每次呼叫上下文。 | | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | -| `__main__.py` | 88 | `je_auto_control_mcp` console script 進入點。 | +| `__main__.py` | 92 | `je_auto_control_mcp` console script 進入點。 | -#### `utils/remote_desktop/`（12,561 行／56 檔） +#### `utils/remote_desktop/`（12,912 行／56 檔） 三條傳輸路徑並存：**TCP**（JPEG 影格）、**WebSocket**（同協定換傳輸）、**WebRTC**（aiortc 視訊 + DataChannel）。 | 檔案 | 行數 | 職責 | | --- | ---: | --- | | `webrtc_host.py` | 716 | WebRTC 主機：串流螢幕視訊並接受檢視端輸入;session 生命週期、DataChannel 接線、檔案收發。 | -| `webrtc_viewer.py` | 672 | WebRTC 檢視端：接收視訊並送出輸入。 | +| `webrtc_viewer.py` | 677 | WebRTC 檢視端：接收視訊並送出輸入。 | | `host.py` | 669 | TCP 主機：接受迴圈、TLS 包裝、連線／認證握手、音訊與剪貼簿廣播、檔案推送、單次 token。 | | `viewer.py` | 634 | TCP 檢視端。 | | `host_service.py` | 558 | 無頭 WebRTC 主機執行器 + 多平台服務安裝器。 | | `host_client.py` | 453 | TCP 主機的每連線處理器：一個檢視端一個實例,擁有它的認證交換、sender／audio／receiver 三條執行緒,以及入站訊息的路由表。 | | `registry.py` | 370 | `AC_remote_*` 指令使用的行程級單例。 | -| `webrtc_transport.py` | 369 | 共用 WebRTC 管線：asyncio 橋接執行緒、螢幕視訊軌、設定。 | +| `webrtc_transport.py` | 411 | 共用 WebRTC 管線：asyncio 橋接執行緒、螢幕視訊軌、設定。 | | `multi_viewer.py` | 339 | 每個連入檢視端各跑一個 `WebRTCDesktopHost` 的協調器。 | | `signaling_server.py` | 427 | 獨立的 WebRTC SDP 交換 rendezvous 服務。 | | `audit_log.py` | 355 | SQLite 雜湊鏈稽核記錄。 | | `host_capture.py` | 297 | TCP 主機的影格與游標產生：螢幕列舉、監視器索引轉擷取區域、預設 JPEG／游標 provider,以及 `FrameProductionMixin`（游標輪詢、擷取迴圈、上線編碼）。 | -| `ws_protocol.py` | 284 | 最小 RFC 6455 WebSocket 框架與握手。 | -| `file_transfer.py` | 339 | 分塊檔案傳輸。 | -| `relay.py` | 314 | NAT 穿透失敗時的 TCP 中繼。 | -| `fingerprint.py` | 245 | TOFU 主機指紋驗證。 | -| `turn_config.py` | 234 | coturn 設定產生器。 | -| `presence.py` | 221 | 多檢視者的執行緒安全在場註冊表。 | -| `jpeg_recorder_encrypted.py` | 223 | AES-GCM 加密版 session 錄影。 | -| `address_book.py` | 213 | 檢視端的主機通訊錄。 | -| `audio.py` / `webrtc_audio.py` / `webrtc_mic.py` | 205 / 189 / 151 | 音訊擷取播放、音訊軌、麥克風上行。 | +| `ws_protocol.py` | 318 | 最小 RFC 6455 WebSocket 框架與握手。 | +| `file_transfer.py` | 371 | 分塊檔案傳輸。 | +| `relay.py` | 315 | NAT 穿透失敗時的 TCP 中繼。 | +| `fingerprint.py` | 246 | TOFU 主機指紋驗證。 | +| `turn_config.py` | 249 | coturn 設定產生器。 | +| `presence.py` | 238 | 多檢視者的執行緒安全在場註冊表。 | +| `jpeg_recorder_encrypted.py` | 239 | AES-GCM 加密版 session 錄影。 | +| `address_book.py` | 229 | 檢視端的主機通訊錄。 | +| `audio.py` / `webrtc_audio.py` / `webrtc_mic.py` | 243 / 207 / 155 | 音訊擷取播放、音訊軌、麥克風上行。 | | `webrtc_files.py` | 249 | 專屬 DataChannel 的分塊檔案傳輸。 | -| `webrtc_host_auth.py` | 237 | 檢視端認證與核准：token 檢查、信任清單／IP 白名單自動放行、手動接受／拒絕、SAS、逾時關閉。 | -| `lan_discovery.py` | 189 | mDNS／Zeroconf 區網探索。 | -| `video_codec.py` | 181 | TCP／WS 路徑的可插拔視訊編解碼。 | -| `webrtc_host_media.py` | 194 | 重新協商與 recvonly 軌管理。aiortc 沒有 `removeTransceiver`,所以開／關不對稱——開是加軌重新 offer,關只能設 inactive 並停掉 receiver。 | -| `hw_codec.py` | 169 | 硬體 H.264 編碼偵測與啟用。 | +| `webrtc_host_auth.py` | 239 | 檢視端認證與核准：token 檢查、信任清單／IP 白名單自動放行、手動接受／拒絕、SAS、逾時關閉。 | +| `lan_discovery.py` | 204 | mDNS／Zeroconf 區網探索。 | +| `video_codec.py` | 197 | TCP／WS 路徑的可插拔視訊編解碼。 | +| `webrtc_host_media.py` | 197 | 重新協商與 recvonly 軌管理。aiortc 沒有 `removeTransceiver`,所以開／關不對稱——開是加軌重新 offer,關只能設 inactive 並停掉 receiver。 | +| `hw_codec.py` | 201 | 硬體 H.264 編碼偵測與啟用。 | | `webrtc_stats.py` | 167 | 把 aiortc 的 `RTCStats` 報告輪詢成精簡 dict。 | -| `connect_coordinator.py` | 149 | 由使用者輸入的目標決定該用哪條傳輸。 | +| `connect_coordinator.py` | 158 | 由使用者輸入的目標決定該用哪條傳輸。 | | `adaptive_bitrate.py` | 148 | 依統計調整主機擷取 FPS。 | -| `signaling_client.py` | 151 | 純標準庫的訊令用戶端。 | +| `signaling_client.py` | 164 | 純標準庫的訊令用戶端。 | | `trust_list.py` | 139 | 自動接受的檢視端信任清單。 | | `webrtc_inspector.py` | 138 | 行程級的 `StatsSnapshot` 滾動視窗。 | -| `input_dispatch.py` | 139 | 在主機端套用輸入訊息。 | -| `session_recorder.py` | 134 | 以 PyAV 把 WebRTC 影格錄成 mp4。 | -| `totp.py` | 142 | RFC 6238 TOTP（零外部相依）。 | +| `input_dispatch.py` | 141 | 在主機端套用輸入訊息。 | +| `session_recorder.py` | 139 | 以 PyAV 把 WebRTC 影格錄成 mp4。 | +| `totp.py` | 146 | RFC 6238 TOTP（零外部相依）。 | | `file_sync.py` | 141 | 輪詢式資料夾鏡像。 | -| `transport.py` | 123 | 可插拔的型別化訊息傳輸。 | +| `transport.py` | 126 | 可插拔的型別化訊息傳輸。 | | `host_access.py` | 112 | TCP 主機的檢視端核准與存取控制：`PendingViewer`、權限字串、分享碼的 TOTP 候選值、IP 白名單。`host` 與 `host_client` 共用,所以獨立成模組。 | -| `protocol.py` | 96 | 長度前綴的 TCP 框架。 | +| `protocol.py` | 98 | 長度前綴的 TCP 框架。 | | `resume_tokens.py` / `session_quality_cache.py` / `rate_limit.py` | 94 / 85 / 84 | 快速重連 token、每 session 品質快取、檢視端限流。 | -| `host_id.py` / `viewer_id.py` | 81 / 77 | 主機與檢視端的持久身分。 | -| `permissions.py` / `clipboard_sync.py` / `wake_on_lan.py` / `session_actions.py` / `auth.py` | 64 / 72 / 56 / 40 / 28 | 逐 session 權限、剪貼簿同步、WOL、SAS 注入與螢幕遮蔽、HMAC 挑戰回應。 | +| `host_id.py` / `viewer_id.py` | 83 / 79 | 主機與檢視端的持久身分。 | +| `permissions.py` / `clipboard_sync.py` / `wake_on_lan.py` / `session_actions.py` / `auth.py` | 64 / 74 / 56 / 40 / 28 | 逐 session 權限、剪貼簿同步、WOL、SAS 注入與螢幕遮蔽、HMAC 挑戰回應。 | | `ws_host.py` / `ws_viewer.py` / `jpeg_recorder.py` | 40 / 29 / 146 | WebSocket 傳輸變體與 TCP 路徑錄影。 | -#### `utils/usb/`（4,472 行）與 `utils/usbip/`（945 行） +#### `utils/usb/`（4,524 行）與 `utils/usbip/`（1,008 行） | 檔案 | 行數 | 職責 | | --- | ---: | --- | | `usb/passthrough/session.py` | 642 | 逐 peer 的 USB 直通 session。 | -| `usb/passthrough/viewer_client.py` | 575 | 檢視端的直通協定用戶端。 | -| `usb/passthrough/backend.py` | 463 | 後端 ABC + libusb 實作。 | +| `usb/passthrough/viewer_client.py` | 600 | 檢視端的直通協定用戶端。 | +| `usb/passthrough/backend.py` | 488 | 後端 ABC + libusb 實作。 | | `usb/passthrough/winusb_backend.py` | 488 | Windows WinUSB 後端（ctypes）。 | | `usb/passthrough/acl.py` | 495 | 逐裝置 ACL。 | | `usb/passthrough/iokit_backend.py` | 221 | macOS IOKit 後端。 | | `usb/passthrough/webrtc_channel.py` | 180 | 把直通協定橋到 WebRTC `usb` DataChannel。 | | `usb/passthrough/loopback.py` | 159 | 行程內 loopback 傳輸（測試用）。 | | `usb/passthrough/protocol.py` | 133 | 線路框格式。 | -| `usb/passthrough/descriptor.py` | 132 | USB 標準裝置描述元解析。 | +| `usb/passthrough/descriptor.py` | 134 | USB 標準裝置描述元解析。 | | `usb/passthrough/key_provider.py` | 125 | ACL 的可插拔 HMAC 金鑰來源。 | | `usb/passthrough/commands.py` | 150 | 無頭直通指令（單一真實來源）。 | | `usb/usb_devices.py` | 296 | 跨平台 USB 裝置列舉。 | | `usb/usb_watcher.py` | 260 | 輪詢式 USB 熱插拔監看。 | -| `usbip/protocol.py` | 330 | USB/IP 線路格式封裝／解析。 | -| `usbip/server.py` | 256 | USB/IP 主機端 TCP 伺服器。 | -| `usbip/libusb_backend.py` | 212 | 以 PyUSB／libusb 執行 URB 的正式後端。 | +| `usbip/protocol.py` | 342 | USB/IP 線路格式封裝／解析。 | +| `usbip/server.py` | 295 | USB/IP 主機端 TCP 伺服器。 | +| `usbip/libusb_backend.py` | 224 | 以 PyUSB／libusb 執行 URB 的正式後端。 | | `usbip/backend.py` | 87 | 可插拔 URB 執行後端。 | -#### `utils/rest_api/`（1,793 行） +#### `utils/rest_api/`（1,881 行） | 檔案 | 行數 | 職責 | | --- | ---: | --- | -| `rest_server.py` | 508 | HTTP 前端主體。 | -| `rest_handlers.py` | 486 | 端點實作。 | -| `rest_openapi.py` | 422 | 走訪路由表產生 OpenAPI 3.1 規格。 | +| `rest_server.py` | 549 | HTTP 前端主體。 | +| `rest_handlers.py` | 524 | 端點實作。 | +| `rest_openapi.py` | 431 | 走訪路由表產生 OpenAPI 3.1 規格。 | | `rest_auth.py` | 157 | Bearer token 驗證 + 逐 client 限流閘門。 | | `rest_metrics.py` | 75 | Prometheus 曝露端點。 | | `rest_registry.py` | 75 | 保存執行中 REST 伺服器的行程級單例。 | @@ -831,7 +836,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | 子套件 | 檔案組成 | | --- | --- | | `accessibility/` | `accessibility_api.py`（公開 API）、`element.py`（dataclass）、`tree.py`（遞迴樹傾印）、`recorder.py`（輪詢式事件錄製）、`backends/`：`base.py` 330 行抽象、`windows_backend.py` 801 行（comtypes UIA）、`windows_reads.py` 142 行（pattern／文字範圍／表頭／元素屬性的純讀取與 `UIA_READ_ERRORS`）、`windows_query.py` 176 行（UIA 搜尋起點、可中斷走訪、快取請求、NULL COM 指標判定與 `UIA_ERRORS`）、`windows_state.py` 98 行（控制項狀態讀取與密碼欄位判定）、`macos_backend.py` 125 行（pyobjc AX）、`null_backend.py` fallback | -| `agent/` | `agent_loop.py`、`computer_use.py`、`backends/`：`anthropic.py`、`anthropic_computer_use.py`（435 行）、`openai.py`、`base.py` | +| `agent/` | `agent_loop.py`、`computer_use.py`、`backends/`：`anthropic.py`、`anthropic_computer_use.py`（644 行）、`_computer_toolset.py`（141 行，GA `computer_toolset_20260801` 的批次、依高解析度層級上限縮放截圖、座標換算與 `zoom` 裁切）、`openai.py`、`base.py` | | `ocr/` | `ocr_engine.py`（門面）、`structure.py`（版面）、`backends/`：`tesseract_backend.py`、`easyocr_backend.py`、`paddleocr_backend.py`、`base.py` | | `vision/` | `vlm_api.py`、`backends/`：`anthropic_backend.py`、`openai_backend.py`、`null_backend.py`、`_parse.py`、`base.py` | | `llm/` | `planner.py`、`backends/`：`anthropic_backend.py`、`null_backend.py`、`base.py` | @@ -849,7 +854,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `action_lint/` | `linter.py`、`schema.py`、`__main__.py`（CI 使用） | | `time_travel/` | `controller.py`、`player.py` | | `dag/` | `graph.py`、`runner.py` | -| `run_history/` | `history_store.py`、`artifact_manager.py` | +| `run_history/` | `history_store.py`、`artifact_manager.py`、`run_outcome.py`（排程／觸發／熱鍵執行有動作失敗時判為錯誤） | | `self_healing/` | `locator.py`、`heal_log.py` | | `ab_locator/` | `runner.py`、`store.py` | | `cost_telemetry/` | `pricing.py`、`store.py` | @@ -875,17 +880,20 @@ GUI 是**選用 extra**（`pip install je_auto_control[gui]`，PySide6 + qt-mate | 模組 | 行數 | 職責 | | --- | ---: | --- | | `gui/__init__.py` | 23 | `start_autocontrol_gui()`：**唯一**會延遲匯入 PySide6 的地方，維持頂層套件 Qt-free。 | -| `main_window.py` | 289 | `QMainWindow`：選單列（File／Actions／View／…）、可關閉分頁、即時語言切換、字級預設、qt-material 主題。分頁分為 core／editing／detection／automation／system 五類。 | -| `main_widget.py` | 423 | 擁有 `QTabWidget`，註冊 48 個分頁，並暴露 show/hide/list API 給選單列。核心分頁在註冊時直接宣告 `(label_key, handler)` 動作對；分頁本體都在下列 mixin。 | +| `main_window.py` | 293 | `QMainWindow`：選單列（File／Actions／View／…）、可關閉分頁、即時語言切換、字級預設、qt-material 主題。分頁分為 core／editing／detection／automation／system 五類。 | +| `main_widget.py` | 430 | 擁有 `QTabWidget`，註冊 48 個分頁，並暴露 show/hide/list API 給選單列。核心分頁在註冊時直接宣告 `(label_key, handler)` 動作對；分頁本體都在下列 mixin。 | | `_auto_click_tab.py` | 286 | 自動點擊分頁的 mixin 建構器。 | -| `_screenshot_tab.py` | 136 | 截圖／取像素分頁 mixin。 | -| `_image_detect_tab.py` | 114 | 影像偵測分頁 mixin。 | +| `_screenshot_tab.py` | 137 | 截圖／取像素分頁 mixin。 | +| `_image_detect_tab.py` | 115 | 影像偵測分頁 mixin。 | | `_script_tab.py` | 115 | 腳本執行分頁 mixin。 | | `_record_tab.py` | 110 | 錄製／回放分頁 mixin。 | | `_report_tab.py` | 88 | 報表分頁 mixin。 | | `_i18n_helpers.py` | 66 | 需要即時語言切換的分頁共用的翻譯註冊 mixin。 | -| `language_wrapper/` | 5,007 | 四語系字典（英／日／簡中／繁中）+ `multi_language_wrapper` 執行期切換器與監聽註冊表。 | -| `selector/` | 179 | 拖曳選取螢幕區域的半透明全螢幕覆蓋層與樣板裁切工具（互動式，但都有對應的程式化 API）。 | +| `_validators.py` | 29 | `int_validator()`／`double_validator()`：以 C locale 驗證的數字輸入框 validator，接受的正是 `int()`／`float()` 讀得懂的寫法（預設 locale 在法文、德文下只收小數逗號）。所有數字 `QLineEdit` 都用它。 | +| `_daemon_thread.py` | 79 | `DaemonThread`：`QThread` 的替代品，保留遠端桌面 worker 用到的介面（`start`／`run`／`isRunning`／`wait`／`requestInterruption`／`started`／`finished`），但 `run()` 跑在 daemon `threading.Thread` 上，刪除物件或程式結束都不會銷毀執行中的執行緒。 | +| `_worker_thread.py` | 192 | `start_worker()`：在 daemon `threading.Thread` 上執行 `QObject` worker 的 `run()`（沒有 `QThread` 可被銷毀），並經由分頁擁有的中繼物件回報結果（回呼一律在 GUI 執行緒；worker 沒處理的例外也送到 `on_fail`）；worker 留在模組登錄表直到 GUI 執行緒看到它結束，回傳 `WorkerHandle`（`isRunning()`）；程式結束時先呼叫 worker 的 `request_stop()`，最多等 10 秒，仍在跑的隨行程結束。 | +| `language_wrapper/` | 5,031 | 四語系字典（英／日／簡中／繁中）+ `multi_language_wrapper` 執行期切換器與監聽註冊表。 | +| `selector/` | 227 | 拖曳選取螢幕區域的半透明全螢幕覆蓋層與樣板裁切工具（互動式，但都有對應的程式化 API）。 | > **分頁指令一律走 Actions 選單**：分頁本身只放輸入、表格與結果檢視，指令由視窗層選單暴露。 > 核心分頁在 `main_widget.py` 註冊時宣告動作；功能分頁實作 `menu_actions()`（目前 40 個檔案有此 hook）。 @@ -944,25 +952,25 @@ GUI 是**選用 extra**（`pip install je_auto_control[gui]`，PySide6 + qt-mate | diagnostics | `diagnostics_tab.py` | 91 | 執行子系統檢查並顯示結果。 | | report | `_report_tab.py` | 81 | 產生 HTML／JSON／XML 報表。 | -#### 遠端桌面 GUI（`gui/remote_desktop/`，19 檔／6,393 行） +#### 遠端桌面 GUI（`gui/remote_desktop/`，19 檔／6,608 行） | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `webrtc_panel.py` | 2,530 | WebRTC 子分頁主體。 | -| `webrtc_dialogs.py` | 493 | WebRTC GUI 用的自訂對話框與清單元件（待審檢視者、信任清單、通訊錄、遠端檔案表、稽核記錄、LAN 瀏覽）。 | +| `webrtc_panel.py` | 2,527 | WebRTC 子分頁主體。 | +| `webrtc_dialogs.py` | 519 | WebRTC GUI 用的自訂對話框與清單元件（待審檢視者、信任清單、通訊錄、遠端檔案表、稽核記錄、LAN 瀏覽）。 | | `advanced_group.py` | 92 | 兩個 WebRTC 面板共用的 Advanced STUN／TURN（含選用硬體編碼器）群組，含它寫回面板的 Protocol。 | | `trusted_group.py` | 70 | WebRTC host 面板的信任 viewer 清單群組（移除／清空／匯入／匯出），含它寫回面板的 Protocol。 | -| `connection_screen.py` | 672 | Quick Connect —— AnyDesk 風格單畫面入口。 | -| `viewer_panel.py` | 542 | 「控制另一台機器」子分頁。 | -| `webrtc_known_hosts.py` | 342 | TOFU 釘選庫瀏覽器：`KnownHostsDialog` 與帶外釘選用的小表單。由 `webrtc_dialogs` 再匯出。 | -| `host_panel.py` | 334 | 「分享這台機器」子分頁。 | +| `connection_screen.py` | 704 | Quick Connect —— AnyDesk 風格單畫面入口。 | +| `viewer_panel.py` | 521 | 「控制另一台機器」子分頁。 | +| `webrtc_known_hosts.py` | 346 | TOFU 釘選庫瀏覽器：`KnownHostsDialog` 與帶外釘選用的小表單。由 `webrtc_dialogs` 再匯出。 | +| `host_panel.py` | 353 | 「分享這台機器」子分頁。 | | `frame_display.py` | 228 | 繪製 JPEG 影格並發出遠端輸入事件的元件。 | -| `webrtc_workers.py` | 195 | 訊令流程的背景 `QThread` worker。 | +| `webrtc_workers.py` | 237 | 訊令流程的背景 worker（`DaemonThread`，長輪詢比面板或程式活得久也不會中止行程）。 | | `tab.py` | 165 | 外層容器分頁。 | -| `_helpers.py` | 189 | 面板共用輔助：翻譯、Qt→AC 鍵滑鼠對應、TLS context、狀態徽章、指紋與時間格式化。 | +| `_helpers.py` | 249 | 面板共用輔助：翻譯、Qt→AC 鍵滑鼠對應、TLS context、狀態徽章、指紋與時間格式化。 | | `remote_screen_window.py` | 140 | 檢視端的彈出視窗。 | -| `tray_icon.py` | 98 | WebRTC 主機的系統匣圖示。 | -| `annotation_overlay.py` | 88 | 主機端標註的透明最上層覆蓋。 | +| `tray_icon.py` | 106 | WebRTC 主機的系統匣圖示。 | +| `annotation_overlay.py` | 136 | 主機端標註的透明最上層覆蓋。 | | `sparkline.py` | 77 | WebRTC 統計面板的迷你走勢圖。 | | `blanking_overlay.py` | 71 | 遠端連線期間的隱私遮蔽全螢幕覆蓋。 | | `viewer_screen_window.py` | 46 | 顯示連入檢視端分享畫面的彈出視窗。 | @@ -1060,26 +1068,26 @@ socket 預設綁 `127.0.0.1`；資源一律用 `with`。 | 層／子系統 | 檔案數 | 行數 | | --- | ---: | ---: | -| `gui/` | 91 | 26,821 | -| `utils/mcp_server/` | 31 | 17,675 | -| `utils/remote_desktop/` | 56 | 12,561 | -| `utils/executor/` | 7 | 9,412 | -| `utils/usb/` | 17 | 4,472 | -| `je_auto_control/`（頂層 3 檔） | 3 | 2,395 | -| `utils/accessibility/` | 14 | 3,032 | -| `wrapper/` | 19 | 3,615 | -| `windows/` | 23 | 1,957 | -| `utils/rest_api/` | 8 | 1,793 | -| `utils/agent/` | 8 | 1,446 | -| `linux_with_x11/` | 19 | 1,236 | -| `linux_wayland/` | 17 | 2,870 | -| `utils/triggers/` | 4 | 1,241 | -| `utils/ocr/` | 9 | 1,126 | -| `utils/usbip/` | 5 | 945 | -| `utils/assertion/` | 3 | 881 | -| `osx/` | 17 | 919 | +| `gui/` | 94 | 27,525 | +| `utils/mcp_server/` | 35 | 18,798 | +| `utils/remote_desktop/` | 56 | 12,912 | +| `utils/executor/` | 7 | 9,504 | +| `utils/usb/` | 17 | 4,524 | +| `je_auto_control/`（頂層 3 檔） | 3 | 2,410 | +| `utils/accessibility/` | 14 | 3,117 | +| `wrapper/` | 19 | 3,631 | +| `windows/` | 23 | 1,959 | +| `utils/rest_api/` | 8 | 1,881 | +| `utils/agent/` | 9 | 1,975 | +| `linux_with_x11/` | 19 | 1,281 | +| `linux_wayland/` | 17 | 2,921 | +| `utils/triggers/` | 4 | 1,377 | +| `utils/ocr/` | 9 | 1,140 | +| `utils/usbip/` | 5 | 1,008 | +| `utils/assertion/` | 3 | 887 | +| `osx/` | 17 | 925 | | `autocontrol-lsp/` | 8 | 744 | -| `utils/hotkey/` | 7 | 837 | -| 其餘模組（約 286 個 `utils/` 子套件 + `android/`／`ios/`／周邊小工具） | 677 | 52,882 | -| **總計** | **1,043** | **148,860** | +| `utils/hotkey/` | 7 | 846 | +| 其餘模組（約 286 個 `utils/` 子套件 + `android/`／`ios/`／周邊小工具） | 679 | 54,959 | +| **總計** | **1,053** | **154,324** | diff --git a/docs/source/Eng/doc/mcp_server/mcp_server_doc.rst b/docs/source/Eng/doc/mcp_server/mcp_server_doc.rst index 62d2fdb45..9dba6643b 100644 --- a/docs/source/Eng/doc/mcp_server/mcp_server_doc.rst +++ b/docs/source/Eng/doc/mcp_server/mcp_server_doc.rst @@ -97,8 +97,12 @@ read-only, and a read-only tool given a ``db`` that does not exist answers with an empty result instead of creating the file. ``ac_assert_http`` only sends ``GET`` or ``HEAD``. -A ``tools/call`` argument that the tool's input schema does not declare is -refused with ``-32602`` (invalid params) before the tool runs. +Arguments that fail the tool's input schema -- a missing or mistyped +property, a value outside an ``enum``, or one the schema does not declare -- +are refused before the tool runs, as a tool execution error: a result with +``isError: true`` whose text says what was wrong, so the model can retry with +corrected arguments (MCP 2025-11-25). An unknown tool or a request that is not +a ``tools/call`` at all is still a ``-32602`` protocol error. Resources, prompts, sampling ============================ @@ -130,7 +134,10 @@ Logging notifications, progress, cancellation - The project logger is forwarded to the client as ``notifications/message`` while a stdio session is active. - Clients can retune the level with ``logging/setLevel``. + Clients can retune the level with ``logging/setLevel``. A 2026-07-28 + request gets the records it produces, and only when it sets + ``io.modelcontextprotocol/logLevel`` (see `Stateless requests + (2026-07-28)`_). - Long-running tools that accept a ``ctx`` parameter receive a :class:`ToolCallContext` and can call ``ctx.progress(value, total, message)`` to push @@ -190,8 +197,13 @@ exits — useful in CI smoke tests and prompt prep: je_auto_control_mcp --list-tools --read-only je_auto_control_mcp --list-resources je_auto_control_mcp --list-prompts + je_auto_control_mcp --read-only # serve only the read-only tools je_auto_control_mcp --fake-backend # swap in the in-memory backend +One ``--list-*`` flag prints its array; several print one object keyed +``tools`` / ``resources`` / ``prompts``. Output, and the stdio server's +messages, are UTF-8 whatever the console's code page. + Registering with Claude Desktop =============================== @@ -256,8 +268,10 @@ box), start the same dispatcher behind HTTP: ``application/json`` by default; if ``Accept`` includes ``text/event-stream`` the response streams progress notifications followed by the final result as SSE events. -- Missing / wrong ``Authorization: Bearer `` returns 401 / - 403 (constant-time compare via ``hmac.compare_digest``). +- A missing or wrong ``Authorization: Bearer `` returns 401 + with a ``WWW-Authenticate: Bearer`` challenge (``error="invalid_token"`` + when a wrong token was sent), as the MCP authorization specification + requires; the compare is constant-time (``hmac.compare_digest``). - ``ssl_context`` wraps the listening socket so the same transport can serve HTTPS. - The default bind is ``127.0.0.1`` per the project's @@ -274,14 +288,27 @@ and are unaffected. To let a browser-based client on another origin in, list its exact origins in ``JE_AUTOCONTROL_MCP_ALLOWED_ORIGINS`` (comma-separated, e.g. ``https://tool.example:8443``). -With ``JE_AUTOCONTROL_MCP_CONFIRM_DESTRUCTIVE=1``, a client that advertised -``elicitation`` must have its session's event stream open for a destructive -call to be confirmed; without one the call is refused rather than run. Only +With ``JE_AUTOCONTROL_MCP_CONFIRM_DESTRUCTIVE=1``, a handshake-era client that +advertised ``elicitation`` must have its session's event stream open for a +destructive call to be confirmed; without one the call is refused rather than run. Only the session a prompt was sent to can answer it. Sessions ======== +``initialize`` agrees on a protocol version: the client's, when it is one the +server speaks (``2025-11-25``, ``2025-06-18``, ``2025-03-26``, ``2024-11-05``), +otherwise the newest of those. A 2025-11-25 client also gets a ``description`` +in ``serverInfo``. 2026-07-28 drops ``initialize`` altogether and is served +per request instead (see `Stateless requests (2026-07-28)`_); an +``initialize`` naming it gets 2025-11-25. Over HTTP a request whose +``MCP-Protocol-Version`` header names a version the server does not speak is +refused with 400 and an ``UnsupportedProtocolVersion`` (``-32022``) error +listing the ones it does. The server declares only server +capabilities (tools, resources, prompts, logging); it sends +``sampling/createMessage``, ``roots/list`` and ``elicitation/create`` only to a +client that declared the matching capability. + ``initialize`` mints a session and returns it in an ``Mcp-Session-Id`` response header. Echo that header on every later request and the server keeps one scope for you — the capabilities @@ -311,10 +338,80 @@ offer. (a standing stream keeps its own session fresh), and the registry evicts the least recently seen once it holds 128. +Stateless requests (2026-07-28) +=============================== + +The server speaks both protocol eras and decides per request. A request +whose ``params._meta`` carries ``io.modelcontextprotocol/protocolVersion`` +is served statelessly, from that request alone; ``initialize``, and every +request without the key, is served as described under `Sessions`_. So an +existing client keeps working unchanged next to a 2026-07-28 one. + +- **Per-request fields.** ``io.modelcontextprotocol/clientCapabilities`` + (an object) is required next to the version; ``clientInfo`` and + ``logLevel`` are optional. A missing or malformed field is ``-32602``. + A version the server does not serve statelessly is ``-32022``, whose + ``data`` lists ``supported`` (``2026-07-28`` first, then the + handshake-era versions, which need ``initialize``) and ``requested``. +- **``server/discover``** answers ``supportedVersions``, the server's + ``capabilities`` (tool-list changes and resource subscriptions, both + through ``subscriptions/listen``) and its identity. Without the per-request fields it is + ``-32602``. +- **Methods.** ``tools/list``, ``tools/call``, ``resources/list``, + ``resources/read``, ``prompts/list``, ``prompts/get`` and + ``subscriptions/listen``. The revision removed ``ping``, + ``logging/setLevel`` and ``resources/(un)subscribe``; they are ``-32601`` + in a stateless request. +- **``subscriptions/listen``** opens one subscription per request. Its + ``notifications`` filter may ask for ``toolsListChanged`` and for + ``resourceSubscriptions`` (a list of URIs; ``autocontrol://screen/live`` is + the subscribable one). The first message is + ``notifications/subscriptions/acknowledged`` with the part the server + will send: ``promptsListChanged`` and ``resourcesListChanged`` are left + out, since those lists never change, and so is a URI that cannot be + subscribed. Every notification after it carries the request's id under + ``_meta["io.modelcontextprotocol/subscriptionId"]``. The request gets an + answer only when the server ends the subscription (``serve_stdio`` + finishing, ``HttpMCPServer.stop()``): a ``complete`` result with the same + ``_meta``. The client ends it with ``notifications/cancelled`` on stdio or + by closing the stream over HTTP, which needs ``Accept: text/event-stream`` + (406 otherwise). An id that is already listening is ``-32600``, and a + malformed filter is ``-32602``. +- **Results.** Every result carries ``resultType`` (``complete``, or + ``input_required`` below) and the server's name, version and description + under ``_meta["io.modelcontextprotocol/serverInfo"]``. ``server/discover``, + the three lists and ``resources/read`` also carry caching hints: + ``cacheScope`` is always ``private``; ``ttlMs`` is an hour for + ``server/discover``, a minute for the lists, and ``0`` for + ``resources/read``, whose content is live. +- **Nothing the client did not ask for.** The capabilities the gates read + are the request's own, not a connection's; the server sends no request of + its own (``request_sampling`` and ``refresh_roots`` raise inside a + stateless request); log records go out only for a request that set + ``logLevel``, at that level or above; and a stdio peer whose first + request was stateless is sent no background log records, and list changes + and resource updates only through its ``subscriptions/listen``. +- **Confirmation** of destructive tools is a multi round-trip: see + `Confirmation prompts (elicitation)`_. +- **Over HTTP** a request is stateless when its ``MCP-Protocol-Version`` + header or its ``_meta`` says 2026-07-28. It must mirror its body into + headers: ``MCP-Protocol-Version`` equal to the ``_meta`` version, + ``Mcp-Method`` equal to ``method``, and for ``tools/call`` / ``prompts/get`` + / ``resources/read`` also ``Mcp-Name`` equal to the tool or prompt name or + the resource URI (``=?base64?...?=`` for a value that is not plain ASCII). + A missing or disagreeing header is 400 with ``HeaderMismatch`` + (``-32020``); a bad version, missing metadata or a missing client + capability is 400; an unknown method is 404. No session is kept: + ``Mcp-Session-Id`` is ignored and none is minted, and ``GET`` / ``DELETE`` + naming 2026-07-28 are 405. A plain JSON ``POST`` works for everything, + confirmation included, since the question comes back in the result; an + SSE ``POST`` additionally carries the call's progress notifications. + Read-only / safe mode ===================== -Set ``JE_AUTOCONTROL_MCP_READONLY=1`` (or pass ``read_only=True`` to +Set ``JE_AUTOCONTROL_MCP_READONLY=1`` (or pass ``--read-only`` to +``je_auto_control_mcp``, or ``read_only=True`` to :func:`build_default_tool_registry`) to drop every tool whose ``readOnlyHint`` is false. Only observers (positions, OCR queries, clipboard reads, history, ...) survive: @@ -355,6 +452,20 @@ at that moment. What that means per transport: comes back as a separate ``POST`` — the client is busy reading the stream it asked on. +A **2026-07-28** request is never sent ``elicitation/create``. The first +call is answered with ``resultType: "input_required"``: the question under +``inputRequests["confirm"]`` and a ``requestState``. The client asks the +user and retries the same call, with the same arguments, the answer in +``inputResponses["confirm"]`` (for example ``{"action": "accept"}``) and the +``requestState`` echoed back. The state is signed with a key that lives only +in the server process, names the tool and a digest of its arguments, +expires after five minutes and is accepted once; a state that fails any of +that is ``-32602``. Over HTTP this needs no session and no open stream. ``decline`` or ``cancel`` is a tool execution error +(``isError: true``) and the tool does not run; a retry without an answer is +asked again. A stateless client that did not declare ``elicitation`` gets +``-32021`` with ``data.requiredCapabilities`` naming it, instead of the +handshake era's unprompted run. + .. warning:: A client that ignores ``Mcp-Session-Id``, or that only ever sends diff --git a/docs/source/Eng/doc/new_features/new_features_doc.rst b/docs/source/Eng/doc/new_features/new_features_doc.rst index 3748f85f4..08cd73f77 100644 --- a/docs/source/Eng/doc/new_features/new_features_doc.rst +++ b/docs/source/Eng/doc/new_features/new_features_doc.rst @@ -256,6 +256,11 @@ keyboard focus, or ``None`` when nothing is focused; with ``app_name`` it is ``None`` unless the focused element belongs to that application. The accessibility recorder follows this element. +Matches with an on-screen rectangle come first; one without (a control on a +hidden tab page reports ``(0, 0, 0, 0)``) comes last, and +``click_accessibility_element`` returns ``False`` rather than click it at the +corner of the screen. + Raises ``AccessibilityNotAvailableError`` on platforms where no backend is installed. Action-JSON commands: ``AC_a11y_list``, ``AC_a11y_find``, ``AC_a11y_find_all``, ``AC_a11y_focused``, ``AC_a11y_click``. GUI: @@ -277,6 +282,12 @@ pixel coordinates:: screen_region=[0, 800, 1920, 1080], # optional crop ) +``None`` / ``False`` means the model did not find the element. A request that +fails (network, authentication, rate limit) raises ``VLMRequestError`` instead +of reading as "not found". The Anthropic backend sends the capture fitted to +the model's image limits and maps the reply back to its pixels; replies such as +``x=512, y=300``, ``{"x": 512, "y": 300}`` or ``512.4, 300.6`` are all read. + Backends (loaded lazily, zero imports at package import time): - Anthropic (``anthropic`` SDK, ``ANTHROPIC_API_KEY``) @@ -525,7 +536,10 @@ GUI: **Remote Desktop** tab opens to the **Quick Connect** screen (AnyDesk-style) by default — huge Host ID on one side, a single input that accepts ``host:port``, ``ws://``, ``wss://``, or a 9-digit Host ID on the other, with *Connect* and *Start hosting* as the two primary -buttons. Recent connections are remembered across sessions. Advanced +buttons. The session opens in a popup window that forwards your mouse +and keyboard to the host. A ``wss://`` target verifies the host's +certificate against the system trust store; for a self-signed host, use +the Advanced viewer and tick *Skip cert verification (self-signed)*. Recent connections are remembered across sessions. Advanced per-transport sub-tabs (legacy TCP / WS host + viewer, WebRTC host + viewer with manual SDP / custom codecs / TLS pinning) stay one click away. WebRTC sub-tabs lazy-load so a stock install without the @@ -1216,6 +1230,10 @@ land in the variable bag:: [["AC_shell_command", {"command": "curl -H \"Authorization: Bearer ${secrets.github_token}\" ..."}]] +``AC_shell_command`` logs only the program it starts, never its arguments, +so a filled-in secret stays out of the log; a program that cannot start +fails the action. + GUI: **Secrets** tab — initialize the vault, unlock it, add / remove entries, change passphrase. The vault file is created with mode 0o600 on POSIX systems; on Windows the default ACL already restricts @@ -1227,7 +1245,8 @@ Webhook (HTTP push) trigger A bundled :mod:`http.server` dispatcher fires an action script when an external service POSTs to a registered path. Configure path, allowed -methods, and an optional bearer token; the request method, path, query, +methods (``GET``, ``POST``, ``PUT``, ``PATCH``, ``DELETE``; any other verb +is refused when the webhook is added), and an optional bearer token; the request method, path, query, headers, raw body, and parsed JSON are seeded into the variable scope:: import je_auto_control as ac @@ -1258,7 +1277,8 @@ Action-JSON commands:: Each fire is recorded in run history as ``trigger`` with source id ``webhook:`` so the dashboard surfaces webhook activity alongside -other triggers. The body is capped at 1 MiB and bearer-token comparison +other triggers; a fire in which any action failed is recorded as an error +(with an error snapshot), as scheduled, triggered and hotkey runs are. The body is capped at 1 MiB and bearer-token comparison uses :func:`hmac.compare_digest`. Bind to ``127.0.0.1`` unless the listener genuinely needs to be reachable from elsewhere on the network. diff --git a/docs/source/Eng/doc/new_features/v103_features_doc.rst b/docs/source/Eng/doc/new_features/v103_features_doc.rst index da18c566b..4231db118 100644 --- a/docs/source/Eng/doc/new_features/v103_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v103_features_doc.rst @@ -39,6 +39,9 @@ Executor commands ``AC_idempotency_begin`` registers/looks up a ``key`` in a named store (optional ``request`` for conflict detection); ``AC_idempotency_complete`` stores the -``response``. Both use a named-instance registry (like circuit breakers / -bulkheads) and are exposed as MCP tools (``ac_idempotency_begin`` / -``ac_idempotency_complete``) and as Script Builder commands under **Flow**. +``response``; ``AC_idempotency_release`` drops an ``in_progress`` key whose work +failed so a retry runs it (a completed key is kept). All three use a +named-instance registry (like circuit breakers / bulkheads), which has no TTL, +and are exposed as MCP tools (``ac_idempotency_begin`` / +``ac_idempotency_complete`` / ``ac_idempotency_release``) and as Script Builder +commands under **Flow**. diff --git a/docs/source/Eng/doc/new_features/v109_features_doc.rst b/docs/source/Eng/doc/new_features/v109_features_doc.rst index c9e5613eb..fd290dc0c 100644 --- a/docs/source/Eng/doc/new_features/v109_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v109_features_doc.rst @@ -28,9 +28,9 @@ Headless API is_mixed_script("pаypal") # True (Latin + Cyrillic) scripts_of("pаypal") # {'LATIN', 'CYRILLIC'} -``confusable_skeleton`` NFKC-normalises (folding fullwidth, ligatures and math -alphanumerics) then maps each remaining cross-script lookalike to its Latin -prototype; invisible format characters (zero-width space, soft hyphen, +``confusable_skeleton`` follows UTS #39: it decomposes with NFKD (folding fullwidth, +ligatures, math alphanumerics and accents), maps each remaining cross-script +lookalike to its Latin prototype, and decomposes again with NFD; invisible format characters (zero-width space, soft hyphen, joiners) are dropped first. ``is_confusable`` is true only for *distinct* strings with equal skeletons. ``detect_homoglyphs`` returns the offending characters with their position and prototype. ``scripts_of`` / ``is_mixed_script`` classify characters diff --git a/docs/source/Eng/doc/new_features/v10_features_doc.rst b/docs/source/Eng/doc/new_features/v10_features_doc.rst index 59c3e6de6..176406f30 100644 --- a/docs/source/Eng/doc/new_features/v10_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v10_features_doc.rst @@ -62,7 +62,9 @@ Two failure kinds, mirroring REFramework: ``kind="business"``. ``stats()`` returns per-status counts (``new`` / ``in_progress`` / -``success`` / ``failed``) for dashboards and run reports. +``success`` / ``failed``) for dashboards and run reports. A database that +cannot be opened or used (not a SQLite file, locked past the timeout) raises +``WorkQueueError``, an ``AutoControlException``. Executor commands diff --git a/docs/source/Eng/doc/new_features/v112_features_doc.rst b/docs/source/Eng/doc/new_features/v112_features_doc.rst index 49fcd3855..cd62ef6a4 100644 --- a/docs/source/Eng/doc/new_features/v112_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v112_features_doc.rst @@ -25,7 +25,8 @@ Headless API format_list(["A", "B", "C", "D"], locale="fr") # 'A, B, C et D' ``style`` is ``"and"`` (conjunction), ``"or"`` (disjunction) or ``"unit"`` -(comma-separated, no conjunction). ``locale`` selects the conjunction word and +(measurements such as "3 ft, 7 in": commas only in English; CLDR's unit +pattern, which ends with the conjunction, in ``es`` / ``fr`` / ``pt`` / ``de``). ``locale`` selects the conjunction word and the serial-comma rule (``en`` / ``es`` / ``fr`` / ``de`` / ``pt``; English uses the Oxford comma, the others do not; an unknown locale falls back to English). One and two element lists, and the empty list, are handled as special cases. diff --git a/docs/source/Eng/doc/new_features/v113_features_doc.rst b/docs/source/Eng/doc/new_features/v113_features_doc.rst index 8ff5831f1..fe0d24189 100644 --- a/docs/source/Eng/doc/new_features/v113_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v113_features_doc.rst @@ -34,7 +34,8 @@ Supported: simple ``{name}`` arguments, ``select`` (e.g. gender), ``plural`` and ``selectordinal`` with the CLDR categories (``zero``/``one``/``two``/``few``/ ``many``/``other``), exact ``=N`` selectors that win over a category, the ``#`` count placeholder, a plural ``offset:`` (``#`` becomes count − offset), nested -arguments, and ICU apostrophe quoting (``''`` → ``'``; ``'{'`` → literal brace). +arguments, and ICU apostrophe quoting (``''`` → ``'``; ``'{'`` → literal brace; +``'#'`` only inside a plural, elsewhere the apostrophes stay). ``plural_rules`` / ``ordinal_rules`` let you inject custom category functions; ``locale`` selects the built-ins (``en``, ``fr``). diff --git a/docs/source/Eng/doc/new_features/v130_features_doc.rst b/docs/source/Eng/doc/new_features/v130_features_doc.rst index 2e43812a6..bd0610491 100644 --- a/docs/source/Eng/doc/new_features/v130_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v130_features_doc.rst @@ -29,7 +29,9 @@ Headless API for box in ssim_changed_regions("golden.png", ignore=[[0, 0, 120, 30]]): print(box["x"], box["y"], box["width"], box["height"]) -``ssim_compare`` returns the mean SSIM over the image (``1.0`` = identical); +``ssim_compare`` returns the mean SSIM over the image, in ``-1..1`` (``1.0`` = +identical), with the constants scaled to the images' dynamic range (255 for 8-bit, +1.0 for 0..1 floats); ``current`` defaults to a screen grab of the optional ``region``. ``ignore`` is a list of ``[x, y, w, h]`` boxes excluded from the score and from change detection. ``ssim_changed_regions`` flags pixels where local dissimilarity ``1 - SSIM`` diff --git a/docs/source/Eng/doc/new_features/v149_features_doc.rst b/docs/source/Eng/doc/new_features/v149_features_doc.rst index 2ed717dff..c41898dd5 100644 --- a/docs/source/Eng/doc/new_features/v149_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v149_features_doc.rst @@ -6,8 +6,10 @@ Perceptual (YIQ) Image Diff with Anti-Alias Suppression metric, and neither ignores **anti-aliased edges** — the #1 source of false-positive visual-diff failures across DPI and font-hinting. ``perceptual_diff`` compares pixels in YIQ space (the pixelmatch colour metric, far closer to human perception than RGB) -and, by default, removes the thin one-pixel edge differences that anti-aliasing -produces (a morphological open), so only *solid* changed regions count. +and, by default, discounts the pixels pixelmatch's anti-aliasing test classifies as +anti-aliasing (a pixel between a darker and a brighter neighbour, next to a flat +area in both images), so a re-rendered edge does not count while a thin real +change, such as edited small text or a 1 px rule, still does. Runs on an injectable image pair (ndarray / path / PIL), so it is headless-testable on synthetic arrays. OpenCV + NumPy come in via ``je_open_cv``; reuses the shared diff --git a/docs/source/Eng/doc/new_features/v157_features_doc.rst b/docs/source/Eng/doc/new_features/v157_features_doc.rst index 13e25a7f5..e59d59fc4 100644 --- a/docs/source/Eng/doc/new_features/v157_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v157_features_doc.rst @@ -30,8 +30,10 @@ Headless API ``read_barcodes(source=None, *, region=None, decoder=None)`` returns a list of ``{"text", "type", "points"}`` dicts, one per detected barcode (``points`` is the -four-corner polygon in image coordinates). ``source`` may be an image path or an -array; when omitted the screen (optionally cropped to ``region``) is grabbed. The +four-corner polygon: in the image's coordinates for a given ``source``, in screen +coordinates for a screen grab, ``region`` included). ``source`` may be an image +path or an array; when omitted the screen (optionally cropped to ``region``) is +grabbed. The grayscale conversion reuses the shared ``visual_match`` haystack loader, so no new image-loading code is added. diff --git a/docs/source/Eng/doc/new_features/v21_features_doc.rst b/docs/source/Eng/doc/new_features/v21_features_doc.rst index cd48da950..726d3b69c 100644 --- a/docs/source/Eng/doc/new_features/v21_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v21_features_doc.rst @@ -30,7 +30,8 @@ variables:: On normal completion the checkpoint is cleared. A failing step raises and leaves the checkpoint on that step, so the next call runs it again. The store is injectable, so resume is unit-tested deterministically without a real crash: -``CheckpointStore.save`` / ``load`` / ``clear``. +``CheckpointStore.save`` / ``load`` / ``clear``. A database that cannot be opened +or used raises ``CheckpointStoreError``, an ``AutoControlException``. Executor / MCP commands: diff --git a/docs/source/Eng/doc/new_features/v27_features_doc.rst b/docs/source/Eng/doc/new_features/v27_features_doc.rst index daa1237c1..77224fdb4 100644 --- a/docs/source/Eng/doc/new_features/v27_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v27_features_doc.rst @@ -39,6 +39,7 @@ Secret scan Walks a JSON-like structure and flags string values that look like secrets — by key name (``password`` / ``token`` / ``api_key`` …), by value pattern (AWS / GitHub tokens, private-key blocks), or by high Shannon entropy — that -should reference the vault (``${secrets.NAME}``). Values already referencing -the vault are ignored; previews are masked. Exposed as ``AC_scan_secrets`` / +should reference the vault (``${secrets.NAME}``). A value that is only a +placeholder (``${secrets.NAME}``) is ignored; one that merely starts with one is +still scanned; previews are masked. Exposed as ``AC_scan_secrets`` / ``ac_scan_secrets``. diff --git a/docs/source/Eng/doc/new_features/v2_features_doc.rst b/docs/source/Eng/doc/new_features/v2_features_doc.rst index f19adcb9a..575fb9bb7 100644 --- a/docs/source/Eng/doc/new_features/v2_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v2_features_doc.rst @@ -120,7 +120,11 @@ Per-call LLM token + USD log with day / model / provider roll-up:: summary = summarise_llm_costs() print(summary.total_usd, summary.by_model) -Pricing table covers Claude 4.x and OpenAI; override per-call. +``summarise_llm_costs()`` with no argument summarises the calls recorded in +``default_cost_store``. The pricing table carries Anthropic's current list +prices for Claude (Fable 5.x, Opus 5.x / 4.x, Sonnet 5 / 4.x, Haiku 4.5 and +older lines; a dated or ``anthropic.``-prefixed id is looked up by its base id) +and OpenAI; override per-call. Executor: ``AC_costs_record / _summary / _list / _clear``. @@ -185,7 +189,10 @@ node is reported as ``skipped`` instead of attempted:: ], }) -Executor: ``AC_run_dag``. GUI: **DAG Runner** tab. +Pass ``stop_event=`` (a ``threading.Event``) to stop a run from another +thread: running nodes finish, and every node not yet started is ``skipped`` +with the error ``"stopped"``. Executor: ``AC_run_dag``. GUI: **DAG Runner** +tab, whose Actions menu has **Stop DAG**. Multi-viewer presence @@ -208,7 +215,13 @@ Computer-use high-level API Wraps :class:`ComputerUseAgentBackend` + :class:`AgentLoop` so a single call drives Anthropic's computer-use tool (``computer_20251124`` on ``claude-opus-5`` by default, sent under its ``computer-use-2025-11-24`` beta; -``tool_type=`` picks another version and ``beta=`` names its beta):: +``tool_type=`` picks another version and ``beta=`` names its beta). With +``model="claude-opus-5-5"``, which accepts nothing else, the backend sends the +GA ``computer_toolset_20260801`` instead: no beta, several actions per turn, +and screenshots scaled into the model's image limits (2576 px on the long +edge and 4784 visual tokens, so a 1080p screen goes unscaled) with the +model's coordinates mapped back to the screen. ``zoom`` is answered with a +full-resolution crop of the region it names:: from je_auto_control import run_computer_use result = run_computer_use( @@ -216,9 +229,15 @@ single call drives Anthropic's computer-use tool (``computer_20251124`` on max_steps=15, wall_seconds=120.0, ) -Auto-detects display size; takes ``max_steps`` + ``wall_seconds`` -budgets so a runaway loop can't drain the API. Executor: -``AC_computer_use``. GUI: **Computer Use** tab. +Auto-detects display size. Screenshots are fitted into the model's image tier +(Claude 4.7 and later: 2576 px / 4784 visual tokens; older models: 1568 px / +1568 tokens) on the beta tool as well, which declares that fitted size as its +display and maps the model's coordinates back to the screen. Takes ``max_steps`` + ``wall_seconds`` +budgets so a runaway loop can't drain the API; setting ``stop_event=`` (a +``threading.Event``) ends the run before its next step, with +``final_message`` ``"stopped"``. Executor: ``AC_computer_use``. GUI: +**Computer Use** tab, whose Actions menu has **Stop**. Closing the window +asks a running job to stop and waits up to 10 seconds for it. WebRunner executor + MCP integration @@ -373,7 +392,8 @@ language and the MCP tool registry. Parameters: * ``goal`` — natural-language objective. * ``backend`` — ``"anthropic"`` (uses ``export_anthropic_tools()`` - with tool-use messages) or ``"openai"`` (uses ``export_openai_tools()`` + with tool-use messages; each screenshot is fitted into the model's image + tier and the ``x`` / ``y`` of a tool call mapped back to the screen) or ``"openai"`` (uses ``export_openai_tools()`` with Chat Completions function calling). * ``max_steps`` (default 25) and ``wall_seconds`` (default 300.0). * ``model`` / ``max_tokens`` — backend-specific overrides. diff --git a/docs/source/Eng/doc/new_features/v34_features_doc.rst b/docs/source/Eng/doc/new_features/v34_features_doc.rst index 7530a5b90..e90cca891 100644 --- a/docs/source/Eng/doc/new_features/v34_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v34_features_doc.rst @@ -12,7 +12,9 @@ The policy supports an **allow** list (default-deny — only matching hosts pass and/or a **deny** list (block these even when otherwise allowed). Patterns are case-insensitive :mod:`fnmatch` globs over the URL hostname, e.g. ``*.example.com`` or ``localhost``. The hostname is matched as urllib will -connect to it: percent-decoded, without a trailing dot, and with an IP literal +connect to it: percent-decoded, IDNA-encoded (a soft hyphen, fullwidth characters +or an ideographic full stop fold away; patterns are encoded the same way), without +a trailing dot, and with an IP literal in any spelling (``2130706433``, ``0x7f.1``, ``[::ffff:127.0.0.1]``) reduced to its usual form. Names are not resolved, so a name that resolves to a denied address is not caught. The module-level policy starts in diff --git a/docs/source/Eng/doc/new_features/v40_features_doc.rst b/docs/source/Eng/doc/new_features/v40_features_doc.rst index 8822f52d7..8a5091131 100644 --- a/docs/source/Eng/doc/new_features/v40_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v40_features_doc.rst @@ -6,8 +6,9 @@ These helpers score similarity, pick the best candidate from a list, and collaps near-duplicates — so a flow can act on "the button that *looks like* Submit" rather than an exact label. -The default backend is the standard library :mod:`difflib`, so the feature works -with **zero extra dependencies**. If the optional ``rapidfuzz`` package is +The default backend is pure Python (named ``difflib`` for compatibility), so the +feature works with **zero extra dependencies**; it computes the same symmetric +Indel ratio as rapidfuzz, ``2 * LCS / (len(a) + len(b))``. If the optional ``rapidfuzz`` package is installed (``pip install je_auto_control[fuzzy]``) it is used instead for speed; scores are normalised to ``0.0..1.0`` either way, so callers never depend on which backend ran. ``BACKEND`` names the active one. Imports no ``PySide6``. diff --git a/docs/source/Eng/doc/new_features/v4_features_doc.rst b/docs/source/Eng/doc/new_features/v4_features_doc.rst index 3b09c8cdc..c1b868985 100644 --- a/docs/source/Eng/doc/new_features/v4_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v4_features_doc.rst @@ -50,7 +50,8 @@ Vision * **Region colour stats** — ``region_color_stats(source, region)`` returns a region's ``average_rgb``, ``dominant_rgb``, and that colour's pixel fraction (quantise colour space → busiest bucket → average its real - pixels). ``AC_region_color_stats``. + pixels). A region reaching past the image is clipped to it. + ``AC_region_color_stats``. * **QR reading** — ``read_qr_codes(source, region)`` decodes QR codes via OpenCV's ``QRCodeDetector`` (no new dependency). ``AC_read_qr``. @@ -61,7 +62,9 @@ Flow control & variables * **Reusable macros** — ``AC_define_macro`` registers a named, parameterised action sub-routine; ``AC_call_macro`` invokes it with ``${arg}`` bindings — the callable function the loop / if primitives - couldn't express. + couldn't express. Parameters are the call's own: after the call (a nested + or recursive one included) the caller's variables of the same names are + back as they were. * **In-process parallel** — ``AC_parallel`` runs branch action lists concurrently, each on a fresh isolated executor so branches never race on shared variables (the in-process complement to the cross-host DAG). @@ -71,7 +74,11 @@ Flow control & variables assertion DSL. * **Read into a variable** — bind external data into the flow scope for later ``${var}`` use: ``AC_ocr_to_var`` (region text), ``AC_shell_to_var`` - (command stdout, decoded with ``encoding`` -- default the locale's), ``AC_read_file_to_var`` (file text), ``AC_http_to_var`` + (command stdout, decoded with ``encoding`` -- default the locale's; a + timeout ends the command and everything it started, and a ``.bat`` / + ``.cmd`` argument holding cmd syntax is refused), ``AC_read_file_to_var`` + (file text; UTF-8 with or without a byte-order mark unless ``encoding`` + says otherwise), ``AC_http_to_var`` (GET body or a dotted JSON path), ``AC_now_to_var`` (strftime), and ``AC_random_to_var`` (seeded int / float / choice). * **Transform a variable** — ``AC_transform_var`` applies upper / lower / @@ -147,7 +154,9 @@ Reporting & notifications * **Desktop notifications** — ``notify(title, message)`` shows a cross-platform toast (``notify-send`` / ``osascript`` / PowerShell); injection-safe (Linux argv, macOS / Windows a static script reading the - strings from environment variables). ``AC_notify``. + strings from environment variables). A notifier that exits non-zero + gives ``shown=False``; Windows toasts use PowerShell's registered app + id, since Windows drops toasts from an unregistered one. ``AC_notify``. GUI diff --git a/docs/source/Eng/doc/new_features/v51_features_doc.rst b/docs/source/Eng/doc/new_features/v51_features_doc.rst index 7937d23ef..cf966623f 100644 --- a/docs/source/Eng/doc/new_features/v51_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v51_features_doc.rst @@ -20,9 +20,12 @@ Syntax Meaning A filter field may be nested (``@.a.b``), and ``[?(@.k)]`` keeps the elements that have ``k``; on an object a filter selects among its member values. The compared value is a JSON number, a quoted string, ``true``, ``false`` or -``null``. Values of different types never compare equal (``true != 1``). +``null``. Values of different types never compare equal (``true != 1``), +and ``<`` / ``>`` order only two numbers or two strings (``<=`` is ``<`` or +``==``, so ``null <= null``). Quoted names and strings decode RFC 9535 +escapes (``['a\'b']`` is the key ``a'b``). A path the subset cannot read -- an unsupported filter or value, a slice -(``[0:2]``) or union (``[0,1]``), an empty ``[]``, an unterminated ``[``, a +(``[0:2]``) or union (``[0,1]``, ``['a','b']``), an empty ``[]``, an unterminated ``[``, a stray character -- raises ``ValueError`` instead of matching something else. Pure standard library (``re``); imports no ``PySide6``. diff --git a/docs/source/Eng/doc/new_features/v55_features_doc.rst b/docs/source/Eng/doc/new_features/v55_features_doc.rst index fe5329e40..e583f22fe 100644 --- a/docs/source/Eng/doc/new_features/v55_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v55_features_doc.rst @@ -5,7 +5,8 @@ The image-redaction module blurs PII in screenshots, but text scraped from a UI, OCR, the clipboard, an LLM prompt/response, or a log line had no string-level equivalent — so PII could leak into action records, audit logs, or a model call. ``detect_pii`` / ``redact_pii_text`` find and mask emails, phone numbers, SSNs, -credit-card numbers, IPv4 addresses, and IBANs over plain text. +credit-card numbers (Luhn-checked), IPv4 addresses, and IBANs (compact or in the +printed groups of four, mod-97-checked) over plain text. Patterns are deliberately simple (no nested quantifiers → no catastrophic backtracking). Pure standard library (``re`` + ``hashlib``); imports no diff --git a/docs/source/Eng/doc/new_features/v59_features_doc.rst b/docs/source/Eng/doc/new_features/v59_features_doc.rst index 58e080fab..0150009b5 100644 --- a/docs/source/Eng/doc/new_features/v59_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v59_features_doc.rst @@ -38,7 +38,8 @@ Headless API ``vex_statement`` validates the inputs: ``status`` must be one of ``VEX_STATUSES`` (``not_affected`` / ``affected`` / ``fixed`` / ``under_investigation``); a ``not_affected`` statement must carry a -``justification`` (one of ``VEX_JUSTIFICATIONS``) or an ``impact_statement``. +``justification`` (one of ``VEX_JUSTIFICATIONS``) or an ``impact_statement``, and an +``affected`` statement an ``action_statement`` (the remediation, as OpenVEX requires). ``build_vex`` wraps statements in an OpenVEX document (pass an explicit ``timestamp`` for a reproducible ``@id``). ``apply_vex`` returns the surviving findings, each non-suppressed match annotated with ``vex_status``. diff --git a/docs/source/Eng/doc/new_features/v5_features_doc.rst b/docs/source/Eng/doc/new_features/v5_features_doc.rst index 8b675ae37..c142cbbb3 100644 --- a/docs/source/Eng/doc/new_features/v5_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v5_features_doc.rst @@ -48,9 +48,13 @@ Turn a recording or an action file into committable, runnable source:: ``target`` is ``pytest`` / ``python`` / ``robot``. The default ``calls`` style maps each ``AC_*`` command to its facade call -(``ac.click_mouse(...)``) and falls back to ``ac.execute_action([...])`` -for flow control and private adapters; the ``actions`` style embeds the -list and replays it through the executor. +(``ac.click_mouse(...)``) and falls back to the executor for flow control, +private adapters and any action holding a ``${...}`` placeholder (only the +executor resolves them); the ``actions`` style embeds the list and replays it +through the executor. Every replay is +``ac.executor.execute_action(..., raise_on_error=True)``, so a generated test +fails at the first failed action. An action list the executor would refuse +(``[1]``, an action with a third element) is refused here too. Executor command: ``AC_generate_code``. CLI: ``je_auto_control codegen``. @@ -116,7 +120,7 @@ Send mail — for example a flow's report — over the standard library:: "username": "bot@x.com", "password": "..."}) TLS is enabled by default (STARTTLS, or implicit SSL when ``use_ssl`` is -set) over a verified default context; supports multiple recipients, CC, +set; the port then defaults to 465 instead of 587) over a verified default context; supports multiple recipients, CC, HTML bodies, and file attachments. Executor command: ``AC_send_email``. diff --git a/docs/source/Eng/doc/new_features/v66_features_doc.rst b/docs/source/Eng/doc/new_features/v66_features_doc.rst index 9b77a45a9..598a7d4e0 100644 --- a/docs/source/Eng/doc/new_features/v66_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v66_features_doc.rst @@ -9,7 +9,8 @@ expander, the calendar layer above cron. Supported rule parts: ``FREQ`` (DAILY/WEEKLY/MONTHLY/YEARLY), ``INTERVAL``, ``COUNT``, ``UNTIL``, ``BYDAY`` (incl. ordinals like ``2MO`` / ``-1FR``), ``BYMONTHDAY`` (incl. negatives), ``BYMONTH``, ``BYSETPOS`` and ``WKST``. -Time-level parts and BYWEEKNO/BYYEARDAY are out of scope. Pure standard library +Time-level parts and BYWEEKNO/BYYEARDAY are out of scope: ``parse_rrule`` raises +``AutoControlException`` for them, and for a rule with both ``COUNT`` and ``UNTIL``. Pure standard library (``datetime`` + ``calendar``); the clock is injectable so ``next_occurrence`` is deterministic. Imports no ``PySide6``. diff --git a/docs/source/Eng/doc/new_features/v68_features_doc.rst b/docs/source/Eng/doc/new_features/v68_features_doc.rst index bfc38fec2..11cd4ec3d 100644 --- a/docs/source/Eng/doc/new_features/v68_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v68_features_doc.rst @@ -42,7 +42,9 @@ Evaluation order mirrors OpenFeature/Unleash/LaunchDarkly: a disabled flag serves ``off_variant`` (reason ``DISABLED``); an unknown flag returns the caller default (reason ``ERROR``); targeting rules are tried in order (``TARGETING_MATCH``); otherwise the fallthrough applies (``DEFAULT`` / -``SPLIT``). Targeting operators include ``eq``/``ne``/``lt``/``gt``/``in``/ +``SPLIT``). A ``serve`` is a variant name, ``{"rollout": {...}}`` or +``{"variant": name}``; anything else (an empty rollout, a missing serve) +serves ``default_variant`` with reason ``ERROR``. Targeting operators include ``eq``/``ne``/``lt``/``gt``/``in``/ ``not_in``/``contains`` and ``semver_*`` (SemVer / PEP 440 precedence: ``1.2`` equals ``1.2.0`` and ``1.0.0-rc.1`` is below ``1.0.0``). Percentage rollout is a consistent-hash bucket of ``sha256("{key}.{salt}.{context_key}")`` so a subject diff --git a/docs/source/Eng/doc/new_features/v76_features_doc.rst b/docs/source/Eng/doc/new_features/v76_features_doc.rst index 0c73bd030..c4c5d7f52 100644 --- a/docs/source/Eng/doc/new_features/v76_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v76_features_doc.rst @@ -34,8 +34,8 @@ Headless API ``tracestate``) tuple. ``new_root_context`` mints a fresh trace; ``child_context`` keeps the trace id and inherited state but allocates a new span id. ``parse_traceparent`` / ``format_traceparent`` round-trip the version-``00`` -header (rejecting bad versions, malformed or all-zero IDs with -``TraceContextError``); ``parse_tracestate`` / ``format_tracestate`` handle the +header (a newer version is read as ``00`` with any extra fields ignored; version +``ff``, malformed or all-zero IDs raise ``TraceContextError``); ``parse_tracestate`` / ``format_tracestate`` handle the vendor list. ``inject_context`` writes the headers; ``extract_context`` reads them back (case-insensitively) and returns ``None`` for a missing or invalid ``traceparent``, so the receiver starts a new trace as W3C Trace Context says. diff --git a/docs/source/Eng/doc/new_features/v79_features_doc.rst b/docs/source/Eng/doc/new_features/v79_features_doc.rst index 7388d658f..395470e3a 100644 --- a/docs/source/Eng/doc/new_features/v79_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v79_features_doc.rst @@ -26,7 +26,7 @@ Headless API ``parse_dotenv`` skips blanks and ``#`` comment lines, strips an optional ``export`` prefix, validates keys, and resolves values: single-quoted values -are literal, double-quoted values process ``\n`` / ``\t`` / ``\\`` / ``\"`` +are literal apart from ``\'`` and ``\\`` (as python-dotenv reads them), double-quoted values process ``\n`` / ``\t`` / ``\\`` / ``\"`` escapes, and unquoted values drop a trailing `` #`` comment and surrounding whitespace. A quoted value ends at its closing quote, so a comment after it is dropped, and it may span several lines. ``dotenv_values`` reads and parses a file; ``load_dotenv`` merges a diff --git a/docs/source/Eng/doc/new_features/v81_features_doc.rst b/docs/source/Eng/doc/new_features/v81_features_doc.rst index 28ae09457..e5b820e65 100644 --- a/docs/source/Eng/doc/new_features/v81_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v81_features_doc.rst @@ -2,7 +2,8 @@ Layered Configuration Resolver ============================== ``json_patch.merge_patch`` merges exactly two documents, ``config_sync`` -resolves by last-write-wins timestamp, and ``AssetStore`` is flat per +resolves by last-write-wins timestamp (a tie goes to a deletion, then to the +same entry on every client, and is reported as a conflict), and ``AssetStore`` is flat per environment. None of them compose an ordered ``defaults < file < env < CLI`` precedence stack with a deep dict merge, nor report *which layer won each key*. This adds that 12-factor resolver. diff --git a/docs/source/Eng/doc/new_features/v86_features_doc.rst b/docs/source/Eng/doc/new_features/v86_features_doc.rst index 19fa168da..d97b4ef87 100644 --- a/docs/source/Eng/doc/new_features/v86_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v86_features_doc.rst @@ -30,7 +30,7 @@ Headless API ``check_foreign_key`` flags non-null child values absent from the parent column (dbt ``relationships``). ``check_unique_key`` reports duplicate single or -composite keys. ``check_accepted_values`` lists non-null values outside the +composite keys; a single-column key ignores nulls, as dbt's ``unique`` does. ``check_accepted_values`` lists non-null values outside the allowed set. ``check_row_count`` verifies the count falls within optional ``minimum`` / ``maximum`` bounds. Each returns an ``ok`` flag plus details. diff --git a/docs/source/Eng/doc/new_features/v89_features_doc.rst b/docs/source/Eng/doc/new_features/v89_features_doc.rst index c8cd2f71c..8a7942459 100644 --- a/docs/source/Eng/doc/new_features/v89_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v89_features_doc.rst @@ -30,7 +30,8 @@ Headless API content_type?}`` dicts), returning ``(content_type, body_bytes)``. Pass an explicit ``boundary`` for a byte-stable body, or call ``new_boundary`` for a fresh token. ``parse_multipart`` reads a body back into ``{fields, files}`` (each -file as ``{name, filename, content_type, content}``). +file as ``{name, filename, content_type, content, content_base64}``: ``content`` +is the part decoded as UTF-8, ``content_base64`` its exact bytes for binary files). Executor commands ----------------- diff --git a/docs/source/Eng/doc/new_features/v8_features_doc.rst b/docs/source/Eng/doc/new_features/v8_features_doc.rst index 41f74076e..4989b8faf 100644 --- a/docs/source/Eng/doc/new_features/v8_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v8_features_doc.rst @@ -34,8 +34,11 @@ Quick start default_popup_watchdog.stop() ``action`` is ``"close"`` (close the matching window) or a key name to -press (``"enter"`` / ``"esc"`` / ...). The guard polls on a background -thread and records every dismissal in ``default_popup_watchdog.hits``. +press (``"enter"`` / ``"esc"`` / ...). A key is pressed only after the popup +has been brought to the front; if Windows refuses that, the rule records an +error rather than press the key in whatever window is active. The guard polls +on a background thread and records every dismissal in +``default_popup_watchdog.hits``. Custom rules diff --git a/docs/source/Eng/doc/new_features/v91_features_doc.rst b/docs/source/Eng/doc/new_features/v91_features_doc.rst index 760ece046..8c65b5dd7 100644 --- a/docs/source/Eng/doc/new_features/v91_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v91_features_doc.rst @@ -7,7 +7,7 @@ login-then-call REST flow could not carry a session headlessly. This parses header; the jar is JSON-serialisable so a session can be saved and reloaded. Pure standard library (``json``); imports no ``PySide6``. The jar is a simple -in-memory name-value store (cookies cleared on ``Max-Age<=0`` / empty value), +in-memory name-value store (cookies cleared on ``Max-Age<=0`` or a past ``Expires``; an empty value is kept), so behaviour is fully deterministic in CI. Headless API @@ -27,7 +27,7 @@ Headless API ``parse_set_cookie`` parses one ``Set-Cookie`` value into ``{name, value, attributes}``. ``CookieJar.update`` applies one or many ``Set-Cookie`` headers -(removing a cookie on an empty value or ``Max-Age<=0``); ``set`` assigns +(removing a cookie on ``Max-Age<=0`` or a past ``Expires``); ``set`` assigns directly; ``cookie_header`` builds the request header; ``to_dict`` / ``from_dict`` and ``save`` / ``load`` persist the jar as JSON. (Domain/path matching is simplified — this is a session-carry jar, not a full RFC 6265 policy engine.) diff --git a/docs/source/Eng/doc/new_features/v95_features_doc.rst b/docs/source/Eng/doc/new_features/v95_features_doc.rst index 11be57bb4..8df4e65c9 100644 --- a/docs/source/Eng/doc/new_features/v95_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v95_features_doc.rst @@ -8,7 +8,8 @@ against declared fields, coercing types and reporting actionable errors — a stdlib analog of pydantic-settings. Pure standard library (``dataclasses``); imports no ``PySide6``. Validation is a -pure function (mapping in, report out), so it is fully deterministic in CI. +function of the mapping and the ``environ`` it is given (default ``os.environ``); +pass ``environ={}`` to make it fully deterministic in CI. Headless API ------------ @@ -26,8 +27,10 @@ Headless API # {"ok": True, "config": {"port": 8080, "env": "dev", "debug": True}, "errors": []} ``ConfigField`` declares a ``type`` (``str`` / ``int`` / ``float`` / ``bool``), -optional ``default``, ``required`` flag, ``choices``, and an ``env`` hint. -``ConfigSchema.validate`` coerces each present value, applies defaults, enforces +optional ``default``, ``required`` flag, ``choices``, and an ``env`` variable +name. A value comes from the mapping, else from that variable, else from the +default (the pydantic-settings order); an ``int`` field refuses a float with a +fraction instead of truncating it. ``ConfigSchema.validate`` coerces each value, applies defaults, enforces required fields and choices, and returns ``{ok, config, errors}`` (errors as ``{field, error}``). ``ConfigSchema.from_dict`` builds a schema from a plain spec, ``validate_config`` does spec-plus-mapping in one call, and ``coerce`` diff --git a/docs/source/Eng/doc/new_features/v9_features_doc.rst b/docs/source/Eng/doc/new_features/v9_features_doc.rst index 121a70b01..2dc9771f9 100644 --- a/docs/source/Eng/doc/new_features/v9_features_doc.rst +++ b/docs/source/Eng/doc/new_features/v9_features_doc.rst @@ -46,8 +46,11 @@ call:: handle_file_dialog("C:/reports/out.csv", action="save") ``action`` is ``open`` / ``save`` / ``folder`` (picking a default dialog -title) or pass an explicit ``window_title``; it waits for the dialog, -types the path, and presses ``confirm_key`` (default Enter). The +title) or pass an explicit ``window_title``; it waits for a window with +exactly that title (case-insensitive), brings it to the front, types the +path, and presses ``confirm_key`` (default Enter). A window that only +contains the title, or one Windows will not bring forward, is not handled: +nothing is typed. The window-wait / type / confirm steps go through an injectable :class:`FileDialogDriver`. Executor command: ``AC_handle_file_dialog``. diff --git a/docs/source/Eng/doc/operations_layer/operations_layer_doc.rst b/docs/source/Eng/doc/operations_layer/operations_layer_doc.rst index 91a516eb7..45fea94da 100644 --- a/docs/source/Eng/doc/operations_layer/operations_layer_doc.rst +++ b/docs/source/Eng/doc/operations_layer/operations_layer_doc.rst @@ -65,6 +65,10 @@ without paying a relay service. Outputs four files: - ``README.txt`` — quick reference with ``turn:`` / ``turns:`` URL, username, secret +A field holding a line break, or a ``user`` holding ``:``, is refused with +``ValueError`` (exit code 2 from the command line): it would otherwise add +its own directives to ``turnserver.conf``. + Headless:: from pathlib import Path @@ -112,6 +116,10 @@ Auth gate never locked out, and the lockout is never global. - A POST body is read only after the route and the token check pass, so unauthenticated requests get 401 / 429 without being parsed. +- A 401 carries ``WWW-Authenticate: Bearer realm="autocontrol"`` (with + ``error="invalid_token"`` when a wrong token was sent). A known path + asked with the other method gets 405 and an ``Allow`` header; an unknown + path gets 404. Headless:: @@ -150,7 +158,9 @@ Read-only (GET): Action (POST): -- ``/execute`` — body ``{"actions": [...]}`` — runs an action list +- ``/execute`` — body ``{"actions": [...], "raise_on_error": false}`` — runs an action list; + with ``raise_on_error`` it stops at the first failing action and answers + ``{"ok": false, "error": ...}`` (success: ``{"ok": true, "result": ...}``) - ``/execute_file`` — body ``{"path": "..."}`` — runs a JSON action file Executor commands:: @@ -209,6 +219,11 @@ Headless:: actions=[["AC_get_mouse_position"]], ) +Pass ``raise_on_error=True`` to have each host stop at its first failing +action and report ``ok: false`` with the error; otherwise ``ok`` only means +the host answered, and action failures are recorded inside ``result``. The +DAG runner's remote nodes and the Admin Console tab both pass it. + Persistence: hosts are saved to ``~/.je_auto_control/admin_hosts.json`` (mode 0600 on POSIX). Reload happens automatically on construction. diff --git a/docs/source/Eng/doc/operations_layer/usb_passthrough_operator_guide.rst b/docs/source/Eng/doc/operations_layer/usb_passthrough_operator_guide.rst index 46d367909..82bc172b7 100644 --- a/docs/source/Eng/doc/operations_layer/usb_passthrough_operator_guide.rst +++ b/docs/source/Eng/doc/operations_layer/usb_passthrough_operator_guide.rst @@ -306,7 +306,8 @@ The same operations are exposed over two more surfaces: * **REST API** — ``GET/POST /usb/passthrough/...``, ``/usb/acl...``, ``/usb/loopback/...``, ``/usb/remote/...`` (bearer-token gated; see ``/openapi.json``). ACL export/import are intentionally *not* on REST - (server-side file paths). + (server-side file paths). ``enabled``, ``allow`` and ``prompt_on_open`` must be JSON + booleans; a string such as ``"false"`` is a 400. * **MCP** — first-class ``ac_usb_*`` tools (``ac_usb_loopback_open`` …) with JSON Schemas, so an agent can call them directly. @@ -319,7 +320,9 @@ What is *not* shipped yet right, list the shared devices over the in-process channel and *Open* one (a descriptor read proves the full stack). The *USB Browser* tab's *Open* button now also works against a **localhost** target via the - same loopback path. + same loopback path. Sharing lasts as long as the panel: when the window + holding it is destroyed, its loopback and hotplug watcher close and the + feature flag it turned on goes back off. - Cross-machine is fully wired: the WebRTC host creates a ``usb`` DataChannel and the viewer exposes ``viewer.usb_client()`` (a ``UsbChannelClient`` with ``list_devices`` / ``open`` / ``resume``). diff --git a/docs/source/Zh/doc/mcp_server/mcp_server_doc.rst b/docs/source/Zh/doc/mcp_server/mcp_server_doc.rst index 9e644612a..71865d2a0 100644 --- a/docs/source/Zh/doc/mcp_server/mcp_server_doc.rst +++ b/docs/source/Zh/doc/mcp_server/mcp_server_doc.rst @@ -1,6 +1,6 @@ -================================ +======================================= MCP 伺服器 (讓 Claude 使用 AutoControl) -================================ +======================================= MCP 伺服器把 AutoControl 包裝成 Model Context Protocol 服務,讓任何 支援 MCP 的客戶端(Claude Desktop、Claude Code、自製 Anthropic / @@ -93,8 +93,10 @@ list-changed 通知與 elicitation。 ``db`` 時回傳空結果,不會建立檔案。``ac_assert_http`` 只送 ``GET`` 或 ``HEAD``。 -``tools/call`` 帶了工具輸入 schema 沒宣告的參數時,會在工具執行前以 -``-32602``(參數無效)拒絕。 +參數不符合工具的輸入 schema 時(缺少或型別錯誤的屬性、不在 ``enum`` 裡的值、schema +沒宣告的參數),會在工具執行前拒絕,並以工具執行錯誤回報:結果帶 ``isError: true``, +文字說明哪裡不對,讓模型能修正參數再試(MCP 2025-11-25)。未知的工具或根本不是 +``tools/call`` 的請求,仍是 ``-32602`` 協定錯誤。 Resources、Prompts、Sampling ============================ @@ -123,7 +125,9 @@ Logging 通知 / Progress / Cancellation - stdio session 期間,專案 logger 會以 ``notifications/message`` 的形式即時推給 client。Client 可用 ``logging/setLevel`` 動態調整 - 等級。 + 等級。2026-07-28 的請求只收到它自己產生的記錄,而且只在它設了 + ``io.modelcontextprotocol/logLevel`` 時才收到(見 `無狀態請求 + (2026-07-28)`_)。 - 接受 ``ctx`` 參數的長時間工具會收到 :class:`ToolCallContext`:呼叫 ``ctx.progress(value, total, message)`` 推送 @@ -180,8 +184,12 @@ CLI 檢視旗標 je_auto_control_mcp --list-tools --read-only je_auto_control_mcp --list-resources je_auto_control_mcp --list-prompts + je_auto_control_mcp --read-only # 伺服器只提供唯讀工具 je_auto_control_mcp --fake-backend # 切換成記憶體版 backend +只給一個 ``--list-*`` 旗標時輸出該陣列;給多個時輸出一個以 ``tools`` / ``resources`` / +``prompts`` 為鍵的物件。不論主控台的碼頁為何,輸出與 stdio 伺服器的訊息一律是 UTF-8。 + 註冊到 Claude Desktop ===================== @@ -244,8 +252,10 @@ HTTP 傳輸(含 SSE / Auth / TLS) - ``POST /mcp`` 接受 JSON-RPC 主體。預設回 ``application/json``; 如果 ``Accept`` 包含 ``text/event-stream``,會以 SSE 串流推送進 度通知,然後送出最終結果。 -- 缺少或錯誤的 ``Authorization: Bearer `` 會回 401 / 403 - (透過 ``hmac.compare_digest`` 做常數時間比對)。 +- 缺少或錯誤的 ``Authorization: Bearer `` 都回 401，並帶 + ``WWW-Authenticate: Bearer`` 挑戰（送了錯誤 token 時加上 + ``error="invalid_token"``），這是 MCP 授權規格的要求；比對透過 + ``hmac.compare_digest`` 以常數時間進行。 - ``ssl_context`` 會包住 socket,讓同一條傳輸支援 HTTPS。 - 預設綁定 ``127.0.0.1``;若要對外,務必同時設定 ``auth_token`` 與(非 localhost 場景)``ssl_context``。 @@ -257,13 +267,23 @@ loopback 時，``Host`` 不是 loopback 名稱的也回 403（防 DNS rebinding 客戶端不送 ``Origin``，不受影響。要讓其他來源的瀏覽器客戶端連線，把完整來源列在 ``JE_AUTOCONTROL_MCP_ALLOWED_ORIGINS``（逗號分隔，例如 ``https://tool.example:8443``）。 -設定 ``JE_AUTOCONTROL_MCP_CONFIRM_DESTRUCTIVE=1`` 時，宣告了 ``elicitation`` 的客戶端 +設定 ``JE_AUTOCONTROL_MCP_CONFIRM_DESTRUCTIVE=1`` 時，宣告了 ``elicitation`` 的握手時代客戶端 必須先開著該 session 的事件串流，破壞性工具才能確認；沒有串流就拒絕執行，而不是直接放行。 確認提示只接受它被送往的那個 session 的回覆。 Session ======= +``initialize`` 會協商協定版本:client 提出的版本若是伺服器支援的(``2025-11-25``、 +``2025-06-18``、``2025-03-26``、``2024-11-05``)就用它,否則用其中最新的。2025-11-25 的 +client 還會在 ``serverInfo`` 拿到 ``description``。完全拿掉 ``initialize`` 的 2026-07-28 +改成逐請求服務(見 `無狀態請求 (2026-07-28)`_);``initialize`` 若指名它,拿到的是 +2025-11-25。走 HTTP 時,``MCP-Protocol-Version`` +標頭寫的若是伺服器不支援的版本,請求會以 400 拒絕,回覆的是列出支援版本的 +``UnsupportedProtocolVersion``(``-32022``)錯誤。伺服器只宣告伺服器端能力(tools、resources、 +prompts、logging);``sampling/createMessage``、``roots/list`` 與 ``elicitation/create`` +只會送給在 initialize 時宣告了對應能力的 client。 + ``initialize`` 會產生一個 session,並用 ``Mcp-Session-Id`` 回應標頭 交給 client。之後每個請求都帶上這個標頭,伺服器就會把它們視為同一個 scope——包含你在 ``initialize`` 聲明的能力,以及進行中呼叫佔用的槽位 @@ -286,11 +306,59 @@ scope——包含你在 ``initialize`` 聲明的能力,以及進行中呼叫佔 - session 有上下界。十分鐘沒被碰過就會被掃掉(常駐串流會讓自己的 session 保持新鮮),而註冊表滿 128 個時,最久沒動的那個會被淘汰。 +無狀態請求 (2026-07-28) +======================= + +伺服器同時支援兩個協定時代,逐請求決定。``params._meta`` 帶著 +``io.modelcontextprotocol/protocolVersion`` 的請求以無狀態方式服務,只看這個請求本身; +``initialize`` 與所有不帶這個鍵的請求,照 `Session`_ 一節的方式服務。所以既有的 +client 不用改,可以和 2026-07-28 的 client 並存。 + +- **逐請求欄位。** 除了版本,還必須有 ``io.modelcontextprotocol/clientCapabilities`` + (物件);``clientInfo`` 與 ``logLevel`` 可省略。欄位缺少或格式不對是 ``-32602``。 + 伺服器不以無狀態方式服務的版本是 ``-32022``,它的 ``data`` 列出 ``supported`` + (``2026-07-28`` 在前,接著是需要 ``initialize`` 的握手時代版本)與 ``requested``。 +- **``server/discover``** 回覆 ``supportedVersions``、伺服器的 ``capabilities`` + (工具清單變更與 resource 訂閱,都經由 ``subscriptions/listen``)與身分。沒有逐請求欄位時是 ``-32602``。 +- **方法。** ``tools/list``、``tools/call``、``resources/list``、``resources/read``、 + ``prompts/list``、``prompts/get`` 與 ``subscriptions/listen``。這個版本移除了 ``ping``、``logging/setLevel`` + 與 ``resources/(un)subscribe``,在無狀態請求裡它們是 ``-32601``。 +- **``subscriptions/listen``** 每個請求開一個訂閱。它的 ``notifications`` 篩選可以要 + ``toolsListChanged`` 與 ``resourceSubscriptions`` (URI 清單;可訂閱的是 + ``autocontrol://screen/live``)。第一則訊息是 ``notifications/subscriptions/acknowledged``, + 列出伺服器會送的部分:``promptsListChanged`` 與 ``resourcesListChanged`` 不列(這兩個清單 + 不會變),無法訂閱的 URI 也不列。之後每則通知都在 + ``_meta["io.modelcontextprotocol/subscriptionId"]`` 帶這個請求的 id。只有伺服器結束訂閱時 + (``serve_stdio`` 結束、``HttpMCPServer.stop()``)這個請求才會收到回覆:一個 ``complete`` + 結果,帶同樣的 ``_meta``。client 在 stdio 用 ``notifications/cancelled`` 結束它,走 HTTP + 則關掉串流;HTTP 需要 ``Accept: text/event-stream``,否則是 406。已經在監聽的 id 是 + ``-32600``,篩選格式錯誤是 ``-32602``。 +- **結果。** 每個結果都帶 ``resultType`` (``complete``,或下面的 ``input_required``), + 並在 ``_meta["io.modelcontextprotocol/serverInfo"]`` 放伺服器的名稱、版本與說明。 + ``server/discover``、三個清單與 ``resources/read`` 另帶快取提示:``cacheScope`` + 一律是 ``private``;``ttlMs`` 在 ``server/discover`` 是一小時、清單是一分鐘、 + ``resources/read`` 是 ``0`` (內容是即時的)。 +- **不送 client 沒要的東西。** 關卡讀的能力是這個請求自己的,不是某條連線的;伺服器 + 不主動送請求(``request_sampling`` 與 ``refresh_roots`` 在無狀態請求裡會丟例外); + 記錄只送給設了 ``logLevel`` 的請求,而且只送該等級以上;第一個請求就是無狀態的 + stdio 對端,不會收到背景記錄,清單變更與 resource 更新也只經由它的 + ``subscriptions/listen`` 送達。 +- 破壞性工具的確認改用多輪往返:見 `破壞性動作確認(Elicitation)`_。 +- **走 HTTP 時**,``MCP-Protocol-Version`` 標頭或 ``_meta`` 寫 2026-07-28 的請求就是無狀態 + 請求。它必須把 body 映到標頭:``MCP-Protocol-Version`` 等於 ``_meta`` 的版本、 + ``Mcp-Method`` 等於 ``method``,``tools/call``/``prompts/get``/``resources/read`` 還要 + ``Mcp-Name`` 等於工具或 prompt 名稱、或 resource URI(不是純 ASCII 的值用 + ``=?base64?...?=``)。標頭缺少或與 body 不符是 400 加 ``HeaderMismatch``(``-32020``); + 版本不對、缺中繼資料或缺 client 能力是 400;未知方法是 404。不保留 session: + ``Mcp-Session-Id`` 會被忽略、也不會發新的;指名 2026-07-28 的 ``GET``/``DELETE`` 是 405。 + 普通的 JSON ``POST`` 就能做所有事,確認也一樣(問題放在結果裡回來);SSE ``POST`` + 另外會送出呼叫的進度通知。 + 唯讀 / 安全模式 =============== -設定 ``JE_AUTOCONTROL_MCP_READONLY=1``(或呼叫 -:func:`build_default_tool_registry` 時傳 ``read_only=True``)只暴 +設定 ``JE_AUTOCONTROL_MCP_READONLY=1``(或對 ``je_auto_control_mcp`` 加上 +``--read-only``,或呼叫 :func:`build_default_tool_registry` 時傳 ``read_only=True``)只暴 露 ``readOnlyHint`` 為 true 的工具(座標、OCR 查詢、剪貼簿讀取、歷 程等): @@ -324,6 +392,17 @@ scope——包含你在 ``initialize`` 聲明的能力,以及進行中呼叫佔 ``elicitation/create``。兩種情況下,答案都要用另一個 ``POST`` 送 回來,因為 client 正忙著讀它問過去的那條串流。 +**2026-07-28** 的請求不會收到 ``elicitation/create``。第一次呼叫的回覆是 +``resultType: "input_required"``:問題放在 ``inputRequests["confirm"]``,另有一個 +``requestState``。client 問過使用者之後,以同樣的參數重送同一個呼叫,把答案放在 +``inputResponses["confirm"]``(例如 ``{"action": "accept"}``),並原樣帶回 +``requestState``。這個 state 以只存在於伺服器行程裡的金鑰簽章,寫明工具與參數摘要, +五分鐘後過期,而且只接受一次;任何一項不符都是 ``-32602``。走 HTTP 時不需要 session, +也不需要開著串流。``decline`` 或 ``cancel`` +是工具執行錯誤(``isError: true``),工具不會執行;沒帶答案的重送會再問一次。沒有 +宣告 ``elicitation`` 的無狀態 client 會收到 ``-32021``,``data.requiredCapabilities`` +寫明缺的能力,而不是像握手時代那樣不經詢問直接執行。 + .. warning:: 如果 client 不回送 ``Mcp-Session-Id``,或是自始至終只送普通的 @@ -388,7 +467,7 @@ Plugin Hot-Reload ``notifications/tools/list_changed``,client 會自動更新工具目錄。 CI 煙霧測試 (Fake Backend) -========================= +========================== Fake backend 把 wrapper 層換成記憶體版的紀錄器,讓沒有顯示伺服器的 CI runner 也能走完所有 MCP 工具: diff --git a/docs/source/Zh/doc/new_features/new_features_doc.rst b/docs/source/Zh/doc/new_features/new_features_doc.rst index 69cfad94e..0961cce4a 100644 --- a/docs/source/Zh/doc/new_features/new_features_doc.rst +++ b/docs/source/Zh/doc/new_features/new_features_doc.rst @@ -242,6 +242,9 @@ Accessibility 元件搜尋 沒有焦點時回傳 ``None``；指定 ``app_name`` 時，焦點元素不屬於該應用程式就回傳 ``None``。無障礙錄製器追蹤的就是這個元素。 +有螢幕矩形的相符元素排在前面；沒有矩形的（隱藏分頁上的控制項回報 ``(0, 0, 0, 0)``） +排在最後，``click_accessibility_element`` 對它回傳 ``False``，不會點到螢幕角落。 + 當前平台若沒有可用後端會拋出 ``AccessibilityNotAvailableError``。 Action-JSON 指令：``AC_a11y_list``、``AC_a11y_find``、``AC_a11y_find_all``、 ``AC_a11y_focused``、``AC_a11y_click``。GUI：**Accessibility** 分頁 @@ -262,6 +265,10 @@ VLM（AI）元件定位 screen_region=[0, 800, 1920, 1080], # 可選：只在此區域搜尋 ) +``None`` / ``False`` 表示模型沒找到元素。請求本身失敗（網路、驗證、速率限制）時會拋出 +``VLMRequestError``，不再當成「找不到」。Anthropic 後端送出的畫面會先縮到模型的影像上限， +回覆再換算回原圖像素；``x=512, y=300``、``{"x": 512, "y": 300}``、``512.4, 300.6`` 這類回覆都讀得到。 + 後端（延遲載入，import ``je_auto_control`` 時不會引入）： - Anthropic (``anthropic`` SDK，``ANTHROPIC_API_KEY``) @@ -494,7 +501,9 @@ GUI：\ **Remote Desktop**\ 分頁預設打開的是 **快速連線** （AnyDesk 風格）— 一邊是超大本機 Host ID，另一邊一個輸入框接受 ``host:port``、 ``ws://``、``wss://`` 或 9 位數 Host ID，搭配 *連線* 與 *開始被遠端* -兩個主要按鈕。近期連線會跨 session 記住。進階的逐傳輸子分頁（既有 +兩個主要按鈕。連線後會開一個彈出視窗，把你的滑鼠與鍵盤轉送到 host。 +``wss://`` 目標會用系統信任的憑證驗證 host；自簽憑證的 host 請改用進階 +viewer，並勾選 *忽略憑證驗證（自簽用）*。近期連線會跨 session 記住。進階的逐傳輸子分頁（既有 TCP / WS host + viewer、WebRTC host + viewer 含手動 SDP / 自訂編碼器 / TLS pinning）仍只差一個 click。WebRTC 子分頁採延遲載入,沒裝 ``[webrtc]`` extra 也能正常開啟整個分頁。 @@ -1140,6 +1149,9 @@ Action JSON 指令:: [["AC_shell_command", {"command": "curl -H \"Authorization: Bearer ${secrets.github_token}\" ..."}]] +``AC_shell_command`` 只記錄它啟動的程式,從不記錄參數,所以填入的 secret 不會進到 log; +程式無法啟動時該動作會失敗。 + GUI: **Secrets** 分頁 — 建立 vault、解鎖、新增 / 移除條目、變更 通行碼。POSIX 系統上 vault 檔以 0o600 建立;Windows 預設 ACL 已限制 只有擁有者能讀取。 @@ -1149,7 +1161,8 @@ Webhook(HTTP push)觸發 ======================= 內建的 :mod:`http.server` dispatcher 在外部服務 POST 到註冊路徑時 -觸發腳本。可設定路徑、允許的方法、可選 bearer token;請求方法、 +觸發腳本。可設定路徑、允許的方法(``GET``、``POST``、``PUT``、``PATCH``、 +``DELETE``;其他方法在註冊時就會被拒絕)、可選 bearer token;請求方法、 路徑、query、headers、原始 body、解析後 JSON 都會種到變數作用域:: import je_auto_control as ac @@ -1180,7 +1193,7 @@ Action JSON 指令:: 每次觸發以 ``trigger`` 來源寫入 run history,source id 為 ``webhook:``,讓 dashboard 把 webhook 活動和其他 trigger 並排 -顯示。Body 上限 1 MiB,bearer token 比對用 +顯示;只要有任何動作失敗,這次觸發就記為錯誤(並附錯誤截圖),排程、觸發器與熱鍵的執行也一樣。Body 上限 1 MiB,bearer token 比對用 :func:`hmac.compare_digest`。除非你真的需要從網路其他地方連入, 否則綁定 ``127.0.0.1``。 diff --git a/docs/source/Zh/doc/new_features/v103_features_doc.rst b/docs/source/Zh/doc/new_features/v103_features_doc.rst index 2419acc33..1249a8f23 100644 --- a/docs/source/Zh/doc/new_features/v103_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v103_features_doc.rst @@ -31,5 +31,7 @@ ---------- ``AC_idempotency_begin`` 在具名儲存中註冊/查找 ``key``(可選 ``request`` 做衝突偵測); -``AC_idempotency_complete`` 儲存 ``response``。兩者使用具名實例登錄(如斷路器/隔艙),並以 MCP 工具 -(``ac_idempotency_begin`` / ``ac_idempotency_complete``)以及 Script Builder 中 **Flow** 分類下的命令提供。 +``AC_idempotency_complete`` 儲存 ``response``;``AC_idempotency_release`` 把工作失敗、仍是 ``in_progress`` +的鍵放掉，讓重試能再跑一次(已完成的鍵會保留)。三者使用具名實例登錄(如斷路器/隔艙，沒有 TTL),並以 MCP 工具 +(``ac_idempotency_begin`` / ``ac_idempotency_complete`` / ``ac_idempotency_release``)以及 Script Builder +中 **Flow** 分類下的命令提供。 diff --git a/docs/source/Zh/doc/new_features/v109_features_doc.rst b/docs/source/Zh/doc/new_features/v109_features_doc.rst index 6343f1a42..399793f84 100644 --- a/docs/source/Zh/doc/new_features/v109_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v109_features_doc.rst @@ -25,8 +25,8 @@ is_mixed_script("pаypal") # True (拉丁 + 西里爾) scripts_of("pаypal") # {'LATIN', 'CYRILLIC'} -``confusable_skeleton`` 先以 NFKC 正規化(折疊全形、連字與數學英數字),再將每個剩餘的跨文字系統仿冒字對映到 -其拉丁原型;不可見的格式字元(零寬空格、軟連字號、連接符)會先被移除。``is_confusable`` 僅在兩個*不同*字串骨架相同時為真。``detect_homoglyphs`` 回傳有問題的字元連同其 +``confusable_skeleton`` 依 UTS #39:先以 NFKD 分解(折疊全形、連字、數學英數字與重音),再將每個剩餘的跨文字系統仿冒字對映到 +其拉丁原型,最後再以 NFD 分解;不可見的格式字元(零寬空格、軟連字號、連接符)會先被移除。``is_confusable`` 僅在兩個*不同*字串骨架相同時為真。``detect_homoglyphs`` 回傳有問題的字元連同其 位置與原型。``scripts_of`` / ``is_mixed_script`` 依 Unicode 區塊將字元分類(忽略數字、標點與空白),因此可單獨 標記一個混用文字系統的權杖。依 UTS #39 的 highly restrictive 等級，拉丁字母搭配漢字 + 平假名 + 片假名 (日文)或漢字 + 諺文(韓文)視為同一套書寫系統，不算混用;全形拉丁字母算拉丁字母。 diff --git a/docs/source/Zh/doc/new_features/v10_features_doc.rst b/docs/source/Zh/doc/new_features/v10_features_doc.rst index 0470b8414..72eed8665 100644 --- a/docs/source/Zh/doc/new_features/v10_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v10_features_doc.rst @@ -57,7 +57,8 @@ Dispatcher / performer :class:`BusinessError` 或傳 ``kind="business"``。 ``stats()`` 回傳各狀態計數(``new`` / ``in_progress`` / ``success`` / -``failed``),供儀表板與執行報告使用。 +``failed``),供儀表板與執行報告使用。資料庫無法開啟或使用(不是 SQLite 檔、鎖定逾時)時丟出 +``WorkQueueError``(屬於 ``AutoControlException``)。 執行器指令 diff --git a/docs/source/Zh/doc/new_features/v112_features_doc.rst b/docs/source/Zh/doc/new_features/v112_features_doc.rst index 4b55bb079..74048d98a 100644 --- a/docs/source/Zh/doc/new_features/v112_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v112_features_doc.rst @@ -20,7 +20,7 @@ format_list(["manzana", "pera", "uva"], locale="es") # 'manzana, pera y uva' format_list(["A", "B", "C", "D"], locale="fr") # 'A, B, C et D' -``style`` 為 ``"and"``(連接)、``"or"``(選擇)或 ``"unit"``(僅以逗號分隔、無連接詞)。``locale`` 選擇連接詞與 +``style`` 為 ``"and"``(連接)、``"or"``(選擇)或 ``"unit"``(度量值，如 "3 ft, 7 in":英文只用逗號;``es`` / ``fr`` / ``pt`` / ``de`` 依 CLDR 單位樣式，結尾用連接詞)。``locale`` 選擇連接詞與 序列逗號規則(``en`` / ``es`` / ``fr`` / ``de`` / ``pt``;英文使用牛津逗號,其餘不使用;未知地區回退為英文)。 一項、兩項與空清單皆以特例處理。未知的 ``style`` 會拋出 ``ValueError``。 diff --git a/docs/source/Zh/doc/new_features/v113_features_doc.rst b/docs/source/Zh/doc/new_features/v113_features_doc.rst index dbb1c5999..156c75866 100644 --- a/docs/source/Zh/doc/new_features/v113_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v113_features_doc.rst @@ -31,7 +31,7 @@ ICU-lite MessageFormat(複數 / 選擇) 支援:簡單 ``{name}`` 參數、``select``(如性別)、``plural`` 與 ``selectordinal`` 搭配 CLDR 類別 (``zero``/``one``/``two``/``few``/``many``/``other``)、優先於類別的精確 ``=N`` 選擇器、``#`` 數量佔位符、 複數 ``offset:``(``#`` 變為 count − offset)、巢狀參數,以及 ICU 單引號跳脫(``''`` → ``'``;``'{'`` → 字面 -大括號)。``plural_rules`` / ``ordinal_rules`` 可注入自訂類別函式;``locale`` 選擇內建規則(``en``、``fr``)。 +大括號;``'#'`` 只在 plural 內跳脫,其他地方單引號照樣保留)。``plural_rules`` / ``ordinal_rules`` 可注入自訂類別函式;``locale`` 選擇內建規則(``en``、``fr``)。 執行器命令 ---------- diff --git a/docs/source/Zh/doc/new_features/v130_features_doc.rst b/docs/source/Zh/doc/new_features/v130_features_doc.rst index d511ca50b..5560803bc 100644 --- a/docs/source/Zh/doc/new_features/v130_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v130_features_doc.rst @@ -24,7 +24,7 @@ SSIM 是標準的視覺回歸度量:容忍輕微光照變化,對結構變化(文 for box in ssim_changed_regions("golden.png", ignore=[[0, 0, 120, 30]]): print(box["x"], box["y"], box["width"], box["height"]) -``ssim_compare`` 回傳整張影像的平均 SSIM(``1.0`` = 完全相同);``current`` 預設為對選用 ``region`` 的螢幕擷取。 +``ssim_compare`` 回傳整張影像的平均 SSIM,範圍 ``-1..1``(``1.0`` = 完全相同),常數依影像的動態範圍縮放(8 位元為 255、0..1 浮點為 1.0);``current`` 預設為對選用 ``region`` 的螢幕擷取。 ``ignore`` 是一組從分數與變化偵測中排除的 ``[x, y, w, h]`` 方框。``ssim_changed_regions`` 標記局部不相似度 ``1 - SSIM`` 超過 ``threshold`` 的像素,將相連者(``min_area`` 以上)分群,回傳 ``{x, y, width, height, area, center}``,由大到小。比較兩張不同尺寸的影像會丟出 ``ValueError``。 diff --git a/docs/source/Zh/doc/new_features/v149_features_doc.rst b/docs/source/Zh/doc/new_features/v149_features_doc.rst index 689e716fd..d5e1bcd78 100644 --- a/docs/source/Zh/doc/new_features/v149_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v149_features_doc.rst @@ -3,8 +3,8 @@ ``visual_regression.image_difference`` 計算原始逐通道最大差像素數,``ssim_compare`` 給出整體結構分數。兩者都未使用 *感知式*色彩度量,也都不忽略**反鋸齒邊緣**——那是跨 DPI 與字體微調時視覺比對誤報的首要來源。``perceptual_diff`` -在 YIQ 空間比較像素(pixelmatch 的色彩度量,比 RGB 更接近人眼感知),並預設移除反鋸齒造成的單像素細邊差異 -(形態學開運算),因此只計算*實心*變化區域。 +在 YIQ 空間比較像素(pixelmatch 的色彩度量,比 RGB 更接近人眼感知),並預設不計入 pixelmatch 反鋸齒判定認定的像素 +(介於較暗與較亮鄰居之間、且兩張影像中都緊鄰平坦區域的像素),因此重新算圖的邊緣不算變化,而細小的真實變化(被改的小字、1 px 的線)仍會計入。 在可注入的影像配對(ndarray / 路徑 / PIL)上執行,因此可對合成陣列做無頭測試。OpenCV + NumPy 透過 ``je_open_cv`` 引入;沿用共用的連通元件輔助與 RGB 載入器。不匯入 ``PySide6``。 diff --git a/docs/source/Zh/doc/new_features/v157_features_doc.rst b/docs/source/Zh/doc/new_features/v157_features_doc.rst index be2729270..02308f750 100644 --- a/docs/source/Zh/doc/new_features/v157_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v157_features_doc.rst @@ -27,8 +27,8 @@ UPC-A / Code-128）的功能——這些正是商品、庫存標籤與物流面 read_barcodes("label.png") ``read_barcodes(source=None, *, region=None, decoder=None)`` 回傳 -``{"text", "type", "points"}`` 字典清單，每偵測到一個條碼一筆（``points`` 為影像 -座標中的四角多邊形）。``source`` 可為影像路徑或陣列；省略時擷取螢幕（可選擇以 +``{"text", "type", "points"}`` 字典清單，每偵測到一個條碼一筆（``points`` 為四角多邊形： +給定 ``source`` 時是該影像的座標，擷取螢幕時是螢幕座標，含 ``region`` 的位移）。``source`` 可為影像路徑或陣列；省略時擷取螢幕（可選擇以 ``region`` 裁切）。灰階轉換重用共用的 ``visual_match`` haystack 載入器，不新增 影像載入程式碼。 diff --git a/docs/source/Zh/doc/new_features/v21_features_doc.rst b/docs/source/Zh/doc/new_features/v21_features_doc.rst index f774e46e0..c22082d5f 100644 --- a/docs/source/Zh/doc/new_features/v21_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v21_features_doc.rst @@ -26,7 +26,8 @@ variables}`` 存入可抽換的儲存後端;之後以相同 ``run_id`` 再執行 result["resumed_from"] # 全新執行為 0;當機後續跑則為 N 正常完成後檢查點會被清除。某一步失敗時會拋出例外，檢查點停在那一步，下次呼叫會重跑它。儲存後端可注入,因此續跑邏輯可在不真的當機的 -情況下做決定性單元測試:``CheckpointStore.save`` / ``load`` / ``clear``。 +情況下做決定性單元測試:``CheckpointStore.save`` / ``load`` / ``clear``。資料庫無法開啟或使用時丟出 +``CheckpointStoreError``(屬於 ``AutoControlException``)。 執行器 / MCP 指令: diff --git a/docs/source/Zh/doc/new_features/v27_features_doc.rst b/docs/source/Zh/doc/new_features/v27_features_doc.rst index 2e00f61fa..bf52f519a 100644 --- a/docs/source/Zh/doc/new_features/v27_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v27_features_doc.rst @@ -37,5 +37,5 @@ fallback_rate, avg_duration_ms, top_brittle}``——在定位器真正失效前, 走訪 JSON 結構並標記看起來像機密的字串值——依鍵名(``password`` / ``token`` / ``api_key`` …)、依值樣式(AWS / GitHub token、私鑰區塊),或 -依高夏農熵——這些應改用保險庫(``${secrets.NAME}``)。已引用保險庫的值會被 -略過;預覽會遮罩。對應 ``AC_scan_secrets`` / ``ac_scan_secrets``。 +依高夏農熵——這些應改用保險庫(``${secrets.NAME}``)。只由一個占位符組成的值(``${secrets.NAME}``)會被 +略過,只是以占位符開頭的值仍會掃描;預覽會遮罩。對應 ``AC_scan_secrets`` / ``ac_scan_secrets``。 diff --git a/docs/source/Zh/doc/new_features/v2_features_doc.rst b/docs/source/Zh/doc/new_features/v2_features_doc.rst index c10723523..4edae758d 100644 --- a/docs/source/Zh/doc/new_features/v2_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v2_features_doc.rst @@ -116,7 +116,9 @@ Executor：``AC_ab_locate / _report / _best_strategy / _clear``。 summary = summarise_llm_costs() print(summary.total_usd, summary.by_model) -內建價格表涵蓋 Claude 4.x 與 OpenAI；可單次呼叫覆寫。 +``summarise_llm_costs()`` 不帶參數時彙總 ``default_cost_store`` 記錄的呼叫。內建價格表是 Anthropic +目前的 Claude 牌價(Fable 5.x、Opus 5.x / 4.x、Sonnet 5 / 4.x、Haiku 4.5 與較舊的系列;帶日期或 +``anthropic.`` 前綴的 id 會以基本 id 查價)與 OpenAI；可單次呼叫覆寫。 Executor：``AC_costs_record / _summary / _list / _clear``。 @@ -178,7 +180,9 @@ Executor：``AC_failure_hook_fire / _list / _clear``。 ], }) -Executor：``AC_run_dag``。GUI：**DAG Runner** 分頁。 +從別的執行緒設定 ``stop_event=``（``threading.Event``）即可停止：執行中的節點會跑完， +尚未開始的節點一律為 ``skipped``、錯誤為 ``"stopped"``。Executor：``AC_run_dag``。 +GUI：**DAG Runner** 分頁，Actions 選單有 **停止 DAG**。 多 viewer 名單 @@ -200,7 +204,10 @@ Computer-use 高階 API 封裝 :class:`ComputerUseAgentBackend` + :class:`AgentLoop`，一次呼叫 即可驅動 Anthropic 的 computer-use tool(預設是 ``claude-opus-5`` 上的 ``computer_20251124``, -以對應的 ``computer-use-2025-11-24`` beta 送出;``tool_type=`` 可換版本,``beta=`` 指定它的 beta):: +以對應的 ``computer-use-2025-11-24`` beta 送出;``tool_type=`` 可換版本,``beta=`` 指定它的 beta)。 +``model="claude-opus-5-5"`` 只接受 GA 的 ``computer_toolset_20260801``,backend 會改送這個形式:不帶 beta、 +一回合可有多個動作,截圖先縮到模型的影像上限內(長邊 2576 px、4784 visual tokens,1080p 螢幕不必縮), +模型給的座標再換算回螢幕座標;``zoom`` 以該區域的全解析度裁切回覆:: from je_auto_control import run_computer_use result = run_computer_use( @@ -208,9 +215,11 @@ Computer-use 高階 API max_steps=15, wall_seconds=120.0, ) -自動偵測螢幕大小；以 ``max_steps`` + ``wall_seconds`` 為預算上限， -避免失控的 loop 把 API 額度耗光。Executor：``AC_computer_use``。 -GUI：**Computer Use** 分頁。 +自動偵測螢幕大小。截圖會縮到模型的影像層級內(Claude 4.7 以後:2576 px／4784 visual tokens;較舊的模型:1568 px／1568 tokens), +beta 工具也一樣:它宣告縮放後的大小為螢幕大小,再把模型給的座標換算回螢幕。以 ``max_steps`` + ``wall_seconds`` 為預算上限， +避免失控的 loop 把 API 額度耗光；設定 ``stop_event=``（``threading.Event``）會在下一步之前結束， +``final_message`` 為 ``"stopped"``。Executor：``AC_computer_use``。 +GUI：**Computer Use** 分頁，Actions 選單有 **停止**。關閉視窗時會請執行中的工作停止，最多等 10 秒。 WebRunner 接入 executor + MCP @@ -355,7 +364,7 @@ helper（``je_auto_control.gui.flow_editor.layout_steps``）可單元 * ``goal`` — 自然語言目標。 * ``backend`` — ``"anthropic"``（透過 ``export_anthropic_tools()`` - 以 tool-use messages 驅動）或 ``"openai"``（``export_openai_tools()`` + 以 tool-use messages 驅動；每張截圖先縮到模型的影像層級內，工具呼叫的 ``x`` / ``y`` 再換算回螢幕）或 ``"openai"``（``export_openai_tools()`` + Chat Completions function calling）。 * ``max_steps``（預設 25）、``wall_seconds``（預設 300.0）。 * ``model`` / ``max_tokens`` — backend 專屬覆寫。 diff --git a/docs/source/Zh/doc/new_features/v34_features_doc.rst b/docs/source/Zh/doc/new_features/v34_features_doc.rst index 0659e11e6..ccf1a4358 100644 --- a/docs/source/Zh/doc/new_features/v34_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v34_features_doc.rst @@ -8,7 +8,7 @@ HTTP 用戶端可連線的主機。它會被每一次 此策略支援**允許(allow)**清單(預設拒絕 —— 僅符合的主機可通過)與/或**拒絕 (deny)**清單(即使其他情況允許也封鎖)。樣式為對 URL 主機名稱進行不分大小寫的 -:mod:`fnmatch` 萬用比對,例如 ``*.example.com`` 或 ``localhost``。比對的是 urllib 實際會連線的主機名稱：先解開百分比編碼、去掉結尾的點，任何寫法的 IP（``2130706433``、``0x7f.1``、``[::ffff:127.0.0.1]``）都換成一般形式；名稱不會解析，所以解析到被拒位址的名稱擋不到。模組層級的策略以 +:mod:`fnmatch` 萬用比對,例如 ``*.example.com`` 或 ``localhost``。比對的是 urllib 實際會連線的主機名稱：先解開百分比編碼、再做 IDNA 編碼(軟連字號、全形字元與全形句點都會被折疊；樣式也同樣編碼)、去掉結尾的點，任何寫法的 IP（``2130706433``、``0x7f.1``、``[::ffff:127.0.0.1]``）都換成一般形式；名稱不會解析，所以解析到被拒位址的名稱擋不到。模組層級的策略以 *allow-all* 模式啟動,因此在操作者鎖定前**不會改變任何行為**。純標準函式庫,不匯入 ``PySide6``。 diff --git a/docs/source/Zh/doc/new_features/v40_features_doc.rst b/docs/source/Zh/doc/new_features/v40_features_doc.rst index 49eec3a1f..b0969a691 100644 --- a/docs/source/Zh/doc/new_features/v40_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v40_features_doc.rst @@ -5,7 +5,8 @@ 從清單中挑出最佳候選,並收合近似重複項 —— 讓流程可以針對「*看起來像* Submit 的按鈕」 動作,而非精確標籤。 -預設後端為標準函式庫 :mod:`difflib`,因此本功能**無需任何額外相依**即可運作。若安裝了 +預設後端為純 Python(為了相容仍名為 ``difflib``),因此本功能**無需任何額外相依**即可運作;它計算與 rapidfuzz 相同、對稱的 +Indel 比例 ``2 * LCS / (len(a) + len(b))``。若安裝了 選用的 ``rapidfuzz`` 套件(``pip install je_auto_control[fuzzy]``)則改用其以加速;無論 何者,分數皆正規化為 ``0.0..1.0``,故呼叫端永不依賴實際執行的後端。``BACKEND`` 標示目 前作用中的後端。不匯入 ``PySide6``。 diff --git a/docs/source/Zh/doc/new_features/v4_features_doc.rst b/docs/source/Zh/doc/new_features/v4_features_doc.rst index b56a8a64b..98e89d2df 100644 --- a/docs/source/Zh/doc/new_features/v4_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v4_features_doc.rst @@ -44,7 +44,8 @@ Builder 項目。視覺與視窗功能的 geometry / IO 操作皆可注入,因 其他平台除非給了 ``scroller=``,否則拋出 ``ValueError``。``AC_scroll_to_find``。 * **區域顏色統計** — ``region_color_stats(source, region)`` 回傳區域的 ``average_rgb``、``dominant_rgb`` 及該色的像素占比(量化色彩空間 → 取 - 最多的 bucket → 平均其真實像素)。``AC_region_color_stats``。 + 最多的 bucket → 平均其真實像素)。超出影像的區域會裁到影像範圍內。 + ``AC_region_color_stats``。 * **讀取 QR code** — ``read_qr_codes(source, region)`` 以 OpenCV 的 ``QRCodeDetector`` 解碼 QR(不需新相依)。``AC_read_qr``。 @@ -54,7 +55,8 @@ Builder 項目。視覺與視窗功能的 geometry / IO 操作皆可注入,因 * **可重用巨集** — ``AC_define_macro`` 註冊具名、帶參數的動作子程序; ``AC_call_macro`` 以 ``${arg}`` 綁定呼叫它——補上 loop / if 原語表達 - 不了的「可呼叫函式」。 + 不了的「可呼叫函式」。參數只屬於這次呼叫:呼叫結束後(包括巢狀或遞迴呼叫), + 呼叫端同名的變數會恢復原值。 * **同進程平行** — ``AC_parallel`` 讓多個分支動作清單並行執行,各自在 獨立的全新 executor 上,因此分支不會在共享變數上互相 race(跨主機 DAG 的同進程版)。 @@ -62,8 +64,9 @@ Builder 項目。視覺與視窗功能的 geometry / IO 操作皆可注入,因 ``AC_assert_duration`` 在區塊耗時超過預算時判失敗——銜接 profiler 與 斷言 DSL 的延遲回歸守門。 * **讀進變數** — 把外部資料綁進流程範圍供後續 ``${var}`` 使用: - ``AC_ocr_to_var``(區域文字)、``AC_shell_to_var``(命令 stdout,以 ``encoding`` 解碼,預設為系統地區設定的編碼)、 - ``AC_read_file_to_var``(檔案文字)、``AC_http_to_var``(GET body 或 + ``AC_ocr_to_var``(區域文字)、``AC_shell_to_var``(命令 stdout,以 ``encoding`` 解碼,預設為系統地區設定的編碼; + 逾時會結束該命令及它啟動的所有程序,``.bat`` / ``.cmd`` 的參數含 cmd 語法時會拒絕)、 + ``AC_read_file_to_var``(檔案文字;除非以 ``encoding`` 指定,否則讀 UTF-8,有無 BOM 皆可)、``AC_http_to_var``(GET body 或 dotted JSON path)、``AC_now_to_var``(strftime)、``AC_random_to_var`` (seeded int / float / choice)。 * **變數轉換** — ``AC_transform_var`` 套用 upper / lower / strip / title / @@ -128,7 +131,9 @@ Builder 項目。視覺與視窗功能的 geometry / IO 操作皆可注入,因 標記版搭檔)。``AC_annotate_screenshot``。 * **桌面通知** — ``notify(title, message)`` 顯示跨平台通知 (``notify-send`` / ``osascript`` / PowerShell);防注入(Linux 用 argv, - macOS / Windows 用從環境變數讀字串的固定腳本)。``AC_notify``。 + macOS / Windows 用從環境變數讀字串的固定腳本)。通知程式以非零結束碼 + 結束時回傳 ``shown=False``;Windows 通知用 PowerShell 已註冊的 app id, + 因為 Windows 會丟掉未註冊 app id 的通知。``AC_notify``。 GUI diff --git a/docs/source/Zh/doc/new_features/v51_features_doc.rst b/docs/source/Zh/doc/new_features/v51_features_doc.rst index 11f95a8ca..709aea964 100644 --- a/docs/source/Zh/doc/new_features/v51_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v51_features_doc.rst @@ -17,8 +17,10 @@ JSONPath 查詢 過濾條件的欄位可以是巢狀的(``@.a.b``),``[?(@.k)]`` 保留有 ``k`` 的元素;對物件套用過濾時，挑的是它的成員值。 比較的值是 JSON 數字、加引號的字串、``true``、``false`` 或 ``null``。 -不同型別的值一律不相等(``true != 1``)。這個子集讀不懂的路徑(不支援的過濾條件或值、切片 ``[0:2]``、 -聯集 ``[0,1]``、空的 ``[]``、沒有收尾的 ``[``、多餘的字元)會拋 ``ValueError``,不會改成比對到別的東西。 +不同型別的值一律不相等(``true != 1``);``<`` / ``>`` 只比較兩個數字或兩個字串(``<=`` 是 ``<`` 或 ``==``, +所以 ``null <= null`` 成立)。加引號的名稱和字串會解碼 RFC 9535 的跳脫序列(``['a\'b']`` 是鍵 ``a'b``)。 +這個子集讀不懂的路徑(不支援的過濾條件或值、切片 ``[0:2]``、 +聯集 ``[0,1]``、``['a','b']``、空的 ``[]``、沒有收尾的 ``[``、多餘的字元)會拋 ``ValueError``,不會改成比對到別的東西。 純標準函式庫(``re``);不匯入 ``PySide6``。 diff --git a/docs/source/Zh/doc/new_features/v55_features_doc.rst b/docs/source/Zh/doc/new_features/v55_features_doc.rst index 0e80b3826..67fcc3d73 100644 --- a/docs/source/Zh/doc/new_features/v55_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v55_features_doc.rst @@ -4,7 +4,7 @@ 影像遮蔽模組會在螢幕截圖中模糊 PII,但從 UI、OCR、剪貼簿、LLM 提示/回應或日誌行擷取的 *文字*卻沒有字串層級的對應 —— 因此 PII 可能洩漏進動作紀錄、稽核日誌或一次模型呼叫。 ``detect_pii`` / ``redact_pii_text`` 可在純文字上找出並遮蔽電子郵件、電話號碼、SSN、信 -用卡號、IPv4 位址與 IBAN。 +用卡號(以 Luhn 驗證)、IPv4 位址與 IBAN(連寫或四碼一組的列印格式,以 mod-97 驗證)。 樣式刻意保持簡單(無巢狀量詞 → 無災難性回溯)。純標準函式庫(``re`` + ``hashlib``);不匯 入 ``PySide6``。 diff --git a/docs/source/Zh/doc/new_features/v59_features_doc.rst b/docs/source/Zh/doc/new_features/v59_features_doc.rst index ae29eb8f4..706e12f65 100644 --- a/docs/source/Zh/doc/new_features/v59_features_doc.rst +++ b/docs/source/Zh/doc/new_features/v59_features_doc.rst @@ -33,7 +33,8 @@ eXchange)正是這個分級訊號的標準。本功能撰寫 `OpenVEX `, so `bearer ` got 401 although RFC 7235 makes the scheme case-insensitive. The scheme is matched case-insensitively and only the token is compared, still with `hmac.compare_digest`. +- **History failure**: `fire()` calls `start_run` before its own try, so a `HistoryStoreError` (a locked database) escaped the handler, the client saw the connection drop and a traceback reached stderr. `_dispatch` answers 500 `{"fired": false}` instead. +- **Not changed**: the REST API and the MCP HTTP transport already refuse a chunked body with 400 rather than accepting it silently. +- **Tests**: `test_webhook_audit.py` (new, 10; the three server cases fail on the previous commit, the seven decoder cases test the new helper). +- **Files**: `utils/http_headers.py`, `triggers/webhook_server.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-13 · 2026-09-24 · macOS and Linux hotkeys stop retrying a failed combo every tick; bind() validates on macOS · #bugfix #audit #macos #linux + +- **What**: the macOS and Linux backends re-sync their bindings about ten times a second, and a combo that could not be parsed (`ctrl+.` has no key code in either table) or, on Linux, grabbed (another client holds it) was retried and logged on every sync for as long as the daemon ran. The Windows backend already remembered failures; the other two never got it. `HotkeyDaemon.bind()` only validated the key on Windows, so macOS accepted `ctrl+.` and failed later, every tick. +- **Fix**: new `FailedCombos` in `hotkey/backends/base.py` records a binding whose current combo failed and skips it until the combo changes; failures of removed bindings are forgotten. The macOS and Linux backends use it for parse failures, and Linux for grab failures too. On macOS a changed combo now also drops the old registration even when the new combo fails, so the old keys stop firing the binding. `bind()` checks the key with the macOS parser on darwin; Linux cannot check without a live X display, so there the memo is what stops the loop. +- **Not changed**: the Windows backend keeps its own equivalent dict; moving it onto `FailedCombos` would be a separate refactor. +- **Tests**: `test_hotkey_failed_combos.py` (new, 6; 5 fail on the previous commit, the sixth tests the new class). They drive `_sync` / `_sync_one` with fakes, so no display, event tap or real hotkey is involved. +- **Files**: `hotkey/backends/base.py`, `hotkey/backends/macos_backend.py`, `hotkey/backends/linux_backend.py`, `hotkey/hotkey_daemon.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-14 · 2026-09-24 · Replay scrolls the recorded way on X11/Wayland, releases where the button went down; paths round; NaN holds refused · #bugfix #audit + +- **Replay scroll**: `_sink_scroll` called `mouse_scroll(value)`, whose `scroll_direction` defaults to `scroll_down` -- read only by X11 and Wayland -- while the recorders and the Windows / macOS backends treat a positive value as up. A wheel-up recorded on Windows replayed as wheel-down on X11. The sink names `scroll_up`, so a positive value means up everywhere. +- **Cleanup release**: after a failed step `replay_timeline` releases every held button with an event that carries no point, and `_sink_mouse_up` filled in `(0, 0)`; on macOS the release was posted in the top-left corner, where a hot corner can fire. A missing point is passed as `None`, which means the current cursor position. +- **Waypoints**: `plan_path` truncated with `int()`, so `-0.6` (a point on a monitor left of the primary one) became `0` and `100.7` became `100`, while `tween_points` rounds. Waypoints are rounded to the nearest pixel. +- **Key hold**: `plan_key_hold` validated with `<= 0`, which NaN passes, so the real key went down before `sleep(NaN)` raised. Non-finite durations and rates are refused up front. +- **Deferred**: the audit's findings in `wrapper/auto_control_keyboard.py` and `wrapper/auto_control_mouse.py` (upper-case letters and `is_shift` on Windows / X11, `\r\n` typing two Enters, the X11 scroll default, NaN in `mouse_scroll`, truncated coordinates) change what the Jeffrey_RPA batch types and scrolls, and that batch is running from this working tree; they are recorded in `Progress.md` until it is idle. +- **Tests**: `test_input_helpers_audit.py` (new, 7; 6 fail on the previous commit, the NaN-rate case already failed there through `round(NaN)`). All input goes to fakes. +- **Files**: `input_macro/input_macro.py`, `mouse_path/mouse_path.py`, `key_hold/key_hold.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: the wrapper findings above (`Progress.md`). + +## U-20260924-15 · 2026-09-24 · Signing, encryption and JWT keys, passwords and tokens are masked in the executor log, its record and the MCP audit file · #security #audit + +- **Executor log and record**: `redact_actions` masked only `AC_secret_*` arguments. The key given to `AC_sign_action_file`, `AC_verify_action_file`, `AC_encrypt_action_file`, `AC_decrypt_action_file`, `AC_jwt_encode` and `AC_jwt_decode` -- and every `password` / `token` argument (`AC_email_trigger_add`, `AC_webhook_add`, `AC_remote_connect`...) -- was written to the `autocontrol_logger` output and used as the result record's key, which REST, MCP, the socket server and run history return. Arguments whose name marks a secret (`SENSITIVE_ARGUMENT_NAMES`: password, passphrase, token, secret, api_key, private_key, client_secret, authorization, access/refresh token) are masked for every command, and `key` for the six keyed commands; `key` elsewhere is a keyboard key and stays readable. Nested commands are masked with their parent, as before. +- **MCP audit file**: `_sanitise` checked top-level argument names only, against a list without `key` or `passphrase`, so `ac_execute_actions` with a nested `AC_secret_unlock` passphrase and `ac_jwt_encode`'s key reached the JSONL file. It walks every dict and list, masks the same names plus `key` (conservative: an audit file has no keyboard keys worth keeping), and runs action lists through `redact_actions`. `REDACTED_KEYS` is now `SENSITIVE_ARGUMENT_NAMES`. +- **Tests**: `test_secret_redaction_audit.py` (new, 11; 10 fail on the previous commit, the eleventh guards readable non-secret arguments). Fake values only. +- **Also in this push**: `test_codegen_audit.py` checks the generated `nan` / `inf` names with `ast` instead of `exec`, which Codacy flagged. +- **Files**: `executor/action_redaction.py`, `mcp_server/audit.py`, both `mcp_server_doc.rst`, `test_codegen_audit.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-16 · 2026-09-24 · Signed-action enforcement covers remote DAG nodes; UserAuthError and CredentialBrokerError join the framework family · #security #audit + +- **Remote DAG nodes**: every local execution path reads action files through `read_executable_action_json`, which enforces `JE_AUTOCONTROL_REQUIRE_SIGNED_ACTIONS`, but `_resolve_remote_actions` loaded a remote node's `action_file` with a plain `json.load` and dispatched it through the admin console. With enforcement on, an unsigned file ran on the remote host. It goes through `read_executable_action_json` now (which also accepts a UTF-8 BOM, as the local paths do); inline `actions` on a node are unchanged. +- **Exception family**: `UserAuthError` (RBAC) and `CredentialBrokerError` (governance) derived from `RuntimeError` only, which `CLAUDE.md` forbids: they escaped every `except AutoControlException` containment boundary. Both derive from `(AutoControlException, RuntimeError)`, as `SecretStoreError` already does; nothing catches them by name, so no handler changes behaviour. +- **Tests**: `test_signing_and_error_family_audit.py` (new, 4; 3 fail on the previous commit, the fourth guards loading without enforcement). +- **Files**: `dag/runner.py`, `rbac/users.py`, `governance/credential_broker.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-17 · 2026-09-24 · Audit hash chain: no re-blessing of cleared hashes, deletions from the top caught, clear() leaves a record · #security #audit + +- **Re-blessing**: `_backfill_chain_locked` ran on every open and re-hashed any row whose `row_hash` was NULL. Editing a row and clearing its hash made the next open chain the forgery, and `verify_chain()` reported `ok`. The backfill is now a one-off migration marked with `PRAGMA user_version = 1`; after it a NULL hash is a broken link. +- **Deleting the oldest rows**: `verify_chain` started from the first row's own `prev_hash` (to allow for pruning), so deleting rows from the top went unnoticed. A `chain_meta` table holds the anchor the first row must point at: the genesis hash, or the hash of the last row automatic pruning removed, updated when it prunes. A log pruned before the anchor existed takes its first row's `prev_hash` as the anchor during the migration. +- **`clear()`**: it wiped the table and left nothing, so a cleared log verified as a clean empty one. It now resets the anchor and starts the new chain with an `audit_log_cleared` event recording how many rows it deleted; `test_audit_log.py`'s clear case now expects that one event. +- **Limits, now documented**: the hashes are unkeyed SHA-256, so whoever can write the database can rebuild the chain, and dropping the newest rows is not detectable from the file alone. Both operations docs say so and suggest keeping the last `row_hash` elsewhere. +- **Tests**: `test_audit_chain_audit.py` (new, 5; 4 fail on the previous commit -- the migration case because the old schema has no `chain_meta` -- and the pruning case guards that automatic pruning still verifies). +- **Files**: `remote_desktop/audit_log.py`, `test_audit_log.py`, both `operations_layer_doc.rst`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-18 · 2026-09-24 · User store keeps a damaged file and refuses shared tokens; secret managers stop overwriting each other; malformed vaults are store errors · #security #audit + +- **Damaged user file**: `UserStore._load` returned an empty store when `users.json` could not be parsed, and the next `add_user` / `set_role` / `rotate_token` saved that, replacing every user on disk. The store now logs the reason, lets nobody sign in, and refuses to overwrite the file until it is repaired or removed. +- **Tags**: `"tags": 5` in the file made the constructor raise a raw `TypeError`; tags that are not a list read as `[]`, and `add_user` normalises its argument the same way. +- **Shared token**: `add_user` accepted a caller-supplied token another user already had, and `authenticate` returns the first match, so a viewer's token could sign in as an admin. A token in use is refused. +- **Lost updates**: each `SecretManager` wrote back its own cached vault, so two on one file (the GUI and a service process) undid each other's `set` / `remove`. Every operation re-reads the vault from disk; one re-keyed elsewhere locks the manager (`SecretStoreLocked`), which must unlock again. There is still no cross-process lock: two writes in the same instant can race, but no longer a whole session apart. +- **Malformed vault**: a missing `salt` raised `KeyError` from `unlock` and `iterations: 0` a `ValueError`; `_load_vault` checks the fields and raises `SecretStoreError`. +- **Docs**: the `utils/rbac` docstring said the REST and MCP servers consult it and the audit log gained a `user_id` field; neither is true, and it now says it is a building block that nothing uses yet (map row updated). +- **Tests**: `test_stores_audit.py` (new, 9; all fail on the previous commit). Temporary directories and fake values only. +- **Files**: `rbac/users.py`, `rbac/__init__.py`, `secrets/secret_store.py`, `architecture_explore.md` (`utils/rbac/` row, line counts), `CHANGELOG.md`. +- **Open items**: RBAC is not wired to the REST API or MCP server (`Progress.md`). + +## U-20260924-19 · 2026-09-24 · REST API: authenticate before reading the body, never lock out the valid token, survive a corrupt audit database · #security #audit + +- **Body before auth**: `do_POST` read and parsed the JSON body before `_dispatch` ran the auth gate, so an unauthenticated client could send 1 MB bodies at will, and a bad one got 400 -- never 401 or 429 -- without touching the rate limit, the lockout or the audit trail. The body is now read only after the route and the gate pass. A rejected or unknown-path request has its declared body drained (capped) before the answer, since Windows otherwise resets the connection before the client reads the 401 / 404. +- **Lockout DoS**: `RestAuthGate.check` tested the lockout before the token, keyed by IP alone -- every local client is 127.0.0.1 and every proxied one the proxy -- so eight bad requests a minute from anyone kept the real token holder out indefinitely. A valid token is checked first and always passes; the lockout answers further wrong tokens, and the per-IP rate limit still applies to everyone. The tokens are random, so the lockout was never what stopped guessing. +- **Corrupt audit database**: `_open_audit_log` caught `OSError` / `RuntimeError` / `ImportError`, but `AuditLog()` raises `AuditLogError` for a corrupt file, so `RestApiServer()` raised instead of running without the audit hook as intended. +- **Comment**: the handler's 30 s timeout bounds each read, not the request; the comment said it stopped stalled clients outright. It now says what it does. +- **Tests**: `test_rest_audit_fixes.py` (new, 5; 4 fail on the previous commit, the fifth guards the authorised path). The `/execute` handler is a fake. +- **Files**: `rest_api/rest_server.py`, `rest_api/rest_auth.py`, `test_http_content_length.py` (its REST case sends the token, since an unauthenticated request is now answered 401 before the header is read), both `operations_layer_doc.rst`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-20 · 2026-09-24 · MCP HTTP: anonymous initialize floods cannot evict a session in use; DELETE checks its path; state of a session dropped mid-request is released · #security #audit #mcp + +- **Eviction**: at the session cap `SessionRegistry.create` evicted the least recently seen session. A client that ignores the session header mints a new session on every `initialize`, and the transport has no token by default, so 128 of them evicted a session a real client was holding (it then got 404 `unknown or expired session`). The victim is now the oldest session never used after its `initialize` -- the same test `_log_eviction` already used to decide how loudly to report it -- and only when every session is in use the oldest of those. +- **DELETE path**: `do_DELETE` never looked at the path, so `DELETE /anything` with a session header ended the session, while GET and POST answer 404 off `/mcp`. It answers 404 too. +- **Orphaned state**: a session evicted, swept or deleted between `_resolve_session` and `handle_line` still had its `initialize` capabilities stored under the dead id after the drop hook had run, and nothing released them (50 injected drops left 50 entries, still there after `stop()`). After handling a request the transport calls `forget_connection` for a session that was closed meanwhile. +- **Tests**: `test_mcp_http_audit_fixes.py` (new, 4; 3 fail on the previous commit, the fourth guards eviction when every session is in use). +- **Files**: `mcp_server/http_sessions.py`, `mcp_server/http_transport.py`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-21 · 2026-09-24 · Socket server reads whole pretty-printed commands; the documented client example works · #bugfix #audit + +- **Framing**: `_read_command` stopped at the first TCP chunk containing a newline. A newline is ordinary whitespace in JSON, so an indented command was cut at the first chunk boundary and failed with `Expecting value`, and one sent in two segments was refused after the first while the client's second write hit a reset. A command now ends at a newline that is the last byte received *and* after which the text either parses or fails before its end (malformed -- answered with the error at once, as before). A parse error exactly at the end means the command is still arriving. Two commands in one connection are still one malformed command: the protocol is one command per connection. +- **Docs**: the socket driver page's client example sent the JSON without the newline terminator and read one `recv`. The server then waited for the terminator until its 30 s read timeout and dropped the connection, so the example got nothing. The example sends the newline and reads until `Return_Data_Over_JE`, and the protocol table states the request and response framing (both languages). +- **Comment**: the handler's timeout bounds each read, not the command; the comment now says so. +- **Not changed**: result values are still written as `str(value)` lines, so a result containing a newline or the marker can confuse a line-based client. Changing that changes the protocol other tools read. +- **Tests**: `test_socket_framing.py` (new, 7; 6 fail on the previous commit, the malformed case guards the immediate answer). The executor is a fake. +- **Files**: `socket_server/auto_control_socket_server.py`, both `socket_driver_doc.rst`, `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-22 · 2026-09-24 · Remote desktop host: failed logins free their slot, view-only means view-only, an allowlist of typos admits nobody · #security #audit + +- **Slot leak**: handlers join the client table before they authenticate, and the host reaps only handlers whose `_shutdown` is set. On an auth failure `start()` only closed the socket, so two wrong-token attempts against `max_clients=2` refused the real viewer until the host restarted. Every failed handshake now calls `stop()`. An approval callback raising outside `(RuntimeError, ValueError, TypeError)` killed the handshake thread the same way; the callback is user code, and any exception now denies the viewer. An unexpected error in the handshake stops the handler before propagating. +- **Handshake deadline**: the 60 s auth timeout bounded each read, so a peer trickling a byte at a time held the handshake -- and its slot -- indefinitely. A watchdog closes the socket when the whole handshake exceeds it. +- **View-only**: only `INPUT` was gated on `PERMISSION_VIEW_ONLY`, so a view-only viewer could set the host clipboard and write files anywhere on the host (a file in the Startup folder is full control at the next logon). `CLIPBOARD` and every `FILE_*` message are dropped for view-only viewers. +- **Allowlist**: `_compile_ip_allowlist` returned `compiled or None`, and `_ip_in_allowlist` treated an empty list as no filtering, so a list whose every entry was invalid (`192.168.1.300`, `10.0.0.1/33`) admitted everyone -- the opposite of the docstring's promise. Such a list admits nobody now; a list of only blank strings is still no list. `test_remote_desktop_ip_allowlist.py` no longer asserts that a compiled empty list admits everyone and gains the typo case. +- **Not changed**: a host can still push a file to any path on a viewer; that is the open `Progress.md` DECIDE item on confining viewer downloads. +- **Tests**: `test_remote_host_access_audit.py` (new, 5; all fail on the previous commit). Input and capture are fakes. +- **Files**: `remote_desktop/host_client.py`, `remote_desktop/host_access.py`, `test_remote_desktop_ip_allowlist.py`, both `new_features_doc.rst` (view-only and allowlist paragraphs, committed separately because another session has edits in those files), `CHANGELOG.md`, `architecture_explore.md` (line counts). +- **Open items**: none. + +## U-20260924-23 · 2026-09-24 · Remote desktop stores keep damaged files aside; interrupted uploads are cleaned up; stopping the relay ends its sessions · #bugfix #audit + +- **Damaged stores**: `TrustList`, `KnownHosts` and `AddressBook` read a damaged file as empty, and the next `add` / `remember` / `upsert` rewrote it with only the new entry -- for `known_hosts` that silently reset every pinned fingerprint, so the next connection re-trusted whatever answered. A non-UTF-8 address book raised `UnicodeDecodeError`, and `{"entries": 5}` / `{"viewers": 5}` a `TypeError`, from the constructor. New `load_json_or_quarantine()` / `quarantine_file()` in `json_store.py` move an unreadable or wrong-shaped file aside as `.corrupt-