Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -731,13 +731,13 @@ The cost is that the press-time context read has to be settled _before_ the requ
path doing so — the read is dispatched before the mic bring-up is joined, so it is almost always
already finished and the wait is spent while the user is still speaking, not while they wait for a
transcript. The _response_
is unchanged and still arrives whole: a single `async throws -> String`, no deltas. The underlying sync STT model
handles audio from ~80 ms up to 120 s (server-side ~30 s inference deadline); those limits live in
is unchanged and still arrives whole: a single `async throws -> String`, no deltas. The dictation API
accepts audio from ~80 ms up to 20 minutes (1200 s; it was 120 s until the API raised it; server-side ~30 s inference deadline); those limits live in
`SyncSTTLimits` and back `DictationSession`'s auto-release timeout, so a held hotkey stops before
the cap.
the cap (at 1195 s).

Every wire constant was swept against the live route on 2026-09-09 (the keyterms list at 2048 chars and the
120 s ceiling are both exact and enforced; the prompt's 4096 is the API's own and hard-rejecting,
then-120 s audio ceiling were both exact and enforced — the ceiling has since been raised to 20 minutes and not re-swept; the prompt's 4096 is the API's own and hard-rejecting,
measured in Unicode scalars on 2026-09-10 — it was `conversation_context`'s 4096, which was **ours**
and merely trimmed, until `stt_prompt` replaced that field; the whole `config` part is separately
limited somewhere past 48 kB; sub-floor audio returns 200 with an
Expand Down
4 changes: 2 additions & 2 deletions App/Blurt/Blurt/Hotkey/DictationKeyTap.swift
Original file line number Diff line number Diff line change
Expand Up @@ -171,8 +171,8 @@ final class DictationKeyTap {
/// consumes `phaseStream()` asynchronously, so if that loop falls behind,
/// dictation N's terminal phase can arrive *after* the gate has armed for
/// dictation N+1 — the reset then clears N+1's live state, its key-up routes to
/// `.none`, and the recording would otherwise run to the ~115 s auto-release cap.
/// Cancelling it turns a silent two-minute hang into a clean stop.
/// `.none`, and the recording would otherwise run to the ~1195 s auto-release cap.
/// Cancelling it turns a silent twenty-minute hang into a clean stop.
///
/// A no-op in every normal flow, where the gate is already idle by the time a
/// terminal phase lands.
Expand Down
2 changes: 1 addition & 1 deletion App/BlurtiOS/BlurtiOS/Sources/DictationCoordinator.swift
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ final class DictationCoordinator {

private func render(_ phase: PipelinePhase) {
self.phase = phase
if phase == .recording { window.extend() }
if phase == .recording { window.extend(atLeast: SyncSTTLimits.maxAudioSeconds) }
timeRecording(phase)
// Delivery always comes before the paste and its terminal phase, and the
// phases render in order, so the dictation's words are waiting by now.
Expand Down
9 changes: 6 additions & 3 deletions App/BlurtiOS/BlurtiOS/Sources/ListeningWindow.swift
Original file line number Diff line number Diff line change
Expand Up @@ -60,10 +60,13 @@ final class ListeningWindow {
}

/// Pushes the window's end out again — called on open and after every
/// dictation, so a window only closes on real silence.
func extend() {
/// dictation, so a window only closes on real silence. `atLeast` holds it
/// open that long whatever the setting says: a dictation's start passes the
/// longest a dictation can run (`SyncSTTLimits.maxAudioSeconds`), so a
/// five-minute window can't end a twenty-minute dictation partway through.
func extend(atLeast floor: TimeInterval = 0) {
let minutes = SharedStore.windowMinutes
let end = minutes > 0 ? Date().addingTimeInterval(TimeInterval(minutes * 60)) : Date.distantFuture
let end = minutes > 0 ? Date().addingTimeInterval(max(TimeInterval(minutes * 60), floor)) : Date.distantFuture
until = end
SharedStore.listeningUntil = end
expiry?.cancel()
Expand Down
6 changes: 3 additions & 3 deletions App/BlurtiOS/BlurtiOSTests/SharedStateTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -59,11 +59,11 @@ struct PhaseSnapshotTests {
}
}

@Test("an in-flight state outlives no dictation past 130 seconds")
@Test("an in-flight state outlives no dictation past 1210 seconds")
func inFlight() {
for state in [PhaseSnapshot.State.connecting, .recording, .processing] {
#expect(!snapshot(state, ageSeconds: 120).isStale)
#expect(snapshot(state, ageSeconds: 131).isStale)
#expect(!snapshot(state, ageSeconds: 1200).isStale)
#expect(snapshot(state, ageSeconds: 1211).isStale)
}
}

Expand Down
2 changes: 1 addition & 1 deletion App/BlurtiOS/DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -801,7 +801,7 @@ notifications that carry nothing and just say "look" (`BlurtiOSCore/SharedState.
| result delivery | the app waits 3 s (poll 250 ms) for the keyboard to take the result; else clipboard + "Copied" | "Pasted" is never a guess |
| command freshness | 10 s | a held press from minutes ago must not start a dictation on resume |
| command retry | one re-signal after 600 ms with no phase change | a missed Darwin notification would leave the gate latched over nothing |
| phase staleness | notices 3 s (max dwell 2 s + 1); in flight 130 s (the 120 s cap + 10) | the keyboard reads the snapshot on every appearance |
| phase staleness | notices 3 s (max dwell 2 s + 1); in flight 1210 s (the 1200 s cap + 10) | the keyboard reads the snapshot on every appearance |
| notice dwell | pasted 1.2 s; copied, error 2 s | the Mac's 0.8 / 1.6 s, a little longer with no hover |
| press delay on the mic | 90 ms | a swipe that starts on the key never starts a dictation it must then cancel |
| tap travel | 12 pt (keys), 24 pt (mic), swipe ≥ 48 pt horizontal and 1.5× the vertical | the carousel's swipe never types or dictates |
Expand Down
4 changes: 2 additions & 2 deletions Sources/BlurtEngine/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Key properties of the design, which your integration can rely on:
- **One request per utterance, chunked upload, whole response.** The dictation API returns the complete transcript — and its LLM-rewritten form — in the response body: no job polling, no incremental deltas, no second request for the cleanup. `TranscriberProtocol.transcribe` is a single `async throws -> String`, so UIs should still show a "transcribing…" state and then the whole result; there is nothing to stream _back_. What is streamed is the request: `transcribe(frames:sampleRate:context:)` is called at press and uploads the recording in chunks as it is captured, so only inference on the last frames lands on the wait the user sits through. There is deliberately no buffered fallback — a body that has already been streamed cannot be replayed, and the transcriber refuses `URLSession`'s replay request rather than silently re-sending an empty one.
- **Cleanup happens server-side, and the opt-out is on the response.** The request's `config.llm_instruction` asks the service to apply our own cleanup instruction (`CleanupInstruction.text` — delete disfluencies, change nothing else) to the verbatim transcript inside the same call. It is one of three steering fields on the request: `config.stt_prompt` primes the _transcription_ with the text that came before — the user's recent dictations, then the text before the cursor (`STTPrompt`, omitted when there is neither) — and `config.keyterms_prompt` boosts the user's key terms (`KeyTerms`, omitted when there are none). `prompt` is `stt_prompt`'s alias and `conversation_context` the array-of-turns field it replaced; a request carries exactly one of the three. The instruction rides **every** request; the **enhanced transcripts** setting (`EnhancedTranscriptsStore`, on by default) then decides which of the two transcripts in the response gets pasted (`AssemblyAITranscriber.transcript(from:enhanced:)`). That ordering is the route's own advice: it rewrites by default, omitting `llm_instruction` only selects the service's wording, and the reference documents no way to decline — the escape it names is to ignore `llm_response` and use `text`. The undocumented `"llm": null` does suppress it (measured 2026-09-10) and Blurt sent it until 2026-09-11; the trade for dropping it is that "off" still waits out and still pays for a rewrite nobody reads. The **active style profile**'s instructions (`StyleProfileStore`, none by default) are appended to that instruction via `CleanupInstruction.sendable(appending:)`, trimmed to the headroom the API's 2048 instruction cap leaves (measured in UTF-8 bytes, the conservative bound — the cap's own unit is unmeasured); no profile, or blank instructions, means the base instruction goes out unchanged. Only the **active** profile's text is ever sent — never a join of the user's profiles, which would blow that cap and 400 the whole request rather than degrading. With the setting on the engine pastes `llm_response`, falling back to the verbatim `text` when the best-effort rewrite failed (`llm_error`) — a degradation, never a user-facing error; with it off it pastes `text` and discards the rewrite unread. There is no client-side LLM pass, no styling stage, and deliberately no hook for one.
- **Latency is pre-paid where possible, and never faked.** `press()` claims `.connecting` before it touches the mic, so a host's pill answers the keypress — but `MicCapture.start()` deliberately holds until the input device is actually delivering frames, and the _start cue_ waits for `.recording`. On a Bluetooth route those are ~1–2 s apart and the OS captures nothing in between, so cueing at the press loses the first words. Route activation itself cannot be pre-paid — measured, not assumed: nothing opens the device before `record()`, so the connecting window is where that cost lives and the gate is what makes it honest. `press()` fires a detached `warmUp()` at the transcriber (pre-opening the HTTPS connection so the request it opens moments later streams from its first frame, ~170 ms cold — measured to coalesce with that request rather than race it) and kicks off the cross-process accessibility read of the focused field without awaiting it — the read is then consumed when the _upload_ opens, with a bounded wait (`DictationSession.contextWaitBudget`, 500 ms), because the streaming route needs the `config` part before any audio — so an unresponsive frontmost app costs the transcript its priming, never a multi-second stall, and that wait is spent while the user is still speaking rather than while they wait for a transcript. On the way out, `release()` flips the phase to `.transcribing` _before_ stopping the mic, so a host's stop cue fires at key-up rather than after the Bluetooth tail linger.
- **A held trigger auto-releases.** `DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 115 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than paying for a request that transcribes nothing (measured: the route answers a sub-floor clip with 200 and an empty transcript, not a 400).
- **A held trigger auto-releases.** `DictationSession` stops recording after `maxRecordingSeconds` (default `SyncSTTLimits.autoReleaseSeconds`, 1195 s — 19 min 55 s) so audio never exceeds what the endpoint accepts, and transcribes what it has. Clips shorter than `SyncSTTLimits.minPCMBytes` (~100 ms of audio — an accidental tap) are dropped as a silent no-op rather than paying for a request that transcribes nothing (measured: the route answers a sub-floor clip with 200 and an empty transcript, not a 400).

## DictationSession

Expand Down Expand Up @@ -193,7 +193,7 @@ func warmUp() async // optional; no-op default

`AssemblyAITranscriber` is a stateless `Sendable` struct. One `POST https://dictation.assemblyai.com/v1/transcribe/live` per utterance, **uploaded in chunks while the recording is still happening**: called at press, it writes the whole `config` part and the `audio` part's headers immediately and then each captured frame as `frames` delivers it, so finishing that stream is what signals end-of-audio. `URLRequest.httpBodyStream` is fed through a `CFStream` bound pair (`ChunkedRequestBody`, in the `AssemblyAI` library with `HTTPTransport`, the wire types and `APIKeyValidator`) because no `URLSession` API takes an `AsyncSequence` body; leaving `Content-Length` unset is what makes the transfer chunked. The body is a JSON `config` part (`sample_rate`, `channels`, and `llm_instruction` carrying `CleanupInstruction.text` with any custom style instructions appended — `CleanupInstruction.sendable(appending:)` — on every request, whatever the enhanced-transcripts setting says) followed by the raw S16LE PCM byte-for-byte, with the API key in `Authorization` and `Blurt/<short version> (macOS <major>.<minor>)` in `User-Agent` — `; dev` appended off a release build, so local installs stay out of the service-side baseline (see `UserAgent`; no model header — the service pins the STT model server-side). That `config` part is written **first**, and this route requires it — it rejects an audio-first body with a 400 (see `AssemblyAITranscriber.streamedBody` for the service's exact wording). That is where the latency comes from — the service opens its upstream STT call as soon as the config lands, so inference overlaps the recording and nothing sits between the last frame and the response. The cost is that the press-time focused-field read has to be settled _before_ the request opens, so `DictationSession.startUpload` awaits it (bounded by `contextWaitBudget`) and hands it in as a value; there is no longer a `context` channel for the body producer to wait on. The response carries the verbatim `text`, the rewritten `llm_response`, and the documented request metadata — `session_id` (quote it when reporting a problem), `audio_duration_ms`, `request_time_ms` and `sync_time_ms`, all logged by `logServerMetrics` and all decoded optionally so a dropped diagnostic can never turn a good dictation into a `malformedResponse`. `transcript(from:enhanced:)` picks between the two transcripts: with enhanced transcripts on it returns the rewrite and falls back to `text` when that is null or blank (the rewrite is best-effort — `llm_error` is logged, never surfaced as a failure); with it off it returns `text`. Its initializer takes an `apiKeyProvider` closure (defaults to `APIKeyStore.current`), a `baseURL`, an `HTTPTransport` — inject a fake transport (see `Tests/BlurtEngineTests/Stubs/FakeHTTPTransport.swift`) to test against canned responses — an `enhancedTranscripts` closure deciding, per request, which of the response's two transcripts to paste (nil, the default, reads `EnhancedTranscriptsStore`), a `customStyle` closure supplying the style instructions appended to the cleanup instruction (nil, the default, reads the active profile from `StyleProfileStore`), and a `rewriteMode` closure choosing the rewrite's model (nil, the default, reads `RewriteModeStore`; Smart sends `config.rewrite_mode: "smart"`, Fast omits it, and nothing is sent while enhanced transcripts are off). `warmUp()` (in `DictationWarmUp.swift`) fires a GET at the route's own `/warm` endpoint — "an unauthenticated no-op" published for exactly this — to pre-pool the connection; it hit the bare host root until 2026-09-11, which pooled the same connection but asked for a page the service never documented answering. It never throws, and any failure just means the real request pays connection setup as before. The reference's two conditions for the pooled connection to still be there are structural here: same client (the injected `transport`, which is the one `transcribe` uses) and same host (`baseURL`, so a data-zone override warms the zone it will call).

The model's limits live in `SyncSTTLimits` (16 kHz sample rate, ~0.1 s–120 s audio, and the auto-release math — the sync STT model behind the dictation service) — the single source shared by the mic, the session, and the request so recorded and declared geometry can't drift.
The model's limits live in `SyncSTTLimits` (16 kHz sample rate, ~0.1 s–1200 s (20 min) audio, and the auto-release math — the sync STT model behind the dictation service) — the single source shared by the mic, the session, and the request so recorded and declared geometry can't drift.

### `InjectorProtocol` → `KeyInjector`

Expand Down
2 changes: 1 addition & 1 deletion Sources/BlurtEngine/STT/AssemblyAITranscriber.swift
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ import os
/// request per utterance covers transcription *and* cleanup. `config` leads the
/// body because the streaming route cannot open its upstream call without it —
/// see `transcribePath` and `streamedBody`. The service picks
/// the STT model server-side and accepts **at most 120 s** of audio (the
/// the STT model server-side and accepts **at most 20 minutes** of audio (the
/// documented cap; there is no documented *minimum* — the ~80 ms floor is this
/// client's own, see `SyncSTTLimits.minPCMBytes`). The rewrite is best-effort
/// with a 5 s server-side deadline, so a failure still returns the verbatim
Expand Down
8 changes: 6 additions & 2 deletions Sources/BlurtEngine/STT/SyncSTTLimits.swift
Original file line number Diff line number Diff line change
Expand Up @@ -9,8 +9,12 @@ public enum SyncSTTLimits {
/// the recorded and declared rates can't drift apart.
public static let sampleRate = 16_000

/// Maximum audio duration the Sync model accepts per request (seconds).
public static let maxAudioSeconds: Double = 120
/// Maximum audio duration the dictation API accepts per request (seconds):
/// 20 minutes. Raised from 120 s when the API began accepting 20-minute
/// audio; the upload is chunked while the user speaks (see
/// `AssemblyAITranscriber.streamedBody`), so a longer cap holds no more of
/// the recording in memory than a slow uplink forces it to.
public static let maxAudioSeconds: Double = 1200

/// Minimum audio duration worth sending (seconds). Not a rejection
/// threshold: measured against `/v1/transcribe/live` (2026-09-09) the route
Expand Down
2 changes: 1 addition & 1 deletion Tests/BlurtEngineTests/HotkeyRaceTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import Testing
/// `release()`/`cancel()` while the press is still inside that call. The
/// session's serial command queue must run the follow-up *after* the press —
/// honoring it against the freshly started recording — rather than dropping it
/// (which would strand the session in `.recording` until the ~115 s
/// (which would strand the session in `.recording` until the ~1195 s
/// auto-release fires).
@Suite("Hotkey press/release races", .timeLimit(.minutes(1)))
struct HotkeyRaceTests {
Expand Down
8 changes: 8 additions & 0 deletions Tests/BlurtEngineTests/SyncSTTLimitsTests.swift
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,14 @@ struct SyncSTTLimitsTests {
#expect(SyncSTTLimits.sampleRate == 16_000)
}

@Test("the audio cap is the dictation API's 20 minutes, auto-released 5 s early")
func audioCapPinned() {
// What a user feels: a held trigger stops recording at 19 min 55 s. Pinned
// so moving either number is a deliberate change, not a side effect.
#expect(SyncSTTLimits.maxAudioSeconds == 1200)
#expect(SyncSTTLimits.autoReleaseSeconds == 1195)
}

@Test("the minimum audio floor sits strictly inside the accepted range")
func minFloorInsideRange() {
#expect(SyncSTTLimits.minAudioSeconds > 0)
Expand Down
Loading