[initcore0]

What & why

RefineOutputGuard only compared writing systems, so a cleanup that changed the language but kept the alphabet was pasted as if it had been cleaned. In a local refine benchmark on 2026-11-01 (scripts/llm-bench.sh, the app's real cleanup prompt, temperature 0):

• Qwen2.5-1.5B (the shipped default) rewrote a German dictation in English (Latin → Latin), and a Russian one too. The existing script check caught the Russian case; the German one passed.
• Qwen3.5-0.8B / 2B (candidate refine upgrades) rewrote a Ukrainian dictation in Russian (Cyrillic → Cyrillic). That passed the guard.

Fix: when the script survives, plain cleanups now also get a language-level check using the on-device NLLanguageRecognizer (NaturalLanguage). It rejects only when all three hold:

• the recognizer is ≥ 0.80 sure of the input's language,
• it's ≥ 0.80 sure the output is a different language,
• the input's language is ≤ 0.10 in the output.

Other rules:

• Both sides need ≥ 20 letters. Below about 4 words language ID is unreliable ("okay" scores en 0.39).
• Easily confused pairs (nb/nn/da, hr/sr/bs, ms/id, zh-Hans/Hant) count as one language. uk/ru deliberately do not, since that swap is the bug.
• Intended translations (expectedOutputScript set, i.e. translate-to-English) keep the old script-only contract.
• A rejection takes the existing fail-open path: the raw transcript is kept, with status "Kept your language (cleanup translated)".

It applies to every caller of outputTranslatedAway: dictation cleanup, live chunks, meeting summaries and Scratchpad AI. All of them already promise "keep the original language". The agent-bridge, spoken-instruction and custom-Mode paths stay exempt through shouldLanguageGuard, unchanged.

The thresholds come from the bench corpus. Legitimate cleanups keep their language at 0.96–1.00, and real translations flip at 1.00. The decision itself is a pure function over hypothesis maps (languageChanged(inputHypotheses:outputHypotheses:)), so thresholds are unit-tested independently of Apple's model.

How it was tested

• swift test passes: 2,645 tests, 0 failures, including 13 new tests in RefineOutputGuardLanguageTests
• The new tests fail without the fix. With the old guard, the Ukrainian → Russian and German → English cases fail (3 assertions), and every legitimate-cleanup case passes on both versions.
• The new tests use real model outputs from the bench run, not hand-written approximations
• App compiles (scripts/dev-app.sh); check-appstate-ratchet.sh OK, with no AppState changes
• OpenWhispCore builds for iOS (xcodebuild -scheme OpenWhispCore -destination 'generic/platform=iOS'): NaturalLanguage is available there, and the import is behind #if canImport(NaturalLanguage)
• ./build.sh: not run on this branch, which was cut before chore(deps): update all native deps + CI, retire WhisperKit fork, ship SpeechAnalyzer again #250's Xcode 27 build fix. CI builds the merge with main.
• Manually verified in the running app: not yet

Privacy checklist

• No new network egress. NLLanguageRecognizer runs fully on device.

Notes

• iOS companion: outputTranslatedAway is public in OpenWhispCore, and its behavior gets stricter (it can now return true for same-script input/output). languageChanged(input:output:) and the pure overload are new public API.
• Possible false rejects are limited to languages the recognizer confuses on borderline text. The cost is losing the cleanup for that dictation, never losing text.
• Unblocks evaluating Qwen3.5 as a refine model. That still needs --reasoning off on llama-server, because Qwen3.5 thinks by default under llama.cpp's template and otherwise returns empty output. That's a separate change.

🤖 Generated with Claude Code