Quill

Offline voice typing for Windows. Hold a key, speak messy English or Chinese, let go — clean, natural English is typed at your cursor. Nothing leaves your computer.

中文说明 · MIT · Windows 10/11

Why

I'm not a native English speaker. When I talk, I hesitate, repeat myself and correct mid-sentence — and I often know exactly what I mean in Chinese but not in English. Dictation tools either type all of that verbatim, or send my voice to the cloud. Quill does the cleanup locally:

You say	Quill types
"I, I want to, um, I want to change the the color, no, the size of the button."	I want to change the size of the button.
"Yesterday I go to the store and buy some apple."	Yesterday I went to the store and bought some apples.
"嗯，我想我想把这个按钮的颜色不对，是大小改一下。" (Chinese)	I want to change the size of this button.
"帮我把这个PR merge一下，然后呃，通知一下team。" (mixed)	Please merge this PR and notify the team.

Features

• Push-to-talk anywhere — hold Right Ctrl (configurable) in any app; the text is pasted at the cursor and your clipboard is restored.
• Cleans up speech, not just grammar — removes fillers, stutters, false starts and self-corrections ("no, I mean…"); fixes tense, articles and plurals.
• Speak Chinese, get English — with AUTO on the home screen Quill tells Chinese from English on its own, no switching; mixed Chinese-English works too. Or get clean Chinese instead (Settings → When speaking Chinese → Chinese).
• Fast — English appears about 1 s after you let go, clean Chinese in about half a second.
• See what changed — an overlay at the bottom of the screen (or the top, in Settings) shows the live text, the words that were fixed and a Chinese translation. With several monitors it appears on the one you're working on.
• Learns your words — names, projects, tools. Write Claude: cloud and Quill decides from context whether "cloud" meant Claude. Once a day it suggests words it probably misheard.

 

How it works

hold key → microphone (16 kHz)
         → recognition   Whisper large-v3-turbo on the GPU, primed with your word list:
                         about once a second for the live overlay, then the whole recording on release
         → word list     Chinese words heard as same-sounding characters are swapped back by pinyin
         → cleanup       Qwen3 4B via Ollama (8B in Settings; falls back to CPU if the GPU is busy)
         → paste at cursor

Reading the whole recording at once is far more accurate than recognizing short pieces between pauses. Recognition runs in a separate process so the UI stays responsive.

Install

Requirements

• Windows 10/11 only (no macOS / Linux), 16 GB RAM.
• An NVIDIA GPU with 6 GB+ VRAM and an NVIDIA driver 527.41 or newer (needed by CUDA 12). Whisper runs on the GPU; there is no CPU mode, and setup.ps1 stops if it finds no NVIDIA GPU. Check with nvidia-smi in PowerShell: it should print a table with a driver version.
• Internet during setup, and winget (it comes with "App Installer" from the Microsoft Store) to install Ollama. If you already have Ollama, winget is not needed.
• About 8 GB of disk: Whisper model 1.6 GB, qwen3:4b 2.6 GB, Ollama itself about 1.5 GB, cuBLAS 0.5 GB, Python 3.12 and the other packages about 1 GB. The optional qwen3:8b adds 5 GB.

git clone https://github.com/FxrcRed/quill.git
cd quill
powershell -ExecutionPolicy Bypass -File setup.ps1          # add -Mirror in mainland China

setup.ps1 installs uv and Python 3.12 (no system Python needed), the packages, the Whisper model, Ollama with qwen3:4b, and a desktop shortcut. It is safe to re-run. If a step fails it stops with a red message and what to do next (usually: fix the cause, open a new PowerShell window, run it again). If something goes missing later, Quill's settings show a Speech model or Cleanup card that says what and have a button that fixes it. For better cleanup, ollama pull qwen3:8b (~5 GB) and pick it in the settings.

Then start Quill from the desktop. The first start loads the Whisper model onto the GPU and starts Ollama: it takes a few seconds to about a minute, longer right after a reboot or on a slow disk. The status card says WAKING until it shows READY, and the hotkey does nothing before READY. Then hold Right Ctrl and speak.

Usage tips

• The language button (top right of the status card) cycles AUTO → EN → CN: AUTO (default) detects Chinese or English for each sentence; EN / CN fix the language you speak. Output is English, unless you set When speaking Chinese → Chinese in Settings.
• Below the status card are HISTORY and WORDS buttons and a settings gear. Settings: microphone, cleanup status (model, GPU memory in use), hotkey (click the key button and press any key), output mode, overlay position (bottom or top of the screen), interface language (English or Chinese; follows Windows by default); the two round icons at the top right are Self-test (checks every part) and Close.
• The shrink button (two arrows, top right) turns Quill to a floating ball at the edge of the screen; click it to come back.
• The moon / sun button (top right) switches between dark and light. Quill follows Windows by default; switching back to the system theme makes it follow again.
• HISTORY keeps the last 20 sentences (the icon in its title row opens the file): the original speech, the English, and the Chinese meaning. You can turn saving off, keep only the last 7 / 30 days, or clear it.
• WORDS: add, edit or delete entries; the icon in its title row opens words.txt in a text editor. One entry per line: Correct spelling: misheard, another misheard (the part after the colon is optional).
• Ollama is started with Quill and stopped when Quill quits; the model leaves GPU memory after it sits idle (after 2 minutes by default; type any number of minutes, or never — in Settings).

Known issues

• Don't use Alt as the hotkey. In some apps, such as the Claude desktop app, tapping Alt opens the menu and steals focus, so the paste fails.

Development

• All of the app is one file: app.pyw (tkinter + Pillow for the UI).
• check.py [ui] [prompt] — runs the checks for whatever you changed: syntax and pyflakes, the end-to-end tests, optionally the UI and correction test sets, then restarts Quill and checks run.log.
• e2e_test.py [zh] — end-to-end test without a microphone (feeds a wav at real-time speed).
• fix_test.py [zh | zhzh] — compare correction output on real-world phrases after changing prompts.
• term_test.py — checks how well the model tells a misheard name (cloud → Claude) from the ordinary word.
• eval_run.py [en | zh | zhzh | all] — runs the correction test set with a fixed seed and diffs it against the last run.
• cer_eval.py [auto] — Chinese character error rate on your own recordings in eval_zh\ (not in git).
• harvest.py — collects test samples by pairing each dictation with the edited message you sent in Claude Code (needs "save_audio": true in settings.json). Its report starts with one line per day: latency, timeouts, and how many sentences you edited before sending (timings come from stats.tsv, one line per sentence written by Quill).
• make_words_demo.py — regenerates the word-list image.
• make_demo.py — regenerates the GIF and screenshot above.
• tools/ — one-off development helpers (window screenshots, UI layout tests, word-list and model comparison scripts, icon and social-preview generators). You don't need them to use Quill. selftest.bat and start.bat are debug launchers (self-test, and start with a console to see errors); the desktop shortcut and start.vbs start Quill normally.
• The project was built together with Claude Code.

Credits

• Whisper large-v3-turbo by OpenAI (MIT), run with faster-whisper
• Qwen3 (Apache-2.0) served by Ollama
• pypinyin (MIT); matching words by sound is an idea from CapsWriter-Offline
• Fonts: Noto Sans SC and Michroma (SIL OFL 1.1)

Models are downloaded at install time and keep their own licenses. Quill itself is MIT.