Asking her about the weather — the card flies out of the rolodex, her answer types as she speaks:
The same file also renders a phone layout (companion app in progress):
A voice-driven HUD for the wall — a smart mirror behind two-way glass, or an interactive display on a plain monitor; same interface either way. Iron Man idiom: pure-black ground, glowing line-work, a rolodex of live cards, a timeline of your next 24 hours, real aircraft on the compass ring, precipitation radar, and a pluggable AI brain that answers from the mirror's own data. macOS-first (tested); a Linux path for N100 / Pi / any Linux box is charted in HARDWARE.md and ~80% of the code already runs there. Start with SETUP.md for the software and HARDWARE.md for the panel, glass, sensor, wall, and the no-Mac / cheaper options.
Bring your own keys and your own AI: everything private lives in config.local.json
and the Keychain. The brain speaks Anthropic by default or any OpenAI-compatible
endpoint (Ollama for fully local). MIT licensed.
Renamed from JARVIS on 2026-08-21. Persona is Alice: British, female, composed.
Stand-in voice is macOS Flo (en_GB); the keeper is Kate (Enhanced), a ~300MB
download via System Settings > Accessibility > Spoken Content > System Voice >
Manage Voices — do it on land. The WAKE WORD is still "hey jarvis" (openWakeWord has
no pretrained "alice" model); a custom "hey alice" model is an on-land task —
either openWakeWord's training notebook or a Picovoice Porcupine custom keyword.
Paths, repo name and launchd labels keep "jarvis" — renaming those buys nothing.
Not a prototype any more. The voice loop runs end to end: wake word, speech recognition on the machine itself, an AI brain that answers from the mirror's own data, and a spoken reply. Eighteen focus cards run on live feeds — weather, air quality, calendar, reminders, mail, traffic with a routed map, flights with live status, aircraft overhead, precipitation radar, ISS passes and launches, news, markets, CI, App Store. A timeline rail shows the next 24 hours. Data refreshes on a five-minute daemon; the whole thing survives a reboot as a launchd appliance.
On top of that sits a ship's-computer layer — ask for a spoken status report across every project you have running, search a library of your own written decisions by voice, dictate a personal log stamped with a stardate and play it back in your own voice, or turn the wall into a window on the live solar system. See BRIDGE-MODE.md for that program and CAPABILITIES.md for everything she can be asked.
Built but waiting on the physical build: voiceprint authorization (code runs;
enrollment happens once the mirror is on a wall), the camera watcher (pipeline
complete, no cameras bought yet), and a presence sensor for wake-on-approach.
A phone companion app is in progress — it renders this same index.html, so
the HUD you see here is the HUD on the phone.
REAL every feed above, the voice loop, the brain, the route map, the clock/date, the render pipeline and state machine SIMULATED the four telemetry meters (processor / memory / mic gain / network are decorative), and the SPACE-key demo turn, which replays a scripted question so you can see the choreography without talking
Open index.html in Chrome. For the real effect:
Chrome > View > Enter Full Screen, or launch kiosk:
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --kiosk --app=file://$PWD/index.html
SPACE or click .... run one fake voice turn (wake -> listen -> think -> respond) C ................. toggle cyan (ambient JARVIS) / red (helmet targeting)
- Pure black background is mandatory: a two-way mirror only shows emitted light, so any grey fill reads as a glowing rectangle. All structure is drawn as lines.
- The radial spoke burst is the voice. It idles low, blooms on LISTENING, and is driven by real microphone amplitude when the voice service is running.
- Landscape and portrait both have real layouts; the phone app uses the portrait one, with the ring on the left and an answer column on the right in landscape.
A query can summon a corner panel to the centre. The inner rings and core readout
fade, the spoke burst migrates outward into a corona, and the panel flies from its
corner and unpacks into a fuller readout. Declared per turn as f:'weather'.
Add a new focusable domain in the FOCUS map: src (which corner panel it flies
from), build() (the expanded card HTML), targets (named data points -> selectors).
Answers are authored with [phrase|target] markers:
"[Eighty-four|temp] and humid in [Atlanta|loc]. Rain likely after [four|hour]."
When the phrase finishes, the matching element in the focus card pulses (brightness surge + slight scale, ~1s) and then holds at raised brightness until the card leaves. The phrase in the spoken line lights up at the same moment, so the eye connects the two. The cue fires off typing position, which stays in step with the spoken line.
A target can be declared as {sel:'...', surge:true} instead of a bare selector.
A surging target doesn't just pulse — it redraws itself. The precip bar rebuilds
from zero with an overshoot, its percentage surfaces above it, and the surrounding
hours dim back. Use it wherever the spoken figure IS the graphic.
Real road geometry from OpenStreetMap (Overpass) + real routes from OSRM. Deliberately NOT map tiles — a raster tile is a photograph, and a photograph behind two-way glass is a glowing grey rectangle. Coordinates in, glowing lines out.
Rendering is split for performance:
- the CITY is static, so it rasterises once to a . ~14k road ways as SVG elements stalls the compositor outright; canvas draws them in a few ms.
- the ROUTE stays in SVG, where it needs to be for the stroke-dash draw-in and for per-segment congestion colouring.
Nothing on the focus path may depend on requestAnimationFrame — a hidden or backgrounded tab never fires it and the card silently never appears. Use timers.
config.local.js and mapdata.js are gitignored and MUST stay that way. Both identify
where the mirror lives: config holds the home coordinates, and mapdata holds a road
network and routes centred on the house. config.example.js is the safe placeholder.
Regenerate map data with tools/build-map.py (see git history) after editing destinations.
python3 tools/refresh-traffic.py
Reads the token from the macOS Keychain (jarvis-mirror / mapbox-token) at runtime.
The token is never stored on disk, never passed as an argument, and every error path is
scrubbed — Mapbox carries the token in the query string, so one unhandled traceback would
otherwise leak it into a log.
freeMin is Mapbox's duration_typical (typical for this time of day), NOT free-flow,
so "vs usual" means what it says. Congestion phrasing is derived from the real per-segment
annotation — never invent a highway name, since step-level data isn't being requested.
3 requests per refresh against a 100k/month free tier. The wall refreshes a fixed set of destinations; the phone app resolves any destination you name out loud, on demand.
~/.venvs/jarvis/bin/python tools/reminders.py [--quiet]
EventKit against whatever macOS has already synced. Needs the one-time Reminders permission, granted per-process — your terminal now, the launchd agent separately.
~/.venvs/jarvis/bin/python tools/refresh-weather.py
No account, no key, no meaningful rate limit. Coordinates come from config.local.json so home never appears in committed source.
~/.venvs/jarvis/bin/python tools/refresh.py # refresh what's stale
~/.venvs/jarvis/bin/python tools/refresh.py --status # ages only
Each source has its own interval (mail 5m, traffic 10m, weather 15m, reminders 15m); one launchd job on a 5-minute timer gives all four sensible cadences.
cp launchd/com.simonbuilds.jarvis.refresh.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/com.simonbuilds.jarvis.refresh.plist
The Mac Outlook profile holds a calendar that stopped being modified in 2023. Dates there are stored as MINUTES SINCE 1601, not Mac absolute time. The current M365 calendar never syncs to this machine; it needs Microsoft Graph, same as work email.
Three launchd agents make the mirror a self-starting appliance: server com.simonbuilds.jarvis.server http.server on 127.0.0.1:8777, KeepAlive refresh com.simonbuilds.jarvis.refresh data refresh every 5 min kiosk com.simonbuilds.jarvis.kiosk tools/kiosk.sh -> caffeinate + Chrome --kiosk
Install: tools/install-kiosk.sh (server + refresh — safe on a work laptop) tools/install-kiosk.sh --full (adds the fullscreen kiosk; DEDICATED machine only)
kiosk.sh waits for the server, then execs Chrome under caffeinate -dis so sleep is
held off only while the mirror is actually running. Chrome uses its own profile dir
(.chrome-kiosk/, gitignored) so it never collides with a normal Chrome session. If
Chrome exits, launchd restarts it: the mirror self-heals.
Manual, one-time, on the dedicated machine (these need the user's password):
- Users & Groups -> automatic login
- Full Disk Access for ~/.venvs/jarvis/bin/python (the launchd TCC trap)
- Reminders/Calendars grants for that same binary on first daemon run
Escape hatch on a live kiosk: Cmd-Q quits Chrome (launchd relaunches in ~2s);
launchctl unload ~/Library/LaunchAgents/com.simonbuilds.jarvis.kiosk.pliststops it.
A SEPARATE server from the mirror: the mirror's data stays on 127.0.0.1, while tools/control-server.py listens on the LAN serving ONLY a control page and a health summary — none of the data files (mail, routes, home coords) are reachable. Every request needs ?t= from config.local.json; anything else is a bare 403. Commands land in controldata.js, which the page polls every 3s.
Phone URL is in phone-remote-url.txt (gitignored — it embeds the token). Controls: brightness (full/boost/night/auto), show any card, palette, demo loop. launchd agent: com.simonbuilds.jarvis.control (loaded; in install-kiosk.sh too).
tools/flights.py polls status for calendar-detected flights, but only inside 48h of departure, on a proximity cadence (12h -> 2h -> 20min), with a hard self-cap of 250 calls/month persisted in .flights-state.json — the free tier can't be exceeded. Key: Keychain jarvis-mirror/rapidapi-key. Cloudflare requires a browser User-Agent. Status merges into the flights card (gate, delay) and the brain's snapshot.