Repository navigation
Conversation
Add a Docker container per bot running Cua's Linux image as the first target, ahead of the owner's Mac. It starts on demand, sleeps after 5 minutes idle, shuts down after 30 minutes asleep or 2 hours after start, and both the bot and the owner can start, sleep and shut it down. Add browser tools over CDP next to the visual tools, and a Computer section with a live thumbnail in the bot details on every client.
Add the computers contract and a native host that runs one Docker container per bot from Cua's Linux image, with the pinned cua-spacesd 0.12.1 binary (sha256-checked) mounted in and a seccomp profile that keeps Chromium's sandbox on. It talks to Docker over its socket and to spacesd through @trycua/cua: screenshots, pointer and keyboard input, windows, the accessibility tree, viewer tickets and thumbnails, plus Chromium driven over CDP with playwright-core. Each container gets its own token in the credentials table, a home volume and loopback-only ports. DOCKER_HOST overrides the Docker socket.
One bots.computer Pi task per bot keeps the computer's time limits in a stored sleep: sleep after 5 minutes idle, shut down after 30 minutes asleep, and shut down 2 hours after start. Host failures retry with backoff, and startup shuts down containers whose bot has no live task. Bots get bot_computer_* tools (start, sleep, shut down, status, windows, read, act, handoff) and bot_browser_* tools over CDP. Acting tools wait while the owner has taken control; trusted reads never boot a computer. Clearing or deleting a bot shuts its computer down or removes it.
Add the server computer domain and computers.* RPCs (status, start, sleep, shut down, touch, release, thumbnail, viewer) and wire the host into the daemon. /computers/<botId>/ forwards Cua's viewer, its gRPC-Web calls, media socket and files, authorized by a short-lived viewer ticket only a paired device can mint. Client credentials are stripped and the viewer runs sandboxed in an opaque origin. The web server and Vite proxy the route, including WebSocket subprotocols.
A Computer section in the bot details shows a live thumbnail, the state with time left, and Start, Sleep and Shut down. The thumbnail opens a large viewer of the live screen with Take over and I'm done; watching keeps the computer awake, and closing the view hands control back.
|
Hey Vasco, big one, so I pushed a set of fixes on top, one commit each (head
Two things worth knowing:
Feel free to push back on any of these if I misread how something is meant to work. |
The viewer's renewal took control unconditionally, so it could take it back from another device or from the bot after a handoff was answered in chat. A renewal now fails with CONFLICT unless this device still holds control, and the viewer falls back to watching.
|
Thanks for these, I went through each one and they all hold up. I gave it a run with Docker up: a separate daemon from this branch on its own port and data, two paired devices, real Cua containers, and a Haiku bot (through the Claude Code provider from #26, merged locally only) for the handoff.
One problem, fixed in Not tested: shutdown after a failed start ( Smaller points, none blocking:
|
What
Each bot can start its own small Linux desktop with Chromium, on demand, and work on it visually or through the browser. The owner can watch it, take over, and shut it down. This covers phases 1-3 of
.agents/plans/pi-cua.md.How it runs
ghcr.io/trycua/linuximage (arm64/amd64, pinned). Orbs downloadscua-spacesd0.12.1, verifies its SHA-256 and mounts it in; it provides screenshots, input, the accessibility tree, viewer tickets and input leases./var/run/docker.sock, or followsDOCKER_HOST=unix://.... OrbStack, Colima or Podman's compatible socket should work too.--no-sandbox.docker pause) after 5 minutes idle -> off (docker stop) after 30 minutes asleep, with a 2-hour hard limit. It survives daemon restarts; deleting a bot removes its container, volume and token.Bot tools
bot_computer_start,_status,_sleep,_shutdown,_read,_windows,_act(screenshot, click, type, keys, scroll, drag) and_handoff(asks the owner to take over, through the question flow).bot_browser_open,_tabs,_snapshot,_act: Chromium over CDP withplaywright-core.Owner controls (web and macOS desktop)
/rooms/direct/$handle/computerwith the Cua viewer, and Take over, which holds an input lease so the bot waits./computers/<botId>/. Access is through a short-lived viewer ticket that only a paired device can mint; client credentials are stripped and the viewer is sandboxed in an opaque origin.New dependencies
@trycua/cua0.10.1playwright-core1.64.1Not included
Requirements
Docker, or a Docker-API-compatible engine, must be running to start a computer. Everything else in Orbs works without it.
Checks
Rebased onto current
main, which needed two merge fixes:bun.lockwas regenerated, andruntime.service.tskeeps both the notes and computer extensions.bun run checkandbun run lintpass (17/17, Fallow clean); the repo has no project test suite. Used on a live dev daemon: a bot's computer ran healthy in Docker Desktop.