DUPLOAI-3742: run the extension installer beside the studio

.env.example

@@ -9,12 +9,14 @@
 #   • UI    : the duplo-ui git SHA (e.g. 02ae64f593513d17a7713fdaeead7b74956085a4)
 #   • Studio: deploy_tag = branch-<sanitized-branch>-<short-sha>  (e.g. branch-feature-plugin-acc46696)
 #   • Agent : a main tag (no extension-specific changes)
+#   • Installer: a main build, main-[short-sha]. It installs your extension builds (see README)
 # ─────────────────────────────────────────────────────────────────────────────
 
 STUDIO_TAG=dev-1.0.6-30d4b919
 AGENT_TAG=dev-0.1.2-3bbb19d
 UI_TAG=146bc5b181363bcf360043e1c0d34676a4713ef5
 XTERM_TAG=dev-0.1.2-1e96285
+INSTALLER_TAG=main-4780968f
 
 # Qdrant vector database (Knowledge Base). The API key is fixed and not a secret — Qdrant is bound to
 # the compose network and localhost only. Change it if you like; compose and scripts/register-qdrant.sh

README.md

@@ -11,7 +11,7 @@ hot-loads into the running platform with no restart.
 
 ## Why you'd want it
 
-- **A whole platform, locally.** Six containers, one command.
+- **A whole platform, locally.** Seven containers, one command.
 - **Your own resource types.** Not plugins bolted on the side — they get a REST API, a portal UI, and a
   provisioning workflow, exactly like the built-in types.
 - **Hot-load, no restart.** Build, deploy, and the new type is live in the UI seconds later.
@@ -49,7 +49,7 @@ region against six security rules, charts the violations over time, and fixes th
 
 | | |
 | --- | --- |
-| `docker-compose.yml` | The platform from published images — mongo, qdrant, studio, agent, ui, xterm |
+| `docker-compose.yml` | The platform from published images: mongo, qdrant, studio, agent, dind, ui, xterm, installer |
 | `.env` / `.env.example` | Image tags, auth, LLM credentials, and the build target |
 | `run.sh` `stop.sh` `logs.sh` | Lifecycle |
 | `scripts/` | Build, deploy, register, and upgrade |
@@ -58,6 +58,69 @@ region against six security rules, charts the violations over time, and fixes th
 | `extensions/terraform/` | The real, shipping **Terraform extension** — source, fetched once by `run.sh`, disconnected from git and yours to modify |
 | `extensions/<name>/` | **Yours** (subject to our and any third party's rights in the underlying software and technology upon which they are built). The one thing an upgrade never touches |
 
+## Installing an extension
+
+The `installer` service installs extensions into the running studio and keeps them current.
+
+**Your own build.** Build it into `extensions/[name]/dist/`:
+
+```bash
+./scripts/build-extension.sh extensions/[name]
+```
+
+The installer installs each build in `extensions/[name]/dist/` that changes while it runs, within about 20 seconds,
+including a rebuild at the same version. The studio may keep running the previous backend DLL after a same-version
+reload, as `deploy-extension.sh` warns, so bump the version for backend changes.
+
+A build already in `dist/` when the installer starts, as after `./run.sh --reset`, waits for its next rebuild, and so
+does an extension you remove. The installer also leaves alone a build older than the installed version, unless that
+version failed to load.
+
+`./scripts/deploy-extension.sh extensions/[name]/dist/extension.zip` hands that build to the installer while it runs,
+waits for the installer to report it, and exits non-zero with the installer's reason when it does not install it. For
+a build the installer leaves alone, with the installer stopped or `EXTENSIONS_ENABLED=false`, for an id in
+`EXTENSIONS_EXCLUDE`, or for a zip outside `extensions/`, it loads the zip itself.
+
+The installer reinstalls a same-version rebuild only over a build it installed itself. After a load it did not make,
+such as `deploy-extension.sh` with the installer stopped or Extension Studio's in-platform flow, bump the version or
+run `./scripts/remove-extension.sh [id]` before relying on the installer for a rebuild.
+
+**From the catalog.** The Available tab lists the extensions your license entitles you to. Install there hands
+the click to this installer only once the UI tag and the studio tag in `.env.example` are images that do that.
+Those two tags predate that handoff. The installer reads that catalog only when `.env` sets these keys, so by
+default it runs on your own builds alone:
+
+```bash
+LICENSE_SERVER_URL=https://console.duplocloud.com
+URL_ROUTE_URL=https://results.prod-apps.duplocloud.net/urls
+CHANNELS_BASE_URL=https://channels.prod-apps.duplocloud.net
+CHANNELS_BUCKET=duplo-helpdesk-channels
+```
+
+Re-run `./run.sh` after changing them. `EXTENSIONS_ENABLED=false` in `.env` stops the installer installing anything,
+and `EXTENSIONS_EXCLUDE` takes a JSON array of extension ids it leaves alone. `docker-compose.yml` lists the other
+`EXTENSIONS_` keys it passes through.
+
+**How it checks an install.** The installer judges an install by the studio's `/healthz` and the extension's row. It
+sees a studio restart only as missed `/healthz` reads spanning at least 5 seconds followed by an answer. A quicker
+restart goes unseen, and the row alone then judges the install.
+
+**A load that hangs the studio.** Under compose the installer backs out only through the studio's API, so it cannot
+back out a build that hangs the studio when it starts. Stop the studio, disable the extension's row by hand, with
+`[id]` its manifest id, and start the studio again:
+
+```bash
+docker compose stop duplo-ai-studio
+docker compose exec mongo mongosh -u authuser -p authpass --authenticationDatabase admin duplo-ai-helpdesk \
+  --eval 'db.loaded_extensions.updateOne({ExtensionId: "[id]", IsCurrent: true}, {$set: {Status: NumberInt(1)}})'
+docker compose start duplo-ai-studio
+```
+
+To install a fixed build, run `./scripts/remove-extension.sh [id]` and then rebuild. The rebuild has to differ in
+content from the build that hung, since the installer may hold that exact build. An install whose load timed out also
+waits until you restart the studio, with `docker compose restart duplo-ai-studio`. Under podman, use `podman compose`
+for each `docker compose` above.
+
 ## Documentation
 
 | Page | What it covers |

SUPPORT.md

@@ -45,8 +45,8 @@ usually a report nobody can act on.
 
 Useful to attach, once redacted: `./logs.sh --no-follow` (or one service, e.g.
 `./logs.sh --no-follow duplo-ai-studio`), `docker compose ps`, `docker compose version`, your OS and
-architecture, and the `STUDIO_TAG` / `UI_TAG` / `AGENT_TAG` / `XTERM_TAG` lines from `.env` — those
-tags are not secret and they pin down exactly which build you are on.
+architecture, and the `STUDIO_TAG` / `UI_TAG` / `AGENT_TAG` / `XTERM_TAG` / `INSTALLER_TAG` lines from `.env`.
+Those tags are not secret and they pin down exactly which build you are on.
 
 ## Never paste
 

docker-compose.yml

@@ -43,16 +43,21 @@ services:
       - qdrant_data:/qdrant/storage
 
   # One-shot: chown the shared named volumes to uid 1001 so the non-root studio + agent
-  # (both appuser 1001) can write them. Named volumes are created root-owned, and run.sh
-  # does `down -v` each start, so this must run every startup (see the services' depends_on).
+  # (both appuser 1001) can write them. Named volumes are created root-owned, on first start and again after
+  # `run.sh --reset` or `stop.sh --wipe` removes them, so this runs every startup (see the services' depends_on).
   # Both processes writing as 1001 keeps ticket/skill/extension dirs interoperable.
+  # The installer's state volume goes to 65532 instead, the distroless nonroot user its image runs as.
   init-perms:
     image: busybox:1.36
-    command: ["sh", "-c", "chown -R 1001:1001 /data /extstudio /csessions && echo 'volumes chowned to 1001'"]
+    command:
+      - sh
+      - -c
+      - chown -R 1001:1001 /data /extstudio /csessions && chown -R 65532:65532 /installer && echo 'volumes chowned'
     volumes:
       - platform_data:/data
       - extension_studio_data:/extstudio
       - claude_sessions:/csessions
+      - installer_state:/installer
 
   duplo-ai-studio:
     image: quay.io/duplocloud/backend:${STUDIO_TAG}
@@ -75,6 +80,7 @@ services:
       # Browser reaches xterm on the published host port, so this must be host-reachable.
       - AIStudio__XtermHost=http://localhost:${XTERM_PORT:-6060}
       - LocalFileStorage__RootPath=/data
+      - Extensions__InstallerUrl=http://installer:8080
     volumes:
       # Shared /data file store (skills + loaded-extension skill files). Shared with the agent so it can
       # read the seeded skills. Survives restart.
@@ -88,6 +94,49 @@ services:
       init-perms:
         condition: service_completed_successfully
 
+  # Installs and upgrades extensions in the studio: each build that changes in extensions/[name]/dist/ while it runs,
+  # and a catalog extension when the Available tab asks. Only the studio calls it, signing each call with the shared JWT
+  # secret, so no host port is published.
+  installer:
+    # Defaults to .env.example's pin, so `docker compose up` before run.sh copies INSTALLER_TAG into .env still starts.
+    image: quay.io/duplocloud/helpdesk-installer:${INSTALLER_TAG:-main-4780968f}
+    restart: unless-stopped
+    environment:
+      - INSTALLER_PLATFORM=compose
+      - INSTALLER_BACKEND_URL=http://duplo-ai-studio:60021
+      # The studio service's image, which the status record reports and holds are keyed on.
+      - INSTALLER_BACKEND_IMAGE=quay.io/duplocloud/backend:${STUDIO_TAG}
+      - INSTALLER_STATE_DIR=/var/lib/installer
+      - INSTALLER_LOCAL_DIR=/extensions
+      # One key at a time rather than env_file, so the installer holds none of .env's LLM, AWS or admin credentials.
+      - Authentication__JwtSharedSecret=${Authentication__JwtSharedSecret:-}
+      - Licensing__Token=${Licensing__Token:-}
+      # The selection. With no selection key set, every pass skips as no-release-config and installs nothing, so the
+      # installer reads an empty value as unset.
+      - EXTENSIONS_ENABLED=${EXTENSIONS_ENABLED:-true}
+      - EXTENSIONS_SOURCE=${EXTENSIONS_SOURCE:-customer}
+      - EXTENSIONS_EXCLUDE=${EXTENSIONS_EXCLUDE:-}
+      - EXTENSIONS_RESET_AT=${EXTENSIONS_RESET_AT:-}
+      - EXTENSIONS_UPDATE_INSTALLED=${EXTENSIONS_UPDATE_INSTALLED:-}
+      - EXTENSIONS_FROZEN=${EXTENSIONS_FROZEN:-}
+      - EXTENSIONS_DRY_RUN=${EXTENSIONS_DRY_RUN:-}
+      - EXTENSIONS_OPT_OUT=${EXTENSIONS_OPT_OUT:-}
+      - EXTENSIONS_LAYERS=${EXTENSIONS_LAYERS:-}
+      - EXTENSIONS_CONFIG_INVALID=${EXTENSIONS_CONFIG_INVALID:-}
+      - EXTENSIONS_ENROLLMENT_ID=${EXTENSIONS_ENROLLMENT_ID:-}
+      # The license server's catalog, off while LICENSE_SERVER_URL is empty, so the installer runs on local builds only.
+      - LICENSE_SERVER_URL=${LICENSE_SERVER_URL:-}
+      - URL_ROUTE_URL=${URL_ROUTE_URL:-}
+      - CHANNELS_BASE_URL=${CHANNELS_BASE_URL:-}
+      - CHANNELS_BUCKET=${CHANNELS_BUCKET:-}
+    volumes:
+      # The status record. A named volume, so it survives a recreate and goes with the database on `down -v`.
+      - installer_state:/var/lib/installer
+      - ./extensions:/extensions:ro
+    depends_on:
+      init-perms:
+        condition: service_completed_successfully
+
   claude-code-agent:
     image: quay.io/duplocloud/duplo-agent:${AGENT_TAG}
     restart: unless-stopped
@@ -304,6 +353,7 @@ volumes:
   extension_studio_data:
   claude_sessions:
   qdrant_data:
+  installer_state:
   dind_storage:
   # Keyed by uid on purpose: these are written as the calling user (see the service's `user:`), and one
   # uid's runtime-created cache subdirectories are not writable by another, so sharing a single pair

docs/architecture.md

@@ -16,7 +16,8 @@ Nothing here builds from source. Every service pulls a published image pinned by
 | `mongo` | `mongo:7.0.14` | 27018 | All platform state, including the `loaded_extensions` records. Matches the version pinned in the helpdesk-helm chart. |
 | `qdrant` | `qdrant/qdrant:v1.18.2` | 6333 | The vector database behind the Knowledge Base. The studio reaches it in-network at `http://qdrant:6333`; the host port is published so its dashboard is available for debugging. Runs with RBAC on, so the studio can mint read-only per-collection tokens instead of sharing the master key. |
 | `xterm` | `quay.io/duplocloud/duplo-xterm` | 6061 | In-browser terminal. The browser reaches it on the published host port, so `AIStudio__XtermHost` must be host-reachable. |
-| `init-perms` | `busybox:1.36` | — | One-shot. Chowns the shared named volumes to uid 1001 so the non-root studio and agent can write them. Named volumes are created root-owned, so this runs on every startup. |
+| `installer` | `quay.io/duplocloud/helpdesk-installer` | none | Installs and upgrades extensions through the studio's API, from your builds in the read-only `extensions/` mount and, when configured, the license server's catalog. The studio calls it at `Extensions__InstallerUrl`, so it publishes no host port. See [Installing an extension](../README.md#installing-an-extension). |
+| `init-perms` | `busybox:1.36` | none | One-shot. Chowns the shared named volumes to uid 1001 so the non-root studio and agent can write them, and the installer's state volume to 65532, its distroless nonroot user. Named volumes are created root-owned, so this runs on every startup. |
 
 Ports shown are the dev kit's `.env.example` defaults, mostly offset from the platform's standard ports so
 this stack can run alongside a full local platform. See [configuration.md](configuration.md#host-ports).
@@ -45,6 +46,7 @@ the UI, but each upload fails. See [configuration.md](configuration.md#knowledge
 | `extension_studio_data` | studio | Unpacked hot-loaded extension bundles. |
 | `claude_sessions` | agent | Agent sessions, under its real home (`/home/appuser/.claude`). |
 | `qdrant_data` | qdrant | Knowledge Base vectors and their payloads. |
+| `installer_state` | installer | The installer's status record, `installer-status.json`. |
 
 Because extensions live in a named volume **and** are recorded in Mongo, they survive a container
 recreate: `ExtensionReplayHostedService` re-registers the active records on boot. That is why

docs/cli-reference.md

@@ -107,7 +107,7 @@ extensions, Mongo, the file store, and the Knowledge Base vectors are lost.
 ```
 
 A thin wrapper around `docker compose logs -f`; every argument is passed straight through. Services are
-`duplo-ai-studio`, `claude-code-agent`, `dind`, `duplo-ui`, `mongo`, `qdrant`, and `xterm`.
+`duplo-ai-studio`, `claude-code-agent`, `dind`, `duplo-ui`, `installer`, `mongo`, `qdrant`, and `xterm`.
 
 ## What each command destroys
 

docs/configuration.md

@@ -110,6 +110,7 @@ Pin the published images the stack runs.
 | `AGENT_TAG` | `main-d27e511` | The `claude-code-agent` image that executes provisioning tickets. A `main` build — it carries no extension-specific changes. |
 | `UI_TAG` | `2a949c536be71efd4501dd60ca6fdda4f7b4676a` | The Angular portal image. A `duplo-ui` git SHA, pinned to a build that renders the license state. |
 | `XTERM_TAG` | `main-d323e0b` | The in-browser terminal image. |
+| `INSTALLER_TAG` | `main-4780968f` | The extension installer image. Its own keys are under [Installing an extension](../README.md#installing-an-extension). |
 | `STUDIO_PLATFORM` | `linux/amd64` | Docker platform for the studio image. On Apple Silicon the studio image is amd64-only and runs emulated. Leave as-is unless you have an arm64 image. |
 
 `STUDIO_TAG` and `UI_TAG` are required — `run.sh` exits if either is empty:

docs/getting-started/build-your-first-extension.md

@@ -228,6 +228,11 @@ hot-loads it. Those last two steps are the same two commands you would run by ha
 ./scripts/deploy-extension.sh extensions/s3-guard/dist/extension.zip
 ```
 
+With the installer service running, the build alone is enough, since the installer loads it. A build loaded
+by `deploy-extension.sh` is not one the installer reinstalls at the same version, so after a script load,
+bump the version or remove the extension before relying on the installer for a rebuild (see
+[Installing an extension](../../README.md#installing-an-extension)).
+
 Their flags are in the [CLI reference](../cli-reference.md). If the build fails to reach the host SDK,
 that symptom is in [troubleshooting.md](../troubleshooting.md#extensions).
 

docs/getting-started/install.md

@@ -169,9 +169,12 @@ You are done when you see:
 
 Build & deploy your extension (scripts read the target from .env — no DUPLO_BASE= prefix needed):
   ./scripts/build-extension.sh  extensions/<name>               # your extensions live in extensions/<name>/
-  ./scripts/deploy-extension.sh extensions/<name>/dist/extension.zip
+  # the installer service loads each build in extensions/<name>/dist/ by itself, within about 20 seconds
+  # without the installer, or on a remote target: ./scripts/deploy-extension.sh extensions/<name>/dist/extension.zip
   # or build every extension:    ./scripts/build-all.sh
   # or build a bundled sample:   ./scripts/build-extension.sh samples/helloworld
+  #   then load it, since the installer reads only extensions/:
+  #   ./scripts/deploy-extension.sh samples/helloworld/dist/extension.zip
   # re-attach the agent to another workspace: ./scripts/register-agent.sh <workspace-id>
 ```