[AdaAibaby]

Summary

Give the client-proxy a per-sandbox bound on concurrent in-flight ingress requests, without fragmenting its shared keepalive connection pool. Gated behind a new flag that defaults to unlimited, so behavior is unchanged until an operator opts in.

Why

Re #3681 (shared-host-service resource accounting). The client-proxy routes every sandbox's ingress through a single shared keepalive pool — pool.Destination.ConnectionKey is the constant "client-proxy" (packages/shared/pkg/proxy/pool/pool.go), deliberately, so connections are reused across sandboxes. It passed a nil ConnectionLimitConfig to the shared reverse proxy (packages/client-proxy/internal/proxy/proxy.go), so there was no per-sandbox bound on concurrent in-flight requests. The only ceilings were process-wide: the pool cap maxClientConns = 16384 and the global resume parking lot (parkingLot.active is one atomic.Int64, default size 512 — not keyed per sandbox, and only for the resume-retry subset).

So one sandbox/team driving many concurrent slow routes consumes client-proxy goroutines/connections bounded only by those global ceilings, not attributed per sandbox — exactly the boundary #3681 asks about. (The orchestrator-side proxy already limits per sandbox via SandboxMaxIncomingConnections; the client-proxy edge had no equivalent.)

The coupling that made this non-trivial

The shared reverse proxy keys both the connection pool (pool.go: p.pool.Upsert(d.ConnectionKey, ...)) and the ingress limiter (handler.go: limiter.TryAcquire(d.ConnectionKey, ...)) off the same ConnectionKey. Naively keying the limiter per sandbox would therefore also split the pool into one-per-sandbox — the opposite of the intended reuse (fd/connection blow-up).

Fix — decouple the limiter key from the pool key

• pool.Destination gains LimitKey + LimiterKey(): the limiter accounts against LimitKey when set, else falls back to ConnectionKey. The pool keeps using ConnectionKey.
• handler.go limiter TryAcquire/Release use d.LimiterKey().
• Embedders that key the pool per lifecycle (orchestrator) set no LimitKey → LimiterKey() returns ConnectionKey → byte-identical to before.
• client-proxy wires a non-nil ConnectionLimitConfig and sets Destination.LimitKey = sandboxId, so limiting is per sandbox while ConnectionKey stays the shared "client-proxy" (pool intact).
• New flag ClientProxyMaxConnectionsPerSandbox, default -1 (no limit) — symmetric with SandboxMaxIncomingConnections. Inert until an operator sets a finite cap.

Changes

File	What
shared/pkg/proxy/pool/destination.go	add LimitKey + LimiterKey() (limiter key ≠ pool key)
shared/pkg/proxy/handler.go	limiter uses d.LimiterKey()
shared/pkg/featureflags/flags.go	new ClientProxyMaxConnectionsPerSandbox (default -1)
client-proxy/internal/proxy/proxy.go	non-nil ConnectionLimitConfig; LimitKey = sandboxId
shared/pkg/proxy/pool/destination_limiterkey_test.go	LimiterKey() fallback/override

Testing

• go test ./packages/shared/pkg/proxy/pool -run LimiterKey ✅, go test ./packages/shared/pkg/proxy ✅
• go test ./packages/orchestrator/pkg/proxy ✅ (fallback makes the orchestrator path identical)
• go build / go vet / gofmt clean for shared + client-proxy
• Pre-existing TestResumeFlights_... panic (redis: invalid URL scheme, needs REDIS_URL) reproduces on clean main; unrelated to this change.

Scope / follow-ups (not in this PR)

• Does not change any default to a finite cap (that is a deployment policy decision).
• The client-proxy server sets no ReadHeaderTimeout — a slow-header bound (cf. the orchestrator tcpfirewall classification work) is a separate change.
• Per-team keying and a per-sandbox parking-lot are possible extensions.