MCP sessions: a connected agent keeps working after a SatGate restart

[matt-dean-git]

Draft. Head: 4675f52 (base 4f7f276). Companion: https://github.com/SatGate-io/satgate-enterprise/pull/522

Problem

Streamable HTTP sessions live in process memory. Every deploy, Fly host event or request that lands on another machine loses them, and the server answers the client's Mcp-Session-Id with 404 "session not found".

What clients do with that 404 (measured on Dean, local gateway restarted mid-session)

Each run used a local satgate-mcp built from main, header auth, a logging proxy in front, and an isolated HOME. The gateway was restarted between two tool calls in one client process.

• Claude Code 2.1.92: Streamable HTTP error: Error POSTing to endpoint: session not found. It keeps sending the dead id, so every later call fails until the client is restarted. It never re-initializes.
• goose 1.53.0: on the 404 it re-initializes and repeats the call in the new session. The user sees no error.
• Hermes 0.22.1: the call that was in flight fails ("transport session expired ... NOT automatically retried"). Hermes then reconnects, and the next call works.

With this branch, all three made their calls across one or more restarts under the same session id with no error: Claude Code (3 turns), goose, and Hermes (whose session survived 3 restarts).

Design

(a) Resume under the same id. Only in header auth mode, and only for the token the id was issued to.

• Session ids are now mcp-<32 hex random>-<32 hex tag>, where the tag is HMAC-SHA256 keyed by SHA-256(opening token) over the random part. No server secret is needed, so any process or machine can check the tag.

• Suppose a POST carries an id this process does not hold. The server serves it again under that id when all of these hold:

  • the bearer passes the same checks as initialize: revoked gets 403 TOKEN_REVOKED (or 401 with the OAuth challenge), missing or failing verification gets 401;
  • the revocation checker does not list the token;
  • the tag matches the bearer (otherwise 404, as before);
  • the id was not DELETEd on this process (a tombstone is kept for the idle TTL).

  The protocol comes from the MCP-Protocol-Version header (400 if unsupported), or 2025-03-26 if the header is missing. The resumed session counts against the session cap and publishes session_connect with resumed: true.

• Unchanged: DELETE and GET of an unknown id, auth modes none and config (they still answer 404), legacy SSE, and every check on a live session.

Why not a Redis session registry: the public module has no Redis. A registry would need a write on every initialize and DELETE, a read on every miss, and a plan for Redis being down. It would also move nothing that matters: a session holds only the token hash, the verified identity and the protocol, and all of that can be rebuilt from the verified token and one header. (a) is stateless across machines, so it also covers requests that land on the second staging machine.

Why bind the id to the token: session ids are logged and shown in events, so they are not secret the way tokens are. With plain (a), anyone holding any valid token could post the id first after a restart and own it. The owner would then get 403 forever, and Claude Code does not recover from that.

Security argument

• The id alone grants nothing. A resume runs the initialize checks on the presented token, and every later request is checked against the token hash exactly as before.

• No takeover. The tag must match the presented token (128-bit tag, constant-time compare). A different token, from the same tenant or another one, gets 404, nothing is created, and the owner can still resume afterwards.

• Revoked or expired tokens are refused before any session exists. A request without a valid token gets 401.

• What a token may call is unchanged: calls go through the same Proxy.handleRequest with the resuming token's verified TokenInfo.

• DELETE still ends a session. The process that ended it never brings the id back. A process that never saw the DELETE (after a restart, or another machine) serves it again only to the same valid token. That token could open an identical session with initialize at any time.

• Limits cannot be bypassed by forcing a new session, because none of them is per session:

  • rate limits are per verified token;
  • budget is per budget id;
  • spend counters and approval holds are per token or per call.

  Resuming keeps the session id, so the enterprise spend caller scope (and its 24 h duplicate guard) is kept. A call re-sent after the restart is still a duplicate (enterprise end-to-end test).

• Memory is bounded: resumed sessions count against the cap, and tombstones use the same cap and sweep.

• Known effect: ids issued before this change have no tag, so they are not resumed. The deploy that ships this still breaks Claude Code sessions one last time.

Tests

New in pkg/mcpserver/streamable_resume_test.go. A "restart" is a new Proxy and SSEServer with an empty session map and the same authenticator.

Test	On main 4f7f276
ResumeAfterRestartSameToken	FAIL: 404 session not found
ResumeWithoutProtocolHeaderUsesDefault	FAIL: 404
ResumeRefusesAnotherToken (other tokens 404 and create nothing; owner resumes afterwards)	FAIL at the owner step: 404
ResumeRefusesRevokedAndInvalidTokens (403 TOKEN_REVOKED, 401)	FAIL: main answers 404 for everything, so the owner could never resume
ResumeRefusesExpiredMacaroon	FAIL: 404 while still valid
ResumeDeletedSession	FAIL at "same valid token on a new process": 404
ResumeGETStreamAfterRestart (GET is 405 before and after and creates nothing; the next POST resumes)	FAIL at the POST: 404
ResumeRespectsSessionCap	FAIL: 404 instead of 503
ResumeConcurrentRequestsShareOneSession	FAIL: 404
ResumeOnlyInHeaderMode	passes on main (guard: unchanged behaviour)
streamable_resume_id_test.go: StreamSessionIDIsBoundToItsToken	new function; does not compile on main

Main run: these tests copied into git archive 4f7f2768, go test -run Resume ./pkg/mcpserver/, exit 1.

Gates on this branch: gofmt -l on the changed files prints nothing; go vet ./... exit 0; go test -race -count=1 ./pkg/mcpserver/... exit 0.

Enterprise bump needed

The enterprise wrapper (internal/mcpgateway/session_spend_http.go) answers 404 for an unknown id before this handler runs. The companion enterprise PR changes that and works with the pinned module, where the answer stays 404. Its end-to-end tests skip until gateway/go.mod is bumped to a version that contains this commit. The bump itself is not part of this lane. With a local -modfile replace pointing at this branch, the enterprise gates (vet, build, go test -race over ./internal/mcpgateway/... ./internal/cloud/...) all pass.

CI (head 4675f52)

Actions run 38971164434: Go, Node.js SDK and Python SDK pass; Vercel passes.

Round 2

Answers the NO-GO reviews on satgate-enterprise#522 (Astra B1-B4, Grok B1-B3). Head: 8f02065 (round 1 head 4675f52).

• Until the linked module resumes sessions, unknown sessions get today's 404 unchanged. This PR adds SSEServer.ResumesStreamSessions() (pkg/mcpserver/streamable_resume.go:90), true only in header auth mode. The enterprise wrapper (#522) type-asserts it and forwards an unknown id only when it is there and true. With the module gateway/go.mod pins today it is absent, so Cloud keeps main's immediate 404.
• An ended session stays ended. An unexpired DELETE tombstone is never dropped (streamable_resume.go:167 markStreamSessionEndedLocked). While the tombstone map (capped at the session cap) is full of unexpired entries, every resume on that process is refused with the existing 404 (streamEndedFullLocked, streamResumeRefusedLocked). A DELETE that finds the map full cannot record its id, so every resume on that process stays refused until that id's own TTL is over (streamEndedFullUntil). Otherwise that id could come back as soon as the earliest tombstone expired. initialize and live sessions are not affected.
• No race with DELETE. The tombstone check runs again under streamMu, in the same critical section as the insert (streamable_http.go:533 addStreamSession -> streamSessionEndedHere -> 404). A DELETE that completes first always wins.
• Tests fail instead of skipping (enterprise side, #522).

New tests (pkg/mcpserver). Each was also run on the round 1 head 4675f52:

Test	Kind	Round 1 head	This head
TestStreamableResumeEndedSessionsStayEndedWhenTombstonesFull	defect	FAIL: ended[2] with the tombstone map full: status=200	PASS
TestAstraDeletedTombstoneCapacity (Astra's adverse test, kept)	defect	FAIL: deleted session revived under capacity pressure: got 200	PASS
TestAstraDeleteRacesResume (Astra's adverse test, kept)	defect	FAIL: resume paused across the DELETE: status=200	PASS
TestStreamableResumeConcurrentDeleteNeverRevives (150 iterations, -race)	defect	FAIL: iteration 0: 18 requests sent after the DELETE returned were served	PASS (also -count=5)
TestStreamableResumesStreamSessionsOnlyInHeaderMode	control (new method)	n/a (method absent)	PASS

Every round 1 test is unchanged and passes.

Gates on this head: gofmt -l on the changed files: empty. go vet ./...: exit 0. go test -race -count=1 ./pkg/mcpserver/...: exit 0 (38.7 s).

CI on 8f02065: Actions run 38990828458: Go pass (3m16s), Node.js SDK pass, Python SDK pass; Vercel and Vercel Preview Comments pass.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
satgate	Ready	Preview	Oct 17, 2026 1:26am UTC