WIP: feat: add opt-in SSE keep-alive to streaming HTTP transports - #233
Draft
sideeffffect wants to merge 1 commit into
Draft
WIP: feat: add opt-in SSE keep-alive to streaming HTTP transports#233sideeffffect wants to merge 1 commit into
sideeffffect wants to merge 1 commit into
Conversation
Add a `keepAlive: Option[FiniteDuration]` parameter to the ox, zio and pekko streaming HTTP server transports. When set, a data-less `ping` Server-Sent Event is emitted on the response stream at that interval while the stream is open, keeping idle connections alive through proxies during long-running tool calls. The events carry no data and are ignored by MCP clients (all three streaming clients skip events without data). Off by default. Closes softwaremill#9 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
sideeffffect
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #9.
What
Long-running tool calls hold the Server-Sent Event response stream open; idle connections can be dropped by proxies/load balancers in between. Each streaming HTTP server transport gains an opt-in
keepAlive: Option[FiniteDuration](defaultNone):When set, a data-less
pingServer-Sent Event (event: ping, nodata:line) is emitted on the response stream at that interval while the stream is open. The events carry no data, so per the SSE spec they are not dispatched as messages — all three streaming clients already skip events viaevent.data.filter(_.nonEmpty)— while the bytes keep the connection warm.How
Injected with each backend's idiomatic operator, only when configured (zero overhead otherwise):
messages.merge(Flow.tick(interval, ping), propagateDoneLeft = true)messages.mergeHaltLeft(ZStream.fromSchedule(Schedule.spaced(interval)).as(ping))Flow[ServerSentEvent].keepAlive(interval, () => ping)In every case the keep-alive stream is tied to the message stream's completion, so the response ends normally when the tool call finishes.
Notes on the design
This addresses the ask in #9 (a convenient keep-alive abstraction) without reviving the deprecated standalone SSE transport, and without a JSON-RPC
pingrequest (which chimp clients would answer withMethodNotFound). sttp'sServerSentEventhas no comment field, so a data-less typed event is the cleanest keep-alive that stays invisible to MCP clients. Kept off by default so it can't change existing behaviour.Tests
One spec per backend (
{Ox,Zio,Pekko}ServerKeepAliveSpec) drives a transport with a short interval and a handler that keeps the stream open, asserting a data-lesspingevent is produced.sbt scalafmtCheckAll compile Test/compile compileDocspasses locally. Marked as draft/WIP.🤖 Generated with Claude Code