Skip to content

feat: let Workflow code read, append to and close a Stream, through Core - #2132

Draft
moetemp wants to merge 11 commits into
moe/streams-apifrom
moe/streams-direct
Draft

moetemp wants to merge 11 commits into
moe/streams-apifrom
moe/streams-direct

Conversation

@moetemp

@moetemp moetemp commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

What changed?

This PR gives Workflow code the Streams API. Workflow code reads a Stream with the same read, read_pages and read_page as a client, and it appends to a Stream and closes it. Core makes these three calls itself, inside the Workflow Task.

It's a prototype, created for the purpose of getting your feedback on the API. Its base is #2129, and Core's side is temporalio/sdk-rust#1801.

events = workflow.get_stream(events_ref, body_type=ModelEvent)
status = workflow.get_stream(status_ref, body_type=str)

# Workflow code decides on each event while the model Activity still runs.
tools = []
async for record in events.read(wait=timedelta(seconds=30)):
    event = record.body
    if event.kind == "tool_call":
        tools.append(
            workflow.start_activity(
                run_tool, event.tool_call, start_to_close_timeout=timedelta(minutes=1)
            )
        )
        await status.append(f"Running {event.tool_call.name}")
results = await asyncio.gather(*tools)
  • The read. With wait, a call waits that long for a new Record and then gives an empty page. With no wait, it gives what is in the Stream. Workflow code gets each page as it arrives, at most 256 Records in one activation.
  • append. Core writes the Records with the Workflow as the Producer and the Workflow Task as the epoch.
  • close. It takes effect after its Workflow Task is in History, so an awaited close spans two Workflow Tasks. A Workflow Task that fails closes nothing, and its retry can still append.
  • create_stream and describe run in a Local Activity.
  • BufferedAppender. It's for client and Activity code, and it raises RuntimeError in Workflow code. Workflow code passes its Records to one append call.
  • History. One core_stream marker for each Workflow Task with a Stream result, whatever the number of Records. By default it has a range, a count and a digest, and a replay reads the Records from the Store again. With workflow_read_replay_from=StreamReplaySource.HISTORY it has the Records.
  • A Store outage. Core tries a call again for up to 30 s before Workflow code gets the error. A replay whose Store stays down fails only its Workflow Task. A replay whose Records are gone fails the task with StreamReplayError, or the Workflow with that class in workflow_failure_exception_types.
  • Settings on StreamConfig: workflow_read_page_records (the cap of 256), workflow_read_min_gap (an opt-in least time between two batches, for fewer activations on a slow Stream) and workflow_read_ahead_bytes (4 MiB).
  • Stores. Core makes these calls on any StreamStore. The bridge has one adapter, which runs the Store's method as a task on the Worker's event loop, never on a Workflow thread. The Stream runtime of feat: add the StreamStore trait and the Stream runtime to temporalio-streams sdk-rust#1799 checks each request and each answer, and it refuses a batch over the limits of the Store. The docs of StreamStore list what a Store must do for Workflow code.

It depends on two things that have no release yet.

Why?

A library that iterates a Stream in Workflow code needs each Record while its Producer still runs. ADK is an example, since its own loop reads the model through an async generator.

It has these costs.

  • A read keeps its Workflow Task open while a call waits on it. So a Signal or an Activity result reaches the Workflow at the heartbeat at the latest (about 8 s by default).
  • A read that waits on a quiet Stream costs one Workflow Task about every 8 s. Code on a Stream that can be quiet for long leaves the loop after an empty page.
  • A read wakes Workflow code for each Record on a Stream that isn't busy, which costs CPU. workflow_read_min_gap trades latency for it.
  • An append is at-least-once. After a Worker crash, a rerun can write a second copy.
  • A call that Core makes on a Store goes through Python. Against a Store inside the bridge, that costs about 100 µs for each call on a local Redis. A read pays it for each page, and an append for each batch.

How did you test it?

The tests are in tests/nexus/test_stream_direct.py and tests/nexus/test_stream_store_workflow.py. They run on the local dev server with a Store that is written in Python, with a Workflow cache and with none.

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

@moetemp moetemp changed the title feat: add iter to the Workflow Stream, with Direct reads and appends in Core feat: add iter to the Workflow Stream, with Direct reads, appends and closes in Core Nov 15, 2026
@moetemp
moetemp force-pushed the moe/streams-direct branch 3 times, most recently from 88cddf9 to 5fe21e2 Compare November 17, 2026 06:51
@moetemp moetemp changed the title feat: add iter to the Workflow Stream, with Direct reads, appends and closes in Core feat: read a Stream in Workflow code, with Direct reads, appends and closes in Core Nov 17, 2026
@moetemp moetemp changed the title feat: read a Stream in Workflow code, with Direct reads, appends and closes in Core feat: let Workflow code read, append to and close a Stream, through Core Nov 17, 2026
@moetemp
moetemp added this pull request to stack #2139 November 18, 2026 01:19
@moetemp
moetemp force-pushed the moe/streams-direct branch 2 times, most recently from 29aef59 to 6b014ea Compare November 18, 2026 15:14
Core makes a Workflow's read, append and close inside the Workflow Task, with one marker for the task. A create and a describe run in a Local Activity.
Core gave the Worker two read settings, so `StreamConfig` passes them on with the same checks.
A gap trades latency for fewer activations of Workflow code.
A read, an append and a close of Workflow code go through Core, which records the marker. Core
gets a runtime that runs the Python Store's calls on the Worker's event loop, so the refusal of
such a Store is gone.
A bug in a Store never gets better, so a retry only delayed the error. Such an exception reaches
Workflow code on the first try as a non-retryable `STORAGE_CORRUPT`, and an outage still retries.
The Workflow `Stream` has `read`, `read_pages` and `read_page` from commit 07b565b6 of the nexgen
fork. A Stream call no longer reaches the generic hook, so the page count test uses
`start_read_records`.
The Redis Store moved to its own package, with the tests of its outages. The test of a Worker with
no Workflows keeps its proof on a small Store of the bridge that fails to build its runtime.
The next commit takes the memory Store out of the bridge. `DictStreamStore` now has the wait of a
read that Core needs, with the resend and the fence of a batch. The test that compared the limits
of a client read with a Workflow read leaves, since a client read of a Python Store has no check.
Core's Stream crate has one `StreamStore` trait and no Store of its own, so the bridge keeps only
the adapter that calls a Store written in Python. The `streams` cargo feature goes, since Core
links the crate in every build.
Workflow code passes its Records to one `append` call, and a flush by time has no meaning there.
A Store doesn't have to refuse a lower epoch of a Producer.
Each Record has what a reader needs to tell the attempts apart.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant