Repository navigation
Conversation
b543267 to
dc75996
Compare
| Experimental. What every Store of a Temporal Stream shares: the error of a failed call, the order | ||
| of a Producer's epochs, and the stored form of a Record. | ||
|
|
||
| ## Errors | ||
|
|
||
| A failed call gives a `StreamError` with a `Reason`. `StreamError::code` gives its gRPC code and | ||
| `StreamError::details` its encoded `StreamErrorDetails`, so every SDK raises the same error for | ||
| the same failure. | ||
|
|
||
| ## The order of epochs | ||
|
|
||
| `epoch::epoch_key` gives the epoch of a Producer as 40 bytes that compare in order. A Store fences | ||
| by this order, so every Store refuses the same writes. `testdata/epochs.json` has the shared | ||
| vectors. `examples/gen_vectors.rs` writes them from a second implementation that shares no code | ||
| with the crate. Run it with `cargo run -p temporalio-streams --example gen_vectors`. | ||
|
|
||
| ## The stored form of a Record | ||
|
|
||
| `record::data_record` gives the bytes that a Store keeps for one Record. They have the Producer of | ||
| the Record and its body, and no offset. `record::BatchLimits` has the limits of one batch: its | ||
| Records, its bytes, and the bytes of one Record. The default is `MAX_BATCH_RECORDS`, | ||
| `MAX_BATCH_BYTES` and `MAX_RECORD_BYTES`, and a Store can set less. The sizes are of the stored | ||
| form. | ||
|
|
||
| ## Protos | ||
|
|
||
| The crate defines no protos. `temporalio-protos` generates them, and `temporalio_streams::proto` | ||
| re-exports them, so a value needs no conversion between the crates. |
There was a problem hiding this comment.
Very LLMy and just restating stuff you could find in the API docstrings. This can be reduced to just a basic summary
There was a problem hiding this comment.
6891b5a cuts it down to the short summary.
| //! Experimental. What every Store of a Temporal Stream shares: the error of a failed call, the | ||
| //! order of a Producer's epochs, and the stored form of a Record. |
There was a problem hiding this comment.
| //! Experimental. What every Store of a Temporal Stream shares: the error of a failed call, the | |
| //! order of a Producer's epochs, and the stored form of a Record. | |
| //! Experimental. Contains shared definitions for Temporal Stream Store implementations. |
Generally speaking I'm pretty anal about us trying to keep our API docstrings sound not so LLMy. I'm not going to comment on every single one but it would be good to take a pass over all of them and fix them up so they make sense contextually and read well.
There was a problem hiding this comment.
Took your wording as is. In 6891b5a I also rewrote the other docstrings that sounded stiff. I'll do another pass when the stack is close to merge.
There was a problem hiding this comment.
I folded #1799 into this PR, so the crate doc mentions the runtime now too.
| Int(i128), | ||
| Arr(Vec<Json>), | ||
| Obj(Vec<(&'static str, Json)>), | ||
| } |
There was a problem hiding this comment.
We shouldn't need a whole independent JSON implementation just for this. We aren't testing serde_json.
There was a problem hiding this comment.
d5d4e11 drops the generator and the vectors too. Nothing outside the crate computes epoch keys, so a unit test in epoch.rs checks the key bytes.
The generator stays a second implementation through its own key math. A custom JSON writer added nothing to that.
The docs read as plain descriptions now, and the README is a short summary like the other crates. The removed tests restated constants or repeated what other tests and the vectors check.
Nothing outside the crate computes epoch keys. The Redis Store calls `epoch_key` and only compares the bytes, so the generator and the JSON vectors had no reader.
The docs read as plain descriptions now. The removed tests repeated other tests or only checked that values pass through to the Store.
The Redis Store keeps the kind number in its Producer keys, so a changed number breaks its data.
temporalio-streams with the types that every Stream Store sharestemporalio-streams with the Stream runtime and shared Store types
The comments didn't say that each read sets its own wait and page size, so the two constants read as fixed settings.
A Store doesn't have to refuse a lower epoch of a Producer. Each Record has what a reader needs to tell the attempts apart.
What was changed
This PR adds the experimental
temporalio-streamscrate. It has the Stream runtime that Core and the SDK bridges call, and the definitions that every Stream Store shares. Core's Worker doesn't use it, and #1801 adds that.StreamStorehas the five operations of the Stream service and the limits of one batch. A bridge implements it for a Store in its own language.StreamRuntimechecks each request, calls the Store, and checks the answer, with the same rules for every Store.StreamErrorandReasongive each failed call one gRPC code and oneStreamErrorDetailsvalue, so every SDK raises the same error for the same failure.epoch::epoch_keyencodes a Producer's epoch as 40 bytes that compare in order. A Store that fences does it by that order.recordhas the stored form of a Record, andBatchLimitsfor the size limits of one batch.idshas the size limits of Stream ids, Producer ids and offsets.The crate defines no protos. It re-exports the Stream protos of #1733 as
temporalio_streams::proto.This PR includes the runtime of #1799, which is closed.
Why?
Core and each SDK bridge need one Store interface and one set of checks, so every Store refuses the same writes. A Store persists the epoch key bytes as its fence, and #1801 hashes them and the
ProducerKindnumbers into a digest that History keeps, so unit tests pin those bytes. The Stores are in temporal-community/sdk-streams#9.Checklist
How was this tested:
cargo test -p temporalio-streams. The runtime tests use a Store that returns the answers that a test sets.Any docs updates needed?