From e817b9c0db99d262635344ef6225fd99a3eae798 Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:48 +0200 Subject: [PATCH 1/6] Split and expand the Chronicle integration documentation - Resolve the contradiction about returned reactor commands and move reactors into chronicle/reactors with an overview and a page on returning commands - Add pages for event metadata, subject, compliance, read models in commands, read-model resolution failures, and the ARCCHR mapping - Split aggregates into defining and injecting pages with complete code - Expand causation, concurrency, and transactional commands from source - Correct the claim that @cratis/cratis is private --- Documentation/chronicle/add-event-sourcing.md | 106 ++++++--------- .../aggregates/defining-an-aggregate-root.md | 91 +++++++++++++ Documentation/chronicle/aggregates/index.md | 105 ++++----------- .../aggregates/injecting-into-commands.md | 84 ++++++++++++ Documentation/chronicle/aggregates/toc.yml | 6 + Documentation/chronicle/code-analysis.md | 36 +++++ Documentation/chronicle/commands/causation.md | 76 +++++++++-- .../chronicle/commands/concurrency.md | 79 +++++++++-- .../chronicle/commands/event-metadata.md | 93 +++++++++++++ Documentation/chronicle/commands/index.md | 3 +- Documentation/chronicle/commands/subject.md | 89 +++++++++++++ Documentation/chronicle/commands/toc.yml | 4 + .../commands/transactional-commands.md | 67 ++++++++-- Documentation/chronicle/compliance.md | 38 ++++++ Documentation/chronicle/index.md | 98 +++++++++++--- .../reactors/command-side-effects.md | 122 +++++++++++++++++ Documentation/chronicle/reactors/index.md | 62 +++++++++ Documentation/chronicle/reactors/toc.yml | 4 + .../chronicle/read-models/failures.md | 61 +++++++++ Documentation/chronicle/read-models/index.md | 78 +++++++---- .../read-models/injecting-into-commands.md | 124 ++++++++++++++++++ Documentation/chronicle/read-models/toc.yml | 4 + .../chronicle/resolving-event-source-id.md | 9 +- Documentation/chronicle/toc.yml | 8 +- Documentation/index.md | 3 +- 25 files changed, 1219 insertions(+), 231 deletions(-) create mode 100644 Documentation/chronicle/aggregates/defining-an-aggregate-root.md create mode 100644 Documentation/chronicle/aggregates/injecting-into-commands.md create mode 100644 Documentation/chronicle/aggregates/toc.yml create mode 100644 Documentation/chronicle/code-analysis.md create mode 100644 Documentation/chronicle/commands/event-metadata.md create mode 100644 Documentation/chronicle/commands/subject.md create mode 100644 Documentation/chronicle/compliance.md create mode 100644 Documentation/chronicle/reactors/command-side-effects.md create mode 100644 Documentation/chronicle/reactors/index.md create mode 100644 Documentation/chronicle/reactors/toc.yml create mode 100644 Documentation/chronicle/read-models/failures.md create mode 100644 Documentation/chronicle/read-models/injecting-into-commands.md diff --git a/Documentation/chronicle/add-event-sourcing.md b/Documentation/chronicle/add-event-sourcing.md index 73652507..3f7c6a86 100644 --- a/Documentation/chronicle/add-event-sourcing.md +++ b/Documentation/chronicle/add-event-sourcing.md @@ -3,7 +3,7 @@ title: Add event sourcing description: Register the experimental Chronicle integration on the application builder with a connection string or a caller-owned client, and prepare the Node entry point. --- -This page adds Chronicle to an Arc application. After it, commands can [return events](commands/index.md). +This page adds Chronicle to an Arc application. When you finish, a command can [return events](commands/index.md) and a query can serve [projected read models](read-models/index.md). The Chronicle kernel runs as a separate process; start one before you run the application. ## Prepare the entry point @@ -27,98 +27,68 @@ const app = await builder.build(); await app.run(); ``` -The equivalent C# setup, alongside the TypeScript `builder.withChronicle(...)` above, is: +Importing `@cratis/arc.chronicle` installs the typed `withChronicle` builder method, so `@cratis/arc.core` keeps no dependency on Chronicle. The exported function `withChronicle(builder, options)` does the same; the [Library sample](https://github.com/Cratis/Arc.TypeScript/blob/main/Samples/Library/main.ts) uses that form. -```csharp -var builder = WebApplication.CreateBuilder(args); -builder.AddCratisArc(configureBuilder: arc => arc.WithChronicle()); -var app = builder.Build(); -app.UseCratisArc(); -app.Run(); -``` +Call `withChronicle` **before** you discover or add artifacts. The integration watches each registration and records every event type, projection, reducer, reactor, and constraint it sees in a catalog for this application. An artifact registered earlier never reaches Chronicle. -Importing `@cratis/arc.chronicle` installs its typed `withChronicle` builder method; Core does not depend on Chronicle. Call `withChronicle` **before** discovering or adding artifacts, so the integration sees your event types, projections, reducers, and reactors. To avoid keeping a connection string in source, put `Cratis:Chronicle:{ConnectionString,EventStore}` in `appsettings.json` or override it with `Cratis__Chronicle__ConnectionString` and `Cratis__Chronicle__EventStore`, then call `builder.withChronicle({})`. Code options win over file and environment settings. The Chronicle engine must run separately. +:::caution[Development credentials] +The connection string above uses the SDK's development credentials and accepts the kernel's self-signed certificate. In production, provide real credentials and `skipTlsValidation=false`. +::: -The experimental private `@cratis/cratis` composition has a shorter TypeScript path (under 20 lines): +## Keep the connection string out of source -```typescript -import 'reflect-metadata'; -import { CratisApplication } from '@cratis/cratis'; -const builder = CratisApplication.createBuilder(); -await builder.discover(new URL('./Features/', import.meta.url)); -const app = await builder.build(); -await app.run(); -``` - -Its `createBuilder()` mirrors C#'s `builder.AddCratis()` followed by `app.UseCratis()`. You can also call `builder.addCratis({ eventStore, connectionString })` after importing `@cratis/cratis` instead of using `CratisApplication.createBuilder()`. Neither path installs authentication automatically. If your routes need authentication, supply an Arc handler in `ArcApplication.createBuilder({ authentication: [...] })` or `CratisApplication.createBuilder({ authentication: [...] })` before hosting. Public routes need no handler. C#'s setup is: +`withChronicle` also reads `Cratis:Chronicle` from the application's configuration, the same `appsettings.json` and `Cratis__...` environment variables Arc uses for its own [configuration](../configuration/index.md): -```csharp -var builder = WebApplication.CreateBuilder(args); -builder.AddCratis(); -var app = builder.Build(); -app.UseCratis(); -app.Run(); +```json title="appsettings.json" +{ + "Cratis": { + "Chronicle": { + "connectionString": "chronicle://localhost:35000", + "eventStore": "Library" + } + } +} ``` -Both paths require a separately running Chronicle server. The TS package is a local preview, not published. The integration has an opt-in live kernel suite; this setup example is not a live-kernel verification. - -:::caution[Development credentials] -The connection string above uses the SDK's development credentials and accepts the kernel's self-signed certificate. In production, provide real credentials and `skipTlsValidation=false`. -::: +Then call `builder.withChronicle({})`. To override the file in a deployment, set `Cratis__Chronicle__ConnectionString` and `Cratis__Chronicle__EventStore`. Values passed in code win over the file and the environment, and a `client` passed in code replaces a configured connection string. Registration fails when no event store is set, or when neither or both of a connection string and a client are set. ## Choose who owns the client | Registration | Ownership | | --- | --- | -| `{ connectionString, eventStore }` | Arc creates the SDK client, with a per-builder catalog of the artifacts you `add` or `discover`, and closes it with the application | -| `{ client, eventStore }` | You pass a caller-owned `IChronicleClient`. Arc never disposes it; your host calls `client.dispose()`. The client must already have a provider that registers the event types, projections, reducers, and reactors you use | +| `{ connectionString, eventStore }` | Arc creates the SDK client with its per-application artifact catalog and closes it with the application | +| `{ client, eventStore }` | You pass a caller-owned `IChronicleClient`. Arc never disposes it; your host calls `client.dispose()`. The client must already have an artifact provider that registers the event types, projections, reducers, and reactors you use | -Pass `eventStore` in either case. Every append and read uses the current execution's tenant as the Chronicle namespace. +Pass `eventStore` in either case. Every append and read uses the current execution's tenant as the Chronicle namespace, so tenancy you configure for Arc also isolates events. -## Return Arc commands from reactors +An Arc-owned client is also wired so that [reactors can return Arc commands](reactors/command-side-effects.md). A caller-owned client needs that handler passed to the SDK before it connects; the reactor page shows how. -With Chronicle SDK 6.6.0 or later, a reactor may return an `@command()` instance or a nonempty array containing **only** Arc commands. Arc executes each command through its validation, authorization, and normal command pipeline, in the triggering event's namespace. A failed command throws at the reactor boundary, so the observer partition fails rather than acknowledging a partial side effect. The commands run in order; a later failure does not undo an earlier committed command. Keep side effects idempotent for re-delivery. +## Use the Cratis composition -For example, a reactor can translate a recorded event into another command's intent: +`@cratis/cratis` composes Arc and the Chronicle integration in one import, the TypeScript counterpart of the .NET `Cratis` package: -```typescript -import { reactor } from '@cratis/chronicle/reactors'; -import type { EventContext } from '@cratis/chronicle/events'; -import { executeCommandsAsSystem } from '@cratis/arc.chronicle'; -import { LiveCreated, FollowUpLive } from './LiveArtifacts.js'; +```typescript title="main.ts" +import 'reflect-metadata'; +import { CratisApplication } from '@cratis/cratis'; -@executeCommandsAsSystem('writers') -@reactor() -export class LiveCommandReactor { - liveCreated(event: LiveCreated, context: EventContext): FollowUpLive { - return new FollowUpLive(context.eventSourceId, event.name); - } -} +const builder = CratisApplication.createBuilder(); +await builder.discover(new URL('./Features/', import.meta.url)); +const app = await builder.build(); +await app.run(); ``` -`LiveCreated` is an SDK `@eventType()` class; `FollowUpLive` is an Arc `@command()` with `@field(String) @key() id` and a `@field(String) name`. The exact integration example is exercised in the [live suite](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Chronicle/Integration/LiveArtifacts.ts). - -With an Arc-owned client (`{ connectionString, eventStore }`), `withChronicle` installs the result handler before observations begin. For a caller-owned client, pass the handler to the SDK when creating the client **before connecting it**: +`CratisApplication.createBuilder(options, chronicle)` creates an Arc builder and registers Chronicle with the `chronicle` options, which default to the `Cratis:Chronicle` configuration. After importing `@cratis/cratis`, `builder.addCratis({ eventStore, connectionString })` does the same on a builder you created yourself. The package re-exports `@cratis/arc.core` and `@cratis/arc.chronicle`, and `@cratis/cratis/testing` re-exports `@cratis/arc.testing` and `@cratis/arc.chronicle/testing`. -```typescript -import { ChronicleClient, ChronicleOptions } from '@cratis/chronicle'; -import { reactorCommandResultHandler } from '@cratis/arc.chronicle'; +`@cratis/cratis` is experimental like the integration it composes, and it is not published to npm. It does not install an authentication handler. If your routes need authentication, pass one in `CratisApplication.createBuilder({ authentication: [...] })`, as you would to `ArcApplication.createBuilder`; see [Authentication](../core/authentication.md). Public routes need no handler. -// Capture the application built later; the SDK invokes this only during observation. -let application: Awaited>; -const client = new ChronicleClient(ChronicleOptions.fromConnectionString(connectionString, { - clientArtifactsProvider: artifacts, - reactorResultHandler: reactorCommandResultHandler(() => application.server, 'Tasks') -})); -builder.withChronicle({ eventStore: 'Tasks', client }); -application = await builder.build(); -``` +## Check it -Here `builder`, `connectionString`, and `artifacts` are your configured Arc builder, Chronicle connection string, and SDK artifact provider. Register reactor, command, and event types before building. `reactorCommandResultHandler` declines event-only returns so Chronicle appends them using its own event-side-effect path. **Do not mix returned commands with events or other values in one array**: Arc rejects the mixture rather than silently dropping an item. Return either all commands or all events. +Run a command that returns an event, then read the event back with the Chronicle Workbench or the `cratis` CLI against the same event store and tenant namespace. A 400 answer with a `constraintViolation` or `concurrencyViolation` reason means Chronicle rejected the append; see [Concurrency](commands/concurrency.md). A connection failure fails the command with an exception. -Returned commands have no Arc principal by default, as in .NET. When a command requires a system role, decorate the **reactor class** with `@executeCommandsAsSystem('role-name')` from `@cratis/arc.chronicle`. This supplies a system principal to returned commands (not to imperative calls made inside the reactor). The SDK identity for their appends is the triggering event's identity by default, or the system identity when the decorator is present; the command's causation includes the event source, event type, sequence number, store, and namespace. The commands retain the triggering event's correlation ID. No distributed transaction spans the reactor's commands and the triggering event. +Next, [return events](commands/index.md) from a command. ## Related -- [Returning events](commands/index.md) - [Chronicle](index.md) +- [Reactors](reactors/index.md) +- [Testing Chronicle commands](../testing/chronicle.md) diff --git a/Documentation/chronicle/aggregates/defining-an-aggregate-root.md b/Documentation/chronicle/aggregates/defining-an-aggregate-root.md new file mode 100644 index 00000000..65564af6 --- /dev/null +++ b/Documentation/chronicle/aggregates/defining-an-aggregate-root.md @@ -0,0 +1,91 @@ +--- +title: Defining an aggregate root +description: Write a Chronicle aggregate root in TypeScript, register event handlers by event class, keep replay free of side effects, and reject invalid changes. +--- + +An aggregate root holds the state one event source's history implies, and the methods that change it. You write the rules once, on the aggregate, and every command that changes an order goes through them. + +## Define the events and the aggregate + +```typescript title="Order.ts" +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { AggregateRoot } from '@cratis/arc.chronicle'; + +@eventType('ItemAdded') +export class ItemAdded { + @field(String) productId: string; + @field(Number) quantity: number; + + constructor(productId: string, quantity: number) { + this.productId = productId; + this.quantity = quantity; + } +} + +export class OrderLimitExceeded extends Error { + constructor() { super('The order total must not exceed 100'); } +} + +export class Order extends AggregateRoot { + quantity = 0; + + constructor() { + super(); + this.on(ItemAdded, event => { this.quantity += event.quantity; }); + } + + canAdd(quantity: number): boolean { + return Number.isSafeInteger(quantity) && quantity > 0 && quantity <= 100 - this.quantity; + } + + addItem(productId: string, quantity: number): void { + if (!this.canAdd(quantity)) throw new OrderLimitExceeded(); + this.apply(new ItemAdded(productId, quantity)); + } +} +``` + +`Order` keeps one piece of state, the total quantity. `canAdd` answers the rule against that state, `addItem` guards it, and `apply()` records the change. The explicit `'ItemAdded'` ID keeps the stored event type stable even if a bundler renames the class; without it, the SDK uses the class name. + +## Register handlers by event class + +`this.on(EventClass, handler)` registers the handler for one event type. The match is by class, not by method name, and each event type can have one handler; registering a second throws. + +The handler runs in two situations: + +- **Replay.** When the aggregate is loaded, Arc fetches the stored events of the handled types and passes each one in, rebuilt as an instance of its class. +- **Apply.** `apply(event)` runs the handler immediately, so the next rule in the same command already sees the new state. + +Keep handlers to state changes. A handler that sends an email or calls a service does so again for every old event on every load. + +The handler takes an optional second `EventContext` argument during replay, when you need the stored event's sequence number or occurred time. During `apply()` there is no context, so treat it as possibly `undefined`. + +Arc loads only the event types the aggregate handles. An aggregate with no handlers can still apply events; it replays nothing. + +## Reject a change + +The aggregate has no failure list. A command rejects a change the usual Arc way: `handle()` asks the aggregate, and returns `rejected(...)` when the rule says no. The caller gets a 400 with your message, and nothing the aggregate applied is appended. [Injecting into commands](injecting-into-commands.md#reject-from-handle) shows the command. + +The throw in `addItem` is a guard for a caller that skipped the question. It fails the command with an exception, which the caller sees as a 500, and nothing is appended either. Throw a class of your own, like `OrderLimitExceeded`, so the log names the domain problem. + +For input you can check without history, such as a positive quantity, a [command validator](../../commands/command-validation.md) answers with a 400 before the aggregate is even loaded. + +## Know what isNew means + +The protected `isNew` property is `true` when the aggregate's route had no events at all when it loaded, including events it does not handle. Use it for "create once" rules: + +```typescript +start(): void { + if (!this.isNew) throw new Error('The order has already started'); + this.apply(new OrderStarted()); +} +``` + +This method fragment assumes an `OrderStarted` event type registered with `this.on(...)` or appended without a handler. + +## What it does not have + +Arc on .NET's aggregate has `Failed(...)` for collecting rule failures, `OnActivate`, and a factory for loading an aggregate by an ID other than the command key. The TypeScript aggregate has none of these. Reject through a thrown error or Arc validation, and load one event source per command key. + +Next, [inject the aggregate into a command](injecting-into-commands.md). diff --git a/Documentation/chronicle/aggregates/index.md b/Documentation/chronicle/aggregates/index.md index 01784ec6..33a41e4d 100644 --- a/Documentation/chronicle/aggregates/index.md +++ b/Documentation/chronicle/aggregates/index.md @@ -1,94 +1,35 @@ --- title: Aggregates -description: Rehydrate a keyed aggregate from Chronicle and enroll its applied events in an Arc command. +description: Decide from one event source's full history with a keyed aggregate root that Arc rehydrates from Chronicle and whose applied events join the command's batch. --- -Use an aggregate when a command needs to decide from one event source's history, rather than from an eventually consistent read model. This API is experimental. It is an optional Chronicle integration; ordinary Arc commands do not need an event store. +An order may hold at most 100 items. To enforce that, the command needs the order's current total, and it needs it to be exact: a read model that lags one event behind could let the 101st item through. An **aggregate** replays the order's own events into memory, checks the rule, and applies the new event. Because it knows which revision it replayed, a concurrent change rejects the append instead of breaking the rule. -## Define the event and aggregate +The API is experimental and part of the optional Chronicle integration. Ordinary Arc commands do not need an event store. -Register handlers by **event class**, not by a method-name convention. A handler runs both for stored events during replay and for events applied in the current command. Keep it free of external effects. +## Aggregate or read model -```typescript title="Order.ts" -import { field } from '@cratis/fundamentals'; -import { eventType } from '@cratis/chronicle/events'; -import { AggregateRoot } from '@cratis/arc.chronicle'; +| | Aggregate | Read model in a command | +| --- | --- | --- | +| State comes from | Replaying the event source's events on every command | A projection Chronicle stored earlier | +| Freshness | Every stored event of the handled types | Whatever the projection has processed | +| Concurrency | The append is rejected when the stream moved after the replay | None; the read model does not lock anything | +| Cost | Grows with the number of events in the stream | One lookup | +| Use it when | A rule depends on exact history of one event source | The command needs context, or a rule can tolerate lag | -@eventType('ItemAdded') -export class ItemAdded { - @field(String) productId: string; - @field(Number) quantity: number; +A command can take both. See [Read models in commands](../read-models/injecting-into-commands.md). - constructor(productId: string, quantity: number) { - this.productId = productId; - this.quantity = quantity; - } -} +## How Arc wires it -export class Order extends AggregateRoot { - quantity = 0; +1. You define a class that extends `AggregateRoot` and registers a handler per event type. +2. A command binds it with `@inject(commandAggregate(Order))`. +3. Before `handle()` runs, Arc loads the events for the command's key, replays them through the handlers, and records the tail it read. +4. `handle()` calls methods on the aggregate, which `apply()` new events. +5. When the command succeeds, the applied events join the command's [batch](../commands/transactional-commands.md), with the recorded tail as the expected revision. - constructor() { - super(); - this.on(ItemAdded, event => { this.quantity += event.quantity; }); - } +## Topics - addItem(productId: string, quantity: number): void { - if (!Number.isSafeInteger(quantity) || quantity <= 0 || quantity > 100 - this.quantity) { - throw new Error('Quantity must be positive and the order total must not exceed 100'); - } - this.apply(new ItemAdded(productId, quantity)); - } -} -``` - -`on(ItemAdded, handler)` takes an optional second `EventContext` argument in the handler when you need the stored event's context. During `apply()`, that context is unavailable, so handle it as optional. Replayed payloads are reconstructed as instances of `ItemAdded`. An aggregate with no handlers can still apply events; it does not read unrelated event types during rehydration. - -## Bind the command - -Call `withChronicle` before registering the artifacts, as shown in [Add event sourcing](../add-event-sourcing.md). Bind the aggregate to the command's `@key()` field and register both the command and event type: - -```typescript title="AddItemToOrder.ts" -import { field } from '@cratis/fundamentals'; -import { command, inject, key } from '@cratis/arc.core'; -import { commandAggregate } from '@cratis/arc.chronicle'; -import { Order } from './Order.js'; - -@command() -export class AddItemToOrder { - @field(String) @key() id = ''; - @field(String) productId = ''; - @field(Number) quantity = 0; - - @inject(commandAggregate(Order)) - handle(order: Order): void { - order.addItem(this.productId, this.quantity); - } -} -``` - -```typescript title="main.ts" -import 'reflect-metadata'; -import { ArcApplication } from '@cratis/arc.core'; -import '@cratis/arc.chronicle'; -import { AddItemToOrder } from './AddItemToOrder.js'; -import { ItemAdded } from './Order.js'; - -const builder = ArcApplication.createBuilder(); -builder.withChronicle({ eventStore: 'Orders', connectionString: 'chronicle://localhost:35000' }); -builder.add(AddItemToOrder, ItemAdded); -const application = await builder.build(); -await application.run(); -``` - -The development connection string requires a running local Chronicle kernel; see [development credentials](../add-event-sourcing.md#choose-who-owns-the-client) before using another environment. Executing `AddItemToOrder` with an empty order records one `ItemAdded`. A later execution replays that event, increments `quantity`, and checks the new total before appending. A rejected or failed command does not append its pending aggregate events. - -`apply()` enrolls events in the command unit of work even when `handle()` returns `void`, calls `commit()` without returning it, or applies again after `commit()`. You may return `order.commit()` explicitly; do **not** also return the same event separately. Unlike the .NET aggregate, this implementation does not have `Failed(...)`, `OnActivate`, or a factory for loading a second source id. Reject invalid input through Arc validation or a failed command result, not a hidden aggregate failure list. - -The command key must be present before the aggregate loads. For a second aggregate **type** on the same key, bind another `commandAggregate(Type)` parameter. Aggregate reads and writes use the current tenant namespace and the command's configured event source type, stream type, and stream id (including `getEventStreamId()`). The concurrency scope uses the unfiltered tail of that route, **including events without a handler**. A concurrent append on that route rejects the batch; an unhandled event does not permanently block the aggregate. `isNew` is true only when that route has no events at all. - -## Boundaries - -Applied events and returned events join one deferred [command batch](../commands/transactional-commands.md). This is not a transaction over other stores. Immediate SDK appends within `handle()` are outside the batch; compensation can run after such an append has already persisted. [Command operations](../../commands/operations/index.md) should not rely on immediate appends being rolled back. - -`commandReadModel(Type)` binds model-bound command parameters. `readModelForValidation(Type, { optional: true })` is available **only to validators of model-bound commands**; it uses the command key and tenant, returns `null` for a missing model, and reads a materialized projection snapshot, not aggregate replay state. Constructor-injected validator read models are not supported. +| Topic | Description | +| --- | --- | +| [Defining an aggregate root](defining-an-aggregate-root.md) | Handlers, state, rules, and what the TypeScript aggregate does not have | +| [Injecting into commands](injecting-into-commands.md) | Binding, identity and routing, commit, concurrency, and boundaries | diff --git a/Documentation/chronicle/aggregates/injecting-into-commands.md b/Documentation/chronicle/aggregates/injecting-into-commands.md new file mode 100644 index 00000000..3a8b19a8 --- /dev/null +++ b/Documentation/chronicle/aggregates/injecting-into-commands.md @@ -0,0 +1,84 @@ +--- +title: Injecting an aggregate into a command +description: Bind a rehydrated aggregate to a command's key with commandAggregate, reject or commit its changes, and know how identity, routing, concurrency, and the batch apply. +--- + +With the aggregate defined, a command asks Arc for it by type. Arc loads it for the command's key, replays its history, and hands it to `handle()`. Whatever the aggregate applies is appended when the command succeeds. + +## Bind the aggregate + +```typescript title="AddItemToOrder.ts" +import { field } from '@cratis/fundamentals'; +import { command, inject, key, rejected, validation } from '@cratis/arc.core'; +import { commandAggregate } from '@cratis/arc.chronicle'; +import { Order } from './Order.js'; + +@command() +export class AddItemToOrder { + @field(String) @key() id = ''; + @field(String) productId = ''; + @field(Number) quantity = 0; + + @inject(commandAggregate(Order)) + handle(order: Order) { + if (!order.canAdd(this.quantity)) { + return rejected(validation('The order total must not exceed 100', ['quantity'])); + } + order.addItem(this.productId, this.quantity); + } +} +``` + +```typescript title="main.ts" +import 'reflect-metadata'; +import { ArcApplication } from '@cratis/arc.core'; +import '@cratis/arc.chronicle'; +import { AddItemToOrder } from './AddItemToOrder.js'; +import { ItemAdded } from './Order.js'; + +const builder = ArcApplication.createBuilder(); +builder.withChronicle({ eventStore: 'Orders', connectionString: 'chronicle://localhost:35000' }); +builder.add(AddItemToOrder, ItemAdded); +const application = await builder.build(); +await application.run(); +``` + +`Order` and `ItemAdded` come from [Defining an aggregate root](defining-an-aggregate-root.md). The aggregate class itself is not registered; `commandAggregate(Order)` is enough. Register the event types it handles, after `withChronicle`. The development connection string needs a running local Chronicle kernel; see [Add event sourcing](../add-event-sourcing.md) for other environments. + +Executing `AddItemToOrder` for an empty order appends one `ItemAdded`. The next execution for the same `id` replays that event, so `quantity` starts from the stored total, and the rule is checked against it. + +## Reject from handle() + +`handle()` returns `rejected(validation(...))` when the aggregate says no. The caller gets a 400 with the message on the `quantity` member, and the command appends nothing, including anything the aggregate applied earlier in the same `handle()`. See [Command outcomes](../../commands/command-outcomes.md). + +## Commit + +`apply()` enrolls the event in the command's batch. You do not have to return anything: the command above returns nothing when it succeeds, and the event is still appended. That also holds when `handle()` calls `order.commit()` without returning it, or applies more events after calling it. + +You may `return order.commit()` to make the commit visible. Do **not** also return the same event yourself; it would be appended twice, and Arc rejects a commit result whose events were already staged. + +## Which events are loaded + +The aggregate belongs to the command's key: the `@key()` field, `getKey()`, or `getEventSourceId()`. A command without a key fails with the exception `A command key is required for Order` before `handle()` runs. + +Loading uses the same route the command's returned events use: the current tenant's namespace, the command's `@eventSourceType`, `@eventStreamType`, and its stream ID from `getEventStreamId()` or `@eventStreamId`. Arc reads the events of the types the aggregate handles, in order, and replays them. See [Event metadata](../commands/event-metadata.md). + +Arc loads each aggregate type once per command. A second parameter of the same type receives the same instance. For a second aggregate **type** on the same key, bind another `commandAggregate(Type)`. There is no way to load an aggregate for a different ID; the key decides. + +## Concurrency + +When the aggregate loads, Arc records the tail of its route, counting every event on the route, handled or not. The events it applies are appended with that tail as an exact [concurrency scope](../commands/concurrency.md). If anything is appended to that route between load and commit, the batch is rejected with a `concurrencyViolation` 400, and nothing is appended. An event the aggregate does not handle moves the tail too, but it cannot block the aggregate permanently: the next load reads the new tail. + +## Boundaries + +- Applied events and returned events join the same [batch](../commands/transactional-commands.md). Nested commands share it, so an aggregate changed in a nested command commits with the outer command. +- An immediate SDK append inside `handle()` is outside the batch. It is stored even when the aggregate's batch is later rejected. +- The batch covers one event log. It is not a transaction over other stores. +- The [command subject](../commands/subject.md) applies to the aggregate's events as it does to returned events. +- `readModelForValidation` and `commandReadModel` read the stored projection, not the aggregate's replayed state. They can disagree while the projection catches up. + +## Related + +- [Aggregates](index.md) +- [Read models in commands](../read-models/injecting-into-commands.md) +- [Testing Chronicle commands](../../testing/chronicle.md) diff --git a/Documentation/chronicle/aggregates/toc.yml b/Documentation/chronicle/aggregates/toc.yml new file mode 100644 index 00000000..2984e4ff --- /dev/null +++ b/Documentation/chronicle/aggregates/toc.yml @@ -0,0 +1,6 @@ +- name: Overview + href: index.md +- name: Defining an aggregate root + href: defining-an-aggregate-root.md +- name: Injecting into commands + href: injecting-into-commands.md diff --git a/Documentation/chronicle/code-analysis.md b/Documentation/chronicle/code-analysis.md new file mode 100644 index 00000000..5ca5062b --- /dev/null +++ b/Documentation/chronicle/code-analysis.md @@ -0,0 +1,36 @@ +--- +title: Chronicle code analysis +description: Which of Arc on .NET's ARCCHR diagnostics for the Chronicle integration apply to TypeScript, which mistakes the runtime catches instead, and which have no check. +--- + +Arc on .NET ships Roslyn analyzers, `ARCCHR0001` to `ARCCHR0010`, that catch Chronicle integration mistakes at build time. `@cratis/eslint-plugin-arc-core` has **no** Chronicle rules. Some of those mistakes cannot happen in the TypeScript API, some are caught when the application builds or runs, and some have no check at all. This page says which is which, so you know what to watch for in review. + +The Arc rules that do exist are listed in [Code analysis](../code-analysis/index.md). + +## ARCCHR mapping + +| .NET diagnostic | TypeScript status | Reason | +| --- | --- | --- | +| ARCCHR0001, aggregate handler signature | N/A | Handlers are registered with `this.on(EventClass, handler)`. The compiler checks the callback type, and a second handler for one event type throws. There is no method-name convention to get wrong. | +| ARCCHR0002, ambiguous command identity | N/A | A command has one key: `getEventSourceId()`, then `getKey()` or the single `@key()` field. Marking a second `@key()` throws when the class is defined. | +| ARCCHR0003, reactor reaches the event log | Not checked | A reactor that appends through its own SDK client does so outside the side-effect path. Return events instead; see [Reactors](reactors/index.md). | +| ARCCHR0004, `[EventType]` repeats the type name | N/A | The SDK falls back to the class name, which a bundler may rename. An explicit ID equal to the class name keeps the stored type stable, so it is not redundant here. | +| ARCCHR0005, Chronicle used but not registered | Partly caught at runtime | A `commandReadModel(Type)` binding with no owner fails `build()`. A returned event without `withChronicle` is not caught: nothing recognizes it, so it becomes the command's ordinary response, and nothing is appended. | +| ARCCHR0006, manual reactor command without `[OnceOnly]` | N/A | The TypeScript SDK has neither `[OnceOnly]` nor `[Replay]`. Every reactor handler must be safe to repeat; see [Returning commands from a reactor](reactors/command-side-effects.md#when-a-command-fails). | +| ARCCHR0007, command injects the event log | Not checked | A handler can reach `ChronicleReadModels.getStore().eventLog` and append immediately. Such an append is outside the command's batch; see [Transactional commands](commands/transactional-commands.md#what-is-outside-the-batch). | +| ARCCHR0008, data annotations `[Key]` | N/A | There is one `@key()`, from `@cratis/arc.core`, and Chronicle reads it. | +| ARCCHR0009, secret-looking command value | Handled at runtime, no lint rule | Values of fields whose names contain `password`, `secret`, `token`, `credential`, or `apiKey` are never recorded in the causation chain. Mark any other secret with `@notAudited()`; see [Causation and auditing](commands/causation.md). | +| ARCCHR0010, raw GUID response does not set the event source | Not checked | An ordinary value in a `tuple(...)` is the response, not the event source. Use `tuple(eventSourceIdResponse(id), event)` to set and return it; see [Resolving the event source ID](resolving-event-source-id.md#return-the-id-to-the-caller). | + +## What to check in review + +The unchecked rows are the ones a reviewer has to catch: + +- a reactor or command that appends through the SDK instead of returning events; +- an application that returns events but never calls `withChronicle`, or calls it after registering artifacts; +- a command meant to append to an existing entity that returns an ID in a tuple instead of `eventSourceIdResponse`. + +## Related + +- [Code analysis](../code-analysis/index.md) +- [Chronicle](index.md) diff --git a/Documentation/chronicle/commands/causation.md b/Documentation/chronicle/commands/causation.md index c7a0aa34..9ad9f7a4 100644 --- a/Documentation/chronicle/commands/causation.md +++ b/Documentation/chronicle/commands/causation.md @@ -1,18 +1,62 @@ --- title: Causation and auditing -description: How the Chronicle integration records the command, its values, and the caller in the causation chain, and how @notAudited keeps secrets out of it. +description: What the Chronicle integration records in the permanent causation chain for each command, which values it leaves out, and how @notAudited and @pii keep a secret or personal value out of it. --- -Events are permanent, and so is the causation chain that says how each one came about. The integration records the command that produced an event, including its property values, so a reader later sees what the command was asked to do. That makes it important to keep secrets out. +Six months from now someone asks why a purchase order exists. The event says what happened. Its **causation chain** says how it came about: which command produced it, and what that command was asked to do. The integration writes that record for you on every event a command returns. + +The chain is stored with the event in the event log, and the event log is immutable. A value recorded there stays for as long as the event does, and every replay reads it. Changing your code later does not remove it. Decide what belongs in a permanent audit record before you add a field to a command. ## What is recorded -- The trusted Arc principal, the correlation ID, and the command's causation are scoped with the SDK's async `run` methods for the append. -- The command's name and field values are part of its causation. -- The SDK stamps one causation chain per batch; see [Transactional commands](transactional-commands.md#causation-in-a-batch). +For this command: + +```typescript +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { command, key } from '@cratis/arc.core'; + +@eventType() +export class PurchaseOrderRaised { + @field(String) supplier: string; + @field(Number) amount: number; + constructor(supplier = '', amount = 0) { this.supplier = supplier; this.amount = amount; } +} + +@command() +export class RaisePurchaseOrder { + @field(String) @key() orderId = ''; + @field(String) supplier = ''; + @field(Number) amount = 0; + + handle(): PurchaseOrderRaised { return new PurchaseOrderRaised(this.supplier, this.amount); } +} +``` + +a request with `{ "orderId": "po-26", "supplier": "ACME", "amount": 1234.56 }` adds an `Arc.Command` entry to the chain of the appended event: + +| Property | Value | +| --- | --- | +| `Command` | `RaisePurchaseOrder` | +| `Value.orderId` | `po-26` | +| `Value.supplier` | `ACME` | +| `Value.amount` | `1234.56` | + +The rules behind that table: + +- The command's class name is always recorded, under `Command`. +- Each field whose value is a string, number, or boolean is recorded as `Value.`, converted to a string and cut at 1,024 characters. +- Every other value is left out: [concepts](../../concepts.md), `Guid`, `Date`, arrays, and nested objects. A command whose fields are all concepts records only its name. +- The event also carries the correlation ID of the request and, as its identity, the signed-in principal, or Chronicle's system identity for an anonymous caller. + +When commands run inside other commands, every event in the batch carries the chain of the outermost command; see [Transactional commands](transactional-commands.md#causation-in-a-batch). Commands a reactor returns get a `ReactorEvent` entry for the triggering event ahead of their own; see [Returning commands from a reactor](../reactors/command-side-effects.md). ## Keep a value out +Three things keep a field's value off the chain. The command is still named either way: an audit trail that hides which commands ran would not be an audit trail. + +### Secrets: `@notAudited()` + ```typescript import { field } from '@cratis/fundamentals'; import { command, key } from '@cratis/arc.core'; @@ -21,14 +65,28 @@ import { notAudited } from '@cratis/arc.chronicle'; @command() export class ConnectAccount { @field(String) @key() id = ''; - @field(String) @notAudited() apiToken = ''; - handle() { /* return an event that does not contain the token */ } + @field(String) @notAudited() confirmationCode = ''; + + handle(): AccountConnected { return new AccountConnected(); } } ``` -`@notAudited()` excludes the field's value from the permanent causation chain. Arc also excludes fields the SDK marks `@pii` and obviously secret-named fields. `@notAudited()` only withholds the value; unlike `@pii`, it does not encrypt it or enroll it in erasure. +This fragment assumes an `AccountConnected` event type. `@notAudited()` from `@cratis/arc.chronicle` withholds the field's value from the causation chain. It only withholds: it does not encrypt the value or enroll it in erasure. Apply it to a public instance field; with standard decorators, a static or private field throws when the class is defined. + +### Personal data: `@pii` + +The SDK's `@pii()` from `@cratis/chronicle/compliance` on a command field also keeps its value off the chain. On the command class itself, it withholds every value the command has. Marking the command does not mark the event you construct from it: annotate the event's own properties where the event stores personal data. See [Compliance](../compliance.md). + +### Secret-looking names + +A field whose name contains `password`, `secret`, `token`, `credential`, or `apiKey`, in any letter case, is never recorded, marked or not. The match is on part of the name, so `tokenCount` is skipped too. Do not rely on the name: a field called `value` that holds an API key is recorded. Mark it. + +## Read the chain + +In the [Chronicle Workbench](/chronicle/workbench/), open an event's context and then its causation entries. The `Arc.Command` entry lists the properties in the table above. ## Related - [Returning events](index.md) -- [Glossary: causation chain](/arc/glossary/) +- [Event metadata](event-metadata.md) +- [Code analysis](../code-analysis.md), for why no TypeScript rule flags an unmarked secret diff --git a/Documentation/chronicle/commands/concurrency.md b/Documentation/chronicle/commands/concurrency.md index de23af03..51d2e709 100644 --- a/Documentation/chronicle/commands/concurrency.md +++ b/Documentation/chronicle/commands/concurrency.md @@ -1,11 +1,49 @@ --- title: Concurrency -description: Reject a Chronicle append when an event source moved, with exact concurrency scopes or routing-decorator tail checks, and map the rejection to validation. +description: Reject a Chronicle append when an event source moved, with routing-decorator tail checks, exact concurrency scopes, or an aggregate, and read the validation result a rejection produces. --- -Two people register the same task at the same moment. Without a concurrency check, both appends succeed. The integration offers two ways to make the second one fail. +Two librarians open the same book and both mark it as lent. Without a check, both commands succeed and the book is lent twice. Optimistic concurrency makes the second append fail: Chronicle compares the stream's current tail with the tail the command expected, and rejects the append when they differ. -## Exact scopes +The integration gives you three ways to state that expectation. Choose by how the command decides. + +| The command decides from | Use | +| --- | --- | +| Nothing it read; it only needs no one else to write in between | A routing decorator with `{ concurrency: true }` | +| An empty stream, such as "create once" | An exact scope with `EventSequenceNumber.beforeFirst` | +| The event source's history | An [aggregate](../aggregates/index.md), which carries the revision it replayed | + +For a rule across event sources, such as a unique author name, use a Chronicle constraint instead. The [Library sample](https://github.com/Cratis/Arc.TypeScript/blob/main/Samples/Library/Features/Authors/Registration/Registration.ts) registers `UniqueAuthorName` beside its command. + +## Check the tail with a routing decorator + +```typescript +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { command, key } from '@cratis/arc.core'; +import { eventSourceType } from '@cratis/arc.chronicle'; + +@eventType() +export class BookLent { + @field(String) borrower: string; + constructor(borrower = '') { this.borrower = borrower; } +} + +@command() +@eventSourceType('Book', { concurrency: true }) +export class LendBook { + @field(String) @key() bookId = ''; + @field(String) borrower = ''; + + handle(): BookLent { return new BookLent(this.borrower); } +} +``` + +`{ concurrency: true }` works the same on `@eventSourceType`, `@eventStreamType`, and `@eventStreamId`. When the command returns its events, the integration reads the tail of each event source in the batch, narrowed to the dimensions you marked, and sends that tail as the expected revision. A write that lands between that read and the append is rejected. + +The tail is read **after** `handle()` has run. A write that landed while `handle()` was deciding is already part of that tail, so this check does not protect a decision made from state read earlier. For a read-modify-write rule, carry the revision you read, as the next two sections do. + +## Pin the revision with an exact scope ```typescript import { field } from '@cratis/fundamentals'; @@ -25,21 +63,44 @@ export class CreateLiveExactlyOnce { } ``` -This excerpt is from the [kernel suite](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Chronicle/Integration/LiveArtifacts.ts), where `LiveCreated` is an `@eventType()` class. `eventsWithConcurrencyScopes(events, scopes)` carries exact, server-authored revisions. `EventSequenceNumber.beforeFirst.value` is the expected revision of a stream with no events, so this command succeeds once per ID. Scope keys may refer to other sources than the appended events. +This excerpt is from the [kernel suite](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Chronicle/Integration/LiveArtifacts.ts), where `LiveCreated` is an `@eventType()` class. `eventsWithConcurrencyScopes(events, scopes)` sends each scope exactly as you wrote it. `EventSequenceNumber.beforeFirst.value` is the expected tail of a stream with no events, so the command succeeds once per ID; the second call is rejected. + +Each scope is keyed by an event source ID and accepts: -The kernel requires at least one event in a batch: **scope-only empty batches fail**. +| Field | Meaning | +| --- | --- | +| `sequenceNumber` | The expected tail, a `bigint` | +| `eventSourceId` | `true` to scope to that event source | +| `eventSourceType`, `eventStreamType`, `eventStreamId` | Narrow the scope to that source type or stream | +| `eventTypes` | Narrow the scope to these event types | -## Routing-decorator tail checks +A scope may name an event source the batch does not append to, which lets a decision about one source guard against changes to another. An exact scope replaces the routing-decorator check for the same source. The kernel requires at least one event in a batch, so **a batch with scopes and no events fails**. -`{ concurrency: true }` on `@eventSourceType`, `@eventStreamType`, or `@eventStreamId` reads the tail immediately before the append and scopes that dimension. It does not pin the revision you used when you read state. For a read-modify-write rule, use exact scopes with the revision you read. +## Let an aggregate carry it + +An aggregate records the tail of its route when it is loaded, and the events it applies are appended with that tail as an exact scope. A concurrent append on the same route between load and commit rejects the batch. See [Aggregates](../aggregates/injecting-into-commands.md). ## When the check fails -Chronicle constraint and concurrency rejections become Arc validation results, so the caller gets 400 and nothing is appended. Unknown, incomplete, contradictory, or partial acknowledgments fail the command. +A concurrency or constraint rejection becomes a validation result. The caller gets 400, the command result carries no response, and nothing from the batch is appended. + +| Rejection | `reason` | `message` | Other fields | +| --- | --- | --- | --- | +| Concurrency | `concurrencyViolation` | `Concurrent modification prevented the append` | `state` with `eventSourceId`, `expectedEventSequenceNumber`, and `actualEventSequenceNumber` | +| Constraint | `constraintViolation` | The constraint's message | `members` holds the constrained property in camel case when Chronicle reports one; `reasonDetail` holds the constraint ID | + +Both have error severity. A client can show the message or retry the command with fresh state. + +An unknown, incomplete, contradictory, or partial acknowledgment from Chronicle is not a rejection. The command fails with an exception, because Arc cannot tell what was stored. + +## Nested commands and tests + +Nested commands that share a batch may each add scopes. Two different scopes for the same event source fail the command instead of silently keeping one. -The in-memory [test scenario](../../testing/chronicle.md) accepts scopes but does not enforce them; only the kernel does. +The in-memory [test scenario](../../testing/chronicle.md) accepts scopes but does not enforce them. Only a kernel does. ## Related - [Transactional commands](transactional-commands.md) - [Resolving the event source ID](../resolving-event-source-id.md) +- [Chronicle constraints](/chronicle/constraints/) diff --git a/Documentation/chronicle/commands/event-metadata.md b/Documentation/chronicle/commands/event-metadata.md new file mode 100644 index 00000000..f763156c --- /dev/null +++ b/Documentation/chronicle/commands/event-metadata.md @@ -0,0 +1,93 @@ +--- +title: Event metadata +description: What every event a command returns carries besides its payload, where the integration takes each value from, and how to override or read it. +--- + +An event records more than its payload. It also records which entity it belongs to, which stream within that entity, whose personal data it carries, when it happened, who caused it, and which request it was part of. You rarely want to set those by hand on every command. The integration resolves each value from the command and the request, and lets you override the ones that differ. + +## What each event carries + +| Metadata | Taken from | Override for one event | +| --- | --- | --- | +| Event source ID | `getEventSourceId()`, then the `@key()` field, then a new UUID; see [Resolving the event source ID](../resolving-event-source-id.md) | `tuple(eventSourceIdResponse(id), event)` or `eventForEventSourceId({ eventSourceId })` | +| Event source type | `@eventSourceType('Author')` on the command | `eventForEventSourceId({ eventSourceType })` | +| Event stream type | `@eventStreamType('Onboarding')` on the command | `eventForEventSourceId({ eventStreamType })` | +| Event stream ID | `getEventStreamId()` on the command, then `@eventStreamId('main')` | `eventForEventSourceId({ eventStreamId })` | +| Subject | `getSubject()`, then a `@subject()` field, then `@eventSubject(...)`, then the event source ID; see [Subject](subject.md) | `eventForEventSourceId({ subject })` | +| Tags | The SDK's `@tag` and `@tags` on the event class | `eventForEventSourceId({ tags })` adds tags for that append | +| Occurred | Set when the event is appended | `eventForEventSourceId({ occurred })` | +| Correlation ID | The request's correlation ID | None | +| Caused by | The signed-in principal, or Chronicle's system identity for an anonymous caller | None | +| Causation | An `Arc.Command` entry with the command name and its values; see [Causation and auditing](causation.md) | None | + +Routing decorators come from `@cratis/arc.chronicle` and apply to every event the command returns. A value set on an `eventForEventSourceId` entry wins over the command's default for that entry only. + +## Set command-wide defaults + +```typescript +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { command, key } from '@cratis/arc.core'; +import { eventSourceType, eventStreamType } from '@cratis/arc.chronicle'; + +@eventType() +export class OnboardingStarted { + @field(String) plan: string; + constructor(plan = '') { this.plan = plan; } +} + +@command() +@eventSourceType('Customer') +@eventStreamType('Onboarding') +export class StartOnboarding { + @field(String) @key() customerId = ''; + @field(String) plan = ''; + + getEventStreamId(): string { return `onboarding-${this.plan}`; } + + handle(): OnboardingStarted { return new OnboardingStarted(this.plan); } +} +``` + +`OnboardingStarted` is appended to the customer's event source, with source type `Customer`, stream type `Onboarding`, and a stream ID computed per command. `getEventStreamId()` runs after validation, so it may read any field. When you also declare `@eventStreamId(...)`, the method wins. + +The same stream settings select which events an [aggregate](../aggregates/injecting-into-commands.md) loads, so an aggregate and the events its command returns agree on the stream. + +## Override one event + +```typescript +import { eventForEventSourceId } from '@cratis/arc.chronicle'; + +handle() { + return eventForEventSourceId({ eventSourceId: 'routed', event: new Registered(), + eventSourceType: 'Override', subject: 'subject-1', tags: ['tag-1'] }); +} +``` + +This excerpt is from the package's [scenario spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Chronicle/testing/for_ChronicleCommandScenario/when_executing/with_returned_events.ts), where `Registered` is an `@eventType()` class. The entry is branded, so an ordinary response object that happens to have `event` and `eventSourceId` fields is never mistaken for an event. Return an array to mix routed entries with plain events in one batch. + +## Read it back + +Chronicle hands the metadata back as an `EventContext`. A reactor receives it as the second handler argument; see [Reactors](../reactors/index.md). A model-bound projection can copy a context value into a read model with the SDK's `@setFromContext`: + +```typescript +import { field } from '@cratis/fundamentals'; +import { fromEvent, setFromContext } from '@cratis/chronicle/projections'; +import { readModel } from '@cratis/arc.core'; + +@readModel() +@fromEvent(OnboardingStarted) +export class Onboarding { + @field(String) id = ''; + @field(String) plan = ''; + @setFromContext(OnboardingStarted, 'occurred') @field(Date) startedAt!: Date; +} +``` + +`plan` is copied from the event by matching name, and `startedAt` takes the event's occurred time. In a test, `result.appendedEvents` on [`ChronicleCommandScenario`](../../testing/chronicle.md) exposes each appended event with its routing, subject, and tags. + +## Related + +- [Returning events](index.md) +- [Resolving the event source ID](../resolving-event-source-id.md) +- [Concurrency](concurrency.md), where the same routing decorators opt into tail checks diff --git a/Documentation/chronicle/commands/index.md b/Documentation/chronicle/commands/index.md index 34dad870..397b9c61 100644 --- a/Documentation/chronicle/commands/index.md +++ b/Documentation/chronicle/commands/index.md @@ -47,7 +47,7 @@ With Chronicle [registered](../add-event-sourcing.md), `POST /api/create-task` a The integration is a [response value handler](../../commands/response-value-handlers.md): it recognizes registered event instances and branded values, and everything else keeps its ordinary meaning. Returning Chronicle events together with [command operations](../../commands/operations/index.md) is rejected before either effect runs. -The event source, routing decorators, and explicit targets are covered in [Resolving the event source ID](../resolving-event-source-id.md). +The event source, routing decorators, and explicit targets are covered in [Resolving the event source ID](../resolving-event-source-id.md). Everything else an appended event carries, such as its subject, causation, and correlation ID, is listed in [Event metadata](event-metadata.md). ## Low-level definitions @@ -55,5 +55,6 @@ The older `defineChronicleCommand` remains. It takes a client, an event store, a ## Related +- [Event metadata](event-metadata.md) - [Transactional commands](transactional-commands.md) - [Testing Chronicle commands](../../testing/chronicle.md) diff --git a/Documentation/chronicle/commands/subject.md b/Documentation/chronicle/commands/subject.md new file mode 100644 index 00000000..585674ba --- /dev/null +++ b/Documentation/chronicle/commands/subject.md @@ -0,0 +1,89 @@ +--- +title: Subject +description: Tell Chronicle whose personal data the events of a command carry, with getSubject(), a @subject() field, @eventSubject, or a routed event, and know the resolution order. +--- + +An order belongs to the order's event source, but the email address on it belongs to a customer. When that customer asks to be forgotten, Chronicle must find every event carrying their data, across every order. The **subject** is how it finds them: Chronicle records a compliance subject on each event, and keys personal-data handling to it. + +By default the subject is the event source ID, which is right when the entity is the person. Set it on the command when the data belongs to someone else. + +## Compute it on the command + +```typescript +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { command, key } from '@cratis/arc.core'; + +@eventType() +export class OrderPlaced { + @field(String) customerId: string; + @field(Number) amount: number; + constructor(customerId = '', amount = 0) { this.customerId = customerId; this.amount = amount; } +} + +@command() +export class PlaceOrder { + @field(String) @key() orderId = ''; + @field(String) customerId = ''; + @field(Number) amount = 0; + + getSubject(): string { return this.customerId; } + + handle(): OrderPlaced { return new OrderPlaced(this.customerId, this.amount); } +} +``` + +`OrderPlaced` is appended to the order, and its subject is the customer. `getSubject()` runs after `handle()`, when the returned events are prepared, so it can read any field. Return a string. + +## Mark a field + +When a field already holds the subject, mark it with the SDK's `@subject()` from `@cratis/chronicle/compliance`: + +```typescript +import { field } from '@cratis/fundamentals'; +import { subject } from '@cratis/chronicle/compliance'; +import { command, key } from '@cratis/arc.core'; + +@command() +export class RegisterCustomer { + @field(String) @key() customerId = ''; + @field(String) @subject() personId = ''; + @field(String) email = ''; + + handle(): CustomerRegistered { return new CustomerRegistered(this.email); } +} +``` + +This command fragment assumes a `CustomerRegistered` event type in your application. + +:::caution[A GUID-valued subject field is ignored] +Arc reads a `@subject()` field only when its value is a string or a concept that wraps a string. A field holding a Fundamentals `Guid`, or a concept wrapping one, is skipped without an error, and the subject falls through to the next source. Declare the field as a string, or implement `getSubject()` and return `this.personId.toString()`. +::: + +## Set a fixed subject + +`@eventSubject('subject')` from `@cratis/arc.chronicle` sets the same subject for every event the command returns. It suits a subject that does not vary per request, such as a system actor. + +## Resolution order + +For each event a command returns, the integration takes the first of: + +1. the `subject` on an [`eventForEventSourceId`](event-metadata.md#override-one-event) entry; +2. `getSubject()` on the command; +3. a `@subject()` field holding a string or string concept; +4. `@eventSubject(...)` on the command; +5. the event's event source ID. + +Events an [aggregate](../aggregates/index.md) applies go through the same order, because the command's batch prepares them the same way. + +The subject never comes from the signed-in user. Authentication tells you who made the request; the subject says whose data the event holds, and the two often differ. + +## What the subject does not do + +Setting a subject does not mark anything as personal data. Values are marked with the SDK's `@pii` decorator, and Chronicle's compliance handling keys them to the subject; the [Chronicle compliance](/chronicle/compliance/) documentation covers that side. Arc does not release encrypted values when it serves a read model. See [Compliance](../compliance.md) for what the integration does and does not handle. + +## Related + +- [Event metadata](event-metadata.md) +- [Compliance](../compliance.md) +- [Chronicle compliance](/chronicle/compliance/) diff --git a/Documentation/chronicle/commands/toc.yml b/Documentation/chronicle/commands/toc.yml index 218a3e3f..30c7e332 100644 --- a/Documentation/chronicle/commands/toc.yml +++ b/Documentation/chronicle/commands/toc.yml @@ -1,5 +1,9 @@ - name: Returning events href: index.md +- name: Event metadata + href: event-metadata.md +- name: Subject + href: subject.md - name: Concurrency href: concurrency.md - name: Causation and auditing diff --git a/Documentation/chronicle/commands/transactional-commands.md b/Documentation/chronicle/commands/transactional-commands.md index e20d1a98..65e09f0c 100644 --- a/Documentation/chronicle/commands/transactional-commands.md +++ b/Documentation/chronicle/commands/transactional-commands.md @@ -1,40 +1,81 @@ --- title: Transactional commands -description: How returned events from nested commands join one event-log batch, when the batch is sent or discarded, and what lies outside it. +description: How returned events from a command and its nested commands join one event-log batch, when the batch is sent or discarded, how it interacts with command operations, and what lies outside it. --- -A command that runs other commands should record either all of their events or none. The integration stages returned events and sends them as one batch when the outermost command succeeds. This is a returned-event batch on one event log, **not** a .NET transaction. +Registering a member runs two commands: one creates the member, the other opens their first loan card. If the second is rejected, you do not want a member without a card. The integration collects the events every command in the chain returns and sends them to Chronicle as **one batch**, after the outermost command has succeeded. Either all of them are appended, or none. + +This is a batch on one event log, sent with the SDK's `appendMany`. It is not a transaction over other databases, HTTP calls, or services. + +## Choose how to append + +| How | Joins the batch | What the command result means | +| --- | --- | --- | +| Return events from `handle()` | Yes | The final verdict, after the batch is sent | +| Return `eventsWithConcurrencyScopes(...)` or `eventForEventSourceId(...)` | Yes | Same | +| `aggregate.apply(...)` on a [command aggregate](../aggregates/injecting-into-commands.md) | Yes, whether or not you return `aggregate.commit()` | Same | +| An SDK `eventLog.append(...)` inside `handle()` | No; it is appended immediately | Nothing about that append | +| A low-level `defineChronicleCommand` | No; it appends on its own | Its own result | + +Return events. It is the path that keeps the command's verdict and the stored events in step. ## One batch per outer command -Returned events from nested Arc commands that share the tenant, correlation ID, and event store join the outer command's single `eventLog.appendMany` call. +Returned events from nested Arc commands that share the tenant, correlation ID, and event store join the outer command's batch. Arc sends the batch after the outer command has run and its result is a success. - If the outer command or any nested command fails, none of the staged events are sent. -- A nested command's success is temporary: it is not a durable append acknowledgment. Only the outer result carries the final commit verdict. -- If the outer command ignores a failed nested command, even one that staged no events, the outer command still fails. -- A detached command that runs after the outer unit completes starts a new unit and appends on its own. +- A nested command's success is provisional: its events are staged, not stored. Only the outer result carries the final verdict. +- If the outer command ignores a failed nested command, even one that staged no events, the outer command still fails, with the message `Nested Chronicle command failed; staged events were discarded`. +- A nested command with a different tenant, correlation ID, or event store fails instead of joining. +- A command that runs after the outer command has completed, such as one started from a timer, starts its own batch. + +Authorization, Arc validation, and a rejecting `provide()` all run before anything is staged, so a rejected command never reaches Chronicle. + +## When the batch is sent -Authorization, Arc validation, and a rejecting `provide()` all run before anything is appended. +The outer command finishes, then Arc sends the batch and turns Chronicle's answer into the command result: + +| Chronicle answers | Command result | +| --- | --- | +| Every event accepted | Success, with the response `handle()` returned | +| A constraint or concurrency rejection | 400 with the validation results, and no response; see [Concurrency](concurrency.md#when-the-check-fails) | +| An error, a partial or incomplete acknowledgment, or no answer | Failure with an exception message; some events may have been stored | + +A request aborted before the batch is sent sends nothing. ## Causation in a batch -The SDK stamps a single causation chain per batch: nested events carry the outer command's causation, not their own command's. +The SDK stamps one causation chain per batch. Every event in it, including events from nested commands, carries the outer command's causation entry, not the entry of the command that returned it. See [Causation and auditing](causation.md). + +## Events and command operations + +A command may return Chronicle events together with [command operations](../../commands/operations/index.md), in a `tuple(...)`. The batch then decides whether the operations are compensated: + +1. Arc runs the operations in order. +2. It sends the batch. +3. On a constraint or concurrency rejection, nothing was appended, so Arc compensates the operations in reverse order. +4. On an unknown, partial, or incomplete outcome, Arc reports the command as failed but **does not compensate**. Events may already be stored, and they cannot be undone. + +Operations therefore need compensation only for a known rejection. This is still not a distributed transaction: an external system that accepted the operation stays changed until compensation runs, and a crash in between is not recovered. ## What is outside the batch -These happen outside the batch and cannot be rolled back with it: +These happen outside the batch and cannot be discarded with it: - an immediate SDK append inside `handle()`; - a low-level `defineChronicleCommand`; -- operations performed by a separate nested command or directly inside a handler. +- anything a handler does directly to another store or service. -An immediate append can already have persisted when a later returned-event batch is rejected. Arc cannot track that immediate append through this scope and **still compensates the command operations**. Do not combine immediate appends with compensated operations when the compensation assumes no events persisted. +An immediate append can already be stored when the batch is later rejected, and Arc cannot see it. Arc **still compensates the command operations** in that case. Do not combine immediate appends with compensated operations when the compensation assumes no events were stored. -A command may return both Chronicle events and [command operations](../../commands/operations/index.md). Arc runs the operations first, then commits the returned-event batch. A constraint or concurrency rejection prevents the append and compensates the operations in reverse order. If Chronicle reports an incomplete, partial, or unknown outcome, Arc reports the command as failed but **does not compensate**: committed events cannot safely be undone. Operations must provide compensation for a known rejection. This is not a distributed transaction over external systems. +## Things to know -Aggregate `apply()` enrolls events in the same batch even if `commit()` is not returned; see [aggregates](../aggregates/index.md). +- **Reads do not see staged events.** A read model or an event-log read inside the chain sees stored events only, not the batch being built. +- **Several reactor commands are several batches.** Each command a reactor returns runs on its own; see [Returning commands from a reactor](../reactors/command-side-effects.md#return-several-commands). +- **Atomic is not the same as current.** The batch is all-or-nothing, but it does not check that the decision was made on the latest state. See [Concurrency](concurrency.md). ## Related - [Concurrency](concurrency.md) - [Returning events](index.md) +- [Command execution scopes](../../commands/command-execution-scopes.md) diff --git a/Documentation/chronicle/compliance.md b/Documentation/chronicle/compliance.md new file mode 100644 index 00000000..bb4656da --- /dev/null +++ b/Documentation/chronicle/compliance.md @@ -0,0 +1,38 @@ +--- +title: Compliance +description: What the experimental Chronicle integration does for personal data, subjects, and audit exclusion in TypeScript, and which parts of Arc on .NET's compliance support it does not have. +--- + +Events are immutable, yet a person can ask for their personal data to be erased. Chronicle resolves that tension by keying personal data to a **subject** and managing encryption keys per subject; destroying the key makes the data unreadable while the events stay. The [Chronicle compliance guide](/chronicle/compliance/) explains that side. + +An Arc application meets compliance at two points: when a command appends events, and when a query serves the read models built from them. The TypeScript integration covers the first. It does not cover the second. + +## What the integration does + +| Concern | In Arc for TypeScript | +| --- | --- | +| Record the subject on appended events | Supported. `getSubject()`, a `@subject()` field, `@eventSubject(...)`, or a routed event's `subject`, falling back to the event source ID. See [Subject](commands/subject.md). | +| Keep command values out of the causation chain | Supported. `@notAudited()`, the SDK's `@pii()` on a command field or class, and secret-looking field names. See [Causation and auditing](commands/causation.md#keep-a-value-out). | +| Mark event or read-model data as personal | Done with the SDK's `@pii()` from `@cratis/chronicle/compliance`. Arc passes your event classes to the SDK unchanged. | +| Release encrypted values when a query serves a read model | **Not implemented.** Arc does not call Chronicle's release operation. | +| Release encrypted values in a command's read model | **Not implemented.** `commandReadModel(Type)` passes the instance as the SDK returns it. | +| Erase a subject's key | Not part of Arc. Use Chronicle's own tooling or the SDK's PII manager. | + +## Releasing values is up to you + +Arc on .NET releases personal data automatically through read-model interception before a response reaches the client. Arc for TypeScript has no equivalent. A query that returns a read model with values Chronicle encrypted returns them as stored. + +The SDK exposes `release(Type, instance)` and `releaseMany(Type, instances)` on `store.readModels`, and `ChronicleReadModels.getStore()` gives you the tenant's store. Arc does not call them, and this repository does not check a release against a kernel. If your read models hold encrypted values, call release where you serve them, and test the result against a real kernel before relying on it. + +Release is not authorization. Decide separately who may read a person's data. + +## Subject and authentication are different + +The subject is whose data an event holds. The signed-in principal is who asked for the change. Arc never derives one from the other. A support agent correcting a customer's address is the principal; the customer is the subject. + +## Related + +- [Subject](commands/subject.md) +- [Causation and auditing](commands/causation.md) +- [Chronicle compliance](/chronicle/compliance/) +- [Capability reference](../reference/capabilities.md#persistence-and-chronicle) diff --git a/Documentation/chronicle/index.md b/Documentation/chronicle/index.md index 5599f3f3..baa688b1 100644 --- a/Documentation/chronicle/index.md +++ b/Documentation/chronicle/index.md @@ -1,42 +1,108 @@ --- title: Chronicle -description: Return Chronicle events from Arc commands and read projected state, with the experimental @cratis/arc.chronicle integration, and know what it guarantees and what it does not. +description: Return Chronicle events from Arc commands, serve projected read models from Arc queries, and let reactors return Arc commands, with the experimental @cratis/arc.chronicle integration. --- -Arc does not require event sourcing: a command can do its work through any service. When you want commands to record facts in [Chronicle](/chronicle/), the event-sourcing database, `@cratis/arc.chronicle` lets a command **return** events instead of appending them inside `handle()`. Arc runs authorization and validation first, then appends the returned events in the trusted tenant's namespace. +A librarian registers an author. You want that registration kept as a fact, and you want the author list on screen to update from it. Without an integration, every command opens a Chronicle client, picks the event store namespace for the caller's tenant, appends inside `handle()`, turns constraint violations into validation results, and takes care never to append when validation has already failed. + +`@cratis/arc.chronicle` removes that plumbing. A command **returns** the event, and Arc appends it after authorization, validation, and `provide()` have passed, in the namespace of the tenant Arc already resolved. Arc does not require event sourcing: a command can do its work through any service. Use this integration when you want commands to record facts in [Chronicle](/chronicle/), the event-sourcing database. :::caution[Experimental] -`@cratis/arc.chronicle` is experimental. It has not been private since v0.12.0, but like every package here it is not published to npm. It is not a substitute for Arc on .NET's transactional Chronicle integration: it provides a single-event-log batch of returned events and keyed aggregates, but no reactor-command integration with the published Chronicle SDK. Its APIs can change. +`@cratis/arc.chronicle` is experimental, and, like every package in this repository, it is not published to npm. Its APIs can change. The [capability reference](../reference/capabilities.md#persistence-and-chronicle) has its status and the checks behind it. ::: -## What is verified +## How the pieces fit -The integration uses the published Chronicle TypeScript SDK, `@cratis/chronicle` 6.7.0, with `@cratis/fundamentals` 7.19.6; both load in native Node ESM with NodeNext resolution. An opt-in suite runs against a real development kernel and checks: +Arc and Chronicle meet at one loop. A command returns an event, Chronicle appends it and projects it into a read model, and an Arc query serves that read model back to the client. -- returned-event batches, readback, and tenant isolation; -- before-first concurrency rejection and aggregate rehydration, commit, and operation compensation; -- command-key read models for existing and missing keys; -- a projected read-model query; -- all of it through real Express, Fastify, and Hono HTTP adapters. +```mermaid +flowchart LR + UI[Client] -->|command| CMD["@command() class · handle()"] + CMD -->|returns an event| EV[(Chronicle event log)] + EV -->|projection| RM["@readModel() · @fromEvent"] + RM -->|query| UI + EV -->|reactor| RE["@reactor() · returns commands"] + RE -->|command| CMD +``` -Validator predicate binding is covered by substitute-based specs, not by the live kernel suite. Run `bash Source/Chronicle/run-integration.sh` to repeat the live checks; Docker is required. The test image `cratis/chronicle:latest-development` is mutable, so pin a compatible image for reproducible deployment testing. The ordinary `yarn test` specs use typed substitutes and never start a kernel. +The [Library sample](https://github.com/Cratis/Arc.TypeScript/tree/main/Samples/Library) walks that loop. Registration returns the event: -## Find your way +```typescript title="Features/Authors/Registration/Registration.ts (excerpt)" +@eventType() +export class AuthorRegistered { + @field(AuthorName) name: AuthorName; + constructor(name: AuthorName = new AuthorName('')) { this.name = name; } +} + +@command() +@roles('Librarian') +export class RegisterAuthor { + @key() @field(AuthorId) id!: AuthorId; + @field(AuthorName) name!: AuthorName; + + handle(): AuthorRegistered { return new AuthorRegistered(this.name); } +} +``` + +The listing projects that event into a read model and serves it: + +```typescript title="Features/Authors/Listing/Listing.ts (excerpt)" +@readModel() +@fromEvent(AuthorRegistered) +export class Author { + @field(AuthorId) id!: AuthorId; + @field(AuthorName) name!: AuthorName; + + @query({ observable: true }, service(ChronicleReadModels)) + static allAuthors(models: ChronicleReadModels): Observable { + return models.observeAll(Author, author => author.id.toString()); + } +} +``` -- [Aggregates](aggregates/index.md) rehydrate by event source and return pending events. -- [Transactional commands](commands/transactional-commands.md) stage nested events and compensate command operations on a known rejection. +The `@key()` field names the event source, so the registration lands in that author's stream. `@fromEvent(AuthorRegistered)` asks Chronicle to copy the event's matching properties into `Author`, keyed by the event source. The observable query then pushes a new author list to the browser whenever the projection changes. Appending and projecting are separate steps inside Chronicle, so a successful command can return before the list has caught up. + +## What the integration adds + +- **Returned events are appended.** `handle()` returns one event, several, or events beside a response. See [Returning events](commands/index.md). +- **Metadata comes from the command.** The event source, stream, subject, and causation are resolved from the command and the request. See [Event metadata](commands/event-metadata.md). +- **Tenants map to namespaces.** Every append and read uses the tenant of the current execution as the Chronicle namespace. +- **Rejections become validation results.** A constraint or concurrency violation answers 400, and nothing is appended. +- **Nested commands share one batch.** Returned events from nested commands are appended together, or not at all. See [Transactional commands](commands/transactional-commands.md). +- **Current state is a parameter.** A command can take its own read model or a rehydrated aggregate as a `handle()` argument. See [Read models in commands](read-models/injecting-into-commands.md) and [Aggregates](aggregates/index.md). +- **Reactors can return commands.** A Chronicle reactor returns an Arc command, and Arc runs it through the full command pipeline. See [Reactors](reactors/index.md). + +## Find your way | Page | Use it when you want to | | --- | --- | | [Add event sourcing](add-event-sourcing.md) | Register Chronicle with the application builder | | [Returning events](commands/index.md) | Return one event, a batch, or events next to a response | +| [Event metadata](commands/event-metadata.md) | See what each appended event carries and where every value comes from | | [Resolving the event source ID](resolving-event-source-id.md) | Choose which event source an event is appended to, and route it | +| [Subject](commands/subject.md) | Record whose personal data an event carries | | [Concurrency](commands/concurrency.md) | Reject an append when the stream moved | -| [Causation and auditing](commands/causation.md) | Keep secrets out of the permanent causation chain | +| [Causation and auditing](commands/causation.md) | See what the permanent causation chain records, and keep secrets out | | [Transactional commands](commands/transactional-commands.md) | Understand the batch across nested commands and its failure rules | -| [Read models](read-models/index.md) | Load projected state into commands and queries | +| [Read models](read-models/index.md) | Serve projected state from queries | +| [Read models in commands](read-models/injecting-into-commands.md) | Decide or validate from the state projected for the command's key | +| [When read model resolution fails](read-models/failures.md) | Understand a rejected command that loads a read model | +| [Aggregates](aggregates/index.md) | Decide from one event source's full history | +| [Reactors](reactors/index.md) | Run a follow-up command when an event is recorded | +| [Compliance](compliance.md) | Know what the integration does, and does not do, for personal data | +| [Code analysis](code-analysis.md) | See which .NET `ARCCHR` diagnostics apply in TypeScript | | [Testing Chronicle commands](../testing/chronicle.md) | Assert returned events without a kernel | +## Where it stops + +- A command's events go to one event log in one event store. There is no transaction across other stores or external calls. +- An event appended directly through the SDK inside `handle()` is outside the command's batch. +- Arc does not release encrypted personal data when it serves a read model. See [Compliance](compliance.md). +- The TypeScript SDK has no replay exclusion for reactors, so a reactor that returns commands must tolerate running again. +- No `ARCCHR` analyzers exist for TypeScript. See [Code analysis](code-analysis.md). + +Start with [Add event sourcing](add-event-sourcing.md). + ## Related - [Chronicle TypeScript client](https://github.com/Cratis/Chronicle.TypeScript), where the SDK is developed diff --git a/Documentation/chronicle/reactors/command-side-effects.md b/Documentation/chronicle/reactors/command-side-effects.md new file mode 100644 index 00000000..3d195765 --- /dev/null +++ b/Documentation/chronicle/reactors/command-side-effects.md @@ -0,0 +1,122 @@ +--- +title: Returning commands from a reactor +description: Return Arc commands from a Chronicle reactor so they run through validation and authorization, give them a system principal, and know the failure, tenancy, and retry rules. +--- + +When a book is added to the catalog, the search index should follow. The indexing command already exists, with its validation and its role check. Instead of calling a service from the reactor and repeating those checks, return the command, and Arc runs it through the same pipeline an HTTP caller would use. + +This relies on the SDK's reactor result hook, available in the Chronicle SDK 6.7.0 the integration requires. + +## Return a command + +```typescript title="Catalog.ts" +import { field } from '@cratis/fundamentals'; +import { eventType, type EventContext } from '@cratis/chronicle/events'; +import { reactor } from '@cratis/chronicle/reactors'; +import { command, key, roles } from '@cratis/arc.core'; +import { executeCommandsAsSystem } from '@cratis/arc.chronicle'; + +@eventType() +export class BookAdded { + @field(String) title: string; + constructor(title = '') { this.title = title; } +} + +@eventType() +export class BookIndexed { + @field(String) title: string; + constructor(title = '') { this.title = title; } +} + +@command() +@roles('CatalogWriter') +export class IndexBook { + @field(String) @key() id = ''; + @field(String) title = ''; + constructor(id = '', title = '') { this.id = id; this.title = title; } + + handle(): BookIndexed { return new BookIndexed(this.title); } +} + +@executeCommandsAsSystem('CatalogWriter') +@reactor() +export class CatalogIndexer { + bookAdded(event: BookAdded, context: EventContext): IndexBook { + return new IndexBook(context.eventSourceId, event.title); + } +} +``` + +Register all four with `builder.add(...)` or `builder.discover(...)` after `withChronicle`. When a `BookAdded` is appended, Chronicle calls `bookAdded`, and Arc executes `IndexBook`: authorization, validation, `provide()`, `handle()`, and the append of `BookIndexed`. The book ID comes from the triggering event's context, not from the event payload. + +## Who runs the command + +A reactor is not an HTTP request, so no caller is signed in. A returned command runs with **no principal** by default, as in Arc on .NET. A command without authorization rules runs normally; one with `@roles`, `@authorize`, or a policy is rejected. + +`@executeCommandsAsSystem('CatalogWriter')` on the reactor class gives the commands it returns an authenticated system principal with exactly the roles you list. Grant only the roles those commands need. The decorator covers returned commands only; a command you execute yourself inside the handler gets nothing from it. + +| Value | Without the decorator | With the decorator | +| --- | --- | --- | +| Arc principal | None | System, with the listed roles | +| Tenant | The triggering event's namespace | Same | +| Correlation ID | The triggering event's correlation ID | Same | +| Causation | A `ReactorEvent` entry with the event source ID, event type, sequence number, event store, and namespace, followed by the command | Same | + +Events the commands append carry Chronicle's system identity in both cases: the command has no signed-in user, and the principal the decorator supplies is the system identity. Arc executes the commands with an allowed severity of Warning, so warnings do not block and errors do. + +## Return several commands + +Return a nonempty array that contains only commands: + +```typescript +bookRemoved(event: BookRemoved, context: EventContext): (ArchiveBook | RemoveFromIndex)[] { + return [new ArchiveBook(context.eventSourceId), new RemoveFromIndex(context.eventSourceId)]; +} +``` + +This handler fragment assumes `BookRemoved` is an event type and `ArchiveBook` and `RemoveFromIndex` are commands in your application. Arc runs the commands in order and stops at the first one that fails. Each command is its own execution with its own [batch](../commands/transactional-commands.md): when the second fails, the first has already committed and stays committed. + +Do not mix commands with events or other values in one array. Arc rejects the mixture, and the handler fails, instead of silently dropping an item. Return either only commands or only events. + +## When a command fails + +A returned command that fails, whether rejected by authorization or validation or by throwing, fails the handler with a message naming the command, the event store, and the namespace. Chronicle marks the observer partition as failed rather than acknowledging a partial side effect. When Chronicle delivers the event again, the handler returns the commands again, including any that succeeded the first time. + +Make the commands safe to repeat. Key them by the triggering event source, check current state in [`provide()` or a read model](../read-models/injecting-into-commands.md), or rely on a Chronicle constraint to reject the duplicate. The TypeScript SDK has no replay exclusion like .NET's `[OnceOnly]`. + +No transaction spans the triggering event and the commands. The triggering event is already committed when the reactor runs. + +## Use a caller-owned client + +With an Arc-owned client (`{ connectionString, eventStore }`), `withChronicle` installs the result handler before observation starts. For a client you create yourself, pass `reactorCommandResultHandler` to the SDK **before the client connects**: + +```typescript title="main.ts (excerpt)" +import { ChronicleClient, ChronicleOptions } from '@cratis/chronicle'; +import { ArcApplication } from '@cratis/arc.core'; +import { ChronicleArtifacts, reactorCommandResultHandler } from '@cratis/arc.chronicle'; +import { BookAdded, CatalogIndexer, IndexBook } from './Catalog.js'; + +const artifacts = new ChronicleArtifacts(); +for (const type of [BookAdded, CatalogIndexer, IndexBook]) artifacts.register(type); + +let application: ArcApplication | undefined; +const client = new ChronicleClient(ChronicleOptions.fromConnectionString('chronicle://localhost:35000', { + clientArtifactsProvider: artifacts, + discoveryPatterns: [], + reactorResultHandler: reactorCommandResultHandler(() => application!.server, 'Catalog') +})); + +const builder = ArcApplication.createBuilder(); +builder.withChronicle({ client, eventStore: 'Catalog' }); +builder.add(BookAdded, CatalogIndexer, IndexBook); +application = await builder.build(); +``` + +The SDK calls the handler only while observing, after `build()` has assigned `application`. The second argument names the event store Arc appends to; a reactor observing a different store fails instead of running commands in the wrong place. The handler declines results that contain no command, so events returned from other reactors keep the SDK's own append path. + +## Related + +- [Reactors](index.md) +- [Transactional commands](../commands/transactional-commands.md) +- [Command pipeline](../../commands/command-pipeline.md) +- [Chronicle reactors](/chronicle/reactors/) diff --git a/Documentation/chronicle/reactors/index.md b/Documentation/chronicle/reactors/index.md new file mode 100644 index 00000000..d1665255 --- /dev/null +++ b/Documentation/chronicle/reactors/index.md @@ -0,0 +1,62 @@ +--- +title: Reactors +description: React to a recorded Chronicle event with a reactor, return follow-up events or Arc commands from it, and know how handlers are found and what a failure does. +--- + +After an author registers, the catalog should build a shelf for them. The registration command should not do that work itself: the shelf belongs to another slice, and the registration is complete the moment the fact is recorded. A projection answers "what does this look like now?". A **reactor** answers "what should happen because of this?". + +Reactors are part of the Chronicle SDK, `@cratis/chronicle`. The Arc integration adds one thing to them: a reactor can return Arc commands, and Arc runs them through its command pipeline. See [Returning commands from a reactor](command-side-effects.md). + +## Write a reactor + +```typescript title="ShelfBuilder.ts" +import { reactor } from '@cratis/chronicle/reactors'; +import type { EventContext } from '@cratis/chronicle/events'; +import { AuthorRegistered } from '../Registration/Registration.js'; +import { CreateShelf } from './CreateShelf.js'; + +@reactor() +export class ShelfBuilder { + authorRegistered(event: AuthorRegistered, context: EventContext): CreateShelf { + return new CreateShelf(context.eventSourceId); + } +} +``` + +`CreateShelf` is an ordinary Arc `@command()` in your application. Register the reactor the way you register every other artifact, with `builder.add(...)` or `builder.discover(...)` after [`withChronicle`](../add-event-sourcing.md). The Arc-owned Chronicle client starts observing it when the application builds. + +## How a handler is found + +The SDK looks for a method named after the event class in camelCase: `AuthorRegistered` is handled by `authorRegistered`. The parameter type plays no part in the match. + +:::caution[A misspelled handler is silently ignored] +A method whose name does not match an event class is never called, and nothing reports it. Rename the event class and the handler stops running. Check the method name first when a reactor appears to do nothing. +::: + +The first argument is the stored event content parsed from JSON, not an instance of your event class. Read its properties, but do not call its methods or test it with `instanceof`. A concept property arrives as its underlying primitive value, so `event.name` on `AuthorRegistered` is a string at runtime even though the class declares an `AuthorName`. The second argument is the event's `EventContext`, which carries the event source ID, sequence number, occurred time, correlation ID, causation, and the identity that caused it. + +The SDK constructs the reactor itself. It has no dependency injection, so a reactor cannot take constructor services. + +## What a handler can return + +| Return | What happens | +| --- | --- | +| Nothing | The event is acknowledged | +| A registered event, or an array of them | The SDK appends them to the triggering event's source and stream as one batch | +| An `EventForEventSourceId` entry, or an array mixing entries and events | The SDK appends each entry to its own target | +| An Arc `@command()` instance, or a nonempty array of only commands | Arc executes each command in order; see [Returning commands](command-side-effects.md) | + +Returning commands and events together in one array fails the handler. Anything else the SDK does not recognize is ignored. + +## When a handler fails + +A handler that throws, or a returned side effect that fails, marks the observer partition for that event source as failed, with the error message. Chronicle's failed-partition handling decides when that event is delivered again. Nothing that already happened is undone, so write handlers that are safe to run twice for the same event. + +The TypeScript SDK has no counterpart of .NET's `[OnceOnly]` replay exclusion. Treat every delivery as one that may be repeated. + +## Topics + +| Topic | Description | +| --- | --- | +| [Returning commands](command-side-effects.md) | Return Arc commands from a reactor and have Arc execute them with validation and authorization | +| [Chronicle reactors](/chronicle/reactors/) | Reactor concepts in the Chronicle documentation | diff --git a/Documentation/chronicle/reactors/toc.yml b/Documentation/chronicle/reactors/toc.yml new file mode 100644 index 00000000..7d727611 --- /dev/null +++ b/Documentation/chronicle/reactors/toc.yml @@ -0,0 +1,4 @@ +- name: Reactors + href: index.md +- name: Returning commands + href: command-side-effects.md diff --git a/Documentation/chronicle/read-models/failures.md b/Documentation/chronicle/read-models/failures.md new file mode 100644 index 00000000..1ad7ed62 --- /dev/null +++ b/Documentation/chronicle/read-models/failures.md @@ -0,0 +1,61 @@ +--- +title: When read model resolution fails +description: Every failure a command-scoped read model can produce, what the caller sees, what it means, and how to fix it. +--- + +A command that loads a read model can fail for two different kinds of reasons. The request can be wrong: no usable key, or an entity that does not exist. The application can be wrong: a read model no integration owns, or a lookup outside a validator. Arc keeps the two apart. Wrong input answers 400 with a validation result, and a misconfiguration fails loudly instead of posing as a missing entity. + +| Message | Cause | Caller sees | +| --- | --- | --- | +| `A command key is required for ` | The command has no usable key | 400 | +| ` was not found for the command key` | The key is valid, but a required read model does not exist | 400 | +| `Expected one read-model resolver for , found 0` | No integration owns the type | `build()` fails | +| `Expected one read-model resolver for , found 2` | Two integrations claim the type | `build()` fails | +| `Command read models can only be resolved during command validation` | `readModelForValidation` was called outside a running validator | The call throws | + +Both 400 answers carry one validation result with error severity, reason `rule`, and no members. The command's own code does not run. + +## A command key is required + +The command resolved no key, so there is nothing to look the read model up by. The key comes from `getEventSourceId()`, a `getKey()` method, or a `@key()` field; see [Resolving the event source ID](../resolving-event-source-id.md). An empty string counts as no key. + +Making the parameter optional does not help: `commandReadModel(Type, { optional: true })` still rejects a command without a key, because the lookup cannot be performed at all. + +:::note[A keyless command still appends] +A Chronicle command with no key appends its events to a new UUID. That UUID is created when the events are appended, after the read model would have been loaded, so it never serves as a lookup key. A command that works on an existing entity must declare that entity's key. +::: + +**Fix:** declare the key on the command. + +## Not found for the command key + +The key is valid, no instance exists for it, and the parameter is required. Arc treats this as invalid input: the command targets an entity that is not there. + +**Fix:** pick the one that matches your intent. + +- **Absence is a business condition.** Declare `commandReadModel(Type, { optional: true })`, or read it with `readModelForValidation(Type, { optional: true })` in a validator, and write the rule around `null`. You get your own message instead of the generic one. +- **The state is required.** Keep the parameter required. The 400 is the intended behavior. + +A projection runs after the event is appended. A command sent straight after the event that creates the entity can arrive before the read model exists. For a rule that must hold regardless of timing, use a Chronicle constraint. + +## No owner, or two owners + +`build()` checks every `commandReadModel(Type)` binding in `handle()` and `provide()` before the application serves anything. A type needs exactly one owner: + +- **Chronicle** owns a type that is a read model in its catalog: projected with `@fromEvent` or another projection decorator, or targeted by a projection or reducer, and registered **after** `withChronicle`. A class registered before `withChronicle`, or not registered at all, is not in the catalog. +- **MongoDB** owns a type listed in `withMongoDB({ readModels })`. +- An application can add its own `ReadModelForCommandResolver` with `builder.addReadModelForCommandResolver(...)`. + +Two owners fail too, instead of one silently winning. Remove the type from one integration. + +`readModelForValidation` is resolved when the validator runs, not at `build()`. A missing owner there makes the rule throw, and the caller gets a 400 with reason `validatorFailed`. + +## Outside a validator + +`readModelForValidation` reads the key and tenant from the validator that is running. Called anywhere else, such as in `handle()` or a query, it has no command to read from and throws. Use `commandReadModel(Type)` in `handle()` and `provide()`. + +## Related + +- [Read models in commands](injecting-into-commands.md) +- [Command validation](../../commands/command-validation.md) +- [Diagnostics](../../reference/diagnostics.md) diff --git a/Documentation/chronicle/read-models/index.md b/Documentation/chronicle/read-models/index.md index f4f23a1d..b2644510 100644 --- a/Documentation/chronicle/read-models/index.md +++ b/Documentation/chronicle/read-models/index.md @@ -1,54 +1,74 @@ --- title: Chronicle read models -description: Serve a Chronicle projection from an Arc query, load it into a command by key with commandReadModel, and look up instances with ChronicleReadModels. +description: Serve a Chronicle projection from Arc queries with ChronicleReadModels, as a snapshot or a live observable, and know the consistency you get. --- -Chronicle projects events into read models. The integration lets the same class be both a Chronicle read model and an Arc read model with queries, and lets a command load the current state of its own event source. +Chronicle projects events into read models and keeps them stored. You still need a query to hand them to the client, scoped to the caller's tenant, and ideally live so the screen updates when a new event lands. With the integration, one class is both the Chronicle read model and the Arc read model that serves it. ## One class, two roles +The [Library sample's author listing](https://github.com/Cratis/Arc.TypeScript/blob/main/Samples/Library/Features/Authors/Listing/Listing.ts): + ```typescript import { field } from '@cratis/fundamentals'; -import { fromEvent } from '@cratis/chronicle/projections'; -import { argument, query, readModel, service } from '@cratis/arc.core'; +import { query, readModel, service } from '@cratis/arc.core'; import { ChronicleReadModels } from '@cratis/arc.chronicle'; +import type { Observable } from 'rxjs'; +import { fromEvent } from '@cratis/chronicle/projections'; +import { AuthorRegistered } from '../Registration/Registration.js'; +import { AuthorId } from '../AuthorId.js'; +import { AuthorName } from '../AuthorName.js'; @readModel() -@fromEvent(LiveCreated) -export class LiveView { - static readonly readModelId = 'ArcTypeScriptLiveView'; - @field(String) id = ''; - @field(String) name = ''; - - @query(argument('id', String), service(ChronicleReadModels)) - static async byId(id: string, models: ChronicleReadModels): Promise { - return models.getById(LiveView, id); +@fromEvent(AuthorRegistered) +export class Author { + @field(AuthorId) id!: AuthorId; + @field(AuthorName) name!: AuthorName; + + @query({ observable: true }, service(ChronicleReadModels)) + static allAuthors(models: ChronicleReadModels): Observable { + return models.observeAll(Author, author => author.id.toString()); + } + + @query(service(ChronicleReadModels)) + static async authorsPage(models: ChronicleReadModels): Promise { + return models.getAll(Author); } } ``` -This excerpt is from the [kernel suite](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Chronicle/Integration/LiveArtifacts.ts), where `LiveCreated` is the event type. Arc's `@readModel()` exposes queries. Chronicle 6.7 infers the same model from `@fromEvent` (or a projection/reducer); do not add Chronicle's deprecated `@readModel()` decorator. Set `static readonly readModelId` only when you need to preserve a custom stored identifier. +Arc's `@readModel()` exposes the queries. Chronicle infers the same class as its read model from `@fromEvent`, or from a projection or reducer that targets it; do not add Chronicle's deprecated `@readModel()` decorator. `@fromEvent(AuthorRegistered)` copies the event's matching properties, and the event source ID becomes `id`. Set `static readonly readModelId` only when you need to keep a custom stored identifier. -`ChronicleReadModels` is a tenant-scoped service. Inject it with `service(ChronicleReadModels)` in a query or `@inject(ChronicleReadModels)` in a command. `getAll(type)` returns all projected instances and `getById(type, id)` returns one or `null`. For live results, `observeAll(type)` returns an RxJS `Observable` (models must expose an `id` convertible to a string); provide a key selector when they do not. `observeById(type, id)` returns an `Observable` that emits `null` when the model is removed. Both emit a snapshot before subscribing to changes; a change between the snapshot and subscription may be missed, so use `watch(type)` and reconcile from the store if gap-free observation matters. `watch(type)` returns an `Observable>`, and `watchIterable(type)` retains the async-iterable path. Unsubscribe to stop watching. Chronicle remains an experimental integration. +`allAuthors` answers a GET with the current list and then pushes a new list on every change. `authorsPage` returns a snapshot, and Arc [pages and sorts](../../queries/model-bound/paging.md) the array in memory, which suits a small catalog but not an unbounded list. -## Load a read model into a command +## What ChronicleReadModels offers -```typescript -@command() -export class ReadLiveInCommand { - @field(String) @key() id = ''; - @inject(commandReadModel(LiveView)) - handle(view: LiveView): string { return view.name; } -} -``` +`ChronicleReadModels` is a scoped service bound to the current tenant's event store. Inject it with `service(ChronicleReadModels)` in a query, or `@inject(ChronicleReadModels)` in a command. + +| Member | Returns | +| --- | --- | +| `getAll(Type)` | Every projected instance in the tenant | +| `getById(Type, id)` or `findInstanceById(Type, id)` | One instance by event source ID, or `null` | +| `observeAll(Type, key?)` | An RxJS `Observable`: a snapshot, then the updated list on every change | +| `observeById(Type, id)` | An `Observable` that emits `null` when the instance is removed | +| `watch(Type)` | An `Observable>` of raw changes | +| `watchIterable(Type)` | The same changes as an async iterable, without RxJS | +| `getStore()` | The tenant's SDK `IEventStore`, for anything else | + +`observeAll` keys the list by each model's `id`. Pass a key selector when your model names its identity differently, or when `id` is a concept, as `allAuthors` does with `author.id.toString()`. Unsubscribe, or let Arc end the subscription, to stop watching. + +## Consistency + +- **Projections are eventually consistent.** A command can succeed before its read model has updated. A client that reads right after a command can see the old state. An observable query catches up on its own. +- **The first list can miss a change.** `observeAll` and `observeById` read a snapshot, then subscribe to changes. A change that lands between the two is missed until the next change. Use `watch(Type)` and reconcile from the store when you need gap-free observation. +- **Everything is tenant-scoped.** Reads use the current execution's tenant as the namespace, like appends. -`commandReadModel(LiveView)` loads the model whose ID is the command's `@key()` value, from the tenant's event store. A missing required model becomes a validation failure; `commandReadModel(LiveView, { optional: true })` passes `null` when the SDK reports absence. Without a usable command key, both forms reject the command. The kernel suite verifies both the existing and the missing case. +## Use state in a command -- Only models identified by Chronicle in the application's artifact catalog qualify. -- If MongoDB owns a model instead, `withMongoDB` provides the same hook for its configured `readModels`. Do not register both integrations as owners of one type. -- Read models are not injected into validators. Make an explicit tenant-scoped lookup in a rule when validation needs state. +A command can take the read model for its own key as a `handle()` or `provide()` parameter, and a validator can read it, with no query round-trip. See [Read models in commands](injecting-into-commands.md). ## Related -- [Command context](../../commands/command-context.md#load-a-read-model-by-key) +- [Observable queries](../../queries/observable-queries.md) +- [When read model resolution fails](failures.md) - [Returning events](../commands/index.md) diff --git a/Documentation/chronicle/read-models/injecting-into-commands.md b/Documentation/chronicle/read-models/injecting-into-commands.md new file mode 100644 index 00000000..c5cadb98 --- /dev/null +++ b/Documentation/chronicle/read-models/injecting-into-commands.md @@ -0,0 +1,124 @@ +--- +title: Read models in commands +description: Take the Chronicle read model projected for a command's key as a handle() or provide() parameter with commandReadModel, read it in a validator with readModelForValidation, and decide what a missing instance means. +--- + +A member reserves a book. The reservation event should carry the book's title, and the command should refuse a book that is not in the catalog. Both answers are already in the `Book` read model Chronicle keeps for that book. You should not need to write a query, call it from the command, and map "not found" to an error yourself. + +Arc loads the read model whose ID is the command's key and passes it in. There are three places to take it: + +| Position | Use it when | Declare it with | +| --- | --- | --- | +| `handle()` | The event you return is computed from the state | `@inject(commandReadModel(Book))` | +| `provide()` | The state is combined with other fetched data before `handle()` decides | `@inject(commandReadModel(Book))` | +| A command validator | The command should be rejected with your own message | `await readModelForValidation(Book, { optional: true })` in an async rule | + +All three read the same instance: Arc loads each read model type at most once per command execution. + +## Take it in handle() + +```typescript title="Reservation.ts" +import { field } from '@cratis/fundamentals'; +import { eventType } from '@cratis/chronicle/events'; +import { fromEvent } from '@cratis/chronicle/projections'; +import { command, commandReadModel, inject, key } from '@cratis/arc.core'; + +@eventType() +export class BookAdded { + @field(String) title: string; + constructor(title = '') { this.title = title; } +} + +@eventType() +export class BookReserved { + @field(String) title: string; + @field(String) member: string; + constructor(title = '', member = '') { this.title = title; this.member = member; } +} + +@fromEvent(BookAdded) +export class Book { + @field(String) id = ''; + @field(String) title = ''; +} + +@command() +export class ReserveBook { + @field(String) @key() bookId = ''; + @field(String) member = ''; + + @inject(commandReadModel(Book)) + handle(book: Book): BookReserved { + return new BookReserved(book.title, this.member); + } +} +``` + +Register the command, the events, and `Book` with `builder.add(...)` or `builder.discover(...)` after `withChronicle`. `ReserveBook` for a book in the catalog appends `BookReserved` with the book's title. For a book that is not, it answers 400 with `Book was not found for the command key`, and `handle()` never runs. + +`Book` is a Chronicle read model because `@fromEvent` projects it. Add Arc's `@readModel()` and queries when clients should read it too; see [Chronicle read models](index.md). + +## Take it in provide() + +`provide()` receives the read model the same way, and its return value becomes the first `handle()` argument: + +```typescript +@inject(commandReadModel(Book)) +provide(book: Book): string { return book.title.trim(); } + +handle(title: string): BookReserved { return new BookReserved(title, this.member); } +``` + +This fragment replaces the `handle()` method of `ReserveBook`. Use `provide()` when the state has to be combined with something else you fetch, such as a rate or a policy, before `handle()` decides. See [Model-bound commands](../../commands/model-bound/index.md#prepare-data-in-provide). + +## Read it in a validator + +Validators do not take read models as constructor parameters. Call `readModelForValidation` inside an asynchronous rule instead: + +```typescript +import { CommandValidator, readModelForValidation, validator } from '@cratis/arc.core'; +import { Book, ReserveBook } from './Reservation.js'; + +@validator(ReserveBook) +export class ReserveBookValidator extends CommandValidator { + constructor() { + super(); + this.ruleFor(command => command.bookId) + .mustAsync(async () => await readModelForValidation(Book, { optional: true }) !== null) + .withMessage('The book is not in the catalog'); + } +} +``` + +`readModelForValidation` uses the command's key and the caller's tenant; it takes no ID, so a rule cannot read another entity by accident. It works only in validators of model-bound commands, while they run. Pass `{ optional: true }` and write the rule around `null`. Without it, a missing model throws inside the rule, and Arc reports a failed validator (reason `validatorFailed`) instead of your message. + +## Required or optional + +The key tells Arc which instance to load. It does not prove that instance exists. Say what absence means: + +| Declaration | Missing instance | +| --- | --- | +| `commandReadModel(Book)` | The command is rejected with 400 before your code runs | +| `commandReadModel(Book, { optional: true })` | Your code receives `null` | + +Choose optional when absence is a state your rule handles, such as "register only if not registered yet". Keep it required when the command cannot mean anything without the state. A read model that was projected asynchronously can lag behind the event that created it, so a command sent straight after creation can find nothing yet. For a rule that must hold regardless, use a Chronicle constraint. [When read model resolution fails](failures.md) lists every failure. + +## Which read models qualify + +- Chronicle resolves a type only when it is a read model in the application's Chronicle catalog: projected with `@fromEvent` or other model-bound projection decorators, or targeted by a projection or reducer, and registered after `withChronicle`. +- A MongoDB read model listed in `withMongoDB({ readModels })` is loaded the same way, from its collection; see [MongoDB](../../mongodb/index.md). Exactly one integration may own a type: `build()` fails when none or two claim it. +- The Drizzle integration does not resolve command read models. + +## Combine it with an aggregate + +A command can take both: projected state as context, and an [aggregate](../aggregates/injecting-into-commands.md) as the thing that changes. List both in `@inject(...)`, in the order of the `handle()` parameters. The read model is a snapshot; it does not lock anything, and it can lag. The aggregate's revision is what protects the decision. + +## Test it + +`ChronicleCommandScenario.givenReadModel(Book, 'book-1', instance)` pins the instance `commandReadModel(Book)` receives. See [Testing Chronicle commands](../../testing/chronicle.md). + +## Related + +- [When read model resolution fails](failures.md) +- [Load a read model by key](../../commands/command-context.md#load-a-read-model-by-key) +- [Command validation](../../commands/command-validation.md) diff --git a/Documentation/chronicle/read-models/toc.yml b/Documentation/chronicle/read-models/toc.yml index 293378dd..6937ac60 100644 --- a/Documentation/chronicle/read-models/toc.yml +++ b/Documentation/chronicle/read-models/toc.yml @@ -1,2 +1,6 @@ - name: Read models href: index.md +- name: Read models in commands + href: injecting-into-commands.md +- name: When resolution fails + href: failures.md diff --git a/Documentation/chronicle/resolving-event-source-id.md b/Documentation/chronicle/resolving-event-source-id.md index 1658f2fd..bb96c6d2 100644 --- a/Documentation/chronicle/resolving-event-source-id.md +++ b/Documentation/chronicle/resolving-event-source-id.md @@ -13,7 +13,11 @@ Every event belongs to an event source, such as one task. The integration takes | A `@key()` field | That field's value | | Neither | A new UUID for each execution | -The key is the same one Arc resolves for the [command context](../commands/command-context.md#give-a-command-a-key). +The key is the same one Arc resolves for the [command context](../commands/command-context.md#give-a-command-a-key), so read models and aggregates loaded for the command use the same event source. + +:::caution[getEventSourceId() must return a string] +`getEventSourceId()` may return a string or a concept that wraps a string. Any other value, including a Fundamentals `Guid` or a concept that wraps one, fails the command with `The command provided an invalid event source id`. Return `this.id.toString()` for a GUID-based identity. An empty string counts as no value, and the `@key()` field is used instead. +::: ## Return the ID to the caller @@ -41,11 +45,12 @@ Class decorators from `@cratis/arc.chronicle` set routing defaults for every eve | `@eventSourceType('Task', { concurrency? })` | The event source type | | `@eventStreamType('Onboarding', { concurrency? })` | The event stream type | | `@eventStreamId('main', { concurrency? })` | The event stream ID | -| `@eventSubject('subject')` | The compliance subject | +| `@eventSubject('subject')` | The compliance subject; see [Subject](commands/subject.md) | `{ concurrency: true }` reads that dimension's tail immediately before the append and scopes it; see [Concurrency](commands/concurrency.md). ## Related - [Returning events](commands/index.md) +- [Event metadata](commands/event-metadata.md) - [Command context](../commands/command-context.md) diff --git a/Documentation/chronicle/toc.yml b/Documentation/chronicle/toc.yml index a15dcdda..dfebd11c 100644 --- a/Documentation/chronicle/toc.yml +++ b/Documentation/chronicle/toc.yml @@ -9,4 +9,10 @@ - name: Read models href: read-models/toc.yml - name: Aggregates - href: aggregates/index.md + href: aggregates/toc.yml +- name: Reactors + href: reactors/toc.yml +- name: Compliance + href: compliance.md +- name: Code analysis + href: code-analysis.md diff --git a/Documentation/index.md b/Documentation/index.md index 9800f999..a2ca803b 100644 --- a/Documentation/index.md +++ b/Documentation/index.md @@ -51,7 +51,7 @@ These excerpts are from the [Tasks sample](https://github.com/Cratis/Arc.TypeScr ## CQRS first, event sourcing optional -Arc is a CQRS framework. A command can validate input, call a service, write to current-state storage, and return a response without any event log. The core has no dependency on event sourcing or on a database. The Chronicle integration is a separate package: experimental, and not private since v0.12.0; see [CQRS without event sourcing](/arc/arc-without-event-sourcing/) for how that boundary works in Arc generally. +Arc is a CQRS framework. A command can validate input, call a service, write to current-state storage, and return a response without any event log. The core has no dependency on event sourcing or on a database. The Chronicle integration is a separate, experimental package; see [CQRS without event sourcing](/arc/arc-without-event-sourcing/) for how that boundary works in Arc generally. ## A server for the clients you already have @@ -77,6 +77,7 @@ Arc for TypeScript is versioned independently of Arc on .NET. GitHub source prev ## Where to go next +- [Why Arc for TypeScript](why-arc-for-typescript.md): who it is for, what it removes, and when it is the wrong fit. - [Get started](getting-started/index.md): run the Tasks sample and call its command and query. - [Coming from Express and NestJS](coming-from-express-and-nestjs.md): compare Arc with the code you write today. - [Hosting overview](overview.md): choose the standalone host or a framework adapter. From 132c0e503716321356b02a735a1edd1859505237 Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:58 +0200 Subject: [PATCH 2/6] Expand MongoDB naming, serializer, and observation documentation - Document the Cratis:MongoDB configuration binding and complete the getting-started code - Cover pluralization, custom policies, stored names in filters, and silent naming mismatches - List read rules and mapping errors, and correct the TimeOnly format - Cover deleted documents, paging observed queries, and stream failure and recovery --- Documentation/mongodb/getting-started.md | 57 +++++++++++-- Documentation/mongodb/index.md | 12 ++- Documentation/mongodb/naming-policies.md | 82 +++++++++++++++++-- .../mongodb/observing-collections.md | 75 +++++++++++++---- Documentation/mongodb/serializers.md | 73 +++++++++++++---- 5 files changed, 248 insertions(+), 51 deletions(-) diff --git a/Documentation/mongodb/getting-started.md b/Documentation/mongodb/getting-started.md index cdd1fe87..302a8219 100644 --- a/Documentation/mongodb/getting-started.md +++ b/Documentation/mongodb/getting-started.md @@ -1,9 +1,9 @@ --- title: Get started with MongoDB -description: Register a MongoDB client and read models with withMongoDB, inject a tenant-scoped collection into model-bound queries, and own the client's lifetime. +description: Register a MongoDB client and read models with withMongoDB, inject a tenant-scoped collection into model-bound queries, configure the connection from appsettings.json, and own the client's lifetime. --- -This page connects one read model to MongoDB and serves it from a query. The code comes from the package's [replica-set integration fixture](https://github.com/Cratis/Arc.TypeScript/tree/main/Source/MongoDB/for_MongoCollection/given). +This page connects one read model to MongoDB and serves it from three queries: a list, a page, and a live list. It follows the package's [replica-set integration fixture](https://github.com/Cratis/Arc.TypeScript/tree/main/Source/MongoDB/for_MongoCollection/given). You need a MongoDB server; the live query needs a replica set. ## Declare the model @@ -17,11 +17,15 @@ export class TaskRecord { } ``` -`@key()` marks the field stored as `_id`; without it, a field named `id` is used. +The collection reads and writes only the fields you declare with `@field`. `@key()` marks the field stored as `_id`; without it, a field named `id` is used, and a model with neither is rejected. ## Inject the collection into queries ```typescript title="TaskQueries.ts" +import { query, queryOptions, readModel, service, type QueryOptions } from '@cratis/arc.core'; +import { mongoCollection, type MongoCollection } from '@cratis/arc.mongodb'; +import { TaskRecord } from './TaskRecord.js'; + const tasks = mongoCollection(TaskRecord); @readModel() @@ -37,13 +41,15 @@ export class TaskQueries { } @query({ observable: true }, service(tasks)) - static async changes(items: MongoCollection) { + static changes(items: MongoCollection) { return items.observe(); } } ``` -This excerpt omits the imports: `readModel`, `query`, `queryOptions`, `service`, and the `QueryOptions` type come from `@cratis/arc.core`; `mongoCollection` and `MongoCollection` from `@cratis/arc.mongodb`. `mongoCollection(TaskRecord)` is a service token for the current tenant's collection. For an owner-restricted query, build a specific filter from the verified principal, such as `items.find({ owner: principal.id })`, instead of forwarding a caller-provided object. +`mongoCollection(TaskRecord)` is a service token. Each request resolves it to the collection in the current tenant's database, so a query never chooses a database itself. `all` returns every task, `page` pushes count, sort, and paging into MongoDB (see [Paging](paging.md)), and `changes` opens a change stream (see [Observing collections](observing-collections.md)). + +For an owner-restricted query, build the filter from the verified principal, such as `items.find({ owner: principal.id })`, instead of forwarding a caller-provided object. ## Register MongoDB @@ -64,7 +70,32 @@ await app.run(); await client.close(); ``` -Importing `@cratis/arc.mongodb` adds `withMongoDB` to the builder. The exported `withMongoDB(builder, options)` function is equivalent. The fixed `default` tenant makes this a single-tenant example; see [Tenancy](tenancy.md) for real tenant routing. +Importing `@cratis/arc.mongodb` adds `withMongoDB` to the builder. The exported `withMongoDB(builder, options)` function is equivalent. List every model you inject in `readModels`; a model left out has no collection token. + +The fixed `default` tenant makes this a single-tenant example, and every request uses the `tasks_default` database. See [Tenancy](tenancy.md) for real tenant routing. + +A GET on the `all` query's route answers with the stored tasks. [Endpoint mapping](../core/endpoint-mapping.md) explains how routes are derived. + +## Configure the connection in appsettings.json + +`withMongoDB` reads `Cratis:MongoDB` from the application's [configuration](../configuration/index.md), so the server address and database can stay out of code: + +```json title="appsettings.json" +{ + "Cratis": { + "MongoDB": { + "server": "mongodb://127.0.0.1:27017", + "database": "tasks" + } + } +} +``` + +```typescript title="main.ts (excerpt)" +builder.add(TaskQueries).withMongoDB({ readModels: [TaskRecord] }); +``` + +Only `server` and `database` bind from configuration; set everything else in code. `Cratis__MongoDB__Server` and `Cratis__MongoDB__Database` override the file in a deployment. Values in code win over configuration, and a `client`, `server`, or `serverResolver` in code replaces a configured `server`. With `server`, Arc creates the client and closes it when the application is disposed. ## Options @@ -73,14 +104,24 @@ Importing `@cratis/arc.mongodb` adds `withMongoDB` to the builder. The exported | `client` | A caller-owned `MongoClient`; Arc leaves it open | | `server` | A MongoDB URI; Arc creates the client and closes it with the application | | `serverResolver(tenantId, context)` | A URI per tenant, for tenants on different servers | -| `database` | The default database name | +| `database` | The default database name; other tenants use `+` | | `databaseNameResolver(tenantId, context)` | A database name per tenant | | `readModels` | Model classes registered for injection and command read-model resolution | | `namingPolicy`, `collectionName`, `ignoreConventions` | See [Naming policies](naming-policies.md) and [Serializers](serializers.md) | | `maxPageSize` | Page size cap, 100 by default, at most 10,000 | | `maxObservableItems` | Observed snapshot cap, 1,000 by default, at most 10,000 | -Specify exactly one of `client`, `server`, or `serverResolver`, and either `database` or `databaseNameResolver`. A missing tenant or empty database name fails rather than reading an implicit default database. +Specify exactly one of `client`, `server`, or `serverResolver`, and either `database` or `databaseNameResolver`. A missing tenant or empty database name fails the request rather than reading an implicit default database. + +## Write documents + +The collection's query methods read. To write from a command, inject the same token and write through the driver collection, encoding the model first: + +```typescript +await items.native.insertOne(items.codec.serialize(task)); +``` + +Encoding through `codec` keeps the stored document in the shape reads expect. See [Serializers](serializers.md#write-through-the-driver). ## Related diff --git a/Documentation/mongodb/index.md b/Documentation/mongodb/index.md index b5bab394..e38ad1ae 100644 --- a/Documentation/mongodb/index.md +++ b/Documentation/mongodb/index.md @@ -6,14 +6,14 @@ description: Serve model-bound queries from tenant-scoped MongoDB collections wi Your read models live in MongoDB, and every tenant has its own database. Wiring a client, choosing the database per request, mapping concepts and GUIDs to BSON, and turning a change stream into a live query is the same code in every service. `@cratis/arc.mongodb` supplies it: your model declares its fields once, and the collection maps them to BSON and returns instances of your model. :::note[Source preview] -`@cratis/arc.mongodb` is optional and not published to npm. It uses the MongoDB 6 driver. +`@cratis/arc.mongodb` is optional and not published to npm. It uses the MongoDB 6 driver. The [capability reference](../reference/capabilities.md#persistence-and-chronicle) has its status and the checks behind it. ::: ## What it provides | Capability | Page | | --- | --- | -| Register collections with `builder.withMongoDB(...)` and inject them into queries | [Get started](getting-started.md) | +| Register collections with `builder.withMongoDB(...)`, or from `Cratis:MongoDB` configuration, and inject them into queries | [Get started](getting-started.md) | | Choose a database, or a server, per tenant | [Tenancy](tenancy.md) | | Map decorated fields, concepts, GUIDs, and dates to BSON | [Serializers](serializers.md) | | Match Arc on .NET's property and collection naming | [Naming policies](naming-policies.md) | @@ -21,7 +21,7 @@ Your read models live in MongoDB, and every tenant has its own database. Wiring | Turn a change stream into an observable query | [Observing collections](observing-collections.md) | | Load a read model by command key | [Command context](../commands/command-context.md#load-a-read-model-by-key) | -`withMongoDB` registers a read-model resolver for the models you list in `readModels`, so a command can declare `@inject(commandReadModel(TaskRecord))` and receive the document whose identity equals the command key. Do not also register another integration, such as Chronicle, as the owner of the same type. +`withMongoDB` registers a read-model resolver for the models you list in `readModels`, so a command can declare `@inject(commandReadModel(TaskRecord))` and receive the document whose identity equals the command key. Do not also register another integration, such as Chronicle, as the owner of the same type; `build()` fails when two claim one type. :::caution[Storage does not authorize a caller] Arc selects a tenant from the execution context, and the collection selects that tenant's database. Your authentication and authorization still have to verify that the caller may use that tenant and read those documents. Never pass untrusted request JSON directly to a MongoDB filter. @@ -31,10 +31,8 @@ Arc selects a tenant from the execution context, and the collection selects that The original `MongoReadModels` remains for low-level `defineQuery` users. It takes a caller-owned client, `databaseForTenant`, and a trusted `filterFor(input, context)`. Its `queryPage` accepts Arc sorting only for fields listed in `sortableFields`, and caps pages at 100 by default. It has no change streams or field codecs; use the model-bound collection for those. -## Verify against a real replica set - -Run `bash Source/MongoDB/run-integration.sh` from the repository root. The script starts a task-owned MongoDB 7 replica set in Docker and removes it afterward. The [integration spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/MongoDB/for_MongoCollection/when_observing_changes/with_a_replica_set.integration.ts) exercises initial snapshots, insertion, deletion, tenant isolation, dependency injection, and provider paging, and a [second spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/MongoDB/for_MongoCollection/when_serving_a_paged_query/with_each_http_adapter.integration.ts) serves sorted pages through Express, Fastify, and Hono. The script exits with 2 when Docker is not available, which means the check did not run. - ## Current boundaries This integration does not supply transactions, a shared watcher or reconnect policy, joined observations, geospatial serializers, resilience middleware, or driver metrics. Do not infer any of those from Arc on .NET. The [capability reference](../reference/capabilities.md#persistence-and-chronicle) has the parity details. + +Start with [Get started](getting-started.md). diff --git a/Documentation/mongodb/naming-policies.md b/Documentation/mongodb/naming-policies.md index 600c4e7e..0f25356a 100644 --- a/Documentation/mongodb/naming-policies.md +++ b/Documentation/mongodb/naming-policies.md @@ -1,16 +1,29 @@ --- title: Naming policies -description: Choose how MongoDB property and collection names are derived, and match Arc on .NET's default or camel-case policy when both read the same database. +description: Choose how MongoDB property and collection names are derived, match Arc on .NET when both read one database, write a custom policy, and use the stored names in your own filters. --- -When a TypeScript service and a .NET service share a MongoDB database, both must agree on property and collection names. The naming policy decides them. +A .NET service writes `Tasks` documents with a `Title` property. Your TypeScript service reads the same database, and its model declares `title`. Unless both sides agree on names, the TypeScript side reads documents with every field missing, and nothing fails. The naming policy decides the stored name of every property and the name of every collection. + +## Choose a preset | Policy | Properties | Collections | .NET equivalent | | --- | --- | --- | --- | -| `defaultMongoNamingPolicy` (default) | Declared names | Pluralized, case-preserving read-model names | Unconfigured MongoDB builder (`new DefaultNamingPolicy()`) | +| `defaultMongoNamingPolicy` (default) | As declared | Pluralized read-model class name, case kept | Unconfigured MongoDB builder (`new DefaultNamingPolicy()`) | | `camelCaseMongoNamingPolicy` | Camel-cased | Pluralized, camel-cased | `.WithCamelCaseNamingPolicy()` | -Both presets keep leading acronyms together. With the default policy, the TypeScript property spelling must match the .NET declaration, for example `Title`, not `title`. +For example, a `TaskRecord` class with `Id` and `Title` fields: + +| | Default | Camel case | +| --- | --- | --- | +| Collection | `TaskRecords` | `taskRecords` | +| `Title` field | `Title` | `title` | +| `URL` field | `URL` | `URL` | +| `@key()` field | `_id` | `_id` | + +Both presets keep a leading acronym together, as .NET does: `URL` stays `URL` under camel case. + +With the default policy, the stored name is exactly the TypeScript field name. To read documents a default-policy .NET service wrote, spell the field as .NET does, `Title`, not `title`. When the .NET side uses camel case, use `camelCaseMongoNamingPolicy` and keep idiomatic TypeScript field names. ## Set a policy @@ -20,12 +33,67 @@ import { camelCaseMongoNamingPolicy } from '@cratis/arc.mongodb'; builder.withMongoDB({ client, database: 'tasks', readModels: [TaskRecord], namingPolicy: camelCaseMongoNamingPolicy }); ``` -## Custom names +The policy applies to every model registered in that `withMongoDB` call, including nested models and derived types. + +## Pluralization + +Both presets pluralize the class name with a few English rules: a name ending in `s`, `x`, `z`, `ch`, or `sh` takes `es`, a consonant followed by `y` becomes `ies`, and everything else takes `s`. So `Category` becomes `Categories` and `Box` becomes `Boxes`. + +.NET pluralizes with Humanizer, which knows irregular words: `Person` becomes `People` there, but `Persons` here. When the two disagree, set the name yourself. + +## Override the collection name + +To change only the collection, keep the policy and pass `collectionName`: + +```typescript +builder.withMongoDB({ client, database: 'library', readModels: [Person], + collectionName: model => model === Person ? 'People' : `${model.name}s` }); +``` + +The function receives each registered model class and must return a nonempty name for all of them, or the request that resolves the collection fails with `MongoDB collection name is required`. + +## Write a custom policy + +A `MongoNamingPolicy` is two functions: + +```typescript +import type { MongoNamingPolicy } from '@cratis/arc.mongodb'; + +export const snakeCase: MongoNamingPolicy = { + propertyName: name => name.replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase(), + collectionName: type => type.name.replace(/([a-z0-9])([A-Z])/g, '$1_$2').toLowerCase() +}; +``` + +Write one when a .NET service uses a customized policy, or when your database already has its own convention. `propertyName` is never called for the key: the `@key()` field is always stored as `_id`. + +## Names in your own filters + +`find`, `queryPage`, and `observe` take a raw MongoDB filter. The filter goes to the driver as written, so it must use **stored** names and **stored** values. For a model with a `status` field and a `Guid` key: + +```typescript +import type { Document, Filter } from 'mongodb'; + +const done = await items.find({ [items.codec.fieldName('status')]: 'done' }); +const one = await items.find({ _id: items.codec.id(taskId) } as Filter); +``` + +`codec.fieldName(name)` converts a model field name to its stored name under the active policy, accepts the wire name too, and throws for a name the model does not declare. `codec.id(value)` encodes a key value, such as a `Guid`, the way it is stored. A `Guid` in any other field is stored as UUID binary, so a plain string will not match it. + +## Sorting and errors + +Client sorting goes through the same mapping. A request with `sortBy=title` resolves `title` to the declared field and then to its stored name. A name that is not a declared field, such as `secret` or `$where`, answers 400 before MongoDB is queried. See [Paging](paging.md). -A `MongoNamingPolicy` supplies `propertyName` and `collectionName` functions. Write your own for irregular plurals handled by .NET's Humanizer or for customized .NET policies. To change only the collection name, pass the `collectionName` option. +| Mistake | What happens | +| --- | --- | +| Field spelled differently from the stored documents | Reads succeed, and the field stays unset on every instance | +| Policy differs from the service that wrote the documents | Same: reads succeed with missing values | +| Collection name differs | Reads return nothing, and writes go to a new collection | +| Unknown sort field | 400 | -Sorting uses the same policy: an Arc wire name such as `title` resolves to the declared property, then to its BSON name. See [Paging](paging.md). +The first three produce no error. When a query returns empty or half-filled models, compare the stored names with `codec.fieldName(...)` and the collection name first. ## Related - [Serializers](serializers.md) +- [Get started with MongoDB](getting-started.md) diff --git a/Documentation/mongodb/observing-collections.md b/Documentation/mongodb/observing-collections.md index 3ba57ebf..35ceeb7e 100644 --- a/Documentation/mongodb/observing-collections.md +++ b/Documentation/mongodb/observing-collections.md @@ -1,36 +1,83 @@ --- title: Observing collections -description: Turn a tenant's MongoDB collection into an observable query with change streams, and know the snapshot, burst, cap, and replica-set rules. +description: Turn a tenant's MongoDB collection into an observable query with change streams, page and sort it, handle deleted documents, and know what happens when the stream fails. --- -A live list backed by MongoDB should update when a document changes, whichever process wrote it. `observe` opens a change stream on the tenant's collection and turns it into an Arc observable source. +A task list backed by MongoDB should update when a document changes, whichever process wrote it: this service, another service, or a script. Polling is either slow or wasteful. `observe` opens a MongoDB change stream on the tenant's collection and turns it into an Arc observable source, so the client receives a new list whenever the collection changes. ```typescript -@query({ observable: true }, service(tasks)) -static changes(items: MongoCollection): Observable { - return items.observe(); +import type { Observable } from 'rxjs'; +import { query, readModel, service } from '@cratis/arc.core'; +import { mongoCollection, type MongoCollection } from '@cratis/arc.mongodb'; +import { TaskRecord } from './TaskRecord.js'; + +const tasks = mongoCollection(TaskRecord); + +@readModel() +export class TaskChanges { + @query({ observable: true }, service(tasks)) + static all(items: MongoCollection): Observable { + return items.observe(); + } } ``` -`items.observe(filter?)` returns an RxJS `Observable` of full snapshots of the matching documents; `items.observeById(id)` emits one document, or `null` after it is deleted. Import `Observable` from `rxjs` (or use `import type`). For an async iterator instead, use `await items.observeIterable(filter?)` or `await items.observeByIdIterable(id)`. Deleting the last matching document from a list appears as an empty list. +A GET on this query answers 200 with the current tasks. Server-sent events and WebSocket subscribers receive the full list now, and again after every change. Change streams need a replica set or a sharded cluster. + +## Choose what to observe + +| Method | Emits | +| --- | --- | +| `observe(filter?)` | Every document matching the filter, as a full list | +| `observeById(id)` | One document, or `null` while it does not exist | +| `observeIterable(filter?)` | The same lists as `observe`, as an async iterable, without RxJS | +| `observeByIdIterable(id)` | The same values as `observeById`, as an async iterable | + +The filter is a raw MongoDB filter in stored names; see [Names in your own filters](naming-policies.md#names-in-your-own-filters). Build it from trusted values, never from request JSON. ## How observation works -1. The stream opens **before** the first snapshot, from a server operation time captured beforehand, so no change between the two is lost. -2. The source has a current value, so a plain GET answers 200, and server-sent events and WebSockets deliver full snapshots on each change. -3. After queued changes, the query is recomputed, even for changes that do not affect the filter. Bursts may be coalesced. -4. `observeById` filters the stream by document key, like Arc on .NET's `ObserveById` and `ObserveSingle`. +1. The change stream opens **before** the first snapshot is read, from a server operation time captured just before, so a change between the two is not lost. +2. The first snapshot is the source's current value, so a plain GET answers 200 without waiting. +3. After each change, and after any changes already queued behind it, the whole query is read again. A burst of changes becomes one new list. +4. For `observe`, every change on the collection triggers a new read, even one that does not affect the filter, so the list you receive is always complete. `observeById` reacts only to changes of its own document. + +Each subscription opens one change stream on its first read. Unsubscribing, closing the connection, or disposing the Arc scope closes the cursor. The async iterables also close when the loop exits. + +## When a document is gone + +- `observe` drops a deleted document from the next list. Deleting the last matching document emits an empty list. +- `observeById` emits `null` when the document is deleted, and emits the document again if it is recreated with the same ID. +- A document that is updated so it no longer matches the filter drops out of the next list, like a deletion. + +A client should treat `null` and an empty list as ordinary states, not as errors. -Each observable instance opens one lazy change stream on its first snapshot read or subscription. Unsubscribe or dispose the Arc scope to close the cursor; the async-iterable API also closes on iterator return. +## Page and sort an observed query + +A client can page and sort an observable query with the same parameters as a snapshot query. Arc applies them to **each** emitted list in memory, with the same options for the whole subscription: + +```text +GET /api/.../all?pageSize=10&page=0&sortBy=title +``` + +Each emission then holds the first ten tasks by title, and `paging.totalItems` counts every document in that emission's list. The database still returns the full matching list on every change; paging only trims what is sent. Keep observed filters narrow, and use [`queryPage`](paging.md) for large collections that do not need to be live. ## Limits -- A full snapshot is capped at 1,000 documents by default; `maxObservableItems` raises it to at most 10,000. Exceeding the cap fails the subscription rather than returning a partial list. -- A standalone MongoDB server is rejected with a replica-set requirement. Replica sets and sharded clusters support change streams. -- The driver resumes resumable stream errors; a non-resumable failure ends the subscription. +- A full list is capped at 1,000 documents by default; `maxObservableItems` raises the cap to at most 10,000. A list that exceeds the cap, on the first read or on any later one, fails the subscription instead of sending a partial list. +- A standalone MongoDB server is rejected with `MongoDB observe requires a replica set with change streams`. Replica sets and sharded clusters support change streams. - Joined observation across collections is not available. +## When the stream fails + +The MongoDB driver resumes the change stream after errors it knows are transient, such as a replica-set election. Your subscription does not notice. + +Anything else ends the subscription with a query failure: a change stream error the driver cannot resume, a read that fails, the item cap, or the change stream ending on its own (`MongoDB change stream ended`). Arc closes the cursor and reports the failure to the subscriber. It does not reopen the stream by itself. + +To recover, subscribe again. A new subscription opens a new change stream and reads a fresh, complete list, so the client catches up on everything that changed in between. + ## Related - [Observable queries](../queries/observable-queries.md) +- [Paging](paging.md) - [MongoDB](index.md) diff --git a/Documentation/mongodb/serializers.md b/Documentation/mongodb/serializers.md index 430b33d0..2bffe14c 100644 --- a/Documentation/mongodb/serializers.md +++ b/Documentation/mongodb/serializers.md @@ -1,9 +1,9 @@ --- title: Serializers -description: How MongoCollection maps decorated fields, concepts, GUIDs, dates, and derived types to BSON, and how to write through the underlying driver without breaking the mapping. +description: How MongoCollection maps decorated fields, concepts, GUIDs, dates, numbers, and derived types to BSON, what fails and why, and how to write through the driver without breaking the mapping. --- -`MongoCollection` reads and writes through a codec derived from the model's `@field` metadata. You get model instances back from reads, and documents stay compatible with what Arc on .NET stores. +A task ID is a `TaskId` concept in your code, a UUID binary in MongoDB, and a string on the wire. Getting each of those conversions right by hand, in every query and every write, is how documents written by one service become unreadable to another. `MongoCollection` does it from the model's `@field` declarations: reads return instances of your model, and the stored documents match what Arc on .NET writes. ## Storage format @@ -11,25 +11,68 @@ description: How MongoCollection maps decorated fields, concepts, GUIDs, dates, | --- | --- | | `@key()` field, or a field named `id` | `_id` | | `Guid` | Standard UUID binary (subtype 4) | -| A concept | Its underlying primitive | +| A concept | Its underlying value, stored by the same rules | | `Date` | BSON date | -| `DateOnly` | BSON date at UTC noon | -| `TimeOnly` | Milliseconds after the Unix epoch | -| `TimeSpan` | String | -| Nested decorated models and arrays | The same mapping, recursively | +| `DateOnly` | BSON date at 12:00 UTC on that day | +| `TimeOnly` | BSON date on 1970-01-01, UTC | +| `TimeSpan` | String, such as `01:02:03.123` | +| `String`, `Number`, `Boolean` | As is | +| Nested decorated model | Embedded document, by the same rules | +| Array with `genericArguments` | Array, each element by the same rules | +| A class with Fundamentals `@derivedType('identifier')` | Its fields plus `_derivedTypeId` | -A class annotated with Fundamentals `@derivedType('identifier')` writes `_derivedTypeId`; reading an unknown discriminator fails instead of creating a base model. No global BSON conventions are installed. +Property names come from the [naming policy](naming-policies.md). No global BSON conventions or class maps are installed, so other code using the same driver is unaffected. -## Read and write +## How reads map documents -- `items.find(filter)` and `items.findById(id)` return model instances. `findById` rejects operator objects as identities. -- `items.codec` is the codec; `items.native` is the underlying `mongodb` driver collection. -- To write elsewhere in your application, encode first: `await items.native.insertOne(items.codec.serialize(task))`. -- Reading through `native` returns driver documents, not model instances. +`items.find(filter)`, `items.findById(id)`, `queryPage`, and `observe` all build model instances the same way: -## Existing documents +- Only declared fields are read. A stored property with no matching `@field` is ignored. +- A declared field missing from the document stays unset on the instance. +- A concept field is rebuilt as an instance of the concept class. +- `Number` accepts BSON doubles and integers, and `Int64` or `Decimal128` values that a JavaScript number represents exactly. A value it cannot represent exactly, such as `Decimal128('0.123456789123456789')`, fails the read with a `RangeError` rather than rounding. +- For a base type with registered derivatives, `_derivedTypeId` selects the concrete class; GUID identifiers compare case-insensitively. -`ignoreConventions: true` bypasses the codec for collections that already store driver-native documents. In that mode, you own field names and conversion. +`findById` accepts a primitive, `Guid`, concept, `ObjectId`, or UUID binary, and rejects an object such as `{ $ne: null }` so a request value cannot become a query operator. + +## When mapping fails + +| Error | Cause | +| --- | --- | +| `MongoDB model requires @field metadata` | The model declares no fields | +| `MongoDB model declares multiple keys` | More than one `@key()` | +| `MongoDB model requires @key() or an id field` | No key | +| `MongoDB model is missing _id` | A stored document has no `_id` | +| `MongoDB Guid requires standard UUID binary` | A `Guid` field holds a string, or legacy UUID binary (subtype 3) | +| `MongoDB number cannot be represented exactly as a JavaScript number` | A lossy `Int64` or `Decimal128` | +| `MongoDB number must be finite` | A `Number` field holds something else | +| `Unknown MongoDB derived type: ` | A `_derivedTypeId` with no registered class | +| `Unsupported MongoDB model type: ` | A field type the codec does not map, such as a class without `@field` declarations | + +The first three fail when the collection is first resolved. The others fail the read, and the query answers with an error rather than a half-built model. + +A `Guid` written by an older .NET driver configuration as legacy UUID binary is the most common of these. Migrate such documents to standard UUID representation before reading them here. + +## Write through the driver + +The collection has no insert or update methods of its own. Write through `items.native`, the driver collection, and encode with `items.codec` so the document matches what reads expect: + +```typescript +import type { Document, Filter } from 'mongodb'; + +await items.native.insertOne(items.codec.serialize(task)); +await items.native.replaceOne({ _id: items.codec.id(task.id) } as Filter, items.codec.serialize(task), { upsert: true }); +``` + +`codec.id` returns `unknown`, so a filter built with it needs the `Filter` cast the package's own code uses. + +`serialize` refuses an instance without a key value, so the driver never invents an `ObjectId` your model cannot read back. A plain object with the model's fields is encoded like an instance. + +Reading through `native` returns raw driver documents. Pass them to `items.codec.deserialize(document)` to get a model instance. + +## Existing documents in another shape + +`ignoreConventions: true` bypasses the codec for collections that already store driver-native documents. Reads assign the raw document's properties to a new instance, writes store the instance's own properties, and identities pass through unchanged. In that mode you own field names and conversion, and a model does not need a key. ## Related From e86fdf99f56f18f9280843e0bbfafe3c7c7b3d6e Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:58 +0200 Subject: [PATCH 3/6] Merge the SQL stub pages and build out the remaining ones Fold read-only access, migrations, and observation limits into the overview and getting-started pages, and add complete code for setup, writes, paging, column codecs, and per-tenant databases. --- Documentation/sql/column-types.md | 28 +++++- Documentation/sql/getting-started.md | 136 +++++++++++++++++++++++++-- Documentation/sql/index.md | 34 ++++++- Documentation/sql/migrations.md | 10 -- Documentation/sql/observing.md | 8 -- Documentation/sql/paging.md | 57 +++++++++-- Documentation/sql/read-only.md | 10 -- Documentation/sql/tenancy.md | 69 +++++++++++++- Documentation/sql/toc.yml | 10 +- 9 files changed, 301 insertions(+), 61 deletions(-) delete mode 100644 Documentation/sql/migrations.md delete mode 100644 Documentation/sql/observing.md delete mode 100644 Documentation/sql/read-only.md diff --git a/Documentation/sql/column-types.md b/Documentation/sql/column-types.md index 20528e09..c7155341 100644 --- a/Documentation/sql/column-types.md +++ b/Documentation/sql/column-types.md @@ -3,7 +3,29 @@ title: SQL column types and conversions description: Declare column codecs for concepts, GUIDs, dates, times, durations, and JSON per SQL dialect, and know where they differ from Arc on .NET. --- -Declare conversions in your Drizzle schema and Fundamentals `@field` metadata on the Arc read model. `pgColumn(codec)`, `mysqlColumn(codec)`, and `sqliteColumn(codec)` wrap a codec in that dialect's Drizzle `customType`. They preserve typed writes and map the driver value back on reads, including provider-paged Arc reads. The SQL read-model codec also checks every declared Arc field has a table column and reconstructs GUIDs, concepts, and temporal values from plain driver columns if a custom column has not already done so. Writes still require an appropriate Drizzle column converter. +A task's title is a `TaskTitle` concept in your code and plain text in the table. Drizzle columns know the driver's types, not Fundamentals concepts, GUIDs, or `DateOnly`. The column codecs close that gap: declare the conversion once on the table column, and both writes and Arc reads get your types back. + +## Declare a column with a codec + +```typescript +import { ConceptAs } from '@cratis/fundamentals'; +import { pgTable } from 'drizzle-orm/pg-core'; +import { conceptCodec, dateOnlyCodec, guidCodec, pgColumn } from '@cratis/arc.drizzle'; + +export class TaskTitle extends ConceptAs { static readonly valueType = String; } + +export const tasks = pgTable('tasks', { + id: pgColumn(guidCodec('postgresql'))('id').primaryKey(), + title: pgColumn(conceptCodec(TaskTitle, 'string', 'postgresql'))('title').notNull(), + due: pgColumn(dateOnlyCodec)('due') +}); +``` + +`pgColumn(codec)`, `mysqlColumn(codec)`, and `sqliteColumn(codec)` wrap a codec in that dialect's Drizzle `customType`. The codec declares the SQL type, converts values on writes, and converts driver values back on reads, including the reads `queryPage` makes. The matching Arc read model declares `@field(Guid) id`, `@field(TaskTitle) title`, and `@field(DateOnly) due`. + +The read-model codec also checks that every declared Arc field has a table column, and rebuilds GUIDs, concepts, and dates from plain driver columns when a column has no custom codec. Writes through a plain column still need the value in the driver's type, so prefer a codec column wherever a model field is not a plain string, number, or boolean. + +## Codecs and SQL types | Codec | PostgreSQL | MySQL | SQLite | | --- | --- | --- | --- | @@ -15,6 +37,8 @@ Declare conversions in your Drizzle schema and Fundamentals `@field` metadata on | `timeSpanCodec` | `text` | `text` | `text` | | `jsonCodec(dialect, validate)` | `jsonb` | `json` | `text` | +## Rules and differences from .NET + Pass the concrete Fundamentals `ConceptAs` subclass and its underlying kind to `conceptCodec`. Because generic types are erased at runtime, declare `static readonly valueType = String`, `Number`, or `Guid` on your concept class; registration rejects an absent or mismatched type marker. Numbers must be finite. For indexed or primary-key MySQL string concepts, pass a fourth argument such as `conceptCodec(TaskName, 'string', 'mysql', 120)` to select `varchar(120)` rather than unindexed `text`. GUIDs use canonical strings in MySQL and SQLite, not binary(16); PostgreSQL uses native UUID. Unlike the .NET SQLite `GuidColumn` migration helper (`BLOB`) paired with a string value converter, this schema and converter both use text. **Do not reuse a .NET SQLite BLOB schema without a data migration.** The MySQL mapping is not yet backed by a live provider test. -`jsonCodec` requires a function that validates untrusted stored data; its decoder parses string values, then calls that function. It does not install EF's reflection-based JSON converter set. `TimeSpan` is serialized as Fundamentals text rather than a native interval, and fractional precision follows Fundamentals' `toString`/`parse`. Text ordering is lexicographic, not duration ordering for negative or multi-day spans: do not sort by this column unless you store a separate numeric duration for ordering. Schema changes and nullability remain your responsibility. See [migrations](migrations.md). +`jsonCodec` requires a function that validates untrusted stored data; its decoder parses string values, then calls that function. It does not install EF's reflection-based JSON converter set. `TimeSpan` is serialized as Fundamentals text rather than a native interval, and fractional precision follows Fundamentals' `toString`/`parse`. Text ordering is lexicographic, not duration ordering for negative or multi-day spans: do not sort by this column unless you store a separate numeric duration for ordering. Schema changes and nullability remain your responsibility. See [Own the schema](getting-started.md#own-the-schema). diff --git a/Documentation/sql/getting-started.md b/Documentation/sql/getting-started.md index 1fab8f05..07170068 100644 --- a/Documentation/sql/getting-started.md +++ b/Documentation/sql/getting-started.md @@ -1,22 +1,142 @@ --- title: Get started with SQL -description: Register a Drizzle database and table with withDrizzle, serve a paged model-bound query, and verify it through Express, Fastify, and Hono. +description: Declare a Drizzle table and an Arc read model, serve a paged query with withDrizzle, write from a command, and keep queries on the read-only handle. --- -For a first SQL-backed Arc query, install `drizzle-orm` and a Drizzle-supported database driver alongside `@cratis/arc.core` and `@cratis/arc.drizzle` from this source workspace. The linked SQLite specs use `sql.js` (WebAssembly, no native build); PostgreSQL integration checks use `pg` and `postgres`. Packages are not published to npm. Run your schema migration before serving requests; `withDrizzle` never creates tables. +This page serves a SQL table through an Arc query. It uses SQLite through `sql.js`, which runs in WebAssembly and needs no native build, so you can follow it on any machine. The code follows the package's [SQLite fixture](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/given/a_sqlite_database.ts). PostgreSQL works the same way with its own Drizzle driver. -The executable SQLite spec defines a [`tasks` table and connection](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/given/a_sqlite_database.ts), a [`TaskRecord`](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/given/TaskRecord.ts) with Fundamentals field metadata and a [model-bound query](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/given/TaskQueries.ts). The host [registers them with `withDrizzle`](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/when_serving_a_sqlite_page/with_each_http_adapter.ts), and the spec calls the generated `/page` route through Express, Fastify, and Hono. Run `yarn vitest run --project @cratis/arc.drizzle`; each adapter returns one sorted task and `paging.totalItems: 2`, while an unknown sort field returns HTTP 400. +Install `drizzle-orm` 0.45 and a Drizzle driver, here `sql.js`, next to `@cratis/arc.core` and `@cratis/arc.drizzle` from this source workspace. The Arc packages are not published to npm. -Resolve the tenant in your Arc host before making SQL queries. For a single-tenant example, construct the builder with `ArcApplication.createBuilder({ tenancy: { resolve: () => 'default' } })`; without a tenant, SQL access fails with “A tenant is required for Drizzle access.” Registration then follows this shape: +## Declare the table and the model -```typescript +The Drizzle table describes the storage. The Arc read model describes what a query returns. + +```typescript title="Tasks.ts" +import { sqliteTable, text } from 'drizzle-orm/sqlite-core'; +import { field, Guid } from '@cratis/fundamentals'; +import { key } from '@cratis/arc.core'; +import { guidCodec, sqliteColumn } from '@cratis/arc.drizzle'; + +export const tasks = sqliteTable('tasks', { + id: sqliteColumn(guidCodec('sqlite'))('id').primaryKey(), + title: text('title').notNull() +}); + +export class TaskRecord { + @field(Guid) @key() id!: Guid; + @field(String) title!: string; +} +``` + +Every `@field` on the model needs a column with the same property name, and the table needs a primary-key column; registration fails otherwise. `sqliteColumn(guidCodec('sqlite'))` stores the `Guid` as text and reads it back as a `Guid`. [Column types](column-types.md) lists the other codecs. + +## Serve a query + +```typescript title="TaskQueries.ts" +import { query, queryOptions, readModel, service, type QueryOptions } from '@cratis/arc.core'; +import { drizzleReadModel, type DrizzleReadModels } from '@cratis/arc.drizzle'; +import { TaskRecord } from './Tasks.js'; + +@readModel() +export class TaskQueries { + @query(service(drizzleReadModel(TaskRecord)), queryOptions()) + static page(tasks: DrizzleReadModels, options: QueryOptions) { + return tasks.queryPage(undefined, options); + } +} +``` + +`drizzleReadModel(TaskRecord)` is a service token for a read-only handle on the current tenant's database. `queryPage` pushes the count, sort, limit, and offset into SQL; see [Paging and sorting](paging.md). + +## Register the database + +```typescript title="main.ts" +import initSqlJs from 'sql.js'; +import { drizzle } from 'drizzle-orm/sql-js'; +import { ArcApplication } from '@cratis/arc.core'; +import '@cratis/arc.drizzle'; +import { TaskRecord, tasks } from './Tasks.js'; +import { TaskQueries } from './TaskQueries.js'; + +const SQL = await initSqlJs(); +const native = new SQL.Database(); +native.run('create table tasks (id text primary key, title text not null)'); +const database = drizzle(native); + +const builder = ArcApplication.createBuilder({ tenancy: { resolve: () => 'default' } }); builder.add(TaskQueries).withDrizzle({ dialect: 'sqlite', - database: db, + database, readModels: [{ type: TaskRecord, table: tasks }] }); +const app = await builder.build(); +await app.run(); ``` -Here `db` is your Drizzle database and `tasks` is its declared table. This fragment belongs in an existing Arc application builder; follow the linked spec for imports, connection creation and serving requests. The exported `withDrizzle(builder, options)` function is equivalent. The single `database` option accepts **only** the `default` tenant; configure `databaseFactory(tenant, context)` before serving other tenants. Arc scopes the handle but does not close your pool or connection. Close it after disposing the application. +Importing `@cratis/arc.drizzle` adds `withDrizzle` to the builder; the exported `withDrizzle(builder, options)` function is equivalent. A GET on the `page` query's route with `pageSize=10&sortBy=title` answers with up to ten tasks sorted by title, and `paging.totalItems` counted in SQL. An unknown sort field answers 400. + +`tenancy.resolve` makes every request use the `default` tenant, the only tenant the single `database` option serves. Without a tenant, SQL access fails with `A tenant is required for Drizzle access`. For more than one tenant, see [Tenancy](tenancy.md). + +The `create table` statement stands in for a migration so the example is self-contained. `withDrizzle` never creates or changes tables; see [Own the schema](#own-the-schema). + +## Write from a command + +A command that writes takes the writable database handle: + +```typescript title="AddTask.ts" +import type { SQLJsDatabase } from 'drizzle-orm/sql-js'; +import { field, Guid } from '@cratis/fundamentals'; +import { command, inject, key } from '@cratis/arc.core'; +import { drizzleDatabase, type DrizzleHandle } from '@cratis/arc.drizzle'; +import { tasks } from './Tasks.js'; + +@command() +export class AddTask { + @field(Guid) @key() id!: Guid; + @field(String) title!: string; + + @inject(drizzleDatabase()) + handle(database: DrizzleHandle): void { + database.native.insert(tasks).values({ id: this.id, title: this.title }).run(); + } +} +``` + +`drizzleDatabase()` resolves to a `DrizzleHandle` whose `native` property is the tenant's Drizzle database, typed as you declare it. Register `AddTask` with `builder.add(...)` like the query. + +## Keep queries read-only + +Queries take `drizzleReadModel(Model)`, commands take `drizzleDatabase()`. The read handle, `DrizzleReadModels`, exposes: + +| Member | Returns | +| --- | --- | +| `queryPage(filter, options)` | One page with the total, counted and cut in SQL | +| `find(filter, sorting?)` | Every match, or throws when there are more than `maxPageSize` | +| `findOne(filter)` | The first match in primary-key order, or `undefined` | +| `table` | The Drizzle table | + +It has no write methods and does not expose the writable database. A filter is a Drizzle `SQL` expression such as `eq(tasks.title, 'a')`, built with bound parameters; never interpolate request input into SQL text. + +This is an API boundary, **not** a database permission. Any code can still inject the writable token, and JavaScript can reach past TypeScript visibility. When queries must not be able to write, give them a connection with read-only database credentials. + +None of these methods accept a cancellation signal, and Arc does not pass the request's signal to the driver. Set timeouts in the driver or the database. + +## Own the schema + +Your application owns the schema and its migrations; Arc has no migration engine. Use [drizzle-kit](https://orm.drizzle.team/docs/drizzle-kit-overview) to generate SQL migrations from the table declarations, review the generated SQL, and apply it in your deployment **before** Arc starts serving requests. + +- A new non-nullable column on a populated table needs a default or a staged backfill. +- With a database per tenant, migrate **every** tenant's database. A migration that succeeded on one tenant proves nothing about the others. +- Do not run schema changes from `databaseFactory`; it runs on requests. + +There is no TypeScript counterpart of .NET's `AddStringColumn` and `AddJsonColumn` EF migration helpers. Declare column types with the [column codecs](column-types.md) and let drizzle-kit generate the SQL. + +## Own the connection + +Arc wraps the database you register in a scoped handle for each request, and never closes it. Close the connection or pool yourself, after `await app.dispose()`. + +## Related -Commands can inject `service(drizzleDatabase())` and access the scoped handle's `.native`; query methods should use `service(drizzleReadModel(TaskRecord))` to avoid accidentally writing from a read model. The Drizzle integration does not register a [command read-model resolver](../commands/command-context.md#load-a-read-model-by-key), so `commandReadModel(TaskRecord)` does not resolve SQL models by command key. A command may explicitly inject `drizzleReadModel(TaskRecord)` for a read instead. [Tenant routing](tenancy.md) explains what you must own. +- [Column types](column-types.md) +- [Paging and sorting](paging.md) +- [Tenancy](tenancy.md) diff --git a/Documentation/sql/index.md b/Documentation/sql/index.md index a1b11603..f006c66f 100644 --- a/Documentation/sql/index.md +++ b/Documentation/sql/index.md @@ -1,12 +1,36 @@ --- title: SQL with Drizzle -description: Serve model-bound queries from application-owned Drizzle databases on SQLite and PostgreSQL, and know what the integration deliberately leaves to you. +description: Serve model-bound queries from application-owned Drizzle databases on SQLite and PostgreSQL, with SQL-side paging, column codecs, and tenant routing, and know what the integration deliberately leaves to you. --- -`@cratis/arc.drizzle` connects Arc queries to application-owned Drizzle databases. It supports SQLite and PostgreSQL with executable database checks; MySQL uses the same SQL query path but has **not** been exercised against a live MySQL server. This is a source preview, not a published npm package. +Your read models live in SQL tables. Every query needs the right database for the tenant, a page and a total count computed in SQL rather than in memory, sorting that a client cannot turn into SQL injection, and conversions for GUIDs, concepts, and dates. `@cratis/arc.drizzle` does that on top of [Drizzle](https://orm.drizzle.team), while your application keeps its schema, its migrations, and its connections. -Use [Get started](getting-started.md) to wire one SQLite database into an Arc read model. Then choose [column conversions](column-types.md), [tenant routing](tenancy.md), [read-only access](read-only.md), [paging and sorting](paging.md), or [migrations](migrations.md). [Observation](observing.md) describes the unsupported live-query boundary. +:::note[Source preview] +`@cratis/arc.drizzle` is not published to npm. SQLite and PostgreSQL are exercised against real databases. MySQL uses the same query path but has **not** been run against a live MySQL server. The [capability reference](../reference/capabilities.md#persistence-and-chronicle) has the status and the checks behind it. +::: -Drizzle's typed, SQL-first schema and query builders let Arc share one read path across PostgreSQL, MySQL, and SQLite while your application retains its SQL and migration ownership. Kysely is a capable typed query builder but does not supply the same table/column mapping used here; Prisma emphasizes its own schema, client generation and migration workflow; TypeORM centers on entities, decorators and unit-of-work patterns rather than this explicit, read-only handle. These are trade-offs, not claims that one ORM replaces another. +## What it provides -Drizzle 0.45.x is pre-1.0 (1.0 is in beta). The peer dependency `^0.45.0` accepts compatible 0.45.x releases, **not** 0.46.x or 1.0; test and update this integration before changing the range. There is no Chronicle requirement or EF Core change tracker. Arc's .NET EF integration also offers SQL Server, spatial Point/LineString/Polygon types, multiple DbContexts and `BaseDbContext` automatic concept conversion; none of those features are ported here. Only one Drizzle database token can be registered per application; adding another `withDrizzle` registration fails at build with a duplicate service. Multiple tenants instead use `databaseFactory` to select one database per tenant. +| Capability | Page | +| --- | --- | +| Register a database and read models with `withDrizzle`, serve a query, write from a command, and keep queries on a read-only handle | [Get started](getting-started.md) | +| Store GUIDs, concepts, dates, times, durations, and JSON per dialect | [Column types](column-types.md) | +| Count, sort, and page in SQL | [Paging and sorting](paging.md) | +| Route each tenant to its own database | [Tenancy](tenancy.md) | + +## Why Drizzle + +Drizzle's typed, SQL-first table declarations and query builders let Arc share one read path across PostgreSQL, MySQL, and SQLite, while your application keeps ownership of its SQL and its migrations. Kysely is a capable typed query builder but does not supply the table and column mapping used here. Prisma centers on its own schema, client generation, and migration workflow. TypeORM centers on entities, decorators, and a unit of work rather than an explicit read-only handle. These are trade-offs, not claims that one ORM replaces another. + +Drizzle 0.45 is before 1.0. The peer dependency `^0.45.0` accepts 0.45 releases, **not** 0.46 or 1.0; the integration has to be tested and updated before that range changes. + +## What it does not do + +- **No schema management.** `withDrizzle` never creates tables, adds columns, or runs migrations. See [Own the schema](getting-started.md#own-the-schema). +- **No live queries.** There is no `observe()`, and Arc does not refresh an observable query when a table changes. SQLite has no cross-process change notification here, and Drizzle does not announce writes. PostgreSQL `LISTEN`/`NOTIFY` would need managed triggers, a listener connection per tenant, resubscription after reconnects, a race-free first read, and tested shutdown; none of that is included. If your application has a reliable, tenant-scoped change source of its own, an Arc [observable query](../queries/observable-queries.md) can consume it. An in-process event after a command write does not see changes made by other processes. +- **No command read models.** `commandReadModel(Type)` does not load SQL models by command key. A command can inject `drizzleReadModel(Type)` and read explicitly. +- **No transactions or change tracking.** There is no unit of work shared with command execution. Use a Drizzle transaction in your command when several writes must succeed together. +- **One registration per application.** A second `withDrizzle` fails at build with a duplicate service. Several tenants use one registration with `databaseFactory`. +- **No EF-only features.** Arc on .NET's Entity Framework integration also has SQL Server, spatial Point, LineString, and Polygon types, several DbContexts, and automatic concept conversion in `BaseDbContext`. None of those are part of this package. + +Start with [Get started](getting-started.md). diff --git a/Documentation/sql/migrations.md b/Documentation/sql/migrations.md deleted file mode 100644 index 84c3ef64..00000000 --- a/Documentation/sql/migrations.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: Migrate a Drizzle schema -description: Own your SQL schema with drizzle-kit migrations, run them before Arc starts, and migrate every tenant deliberately. ---- - -Own the schema in the application and use [drizzle-kit](https://orm.drizzle.team/docs/drizzle-kit-overview) to generate and apply SQL migrations. Run migrations **before** starting Arc or admitting tenant requests. `withDrizzle` does not discover migrations, create tables or add columns at runtime. - -For a new column, add it to your Drizzle table declaration, generate a migration with drizzle-kit, inspect the generated SQL, and apply it with your deployment's migration step. A non-nullable column on a populated table may need a default or a staged backfill. Review provider-specific SQL and test the result against existing data before deployment. There is no TypeScript counterpart to .NET's `AddStringColumn` / `AddJsonColumn` EF migration extensions; use the [column codecs](column-types.md) in the declared schema and drizzle-kit in the migration workflow instead of maintaining a second migration engine. - -With per-tenant databases or schemas, migrate **each authorized tenant target** deliberately; a successful migration on the default tenant does not prove the others are ready. Do not run uncontrolled schema changes in `databaseFactory`. diff --git a/Documentation/sql/observing.md b/Documentation/sql/observing.md deleted file mode 100644 index ef904583..00000000 --- a/Documentation/sql/observing.md +++ /dev/null @@ -1,8 +0,0 @@ ---- -title: SQL observation limits -description: Why the Drizzle integration does not provide live observation, and what an application-owned change source must handle instead. ---- - -This integration does **not** expose `observe()` or automatically refresh Arc observable queries when SQL changes. SQLite has no cross-process notification mechanism in this adapter; Drizzle query execution alone does not announce writes. PostgreSQL `LISTEN/NOTIFY` would require managed triggers, a dedicated listener connection per isolation boundary, resubscription on reconnect, a race-free initial snapshot and tested shutdown. No such producer or trigger installation is included. MySQL is also unverified for observation. - -If your application already has a reliable, tenant-scoped change source, Arc's core observable-query APIs can consume that application-owned source. Test initial-read races, external writes, failure and subscription disposal independently. An in-process event after a command write does not observe changes made by other processes. Do not treat this as parity with .NET EF's provider-dependent observation support. diff --git a/Documentation/sql/paging.md b/Documentation/sql/paging.md index 9be8bbd5..7d9d892f 100644 --- a/Documentation/sql/paging.md +++ b/Documentation/sql/paging.md @@ -1,14 +1,59 @@ --- title: Page and sort SQL read models -description: Push count, sort, limit, and offset into SQL with DrizzleReadModels.queryPage, and know the sort-field and consistency rules. +description: Push count, sort, limit, and offset into SQL with DrizzleReadModels.queryPage, filter with a typed predicate, and know the sort-field, size, and consistency rules. --- -Declare a Drizzle table with at least one column marked `.primaryKey()` and register it with `readModels`. A composite primary key declared only through Drizzle's table extras does not mark individual columns for this adapter's stable tie-breaker. Your model-bound query receives Arc's `queryOptions()` and calls `DrizzleReadModels.queryPage(filter, options)`. The method requires `options.paging`, with a nonnegative safe page index and a positive page size no larger than `maxPageSize` (100 by default, maximum configurable value 10,000). The filter is an optional application-built Drizzle `SQL` expression. +A task table grows to a million rows, and the client shows ten at a time. Loading every row to cut a page in memory stops working long before that. `DrizzleReadModels.queryPage` sends the count, the sort, and the page to the database and returns an Arc page with the real total. -Arc pushes `count(*)`, ordering, `limit` and `offset` to the selected tenant database. The count is calculated before the page, so `totalItems` is not the length of the page. Requested `sorting.field` must be both a declared Arc `@field` and a property returned by `getTableColumns(table)`; only those fields and the primary key are selected. A private table column cannot be used to infer its order; an unknown field or unsupported direction fails before SQL runs and the Arc HTTP pipeline maps unknown fields to 400. All queries sort by the primary key as a stable tie-breaker when needed. The query never interpolates the client field as SQL text. +## Page a query -`find(filter, sorting?)` returns all matches only when the count is at most `maxPageSize`; otherwise it throws. `findOne(filter)` returns the first match ordered by primary key, or `undefined`. Both use Drizzle's mapped selection and avoid loading an unbounded result. +```typescript +import { eq } from 'drizzle-orm'; +import { query, queryOptions, readModel, service, type QueryOptions } from '@cratis/arc.core'; +import { drizzleReadModel, type DrizzleReadModels } from '@cratis/arc.drizzle'; +import { TaskRecord, tasks } from './Tasks.js'; -Count and page are separate statements. A concurrent write between them can change membership: this adapter does not promise snapshot isolation or an EF transaction. If the page length no longer agrees with the count, Arc may reject the result as malformed rather than return an inconsistent page. Use an application-managed transaction when that consistency matters; there is no automatic transaction shared with command execution. Offset paging may become expensive for deep pages, so index the filtered and sorted fields and set an appropriate maximum. +@readModel() +export class TaskQueries { + @query(service(drizzleReadModel(TaskRecord)), queryOptions()) + static page(items: DrizzleReadModels, options: QueryOptions) { + return items.queryPage(undefined, options); + } -The [adapter HTTP spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/when_serving_a_sqlite_page/with_each_http_adapter.ts) exercises sorting, count and field rejection through all three hosts. + @query(service(drizzleReadModel(TaskRecord)), queryOptions()) + static open(items: DrizzleReadModels, options: QueryOptions) { + return items.queryPage(eq(tasks.title, 'open'), options); + } +} +``` + +`TaskRecord` and `tasks` are the model and table from [Get started](getting-started.md). A GET with `page=0&pageSize=10&sortBy=title` runs a `count(*)` and a `select ... order by ... limit 10 offset 0` in the tenant's database, and answers with ten tasks and `paging.totalItems` from the count. The second query filters with a typed Drizzle predicate first; a filter is optional, and `undefined` means every row. + +Build filters from trusted values with Drizzle's operators, which bind parameters. Never interpolate request text into SQL. + +## Rules + +| Rule | Detail | +| --- | --- | +| Paging is required | `queryPage` throws without `options.paging`. Use `find` for an unpaged read. | +| Page size | At least 1, at most `maxPageSize`: 100 by default, configurable up to 10,000 with `withDrizzle({ maxPageSize })` | +| Page index | A nonnegative safe integer | +| Sort field | Must be a declared `@field` on the model **and** a column of the table. An unknown field, or a table column the model does not declare, answers 400 before any SQL runs | +| Sort direction | `asc` or `desc` | +| Tie-breaker | The primary key, ascending, so pages stay stable when sort values repeat | +| Selected columns | Only the model's declared fields and the primary key; other columns are never read | + +The client's sort field is matched against the declared fields and never becomes SQL text. A table needs at least one column marked `.primaryKey()`. A composite key declared only through Drizzle's table extras does not mark individual columns, so the adapter cannot use it as a tie-breaker, and registration fails. + +## Unpaged reads + +`find(filter, sorting?)` returns every match only when there are at most `maxPageSize` of them, and throws `Drizzle find exceeds maxPageSize` otherwise, instead of silently truncating. `findOne(filter)` returns the first match in primary-key order, or `undefined`. + +## Consistency + +The count and the page are separate statements. A write between them can change which rows are counted or returned; the adapter does not promise snapshot isolation. When the page no longer agrees with the count, Arc may reject the result as malformed instead of returning an inconsistent page. Offset paging gets slower for deep pages, so index the filtered and sorted columns and keep `maxPageSize` sensible. + +## Related + +- [Paging and sorting](../queries/model-bound/paging.md) for the request parameters +- [Get started with SQL](getting-started.md) diff --git a/Documentation/sql/read-only.md b/Documentation/sql/read-only.md deleted file mode 100644 index faf7f692..00000000 --- a/Documentation/sql/read-only.md +++ /dev/null @@ -1,10 +0,0 @@ ---- -title: Keep SQL queries read-only -description: Use the read-only DrizzleReadModels handle in queries, inject the writable database only in commands, and know that this is an API boundary, not a permission. ---- - -Inject `service(drizzleReadModel(Model))` into a read-model query. The scoped `DrizzleReadModels` handle exposes `queryPage(filter, options)`, `find(filter, sorting?)`, `findOne(filter)` and its table; `find` rejects more than `maxPageSize` matches instead of silently truncating results, and `findOne` returns the first primary-key-ordered match or `undefined`. None of these APIs accept a query cancellation signal or propagate Arc's `context.signal` to the driver; timeouts and cancellation must be managed by the application or database driver. The handle does not expose a writable Drizzle database. Use a typed Drizzle `SQL` predicate built with bound parameters, not string-interpolated request input. If a command needs to write, inject `service(drizzleDatabase())` and use the returned `.native` database. - -This is an API boundary, **not** a database permission boundary. It cannot stop a caller from injecting the writable token elsewhere, and JavaScript can bypass TypeScript visibility. Provision read-only database credentials for queries if writes must be prohibited. Unlike .NET's `ReadOnlyDbContext`, there is no EF tracking or SaveChanges interceptor to turn off; Arc has no unit of work or automatic SQL transaction here. - -The registered pool is application-owned. Disposing a query scope releases its Arc handle, not its underlying connection or pool. Close that pool explicitly at application shutdown after `await app.dispose()`. diff --git a/Documentation/sql/tenancy.md b/Documentation/sql/tenancy.md index 90cf3f4c..c4a4c8e2 100644 --- a/Documentation/sql/tenancy.md +++ b/Documentation/sql/tenancy.md @@ -1,10 +1,71 @@ --- title: Route SQL by tenant -description: Route SQL access to a database per tenant with databaseFactory, and own the pools, credentials, and isolation checks. +description: Give each tenant its own SQL database with databaseFactory, keep single-tenant applications on the default tenant, and own the pools, credentials, and isolation checks. --- -Use `databaseFactory(tenant, context)` when tenant data must be isolated. Arc resolves it in each execution scope after its normal tenant resolution, lowercases the tenant ID, and rejects a missing tenant or an empty resolver result. The factory may return a connection or a pooled Drizzle database and may reuse one per tenant; **the application owns creation, cache limits, credentials and shutdown**. Arc never disposes a returned database. Avoid creating an unbounded pool for every untrusted tenant ID. +Two customers share your service, and neither may ever see the other's tasks. The safest line between them is a database each. With `databaseFactory`, every request's tenant selects the database its queries and commands use, and no query can ask for another tenant's data. -`database` is the short path for a single default-tenant database; requesting another tenant fails closed. For a shared database with tenant-specific schemas, return a pool/connection already confined to that tenant, such as one with a pinned schema search path. Never change a shared pooled connection's search path per request without a transaction and guaranteed reset. The adapter does not prepend a tenant predicate or validate your factory's tenant-to-database mapping. Verify cross-tenant reads and writes under your actual pool and authorization configuration. +## A database per tenant -.NET EF pooled contexts do not automatically infer per-tenant databases. This integration deliberately makes resolution explicit, similar to MongoDB's tenant-aware factory, rather than silently reusing one pool across tenants. See the [SQLite tenant-isolation spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/when_paging_across_tenants/with_sqlite.ts). +```typescript +import { drizzle, type NodePgDatabase } from 'drizzle-orm/node-postgres'; +import { Pool } from 'pg'; +import '@cratis/arc.drizzle'; +import { TaskRecord, tasks } from './Tasks.js'; + +const databases = new Map(); +const allowedTenants = new Set(['acme', 'globex']); + +builder.withDrizzle({ + dialect: 'postgresql', + databaseFactory: tenant => { + if (!allowedTenants.has(tenant)) throw new Error(`Unknown tenant ${tenant}`); + let database = databases.get(tenant); + if (!database) { + database = drizzle(new Pool({ connectionString: `postgres://localhost/tasks_${tenant}` })); + databases.set(tenant, database); + } + return database; + }, + readModels: [{ type: TaskRecord, table: tasks }] +}); +``` + +This excerpt assumes an Arc `builder` and a PostgreSQL `tasks` table declared with `pgTable`, in the shape [Get started](getting-started.md) shows for SQLite. Arc calls `databaseFactory` once per execution scope, after it has resolved the tenant, with the tenant ID in lowercase. A missing tenant, or a factory that returns nothing, fails the request. + +## What you own + +Arc selects; your factory decides. The application owns: + +- **Creation and caching.** Reuse one pool per tenant, as above. A factory that creates a pool for any tenant ID a request names lets a caller exhaust your connections. +- **The mapping.** Arc does not check that the database you return belongs to the tenant, and does not add a tenant predicate to any query. +- **Credentials.** Give each tenant's pool credentials that reach only that tenant's database. +- **Shutdown.** Arc never closes a database the factory returns. Close your pools after `await app.dispose()`. +- **Migrations.** Migrate every tenant's database before serving it; see [Own the schema](getting-started.md#own-the-schema). + +Choosing the tenant's database does not prove the caller belongs to that tenant. Configure a membership check or derive the tenant from the principal; see [Tenancy](../tenancy/index.md). + +## A schema per tenant + +For one database with a schema per tenant, return a pool or connection that is already confined to that tenant's schema, such as one with a fixed search path. Never change the search path of a shared pooled connection per request unless a transaction guarantees it is reset. + +## Single-tenant applications + +The `database` option is the short path for one database. It serves only the `default` tenant, and a request for any other tenant fails closed with `Drizzle database is only available for the default tenant`. Resolve every request to that tenant: + +```typescript +const builder = ArcApplication.createBuilder({ tenancy: { resolve: () => 'default' } }); +``` + +Set exactly one of `database` and `databaseFactory`; both, or neither, fails `withDrizzle`. + +## Verify the boundary + +Test cross-tenant reads and writes with your real pool configuration and authorization: read as one tenant, write as another, and check that nothing crosses. The package's [SQLite tenant-isolation spec](https://github.com/Cratis/Arc.TypeScript/blob/main/Source/Drizzle/for_DrizzleReadModels/when_paging_across_tenants/with_sqlite.ts) shows the shape of such a check. + +.NET EF pooled contexts do not infer a database per tenant either. This integration makes the choice explicit, like the MongoDB integration's per-tenant resolvers, rather than reusing one pool across tenants. + +## Related + +- [Tenancy](../tenancy/index.md) +- [Get started with SQL](getting-started.md) diff --git a/Documentation/sql/toc.yml b/Documentation/sql/toc.yml index 179625ee..b0804fee 100644 --- a/Documentation/sql/toc.yml +++ b/Documentation/sql/toc.yml @@ -4,13 +4,7 @@ href: getting-started.md - name: Column types and conversions href: column-types.md -- name: Tenancy - href: tenancy.md -- name: Read-only queries - href: read-only.md - name: Paging and sorting href: paging.md -- name: Migrations - href: migrations.md -- name: Observation limits - href: observing.md +- name: Tenancy + href: tenancy.md From 021825ecfe0764ad6ec6e5d3cea1c98f203593fa Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:58 +0200 Subject: [PATCH 4/6] Move repository scripts out of the vertical slices page Add a contributor page for regenerating and checking the samples, and correct the release page's statement about private packages. --- Documentation/contributing/releases.md | 2 +- Documentation/contributing/samples.md | 29 ++++++++++++++++++++++++++ Documentation/contributing/toc.yml | 2 ++ Documentation/vertical-slices.md | 6 ++++-- 4 files changed, 36 insertions(+), 3 deletions(-) create mode 100644 Documentation/contributing/samples.md diff --git a/Documentation/contributing/releases.md b/Documentation/contributing/releases.md index 375f621d..18a5225b 100644 --- a/Documentation/contributing/releases.md +++ b/Documentation/contributing/releases.md @@ -66,6 +66,6 @@ The workflow is **manual only**, not triggered by pushes or pull requests. The p ## Before enabling publication -Review the package inventory and versioning scheme, including whether any packages stay private (the experimental Chronicle integration has not been `private` since v0.12.0), verify every published manifest against the planned version, configure npm trusted publishing and a complete post-publish verification gate, and review failure behavior before granting write permissions or adding any release effect. A release must fail rather than report success after a partial publish. Neither this page nor the preview authorizes a publication workflow. +Review the package inventory and versioning scheme, including whether any packages should stay private (none of the eleven packages under `Source` is marked `private`, including the experimental `@cratis/arc.chronicle` and `@cratis/cratis`; only the samples and contract-test fixtures are), verify every published manifest against the planned version, configure npm trusted publishing and a complete post-publish verification gate, and review failure behavior before granting write permissions or adding any release effect. A release must fail rather than report success after a partial publish. Neither this page nor the preview authorizes a publication workflow. **Major releases are never automatic.** They require a human merge and may proceed only after Arc parity has been verified. A minor/patch preview is not a shortcut around those requirements. Merge pull requests with a true merge commit, not squash or rebase; merging and publishing are outside this workflow. diff --git a/Documentation/contributing/samples.md b/Documentation/contributing/samples.md new file mode 100644 index 00000000..f04883db --- /dev/null +++ b/Documentation/contributing/samples.md @@ -0,0 +1,29 @@ +--- +title: Change a sample +description: Regenerate proxies and metadata, lint, and check client generation after you change the Tasks or Library sample in this repository. +--- + +The Tasks and Library samples in this repository are checked by the same gate as the packages. Both samples commit their generated metadata, and Library also commits its browser proxies, so a change to a decorated artifact has to be followed by a regeneration, or `yarn ci` fails. This page is for contributors to this repository; applications have their own scripts. + +## After you change a slice + +Run these from the repository root. The `generate-proxies` scripts run the proxy generator from its `dist` folder, so run `yarn build` first. + +| Command | What it does | +| --- | --- | +| `yarn workspace @cratis/arc.core.sample.tasks generate-proxies` | Regenerates `Samples/Tasks/Features/generatedMetadata.ts`, and compiles the Tasks proxies into `dist/proxies` | +| `yarn workspace @cratis/arc.sample.library generate-proxies` | Regenerates `Samples/Library/Features/generatedMetadata.ts` and the proxies in `Samples/Library/Web/src/generated` | +| `yarn check:metadata` | Fails when either sample's committed metadata differs from its source | +| `yarn lint:tasks:arc` | Runs the Arc ESLint rules over `Samples/Tasks/Features` with type information | +| `yarn test:client-generation` | Builds, regenerates the Tasks proxies, compiles the client fixtures, and runs the generated proxies against Express, Fastify, and Hono | + +Commit the regenerated files with the source change. Do not hand-edit generated metadata or browser proxies. + +## Before you open a pull request + +`yarn ci` runs all of the above with the rest of the gate. [Contributing](https://github.com/Cratis/Arc.TypeScript/blob/main/CONTRIBUTING.md) lists every step, and [Preview a TypeScript release](releases.md) covers release checks. + +## Related + +- [Keep a behavior together in a vertical slice](../vertical-slices.md) +- [Proxy generation](../proxy-generation/index.md) diff --git a/Documentation/contributing/toc.yml b/Documentation/contributing/toc.yml index 767bc440..4e9dcec7 100644 --- a/Documentation/contributing/toc.yml +++ b/Documentation/contributing/toc.yml @@ -1,2 +1,4 @@ +- name: Change a sample + href: samples.md - name: Preview a release href: releases.md diff --git a/Documentation/vertical-slices.md b/Documentation/vertical-slices.md index c5a59638..ab2fac5f 100644 --- a/Documentation/vertical-slices.md +++ b/Documentation/vertical-slices.md @@ -20,6 +20,8 @@ Features/Authors/ This is an **application convention**, not a requirement of the Arc runtime. The [C# Library registration slice](https://github.com/Cratis/Samples/blob/main/Library/Lending/Authors/Registration/Registration.cs) puts `RegisterAuthor`, `AuthorRegistered`, and `UniqueAuthorName` in `Registration.cs`; [Studio's registration slice](https://github.com/Cratis/Studio/blob/main/Source/Core/Projects/Registration/Registration.cs) also keeps its command, constraint, and event together. In a Chronicle TypeScript slice, place its event alongside the command, give it a constructor and `@field` declarations, and return an event instance from `handle()` as Library does. Arc without Chronicle needs no event; Tasks demonstrates that path. -Discover the **folder**, not a single class: `await builder.discover(new URL('./Features/', import.meta.url))` registers every exported, decorated artifact in each module. The source proxy generator reads all exported command and read-model declarations in a file and still writes one browser proxy **per operation**. It also records each exported validator in generated metadata, including validators beside their commands; the sample's `generate-proxies` command regenerates both outputs. `yarn lint:tasks:arc` checks the co-located artifacts with the Arc ESLint plugin. +Discover the **folder**, not a single class: `await builder.discover(new URL('./Features/', import.meta.url))` registers every exported, decorated artifact in each module. The [proxy generator](proxy-generation/index.md) reads every exported command and read-model declaration in a file and still writes one browser proxy **per operation**. It also records each exported validator in [generated metadata](proxy-generation/generated-artifact-metadata.md), including validators beside their commands. The [Arc ESLint rules](code-analysis/index.md) check co-located artifacts the same way as artifacts in separate files. -When you add a slice, run `yarn workspace @cratis/arc.core.sample.tasks generate-proxies`, `yarn check:metadata`, `yarn lint:tasks:arc`, and `yarn test:client-generation`. Do not hand-edit generated metadata or browser proxies. Framework implementations under `Source/` use one type per file instead; this layout is for application code and documentation examples. +After you add or change a slice, regenerate the proxies and metadata with `arc-proxygenerator`, and let your lint run include the slice folder. Never hand-edit generated metadata or browser proxies; change the decorated source and regenerate. + +Framework implementations under `Source/` use one type per file instead; this layout is for application code and documentation examples. Working on the samples in this repository has its own scripts, listed in [Change a sample](contributing/samples.md). From 705c0401a0ee4b13d763977cb71f5af9b3fe31ab Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:58 +0200 Subject: [PATCH 5/6] Make the capability reference the home of integration evidence - Move the Chronicle, MongoDB, SQL, and proxy generation check descriptions into the capability reference and fix the PostgreSQL spec path - Add rows for Chronicle compliance, reactor replay exclusion, and code analysis, and rename the Cratis composition row - Add a glossary and a diagnostics reference --- Documentation/reference/capabilities.md | 41 ++++++++++- Documentation/reference/diagnostics.md | 98 +++++++++++++++++++++++++ Documentation/reference/glossary.md | 67 +++++++++++++++++ Documentation/reference/index.md | 4 +- Documentation/reference/toc.yml | 4 + 5 files changed, 209 insertions(+), 5 deletions(-) create mode 100644 Documentation/reference/diagnostics.md create mode 100644 Documentation/reference/glossary.md diff --git a/Documentation/reference/capabilities.md b/Documentation/reference/capabilities.md index 2efc4cd2..f90c57b1 100644 --- a/Documentation/reference/capabilities.md +++ b/Documentation/reference/capabilities.md @@ -103,9 +103,12 @@ Evidence paths are relative to the repository root. Spec folders follow `for_` | A class passed to `add()` has no Arc or integration decorator | +| `Conflicting namespaces for ` | One class was registered under two different namespaces | +| `Duplicate validator target: ` | Two validators target one class | +| `Unbound constructor parameters on ` | A service's constructor parameters have no tokens | +| `Service requires an implementation` | A `serviceToken` was registered without a class or factory | +| `Missing service: ` | A declared dependency is not registered | +| `Service dependency cycle: ` | Services depend on each other in a loop | +| `Captive service dependency: ` | A singleton depends on a scoped or transient service | +| `Expected one read-model resolver for , found ` | A `commandReadModel(Type)` has no owning integration, or two; see [When read model resolution fails](../chronicle/read-models/failures.md) | +| `Multiple identity details providers found` | More than one `@identityDetailsProvider()` | +| `Import @cratis/arc. before calling with()` | An integration method was called without importing its package | +| `Chronicle requires eventStore and exactly one of connectionString or client` | Incomplete `withChronicle` options or configuration | +| `MongoDB requires exactly one of client, server, or serverResolver` | Incomplete `withMongoDB` options or configuration | +| `Drizzle requires exactly one of database or databaseFactory` | Incomplete `withDrizzle` options | +| `Invalid Cratis configuration: .` | An `appsettings.json` or `Cratis__...` value has the wrong type; the message names the key, never the value | + +The service messages are explained in [Dependency injection](../dependency-injection.md). + +## At request time: results and status codes + +A command or query result carries `validationResults`, and each result has a `reason`: + +| `reason` | Meaning | +| --- | --- | +| `rule` | A validator rule failed, or a required command read model is missing or has no key | +| `malformedRequest` | The input does not match the declared fields | +| `validatorFailed` | A validator threw; the error goes to `logger`, and the caller sees a generic message without the exception text | +| `dependencyUnavailable` | A service a validator or handler needs could not be resolved | +| `constraintViolation` | Chronicle rejected an append for a constraint; `reasonDetail` names it | +| `concurrencyViolation` | Chronicle rejected an append because the stream moved; `state` holds the revisions | + +The HTTP status follows the [HTTP contract reference](http-contract.md#status-codes): 400 for validation, 401 and 403 for authentication and authorization, 405 for an unsupported method, 408 and 503 for observable waits and limits, and 500 for exceptions. Outside development, a 500 carries `An unexpected error occurred` and no stack trace; the original error goes to the `logger` option. See [Configuration](../configuration/index.md#errors-and-logging). + +## On a running server + +| Surface | Shows | +| --- | --- | +| `GET /.cratis/commands`, `GET /.cratis/queries` | Every command and query with its route and input JSON Schema; see [Introspection](../introspection/index.md) | +| `GET /.cratis/identity-details/schema` | The identity details schema | +| `GET /openapi.json` | The OpenAPI 3.1 document; see [OpenAPI](../open-api/index.md) | +| `GET /.cratis/queries/health` | The authenticated caller's own observable hub connections, when `query.enableObservableHealth` is on; see [Query health](../queries/query-health.md) | +| OpenTelemetry | Spans such as `cratis.arc.command.execute` and the `cratis.arc.operation.duration` histogram from the `Cratis.Arc` source; see [Observability](../observability.md) | +| `logger(error, correlationId)` | Every failure with its correlation ID, and unknown configuration keys | + +Every response carries its correlation ID, in `X-Correlation-ID` unless you renamed the header. Search your logs and traces by it. + +For the Chronicle side of a running system, such as failed observer partitions, use the Chronicle Workbench or the `cratis` CLI against the same event store and tenant namespace. + +## Related + +- [Troubleshooting](../troubleshooting.md) +- [Capability reference](capabilities.md) +- [Glossary](glossary.md) diff --git a/Documentation/reference/glossary.md b/Documentation/reference/glossary.md new file mode 100644 index 00000000..fcaec6eb --- /dev/null +++ b/Documentation/reference/glossary.md @@ -0,0 +1,67 @@ +--- +title: Glossary +description: One line per Arc for TypeScript term, with the decorator or function behind it and the page that covers it in full. +--- + +Terms shared by every Arc implementation are defined once in the [Arc glossary](/arc/glossary/). This page adds the TypeScript names for them, and the terms only Arc for TypeScript has. + +## Commands + +- **Command**: a class decorated `@command()` whose instance `handle()` method carries out an intent. See [Model-bound commands](../commands/model-bound/index.md). +- **`handle()`**: the command's method that decides and returns a response, events, operations, or nothing. +- **`provide()`**: an optional command method that runs after validation, before `handle()`, to load data `handle()` needs; its return value is `handle()`'s first argument. +- **Command key**: the value that identifies what a command acts on, from `@key()`, `getKey()`, or with Chronicle, `getEventSourceId()`. See [Command context](../commands/command-context.md#give-a-command-a-key). +- **Command context**: the command, its resolved key, request identity, and values, available to handlers through `commandContext()`. See [Command context](../commands/command-context.md). +- **Outcome**: a value created by `response(...)`, `rejected(...)`, or `denied(...)` that ends a command with a response, a 400, or a 403. See [Command outcomes](../commands/command-outcomes.md). +- **`tuple(...)`**: a return value that carries several values from `handle()`, at most one of which becomes the response. +- **Response value handler**: a `CommandResponseValueHandler` that consumes a returned value on the server, such as a Chronicle event. See [Response value handlers](../commands/response-value-handlers.md). +- **Command operation**: a returned `CommandOperation` that Arc executes and, after a known failure, compensates. See [Command operations](../commands/operations/index.md). + +## Queries + +- **Read model**: a class decorated `@readModel()` whose static `@query(...)` methods are queries. See [Model-bound queries](../queries/model-bound/index.md). +- **Parameter descriptor**: `argument(name, Type)`, `service(token)`, or `queryOptions()` in `@query(...)`, telling Arc where each query parameter comes from. +- **Observable query**: a query declared `@query({ observable: true }, ...)` that returns an RxJS observable or an async iterable, served as a snapshot, server-sent events, or WebSocket frames. See [Observable queries](../queries/observable-queries.md). +- **`queryPage(items, total, sorting?)`**: the result a query returns when its data source has already cut the page. See [Paging and sorting](../queries/model-bound/paging.md). +- **Read-model interceptor**: a `@readModelInterceptor()` class that transforms each read-model instance before it is encoded. See [Intercept read models](../queries/read-model-interception.md). + +## Types and validation + +- **Field**: a property declared with Fundamentals `@field(Type)`. Only declared fields are bound, validated, encoded, stored, and generated. See [Concepts](../concepts.md). +- **Concept**: a class extending Fundamentals `ConceptAs` with `static readonly valueType`, wrapping one primitive domain value. See [Concepts](../concepts.md). +- **Validator**: a class extending `CommandValidator`, `QueryValidator`, `ConceptValidator`, or `ModelValidator`, decorated `@validator(Target)`, with rules declared by `ruleFor`. See [Command validation](../commands/command-validation.md). +- **Allowed severity**: the highest validation severity that does not block a command; over HTTP it is capped at Warning. See [Validation severity filtering](../commands/validation-severity-filtering.md). + +## Services and hosting + +- **Application builder**: the builder from `ArcApplication.createBuilder()` where you add or discover artifacts, register services, and attach integrations before `build()`. See [Dependency injection](../dependency-injection.md). +- **Discovery**: `builder.discover(folderUrl)`, which imports a folder and registers every exported, decorated artifact. +- **Service token**: a class, or a `serviceToken(name)`, that identifies a service in Arc's container. +- **Generated metadata**: the file `arc-proxygenerator` writes so that a bare `@query()` or `handle()` can infer its arguments and services. See [Generated artifact metadata](../proxy-generation/generated-artifact-metadata.md). +- **`ArcServer`**: the built pipeline that executes commands and queries; hosts and `executeCommand`, `performQuery`, and `openObservableQuery` use it. See [Calling commands from code](../commands/calling-commands-from-code.md). +- **Host adapter**: `cratisArc` from `@cratis/arc.express`, `@cratis/arc.fastify`, or `@cratis/arc.hono`, which serves an Arc application inside that framework. See [Host adapters](../hosts/index.md). +- **Execution context**: the tenant, principal, correlation ID, allowed severity, and cancellation signal of one request or direct call. +- **Authentication handler**: an ordered handler that turns a request into a principal, such as `jwtBearer()`. See [Authentication](../core/authentication.md). +- **Identity details provider**: an `@identityDetailsProvider()` class whose result `/.cratis/me` returns. See [Identity](../identity/index.md). + +## Client + +- **Proxy**: a generated TypeScript class for one command or query that the published `@cratis/arc` client executes. See [Proxy generation](../proxy-generation/index.md). +- **`arc-proxygenerator`**: the CLI that reads decorated TypeScript source and writes proxies and generated metadata. + +## Chronicle + +- **Returned event**: an `@eventType()` instance returned from `handle()`, which the integration appends. See [Returning events](../chronicle/commands/index.md). +- **Batch**: the returned events of an outer command and its nested commands, appended together after the outer command succeeds. See [Transactional commands](../chronicle/commands/transactional-commands.md). +- **Concurrency scope**: the expected tail of an event source, set by `{ concurrency: true }`, `eventsWithConcurrencyScopes`, or an aggregate. See [Concurrency](../chronicle/commands/concurrency.md). +- **Subject**: the compliance identity recorded on an appended event, from `getSubject()`, `@subject()`, `@eventSubject`, or the event source ID. See [Subject](../chronicle/commands/subject.md). +- **Aggregate root**: a class extending `AggregateRoot`, bound with `commandAggregate(Type)`, rehydrated from the command key's events. See [Aggregates](../chronicle/aggregates/index.md). +- **Command read model**: a read model loaded for the command's key with `commandReadModel(Type)` or `readModelForValidation(Type)`. See [Read models in commands](../chronicle/read-models/injecting-into-commands.md). +- **Reactor command**: an Arc command returned from a Chronicle reactor and executed through the command pipeline. See [Returning commands from a reactor](../chronicle/reactors/command-side-effects.md). + +## Storage + +- **Collection token**: `mongoCollection(Type)`, the service token for the current tenant's MongoDB collection. See [Get started with MongoDB](../mongodb/getting-started.md). +- **Naming policy**: the `MongoNamingPolicy` that decides stored property and collection names. See [Naming policies](../mongodb/naming-policies.md). +- **Read handle**: `drizzleReadModel(Type)`, a read-only SQL handle for queries; `drizzleDatabase()` is the writable counterpart for commands. See [Get started with SQL](../sql/getting-started.md). +- **Column codec**: a conversion such as `guidCodec` or `conceptCodec`, wrapped in `pgColumn`, `mysqlColumn`, or `sqliteColumn`. See [Column types](../sql/column-types.md). diff --git a/Documentation/reference/index.md b/Documentation/reference/index.md index f645fbdf..566fac9e 100644 --- a/Documentation/reference/index.md +++ b/Documentation/reference/index.md @@ -11,8 +11,10 @@ These pages are for looking things up. Narrative guides link here instead of res | [HTTP contract reference](http-contract.md) | Look up every route, method, header, and status code the server uses | | [Packages](packages.md) | See what each package exports and what it needs | | [Wire format](wire-format.md) | Check derived-type discriminators, naming, enums, and number encoding | +| [Diagnostics](diagnostics.md) | Find every lint rule, build error, validation reason, status code, and runtime surface that tells you why Arc behaves as it does | +| [Glossary](glossary.md) | Look up an Arc for TypeScript term and the decorator or function behind it | | [Configuration](../configuration/index.md) | Look up every `ArcOptions` setting | | [Decorator reference](../decorators.md) | Look up every decorator and parameter descriptor | | [Arc HTTP contract](/arc/http-contract/) | Read the language-neutral contract, maintained in the Arc repository | | [Arc capability matrix](/arc/capabilities/) | Compare Arc on .NET and Arc for Kotlin and Java | -| [Glossary](/arc/glossary/) | Look up shared Arc terms | +| [Arc glossary](/arc/glossary/) | Look up terms shared by every Arc implementation | diff --git a/Documentation/reference/toc.yml b/Documentation/reference/toc.yml index 82c9e4bc..b5984728 100644 --- a/Documentation/reference/toc.yml +++ b/Documentation/reference/toc.yml @@ -9,3 +9,7 @@ href: packages.md - name: Wire format href: wire-format.md + - name: Diagnostics + href: diagnostics.md + - name: Glossary + href: glossary.md From f0059df8f395b92c97666986efe24911f5d6d875 Mon Sep 17 00:00:00 2001 From: woksin Date: Fri, 25 Sep 2026 08:12:58 +0200 Subject: [PATCH 6/6] Add a Why Arc for TypeScript page --- Documentation/toc.yml | 2 + Documentation/why-arc-for-typescript.md | 57 +++++++++++++++++++++++++ 2 files changed, 59 insertions(+) create mode 100644 Documentation/why-arc-for-typescript.md diff --git a/Documentation/toc.yml b/Documentation/toc.yml index 5e9db07b..d385185a 100644 --- a/Documentation/toc.yml +++ b/Documentation/toc.yml @@ -1,5 +1,7 @@ - name: Overview href: index.md +- name: Why Arc for TypeScript + href: why-arc-for-typescript.md - name: Coming from Express and NestJS href: coming-from-express-and-nestjs.md - name: Getting Started diff --git a/Documentation/why-arc-for-typescript.md b/Documentation/why-arc-for-typescript.md new file mode 100644 index 00000000..6e807cdc --- /dev/null +++ b/Documentation/why-arc-for-typescript.md @@ -0,0 +1,57 @@ +--- +title: Why Arc for TypeScript +description: The endpoint plumbing Arc removes from a Node.js backend, who it is built for, what you get in return, and when a different approach fits better. +--- + +## Every endpoint repeats the same plumbing + +A Node.js backend for a rich frontend writes the same shape for every operation: a route, a body parser, input checks that produce some error format, a call into a service, a status code, a response shape, an authorization check, a tenant lookup, and a typed client in the frontend that someone keeps in step by hand. Add live data and each list also needs its own server-sent events or WebSocket endpoint. None of that is the behavior you set out to write. It is the cost of getting a TypeScript class across HTTP. + +## What Arc does instead + +You put the behavior on the model. A class decorated `@command()` with a `handle()` method is the command, its validation target, and its endpoint. A class decorated `@readModel()` with static `@query()` methods is the read model and the endpoints that serve it. Arc runs every command and query through one pipeline that owns binding, authorization, validation, tenancy, correlation, error redaction, and status codes, and serves the result over the same [HTTP contract](/arc/http-contract/) as Arc on .NET. + +```mermaid +graph LR + Client[Generated proxy or HTTP client] --> Host[Node host, Express, Fastify, or Hono] + Host --> Pipeline[Arc command or query pipeline] + Pipeline --> Model["@command() or @readModel() class"] + Model -. optional .-> Storage[MongoDB, SQL, or Chronicle] + Source[Decorated TypeScript source] --> Generator[arc-proxygenerator] + Generator --> Client +``` + +`arc-proxygenerator` reads the same decorated source and writes typed proxies for the published `@cratis/arc` client, including React hooks, so the frontend's types come from the backend's code. + +## What you get + +| Without Arc | With Arc | +| --- | --- | +| A route and handler per operation, plus the service call | One decorated class; `handle()` or the static query method is the behavior | +| Input checks and error shapes repeated per route | `@field` binding, `CommandValidator` and `ConceptValidator` rules, and one validation result shape with a `/validate` route for every command | +| A hand-kept client, or a spec you maintain beside the code | Proxies generated from your TypeScript source, and OpenAPI served from the same metadata | +| A custom SSE or WebSocket endpoint per live list | Return an RxJS observable; Arc serves a snapshot, server-sent events, WebSockets, and multiplexed hubs | +| Authorization checks sprinkled through handlers | `@authorize`, `@roles`, `@allowAnonymous`, and named policies, evaluated before validation | +| Tenant and correlation handling in every handler | One resolved tenant and correlation ID per request, passed to storage integrations | +| Hand-written event-log plumbing | Optionally, return Chronicle events from `handle()` and let the integration append them | + +## Who it is for + +- **Teams with an Arc frontend.** If your frontend uses `@cratis/arc` and `@cratis/arc.react`, this server speaks their contract, and the generator writes their proxies. +- **Teams running Arc on .NET and Node side by side.** Both implementations follow one HTTP contract, and a paired suite checks the same requests against both; see [How parity is checked](reference/capabilities.md#how-parity-is-checked). +- **CQRS applications, with or without event sourcing.** The core needs no database and no event store. MongoDB, SQL through Drizzle, and Chronicle are separate, optional packages. + +## When it is the wrong fit + +- **You need a published, stable package today.** Nothing is on npm yet, the version is 0.x, and APIs can still change. You build from this repository's workspace. +- **You need everything Arc on .NET does.** Parity is incomplete and tracked area by area. Read the [capability reference](reference/capabilities.md) before you plan around a feature; the Chronicle integration in particular is experimental. +- **You design your HTTP API resource by resource.** Arc's contract is fixed: commands are POST requests, queries are GET or HTTP `QUERY` requests, and results come in Arc's envelopes. If clients depend on a hand-shaped REST or custom wire format, Arc's conventions will fight you. +- **You need live queries over SQL.** The Drizzle integration serves snapshots and pages, not change notifications; see [SQL with Drizzle](sql/index.md). +- **You deploy to a runtime Arc has not been run on.** The Node host and adapters are checked, and the Fetch entry runs in Deno; Bun, Cloudflare Workers, and Next.js deployments have not been exercised. See [Fetch API runtimes](hosts/fetch-runtimes.md). +- **You have a handful of endpoints and no frontend to generate for.** The pipeline and conventions pay off across many operations and a typed client. A single webhook receiver does not need them. + +## Where to go next + +- [Get started](getting-started/index.md): run the Tasks sample and call its command and query. +- [Hosting overview](overview.md): choose the standalone Node host or a framework adapter. +- [Architecture](architecture.md): the core, the adapters, and how Arc concepts map to TypeScript.