diff --git a/.claude/commands/coreex-aspire.md b/.claude/commands/coreex-aspire.md new file mode 100644 index 000000000..90dde6685 --- /dev/null +++ b/.claude/commands/coreex-aspire.md @@ -0,0 +1,14 @@ +--- +description: "Add or extend .NET Aspire orchestration and end-to-end tests in a CoreEx solution. USE FOR: AppHost wiring for new hosts, service-bus queue/topic topology in Test.Aspire, MockHost stubs for third-party HTTP, OnBeforeStartAsync/OnAfterStartAsync setup, E2E tests across domains. DO NOT USE FOR: non-Aspire hosts (use coreex-scaffold) or per-host integration tests (use coreex-test-api / coreex-test-subscribe / coreex-test-relay)." +allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] +--- + + + +Read `.github/skills/coreex-aspire/SKILL.md` and follow the instructions in that file. diff --git a/.claude/commands/coreex-command-publish-e2e.md b/.claude/commands/coreex-command-publish-e2e.md new file mode 100644 index 000000000..b859aa4d0 --- /dev/null +++ b/.claude/commands/coreex-command-publish-e2e.md @@ -0,0 +1,14 @@ +--- +description: "Add the sending/publishing of a command (not an event) to another domain end-to-end: adapter, transactional call site (outbox vs direct), NamedDestinationProvider queue routing, Service Bus topology, API/Relay tests and Aspire E2E. USE FOR: a domain must instruct another domain to do something. DO NOT USE FOR: publishing an event (use coreex-app-service), receiving a command (use coreex-command-subscribe-e2e), or synchronous request/response calls (use coreex-adapter)." +allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] +--- + + + +Read `.github/skills/coreex-command-publish-e2e/SKILL.md` and follow the instructions in that file. diff --git a/.claude/commands/coreex-command-subscribe-e2e.md b/.claude/commands/coreex-command-subscribe-e2e.md new file mode 100644 index 000000000..a98664456 --- /dev/null +++ b/.claude/commands/coreex-command-subscribe-e2e.md @@ -0,0 +1,14 @@ +--- +description: "Add the receiving/handling of a command (not an event) end-to-end: contract, app service, subscriber, dedicated keyed command-queue receiver in the Subscribe host, Service Bus topology, Subscribe tests and Aspire E2E. USE FOR: this domain must act on a command addressed to it. DO NOT USE FOR: subscribing to an event (use coreex-subscriber-e2e), sending a command (use coreex-command-publish-e2e), or subscribing to another domain's command (never valid)." +allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] +--- + + + +Read `.github/skills/coreex-command-subscribe-e2e/SKILL.md` and follow the instructions in that file. diff --git a/.claude/commands/coreex-repository.md b/.claude/commands/coreex-repository.md index 02b544b54..913349e30 100644 --- a/.claude/commands/coreex-repository.md +++ b/.claude/commands/coreex-repository.md @@ -1,5 +1,5 @@ --- -description: "Create or modify a CoreEx Infrastructure-layer repository. USE FOR: new repository class, adding CRUD operations, adding a custom query (QueryArgsConfig), bidirectional mapper (BiDirectionMapper), EfDb model accessor, Result pipeline variants. DO NOT USE FOR: Application-layer service logic, domain invariants, typed HTTP clients/adapters (those follow adapter conventions in the infrastructure instructions, not this skill)." +description: "Create or modify a CoreEx Infrastructure-layer repository. USE FOR: new repository class, adding CRUD operations, adding a custom query (QueryArgsConfig), bidirectional mapper (BiDirectionMapper), EfDb/CosmosDb model accessor, Result pipeline variants. DO NOT USE FOR: Application-layer service logic, domain invariants, typed HTTP clients/adapters (those follow adapter conventions in the infrastructure instructions, not this skill)." allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] --- diff --git a/.claude/commands/coreex-scaffold.md b/.claude/commands/coreex-scaffold.md index b07b9d680..bf4eb1470 100644 --- a/.claude/commands/coreex-scaffold.md +++ b/.claude/commands/coreex-scaffold.md @@ -1,5 +1,5 @@ --- -description: "Guide a developer through CoreEx solution shaping after bootstrap, using a short plain-English interview that turns user answers into safe dotnet new template inputs. USE FOR: bootstrap-only repos, deciding API-only vs API plus relay vs API plus subscriber, choosing SQL Server vs Postgres vs no database, choosing refdata/outbox/DDD/ROP options, installing CoreEx.Template, checking current solution shape, adding missing Api/Relay/Subscribe hosts to an existing repo, adding a .NET Aspire AppHost for local multi-host orchestration and dashboard visibility, and optionally preparing a first local runnable state with local dependency assets plus database/code-generation steps. DO NOT USE FOR: unrelated runtime debugging, bootstrap creation, or forcing root re-scaffolding over an existing solution. INVOKES: workspace inspection, ask-questions style interviews, dotnet new install/list, dry-run validation, solution wiring, optional local dependency asset creation, focused build/test validation, and either template generation or manual retrofit work depending on repo shape." +description: "Guide a developer through CoreEx solution shaping after bootstrap, using a short plain-English interview that turns user answers into safe dotnet new template inputs. USE FOR: bootstrap-only repos, deciding API-only vs API plus relay vs API plus subscriber, choosing SQL Server vs Postgres vs Cosmos vs no database, choosing refdata/outbox/DDD/ROP options, installing CoreEx.Template, checking current solution shape, adding missing Api/Relay/Subscribe hosts to an existing repo, adding a .NET Aspire AppHost for local multi-host orchestration and dashboard visibility, and optionally preparing a first local runnable state with local dependency assets plus database/code-generation steps. DO NOT USE FOR: unrelated runtime debugging, bootstrap creation, or forcing root re-scaffolding over an existing solution. INVOKES: workspace inspection, ask-questions style interviews, dotnet new install/list, dry-run validation, solution wiring, optional local dependency asset creation, focused build/test validation, and either template generation or manual retrofit work depending on repo shape." allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] --- diff --git a/.claude/commands/coreex-subscriber-e2e.md b/.claude/commands/coreex-subscriber-e2e.md index 8a78b83bd..a640b0178 100644 --- a/.claude/commands/coreex-subscriber-e2e.md +++ b/.claude/commands/coreex-subscriber-e2e.md @@ -1,5 +1,5 @@ --- -description: "Create a complete new event or command subscriber end-to-end in a single guided workflow: event/command DTO contract (if new), subscriber handler, optional application service and repository, and integration tests. USE FOR: adding a brand-new subscriber to an existing CoreEx Subscribe host for any scenario — command handling, event-data-sync replication, or event-driven business-process choreography. DO NOT USE FOR: modifying an existing subscriber (use coreex-subscriber directly), API endpoint work (use coreex-api-e2e), or setting up the Subscribe host itself (use coreex-scaffold)." +description: "Create a complete new event subscriber end-to-end in a single guided workflow: event DTO contract (if new), subscriber handler, optional application service and repository, and integration tests. USE FOR: adding a brand-new subscriber to an existing CoreEx Subscribe host for event-data-sync replication or event-driven business-process choreography. DO NOT USE FOR: command handling (use coreex-command-subscribe-e2e), sending a command (use coreex-command-publish-e2e), modifying an existing subscriber (use coreex-subscriber directly), API endpoint work (use coreex-api-e2e), or setting up the Subscribe host itself (use coreex-scaffold)." allowed-tools: [Read, Glob, Grep, Edit, Write, Bash] --- diff --git a/.github/INSTRUCTION_AUTHORING.md b/.github/INSTRUCTION_AUTHORING.md index 187cf4a1d..a0da7d5d1 100644 --- a/.github/INSTRUCTION_AUTHORING.md +++ b/.github/INSTRUCTION_AUTHORING.md @@ -58,7 +58,7 @@ for every POST") is an invariant and stays; a numbered "detect → confirm → s procedure and moves to the skill. **Project-wide choices live in state, not in prompts.** The solution-root `AGENTS.md` "Feature Configuration" -records `data-provider`, `rop-enabled`, `refdata-enabled`, `outbox-enabled`, and `messaging-provider`. Whether a Domain layer is present is inferred from the existence of `src/*.Domain/` (added via `dotnet new coreex-domain`). Instructions and skills must **read that recording before asking** and re-state resolved +records `data-provider` (`SqlServer` / `Postgres` / `Cosmos` / `None`), `rop-enabled`, `refdata-enabled`, `outbox-enabled`, and `messaging-provider`. Whether a Domain layer is present is inferred from the existence of `src/*.Domain/` (added via `dotnet new coreex-domain`). Instructions and skills must **read that recording before asking** and re-state resolved values for confirmation rather than re-prompting (the global rule lives in `.github/copilot-instructions.md`). --- @@ -207,9 +207,9 @@ A development-time console tool. Reads a single `ref-data.yaml` file (validated The entry point is a one-line `Program.cs` in the `*.CodeGen` project. Templates live in `CoreEx.CodeGen/RefData/Templates/` (embedded in the NuGet package) and are the only place structural changes to the generated shape belong. -### DbEx (`*.Database` project) +### `*.Database` tooling project -A development-time migration and generation tool. Reads YAML configuration and SQL migration scripts to produce database schema, outbox infrastructure, and EF Core scaffolding. +For relational domains this is a DbEx-based migration and generation tool that reads YAML configuration and SQL migration scripts to produce database schema, outbox infrastructure, and EF Core scaffolding. For Cosmos domains it is a `CosmosDbConsole` provisioning app that declares containers and imports seed data; there are no SQL migration scripts or `dbex.yaml`. | Output pattern | What to change instead | |---|---| diff --git a/.github/agents/README.md b/.github/agents/README.md index ee495880f..9a9ed44ce 100644 --- a/.github/agents/README.md +++ b/.github/agents/README.md @@ -13,13 +13,13 @@ It is defined once in `coreex-expert.agent.md` and works in both tools from the ## How the agent resolves guidance -The agent relies on a local doc cache rather than making live GitHub fetches on every question. The cache is populated by `/coreex-docs-sync` and stored under `.github/docs/coreex/`. +The agent prefers the local doc cache installed by `dotnet new coreex-ai` at the pinned `CoreEx.Template` version. `/coreex-docs-sync` refreshes that cache and the entire AI asset bundle together; it does not fetch GitHub `main`. ``` Developer asks a CoreEx question │ ▼ - Read .github/docs/coreex/.manifest + Read .github/docs/coreex/manifest.txt │ ┌─────┴─────┐ │ │ @@ -28,12 +28,12 @@ Developer asks a CoreEx question │ └──► offer to run /coreex-docs-sync first │ (never runs silently — always asks first) │ - check staleness + compare coreex-version with project package references │ ┌────┴────┐ │ │ - fresh stale - │ (>30 days OR coreex-version in manifest + matches differs + │ (coreex-version in manifest │ differs from version in project files) │ │ │ └──► recommend running /coreex-docs-sync @@ -41,25 +41,25 @@ Developer asks a CoreEx question ▼ Use local cache (always preferred over live GitHub fetches) │ - ├── .github/docs/coreex/*.md ← 10 architecture docs + ├── .github/docs/coreex/*.md ← 12 architecture/setup docs │ └── .github/docs/coreex/agents/*.md ← 18 per-package AI guides │ - read manifest referenced-packages + inspect project package references to distinguish: • package already in project → guide on current usage • package not yet in project → recommend adopting it ``` -The `referenced-packages` field in the manifest lets the agent distinguish between a package the project already uses and one it would need to add — without restricting which guides get synced. +The manifest records only `coreex-version`. Package adoption is determined from the project's package references, not a `referenced-packages` field or a sync timestamp. --- ## The local doc cache -`/coreex-docs-sync` fetches from the CoreEx GitHub repository and writes two folders: +`CoreEx.Template` packages the documentation at release time. `coreex-ai` installs it; `/coreex-docs-sync` reinstalls the bundle from the release matching the project's CoreEx NuGet version. -**`.github/docs/coreex/`** — 10 architecture docs: +**`.github/docs/coreex/`** — 12 architecture/setup docs: | File | Content | |------|---------| @@ -73,17 +73,18 @@ The `referenced-packages` field in the manifest lets the agent distinguish betwe | `testing.md` | Unit, integration, API, Subscribe, and Relay test patterns | | `tooling.md` | CodeGen and Database project run order, generated-file ownership | | `aspire.md` | Aspire orchestration for local distributed development and E2E testing | +| `local-dev.md` | Local infrastructure, connection strings, and startup sequences | +| `application-scaffolding-guide.md` | Choosing and composing solution/host templates | -**`.github/docs/coreex/agents/`** — 18 per-package AI usage guides: the base `CoreEx` package plus one per `src/CoreEx.*` package (including `CoreEx.Cosmos`, which is newly published as a preview-quality package — its API surface may still change without following strict semver until it stabilizes). All 18 are synced unconditionally so the agent can guide on any package — including ones the project hasn't adopted yet. +**`.github/docs/coreex/agents/`** — 18 per-package AI usage guides: the base `CoreEx` package plus one per `src/CoreEx.*` package (including `CoreEx.Cosmos`). All 18 are synced unconditionally so the agent can guide on any package — including ones the project hasn't adopted yet. -**`.github/docs/coreex/.manifest`** — records `synced` date, `coreex-version`, and `referenced-packages`. +**`.github/docs/coreex/manifest.txt`** — records `coreex-version`, stamped by the installed template release. ### Staleness triggers | Trigger | Agent action | |---------|-------------| | Cache absent | Offers to run `/coreex-docs-sync` before the first GitHub fetch | -| `synced` date > 30 days | Recommends a refresh | | `coreex-version` in manifest ≠ version in project files | Recommends a refresh | --- @@ -95,7 +96,7 @@ An earlier design synced only the packages the project already references. This - The agent cannot recommend adopting a package (e.g. `CoreEx.Caching.FusionCache`) if it has no knowledge of what that package offers. - All 18 guides are small markdown files — the total download is negligible. - Syncing all unconditionally removes the need to re-run after adding a new package. -- The `referenced-packages` manifest field preserves the "in project vs. not yet" distinction without making it a gate on what gets synced. +- Inspecting project package references preserves the "in project vs. not yet" distinction without making it a gate on which guides get installed. --- @@ -119,10 +120,10 @@ dotnet new coreex-ai --app-folder `dotnet new coreex-ai` installs everything the agent depends on: - `.github/instructions/` — the scoped, auto-injected instruction files -- `.github/prompts/` — the `coreex-scaffold` prompt plus one prompt per per-capability (L1) skill -- `.github/skills/` — the full skill suite (`coreex-docs-sync`, `acquire-codebase-knowledge`, `coreex-scaffold`, `aspire`, and the 15 L1 skills) +- `.github/prompts/` — 22 wrappers: bootstrap, scaffold, 16 L1 skills and four L2 workflows (`coreex-docs-sync` is a direct skill, without a prompt wrapper) +- `.github/skills/` — 23 CoreEx skills: bootstrap, scaffold, docs-sync, 16 L1 skills and four L2 workflows; generic `acquire-codebase-knowledge` and `aspire` skills are framework-repository-only - `.github/agents/coreex-expert.agent.md` — this agent -- `.claude/commands/` — the Claude Code equivalents +- `.claude/commands/` — 24 commands: one for each CoreEx skill plus expert guidance - `.github/docs/coreex/` — the local docs cache (architecture docs + per-package guides) the expert reads first, already populated at the pinned version — no separate sync step needed on first install Later, when the project's `CoreEx` NuGet version is bumped, re-run `/coreex-docs-sync` to refresh the whole bundle (instructions, skills, prompts, this agent, and the docs cache) to the matching `CoreEx.Template` release — the agent recommends this automatically when the cached `coreex-version` no longer matches the project (see the resolution flowchart above). diff --git a/.github/agents/coreex-expert.agent.md b/.github/agents/coreex-expert.agent.md index 0f865b88b..16294c6c3 100644 --- a/.github/agents/coreex-expert.agent.md +++ b/.github/agents/coreex-expert.agent.md @@ -33,16 +33,16 @@ These files are present when the CoreEx AI workflow set has been copied into the - `.github/instructions/coreex-domain.instructions.md` — DDD aggregates, `Entity`, mutation guards, `Result` pipelines. - `.github/instructions/coreex-application-services.instructions.md` — service shape, `TransactionAsync`, validation-before-transaction, event enqueuing. - `.github/instructions/coreex-validators.instructions.md` — `Validator`, rule chains, `CommonValidator`, `ValidateAndThrowAsync`. -- `.github/instructions/coreex-repositories.instructions.md` — `EfDbModel`, `IBiDirectionMapper`, `QueryArgsConfig`, paging. +- `.github/instructions/coreex-repositories.instructions.md` — `EfDb`/`CosmosDb`, `IBiDirectionMapper`, `QueryArgsConfig`, paging. - `.github/instructions/coreex-api-controllers.instructions.md` — controller shape, `WebApi` helpers, `[IdempotencyKey]`, PATCH. - `.github/instructions/coreex-event-subscribers.instructions.md` — subscriber classes, `[Subscribe]`, `SubscribedManager`, error handling. - `.github/instructions/coreex-host-setup.instructions.md` — `Program.cs` shape, middleware order, service registration, outbox relay hosts. -- `.github/instructions/coreex-tooling.instructions.md` — `*.CodeGen` and `*.Database` projects, `ref-data.yaml`, DbEx, generated-file ownership. +- `.github/instructions/coreex-tooling.instructions.md` — `*.CodeGen` and `*.Database` projects, `ref-data.yaml`, DbEx or Cosmos provisioning, generated-file ownership. - `.github/instructions/coreex-tests.instructions.md` — `UnitTestEx`, `NUnit`, `AwesomeAssertions`, outbox/event expectations, seed data. ### Per-package AI usage guides -Check `.github/docs/coreex/agents/` for locally cached guides first (see [Local doc cache](#local-doc-cache)). `/coreex-docs-sync` caches guides for **all** CoreEx packages — check the manifest's `referenced-packages` field to distinguish packages already in the project from ones the project would need to add. +Check `.github/docs/coreex/agents/` for locally cached guides first (see [Local doc cache](#local-doc-cache)). `/coreex-docs-sync` installs guides for **all** CoreEx packages — inspect the project's package references to distinguish packages already adopted from ones it would need to add. The manifest records only `coreex-version`. If a guide is not cached locally, fetch from GitHub: @@ -51,6 +51,7 @@ If a guide is not cached locally, fetch from GitHub: - [CoreEx.AspNetCore.NSwag](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.AspNetCore.NSwag/AGENTS.md) — NSwag/OpenAPI integration. - [CoreEx.Azure.Messaging.ServiceBus](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Azure.Messaging.ServiceBus/AGENTS.md) — Service Bus publisher, subscribers, error handling. - [CoreEx.Caching.FusionCache](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Caching.FusionCache/AGENTS.md) — `IHybridCache`, Redis backplane, idempotency provider. +- [CoreEx.Cosmos](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Cosmos/AGENTS.md) — `CosmosDb`, `CosmosDbContainer`, transactional-batch outbox, Change Feed relay, provisioning. - [CoreEx.CodeGen](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.CodeGen/AGENTS.md) — `CodeGenConsole`, `ref-data.yaml`, generated-file ownership. - [CoreEx.Data](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Data/AGENTS.md) — `IUnitOfWork`, `TransactionAsync`, `QueryArgsConfig`, `DataResult`. - [CoreEx.Data.GraphQL](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Data.GraphQL/AGENTS.md) — GraphQL-lite `IGraphQLEngine`, `where`/`orderBy` bridged to `QueryArgsConfig`, Relay Cursor Connections paging. @@ -86,7 +87,7 @@ Check `.github/docs/coreex/` for a local cache first (see [Local doc cache](#loc | Folder | Contents | |---|---| -| `.github/docs/coreex/` | 11 sample architecture docs (layers, patterns, each layer walkthrough, testing, tooling, Aspire, application scaffolding) | +| `.github/docs/coreex/` | 12 architecture/setup docs (local development, layers, patterns, each layer walkthrough, testing, tooling, Aspire, application scaffolding) | | `.github/docs/coreex/agents/` | AI usage guides for **all** CoreEx packages — available for guidance even on packages not yet adopted by this project | A manifest at `.github/docs/coreex/manifest.txt` records `coreex-version` — stamped directly by the template at generation time, never hand-written. @@ -111,6 +112,7 @@ Do not run `/coreex-docs-sync` silently — always offer and wait for confirmati - Separate explanation, plan, and implementation guidance clearly. - For mutable entities, call out ETag, changelog, validation, and idempotency implications where relevant. - For messaging, explicitly distinguish API-only, API plus outbox relay, API plus subscriber, and full orchestration shapes. +- For Service Bus sessions, treat publisher key-to-session mapping, destination-specific bucket counts, receiver concurrency, and relational outbox partition sizing as separate controls; verify destination-specific profiles against the exact resolved queue/topic name. - The Api/Relay/Subscribe host split is a workload-isolation convention, not a technical requirement — for a small, low-traffic solution, consolidating hosted-service processing (outbox relay, subscriber receiving) into the Api host is a legitimate simplification. Mention it when a user's stated scale/traffic profile suggests the extra processes may not be earning their operational cost; see [Hosts Layer Guide § Choosing a Host Topology](https://github.com/Avanade/CoreEx/blob/main/samples/docs/hosts-layer.md#choosing-a-host-topology-split-vs-consolidate). - Never recommend editing `*.g.cs`, `*.g.sql`, or `*.g.pgsql` files — direct the user to the owning generator instead (Roslyn source generator for `*.g.cs`; `*.Database` project for `*.g.sql`/`*.g.pgsql`). @@ -123,7 +125,9 @@ These skills are part of the CoreEx AI workflow set and live in `.github/skills/ | Request | Skill | |---------|-------| | New entity with CRUD API — full stack from database table to HTTP endpoint | `/coreex-api-e2e` | -| New event or command subscriber — handler, optional orchestration, and tests | `/coreex-subscriber-e2e` | +| New event subscriber (event-data-sync or business-process) — handler, optional orchestration, and tests | `/coreex-subscriber-e2e` | +| **Send a command** to another domain (not an event) — adapter, outbox/direct call site, queue routing, topology, tests, Aspire | `/coreex-command-publish-e2e` | +| **Handle a command** addressed to this domain (not an event) — contract, subscriber, dedicated command-queue receiver, topology, tests, Aspire | `/coreex-command-subscribe-e2e` | **Add or modify a single capability** → route to the matching per-capability (L1) skill: @@ -131,23 +135,27 @@ These skills are part of the CoreEx AI workflow set and live in `.github/skills/ |---------|-------| | Contract / DTO / entity | `/coreex-contract` | | Reference data | `/coreex-refdata` | -| Database table or migration | `/coreex-db-migration` | +| Relational schema/migration or Cosmos container provisioning | `/coreex-db-migration` | | Repository, mapper, query config | `/coreex-repository` | | External integration / adapter | `/coreex-adapter` | | Application service | `/coreex-app-service` | | Validator | `/coreex-validator` | -| Authorization / business policy | `/coreex-policy` | +| I/O-dependent business guards in application policies | `/coreex-policy` | | DDD aggregate / domain entity | `/coreex-aggregate` | | API controller / endpoint | `/coreex-api` | -| Event subscriber | `/coreex-subscriber` | +| GraphQL-lite query bridge over existing query configuration | `/coreex-graphql` | +| Event or command subscriber | `/coreex-subscriber` | | API tests | `/coreex-test-api` | | Subscriber tests | `/coreex-test-subscribe` | | Outbox relay tests | `/coreex-test-relay` | +| Aspire AppHost, service-bus topology, MockHost stubs, E2E tests | `/coreex-aspire` | **Broader routing:** - Greenfield solution or host scaffolding, or adding a .NET Aspire AppHost for local multi-host orchestration/dashboard → `/coreex-scaffold` (`coreex-scaffold` skill), which runs the matching [CoreEx.Template](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Template/README.md) `dotnet new coreex*` commands. -- Repo mapping or onboarding documentation → `/acquire-codebase-knowledge`. +- Blank repository after installing `coreex-ai` → `/coreex-bootstrap`, which creates root guidance and chains to `/coreex-scaffold`. +- CoreEx package version changed or AI bundle version mismatches → `/coreex-docs-sync`. +- Repo mapping or onboarding documentation → `/acquire-codebase-knowledge` where installed (framework-repository-only; not shipped by `coreex-ai`). - Retrofit that no single skill covers → inspect the current code and recommend the smallest manual changes aligned to the samples and instructions. ## Response format diff --git a/.github/copilot-instructions.md b/.github/copilot-instructions.md index 5b0527a3e..b08fa8480 100644 --- a/.github/copilot-instructions.md +++ b/.github/copilot-instructions.md @@ -12,7 +12,7 @@ CoreEx is a modular .NET framework for enterprise APIs and distributed services. ## Project Configuration State (read before asking) A CoreEx solution scaffolded with `dotnet new coreex` records its project-wide choices in the **solution-root -`AGENTS.md` "Feature Configuration"** block: `data-provider` (SQL Server / PostgreSQL / None), `refdata-enabled`, +`AGENTS.md` "Feature Configuration"** block: `data-provider` (SQL Server / PostgreSQL / Cosmos / None), `refdata-enabled`, `rop-enabled` (exception vs `Result`), `outbox-enabled`, and `messaging-provider`. Whether a Domain layer is present is inferred from the existence of `src/*.Domain/` (added via `dotnet new coreex-domain`). @@ -30,7 +30,7 @@ solution; the rule applies to consumer solutions where these assets are installe - `gen\CoreEx.Generator\`: Roslyn source generator for contracts. - `tests\`: framework-level tests. - `samples\src\Contoso.*\`: sample domains split by layer/host. -- `samples\aspire\AppHost.cs`: orchestration entrypoint. +- `samples\aspire\Contoso.Aspire\AppHost.cs`: orchestration entrypoint (plus `Contoso.Aspire.MockHost` for stub dependencies and `Contoso.Test.Aspire` for automated cross-domain smoke testing). - `coreex-starter\`: separate starter template repo — ignore unless user wants starter changes. ## Build, Test, and Run @@ -41,6 +41,7 @@ solution; the rule applies to consumer solutions where these assets are installe - **`CoreEx.Template` changes**: `dotnet build`/`dotnet test` do **not** exercise `src/CoreEx.Template/content/**` — that's raw `dotnet new` template content, not compiled C#. Any change under `src/CoreEx.Template/content/` (a host's `Program.cs`/`GlobalUsing.cs`/`.csproj`, a `template.json` symbol, etc.) must be validated by actually scaffolding it: run [`tools/validate-template-pack.ps1`](../tools/validate-template-pack.ps1), which packs the template, installs it, scaffolds every parameter combination it knows about into temp directories, and `dotnet build`s the ones flagged `Build = $true`. It also runs in CI (`.github/workflows/CI.yml`). If you add a new template, host, or parameter combination, add a matching scenario to the script's `$testScenarios` array — parameter-conditional bugs (an unconditional `global using`/`ProjectReference` that should have been gated behind a symbol like `has-data-provider` or `implement-servicebus`) only surface when the generated code is actually compiled, which most existing host-template scenarios don't yet do since they scaffold in isolation without their `coreex` solution siblings. - **Linting**: No separate `dotnet format`. Build is the lint pass (nullable, LangVersion=preview, TreatWarningsAsErrors in `src\Directory.Build.props`). - **Formatting**: 4 spaces for `*.cs`, 2 spaces for `*.json|*.xml|*.yaml|*.props|*.csproj|*.sln|*.sql` per `.editorconfig`. +- **Line endings**: `.editorconfig` (`end_of_line = lf`) and `.gitattributes` (`* text=auto eol=lf`) both mandate LF. Tooling (`dotnet new` templating, Roslyn/Handlebars code generators, and file-writing AI tools) does not consult either file and can emit CRLF on Windows regardless. Before finishing any task that created or edited files, run `git add --renormalize ` (or `.` if changes are broad) as a closing step — it is a no-op if nothing drifted, so always safe to run. Do this in addition to, not instead of, `run_build`. - **Ad-hoc spike/reflection projects**: this repo uses Central Package Management (root `Directory.Packages.props`). A throwaway `dotnet new console` project scaffolded *inside* the repo tree (even outside `src\`/`tests\`) will silently inherit it and can fail to restore (`NU1008`) if it references a package/version not centrally pinned. Scaffold spike projects outside the repo tree, or set `false` in the spike project's own `.csproj` to opt out locally. ## Local Development Infrastructure @@ -56,9 +57,9 @@ podman compose -f docker-compose.yml up -d # Podman preferred; `docker compose | `db-sql-server` | 1433 | Shopping domain database; Service Bus emulator backing store | | `db-postgres` | 5432 | Products domain database | | `redis-cache` | 6379 | FusionCache Redis backplane (all domains) | -| `servicebus-emulator` | 5672 AMQP, 5300 mgmt | Azure Service Bus emulator; namespace `sbemulatorns`; topic `contoso` with subscriptions `products` and `shopping` (both session-enabled); config at `servicebus/Config.json` | +| `servicebus-emulator` | 5672 AMQP, 5300 mgmt | Azure Service Bus emulator; namespace `sbemulatorns`; topic `contoso` with subscriptions `products` and `shopping` (both session-enabled) plus session-enabled command queue `contoso-products`; config at `servicebus/Config.json` | | `dts-emulator` | 8080, 8082 | Azure Durable Task Scheduler emulator; task hubs `default` and `order` | -| `cosmos-emulator` | 8081, 10251-10254 | Azure Cosmos DB emulator; backs the Customers domain database and `CoreEx.Cosmos.Test.Unit`; under rootless Podman prefer `--privileged` or host networking if it doesn't come up cleanly (see `docker-compose.yml` comment) | +| `cosmos-emulator` | 8081 (HTTPS gateway), 8083 (health, `/ready`), 1234 (Data Explorer) | Azure Cosmos DB **vNext** (Linux) emulator, host networking; backs the Customers domain database and `CoreEx.Cosmos.Test.Unit`; no container cap (see `docker-compose.yml` comment) | | `aspire-dashboard` | 18888 UI, 4317 OTLP | Standalone OpenTelemetry dashboard; usable without running the full Aspire AppHost | Connection strings for each service in development are in each host's `appsettings.Development.json` under the `Aspire:` configuration key hierarchy. See [`samples/docs/local-dev.md`](../samples/docs/local-dev.md) for full detail, connection string patterns, and startup sequences. @@ -67,10 +68,10 @@ Connection strings for each service in development are in each host's `appsettin - **Two roles**: framework packages (`src\`) + sample reference implementations (`samples\`). - **Business layers** (strict inward dependency — inner layers have no knowledge of outer): `*.Contracts` → `*.Domain` (optional) → `*.Application` → `*.Infrastructure`. Confirmed by project references: `*.Domain` references only `*.Contracts`; `*.Application` references `*.Contracts` and `*.Domain`; `*.Infrastructure` references `*.Application` (and transitively `*.Domain`) — never the reverse in either case. - **Host layers** (composition roots, no business logic): `*.Api`, `*.Relay`, `*.Subscribe`. -- **Design-time tooling** (no runtime presence): `*.CodeGen` (generates reference-data layer from `ref-data.yaml`) and `*.Database` (schema, seeding, outbox infrastructure via DbEx). -- **Sample flow**: Controllers → `WebApi` helpers → Application services (validate + `IUnitOfWork`) → Infrastructure repositories (EF + explicit mappers) → transactional outbox → relay publishes to Service Bus → subscribers consume. -- **Polyglot data**: Products uses PostgreSQL (`CoreEx.Database.Postgres` + `CoreEx.EntityFrameworkCore`); Shopping uses SQL Server (`CoreEx.Database.SqlServer` + `CoreEx.EntityFrameworkCore`); Customers uses Azure Cosmos DB (`CoreEx.Cosmos` — preview, API surface may still change without following strict semver until it stabilizes; direct SDK access, no EF Core, no `*.Database` project; containers are code-first via `ReplaceOrCreateContainerAsync`). Layers above Infrastructure are database-agnostic. -- **Primary domains**: Products and Shopping complete; Customers (Cosmos DB) demonstrates typed CRUD + transactional outbox but has no Relay/Subscribe host yet; Orders WIP. See `samples\README.md` for topology. +- **Design-time tooling** (no runtime presence): `*.CodeGen` (generates reference-data layer from `ref-data.yaml`) and `*.Database` (schema, seeding, and outbox infrastructure via DbEx for relational domains; a Cosmos DB provisioning console for Cosmos domains). +- **Sample flow**: Controllers → `WebApi` helpers → Application services (validate + `IUnitOfWork`) → Infrastructure repositories (EF/CoreEx.Cosmos + explicit mappers) → transactional outbox → relay publishes to Service Bus → subscribers consume. +- **Polyglot data**: Products uses PostgreSQL (`CoreEx.Database.Postgres` + `CoreEx.EntityFrameworkCore`); Shopping uses SQL Server (`CoreEx.Database.SqlServer` + `CoreEx.EntityFrameworkCore`); Customers uses Azure Cosmos DB (`CoreEx.Cosmos`; direct SDK access, no EF Core, and a `*.Database` provisioning console instead of DbEx migrations/schema). Layers above Infrastructure are database-agnostic. +- **Primary domains**: Products and Shopping complete; Customers (Cosmos DB) demonstrates typed CRUD + transactional outbox + a change-feed Relay host, but has no Subscribe host; Orders WIP. See `samples\README.md` for topology. - **Aspire**: orchestrates all sample hosts in `samples\aspire\Contoso.Aspire\AppHost.cs` for local distributed development and E2E testing. ## Key Conventions That Matter in This Repo @@ -83,6 +84,7 @@ Connection strings for each service in development are in each host's `appsettin ### Contracts and Source Generation - Contracts are commonly declared as `[Contract] public partial class ...`. - Mutable contracts often implement `IIdentifier`, `IETag`, and `IChangeLog`. +- Use `[NonNullable]` for properties that are mandatory/required but declared nullable (`string?`, `int?`) for DTO flexibility, so OpenAPI reports them as non-nullable. - Use `[ReadOnly(true)]` for server-managed fields and `[ReferenceData]` for reference-data-backed code properties. - Canonical casing transformations belong in property setters when already established by the model (for example `Sku` uppercasing in `ProductBase`). - Favor the existing source-generation approach; do not hand-write members that are meant to be generated. @@ -149,9 +151,9 @@ Connection strings for each service in development are in each host's `appsettin - Sample: `WithGenericTester` (unit) or `WithApiTester` (API/Subscribe/Relay). - Integration tests: per-class named seed files `Data\read-data.seed.yaml` / `Data\mutate-data.seed.yaml` (Products), or a single `Data\data.yaml` (Orders/Shopping), in Test.Common + `Resources\` JSON expectations. - **Intra-domain dependencies are real; inter-domain dependencies are always mocked.** Own database, cache, and outbox are started and seeded in `[OneTimeSetUp]`. Cross-domain HTTP calls and direct broker publishes are replaced with `MockHttpClientFactory` / `UseExpectedAzureServiceBusPublisher()`. -- Outbox assertion helpers are database-specific: `UseExpectedPostgresOutboxPublisher()` for Products; `UseExpectedSqlServerOutboxPublisher()` for Shopping. Do not use the SQL Server helper in Products tests. +- Outbox assertion helpers are database-specific: `UseExpectedPostgresOutboxPublisher()` for Products; `UseExpectedSqlServerOutboxPublisher()` for Shopping; `UseExpectedCosmosDbOutboxPublisher()` for Customers. Do not mix helpers across providers. - Mock downstream HTTP calls; do not assume live APIs. -- **Outbox test-coverage split — don't re-flag as a gap without checking both places**: `tests\CoreEx.Database.SqlServer.Test.Unit` / `tests\CoreEx.Database.Postgres.Test.Unit` cover library-level logic only (parameter extensions, error-code mapping, `DatabaseRecord`/`DatabaseCommand` behavior) against a live database via DbEx migrations in the sibling `*.Test.Console` project — they do **not** exercise `SqlServerOutboxPublisher`/`SqlServerOutboxRelay` or `PostgresOutboxPublisher`/`PostgresOutboxRelay` end-to-end. That coverage lives in `samples\tests\Contoso.Products.Test.Relay` (Postgres) and `samples\tests\Contoso.Shopping.Test.Relay` (SQL Server): both spin up the real relay host (`WithApiTester<...Relay.Program>`), publish real events via the outbox, let the live `*OutboxRelayHostedService` poll and relay them, and assert what actually lands on the Service Bus emulator (`Test.GetAndClearAzureServiceBusAsync`). Keep the two relay test projects in parity — a fix or new scenario added to one should be mirrored in the other. +- **Outbox test-coverage split — don't re-flag as a gap without checking both places**: `tests\CoreEx.Database.SqlServer.Test.Unit` / `tests\CoreEx.Database.Postgres.Test.Unit` cover library-level logic only (parameter extensions, error-code mapping, `DatabaseRecord`/`DatabaseCommand` behavior) against a live database via DbEx migrations in the sibling `*.Test.Console` project — they do **not** exercise `SqlServerOutboxPublisher`/`SqlServerOutboxRelay` or `PostgresOutboxPublisher`/`PostgresOutboxRelay` end-to-end. That coverage lives in `samples\tests\Contoso.Products.Test.Relay` (Postgres) and `samples\tests\Contoso.Shopping.Test.Relay` (SQL Server): both spin up the real relay host and assert delivery on the Service Bus emulator. Customers exercises Cosmos outbox writes in `samples\tests\Contoso.Customers.Test.Api`, and real two-container forwarding, cleanup, first-start exclusion, and management in `samples\tests\Contoso.Customers.Test.Relay`. Aspire additionally covers the real Customers API-to-broker flow using an isolated observation subscription. ### House Rules - Code comments end with a period/full stop. @@ -162,6 +164,7 @@ Connection strings for each service in development are in each host's `appsettin - Single-line `if` bodies do not need braces: `if (x) return;` - Use expression-bodied syntax (`=>`) when the entire method or property body is a single expression. - Private instance fields are always prefixed with `_`. +- Files end with exactly one trailing newline — never extra blank lines after the last line of content (check the tail of every file you create or edit). ### Generated Code Never create or edit `*.g.cs`, `*.g.sql`, or `*.g.pgsql` files directly. Each generator owns its outputs: @@ -193,10 +196,12 @@ see [coreex-ai-workflows.md](./coreex-ai-workflows.md). |---------|------|-------------| | `CoreEx.Template` | Template pack | Deterministic `dotnet new` scaffolding. Pin the version — `dotnet new install CoreEx.Template::` — then `dotnet new coreex` (solution), `coreex-api` / `coreex-relay` / `coreex-subscribe` (hosts), `coreex-domain` (optional DDD layer), `coreex-aspire` (optional local-orchestration AppHost), or `coreex-ai` (AI workflow assets). Validate any change to `src/CoreEx.Template/content/**` with [`tools/validate-template-pack.ps1`](../tools/validate-template-pack.ps1) — see "Build, Test, and Run" above. | | `CoreEx Expert` | Agent | Architecture guidance, pattern recommendations, and design review. Invoke via `/coreex-expert` (or `@coreex-expert`). | +| `/coreex-bootstrap` | Skill-backed prompt | Initialize root guidance in a blank repository after `coreex-ai`, then chain to `/coreex-scaffold`. | | `/coreex-scaffold` | Skill-backed prompt | Guided greenfield solution scaffolding (chooses the smallest safe shape, runs the `dotnet new coreex*` commands). | | `/coreex-docs-sync` | Skill | Refresh the whole AI asset bundle (instructions, skills, prompts, the `coreex-expert` agent, and the `.github/docs/coreex/` doc cache) to a new pinned CoreEx version after a version bump. | -| `/coreex-` | Skills (L1) + matching prompts | Add or modify one building block: `coreex-contract`, `coreex-refdata`, `coreex-db-migration`, `coreex-repository`, `coreex-adapter`, `coreex-app-service`, `coreex-validator`, `coreex-policy`, `coreex-aggregate`, `coreex-api`, `coreex-graphql`, `coreex-subscriber`, and `coreex-test-api` / `coreex-test-subscribe` / `coreex-test-relay`. Each skill has a `.prompt.md` wrapper for Copilot. | -| `/acquire-codebase-knowledge`, `/aspire` | Skills | Repo onboarding documentation; local Aspire orchestration. | +| `/coreex-` | Skills (L1) + matching prompts | Add or modify one building block: `coreex-contract`, `coreex-refdata`, `coreex-db-migration`, `coreex-repository`, `coreex-adapter`, `coreex-app-service`, `coreex-validator`, `coreex-policy`, `coreex-aggregate`, `coreex-api`, `coreex-graphql`, `coreex-subscriber`, `coreex-aspire`, and `coreex-test-api` / `coreex-test-subscribe` / `coreex-test-relay`. | +| `/coreex-*-e2e` | Skills (L2) + matching prompts | Complete new slices: `coreex-api-e2e`, `coreex-subscriber-e2e`, `coreex-command-publish-e2e`, and `coreex-command-subscribe-e2e`. | +| `/acquire-codebase-knowledge`, `/aspire` | Repository-only skills | Repo onboarding documentation; local Aspire orchestration. Not shipped by `coreex-ai`. | ## Guidance for Authoring Instructions and Skills @@ -205,4 +210,4 @@ When creating or maintaining Copilot instruction files and skills: - **Instruction files** (`.instructions.md`) — see [INSTRUCTION_AUTHORING.md](./INSTRUCTION_AUTHORING.md) for standards on YAML frontmatter, section order, and content rules. - **Skill files** (`SKILL.md`) — see [SKILL_AUTHORING.md](./SKILL_AUTHORING.md) for the directory structure pattern (`references/`, `assets/`), lean main file rules (<300 lines), and cross-referencing guidelines. -Both documents define durable patterns for creating guidance that is discoverable, maintainable, and context-efficient. \ No newline at end of file +Both documents define durable patterns for creating guidance that is discoverable, maintainable, and context-efficient. diff --git a/.github/coreex-ai-workflows.md b/.github/coreex-ai-workflows.md index ba81970bc..5b8e152a5 100644 --- a/.github/coreex-ai-workflows.md +++ b/.github/coreex-ai-workflows.md @@ -28,7 +28,7 @@ This folder contains the AI artefacts that give GitHub Copilot and Claude Code a - Claude Code: `@coreex-expert` - Copilot Chat: switch to **Agent** mode and select **CoreEx Expert** -The agent uses a local doc cache (populated by `/coreex-docs-sync`) to avoid live GitHub fetches on every question. It covers all 18 CoreEx packages, distinguishing those already in the project from ones the project could adopt. See the [agent README](./agents/README.md) for the resolution flowchart, cache structure, and adoption guide. +The agent uses a local doc cache installed by `dotnet new coreex-ai` and refreshed by `/coreex-docs-sync` to avoid live GitHub fetches on every question. It covers all 18 CoreEx package guides, distinguishing packages referenced by the project from ones it could adopt. See the [agent README](./agents/README.md) for the resolution flowchart, cache structure, and adoption guide. ## Instructions @@ -40,13 +40,14 @@ Instructions are passive — no action is needed to activate them. The global fi | `coreex-contracts.instructions.md` | Contract files — `[Contract]`, `[ReferenceData]`, source generation | | `coreex-application-services.instructions.md` | Application services — `TransactionAsync`, validation, event enqueuing | | `coreex-validators.instructions.md` | Validator files — `Validator`, rule chains | -| `coreex-repositories.instructions.md` | Repository files — `EfDbModel`, mappers, `QueryArgsConfig`, paging | +| `coreex-repositories.instructions.md` | Repository files — `EfDb`/`CosmosDb`, mappers, `QueryArgsConfig`, paging | | `coreex-api-controllers.instructions.md` | Controller files — `WebApi` helpers, `[IdempotencyKey]`, PATCH | | `coreex-event-subscribers.instructions.md` | Subscriber files — `[Subscribe]`, `SubscribedManager`, error handling | | `coreex-host-setup.instructions.md` | `Program.cs` files — middleware order, service registration, outbox relay | -| `coreex-tooling.instructions.md` | CodeGen and Database projects — `ref-data.yaml`, DbEx, generated-file ownership | +| `coreex-tooling.instructions.md` | CodeGen and Database projects — `ref-data.yaml`, DbEx or Cosmos provisioning, generated-file ownership | | `coreex-tests.instructions.md` | Test files — UnitTestEx, NUnit, AwesomeAssertions, outbox/event assertions | | `coreex-domain.instructions.md` | Domain files — aggregates, mutation guards, `Result` pipelines | +| `coreex-aspire.instructions.md` | Aspire files — AppHost wiring, service-bus topology, MockHost/third-party HTTP, `Test.Aspire` lifecycle | ## Prompts, Skills, and Templates @@ -55,8 +56,8 @@ Instructions are passive — no action is needed to activate them. The global fi Three artefact types cooperate, each with a distinct job: - **Instruction** (`instructions/*.instructions.md`) — invariant rules that are *auto-injected* whenever an edited file matches the instruction's `applyTo` glob. You never invoke them; they passively shape every edit to a matching file. -- **Skill** (`skills/coreex-*/`) — a reasoning *workflow* you invoke explicitly to create or modify something. Each skill owns a `SKILL.md` and a `references/workflow.md` that drive the interaction. GitHub Copilot invokes a skill directly by reading `SKILL.md`; Claude Code requires the matching `.claude/commands/coreex-.md` thin wrapper (see below) for `/coreex-` to work. -- **Prompt** (`prompts/coreex-*.prompt.md`) — the GitHub Copilot entry point that *delegates to the matching skill's workflow*. Skills and prompts map 1:1 by name, so `/coreex-contract` and `coreex-contract.prompt.md` run the same workflow. +- **Skill** (`skills/coreex-*/`) — a reasoning *workflow* you invoke explicitly to create or modify something. Each skill owns a `SKILL.md`; longer workflows also use `references/`. GitHub Copilot invokes a skill directly by reading `SKILL.md`; Claude Code requires the matching `.claude/commands/coreex-.md` thin wrapper (see below) for `/coreex-` to work. +- **Prompt** (`prompts/coreex-*.prompt.md`) — the GitHub Copilot entry point that *delegates to the matching skill's workflow*. All L1/L2 workflows plus bootstrap and scaffold have matching prompts; `coreex-docs-sync` is invoked directly as a skill. - **Claude command** (`.claude/commands/coreex-*.md`) — the Claude Code entry point, mirroring the prompt's role: a thin wrapper ("Read `.github/skills/coreex-/SKILL.md` and follow it") that makes `/coreex-` work in Claude Code. Every `coreex-*` skill ships one, 1:1 by name — this is what gives Claude Code and Copilot equivalent slash-command coverage over the same skill catalog. **Feature Configuration.** A CoreEx solution's project-wide choices — `data-provider`, `refdata-enabled`, `rop-enabled`, `outbox-enabled`, `messaging-provider` — are persisted in the solution-root `AGENTS.md` **Feature Configuration** block. Skills read that block before asking anything, so recorded decisions are not re-prompted from one skill to the next. Whether a Domain layer is present is inferred from the existence of the `src/*.Domain/` project rather than a flag, since the Domain layer is now an independent addon template (`coreex-domain`). @@ -72,42 +73,47 @@ A version mismatch between the installed AI-asset bundle (`.github/docs/coreex/m ### Command catalog +The consumer bundle contains **23 CoreEx skills** (16 L1, four L2, bootstrap, scaffold, and docs-sync), **22 prompt wrappers**, **24 Claude commands** (one per skill plus expert guidance), **12 scoped instructions** plus the consumer global instruction, and one expert agent. The two generic repository-only skills below are not shipped by `coreex-ai`. + | Command | Type | What it does | |---------|------|-------------| | [`CoreEx.Template`](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Template/README.md) | Template pack | Deterministic `dotnet new` scaffolding for a CoreEx solution plus API, relay, and subscriber hosts. Always install with an explicit pinned version — `dotnet new install CoreEx.Template::` (see [Version-pin discipline](#version-pin-discipline)) — then run the `coreex*` templates in a terminal. `dotnet new coreex-ai` installs the full AI workflow set (instructions, prompts, skills, the `coreex-expert` agent, `.claude/commands/`, and the `.github/docs/coreex/` docs cache, self-describing via `manifest.txt`) into a consuming project as one version-pinned bundle. | | [`/acquire-codebase-knowledge`](./skills/acquire-codebase-knowledge/README.md) | Skill | Maps an unfamiliar codebase and produces seven structured onboarding documents. Generic (no CoreEx dependency) and framework-repo-only — not copied into consumer repos by `dotnet new coreex-ai`. | | [`/coreex-scaffold`](./skills/coreex-scaffold/README.md) · [prompt](./prompts/coreex-scaffold.prompt.md) | Skill + prompt | Guides greenfield solution scaffolding, chooses the smallest safe CoreEx.Template shape, and runs the matching `dotnet new coreex*` commands. | +| [`/coreex-bootstrap`](./skills/coreex-bootstrap/SKILL.md) · [prompt](./prompts/coreex-bootstrap.prompt.md) | Skill + prompt | Initializes root guidance in a blank repository after `coreex-ai`, then chains to `coreex-scaffold`. | | [`/coreex-docs-sync`](./skills/coreex-docs-sync/README.md) | Skill | Refreshes the whole AI asset bundle (instructions, skills, prompts, agent, `.claude/commands/`, and the `.github/docs/coreex/` docs cache) as one version-pinned `dotnet new coreex-ai --force` reinstall, matching the project's referenced `CoreEx` NuGet version. No live GitHub `main` fetch. | | [`/aspire`](./skills/aspire/README.md) | Skill | Orchestrates Aspire distributed apps locally: start, stop, logs, debug. Generic (wraps the Aspire CLI, no CoreEx dependency) and framework-repo-only — not copied into consumer repos by `dotnet new coreex-ai`. | #### Per-capability skills (L1) -Fifteen skills add or modify a single CoreEx capability on an existing solution. Each is invoked as `/coreex-` in Claude Code (via its `.claude/commands/coreex-.md` thin wrapper) or via the matching [`prompts/coreex-.prompt.md`](./prompts/) in Copilot — 1:1 by name across all three (skill, prompt, Claude command). Every skill reads the solution-root `AGENTS.md` **Feature Configuration** first to avoid redundant questioning. +Sixteen skills add or modify a single CoreEx capability on an existing solution. Each is invoked as `/coreex-` in Claude Code (via its `.claude/commands/coreex-.md` thin wrapper) or via the matching [`prompts/coreex-.prompt.md`](./prompts/) in Copilot — 1:1 by name across all three (skill, prompt, Claude command). Every skill reads the solution-root `AGENTS.md` **Feature Configuration** first to avoid redundant questioning. | Skill / prompt | Capability | |----------------|-----------| | [`coreex-contract`](./skills/coreex-contract/) | Hand-authored contract (DTO/entity) — root, subordinate, request/response, base class | -| [`coreex-refdata`](./skills/coreex-refdata/) | Reference data type + `ref-data.yaml` entry | -| [`coreex-db-migration`](./skills/coreex-db-migration/) | Database table / DbEx migration | -| [`coreex-repository`](./skills/coreex-repository/) | EF Core repository, mapper, and query configuration | +| [`coreex-refdata`](./skills/coreex-refdata/) | Reference data types, seed data, `ref-data.yaml`, and opt-in mutation APIs | +| [`coreex-db-migration`](./skills/coreex-db-migration/) | Database table change or Cosmos container provisioning change | +| [`coreex-repository`](./skills/coreex-repository/) | EF Core or Cosmos repository, mapper, and query configuration | | [`coreex-adapter`](./skills/coreex-adapter/) | External-integration adapter | | [`coreex-app-service`](./skills/coreex-app-service/) | Application service orchestration | | [`coreex-validator`](./skills/coreex-validator/) | Fluent `Validator` | -| [`coreex-policy`](./skills/coreex-policy/) | Authorization / business policy | +| [`coreex-policy`](./skills/coreex-policy/) | I/O-dependent business guards in application policies | | [`coreex-aggregate`](./skills/coreex-aggregate/) | DDD aggregate / domain entity | | [`coreex-api`](./skills/coreex-api/) | API controller / endpoint | | [`coreex-graphql`](./skills/coreex-graphql/) | GraphQL-lite query bridge (`CoreEx.Data.GraphQL`) on an API host | -| [`coreex-subscriber`](./skills/coreex-subscriber/) | Event subscriber | +| [`coreex-subscriber`](./skills/coreex-subscriber/) | Event or command subscriber | | [`coreex-test-api`](./skills/coreex-test-api/) | API tests | | [`coreex-test-subscribe`](./skills/coreex-test-subscribe/) | Subscriber tests | | [`coreex-test-relay`](./skills/coreex-test-relay/) | Outbox relay tests | +| [`coreex-aspire`](./skills/coreex-aspire/) | Aspire AppHost wiring, service-bus topology, MockHost stubs, and `Test.Aspire` E2E tests | #### End-to-end workflow skills (L2) -Two skills orchestrate a complete vertical slice by gathering all context upfront and invoking the appropriate L1 skills in sequence — no repeated questions. They are **creation-only** — modifications to an existing entity or subscriber always target the relevant L1 skill directly. +Four skills orchestrate a complete vertical slice by gathering all context upfront and invoking the appropriate L1 skills in sequence — no repeated questions. They are **creation-only** — modifications to an existing entity or subscriber always target the relevant L1 skill directly. | Skill / prompt | What it orchestrates | |----------------|---------------------| | [`coreex-api-e2e`](./skills/coreex-api-e2e/) | New entity with CRUD API — contract → migration → repository → validator → policy (if needed) → app-service → API endpoint → integration tests | -| [`coreex-subscriber-e2e`](./skills/coreex-subscriber-e2e/) | New event or command subscriber — contract (if new) → migration + repository (if new entity) → app-service (if needed) → subscriber handler → integration tests | - +| [`coreex-subscriber-e2e`](./skills/coreex-subscriber-e2e/) | New event subscriber (data-sync / business-process; for **commands** use the two command skills below) — contract (if new) → migration + repository (if new entity) → app-service (if needed) → subscriber handler → integration tests | +| [`coreex-command-publish-e2e`](./skills/coreex-command-publish-e2e/) | Send a **command** to another domain — adapter → transactional call site (outbox vs direct) → `NamedDestinationProvider` queue routing → Test.Common `ServiceBus` queue → API + Relay tests → Aspire E2E | +| [`coreex-command-subscribe-e2e`](./skills/coreex-command-subscribe-e2e/) | Handle a **command** addressed to this domain — contract (if payload) → migration + repository (if new state) → app-service → subscriber → dedicated keyed command-queue receiver → `ServiceBus` queue → Subscribe tests → Aspire E2E | diff --git a/.github/instructions/coreex-application-services.instructions.md b/.github/instructions/coreex-application-services.instructions.md index 2d4c7c9d9..e2e3decb6 100644 --- a/.github/instructions/coreex-application-services.instructions.md +++ b/.github/instructions/coreex-application-services.instructions.md @@ -92,7 +92,7 @@ public async Task UpdateAsync(Product product, CancellationToken cancel ## Validation -Validators live in `Application/Validators/` and are **not registered in DI** — they are not injected into services (see [DI Registration Principle](#di-registration-principle) below). Choose the base class based on whether the validator needs injected dependencies: +Application business/request validators live in `Application/Validators/` and are **not registered in DI** — they are not injected into services (see [DI Registration Principle](#di-registration-principle) below). External HTTP response-contract rules belong in the Infrastructure client instead; see [client response validation](/.github/instructions/coreex-repositories.instructions.md#client-response-validation). Choose the base class based on whether the application validator needs injected dependencies: **`Validator`** — use when no constructor injection is required. Exposes a static `Default` singleton; always call via the singleton: @@ -352,6 +352,30 @@ Each has `Async` and `AsAsync` variants too (e.g. `ThenAsync`, `ThenAsAsync`). S Failure factories (return a failed result of the matching type): `Result.ValidationError(...)`, `NotFoundError(...)`, `BusinessError(...)`, `ConflictError(...)`, `ConcurrencyError(...)`, `DuplicateError(...)`, `AuthenticationError(...)`, `AuthorizationError(...)`, `TransientError(...)`. Success factories: `Result.Success`, `Result.Ok(value)`. See the [CoreEx Results README](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx/Results/README.md) for the full set. +#### Propagating a failure (guard clauses) + +Prefer a `Then*` chain — it short-circuits on failure and re-types for you. An explicit guard clause is also fine (e.g. between awaits); when you use one, **hand the existing result straight back** — never unwrap its `Error` to build a new one: + +| You hold | Method returns | Write | +|---|---|---| +| `Result` | `Result` | `return r;` (implicit `Result` → `Result`) | +| `Result` | `Result` | `return r;` | +| `Result` | `Result` (different type) | `return r.AsResult();` (drops to `Result`, then implicitly converts) | +| `Result` | `Result` | `return r.AsResult();` | + +```csharp +var loaded = await _repository.GetAsync(id, cancellationToken).ConfigureAwait(false); +if (loaded.IsFailure) return loaded.AsResult(); // Result → Result + +var concurrency = ETag.CompareWithResult(etag, loaded.Value.ETag); +if (concurrency.IsFailure) return concurrency; // Result → Result + +// ❌ Wrong — reaches into Error to rebuild what already exists (and needs a '!' to do it). +if (loaded.IsFailure) return Result.Fail(loaded.Error!); +``` + +The implicit `Result` → `Result` conversion defaults the value on success, so use it (and `AsResult()`) only on the **failure** path of a guard — to continue on success, produce the real value (`ThenAs`, or construct the `Result`). Do not use the implicit `Result` → `T` conversion in a service either; it reads `Value`, which throws on failure. + **`ThrowOnError()` returns `Result`, not `T`** — unwrapping to `T` happens via `Result`'s implicit conversion operator, which only fires in a **target-typed** context: ```csharp @@ -439,6 +463,8 @@ var result = await new ProductPolicy(_productAdapter).EnsureExistsAsync(productI ## Application-Level Mapping +For collection projections, follow [Mapper Projection Nullability](/.github/instructions/coreex-conventions.instructions.md#mapper-projection-nullability); preserve the lambda selectors in the examples below. + When a domain has a Domain layer, an `Application/Mapping/` sub-folder holds mappers that translate between the **Domain aggregate** and the **Contract**. This mapping is an Application-layer concern because it sits at the public surface boundary — it is not tied to any persistence technology. Use `Mapper` (uni-directional). Mappers are **not registered in DI** — call them via the static `Map()` method directly at the point of use (see [DI Registration Principle](#di-registration-principle) below): @@ -524,6 +550,7 @@ Always call `.ConfigureAwait(false)` on every `await` inside service and reposit - Do not perform a mutating `TransactionAsync` without adding its event (`_unitOfWork.Events.Add(...)`) when eventing is enabled — the write and the event must be committed together. - Do not use `WhereMutated(_ => ...)` on a delete — `DataResult` (delete) has no value; use the parameterless `WhereMutated(() => ...)`. The value-carrying `WhereMutated(v => ...)` is only for `DataResult` (create/update). - Do not reach for a non-`As` Result operator when the delegate changes the result type — use the `As` variant (e.g. `ThenAs`/`ThenAsAsync`); the `As` suffix exists to make the type change explicit. +- Do not propagate a failure with `Result.Fail(r.Error!)` / `Result.Fail(r.Error!)` / `return r.Error!;` — return the result itself (`return r;`) or `r.AsResult()` when the result type differs; prefer a `Then*` chain. Avoid touching `.Error` in service code at all. - Do not generate service operations the user did not confirm — confirm the CRUD set (Get/Create/Update/Delete, each default-selected) when operations are unspecified, and never add a Query operation without an explicit request. ## Further Reading diff --git a/.github/instructions/coreex-aspire.instructions.md b/.github/instructions/coreex-aspire.instructions.md new file mode 100644 index 000000000..0c27dd8c7 --- /dev/null +++ b/.github/instructions/coreex-aspire.instructions.md @@ -0,0 +1,126 @@ +--- +applyTo: "**/aspire/**/*.cs" +description: "Aspire conventions: AppHost wiring, MockHost stubs, Test.Aspire lifecycle (OnBeforeStartAsync/OnAfterStartAsync), Service Bus topology, and how HTTP calls outside the domain are resolved" +tags: ["aspire", "apphost", "mockhost", "e2e", "service-bus", "http", "wiremock"] +--- + + + +# Aspire Conventions + +> **Related skill:** to wire a new host, queue, inter-domain call, third-party stub or E2E scenario into Aspire, invoke +> [`coreex-aspire`](/.github/skills/coreex-aspire/SKILL.md). This file holds the invariants that must hold on **any** edit to +> an `aspire/**` file. For the full narrative see [Aspire & End-to-End](/.github/docs/coreex/aspire.md) (docs-sync cache). + +## Project roles + +| Project | Role | Rules | +|---|---|---| +| `*.Aspire` (AppHost) | Declares hosts + their dependencies; runs them together with a dashboard | Wiring only — no business logic, no DI registrations for the hosts | +| `*.Aspire.MockHost` | WireMock.Net console host that stands in for HTTP dependencies **outside** the AppHost | Started via `AddMockHostProject`; carries no stubs itself — stubs are added by the test (or at AppHost start) | +| `*.Test.Aspire` | NUnit project deriving `WithAspireTester`; starts the whole AppHost and asserts cross-host flows | The **only** place inter-domain dependencies are real | + +`dotnet new coreex-aspire` generates all three; the `AddEndpoints` / `AddHostedServiceSupport` / `DisableHttpCertificateValidation` / +`AddMockHostProject` / `WithMockHostEnvironment` / `HttpMock` helpers come from `CoreEx.UnitTesting` + `UnitTestEx.Aspire` +(`` in the AppHost csproj) — there is no local `Extensions.cs`. + +## AppHost wiring + +Infrastructure (relational databases, the Cosmos emulator, Redis, and the Service Bus emulator) runs from `docker-compose.yml`, **not** Aspire. Model each dependency as a visible connection-string resource with the name the host already expects (`SqlServer`, `Postgres`, or `Cosmos`; plus `redis` / `ServiceBus`), then add one `AddProject` per host, referencing only what it needs. `AddExternalConnectionString` is supplied by `CoreEx.UnitTesting` in the `UnitTestEx` namespace alongside the other Aspire helpers; do not duplicate it locally. + +```csharp +var serviceBus = builder.AddExternalConnectionString("ServiceBus").WithIconName("MailMultiple"); + +var productsApi = builder.AddProject("products-api") + .WithReference(postgres).WithReference(redis).AddEndpoints("/health/ready/detailed"); + +builder.AddProject("products-subscribe") + .WithReference(postgres).WithReference(redis).WithReference(serviceBus) + .AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); // relay + subscribe hosts always get hosted-service support +``` + +- Resource names are kebab-case `{domain}-{host}` (`products-api`, `shopping-subscribe`). They are the names used by `Test.Http("products-api")`, `WaitForResourceAsync`, and service discovery — never rename casually. +- Each new host needs a `` in the AppHost csproj (this is what generates `Projects.X`). +- Relay: data provider + Service Bus. Subscribe: data provider + Redis + Service Bus. Api: data provider + Redis (+ Service Bus only if it publishes directly, not via outbox). For Cosmos domains, use `builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint")`; relational domains keep their existing SQL Server/Postgres connection names. +- The helper wraps secret `ConnectionStrings:{name}` parameters in visible `ConnectionStringResource` nodes. Plain `AddConnectionString(name)` creates parameter-only resources excluded from Aspire 13.5.4's Graph/Table views. Older AppHosts should update `CoreEx.UnitTesting` and adopt the scaffold pattern without overwriting custom wiring. +- A running connection-string node means configuration resolved, not infrastructure health; host readiness checks still probe dependencies. Cosmos needs its configured endpoint URL for peer matching because the dashboard parser does not recognise `AccountEndpoint`. Keep credentials out of URLs, use explicit relational ports, and inspect `server.address`/`server.port`/`peer.service` before attributing a mislabelled span to a provider. + +## HTTP calls outside the domain + +A host that calls something outside itself falls into exactly one of three cases. Pick by *where the callee lives*, never by convenience. + +| Callee | Wire it with | Test.Aspire | +|---|---|---| +| **Another domain in the same AppHost** (e.g. Shopping → Products API) | `.WithReference(calleeApi)` on **every calling host**; the caller's `appsettings.Development.json` sets `BaseAddress` to `https+http://{resource-name}` and the client is registered with `builder.AddTypedHttpClient("ConfigSection")` | Real call — **no stub**. Just wait for the callee (`WaitForResourceAsync`) | +| **Third-party API** (SendGrid, payment gateway, …) | `var mockhost = builder.AddMockHostProject("mock-host");` then `.WithMockHostEnvironment("SendGrid__BaseAddress", mockhost, "http")` on **every calling host** (this overrides the configured `BaseAddress`) | Stub in `OnAfterStartAsync` (below) | +| **Another domain that is *not* in this AppHost** (separate repo/solution) | Treat as third-party: MockHost + `WithMockHostEnvironment`, stubbing that domain's HTTP contract | Stub in `OnAfterStartAsync` | + +```csharp +// AppHost.cs — third-party API stood up by the MockHost. +var mockhost = builder.AddMockHostProject("mock-host"); +builder.AddProject("shopping-subscribe") + /* ...references... */ + .WithMockHostEnvironment("SendGrid__BaseAddress", mockhost, "http"); +``` + +```csharp +// Test.Aspire — stub the third-party route once the MockHost is running. +protected override async Task OnAfterStartAsync(DistributedApplication app) +{ + await app.WaitForResourceAsync(["products-api", "shopping-api"]); + await app.HttpMock("mock-host", "http").Request(HttpMethod.Post, "/v3/mail/send").WithAnyBody().Respond.WithAsync(HttpStatusCode.Accepted); +} +``` + +- The `WithMockHostEnvironment` key is the configuration path with `__` separators and must match the section the typed client reads (`AddTypedHttpClient("SendGrid")` reads `SendGrid:BaseAddress`). +- The AppHost-scaffolded `mockhost` line is commented out until a stub is genuinely needed — uncomment it, do not add a second MockHost. +- Auth material for a third-party client (`ApiKey` etc.) must still resolve to *something* in `appsettings.Development.json`, or the typed-client factory throws before the call reaches the mock. +- Never call the real third-party API from Aspire tests. Never stub an intra-AppHost domain — that hides the very integration Aspire exists to verify. + +## Test.Aspire lifecycle + +`WithAspireTester` starts the AppHost between two hooks. The `Test.*` migrate/reset helpers of the intra-domain testers are **not** available here — use the `app.*` equivalents. + +```csharp +public class E2ETest : WithAspireTester +{ + protected override async Task OnBeforeStartAsync(DistributedApplication app) + { + // Runs BEFORE any host starts: seed state and create broker entities so receivers/relays find them on startup. + await app.MigratePostgresDataAsync("Postgres", ["mutate-data.seed.yaml"], Products.Database.Program.ConfigureMigrationArgs); + // Cosmos variant: await app.MigrateCosmosDataAsync("Cosmos", "contoso", ["no-data.seed.yaml"], Customers.Database.Program.ConfigureProvisionArgs); + await app.ClearRedisCacheAsync("redis"); + await app.ResetAzureServiceBusAsync("ServiceBus", ServiceBus.GetQueues(), ServiceBus.GetTopicsAndSubscriptions()); + } + + protected override async Task OnAfterStartAsync(DistributedApplication app) + => await app.WaitForResourceAsync(["products-api", "shopping-api"]); // + HttpMock stubs (above) +} +``` + +- **Service Bus topology is code**, in a static `ServiceBus` class (`GetQueues()` / `GetTopicsAndSubscriptions()`). A single-domain solution reuses its `Test.Common` `ServiceBus`; a multi-domain AppHost owns one **union** class listing every domain's topic, subscriptions **and command queues**. A queue that is missing here makes the relay fail to send and the command receiver fail to start. +- **Cosmos test state is also provisioned through code.** Use `app.MigrateCosmosDataAsync(..., DbMigration.ConfigureProvisionArgs)` to create/reset containers and import the named `Data/*.seed.yaml` resources; do not drop containers manually between Aspire tests, because `MigrateCosmosDataAsync` also avoids the stale-client-cache issues that follow an out-of-band reset. +- Command queues are `RequiresSession = true` (the receivers are session receivers); topic subscriptions likewise. +- Keep the producer's session policy aligned with the exact resolved destination name: `CoreEx:Host:ServiceBus:Destinations:{destination}` overrides are configured in the **publishing host** (`Api`, `Subscribe`, or `Relay`), not in the Aspire topology. Session bucket count controls producer-side key grouping; receiver `MaxConcurrentSessions` controls consumer parallelism, and neither is the broker entity's physical partition count. +- Drive flows over real HTTP with `Test.Http("resource-name")`. For anything asynchronous (outbox → relay → broker → subscriber) **poll with a bounded loop** (`Test.Delay(1000, "reason")` up to N attempts, then `Assert.Fail`) until the observable effect appears; assert a stubbed third-party call by polling the MockHost's WireMock admin API (`await Test.HttpMock("mock-host", "http").GetAdminApiAsync()` then `GetRequestsAsync()`) for a request matching the path (`/v3/mail/send`) **and** a body fragment unique to the flow (e.g. the recipient e-mail) — never scrape logs, and never match on path alone (it can't tell your request from someone else's). +- The Aspire tester has no broker-peek helper. A command's delivery to its queue is proven in `*.Test.Relay` (`GetAndClearAzureServiceBusAsync`); Aspire proves the *effect* in the consuming host. + +## Do Not + +- Do not run infrastructure through Aspire (`AddPostgres`, `AddRedis`, `AddAzureServiceBus`, or Cosmos-provisioning resources) — the emulators come from `docker-compose.yml`. +- Do not hard-code ports or URLs in tests or `BaseAddress` — use resource names / service discovery. +- Do not put business logic, migrations or seed data in the AppHost. +- Do not use `[OneTimeSetUp]` for AppHost preparation — the hooks are `OnBeforeStartAsync` / `OnAfterStartAsync`. +- Do not stub an HTTP dependency that is itself a resource in the AppHost. + +## Further Reading + +- [Aspire & End-to-End](/.github/docs/coreex/aspire.md) and [Local Development](/.github/docs/coreex/local-dev.md) — docs-sync cache. +- Related skills: [`coreex-aspire`](/.github/skills/coreex-aspire/SKILL.md), [`coreex-command-publish-e2e`](/.github/skills/coreex-command-publish-e2e/SKILL.md), [`coreex-command-subscribe-e2e`](/.github/skills/coreex-command-subscribe-e2e/SKILL.md). +- Illustrative (CoreEx sample — not in your project): [`AppHost.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Aspire/AppHost.cs), [`E2ETest.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Test.Aspire/E2ETest.cs), [`ServiceBus.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Test.Aspire/ServiceBus.cs). diff --git a/.github/instructions/coreex-contracts.instructions.md b/.github/instructions/coreex-contracts.instructions.md index a756c8c0c..16f687c72 100644 --- a/.github/instructions/coreex-contracts.instructions.md +++ b/.github/instructions/coreex-contracts.instructions.md @@ -197,6 +197,29 @@ public partial class Employee : IIdentifier, IETag, IChangeLog Decorate server-assigned properties with `[ReadOnly(true)]` to signal that clients cannot supply them. Common examples: `Id`, `ETag`, `ChangeLog`, `CategoryCode` (derived from SubCategory). NSwag/OpenAPI automatically excludes these from inbound request schemas. +## NonNullable Properties + +Contract properties are routinely declared nullable (`string?`, `int?`, `DateOnly?`) for DTO flexibility — partial payloads, merge-patch, and validation that reports a missing value rather than failing deserialization. Where a property is nevertheless **mandatory/required (ultimately not nullable)** in the contract, decorate it with `[NonNullable]` (`CoreEx.Entities.NonNullableAttribute`). The NSwag operation processor then emits it as `nullable: false` in the OpenAPI schema regardless of the CLR nullability. The C# type and validation are unaffected — `[NonNullable]` is documentation of the contract only, so keep the matching validator rule (e.g. `.Mandatory()`). + +```csharp +[Contract] +public partial class Person +{ + /// Gets or sets the first name. + [NonNullable] + public string? FirstName { get; set; } // Mandatory; nullable in code only for DTO flexibility. + + /// Gets or sets the middle name. + public string? MiddleName { get; set; } // Genuinely optional — no attribute. +} +``` + +Do **not** apply it to optional properties, and do not use `System.Diagnostics.CodeAnalysis.NotNull` for this purpose (that is a compiler flow-analysis attribute with no OpenAPI effect). + +### `[ReadOnly]` properties and `[NonNullable]` + +Contracts are commonly shared between request and response bodies, so a server-assigned property is legitimately null on create. Never mark `Id`, `ETag` or `ChangeLog` with `[NonNullable]` — they are lifecycle fields that are null in requests, and non-nullable members in generated clients cause confusion. For any other `[ReadOnly]` property, apply `[NonNullable]` only when its containing type is never used as a request body (e.g. `BasketItem`, which is only returned within the read-only `Basket.Items`); otherwise leave it unmarked (e.g. `ProductBase.CategoryCode`). + ## Reference Data Properties Use `[ReferenceData]` on code properties that back a reference data relationship. Two conditions must both be met for the source generator to emit the navigation accessor: @@ -318,6 +341,27 @@ public partial class ProductLite : ProductBase } ``` +### Patch / subset contracts — do not inherit + +Inheritance is for genuine supersets only: every inherited property must remain meaningful on the derived type. Do **not** inherit a sibling contract for a merge-patch or partial-update request merely because field names overlap. If the contract only needs a subset of the other contract's properties, author a **standalone** class containing just those properties (plus `IETag` where concurrency applies). Never inherit and then suppress the unwanted members with `new` / `[JsonIgnore]` / `[ReadOnly]` — the contract then advertises (in OpenAPI, validation and the patch surface) fields that are silently ignored, and inherited non-nullable members (e.g. `decimal`) take default values that a merge-patch can mistake for intent. + +```csharp +// ❌ Inherits everything, then has to ignore most of it. +public partial class BookingTravellerNotesPatch : BookingTraveller { /* hides/ignores the rest */ } + +// ✅ Standalone — only what the patch actually changes. +[Contract] +public partial class BookingTravellerNotesPatch : IETag +{ + /// Gets or sets the notes. + public string? Notes { get; set; } + + /// + [ReadOnly(true)] + public string? ETag { get; set; } +} +``` + ## Reference Data Contracts Reference data contracts are **generated, not hand-authored**. The source of truth is the `entities:` section of `ref-data.yaml` in the domain's `*.CodeGen` project. Running the CodeGen generates all artefacts across every layer -- contract class, API endpoint, service method, repository interface, repository implementation, and mapper -- as `.g.cs` files that must never be edited directly. @@ -437,6 +481,7 @@ Never create or edit `*.g.cs` files directly. - Do not reference another domain's Contracts assembly to consume its events — declare a local adapter model instead. - Do not omit `[Contract]` and `partial` from hand-authored contract classes without an explicit user request — all hand-authored contracts use `[Contract]` + `partial` by default. +- Do not inherit a sibling contract for a patch/partial-update/subset request and then hide or ignore the unwanted members (`new`, `[JsonIgnore]`) — author a standalone class with only the needed properties (+ `IETag` if applicable). See *Patch / subset contracts — do not inherit*. - Do not implement members that the Roslyn source generator emits (equality, cloning, serialization helpers). - Do not place domain rules, validators, or service calls in contract classes. - Do not leave contract properties without a `` — every property gets one (standard `Id`/`ETag`/`ChangeLog` may use ``). See the *Documentation Comments* section. diff --git a/.github/instructions/coreex-conventions.instructions.md b/.github/instructions/coreex-conventions.instructions.md index 4c20fcc45..bca366dac 100644 --- a/.github/instructions/coreex-conventions.instructions.md +++ b/.github/instructions/coreex-conventions.instructions.md @@ -27,6 +27,22 @@ Every project must have `Nullable` and `ImplicitUsings` enabled. For consuming p Nullable warnings are treated as errors. Never suppress a nullable warning with the null-forgiving operator (`!`) without a clear reason in a comment. +## Mapper Projection Nullability + +When projecting through a CoreEx mapper with `[return: NotNullIfNotNull(nameof(source))]` into a collection of non-nullable elements, use a lambda that calls the mapper rather than an inferred method-group selector. `Select(Mapper.Map)` can infer a nullable result element type from the declared return type; the lambda lets nullable flow analysis apply the conditional return contract to each argument: + +```csharp +// Correct when source.Items has non-nullable elements. +Items = [.. source.Items.Select(i => BasketItemMapper.Map(i))] + +// Avoid: inferred nullable result elements can produce CS8604 at the collection spread. +Items = [.. source.Items.Select(BasketItemMapper.Map)] +``` + +The same rule applies to bidirectional mapper selectors: use `x => ProductMapper.To.Map(x)` / `x => ProductMapper.From.Map(x)`. Do not simplify these lambdas into method groups. This is a conditional-nullability inference rule, not a general ban on method groups. + +If source elements are nullable, a lambda still returns nullable elements: preserve nulls in a nullable destination or explicitly guard them according to the intended behavior. Do not add `!`, warning suppression, or null-dropping filters merely to hide the warning; `Where(x => x is not null)` also does not change the sequence's nullable element type. + ## Global Usings Every project has a single `GlobalUsing.cs` at the project root that declares all namespace imports. Do not add `using` statements to individual source files. @@ -227,8 +243,15 @@ private readonly IUnitOfWork _unitOfWork; private readonly ILogger _logger; ``` +## End of File + +Every file ends with **exactly one** line terminator — the closing `}` (or last line of content) followed by a single newline. No blank lines after it, and no trailing whitespace. This applies to every file you create or edit (`.cs`, `.json`, `.yaml`, `.md`, `.csproj`, `.sql`, etc.), including generated-then-hand-tweaked output and template content. + +Create/edit tooling frequently appends extra `\n`s (particularly when `new_str` ends with a blank line, or when a trailing block is deleted and the blank lines before it are left behind) — check the tail of the file before finishing. + ## Do Not +- Do not leave blank lines at the end of a file — end with exactly one newline after the last line of content (the `.editorconfig` has `insert_final_newline = true`, which guarantees one, not zero extra). - Do not emit `#nullable enable` or `#nullable restore` pragma directives in hand-authored files — nullable is enabled project-wide via `enable` in `Directory.Build.props`. These pragmas are reserved for auto-generated `.g.cs` files produced by code generators. - Do not add `using` statements to individual `.cs` files — declare all imports in `GlobalUsing.cs`. - Do not leave `GlobalUsing.cs` unsorted after editing — re-sort the whole file alphabetically (all namespaces equally, no `System.*` grouping) rather than appending new entries at the end. @@ -241,6 +264,7 @@ private readonly ILogger _logger; - Do not use `Guid.NewGuid()` — use `Runtime.NewGuid()`. - Do not omit or drop `CancellationToken` — every `async`/`Task`-returning method takes one (`CancellationToken cancellationToken = default`, last parameter) and passes it to every downstream awaitable call. - Do not pass `cancellationToken` positionally to a method with 3+ optional parameters (e.g. `ToMappedItemsResultAsync`/`ToItemsResultAsync`) — pass it as `cancellationToken:` by name; a bare positional token can silently bind to an unrelated `bool` parameter (e.g. `autoCount`) and fail to compile. +- Do not rebuild a failed `Result`/`Result` from its `.Error` (`Result.Fail(r.Error!)`, `return r.Error!;`) — return the result itself (`return r;`), or `r.AsResult()` when the result type differs, or use a `Then*` chain. See the Result<T> guard-clause rules in `coreex-application-services.instructions.md`. - Do not replace a private backing field with an auto-property simply because it could be one — backing fields are a valid developer choice. - Do not leave interface members or contract properties undocumented — each gets a ``. - Do not invert the doc convention — summaries go on **interfaces and contract properties**; the **implementing** class member gets `` (not a fresh summary). Summarising the concrete class while leaving the interface/contract undocumented is backwards. diff --git a/.github/instructions/coreex-event-subscribers.instructions.md b/.github/instructions/coreex-event-subscribers.instructions.md index 9ef697f41..e62172913 100644 --- a/.github/instructions/coreex-event-subscribers.instructions.md +++ b/.github/instructions/coreex-event-subscribers.instructions.md @@ -23,7 +23,7 @@ tags: ["subscribers", "messaging", "service-bus", "event-handling", "integration | Package | Key types provided | |---|---| | `CoreEx.Events` | `SubscribedBase`, `SubscribedBase`, `[Subscribe(...)]`, `EventSubscriberArgs`, `ErrorHandler`, `ErrorHandling`, `EventData`, `.Key` | -| `CoreEx.Azure.Messaging.ServiceBus` | `ServiceBusSessionReceiverOptions`, `.AzureServiceBusReceiving()`, `.WithSessionReceiver()`, `.WithSubscribedSubscriber()`, `.WithHostedService()` | +| `CoreEx.Azure.Messaging.ServiceBus` | `ServiceBusSessionReceiverOptions`, `.AzureServiceBusReceiving()`, `.WithSessionReceiver()`, `.WithSubscribedSubscriber()`, `.WithKeyedSubscribedSubscriber()`, `.WithHostedService()` | | `CoreEx` | `[ScopedService]`, `.ThrowIfNull()`, `.Required()`, `Result`, `Result.Success`, `IValidator` | ## Subscriber Structure @@ -170,13 +170,13 @@ builder.Services .AddDefaultCacheKeyProvider() .AddHybridCacheIdempotencyProvider(); -// 3. Infrastructure — database, EF, outbox publisher (for transactional writes inside subscribers) +// 3. Infrastructure — database, relational EF or Cosmos, and (only when outbox-enabled) the default outbox publisher for transactional writes inside subscribers. // SQL Server variant: builder.AddSqlServerClient("SqlServer"); builder.Services .AddSqlServerDatabase() .AddSqlServerUnitOfWork() - .AddSqlServerOutboxPublisher() // outbox publisher becomes the default IEventPublisher + .AddSqlServerOutboxPublisher() // only when outbox-enabled; becomes the default IEventPublisher when registered .AddDbContext() .AddEfDb(); @@ -186,18 +186,34 @@ builder.Services // .AddPostgresDatabase() // .AddPostgresUnitOfWork() // .AddEventFormatter() -// .AddPostgresOutboxPublisher() +// .AddPostgresOutboxPublisher() // only when outbox-enabled; becomes the default IEventPublisher when registered // .AddDbContext() // .AddEfDb(); +// Cosmos variant (use instead of the relational block): +// builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => +// { +// o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; +// if (builder.Environment.IsDevelopment()) +// { +// o.ConnectionMode = ConnectionMode.Gateway; +// o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); +// } +// }); +// builder.Services +// .AddCosmosDb("my-database-id") +// .AddCosmosDbUnitOfWork() +// .AddCosmosDbEventPublisher() // only when outbox-enabled; becomes the default IEventPublisher when registered +// .AddCosmosDbHealthCheck(); + // 4. Azure Service Bus publisher — direct publish capability (not the default IEventPublisher) builder.AddAzureServiceBusClient("ServiceBus"); builder.Services.AddAzureServiceBusPublisher((_, c) => { c.SessionIdStrategy = ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId; -}, addAsDefaultIEventPublisher: false); // false because outbox publisher is already the default +}, addAsDefaultIEventPublisher: false); // false because, when outbox-enabled, the database-backed outbox publisher remains the default -// 5. Event formatter + subscriber manager +// 5. Event formatter + subscriber manager (AddNamedDestinationProvider() is registered with the other CoreEx services: events to the shared topic, commands to `{topic}-{domain}` queues) builder.Services .AddEventFormatter() .AddSubscribedManager((_, c) => c.AddSubscribersUsing()); @@ -209,9 +225,9 @@ builder.Services.AzureServiceBusReceiving() var o = ServiceBusSessionReceiverOptions.CreateForTopicSubscription(); o.SessionProcessorOptions.MaxConcurrentSessions = 4; return o; - }) - .WithSubscribedSubscriber() // routes received messages through the SubscribedManager - .WithHostedService() // runs the receiver as a BackgroundService + }, "receiver-events") // keyed by default so further receivers (e.g. a command queue) can be added alongside + .WithKeyedSubscribedSubscriber("subscriber-events") // routes received messages through the SubscribedManager + .WithHostedService("hosted-subscriber-events") // runs the receiver as a BackgroundService (also the health-check/hosted-service management name) .Build(); // 7. External API clients (if needed — for domains with inter-domain HTTP calls) @@ -228,7 +244,7 @@ builder.Services.AddOpenApiDocument(s => builder.WithCoreExTelemetry() .WithCoreExServiceBusTelemetry() - .WithCoreExSqlServerTelemetry() // or .WithCoreExPostgresTelemetry() for PostgreSQL + .WithCoreExSqlServerTelemetry() // or .WithCoreExPostgresTelemetry() / .WithCoreExCosmosDbTelemetry() for the chosen provider .UseOtlpExporter(); // 9. Build and middleware pipeline @@ -253,17 +269,36 @@ app.Run(); `MapHostedServices()` exposes runtime management endpoints to **pause and resume** the receiver per partition without restarting the process. +## Command Subscribers (Queue) + +A command is addressed to **one** domain and travels on that domain's **queue** (`{CoreEx:Events:Destination}-{domain}`, e.g. `contoso-products`), not the shared event topic. The `NamedDestinationProvider` (`AddNamedDestinationProvider()`) does the routing on the publishing side; the consuming Subscribe host needs its **own** receiver for that queue: + +```csharp +builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForQueue("{destination}-{domain}"), "receiver-commands") + .WithKeyedSubscribedSubscriber("subscriber-commands") + .WithHostedService("hosted-subscriber-commands") + .Build(); +``` + +- Each receiver, subscriber and hosted service needs its own **service key**; the event receiver uses `receiver-events` / `subscriber-events` / `hosted-subscriber-events`. Tests resolve `GetRequiredKeyedService("subscriber-commands")`. +- The queue name must equal the `NamedDestinationProvider` result exactly and must exist (Test.Common `ServiceBus.GetQueues()`, session-enabled; Aspire topology class), otherwise the hosted service fails to start. +- Publisher-side `CoreEx:Host:ServiceBus:Destinations:{destination}` overrides use that exact resolved queue/topic name; the default `NamedDestinationProvider` forms command queues as `{CoreEx:Events:Destination}-{target-domain}`. Session bucket count is not receiver concurrency: tune `MaxConcurrentSessions` independently, and only bucket keys when the ordering/concurrency trade-off is intentional. +- The **consuming domain owns the command contract** and subject (`{parent}.{this-domain}.{entity}.{action}[.v{n}]`). Never subscribe to another domain's command. +- Commands are at-least-once: handlers must be idempotent; map only *expected* outcomes in the `ErrorHandler`. +- Full slice (contract, wiring, topology, tests, Aspire): [`coreex-command-subscribe-e2e`](/.github/skills/coreex-command-subscribe-e2e/SKILL.md). Publishing side: [`coreex-command-publish-e2e`](/.github/skills/coreex-command-publish-e2e/SKILL.md). + ## Do Not - Do not embed business logic in subscriber classes — delegate immediately to an Application-layer service or adapter. - Do not use MediatR or in-process event dispatchers — subscribers react to integration events from the broker only. - Do not manually register subscriber classes in DI — `AddSubscribersUsing()` discovers them automatically via `[ScopedService]`. - Do not omit `AddEventFormatter()` from `Program.cs` — it is required for message parsing and deserialization. -- Do not set `addAsDefaultIEventPublisher: true` for the Service Bus publisher when the outbox publisher is the intended default `IEventPublisher`. +- Do not set `addAsDefaultIEventPublisher: true` for the Service Bus publisher when an outbox publisher is registered (all providers, and only when `outbox-enabled` is true) and is the intended default `IEventPublisher`. ## Further Reading - [Hosts Layer Guide — Subscribe Host](/.github/docs/coreex/hosts-layer.md) — Subscribe host architecture, Program.cs shape, and subscriber patterns (docs-sync cache; after `/coreex-docs-sync`). Source: [samples/docs/hosts-layer.md](https://github.com/Avanade/CoreEx/blob/main/samples/docs/hosts-layer.md). - [Pattern Catalog](/.github/docs/coreex/patterns.md) — Subscribe, Publish, Transactional Outbox, and Event-Driven Replication pattern entries (docs-sync cache; after `/coreex-docs-sync`). Source: [samples/docs/patterns.md](https://github.com/Avanade/CoreEx/blob/main/samples/docs/patterns.md). - [CoreEx.Azure.Messaging.ServiceBus guide](/.github/docs/coreex/agents/CoreEx.Azure.Messaging.ServiceBus.md) — `SubscribedBase`, `ErrorHandler`, and Service Bus receiver configuration (docs-sync cache; after `/coreex-docs-sync`). Source: [CoreEx.Azure.Messaging.ServiceBus README](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Azure.Messaging.ServiceBus/README.md). -- Related skill: [`coreex-subscriber`](/.github/skills/coreex-subscriber/SKILL.md) — invoke to scaffold an event subscriber. +- Related skill: [`coreex-subscriber`](/.github/skills/coreex-subscriber/SKILL.md) — invoke to scaffold a subscriber; [`coreex-command-subscribe-e2e`](/.github/skills/coreex-command-subscribe-e2e/SKILL.md) for a command end-to-end. diff --git a/.github/instructions/coreex-host-setup.instructions.md b/.github/instructions/coreex-host-setup.instructions.md index 6d2309f38..8d2308981 100644 --- a/.github/instructions/coreex-host-setup.instructions.md +++ b/.github/instructions/coreex-host-setup.instructions.md @@ -25,6 +25,27 @@ The host is a **composition root only** — no business logic. There are three h > **Further Reading**: [Hosts Layer Guide](/.github/docs/coreex/hosts-layer.md) · [Layer Dependencies](/.github/docs/coreex/layers.md) · [Pattern Catalog](/.github/docs/coreex/patterns.md) (docs-sync cache; after `/coreex-docs-sync`). Source: [samples/docs/hosts-layer.md](https://github.com/Avanade/CoreEx/blob/main/samples/docs/hosts-layer.md) · [samples/docs/layers.md](https://github.com/Avanade/CoreEx/blob/main/samples/docs/layers.md) · [samples/docs/patterns.md](https://github.com/Avanade/CoreEx/blob/main/samples/docs/patterns.md). +## Service Bus Session Routing + +`CoreEx:Host:ServiceBus:SessionIdPartitionSize` is a publisher-wide default, not a Service Bus physical partition count. When a publisher sends to multiple destinations with different ordering/concurrency profiles, override its strategy or bucket count by the exact resolved destination: + +```json +"CoreEx": { + "Host": { + "ServiceBus": { + "SessionIdPartitionSize": 8, + "Destinations": { + "contoso-products": { + "SessionIdPartitionSize": 2 + } + } + } + } +} +``` + +Use a stable business key with `UsePartitionKeyAsIs` when per-entity ordering is needed. Hashed buckets can make high-cardinality keys manageable, but unrelated keys may collide; more buckets do not increase throughput unless the receiver also allows enough concurrent sessions. Keep receiver concurrency settings separate from publisher session mapping, and keep both separate from the shared relational outbox `CoreEx:Host:Outbox:PartitionSize`. + --- ## Scaffolding an API host @@ -35,7 +56,7 @@ An API host is **not** part of the base `coreex` solution — it is added on dem > 1. **Detect** the host: look for `**/*.Api/*.Api.csproj`. The file system is authoritative for project existence (unlike database state) — no further checking is needed. > 2. **If present**, skip to authoring controllers (see [coreex-api-controllers](./coreex-api-controllers.instructions.md)). > 3. **If absent, confirm creation** with the user — default the name to `{Solution}.Api` and the physical location to `src/` (the template default). Do not create it without confirmation. -> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check `dbex.yaml`). These default the `coreex-api` template — which takes **`data-provider`, `refdata-enabled`, `outbox-enabled`** (a subset of the solution options). Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and `dbex.yaml` disagree, **stop and flag** rather than guessing. +> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check the real provider artefacts — `*.Database/dbex.yaml` for relational domains, `*.Database/Program.cs` for Cosmos domains). These default the `coreex-api` template — which takes **`data-provider`, `refdata-enabled`, `outbox-enabled`** (a subset of the solution options). Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and those provider artefacts disagree, **stop and flag** rather than guessing. > 5. **Scaffold** with the recovered values, naming consistently with the solution so the derived `domain-name`/`solution-name` tokens align with the existing projects: > ``` > dotnet new coreex-api -n {Solution}.Api --data-provider --refdata-enabled --outbox-enabled @@ -66,7 +87,7 @@ The scaffolded `Program.cs` wiring is described below; it is generated **complet > // all hosts — scans the assemblies (via the stable AssemblyMarker types) for [ScopedService] types, registering them all (incl. CodeGen-generated): > builder.Services.AddDynamicServicesUsing(typeof({Solution}.Application.AssemblyMarker).Assembly, typeof({Solution}.Infrastructure.AssemblyMarker).Assembly); > ``` -> The generated `ReferenceDataService` is `[ScopedService]`-decorated, so the assembly scan picks it up automatically and the orchestrator binds to it via `IReferenceDataProvider` — no manual step. Before CodeGen there are simply no ref-data entities to serve (requests return empty), which is correct. **Do not** add `AddReferenceDataOrchestrator()` or `AddDynamicServicesUsing<…generated types…>()` — that reintroduces the compile-time dependency the marker approach removes. +> The generated `ReferenceDataProvider` is `[ScopedService]`-decorated, so the assembly scan picks it up automatically and the orchestrator binds to it via `IReferenceDataProvider` — no manual step. Before CodeGen there are simply no ref-data entities to serve (requests return empty), which is correct. **Do not** add `AddReferenceDataOrchestrator()` or `AddDynamicServicesUsing<…generated types…>()` — that reintroduces the compile-time dependency the marker approach removes. --- @@ -90,7 +111,7 @@ Like the Relay host, the Subscribe host is **fully template-generated**: the `co > 1. **Detect** it: look for `**/*.Subscribe/*.Subscribe.csproj`. The file system is authoritative (unlike database state) — no further checking is needed. > 2. **If present**, skip to authoring subscribers (see [coreex-event-subscribers](./coreex-event-subscribers.instructions.md)). > 3. **If absent, confirm creation** with the user — default the name to `{Solution}.Subscribe` and the physical location to `src/` (the template default). Do not create it without confirmation. -> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check `dbex.yaml`). The `coreex-subscribe` template takes **`data-provider`**, **`messaging-provider`**, and **`refdata-enabled`** — default all three from the recorded `coreex` selections. Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and `dbex.yaml` disagree, **stop and flag** rather than guessing. +> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check the real provider artefacts — `*.Database/dbex.yaml` for relational domains, `*.Database/Program.cs` for Cosmos domains). The `coreex-subscribe` template takes **`data-provider`**, **`messaging-provider`**, and **`refdata-enabled`** — default all three from the recorded `coreex` selections. Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and those provider artefacts disagree, **stop and flag** rather than guessing. > 5. **Scaffold** with the recovered values, naming consistently with the solution so the derived `domain-name`/`solution-name` tokens align with the existing projects: > ``` > dotnet new coreex-subscribe -n {Solution}.Subscribe --data-provider --messaging-provider --refdata-enabled @@ -125,7 +146,7 @@ Unlike the Api host, the Relay is a **one-off, fully template-generated** host: > 1. **Detect** it: look for `**/*.Relay/*.Relay.csproj`. The file system is authoritative (unlike database state) — no further checking is needed. > 2. **If present, STOP — it already exists.** The Relay is a single per-solution one-off; **immediately report it as pre-existing and do nothing else** — do not re-scaffold, modify, or "augment" it. (Contrast the Api host, where an existing host means "go author controllers"; the Relay has **no** follow-on work.) > 3. **If absent, confirm creation** with the user — default the name to `{Solution}.Relay` and the physical location to `src/` (the template default). Do not create it without confirmation. -> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check `dbex.yaml`). The `coreex-relay` template takes **`data-provider`** and **`messaging-provider`** — default **both** from the recorded `coreex` selections. Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and `dbex.yaml` disagree, **stop and flag** rather than guessing. +> 4. **Recover the original selections** from the solution-root `AGENTS.md` "Feature Configuration" (cross-check the real provider artefacts — `*.Database/dbex.yaml` for relational domains, `*.Database/Program.cs` for Cosmos domains). The `coreex-relay` template takes **`data-provider`** and **`messaging-provider`** — default **both** from the recorded `coreex` selections. Re-state the resolved values for confirmation rather than re-prompting; if `AGENTS.md` and those provider artefacts disagree, **stop and flag** rather than guessing. > 5. **Scaffold** with the recovered values, naming consistently with the solution so the derived `domain-name`/`solution-name` tokens align with the existing projects: > ``` > dotnet new coreex-relay -n {Solution}.Relay --data-provider --messaging-provider @@ -158,15 +179,16 @@ The scaffolded Relay `Program.cs` wiring is described in [Outbox Relay Host](#ou | `CoreEx.AspNetCore` | `AddMvcWebApi()`, `AddHttpWebApi()`, `AddExecutionContext()`, `UseCoreExExceptionHandler()`, `UseExecutionContext()`, `UseIdempotencyKey()`, `MapHealthChecks()` | | `CoreEx.AspNetCore.NSwag` | `AddOpenApiDocument()`, `AddCoreExConfiguration()`, `UseOpenApi()`, `UseSwaggerUi()` | | `CoreEx.Caching.FusionCache` | `AddFusionCache()`, `AddFusionHybridCache()`, `AddDefaultCacheKeyProvider()`, `AddHybridCacheIdempotencyProvider()` | -| `CoreEx.Database.SqlServer` | `AddSqlServerDatabase()`, `AddSqlServerUnitOfWork()`, `AddSqlServerOutboxPublisher()`, `AddSqlServerClient("SqlServer")` | -| `CoreEx.Database.Postgres` | `AddPostgresDatabase()`, `AddPostgresUnitOfWork()`, `AddPostgresOutboxPublisher()`, `AddAzureNpgsqlDataSource("Postgres")` | -| `CoreEx.EntityFrameworkCore` | `AddDbContext()`, `AddEfDb()` | +| `CoreEx.Database.SqlServer` | `AddSqlServerDatabase()`, `AddSqlServerUnitOfWork()`, `AddSqlServerOutboxPublisher()` (only when outbox-enabled), `AddSqlServerClient("SqlServer")` | +| `CoreEx.Database.Postgres` | `AddPostgresDatabase()`, `AddPostgresUnitOfWork()`, `AddPostgresOutboxPublisher()` (only when outbox-enabled), `AddAzureNpgsqlDataSource("Postgres")` | +| `CoreEx.Cosmos` | `AddAzureCosmosClient("Cosmos", ...)`, `AddCosmosDb("{db-id}")`, `AddCosmosDbUnitOfWork()`, `AddCosmosDbEventPublisher()` (only when outbox-enabled), `AddCosmosDbHealthCheck()` | +| `CoreEx.EntityFrameworkCore` | `AddDbContext()`, `AddEfDb()` (relational providers only) | | `CoreEx.Events` | `AddEventFormatter()` | | `CoreEx.RefData` | `AddReferenceDataOrchestrator()` | | `CoreEx.Data.GraphQL` | `AddCoreExGraphQLLite((o, sp) => o.AddQuery(...).AddGet(...))`, `MapCoreExGraphQLLite("/query")` — additive, opt-in GraphQL-lite bridge over existing `QueryArgsConfig`-driven `QueryAsync`/`GetAsync` methods; add only when explicitly asked | | `Aspire.StackExchange.Redis.DistributedCaching` | `AddRedisDistributedCache("redis")` | | `FusionCache.Backplane.StackExchangeRedis` | `RedisBackplane`, `RedisBackplaneOptions` | -| `OpenTelemetry.*` | `WithCoreExTelemetry()`, `WithCoreExSqlServerTelemetry()` / `WithCoreExPostgresTelemetry()`, `UseOtlpExporter()` | +| `OpenTelemetry.*` | `WithCoreExTelemetry()`, `WithCoreExSqlServerTelemetry()` / `WithCoreExPostgresTelemetry()` / `WithCoreExCosmosDbTelemetry()`, `UseOtlpExporter()` | ### Subscribe Host @@ -175,9 +197,10 @@ The scaffolded Relay `Program.cs` wiring is described in [Outbox Relay Host](#ou | `CoreEx.AspNetCore` | `AddMvcWebApi()`, `AddHttpWebApi()`, `AddExecutionContext()`, `AddHostedServiceManager()`, `UseCoreExExceptionHandler()`, `UseExecutionContext()`, `MapHealthChecks()`, `MapHostedServices()` | | `CoreEx.Caching.FusionCache` | `AddFusionCache()`, `AddFusionHybridCache()`, `AddDefaultCacheKeyProvider()`, `AddHybridCacheIdempotencyProvider()` | | `CoreEx.Events` | `AddEventFormatter()`, `AddSubscribedManager()` | -| `CoreEx.Database.SqlServer` | `AddSqlServerDatabase()`, `AddSqlServerUnitOfWork()`, `AddSqlServerOutboxPublisher()`, `AddSqlServerClient("SqlServer")` | -| `CoreEx.Database.Postgres` | `AddPostgresDatabase()`, `AddPostgresUnitOfWork()`, `AddPostgresOutboxPublisher()`, `AddAzureNpgsqlDataSource("Postgres")` | -| `CoreEx.EntityFrameworkCore` | `AddDbContext()`, `AddEfDb()` | +| `CoreEx.Database.SqlServer` | `AddSqlServerDatabase()`, `AddSqlServerUnitOfWork()`, `AddSqlServerOutboxPublisher()` (only when outbox-enabled), `AddSqlServerClient("SqlServer")` | +| `CoreEx.Database.Postgres` | `AddPostgresDatabase()`, `AddPostgresUnitOfWork()`, `AddPostgresOutboxPublisher()` (only when outbox-enabled), `AddAzureNpgsqlDataSource("Postgres")` | +| `CoreEx.Cosmos` | `AddAzureCosmosClient("Cosmos", ...)`, `AddCosmosDb("{db-id}")`, `AddCosmosDbUnitOfWork()`, `AddCosmosDbEventPublisher()` (only when outbox-enabled), `AddCosmosDbHealthCheck()` | +| `CoreEx.EntityFrameworkCore` | `AddDbContext()`, `AddEfDb()` (relational providers only) | | `CoreEx.RefData` | `AddReferenceDataOrchestrator()` | | `CoreEx.Azure.Messaging.ServiceBus` | `AddAzureServiceBusClient("ServiceBus")`, `AddAzureServiceBusPublisher(..., addAsDefaultIEventPublisher: false)`, `AzureServiceBusReceiving()`, `WithCoreExServiceBusTelemetry()` | | `Aspire.StackExchange.Redis.DistributedCaching` | `AddRedisDistributedCache("redis")` | @@ -207,7 +230,7 @@ builder.AddHostSettings(); builder.Services .AddPrecisionTimeProvider() .AddExecutionContext() - .AddReferenceDataOrchestrator() // non-generic — binds the IReferenceDataProvider (CodeGen-generated ReferenceDataService) from DI at runtime + .AddReferenceDataOrchestrator() // non-generic — binds the IReferenceDataProvider (CodeGen-generated ReferenceDataProvider) from DI at runtime .AddMvcWebApi() .AddHttpWebApi(); @@ -226,14 +249,14 @@ builder.Services .AddDefaultCacheKeyProvider() .AddHybridCacheIdempotencyProvider(); -// Database, EF, outbox publisher. +// Database, relational EF or Cosmos, and (only when outbox-enabled) the default outbox publisher. // SQL Server variant: builder.AddSqlServerClient("SqlServer"); builder.Services .AddSqlServerDatabase() .AddSqlServerUnitOfWork() .AddEventFormatter() - .AddSqlServerOutboxPublisher() + .AddSqlServerOutboxPublisher() // only when outbox-enabled .AddDbContext() .AddEfDb(); @@ -243,15 +266,32 @@ builder.Services // .AddPostgresDatabase() // .AddPostgresUnitOfWork() // .AddEventFormatter() -// .AddPostgresOutboxPublisher() +// .AddPostgresOutboxPublisher() // only when outbox-enabled // .AddDbContext() // .AddEfDb(); +// Cosmos variant (use instead of the relational block): +// builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => +// { +// o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; +// if (builder.Environment.IsDevelopment()) +// { +// o.ConnectionMode = ConnectionMode.Gateway; +// o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); +// } +// }); +// builder.Services +// .AddCosmosDb("my-database-id") +// .AddEventFormatter() +// .AddCosmosDbEventPublisher() // only when outbox-enabled +// .AddCosmosDbUnitOfWork() +// .AddCosmosDbHealthCheck(); + builder.Services.PostConfigureAllHealthChecks(); builder.Services.AddControllers(); builder.Services.AddOpenApiDocument(s => { s.Title = builder.Environment.ApplicationName; s.AddCoreExConfiguration(); }); -builder.WithCoreExTelemetry().WithCoreExSqlServerTelemetry().UseOtlpExporter(); +builder.WithCoreExTelemetry().WithCoreExSqlServerTelemetry().UseOtlpExporter(); // or .WithCoreExPostgresTelemetry() / .WithCoreExCosmosDbTelemetry() for the chosen provider var app = builder.Build(); app.UseCoreExExceptionHandler(); @@ -268,13 +308,14 @@ app.Run(); ``` Key points: -- **`AddDynamicServicesUsing(...)` registers per _assembly_, not per service.** It scans each supplied assembly for all `[ScopedService]`-decorated types and registers them. The template anchors each assembly on its neutral **`AssemblyMarker`** type (`typeof(MyApp.Application.AssemblyMarker).Assembly`, `…Infrastructure…`) — these are the **only** markers the clean scaffold ships, and they exist **solely** for this anchoring. Use the neutral marker, **not** a domain type like `typeof(ReferenceDataService).Assembly`: the marker reads as "scan this whole assembly," never misleads a reader into thinking the scan is scoped to one type, always exists (so there is no compile-time dependency on CodeGen output and no bootstrap/uncomment step), and survives renames. **Adding a new entity does not change this line** — the new service/repository is picked up automatically by the existing assembly scan. Add another assembly only when you introduce a **new project** containing `[ScopedService]` types (e.g. the Subscribe host passes `typeof(Program).Assembly` for its subscribers). Do **not** add per-namespace placeholder/marker types to satisfy `global using`s — those follow the code (see `coreex-conventions.instructions.md`). -- Prefer the **non-generic `AddReferenceDataOrchestrator()`** (binds `IReferenceDataProvider` from DI at runtime) over `AddReferenceDataOrchestrator()` — the non-generic form needs no generated type, so it compiles from scaffold time. It also auto-registers `ReferenceDataQuery.Default`, enabling `ReferenceDataOrchestrator.QueryAsync(QueryArgs?, PagingArgs?, CancellationToken)` on all ref-data endpoints out of the box. +- **`AddDynamicServicesUsing(...)` registers per _assembly_, not per service.** It scans each supplied assembly for all `[ScopedService]`-decorated types and registers them. The template anchors each assembly on its neutral **`AssemblyMarker`** type (`typeof(MyApp.Application.AssemblyMarker).Assembly`, `…Infrastructure…`) — these are the **only** markers the clean scaffold ships, and they exist **solely** for this anchoring. Use the neutral marker, **not** a domain type like `typeof(ReferenceDataProvider).Assembly`: the marker reads as "scan this whole assembly," never misleads a reader into thinking the scan is scoped to one type, always exists (so there is no compile-time dependency on CodeGen output and no bootstrap/uncomment step), and survives renames. **Adding a new entity does not change this line** — the new service/repository is picked up automatically by the existing assembly scan. Add another assembly only when you introduce a **new project** containing `[ScopedService]` types (e.g. the Subscribe host passes `typeof(Program).Assembly` for its subscribers). Do **not** add per-namespace placeholder/marker types to satisfy `global using`s — those follow the code (see `coreex-conventions.instructions.md`). +- Prefer the **non-generic `AddReferenceDataOrchestrator()`** (binds `IReferenceDataProvider` from DI at runtime) over `AddReferenceDataOrchestrator()` — the non-generic form needs no generated type, so it compiles from scaffold time. It also auto-registers `ReferenceDataQuery.Default`, enabling `ReferenceDataOrchestrator.QueryAsync(QueryArgs?, PagingArgs?, CancellationToken)` on all ref-data endpoints out of the box. - `AddReferenceDataOrchestrator()` and `AddDynamicServicesUsing(...)` are shared with Subscribe hosts — both API and Subscribe hosts are full application-layer consumers. - FusionCache (L1/L2) and `AddHybridCacheIdempotencyProvider()` are shared with Subscribe hosts — both need caching for reference data and idempotency for safe duplicate handling. - `AddEventFormatter()` is required wherever events are published or parsed. -- `AddSqlServerOutboxPublisher()` / `AddPostgresOutboxPublisher()` take no generic type parameter. +- `AddSqlServerOutboxPublisher()` / `AddPostgresOutboxPublisher()` / `AddCosmosDbEventPublisher()` take no domain-type generic parameter. Register them only when `outbox-enabled` is true; otherwise keep the host on its direct-publisher-only path. - `UseIdempotencyKey()` must come **after** `UseExecutionContext()`. +- Generated mutable reference-data controllers apply `[Authorize]` by default. The scaffold's `UseAuthentication()` call is intentionally commented until an authentication scheme is configured; before enabling mutable entities, register the intended authentication scheme and enable authentication middleware. Override `mutableAttribute` with a policy/role for the intended access level, or use `[AllowAnonymous]` only when anonymous writes are an explicit requirement (as in the Products and Customers samples). - If the domain also publishes directly to Service Bus (e.g. for cross-domain adapters), add `AddAzureServiceBusPublisher(..., addAsDefaultIEventPublisher: false)` so the outbox publisher remains the default `IEventPublisher`. - `MapHealthChecks()`'s **basic** `live`/`startup`/`ready` endpoints are intentionally left anonymous — they're conventionally probed by container orchestrators without credentials. Its **detailed** endpoints (`/health/*/detailed`) are different: they emit the full `HealthReport`, which can include component names and exception details, so `HealthCheckOptions.AreDetailedEndpointsEnabled` defaults to `false` (secure by default) and must be explicitly opted into, as shown above. Once opted in, they should also be secured — pass `detailedGroupConfigure: g => g.RequireAuthorization()`, **commented out by default** (as shown above) since it 500s until an authentication scheme and authorization services are registered; `MapHealthChecks` logs a `Warning` at startup if detailed endpoints are enabled without it. Uncomment alongside `UseAuthentication()` once a scheme is configured. - **`CoreEx.Data.GraphQL` is additive, not part of the base scaffold** — add it only when explicitly asked for a GraphQL query surface; use the `coreex-graphql` skill to wire it (`.github/skills/coreex-graphql/SKILL.md`) rather than improvising. Register roots with `builder.Services.AddCoreExGraphQLLite((o, sp) => o.AddQuery("products", ProductQueryArgsConfig.Default, async (qa, pa, ct) => await CoreEx.ExecutionContext.GetRequiredService().QueryAsync(qa, pa, ct).ConfigureAwait(false)).AddGet("product", (args, ct) => CoreEx.ExecutionContext.GetRequiredService().GetAsync(args.GetIdentifier(), ct)))` — `GetIdentifier` validates presence and type of the named argument (default `"id"`) — it casts to `TId`, it does not convert — and throws an `ArgumentException`, mapped by the engine to `ARGUMENT_ERROR`, if it's missing/empty/wrong-typed — then `app.MapCoreExGraphQLLite("/query");` after `app.MapControllers();`. Resolve scoped dependencies (repositories, application services) per-invocation via `CoreEx.ExecutionContext.GetRequiredService()` — never capture an instance from the root `IServiceProvider` at registration time, since `IGraphQLEngine` is a singleton. It bridges GraphQL `where`/`orderBy` 1:1 onto the entity's existing `QueryArgsConfig` — no new resolver/filter logic. `MapCoreExGraphQLLite` executes through `WebApi.PostAsync(...)`, so standard exception-handling middleware applies as a safety net; add `.WithCoreExGraphQLTelemetry()` alongside the host's other OpenTelemetry tracing extensions to trace `ExecuteAsync` calls. To expose all reference data types as GraphQL roots in one call, use `o.AddReferenceDataQueries(sp, ReferenceDataQueryArgsConfig.Default)` — this bulk-registers every type known to the `ReferenceDataOrchestrator`, keyed as `ref_` where `` is the type's `AlternateNames` entry where registered, otherwise its own `Type.Name`; pass `excludeTypes` to opt specific types out. See [`CoreEx.Data.GraphQL` AGENTS.md](https://github.com/Avanade/CoreEx/blob/main/src/CoreEx.Data.GraphQL/AGENTS.md). @@ -318,7 +359,7 @@ builder.AddSqlServerClient("SqlServer"); builder.Services .AddSqlServerDatabase() .AddSqlServerUnitOfWork() - .AddSqlServerOutboxPublisher() + .AddSqlServerOutboxPublisher() // only when outbox-enabled .AddDbContext() .AddEfDb(); @@ -327,10 +368,26 @@ builder.Services // builder.Services // .AddPostgresDatabase() // .AddPostgresUnitOfWork() -// .AddPostgresOutboxPublisher() +// .AddPostgresOutboxPublisher() // only when outbox-enabled // .AddDbContext() // .AddEfDb(); +// Cosmos variant (use instead of the relational block): +// builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => +// { +// o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; +// if (builder.Environment.IsDevelopment()) +// { +// o.ConnectionMode = ConnectionMode.Gateway; +// o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); +// } +// }); +// builder.Services +// .AddCosmosDb("my-database-id") +// .AddCosmosDbEventPublisher() // only when outbox-enabled +// .AddCosmosDbUnitOfWork() +// .AddCosmosDbHealthCheck(); + // Service Bus: keep outbox publisher as the default IEventPublisher. builder.AddAzureServiceBusClient("ServiceBus"); builder.Services.AddAzureServiceBusPublisher((_, c) => @@ -349,9 +406,9 @@ builder.Services.AzureServiceBusReceiving() var o = ServiceBusSessionReceiverOptions.CreateForTopicSubscription(); o.SessionProcessorOptions.MaxConcurrentSessions = 4; return o; - }) - .WithSubscribedSubscriber() - .WithHostedService() + }, "receiver-events") + .WithKeyedSubscribedSubscriber("subscriber-events") + .WithHostedService("hosted-subscriber-events") .Build(); builder.Services.PostConfigureAllHealthChecks(); @@ -360,7 +417,7 @@ builder.Services.AddOpenApiDocument(s => { s.Title = builder.Environment.Applica builder.WithCoreExTelemetry() .WithCoreExServiceBusTelemetry() - .WithCoreExSqlServerTelemetry() // or .WithCoreExPostgresTelemetry() for PostgreSQL + .WithCoreExSqlServerTelemetry() // or .WithCoreExPostgresTelemetry() / .WithCoreExCosmosDbTelemetry() for the chosen provider .UseOtlpExporter(); var app = builder.Build(); @@ -380,10 +437,10 @@ app.Run(); Key points: - Subscribe hosts **do** include `AddReferenceDataOrchestrator()` and `AddDynamicServicesUsing(...)` — subscribers call application services that need reference data for validation and business logic. The Subscribe host passes **three** assemblies to `AddDynamicServicesUsing`: its own (`typeof(Program).Assembly`, for the subscriber types) plus the Application and Infrastructure `AssemblyMarker` assemblies. - Subscribe hosts **do** include FusionCache (L1/L2) and `AddHybridCacheIdempotencyProvider()` — caching is required for reference data; idempotency is required to safely handle duplicate message delivery. -- Subscribe hosts **do** include database/EF Core and outbox publisher — subscribers persist domain data and publish outbound events. +- Subscribe hosts **do** include domain persistence — relational hosts use EF Core; Cosmos hosts use `CosmosDb` — and, when `outbox-enabled` is true, the provider-specific outbox publisher as the default `IEventPublisher`. - `AddHostedServiceManager()` must be registered before `AzureServiceBusReceiving()`. - `AddSubscribersUsing()` scans the assembly of `T` and auto-registers all `[Subscribe]`-decorated classes — no manual registration per subscriber. -- `AddAzureServiceBusPublisher(..., addAsDefaultIEventPublisher: false)` keeps the outbox publisher as the default `IEventPublisher` for transactional writes. +- `AddAzureServiceBusPublisher(..., addAsDefaultIEventPublisher: false)` keeps the outbox publisher as the default `IEventPublisher` for transactional writes when one is registered; the broker publisher stays available for explicit/direct paths. - `MapHostedServices()` must come **after** `MapHealthChecks()`. - `MapHealthChecks()`'s **detailed** endpoints and `MapHostedServices()`'s pause/resume endpoints are both intended to be secured via `RequireAuthorization()` (both admin/diagnostic surfaces), but the calls above are **commented out by default** — they 500 until an authentication scheme is registered. Uncomment alongside `UseAuthentication()`. The basic `live`/`startup`/`ready` health checks stay anonymous for orchestrator probes. `HealthCheckOptions.AreDetailedEndpointsEnabled` defaults to `false` (secure by default), so it must be explicitly opted into as shown above; `MapHealthChecks` logs a `Warning` at startup if left enabled without `detailedGroupConfigure`. See the equivalent API host bullet for the reasoning. @@ -391,7 +448,7 @@ Key points: ## Outbox Relay Host -The Outbox Relay host is minimal: it polls the outbox table and forwards committed events to Azure Service Bus. It has **no application logic** — no controllers, no OpenAPI, no FusionCache, no reference data, no EF Core DbContext. It only needs database connectivity to read the outbox table and Service Bus connectivity to publish. +The Outbox Relay host is minimal: it reads committed outbox events and forwards them to Azure Service Bus. It has **no application logic** — no controllers, no OpenAPI, no FusionCache, no reference data, no EF Core DbContext. Relational providers poll the outbox table; Cosmos uses a Change Feed Processor over the outbox-hosting container. It only needs provider-specific database connectivity plus Service Bus connectivity to publish. ```csharp builder.Services @@ -418,6 +475,21 @@ builder.AddSqlServerOutboxRelayHostedService(); // .AddPostgresOutboxRelay(); // builder.AddPostgresOutboxRelayHostedService(); +// Cosmos variant: +// builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => +// { +// o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; +// if (builder.Environment.IsDevelopment()) +// { +// o.ConnectionMode = ConnectionMode.Gateway; +// o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); +// } +// }); +// builder.Services +// .AddCosmosDb("my-database-id") +// .AddCosmosDbHealthCheck(); +// builder.AddCosmosDbOutboxRelayHostedService("ref-data"); + // Service Bus publisher — this IS the default IEventPublisher for the relay. builder.AddAzureServiceBusClient("ServiceBus"); builder.Services.AddAzureServiceBusPublisher((_, c) => @@ -427,7 +499,7 @@ builder.Services.AddAzureServiceBusPublisher((_, c) => builder.Services.PostConfigureAllHealthChecks(); -builder.WithCoreExTelemetry().WithCoreExSqlServerTelemetry().WithCoreExServiceBusTelemetry().UseOtlpExporter(); +builder.WithCoreExTelemetry().WithCoreExSqlServerTelemetry().WithCoreExServiceBusTelemetry().UseOtlpExporter(); // or .WithCoreExPostgresTelemetry() / .WithCoreExCosmosDbTelemetry() for the chosen provider var app = builder.Build(); app.UseCoreExExceptionHandler(); @@ -441,9 +513,9 @@ app.Run(); ``` Key points: -- The Relay host has **no application-layer dependencies** — no `AddReferenceDataOrchestrator`, no `AddDynamicServicesUsing`, no FusionCache, no EF Core DbContext, no domain services. -- `AddSqlServerOutboxRelay()` / `AddPostgresOutboxRelay()` take no configuration lambda. -- `AddSqlServerOutboxRelayHostedService()` / `AddPostgresOutboxRelayHostedService()` register the background relay pump — call these on `builder`, not `builder.Services`. +- The Relay host has **no application-layer dependencies** — no `AddReferenceDataOrchestrator`, no `AddDynamicServicesUsing`, no FusionCache, no EF Core DbContext, no domain services. The Cosmos variant likewise stays host-only: register `AddCosmosDb(...)` for access plus the relay hosted service, but never a `CosmosDbEventPublisher` (the relay publishes to Service Bus, not back into Cosmos). +- `AddSqlServerOutboxRelay()` / `AddPostgresOutboxRelay()` take no configuration lambda. Cosmos relay hosts do not register a separate relay service in `builder.Services`; they register `AddCosmosDb(...)` and `AddCosmosDbHealthCheck()` on `builder.Services`, then the hosted service on `builder`. +- `AddSqlServerOutboxRelayHostedService()` / `AddPostgresOutboxRelayHostedService()` / `AddCosmosDbOutboxRelayHostedService("")` register the background relay pump — call these on `builder`, not `builder.Services`. Cosmos relay instances are Change Feed Processor-based, use a shared `$outbox-leases` container that they never create (declare it in the Database project with `.OutboxLeaseContainer()`; startup fails fast if missing), and use `CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:ServicesCount` for scaling. - `UseAuthentication()` / `UseAuthorization()` are present but **commented out** — the Relay host (unlike API/Subscribe) has no `AddControllers()` or other MVC registration, so calling `UseAuthorization()` without first registering `builder.Services.AddAuthorization()` throws at startup (confirmed: `AddControllers()` in the API/Subscribe hosts registers authorization services transitively, which is why they can call it directly). Uncomment both together, once a scheme and policy are registered, so the `detailedGroupConfigure`/`groupConfigure` calls below can also be uncommented and take effect. - `MapHealthChecks()`'s **basic** `live`/`startup`/`ready` endpoints are intentionally left anonymous for container-orchestrator probes; its **detailed** endpoints emit the full `HealthReport` (component names, exception details), so `HealthCheckOptions.AreDetailedEndpointsEnabled` defaults to `false` (secure by default) and is explicitly opted into above, and should be secured via `detailedGroupConfigure: g => g.RequireAuthorization()`. `MapHostedServices()`'s pause/resume admin endpoints should be secured the same way via `groupConfigure` (its remarks call this out as "highly recommended"). Both `detailedGroupConfigure` and `groupConfigure` are **commented out by default** here — same as `UseAuthentication()`/`UseAuthorization()` above — since they'd otherwise 500 every request until an authentication scheme is actually registered; uncomment alongside those. - No `AddControllers()`, no `AddOpenApiDocument()`, no `UseOpenApi()`, no `UseSwaggerUi()`, no `UseIdempotencyKey()`. diff --git a/.github/instructions/coreex-repositories.instructions.md b/.github/instructions/coreex-repositories.instructions.md index 09c1eed87..46bf7ad15 100644 --- a/.github/instructions/coreex-repositories.instructions.md +++ b/.github/instructions/coreex-repositories.instructions.md @@ -1,7 +1,7 @@ --- applyTo: "**/Infrastructure/**/*.cs" -description: "Repository and infrastructure conventions: EFCore, mapping, typed HTTP clients, adapter implementations, and data-access patterns" -tags: ["repositories", "infrastructure", "data-access", "efcore", "mapping", "adapters"] +description: "Repository and infrastructure conventions: EFCore/Cosmos, mapping, typed HTTP clients, adapter implementations, and data-access patterns" +tags: ["repositories", "infrastructure", "data-access", "efcore", "cosmos", "mapping", "adapters"] --- + +Guide this workspace through adding or extending Aspire AppHost wiring and Test.Aspire end-to-end tests. + +Use `.github/skills/coreex-aspire/SKILL.md` and its referenced workflow as the authoritative workflow. + +Operational contract: +- Read the solution-root `AGENTS.md` **Feature Configuration** first; inspect the existing `aspire/` projects before asking anything. +- Decide every HTTP call outside the domain explicitly: an in-AppHost domain uses `WithReference` (no stub); a third party or out-of-AppHost domain uses the MockHost with `WithMockHostEnvironment` and `app.HttpMock` stubs. +- Use the `OnBeforeStartAsync` / `OnAfterStartAsync` lifecycle with `app.*` helpers; poll asynchronous effects with a bounded loop. +- If any prompt text conflicts with the skill, the skill wins. + +Outcome: +- The AppHost, service-bus topology and Test.Aspire E2E are wired and the outside-the-domain HTTP strategy is explicit and tested. diff --git a/.github/prompts/coreex-command-publish-e2e.prompt.md b/.github/prompts/coreex-command-publish-e2e.prompt.md new file mode 100644 index 000000000..711a0f2d9 --- /dev/null +++ b/.github/prompts/coreex-command-publish-e2e.prompt.md @@ -0,0 +1,25 @@ +--- +description: Add the sending/publishing of a command to another domain end-to-end — adapter, transactional call site, queue routing, Service Bus topology, API and Relay tests, and Aspire E2E in one guided workflow +--- + + + +Guide this workspace through adding the publishing of a command (not an event) to another domain end-to-end. + +Use `.github/skills/coreex-command-publish-e2e/SKILL.md` and its referenced workflow as the authoritative workflow. + +Operational contract: +- Read the solution-root `AGENTS.md` **Feature Configuration** first (`messaging-provider`, `outbox-enabled`, `data-provider` — `SqlServer` / `Postgres` / `Cosmos` / `None`, `rop-enabled`). +- Verify every publishing host uses `AddNamedDestinationProvider()` — `AddFixedDestinationProvider` would send commands to the shared topic. +- Batch all interview questions into one turn: target domain and location, command entity/action, key-only or payload, trigger, outbox vs direct, outbound HTTP. +- Invoke `coreex-adapter`, `coreex-app-service`, add the `{Destination}-{target}` queue to Test.Common `ServiceBus`, then `coreex-test-api`, `coreex-test-relay` and `coreex-aspire`. +- If any prompt text conflicts with the skill, the skill wins. + +Outcome: +- The command is built in an adapter, emitted inside the unit of work, routed to the target domain queue, and covered by API, Relay and (where applicable) Aspire tests. diff --git a/.github/prompts/coreex-command-subscribe-e2e.prompt.md b/.github/prompts/coreex-command-subscribe-e2e.prompt.md new file mode 100644 index 000000000..0102c14f0 --- /dev/null +++ b/.github/prompts/coreex-command-subscribe-e2e.prompt.md @@ -0,0 +1,25 @@ +--- +description: Add the receiving/handling of a command end-to-end — contract, app service, subscriber, dedicated keyed command-queue receiver, Service Bus topology, Subscribe tests and Aspire E2E in one guided workflow +--- + + + +Guide this workspace through adding the subscribing to (handling of) a command (not an event) end-to-end. + +Use `.github/skills/coreex-command-subscribe-e2e/SKILL.md` and its referenced workflow as the authoritative workflow. + +Operational contract: +- Read the solution-root `AGENTS.md` **Feature Configuration** first (`messaging-provider`, `outbox-enabled`, `data-provider` — `SqlServer` / `Postgres` / `Cosmos` / `None`, `rop-enabled`). +- Inspect the Subscribe host: if the event receiver is unkeyed, retrofit it to the keyed pattern before adding the command receiver. +- Batch all interview questions into one turn: command entity/action, key-only or payload, triggered action, persistence, expected error outcomes, publisher location, outbound HTTP. +- Invoke `coreex-contract` (if payload), `coreex-db-migration` + `coreex-repository` (if new state), `coreex-app-service`, `coreex-subscriber`, wire `receiver-commands` / `subscriber-commands` / `hosted-subscriber-commands`, add the queue to Test.Common `ServiceBus`, then `coreex-test-subscribe` and `coreex-aspire`. +- If any prompt text conflicts with the skill, the skill wins. + +Outcome: +- The command handler is registered with a dedicated command-queue receiver in the Subscribe host, the queue exists in the topology, and success, expected-error and unsubscribed tests pass. diff --git a/.github/prompts/coreex-contract.prompt.md b/.github/prompts/coreex-contract.prompt.md index 7de747e2d..427dc6a13 100644 --- a/.github/prompts/coreex-contract.prompt.md +++ b/.github/prompts/coreex-contract.prompt.md @@ -19,6 +19,7 @@ Operational contract: - `[Contract]` + `partial` on all contract classes by default; only omit when explicitly asked. - Only `[ReferenceData]`-decorated properties are `partial` — never mark plain properties `partial` (CS9248). - `[ReadOnly(true)]` on all server-assigned or derived fields. +- `[NonNullable]` on mandatory/required properties that are declared nullable (`string?`, `int?`) for DTO flexibility, so OpenAPI reports them as non-nullable. Never on `Id`/`ETag`/`ChangeLog`; on other `[ReadOnly]` properties only when the type is never a request body. - Apply `[Schema]` only when the user explicitly requests a custom event schema name or version. - Never hand-author generated members or create/edit `*.g.cs` files. - If any prompt text conflicts with the skill, the skill wins. diff --git a/.github/prompts/coreex-repository.prompt.md b/.github/prompts/coreex-repository.prompt.md index b5b7c6cfc..cc9b81eb3 100644 --- a/.github/prompts/coreex-repository.prompt.md +++ b/.github/prompts/coreex-repository.prompt.md @@ -1,5 +1,5 @@ --- -description: Create or modify a CoreEx Infrastructure-layer repository — EfDb CRUD delegates, QueryArgsConfig queries, BiDirectionMapper, and Result pipelines +description: Create or modify a CoreEx Infrastructure-layer repository — EfDb/CosmosDb CRUD delegates, QueryArgsConfig queries, BiDirectionMapper, and Result pipelines --- + +# CoreEx: Aspire Wiring & E2E + +Makes a code change *visible and testable* in the Aspire environment: AppHost resources and references, MockHost stubs for anything outside the AppHost, the code-based Service Bus topology, and a cross-host E2E test. Aspire is where **inter-domain dependencies are real** — everything intra-domain tests deliberately mock is exercised here. + +## When to Use + +- A new Api / Relay / Subscribe host must join the AppHost +- A change introduces a **Service Bus queue** (e.g. a command queue) that the E2E topology must create +- A new queue/topic needs session requirements or a destination-specific producer session profile +- A host now calls **another domain's HTTP API** (needs service discovery) or a **third-party API** (needs a MockHost stub) +- A cross-host flow needs an automated E2E test (`*.Test.Aspire`) +- Invoked by [`coreex-command-publish-e2e`](../coreex-command-publish-e2e/SKILL.md) and [`coreex-command-subscribe-e2e`](../coreex-command-subscribe-e2e/SKILL.md) for their final step + +## When Not to Use + +- Running / stopping / inspecting a live AppHost (logs, traces, resource status) — use the `aspire` skill +- Intra-domain tests — `coreex-test-api`, `coreex-test-subscribe`, `coreex-test-relay` +- Generating the AppHost for the first time — `coreex-scaffold` (`dotnet new coreex-aspire`) +- Writing the typed HTTP client / adapter — `coreex-adapter` (this skill only wires it into Aspire) + +> **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` **Feature Configuration** +> (`data-provider`, `messaging-provider`, `outbox-enabled`) and inspect the existing `*.Aspire/AppHost.cs`, `*.Aspire.MockHost` +> and `*.Test.Aspire` before interviewing. Re-state resolved values for confirmation. + +## The Deciding Question — where does the callee live? + +| Callee | Mechanism | Stub in Test.Aspire? | +|---|---|---| +| Another domain **in the same AppHost** | `.WithReference(calleeApi)` on each caller + `BaseAddress = https+http://{resource-name}` | **No** — real call | +| A **third-party** API | MockHost + `.WithMockHostEnvironment("Section__BaseAddress", mockhost, "http")` | **Yes** — `app.HttpMock(...)` in `OnAfterStartAsync` | +| Another domain **outside** this AppHost | Same as third-party | **Yes** | + +Get this wrong and the E2E either hides a real integration (stubbing an in-AppHost domain) or hits the internet (not stubbing a third party). + +## Workflow Overview + +1. **Inspect** — read AppHost, MockHost, Test.Aspire, Test.Common `ServiceBus`, the calling host's `Program.cs` + `appsettings.Development.json`. +2. **Interview** — what changed; where each callee lives; which async effect the E2E should prove. +3. **AppHost** — add/adjust visible external resources via `CoreEx.UnitTesting`'s `AddExternalConnectionString`, `AddProject` + `WithReference` (+ `ProjectReference`); service-discovery `BaseAddress`; MockHost wiring. +4. **Topology** — add queues/topics/subscriptions to the `ServiceBus` class used by Test.Aspire (union class if multi-domain); mark session-enabled entities with `RequiresSession = true` and check the producer's exact destination-specific session profile. +5. **Test.Aspire** — `OnBeforeStartAsync` (migrate, cache, broker reset) / `OnAfterStartAsync` (wait, stubs); E2E test with bounded polling. +6. **Validate** — `dotnet build` the AppHost + Test.Aspire; `dotnet test` when infrastructure is up. + +For step-by-step guidance see [`references/workflow.md`](references/workflow.md). + +## Key References + +- [`/.github/instructions/coreex-aspire.instructions.md`](/.github/instructions/coreex-aspire.instructions.md) — AppHost / MockHost / Test.Aspire invariants +- [`/.github/instructions/coreex-tests.instructions.md`](/.github/instructions/coreex-tests.instructions.md) — intra-domain test conventions (what Aspire deliberately does *not* re-cover) +- [`/.github/instructions/coreex-host-setup.instructions.md`](/.github/instructions/coreex-host-setup.instructions.md) — `AddTypedHttpClient`, receivers, `Program.cs` +- Related skills: [`coreex-adapter`](../coreex-adapter/SKILL.md) (typed HTTP client), [`coreex-scaffold`](../coreex-scaffold/SKILL.md) (AppHost scaffold) +- [Aspire & End-to-End](/.github/docs/coreex/aspire.md) — optional (after `/coreex-docs-sync`) +- Illustrative examples (CoreEx sample — not present in your project): [`AppHost.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Aspire/AppHost.cs), [`E2ETest.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Test.Aspire/E2ETest.cs), [`ServiceBus.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/aspire/Contoso.Test.Aspire/ServiceBus.cs) diff --git a/.github/skills/coreex-aspire/references/workflow.md b/.github/skills/coreex-aspire/references/workflow.md new file mode 100644 index 000000000..77d43401b --- /dev/null +++ b/.github/skills/coreex-aspire/references/workflow.md @@ -0,0 +1,188 @@ + + +# coreex-aspire: Workflow + +--- + +## Phase 1 — Inspect (before asking anything) + +Read, in this order: + +1. Solution-root `AGENTS.md` **Feature Configuration** — `data-provider`, `messaging-provider`, `outbox-enabled`. +2. `aspire/*.Aspire/AppHost.cs` — existing resources, `WithReference` chains, whether `mockhost` is declared or commented out. +3. `aspire/*.Aspire/*.csproj` — `ProjectReference`s (a host missing here has no `Projects.X`). +4. `aspire/*.Test.Aspire/` — the test class, its hooks, and which `ServiceBus` class it uses (`Common.ServiceBus` for a single-domain solution; a local union class for a multi-domain AppHost). +5. The calling host's `Program.cs` (`AddTypedHttpClient<...>("Section")`, receivers) and `appsettings*.json` (`BaseAddress`, `QueueOrTopicName`, and any `CoreEx:Host:ServiceBus:Destinations` profile). + +If there is **no** `aspire/` folder, stop: the AppHost does not exist. Offer `dotnet new coreex-aspire` via `coreex-scaffold`; do not hand-roll one. + +## Phase 2 — Interview (one batched turn) + +| # | Question | Gates | +|---|---|---| +| 1 | What changed? new host / new queue / new inter-domain HTTP call / new third-party HTTP call / new E2E scenario (any combination) | Which of Phases 3–5 run | +| 2 | For each HTTP callee: is it a domain **in this AppHost**, a **third-party** API, or a domain **outside** this AppHost? | Service discovery vs MockHost (see the deciding table in `SKILL.md`) | +| 3 | For a third-party/out-of-AppHost callee: which routes does the flow hit, and what should each return? | The `HttpMock` stubs | +| 4 | Which asynchronous effect proves the flow (e.g. "reservation status becomes Confirmed")? | The polling assertion | + +Do not ask what Phase 1 already answered. + +## Phase 3 — AppHost + +### External infrastructure + +Use **`CoreEx.UnitTesting`'s** `AddExternalConnectionString` helper in the `UnitTestEx` namespace, alongside +`AddEndpoints` and the other Aspire helpers. Keep existing connection names and references: + +```csharp +var db = builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint").WithIconName("DatabaseMultiple"); +var redis = builder.AddExternalConnectionString("redis").WithIconName("Database"); +var serviceBus = builder.AddExternalConnectionString("ServiceBus").WithIconName("MailMultiple"); +``` + +For relational providers use `"Postgres"` or `"SqlServer"` without `endpointKey`. The helper wraps secret +parameters from `ConnectionStrings:{name}` in visible connection-string nodes. Plain `AddConnectionString(name)` +is parameter-only and absent from Aspire 13.5.4's Graph/Table views. For older AppHosts, update the pinned +`CoreEx.UnitTesting` package to a release containing the helper, then change the declarations manually; +do not regenerate with `--force` or add a duplicate local implementation. + +Compose retains lifecycle ownership. Node status means configuration resolved; hosts still perform readiness +probes. Derive Cosmos's URL from `AccountEndpoint`, not a hard-coded emulator port, and never expose credentials +in URLs. Set explicit relational ports (`5432`/`1433` for default local services). Dashboard trace attribution +uses endpoint attributes, not provider names; inspect `server.address`, `server.port`, and `peer.service` if a span +is mislabelled. Secret backing parameters can also participate in matching, so do not promise that graph visibility +alone fixes all trace labels. + +### New host + +```csharp +builder.AddProject("billing-subscribe") + .WithReference(db).WithReference(redis).WithReference(serviceBus) // only what the host needs + .AddEndpoints("/health/ready/detailed") + .AddHostedServiceSupport(); // relay + subscribe hosts +``` + +Add the `` to the AppHost csproj first, otherwise `Projects.X` does not exist. The resource name (`billing-subscribe`) becomes the name used in every test — choose it once. + +### Inter-domain HTTP callee **in this AppHost** + +1. On **every** calling host in the AppHost: `.WithReference(calleeApi)`. +2. In each calling host's `appsettings.Development.json`, point the typed client's section at the *resource name*: + +```json +"ProductsApi": { "BaseAddress": "https+http://products-api" } +``` + +3. Client registration is unchanged: `builder.AddTypedHttpClient("ProductsApi");` + +No MockHost, no stub — the call is real. + +### Third-party / out-of-AppHost HTTP callee + +1. Uncomment (or add once) the MockHost resource: `var mockhost = builder.AddMockHostProject("mock-host");` +2. On **every** calling host: `.WithMockHostEnvironment("SendGrid__BaseAddress", mockhost, "http")`. The key must equal the configuration path the typed client reads (`"SendGrid"` section → `SendGrid__BaseAddress`). +3. Confirm any other value the typed client factory reads at startup (API keys, etc.) resolves in the host's `appsettings.Development.json` — a missing key throws before the request reaches the mock. +4. Stubs are added in Test.Aspire (Phase 5), not in `MockHost/Program.cs` — the MockHost stays the one-liner `WireMockConsole.RunAsync(...)`. + +## Phase 4 — Service Bus topology + +Edit the `ServiceBus` class Test.Aspire resets against: + +```csharp +public static CreateQueueOptions[]? GetQueues() => +[ + new CreateQueueOptions("contoso-products") { RequiresSession = true } // one per command-receiving domain: {Destination}-{domain} +]; +``` + +- A **single-domain** solution reuses `Common.ServiceBus` from `Test.Common` (already correct for its own topic/subscription); add the queue there once and Relay/Subscribe/Aspire tests all see it. +- A **multi-domain** AppHost owns a local union class listing every domain's topic, subscriptions and command queues. Update it whenever any domain adds an entity. +- Queue name = the `NamedDestinationProvider` result: `{CoreEx:Events:Destination}-{target-domain-lowercase}`. It must equal the consumer's `ServiceBusSessionReceiverOptions.CreateForQueue(...)` string exactly. +- If publishing to a session-enabled entity, configure any `CoreEx:Host:ServiceBus:Destinations:{destination}` override in each host that publishes there (including the Relay); the key must exactly match the resolved destination. The session bucket count is producer-side key grouping, while `MaxConcurrentSessions` is consumer-side concurrency. Set `RequiresSession = true` in topology to match session-based publishing/receiving. +- Entities are recreated by `ResetAzureServiceBusAsync` in `OnBeforeStartAsync`, so the emulator `Config.json` can stay empty. + +## Phase 5 — Test.Aspire + +```csharp +public class E2ETest : WithAspireTester +{ + protected override async Task OnBeforeStartAsync(DistributedApplication app) + { + await app.MigratePostgresDataAsync("Postgres", ["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs); // or MigrateSqlServerDataAsync / MigrateCosmosDataAsync provider-specifically + await app.ClearRedisCacheAsync("redis"); + await app.ResetAzureServiceBusAsync("ServiceBus", ServiceBus.GetQueues(), ServiceBus.GetTopicsAndSubscriptions()); + } + + protected override async Task OnAfterStartAsync(DistributedApplication app) + { + await app.WaitForResourceAsync(["billing-api", "billing-subscribe"]); + // Third-party / out-of-AppHost callees only: + await app.HttpMock("mock-host", "http").Request(HttpMethod.Post, "/v3/mail/send").WithAnyBody().Respond.WithAsync(HttpStatusCode.Accepted); + } +} +``` + +- Hooks, not `[OneTimeSetUp]`. `Test.*` intra-domain helpers do not exist on the Aspire tester; use `app.*`. For Cosmos, always go through `app.MigrateCosmosDataAsync(..., DbMigration.ConfigureProvisionArgs)` rather than resetting containers manually, so the declared container topology and the client state stay aligned. +- Add `using`/`global using` for `Aspire.Hosting` (`DistributedApplication`) in the project's single `GlobalUsing.cs`. +- Stubs use `WithAnyBody()` unless the body matters; assert stub usage only when the flow's correctness depends on the request shape. + +### The E2E test + +Drive the flow from its real entry point over HTTP, then **poll** for the async effect: + +```csharp +[Test] +public void Checkout_ConfirmsReservation() +{ + Test.Checkpoint("Checkout the basket; the reservation is confirmed asynchronously via the outbox, Service Bus and the Products Subscribe host."); + var basket = Test.Http("shopping-api").Run(HttpMethod.Post, $"/api/baskets/{basketId}/checkout").AssertOK().Value!; + + var movements = GetMovements(basket.Id!); + for (var attempt = 1; movements.Any(m => m.Status != MovementStatus.Confirmed); attempt++) + { + if (attempt > 10) Assert.Fail("The reservation was not confirmed after 10 attempts."); + Test.Delay(1000, $"Waiting for the reservation to be confirmed; iteration {attempt}."); + movements = GetMovements(basket.Id!); + } + + // Third-party stub only: poll the MockHost's WireMock journal for a call to *this* route addressed to *this* recipient. + var api = await Test.HttpMock("mock-host", "http").GetAdminApiAsync().ConfigureAwait(false); + for (var attempt = 1; !await SendReceivedAsync(api, email).ConfigureAwait(false); attempt++) + { + if (attempt > 30) Assert.Fail($"No mail-send request addressed to '{email}' was received after 30 attempts."); + Test.Delay(1000, $"Waiting for the mail-send request; iteration {attempt}."); + } +} + +private static async Task SendReceivedAsync(WireMock.Client.IWireMockAdminApi api, string email) +{ + var requests = await api.GetRequestsAsync().ConfigureAwait(false); + return requests.Any(r => r.Request?.Path == "/v3/mail/send" && r.Request.Body?.Contains(email, StringComparison.OrdinalIgnoreCase) == true); +} +``` + +- Bounded loop of `Test.Delay(ms, reason)` (e.g. 10 x 1 s), failing with `Assert.Fail` — never an unbounded wait or an unexplained `Thread.Sleep`. +- `Test.Checkpoint(...)` narrates each step in the output; use it as the sample E2E does. +- `Test.HttpMock("mock-host", "http").GetAdminApiAsync()` + `GetRequestsAsync()` inspects the WireMock request journal to prove a stubbed third-party route was really called. Match on the request `Path` **and** a body fragment unique to the flow (e.g. the recipient) — path alone proves nothing once other flows hit the same stub. This replaces log scraping (`AssertLogContains`), which is fragile. The test method becomes `async Task`. +- Assert the **effect in the consuming host** (via its API), not the broker: the Aspire tester has no peek helper. Broker delivery of a specific message is a `*.Test.Relay` concern. +- Where the consumer is *outside* the AppHost there is no effect to poll — keep the E2E to publishing health (host ready, relay hosted service healthy) and leave delivery to `Test.Relay`. + +## Phase 6 — Validate + +1. `dotnet build` the AppHost and `*.Test.Aspire` projects — 0 errors/warnings. +2. With infrastructure up (`docker compose up -d`) run `dotnet test aspire/*.Test.Aspire`. If infrastructure is unavailable, say so explicitly and report the build result only. +3. Confirm each `WithMockHostEnvironment` key has a matching configuration section and a stub, and that no in-AppHost domain is stubbed. + +## Guardrails + +- **Never stub an in-AppHost domain.** Aspire exists to run those for real. +- **Never let a test reach a real third party.** No `WithMockHostEnvironment` + no stub means an outbound internet call. +- **Every calling host** of a mocked/discovered callee needs the reference/environment — Api *and* Subscribe when both call it. +- **Queue names are contracts** between publisher config, consumer receiver and the topology class — they must be identical. +- **One MockHost.** Reuse `mock-host`; add stubs, not hosts. diff --git a/.github/skills/coreex-bootstrap/SKILL.md b/.github/skills/coreex-bootstrap/SKILL.md index 7c3a3cdd5..075f3cb23 100644 --- a/.github/skills/coreex-bootstrap/SKILL.md +++ b/.github/skills/coreex-bootstrap/SKILL.md @@ -87,6 +87,7 @@ Use `/coreex-expert` when you need CoreEx architecture guidance before choosing @.github/instructions/coreex-conventions.instructions.md @.github/instructions/coreex-contracts.instructions.md @.github/instructions/coreex-application-services.instructions.md +@.github/instructions/coreex-aspire.instructions.md @.github/instructions/coreex-validators.instructions.md @.github/instructions/coreex-repositories.instructions.md @.github/instructions/coreex-api-controllers.instructions.md @@ -103,5 +104,5 @@ Use `/coreex-expert` when you need CoreEx architecture guidance before choosing | `/coreex-bootstrap` | Generates root guidance files for a blank CoreEx repo, then chains to `/coreex-scaffold` | | `/coreex-scaffold` | Guided interview → derives and runs the right `dotnet new coreex*` commands for this solution shape | | `/coreex-expert` | Architecture guidance, pattern decisions, and sample-aligned implementation advice | -| `/coreex-docs-sync` | Fetches and caches CoreEx architecture docs and per-package AI guides under `.github/docs/coreex/` | +| `/coreex-docs-sync` | Refreshes the entire AI asset bundle and docs cache from the CoreEx.Template release matching this project's CoreEx NuGet version | ``` diff --git a/.github/skills/coreex-command-publish-e2e/SKILL.md b/.github/skills/coreex-command-publish-e2e/SKILL.md new file mode 100644 index 000000000..6e154e2bb --- /dev/null +++ b/.github/skills/coreex-command-publish-e2e/SKILL.md @@ -0,0 +1,71 @@ +--- +name: coreex-command-publish-e2e +description: "Add the sending/publishing of a command to another domain end-to-end: adapter (interface + Infrastructure implementation + wire DTO), application-service call site (outbox vs direct broker), destination routing (NamedDestinationProvider -> per-domain queue), Service Bus topology in Test.Common, API/Relay tests, and Aspire E2E. USE FOR: a domain must instruct another domain to do something (a command, not an event) and the developer wants code, infrastructure, tests and Aspire in one guided workflow. DO NOT USE FOR: publishing an event (fire-and-forget; use coreex-app-service — events go to the shared topic), receiving a command (use coreex-command-subscribe-e2e), or a synchronous request/response call to another domain (use coreex-adapter with a typed HTTP client)." +argument-hint: "Optional: target domain, command entity + action (e.g. products reservation confirm), payload or key-only, triggering operation, and whether the target domain is in this AppHost" +tags: ["coreex", "command", "publish", "outbox", "service-bus", "queue", "end-to-end", "vertical-slice", "aspire", "messaging"] +--- + + + +# CoreEx: Publish a Command (End-to-End) + +Guides you through making one domain **send a command** to another: the adapter that builds it, the transactional call site, the routing to the target's queue, the broker topology, the tests and the Aspire E2E — gathered in one interview so nothing is asked twice. + +## Events vs Commands — decide first + +| | **Event** | **Command** | +|---|---|---| +| Intent | "This happened" — reactive, fire-and-forget | "Do this" — addressed to **one** consumer | +| Built with | `EventData.CreateEventWith(...)` | `EventData.CreateCommand("{target-domain}", "{entity}", "{action}")` | +| Destination (`NamedDestinationProvider`) | Shared **topic** (`CoreEx:Events:Destination`) — every domain subscribes | **Queue** `{Destination}-{target-domain}` (e.g. `contoso-products`) — only the target sees it | +| Contract owner | Publisher | **Target** domain (`DomainName` = the owner of the message contract) | + +If the developer means an event, stop and use `coreex-app-service` — nothing on this page applies. + +## When to Use + +- "Shopping must tell Products to confirm the reservation" — a domain instructs another domain +- A compensating command (cancel/undo) must be sent, possibly bypassing the outbox +- The developer wants code + broker topology + tests + Aspire covered in one pass + +## When Not to Use + +- Publishing an event — `coreex-app-service` +- Receiving/handling a command — `coreex-command-subscribe-e2e` +- A synchronous call that needs an answer — `coreex-adapter` (typed HTTP client) + +## Workflow Overview + +1. **Read Feature Configuration** — `messaging-provider` must be `ServiceBus`; `outbox-enabled` decides outbox vs direct; `rop-enabled` decides `Result` vs exceptions. +2. **Verify routing** — `AddNamedDestinationProvider()` is registered in every host that publishes; migrate away from `AddFixedDestinationProvider` (it would send commands to the shared topic). +3. **Interview** — target domain, command shape, payload, trigger, outbox vs direct, target location; one batched turn. +4. **Execute L1 sequence** (table below). +5. **Topology + tests + Aspire** — queue in Test.Common `ServiceBus`, API/Relay tests, `coreex-aspire`. +6. **Validate** — build, unit/integration tests, report. + +For full step-by-step guidance see [`references/workflow.md`](references/workflow.md). + +## L1 Sequence + +| Step | Skill | Condition | +|---|---|---| +| 1 | [`coreex-adapter`](../coreex-adapter/SKILL.md) | Always — the adapter interface + implementation that builds the `EventData` command; the wire DTO for a payload lives in `Infrastructure/Clients/{Target}/` | +| 2 | [`coreex-app-service`](../coreex-app-service/SKILL.md) | Always — the call site inside `_unitOfWork.TransactionAsync(...)` (or the compensation path) | +| 3 | *(inline — Service Bus topology)* | Always — add the queue to Test.Common `ServiceBus.GetQueues()` | +| 4 | [`coreex-test-api`](../coreex-test-api/SKILL.md) | Always — assert the outbox row: destination queue, subject, key/value | +| 5 | [`coreex-test-relay`](../coreex-test-relay/SKILL.md) | Always — prove the relay delivers the command to the queue | +| 6 | [`coreex-aspire`](../coreex-aspire/SKILL.md) | When an `aspire/` folder exists — topology, consumer wiring, E2E | + +## Key References + +- [`/.github/instructions/coreex-application-services.instructions.md`](/.github/instructions/coreex-application-services.instructions.md) — unit of work, adapters +- [`/.github/instructions/coreex-repositories.instructions.md`](/.github/instructions/coreex-repositories.instructions.md) — adapter/client implementation conventions +- [`/.github/instructions/coreex-tests.instructions.md`](/.github/instructions/coreex-tests.instructions.md) — outbox assertions, relay tests, `Test.ResetAzureServiceBusAsync` +- [`/.github/instructions/coreex-aspire.instructions.md`](/.github/instructions/coreex-aspire.instructions.md) — Aspire wiring +- Illustrative examples (CoreEx sample — not present in your project): [`ProductAdapter`](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Shopping.Infrastructure/Adapters/Products/ProductAdapter.cs) (`CreateConfirmReservationCommand`, `CancelReservationAsync`), [`BasketService.CheckoutAsync`](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Shopping.Application/BasketService.cs), [relay queue test](https://github.com/Avanade/CoreEx/blob/main/samples/tests/Contoso.Shopping.Test.Relay/RelayTests.cs) diff --git a/.github/skills/coreex-command-publish-e2e/references/workflow.md b/.github/skills/coreex-command-publish-e2e/references/workflow.md new file mode 100644 index 000000000..8869524ad --- /dev/null +++ b/.github/skills/coreex-command-publish-e2e/references/workflow.md @@ -0,0 +1,150 @@ + + +# coreex-command-publish-e2e: Workflow + +Full step-by-step workflow for making a domain **send a command** to another domain, end-to-end. All context is gathered upfront so no L1 skill re-asks a question already answered here. + +--- + +## Phase 1 — Read Feature Configuration + +Read the solution-root `AGENTS.md` **Feature Configuration** block and cross-check against reality. + +| Feature | How it shapes the sequence | +|---|---| +| `messaging-provider` | Must be `ServiceBus`. Anything else: stop and flag — commands rely on broker queues. | +| `outbox-enabled` | `true` → the command is added to the default (outbox) `IEventPublisher`. `false` → the direct broker publisher is the default; there is no atomic guarantee. | +| `rop-enabled` | Exception flow vs `Result` at the call site. | +| `data-provider` | Selects the outbox helpers used by the tests (`ExpectSqlServerOutboxEvents` vs `ExpectPostgresOutboxEvents` vs `ExpectCosmosDbOutboxEvents`). | + +Then verify **routing** (this is the most common trap): + +- Every host that publishes commands (Api, Subscribe) must call `AddNamedDestinationProvider()`. **`AddFixedDestinationProvider` sends everything — including commands — to the one shared topic**; if found, replace it (events are unaffected, they still go to `CoreEx:Events:Destination`). +- `CoreEx:Events:Destination` and the host's `DomainName` (from `IHostSettings`) must be set. The command queue name is derived as `{Destination}-{target-domain}` (used verbatim, so mind the case) — e.g. `contoso` + `products` → `contoso-products`. +- The relay needs **no** destination provider: the destination is persisted in the outbox row when the command is added. +- If the session routing profile differs from the publisher-wide default, configure `CoreEx:Host:ServiceBus:Destinations:{exact-queue-name}` in each host that can publish the command directly and in the Relay that publishes its outbox row. `SessionIdPartitionSize` controls producer key-to-session bucketing; the target receiver's `MaxConcurrentSessions` is independent. + +--- + +## Phase 2 — Interview + +One batched turn. Restate anything already resolved rather than re-asking. + +| # | Question | Why needed | +|---|---|---| +| 1 | Which **domain receives** the command, and is it in **this solution / AppHost** or external? | Queue name; Aspire wiring; whether a consumer exists to E2E against | +| 2 | Command **entity and action** (e.g. `reservation` / `confirm`) | `EventData.CreateCommand("{target}", "{entity}", "{action}")`; subject `{parent}.{target}.{entity}.{action}` | +| 3 | **Key-only** (`WithKey(id)`) or a **payload**? If payload: fields | Payload DTO in `Infrastructure/Clients/{Target}/`; subject `.v1` suffix when a payload exists | +| 4 | What **triggers** it — which application-service operation? Must it commit atomically with a database write? | Outbox (default) vs direct publisher | +| 5 | Is there a **compensating** command (undo/cancel on failure)? | Direct-to-broker path bypassing the outbox | +| 6 | Does the sending flow also make **synchronous HTTP calls** to the target or a third party? | Aspire wiring rules (see Phase 5) | + +The **contract of the command is owned by the target domain**. If the target is in this solution, read its Contracts/Subscribe project for the agreed subject and payload; if external, agree the subject/payload with its owner and record it. + +--- + +## Phase 3 — L1 Sequence + +### Step 1 — Adapter (`coreex-adapter`) + +Invoke [`coreex-adapter`](../coreex-adapter/SKILL.md) for: + +- **Interface** `IXxxAdapter` in `Application/Adapters/{Target}/` with a domain-idiomatic method, e.g. `CreateConfirmReservationCommand(string reservationId)` (synchronous, returns `void`/`Result` — it only *adds* to the publisher) and, if needed, an async `CancelReservationAsync(...)`. +- **Implementation** in `Infrastructure/Adapters/{Target}/`. Build the command with `EventData.CreateCommand("{target}", "{entity}", "{action}").WithKey(id)` (or `CreateCommandWith` / `CreateCommand` for a payload) and add it to the injected `IEventPublisher`. +- **Payload DTO** (if any) in `Infrastructure/Clients/{Target}/` — the wire schema stays out of Application (anti-corruption). +- `WithKey(id)` / `WithPartitionKey(...)` is what gives per-entity ordering: the command queue is session-enabled, so a key/partition key is effectively mandatory. + +Outbox vs direct: + +| Path | Inject | Use | +|---|---|---| +| **Outbox** (default) | `IEventPublisher` | The command is part of the unit of work: committed with the DB write or not at all. | +| **Direct** | `[FromKeyedServices("AzureServiceBus")] IEventPublisher` | Compensation after a failure where the transaction has already rolled back; publish, then `PublishAsync(CancellationToken.None)` so cancellation of the request doesn't abandon the undo. | + +Never call `ServiceBusSender`/`HttpClient` directly from a service. + +### Step 2 — Call site (`coreex-app-service`) + +Invoke [`coreex-app-service`](../coreex-app-service/SKILL.md). Call the adapter **inside** `_unitOfWork.TransactionAsync(...)` alongside the mutation, so the outbox row and the data commit atomically. Do not publish the command before the transaction, nor after it (loses atomicity). The compensating path sits in the failure branch, outside the (rolled-back) transaction. + +### Step 3 — Service Bus topology (inline) + +In Test.Common `ServiceBus.GetQueues()` add a **session-enabled** queue for the target: + +```csharp +public static CreateQueueOptions[]? GetQueues() => +[ + new CreateQueueOptions("{destination}-{target}") { RequiresSession = true } +]; +``` + +- The name must equal the `NamedDestinationProvider` result exactly; a mismatch surfaces as the relay failing to send. +- Legacy solutions that still use the emulator `servicebus/Config.json`: add the queue there and restart the emulator. +- This makes `Test.ResetAzureServiceBusAsync()` (Relay/Subscribe/Aspire tests) create it. The **target's** own Test.Common defines the receiving side — see `coreex-command-subscribe-e2e`. + +### Step 4 — API test (`coreex-test-api`) + +Invoke [`coreex-test-api`](../coreex-test-api/SKILL.md). For the triggering operation assert: + +```csharp +.ExpectSqlServerOutboxEvents(e => e.AssertMetadata("{destination}-{target}", "{parent}.{target}.{entity}.{action}", id)); // ExpectPostgresOutboxEvents / ExpectCosmosDbOutboxEvents provider-specifically. +``` + +- The first argument is the **queue** destination (not the topic) — this is the assertion that proves routing. +- Direct/compensating publishes: `.ExpectAzureServiceBusEvents(...)`. +- Any outbound HTTP from the same flow is mocked: `MockHttpClientFactory.Create()`, `mcf.CreateClient("{ClientName}").Request(...)`, `Test.ReplaceHttpClientFactory(mcf)`. Intra-domain dependencies are real; the target domain is **always** mocked here. +- Add the failure case: the compensating command is emitted (and the outbox command is not). + +### Step 5 — Relay test (`coreex-test-relay`) + +Invoke [`coreex-test-relay`](../coreex-test-relay/SKILL.md). The relay host test publishes via the outbox, lets the live relay poll, then reads the queue: + +```csharp +var msgs = await Test.GetAndClearAzureServiceBusAsync(ServiceBusSessionReceiverOptions.CreateForQueue("{destination}-{target}")); +``` + +Assert subject and body. If the queue is missing from `ServiceBus.GetQueues()`, this test fails — that is the intended guard. + +--- + +## Phase 4 — Aspire (`coreex-aspire`) + +When an `aspire/` folder exists, invoke [`coreex-aspire`](../coreex-aspire/SKILL.md): + +1. Add the queue to the Aspire topology class (`Contoso.Test.Aspire/ServiceBus.cs` equivalent), session-enabled, same name. +2. Target **in the AppHost**: nothing else to wire for the broker; the E2E asserts the **effect in the target domain** (e.g. `GET` on the target API until the entity shows the confirmed state — bounded loop, not a fixed delay). There is no broker-peek helper on the Aspire tester; delivery is proven by the Relay test. +3. Target **outside the AppHost**: only the relay-to-broker leg can be proven; document that the consumer is out of scope, or stub it. + +## Phase 5 — HTTP calls outside the domain (only if Q6 = yes) + +| Callee | Wiring | +|---|---| +| A domain **in the AppHost** | `.WithReference(theApi)` on the calling hosts; `BaseAddress` = `https+http://{resource-name}` in `appsettings.Development.json`; no stub. | +| A **third party** or a domain **not in the AppHost** | Add the MockHost project; `.WithMockHostEnvironment("{Section}__BaseAddress", mockhost, "http")` on the calling host; stub each route in `OnAfterStartAsync` with `app.HttpMock("mock-host", "http").Request(...).Respond...`. | + +Full detail and examples: [`/.github/instructions/coreex-aspire.instructions.md`](/.github/instructions/coreex-aspire.instructions.md). + +--- + +## Phase 6 — Validate + +1. `dotnet build` the solution (warnings are errors). +2. Run the API tests for the triggering endpoint and the Relay test project. +3. Run the Aspire test only if Docker/Podman infrastructure is available; otherwise say it was not run. +4. Report: files created, queue name, subject, outbox vs direct, what was **not** verified. + +## Common Pitfalls + +| Symptom | Cause | +|---|---| +| Command lands on the shared topic | Host still uses `AddFixedDestinationProvider` | +| Relay logs "entity not found" / send failure | Queue missing from `ServiceBus.GetQueues()` / emulator config or name mismatch | +| Command processed out of order | No `WithKey`/`WithPartitionKey`; queue is session-based | +| Command emitted even though the DB write failed | Adapter called outside `TransactionAsync` | +| Command never consumed | Target Subscribe host has no command receiver — see `coreex-command-subscribe-e2e` | diff --git a/.github/skills/coreex-command-subscribe-e2e/SKILL.md b/.github/skills/coreex-command-subscribe-e2e/SKILL.md new file mode 100644 index 000000000..221a0145c --- /dev/null +++ b/.github/skills/coreex-command-subscribe-e2e/SKILL.md @@ -0,0 +1,73 @@ +--- +name: coreex-command-subscribe-e2e +description: "Add the receiving/handling of a command end-to-end: command contract (owned here), optional persistence, application service, command subscriber, the dedicated keyed queue receiver + subscriber + hosted service in the Subscribe host, Service Bus topology in Test.Common, Subscribe tests, and Aspire E2E. USE FOR: this domain must accept and act on a command addressed to it (queue `{Destination}-{this-domain}`) and the developer wants code, host wiring, infrastructure, tests and Aspire in one guided workflow. DO NOT USE FOR: subscribing to an event (use coreex-subscriber-e2e), sending a command (use coreex-command-publish-e2e), or subscribing to another domain's command (never valid — a command is consumed only by its owning domain)." +argument-hint: "Optional: command entity + action (e.g. reservation confirm), payload or key-only, the action it triggers, whether state is persisted, and who publishes it (in this AppHost or external)" +tags: ["coreex", "command", "subscribe", "subscriber", "service-bus", "queue", "end-to-end", "vertical-slice", "aspire", "messaging"] +--- + + + +# CoreEx: Subscribe to a Command (End-to-End) + +Guides you through making a domain **receive and act on a command**: the contract it owns, the handler, the **dedicated command-queue receiver** in the Subscribe host, the broker topology, the tests and the Aspire E2E — all context gathered once. + +## Events vs Commands — decide first + +| | **Event subscriber** (`coreex-subscriber-e2e`) | **Command subscriber** (this skill) | +|---|---|---| +| Source | Shared **topic**, this domain's subscription | This domain's **queue** `{Destination}-{this-domain}` (e.g. `contoso-products`) | +| Contract owner | The publisher | **This domain** — it defines subject + payload; publishers conform | +| Host wiring | Receiver `receiver-events` → `subscriber-events` → `hosted-subscriber-events` | **Second** receiver `receiver-commands` → `subscriber-commands` → `hosted-subscriber-commands` | +| Failure semantics | Usually tolerant (other domains' events you don't care about are ignored) | Explicit `ErrorHandler` for expected business outcomes; never silently drop a command | + +A domain **never** subscribes to another domain's command queue. If the message is addressed to someone else, stop. + +## When to Use + +- "Products must handle a `reservation confirm` command sent by Shopping" +- Adding a command handler to a Subscribe host that already handles events (or has no command queue yet) +- The developer wants contract + handler + wiring + topology + tests + Aspire in one pass + +## When Not to Use + +- Reacting to an event — `coreex-subscriber-e2e` +- Sending a command — `coreex-command-publish-e2e` +- Creating the Subscribe host itself — `coreex-scaffold` + +## Workflow Overview + +1. **Read Feature Configuration** — `messaging-provider`, `data-provider` (`SqlServer` / `Postgres` / `Cosmos` / `None`), `rop-enabled`, `outbox-enabled`. +2. **Inspect the host** — is the existing event receiver already keyed? Is there already a command receiver? (Retrofit rules in the workflow.) +3. **Interview** — command shape, action, persistence, publisher location; one batched turn. +4. **Execute L1 sequence** (table below). +5. **Host wiring + topology + tests + Aspire.** +6. **Validate** — build, Subscribe tests, report. + +For full step-by-step guidance see [`references/workflow.md`](references/workflow.md). + +## L1 Sequence + +| Step | Skill | Condition | +|---|---|---| +| 1 | [`coreex-contract`](../coreex-contract/SKILL.md) | Only if the command carries a payload (a key-only command needs no contract) | +| 2 | [`coreex-db-migration`](../coreex-db-migration/SKILL.md) + [`coreex-repository`](../coreex-repository/SKILL.md) | Only if the command persists new state (`data-provider ≠ None`) | +| 3 | [`coreex-validator`](../coreex-validator/SKILL.md) / [`coreex-policy`](../coreex-policy/SKILL.md) | Only if the payload/state needs validating | +| 4 | [`coreex-app-service`](../coreex-app-service/SKILL.md) | Always — the action the command triggers | +| 5 | [`coreex-subscriber`](../coreex-subscriber/SKILL.md) | Always — `[Subscribe("{parent}.{this-domain}.{entity}.{action}[.v1]")]` | +| 6 | *(inline — host wiring + topology)* | Always — command receiver in `Program.cs`; queue in Test.Common `ServiceBus` | +| 7 | [`coreex-test-subscribe`](../coreex-test-subscribe/SKILL.md) | Always — keyed `subscriber-commands` tests | +| 8 | [`coreex-aspire`](../coreex-aspire/SKILL.md) | When an `aspire/` folder exists | + +## Key References + +- [`/.github/instructions/coreex-event-subscribers.instructions.md`](/.github/instructions/coreex-event-subscribers.instructions.md) — subscriber conventions, keyed receivers, subject naming +- [`/.github/instructions/coreex-host-setup.instructions.md`](/.github/instructions/coreex-host-setup.instructions.md) — Subscribe host `Program.cs` +- [`/.github/instructions/coreex-tests.instructions.md`](/.github/instructions/coreex-tests.instructions.md) — Subscribe test conventions +- [`/.github/instructions/coreex-aspire.instructions.md`](/.github/instructions/coreex-aspire.instructions.md) — Aspire wiring +- Illustrative examples (CoreEx sample — not present in your project): [`ReservationConfirmSubscriber`](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Products.Subscribe/Subscribers/ReservationConfirmSubscriber.cs), [Products.Subscribe `Program.cs`](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Products.Subscribe/Program.cs) (two keyed receivers) diff --git a/.github/skills/coreex-command-subscribe-e2e/references/workflow.md b/.github/skills/coreex-command-subscribe-e2e/references/workflow.md new file mode 100644 index 000000000..cb3d811a5 --- /dev/null +++ b/.github/skills/coreex-command-subscribe-e2e/references/workflow.md @@ -0,0 +1,147 @@ + + +# coreex-command-subscribe-e2e: Workflow + +Full step-by-step workflow for making a domain **receive and handle a command**, end-to-end. All context is gathered upfront so no L1 skill re-asks a question already answered here. + +--- + +## Phase 1 — Read Feature Configuration + +Read the solution-root `AGENTS.md` **Feature Configuration** block and cross-check against reality. + +| Feature | How it shapes the sequence | +|---|---| +| `messaging-provider` | Must be `ServiceBus` (commands need a broker queue). Otherwise stop and flag. | +| `data-provider` | `None` → no migration/repository. Otherwise it selects the persistence path (`DbEx`/EF vs Cosmos provisioning/containers) and the outbox helpers in tests. | +| `rop-enabled` | Exception vs `Result` in the app service. | +| `outbox-enabled` | If the handler itself emits events, they go through the outbox inside the unit of work. | + +## Phase 2 — Inspect the Subscribe host + +Open `*.Subscribe/Program.cs` and determine: + +| Finding | Action | +|---|---| +| Event receiver is **keyed** (`receiver-events` / `subscriber-events` / `hosted-subscriber-events`) | Add the command receiver alongside it. | +| Event receiver is **unkeyed** (`WithSessionReceiver(...)`, `WithSubscribedSubscriber()`, `WithHostedService()` with no key) | **Retrofit** it to the keyed pattern first — two receivers cannot share the default registration. Add the keys `receiver-events`, `subscriber-events`, `hosted-subscriber-events`, then update **every existing test** to `GetRequiredKeyedService("subscriber-events")`, and any health-check/`/hosted-services/...` references. | +| A command receiver already exists | Just add the subscriber; skip the wiring step. | +| `AddNamedDestinationProvider()` missing | Add it (replacing any `AddFixedDestinationProvider`) if this host publishes anything. | + +## Phase 3 — Interview + +One batched turn. Restate resolved values. + +| # | Question | Why needed | +|---|---|---| +| 1 | Command **entity and action** (e.g. `reservation` / `confirm`)? | Subject `{parent}.{this-domain}.{entity}.{action}`; class name `{Entity}{Action}Subscriber` | +| 2 | **Key-only** or a **payload**? If payload: fields + validation rules | Contract in `*.Contracts`; `.v1` subject suffix; `ValueValidator` | +| 3 | What **action** does it trigger — new or existing app-service operation? | Step 4 | +| 4 | Does it **persist new state**? | Steps 2/3 (migration + repository) | +| 5 | Which **outcomes are expected, not errors** (e.g. "already applied", "not found because it expired")? | `ErrorHandler` mapping; tests | +| 6 | Who **publishes** it — a domain in this AppHost, or external? Does the handler make **outbound HTTP calls**? | Aspire E2E shape and wiring | + +Subject rule: `{parent}.{domain}.{entity}.{action}` (e.g. `contoso.products.reservation.confirm`); append `.v1` only when there is a payload. The domain segment **must be this domain** — it is the owner. + +## Phase 4 — L1 Sequence + +1. **Contract** ([`coreex-contract`](../coreex-contract/SKILL.md)) — payload only. The command's contract is owned and versioned here; publishers conform to it. +2. **Migration + repository** ([`coreex-db-migration`](../coreex-db-migration/SKILL.md), [`coreex-repository`](../coreex-repository/SKILL.md)) — only when new state is persisted. +3. **Validator/policy** ([`coreex-validator`](../coreex-validator/SKILL.md), [`coreex-policy`](../coreex-policy/SKILL.md)) — payload validation via `ValueValidator` in the subscriber; I/O guards via a policy. +4. **App service** ([`coreex-app-service`](../coreex-app-service/SKILL.md)) — the action; mutation + any events inside `_unitOfWork.TransactionAsync(...)`. **Commands must be idempotent**: delivery is at-least-once, so re-applying an already-applied command must succeed (or map to `CompleteAsInformation`). +5. **Subscriber** ([`coreex-subscriber`](../coreex-subscriber/SKILL.md)) — `[ScopedService, Subscribe("...")]` on a `SubscribedBase` (or `SubscribedBase`). Set `ErrorHandler` for the expected outcomes from Q5, e.g.: + +```csharp +internal static readonly ErrorHandler DefaultErrorHandler = new ErrorHandler() + .Add(ex => ex.ErrorCode == "pending-reservation-not-found" ? ErrorHandling.CompleteAsInformation : null); +``` + + Everything not mapped is a real failure and must be retried/dead-lettered — do not blanket-swallow. Ensure the subscriber's assembly is covered by `AddSubscribersUsing()` and `AddDynamicServicesUsing()` in `Program.cs`. + +## Phase 5 — Host wiring (inline) + +Add the second receiver in `*.Subscribe/Program.cs`, after the event receiver. The queue string **must equal** the `NamedDestinationProvider` result `{CoreEx:Events:Destination}-{this-domain}`: + +```csharp +// Commands: the '{destination}-{domain}' queue (addressed to this domain only via the NamedDestinationProvider). +builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => + { + var o = ServiceBusSessionReceiverOptions.CreateForQueue("{destination}-{domain}"); + o.SessionProcessorOptions.MaxConcurrentSessions = 4; + return o; + }, "receiver-commands") + .WithKeyedSubscribedSubscriber("subscriber-commands") + .WithHostedService("hosted-subscriber-commands") + .Build(); +``` + +- The hosted-service key is also the **health-check name** and the `/hosted-services//...` pause/resume name — keep it descriptive. +- The queue must be session-enabled in topology. Its publisher may use `CoreEx:Host:ServiceBus:Destinations:{exact-queue-name}` for a destination-specific session strategy or bucket count; that producer-side setting is independent of this receiver's `MaxConcurrentSessions`. +- A missing queue makes the hosted service fail to start (health check goes unhealthy) — that is the symptom to expect if topology is skipped. +- The receiving Subscribe host does **not** need an outbound publisher unless the handler emits events. + +### Topology + +In this domain's Test.Common `ServiceBus.GetQueues()`: + +```csharp +new CreateQueueOptions("{destination}-{domain}") { RequiresSession = true } +``` + +(Legacy: `servicebus/Config.json` + emulator restart.) Sessions are required because publishers key commands for per-entity ordering. + +## Phase 6 — Subscribe tests (`coreex-test-subscribe`) + +Invoke [`coreex-test-subscribe`](../coreex-test-subscribe/SKILL.md). Per command: + +- **Success**: build the command, resolve the **keyed** subscriber, receive it, assert the state change / outbox events: + +```csharp +var ed = EventData.CreateCommand("{domain}", "{entity}", "{action}").WithKey(id); +var sbm = Test.CreateCloudEventFrom(ed).ToServiceBusReceivedMessage(); +var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); +var r = await sbs.ReceiveAsync(sbm); +r.IsSuccess.Should().BeTrue(); +``` + +- **Expected-outcome** case per `ErrorHandler` mapping (e.g. not found → `IsSuccess` with information handling). +- **Validation failure** if there is a payload. +- **Idempotency**: receive the same command twice. +- **Unsubscribed** (once per host): an unmatched subject fails with `EventSubscriberHandledException` / `ErrorHandling.CompleteAsSilent` and message `No subscriber matched the event.` +- Any outbound HTTP from the handler is mocked with `MockHttpClientFactory` + `Test.ReplaceHttpClientFactory(mcf)`. +- Event receiver tests use `"subscriber-events"`; command tests use `"subscriber-commands"`. Do not mix. +- Optionally assert the hosted service and health-check keys exist. + +## Phase 7 — Aspire (`coreex-aspire`) + +When an `aspire/` folder exists, invoke [`coreex-aspire`](../coreex-aspire/SKILL.md): + +1. Add the queue (session-enabled) to the Aspire topology class, same name. +2. The Subscribe host must be in the AppHost with `WithReference` to the service bus and its database (existing pattern). +3. **Publisher in the AppHost**: the E2E drives the publishing operation over HTTP and polls the **effect in this domain** (bounded loop) — that proves outbox → relay → queue → subscriber end to end. +4. **Publisher external**: cannot E2E the origin; drive the effect only if a stub/simulated send is feasible, otherwise state the limitation. +5. **Handler makes outbound HTTP calls**: in-AppHost domain → `WithReference` + `https+http://{resource}`; third party → MockHost + `WithMockHostEnvironment(...)` + `app.HttpMock(...)` stubs. See [`/.github/instructions/coreex-aspire.instructions.md`](/.github/instructions/coreex-aspire.instructions.md). + +## Phase 8 — Validate + +1. `dotnet build` (warnings are errors). +2. Run the Subscribe test project. Run Aspire only if Docker/Podman infra is available; otherwise say so. +3. Report: subject, queue name, keys, files created, and anything not verified. + +## Common Pitfalls + +| Symptom | Cause | +|---|---| +| `InvalidOperationException` resolving `ServiceBusSubscribedSubscriber` | Unkeyed lookup after keying the receivers — use the key | +| Command hosted service unhealthy / fails to start | Queue not created (Test.Common/emulator/Aspire topology) or name mismatch | +| Commands never arrive | Publisher still uses `AddFixedDestinationProvider` (sent to the topic), or queue name differs from the provider result | +| "No subscriber matched the event." | Subject domain/entity/action mismatch, or the subscriber's assembly isn't registered in `AddSubscribersUsing` | +| Command handled twice with side effects | Handler not idempotent | +| Subscribing to `{other-domain}` command | Never valid — coordinate with the owning domain instead | diff --git a/.github/skills/coreex-contract/SKILL.md b/.github/skills/coreex-contract/SKILL.md index 6357be4a5..72d02f407 100644 --- a/.github/skills/coreex-contract/SKILL.md +++ b/.github/skills/coreex-contract/SKILL.md @@ -23,7 +23,7 @@ Guides you through creating or modifying a hand-authored contract (DTO/entity) i - New subordinate, line-item, or request/response contract (accessed via a parent — may still carry its own identifier) - Adding or changing properties on an existing contract - Wiring a ref-data navigation property (`[ReferenceData]`) on an existing contract -- Extracting a shared base class when ≥2 contracts repeat the same fields +- Extracting a shared base class when ≥2 contracts repeat the same fields **and** every shared field is meaningful on each (genuine superset) — not for patch/subset request contracts ## When Not to Use @@ -46,8 +46,10 @@ Guides you through creating or modifying a hand-authored contract (DTO/entity) i - `[Contract]` + `partial` on **all** contract classes by default - Only `[ReferenceData]` properties are `partial` — never make plain properties `partial` (CS9248) - `[ReadOnly(true)]` on all server-assigned fields (`Id`, `ETag`, `ChangeLog`, computed/derived) +- `[NonNullable]` on any property that is mandatory/required (ultimately not nullable) but declared nullable (`string?`, `int?`) for DTO flexibility — drives `nullable: false` in OpenAPI; not the same as `System.Diagnostics.CodeAnalysis.NotNull`. Never on `Id`/`ETag`/`ChangeLog`; on other `[ReadOnly]` properties only when the type is never a request body - Every property needs a `` XML doc comment - Same contract type for both API response and event payload — never split them +- Patch/subset contracts (merge-patch, partial-update requests): standalone class with only the needed fields (+ `IETag`) — never inherit a sibling and suppress the rest; inheritance is for genuine supersets only For full workflow, decision trees, and code examples see [`references/workflow.md`](references/workflow.md). diff --git a/.github/skills/coreex-contract/references/workflow.md b/.github/skills/coreex-contract/references/workflow.md index b4ae6b43a..5b606bcc1 100644 --- a/.github/skills/coreex-contract/references/workflow.md +++ b/.github/skills/coreex-contract/references/workflow.md @@ -82,6 +82,7 @@ public partial class {Name} : IIdentifier, IETag, IChangeLog // add ba public string? Id { get; set; } /// Gets or sets the ... + [NonNullable] // Only where mandatory/required yet declared nullable for DTO flexibility. public string? SomeField { get; set; } /// Gets or sets the SKU (upper-cased on set). @@ -109,6 +110,7 @@ public partial class {Name} : IIdentifier, IETag, IChangeLog // add ba Key assembly rules: - `[Contract]` + `partial` on the class — always. - `[ReadOnly(true)]` on `Id`, `ETag`, `ChangeLog`, and any server-assigned/derived field. +- `[NonNullable]` on any property that is mandatory/required (ultimately not nullable) but declared nullable (`string?`, `int?`) for DTO flexibility. Ask/infer from the requirements which properties are required; leave genuinely optional properties unmarked. This only affects the OpenAPI schema (`nullable: false`) — keep the matching validator rule. Never substitute `System.Diagnostics.CodeAnalysis.NotNull`. Never mark `Id`, `ETag` or `ChangeLog` (null on create); mark other `[ReadOnly]` properties only when the containing type is never a request body. - Only `[ReferenceData]` properties are `partial` — all others are plain auto-properties. - `[Localization("label")]` — only when the auto-derived sentence-case label would be wrong/undesired. Never add when value equals the default. - Casing transforms belong in the setter: `set => field = value?.ToUpper()`. @@ -162,6 +164,8 @@ public partial class ProductLite : ProductBase } ``` +**A5 applies to genuine supersets only** — every inherited property must stay meaningful on the derived type. For a patch/merge-patch or other partial-update request that needs only a *subset* of a sibling contract's fields, do **not** inherit it and suppress the rest (`new`, `[JsonIgnore]`, ignoring members): author a standalone `[Contract]` class with just the properties it needs (+ `IETag` where concurrency applies). Overlapping field names alone are not a reason to extract a base class or inherit. Path B (request contracts) is the right home for these. + --- ## Path B — New Subordinate or Request Contract diff --git a/.github/skills/coreex-db-migration/SKILL.md b/.github/skills/coreex-db-migration/SKILL.md index 9d6564970..3be7e688c 100644 --- a/.github/skills/coreex-db-migration/SKILL.md +++ b/.github/skills/coreex-db-migration/SKILL.md @@ -1,7 +1,7 @@ --- name: coreex-db-migration -description: "Add or change a database table for a CoreEx domain. USE FOR: new transactional table, new reference-data table, altering an existing table (columns, indexes, constraints), or any other schema change (indexes, functions, stored procs). Scaffolds the correct migration script, updates dbex.yaml, applies the migration, and regenerates Infrastructure persistence models. DO NOT USE FOR: outbox provisioning (use dotnet run -- script outbox directly), seed data only changes (dotnet run -- Data), or CoreEx contract/service generation (that is *.CodeGen, not *.Database)." -argument-hint: "Optional: entity/table name, schema, SQL Server vs PostgreSQL, nature of change" +description: "Add or change relational database schema or Cosmos container provisioning for a CoreEx domain. USE FOR: new transactional table, new reference-data table, altering an existing table (columns, indexes, constraints), Cosmos container/provisioning changes, or any other persistence-shape change that must flow through Infrastructure access. For relational domains this scaffolds the correct migration script, updates dbex.yaml, applies the migration, and regenerates EF persistence models; for Cosmos domains it updates ConfigureProvisionArgs/seed data and the typed accessor expectations. DO NOT USE FOR: outbox provisioning in relational domains (use dotnet run -- script outbox directly), seed data only changes (dotnet run -- Data), or CoreEx contract/service generation (that is *.CodeGen, not *.Database)." +argument-hint: "Optional: entity/table/container name, schema or database id, SQL Server vs PostgreSQL vs Cosmos, nature of change" tags: ["database", "migration", "dbex", "schema", "efcore", "coreex"] --- @@ -15,14 +15,15 @@ tags: ["database", "migration", "dbex", "schema", "efcore", "coreex"] # CoreEx: DB Migration -Guides you through any database schema change for a CoreEx domain — from choosing the right migration script through to regenerated EF persistence models. +Guides you through any persistence-shape change for a CoreEx domain — relational schema migrations or Cosmos container/provisioning changes — through to regenerated access-layer artefacts. ## When to Use -- Adding a new table for a new entity (transactional or reference-data) -- Altering an existing table — adding/modifying/removing columns, indexes, or constraints -- Any other schema change that needs to flow through to regenerated `*.g.cs` Infrastructure files -- Non-entity schema changes (adding an index, a unique constraint, a function) +- Adding a new relational table for a new entity (transactional or reference-data) +- Altering an existing relational table — adding/modifying/removing columns, indexes, or constraints +- Adding or changing a Cosmos container declaration, partition-key shape, or seed-data import for a CoreEx entity +- Any other persistence-shape change that needs to flow through to regenerated `*.g.cs` Infrastructure files or typed accessors +- Non-entity relational schema changes (adding an index, a unique constraint, a function) ## When Not to Use @@ -32,14 +33,16 @@ Guides you through any database schema change for a CoreEx domain — from choos - Runtime or deployment issues > **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` -> **Feature Configuration** for `data-provider` (SQL Server / PostgreSQL — gates this whole skill; `None` -> means no database) and `outbox-enabled`. Only prompt for what is unrecorded; re-state resolved values +> **Feature Configuration** for `data-provider` (`SqlServer` / `Postgres` / `Cosmos` — gates this whole skill; `None` +> means no persistence project) and `outbox-enabled`. Only prompt for what is unrecorded; re-state resolved values > for confirmation. ## Quick Reference All commands run from the `*.Database` project directory. +### Relational providers (`SqlServer` / `Postgres`) + | Task | Command | |---|---| | Bring DB up to date | `dotnet run -- database` | @@ -51,6 +54,16 @@ All commands run from the `*.Database` project directory. | Apply everything + regenerate | `dotnet run -- All` | | Drop + full rebuild (destructive, confirm first) | `dotnet run -- dropandall --accept-prompts` | +### Cosmos provider + +| Task | Command / file | +|---|---| +| Declare or change containers | Edit `Program.ConfigureProvisionArgs(CosmosDbProvisionArgs)` | +| Add or amend seed data | Edit `Data/*.seed.yaml` / `Data/*.json` (top-level key = container id) | +| Create/import baseline | `dotnet run -- All` | +| Recreate containers + data | `dotnet run -- ResetAndData` | +| Drop + recreate database (destructive, confirm first) | `dotnet run -- DropAndAll` | + ## Naming - Script commands produce a file named `yyyyMMdd-HHmmss-.{sql|pgsql}` using the current UTC date and time. @@ -65,7 +78,7 @@ All commands run from the `*.Database` project directory. | PostgreSQL | `.pgsql` | `snake_case` | | SQL Server | `.sql` | `PascalCase` | -Check the project's `*.Database/Program.cs` or `appsettings.json` to confirm the provider in use. +Check the project's `*.Database/Program.cs` or host wiring to confirm the provider in use. Cosmos domains declare containers in `ConfigureProvisionArgs(...)` instead of `dbex.yaml`/migration scripts. For the full step-by-step decision tree, SQL column templates, and guardrails see [`references/workflow.md`](references/workflow.md). @@ -86,9 +99,10 @@ For the full workflow, example YAML, DDD aggregate vs CRUD service guidance, and ## Key References -- [`/.github/instructions/coreex-tooling.instructions.md`](/.github/instructions/coreex-tooling.instructions.md) — DbEx command reference, `dbex.yaml` structure, SQL conventions, outbox provisioning +- [`/.github/instructions/coreex-tooling.instructions.md`](/.github/instructions/coreex-tooling.instructions.md) — relational DbEx command reference **and** Cosmos provisioning-console conventions - [`/.github/instructions/coreex-repositories.instructions.md`](/.github/instructions/coreex-repositories.instructions.md) — what the generated `*.g.cs` feeds into - Related skills: [`coreex-refdata`](../coreex-refdata/SKILL.md) (reference-data tables + CodeGen), [`coreex-repository`](../coreex-repository/SKILL.md) (maps the generated persistence models) - Illustrative examples (CoreEx sample — not present in your project): - [PostgreSQL domain database project](https://github.com/Avanade/CoreEx/tree/main/samples/src/Contoso.Products.Database) — canonical `.pgsql` / `snake_case` migrations, `dbex.yaml`, seed files - [SQL Server domain database project](https://github.com/Avanade/CoreEx/tree/main/samples/src/Contoso.Shopping.Database) — canonical `.sql` / `PascalCase` migrations, `dbex.yaml`, seed files + - [Cosmos domain database project](https://github.com/Avanade/CoreEx/tree/main/samples/src/Contoso.Customers.Database) — `CosmosDbConsole`, `ConfigureProvisionArgs`, and container-keyed seed files diff --git a/.github/skills/coreex-db-migration/references/workflow.md b/.github/skills/coreex-db-migration/references/workflow.md index ad798cfb6..b7d017bed 100644 --- a/.github/skills/coreex-db-migration/references/workflow.md +++ b/.github/skills/coreex-db-migration/references/workflow.md @@ -8,11 +8,27 @@ # DB Migration Workflow -All commands run from the `*.Database` project directory for the target domain. +All commands run from the `*.Database` project directory for the target domain. Branch immediately on the recorded `data-provider`. --- -## Phase 1: Establish Baseline +## Phase 0: Provider gate + +- **`SqlServer` / `Postgres`** → follow the relational workflow below (DbEx migration scripts, `dbex.yaml`, generated EF models). +- **`Cosmos`** → skip `dbex.yaml`, `Migrations/`, and SQL scripts entirely. Use the Cosmos path in Phase 1A/2A below: update `Program.ConfigureProvisionArgs(...)`, keep seed data keyed by container id, and align the typed `*CosmosDb` accessor/repository expectations with the container ids you declare. +- **`None`** → this skill is not applicable. + +## Phase 1A: Cosmos baseline + +1. Open `*.Database/Program.cs` and identify the current `ConfigureProvisionArgs(CosmosDbProvisionArgs)` declarations (`.Container(...)`, `.ReferenceDataContainer("ref-data")`, `.OutboxLeaseContainer()` (shared relay lease container; must be declared wherever a Relay host runs), indexing/unique-key customisation). +2. Review the existing `Data/*.seed.yaml` / `*.json` resources. Top-level keys must equal declared container ids. Transactional containers use raw camelCase document bodies; `ref-data` keeps the grouped `$^TypeName` shorthand. +3. Decide whether the change is: + - a **new container** (declare it in `ConfigureProvisionArgs`, then wire the matching `Container(...)` or `ToMappedModel(...)` accessor in `*CosmosDb.cs`), + - an **existing container shape change** (indexing, partition key, seed shape, or mapped model expectations), or + - **seed data only**. +4. Apply via `dotnet run -- All` (or `ResetAndData` when you need to recreate declared containers). + +## Phase 1B: Relational baseline 1. **Bring the database up to date** (non-destructive — Create → Migrate → Schema → Data): ``` @@ -30,7 +46,20 @@ All commands run from the `*.Database` project directory for the target domain. --- -## Phase 2: Choose Script Type +## Phase 2A: Cosmos path + +For `data-provider = Cosmos`, there is no `inspect`/`script` step. Use this decision tree instead: + +| Need | Action | +|---|---| +| New transactional container | Add `.Container("", ...)` to `ConfigureProvisionArgs`, add/update the matching `Container("")` or `.ToMappedModel<...>()` accessor on `*CosmosDb`, seed with `Data/*.seed.yaml`, then run `dotnet run -- All` | +| Existing container shape/indexing change | Update the `.Container(...)` configuration and any matching persistence/repository assumptions, then `dotnet run -- ResetAndData` if the container must be recreated | +| Reference-data container change | Update `ReferenceDataContainer("ref-data")`-adjacent seed data and any `WithTypeDiscriminator()` accessors; keep the container id stable unless the developer explicitly wants to migrate data | +| Seed data only | Edit `Data/*.seed.yaml` / `*.json`, then run `dotnet run -- Data` | + +When a Cosmos change introduces a new container, the repository/accessor follow-on work is part of the same task: declare the container in `ConfigureProvisionArgs`, expose it from `*CosmosDb`, and align tests/seed files to the same id. + +## Phase 2B: Choose Script Type (relational) Branch on the combination of what the developer needs and the inspect result. diff --git a/.github/skills/coreex-docs-sync/README.md b/.github/skills/coreex-docs-sync/README.md index 874d20545..a9868bf0c 100644 --- a/.github/skills/coreex-docs-sync/README.md +++ b/.github/skills/coreex-docs-sync/README.md @@ -51,9 +51,10 @@ No arguments required. ``` .github/instructions/*.instructions.md -.github/skills/** ← coreex-docs-sync, coreex-scaffold, all L1 skills, and all L2 skills +.github/skills/coreex-*/ ← 23 skills: bootstrap, scaffold, docs-sync, 16 L1 and four L2 .github/prompts/*.prompt.md .github/agents/coreex-expert.agent.md +.github/coreex-ai-workflows.md .claude/commands/** .github/docs/coreex/ manifest.txt ← coreex-version, stamped by the template — not hand-written diff --git a/.github/skills/coreex-refdata/SKILL.md b/.github/skills/coreex-refdata/SKILL.md index 22a4e2d08..a32d3cacc 100644 --- a/.github/skills/coreex-refdata/SKILL.md +++ b/.github/skills/coreex-refdata/SKILL.md @@ -15,7 +15,7 @@ tags: ["refdata", "codegen", "database", "contracts", "reference-data", "coreex" # CoreEx: Reference Data -Guides you through the complete end-to-end workflow for adding or modifying a reference data (Code/Text lookup) type in a CoreEx domain — covering both the database side (`*.Database`) and the CodeGen side (`*.CodeGen`). +Guides you through the complete end-to-end workflow for adding or modifying a reference data (Code/Text lookup) type in a CoreEx domain — covering both the persistence side (`*.Database`, relational DbEx or Cosmos provisioning) and the CodeGen side (`*.CodeGen`). ## When to Use @@ -23,6 +23,7 @@ Guides you through the complete end-to-end workflow for adding or modifying a re - Adding extra stored properties to an existing type (beyond the standard `IReferenceData` fields) - Adding or amending seed rows for an existing type - Modifying an existing type's pluralization or identifier type +- Making a type mutable (`mutability` — generated create/patch/activate/deactivate/delete endpoints and service) - Wiring an existing ref-data type into a new or existing contract via `[ReferenceData]` ## When Not to Use @@ -36,22 +37,24 @@ Guides you through the complete end-to-end workflow for adding or modifying a re > **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` > **Feature Configuration** for `refdata-enabled` (gates this whole skill) and `data-provider` -> (SQL Server / PostgreSQL — drives casing and script extension). Only prompt for what is unrecorded; +> (`SqlServer` / `Postgres` / `Cosmos` — drives whether the database step is DbEx or Cosmos provisioning, plus casing/seed shape). Only prompt for what is unrecorded; > re-state resolved values for confirmation. **Two separate `dotnet run` steps in two different projects — never conflate them.** | Task | Project | Command / File | |---|---|---| -| Bring DB up to date (inspect-safe baseline) | `*.Database` | `dotnet run -- database` | -| Inspect table state | `*.Database` | `dotnet run -- inspect ` | -| New ref-data migration script | `*.Database` | `dotnet run -- script refdata
` | -| Add/amend seed rows | `*.Database` | Edit `Data/ref-data.seed.yaml` | -| Register table in EF model | `*.Database` | Edit `dbex.yaml` → `tables:` | -| Apply DB + regenerate EF models | `*.Database` | `dotnet run -- All` | +| Bring persistence baseline up to date | `*.Database` | Relational: `dotnet run -- database`; Cosmos: inspect `Program.ConfigureProvisionArgs(...)` + run `dotnet run -- All` | +| Inspect table/container state | `*.Database` | Relational: `dotnet run -- inspect
`; Cosmos: review declared container ids + seed files | +| New ref-data persistence shape | `*.Database` | Relational: `dotnet run -- script refdata
`; Cosmos: keep `ReferenceDataContainer("ref-data")` and update its seed data / accessors | +| Add/amend seed rows | `*.Database` | Edit `Data/ref-data.seed.yaml` (relational or Cosmos `ref-data` container) | +| Register table in EF model | `*.Database` | Relational only: edit `dbex.yaml` → `tables:` | +| Apply persistence + regenerate EF models/accessors | `*.Database` | Relational: `dotnet run -- All`; Cosmos: `dotnet run -- All` (provisions/imports; no EF generation) | | Add/update entity definition | `*.CodeGen` | Edit `ref-data.yaml` → `entities:` | | Generate all ref-data artefacts | `*.CodeGen` | `dotnet run` | +> **Mutability is opt-in and secured by default, but has no referential-integrity protection.** Mutable endpoints require authorization by default (`[Authorize]`); use `mutableAttribute` to apply a specific policy/role or `[AllowAnonymous]` only when intentionally public. Delete/deactivate never check whether the code is still referenced elsewhere. Warn the user and point them at the `PreCheckAsync` hook — see "Mutable types" in `references/workflow.md`. + ## The Two YAML Files — Never Confuse Them | File | What it contains | @@ -63,10 +66,11 @@ Putting entity definitions in the seed file, or seed rows in the entity file, is ## Polyglot Note -| Provider | Casing | Script ext | +| Provider | Persistence shape | Seed shape | |---|---|---| -| PostgreSQL | `snake_case` | `.pgsql` | -| SQL Server | `PascalCase` | `.sql` | +| PostgreSQL | `snake_case` tables + `.pgsql` scripts | `Data/ref-data.seed.yaml` under the schema key | +| SQL Server | `PascalCase` tables + `.sql` scripts | `Data/ref-data.seed.yaml` under the schema key | +| Cosmos | `ReferenceDataContainer("ref-data")` in `ConfigureProvisionArgs(...)` | `Data/ref-data.seed.yaml` keyed by `ref-data`, with grouped `$^TypeName` entries | Check the project's `*.Database/Program.cs` to confirm the provider in use. @@ -80,3 +84,4 @@ For the full decision tree, YAML examples, seeding rules, and guardrails see [`r - Illustrative examples (CoreEx sample — not present in your project): - [PostgreSQL CodeGen definition](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Products.CodeGen/ref-data.yaml) and its [seed file](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Products.Database/Data/ref-data.seed.yaml) - [SQL Server CodeGen definition](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Shopping.CodeGen/ref-data.yaml) and its [seed file](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Shopping.Database/Data/ref-data.seed.yaml) + - [Cosmos CodeGen definition](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Customers.CodeGen/ref-data.yaml) and its [seed file](https://github.com/Avanade/CoreEx/blob/main/samples/src/Contoso.Customers.Database/Data/ref-data.seed.yaml) diff --git a/.github/skills/coreex-refdata/references/workflow.md b/.github/skills/coreex-refdata/references/workflow.md index ad08f3bb1..f22027227 100644 --- a/.github/skills/coreex-refdata/references/workflow.md +++ b/.github/skills/coreex-refdata/references/workflow.md @@ -14,21 +14,21 @@ Full step-by-step workflow for adding or modifying a reference data type. Follow ## Phase 1 — Establish Baseline -Before any change, bring the database to a known good state and identify the scope. +Before any change, bring the persistence baseline to a known good state and identify the scope. -1. Identify the database provider in use (PostgreSQL → `.pgsql` / `snake_case`; SQL Server → `.sql` / `PascalCase`). Check `*.Database/Program.cs` if unsure. +1. Identify the data provider in use: PostgreSQL (`.pgsql` / `snake_case`), SQL Server (`.sql` / `PascalCase`), or Cosmos (`ConfigureProvisionArgs(...)` + container-keyed seed data). Check `*.Database/Program.cs` if unsure. 2. From the `*.Database` project directory: - ``` - dotnet run -- database - ``` - This is non-destructive (`Create` → `Migrate` → `Schema` → `Data`). **If it fails, stop and surface the verbatim error — do not continue.** -3. Identify all tables involved: the ref-data entity's own table, plus any ref-data tables referenced via `^Type` properties. + - **Relational**: run `dotnet run -- database`. This is non-destructive (`Create` → `Migrate` → `Schema` → `Data`). **If it fails, stop and surface the verbatim error — do not continue.** + - **Cosmos**: review `Program.ConfigureProvisionArgs(CosmosDbProvisionArgs)` and the existing `Data/*.seed.yaml` resources, then run `dotnet run -- All` if the baseline needs provisioning/importing. +3. Identify the persistence artefacts involved: + - **Relational**: the ref-data entity's own table, plus any ref-data tables referenced via `^Type` properties. + - **Cosmos**: the `ref-data` container, plus any generated repository/accessor expectations that depend on its discriminator entries. --- ## Phase 2 — Choose Your Path -Inspect the table(s) to determine which path applies: +For **relational** providers, inspect the table(s) to determine which path applies: ``` dotnet run -- inspect
@@ -42,6 +42,8 @@ dotnet run -- inspect
| Seed rows only, no schema change | **Path D** — seed rows only | | Wire existing type into a contract | **Path E** — contract wiring only | +For **Cosmos**, the persistence path is simpler: keep the `ref-data` container declaration, update its grouped seed data, and set `repository: Cosmos` in `ref-data.yaml` when the generated repository should target Cosmos. There is no table-inspect/dbex path. + --- ## Path A — New Ref-Data Type (new table + seed rows + CodeGen) @@ -254,7 +256,7 @@ Add or update the entry under `entities:`. The standard `IReferenceData` propert ```yaml # yaml-language-server: $schema=https://raw.githubusercontent.com/Avanade/CoreEx/refs/heads/main/schema/coreex-refdata.json collectionSortOrder: Code -repository: EntityFramework +repository: EntityFramework # use `Cosmos` in Cosmos-backed domains entities: - name: Brand # minimal — no extra properties - name: SubCategory @@ -283,6 +285,8 @@ Key `entities:` options: | `properties[].name` | — | Additional property name | | `properties[].type` | — | CLR type; prefix `^` for navigation accessor | | `properties[].excludeContract` | `false` | Persistence model only | +| `mutability` | `None` | `None` (read-only), `CreateUpdate` or `CreateUpdateDelete` — opts the type in to generated write endpoints; see [Mutable types](#mutable-types-optional) | +| `mutableAttribute` | `[Authorize]` | Replaces the default attribute on the generated `{Name}Controller`; set a policy/role or `[AllowAnonymous]` only when intended — `attribute` does **not** cover write endpoints | ### Run CodeGen @@ -297,11 +301,22 @@ On success, CodeGen emits `.g.cs` files across all layers: | Artefact | Layer | |---|---| | `.g.cs` | Contracts | -| `Controller.g.cs` | API host | -| `Service.g.cs` | Application | -| `IRepository.g.cs` | Application | -| `Repository.g.cs` | Infrastructure | +| `ReferenceDataController.g.cs` (read-only GETs, all entities) | API host | +| `ReferenceDataProvider.g.cs` (`IReferenceDataProvider`) | Application | +| `ReferenceDataRepository.g.cs` / `IReferenceDataRepository.g.cs` | Infrastructure / Application | | `Mapper.g.cs` | Infrastructure | +| `Controller.g.cs` — **mutable entities only** | API host | +| `IReferenceDataService.g.cs` + `ReferenceDataService.g.cs` — **only if any entity is mutable** | Application | + +### Mutable types (optional) + +Types are **read-only by default**. Only set `mutability` when the user explicitly wants the type to be maintained through the API; ask if unclear. + +- **EF or Cosmos only** — requires `repository: EntityFramework` or `Cosmos`; CodeGen fails fast otherwise. For Cosmos the ref-data container must be dedicated, have a `/typeDiscriminator` + `/code` unique key, and be registered with `new CosmosDbOptions().Container("", c => c.WithReferenceDataOutboxEvent())` (duplicate codes surface as a 409). The persistence model needs columns for any optional property clients set (`Description`, `StartsOn`, `EndsOn`) or the request is rejected with a 400 (`not-supported`). +- Generates `POST`, `PUT {id}`, `PATCH {id}`, `POST {id}/activate`, `POST {id}/deactivate` and — for `CreateUpdateDelete` — `DELETE {id}` under `/api/refdata/{route}`. Create always yields an inactive item; `code` is immutable; an active item cannot be deleted. Mutable controllers default to `[Authorize]`; recommend a project-specific policy/role through `mutableAttribute` where available. Use `[AllowAnonymous]` only when the API is deliberately public (as in the Products and Customers sample configurations). +- **No usage/cascade check.** Delete and deactivate do not verify the value is unreferenced, and other tables usually store the code — removing or deactivating an in-use value silently invalidates that data. **Always warn the user**, and offer to add a veto-only check via the `PreCheckAsync` hook (a lightweight allow/deny guard — never side effects or cascades; if the user needs more, e.g. atomic reassignment of dependents, hand-write that repository/service logic instead of using the generated mutation) in a hand-written `partial class ReferenceDataService` (`partial void OnInitialization()`; runs for activate/deactivate/delete only, before the transaction, one delegate for all mutable types). Never edit the `.g.cs`. +- Add API tests (`coreex-test-api`) for the write endpoints, including the `PreCheckAsync` veto. +- Detail: `src/CoreEx.CodeGen/README.md` → "Readonly vs Mutation" (CoreEx repo). **On failure, relay verbatim error output — do not create or edit `.g.cs` files to work around it. Fix `ref-data.yaml` and re-run.** @@ -332,7 +347,7 @@ Do **not** pre-add empty-namespace usings — wait until the generated code that ## Guardrails -- **Never edit `.g.cs` files** — they are owned by `*.CodeGen` (contract, controller, service, repository, mapper) or `*.Database` (persistence model). Regenerate instead. +- **Never edit `.g.cs` files** — they are owned by `*.CodeGen` (contract, controllers, provider, service, repository, mapper) or `*.Database` (persistence model). Regenerate instead. - **Two separate YAML files** — `*.CodeGen/ref-data.yaml` (entity definitions) vs `*.Database/Data/ref-data.seed.yaml` (seed rows). Wrong file = runtime failure. - **Never include `{Name}Id` in seed rows** — `$^` auto-generates the id. Including it is always a bug. - **Always use `$^` on ref-data table entries** — regardless of identifier type (`string`, `Guid`, `int`). diff --git a/.github/skills/coreex-repository/SKILL.md b/.github/skills/coreex-repository/SKILL.md index 25078d342..4297d692a 100644 --- a/.github/skills/coreex-repository/SKILL.md +++ b/.github/skills/coreex-repository/SKILL.md @@ -1,7 +1,7 @@ --- name: coreex-repository -description: "Create or modify a CoreEx Infrastructure-layer repository. USE FOR: new repository class, adding CRUD operations, adding a custom query (QueryArgsConfig), bidirectional mapper (BiDirectionMapper), EfDb model accessor, Result pipeline variants. DO NOT USE FOR: Application-layer service logic, domain invariants, typed HTTP clients/adapters (those follow adapter conventions in the infrastructure instructions, not this skill)." -argument-hint: "Optional: entity name, database type (PostgreSQL/SQL Server), operations needed (get/create/update/delete/query), new or existing repository; for query: filtering (default: yes), ordering (default: yes), paging (default: yes), count support (default: no), then per filter field: name + property type + allowed operators + case-insensitive? + model mapping if different" +description: "Create or modify a CoreEx Infrastructure-layer repository. USE FOR: new repository class, adding CRUD operations, adding a custom query (QueryArgsConfig), bidirectional mapper (BiDirectionMapper), EfDb/CosmosDb model accessor, Result pipeline variants. DO NOT USE FOR: Application-layer service logic, domain invariants, typed HTTP clients/adapters (those follow adapter conventions in the infrastructure instructions, not this skill)." +argument-hint: "Optional: entity name, database type (PostgreSQL/SQL Server/Cosmos), operations needed (get/create/update/delete/query), new or existing repository; for query: filtering (default: yes), ordering (default: yes), paging (default: yes), count support (default: no), then per filter field: name + property type + allowed operators + case-insensitive? + model mapping if different" tags: ["repository", "infrastructure", "efcore", "mapping", "coreex", "data-access", "result"] --- @@ -15,15 +15,15 @@ tags: ["repository", "infrastructure", "efcore", "mapping", "coreex", "data-acce # CoreEx: Repository -Guides you through creating or modifying a CoreEx Infrastructure-layer repository in `Infrastructure/Repositories/`. Covers CRUD delegates, custom queries, bidirectional mappers, EfDb accessors, and `Result` pipelines. +Guides you through creating or modifying a CoreEx Infrastructure-layer repository in `Infrastructure/Repositories/`. Covers CRUD delegates, custom queries, bidirectional mappers, EfDb or CosmosDb accessors, and `Result` pipelines. ## When to Use -- New repository class for an entity (scaffold, EfDb accessor, mapper, registration) +- New repository class for an entity (scaffold, EfDb/CosmosDb accessor, mapper, registration) - Adding a CRUD operation (get/create/update/delete) using EfDb delegate shortcuts - Adding a custom query method with `QueryArgsConfig` dynamic filtering/ordering - Adding a `BiDirectionMapper` (Contract ↔ Persistence) in `Infrastructure/Mapping/` -- Adding or modifying an `EfDbMappedModel` or `EfDbModel` accessor on `*EfDb` +- Adding or modifying an `EfDbMappedModel` / `EfDbModel` accessor on `*EfDb`, or a `CosmosDbContainer` / `CosmosDbMappedContainer<...>` accessor on `*CosmosDb` - Switching a method to the `Result` / `*WithResultAsync` pipeline ## When Not to Use @@ -35,8 +35,8 @@ Guides you through creating or modifying a CoreEx Infrastructure-layer repositor ## Quick Reference **Clarifying questions before writing any code:** -0. **Resolve from state first.** Read the solution-root `AGENTS.md` **Feature Configuration** for `data-provider` (SQL Server / PostgreSQL — gates this skill) and check whether a `*.Domain` project exists (Domain-mapping vs contract-mapping). Only ask for what is unresolved; re-state resolved values for confirmation. -1. Which entity? Which database type (PostgreSQL / SQL Server)? Check the project's `Program.cs`. +0. **Resolve from state first.** Read the solution-root `AGENTS.md` **Feature Configuration** for `data-provider` (`SqlServer` / `Postgres` / `Cosmos` — gates this skill) and check whether a `*.Domain` project exists (Domain-mapping vs contract-mapping). Only ask for what is unresolved; re-state resolved values for confirmation. +1. Which entity? Which database type (`PostgreSQL` / `SQL Server` / `Cosmos`)? Check the project's `Program.cs`. 2. New repository or adding to an existing one? 3. Operations needed: Get / Create / Update / Delete / Query? 4. Does the project use `Result` / ROP pipelines? (→ `*WithResultAsync` — per-project style choice, not tied to DDD) @@ -44,12 +44,12 @@ Guides you through creating or modifying a CoreEx Infrastructure-layer repositor **Key rules at a glance:** - `[ScopedService]` on every repository class — auto-registers in DI -- Primary constructor: `public class ProductRepository(ProductsEfDb ef) : IProductRepository`; guard with `ThrowIfNull()` -- Use EfDb delegate shortcuts (`GetAsync`, `CreateAsync`, `UpdateAsync`, `DeleteAsync`) — never write raw `DbContext` CRUD +- Primary constructor: relational example `public class ProductRepository(ProductsEfDb ef) : IProductRepository`; Cosmos example `public class CustomerRepository(CustomersCosmosDb cosmos) : ICustomerRepository`; always guard the injected accessor with `ThrowIfNull()` +- Use the provider's typed accessors: `EfDb` delegate shortcuts (`GetAsync`, `CreateAsync`, `UpdateAsync`, `DeleteAsync`) for relational domains; `CosmosDbMappedContainer` / `CosmosDbContainer` delegates for Cosmos domains. Never write raw `DbContext` CRUD or instantiate a `CosmosClient` in the repository. - `DataResult` return for Create/Update; `DataResult` for Delete — includes mutation flag for event decisions - `*WithResultAsync` variants for `Result` ROP pipelines (per-project style choice) - `BiDirectionMapper`: override **both** `OnMap` overloads; map `Id` explicitly; **never** map `ETag` or `ChangeLog` — base mapper owns them -- `QueryArgsConfig`: create a dedicated `{Name}QueryArgsConfig : QueryArgsConfig<{Name}QueryArgsConfig>` class per entity in `Infrastructure/Repositories/`; access via `.Default`; call `.Parse(query).ThrowOnError()` before use — never instantiate per-request +- `QueryArgsConfig`: create a dedicated `{Name}QueryArgsConfig : QueryArgsConfig<{Name}QueryArgsConfig>` class per entity in `Infrastructure/Repositories/`; access via `.Default`; call `.Parse(query).ThrowOnError()` before use — never instantiate per-request. Relational queries materialize from `IQueryable` extensions; Cosmos queries materialize from `CosmosDbQuery` instance methods (`Container.Query(...).ToMappedItemsResultAsync(...)`). - `AddReferenceDataField(field, model, ...)`: `field` is always the contract's generated nav property (`{Name}`, never `{Name}Code`) — a fixed, documented source-generator convention, **not** something to verify by exploring generated code - `ToMappedItemsResultAsync(mapper, paging, cancellationToken: cancellationToken)`: always pass `cancellationToken` **by name** — `autoCount` (`bool`, defaults `true`) sits before it in the signature, and a bare positional token there fails to compile - Always `.ConfigureAwait(false)` on every `await` diff --git a/.github/skills/coreex-repository/references/workflow.md b/.github/skills/coreex-repository/references/workflow.md index 21acc01d0..406cb5929 100644 --- a/.github/skills/coreex-repository/references/workflow.md +++ b/.github/skills/coreex-repository/references/workflow.md @@ -18,7 +18,7 @@ Answer these questions before emitting any code. | Question | Default | Notes | |---|---|---| -| Which entity and database type? | Ask | PostgreSQL: `PostgresDatabase` + `UseNpgsql`; SQL Server: `SqlServerDatabase` + `UseSqlServer` — check the project's `Program.cs` | +| Which entity and database type? | Ask | PostgreSQL: `PostgresDatabase` + `UseNpgsql`; SQL Server: `SqlServerDatabase` + `UseSqlServer`; Cosmos: `AddAzureCosmosClient("Cosmos")` + `AddCosmosDb("{db-id}")` — check the project's `Program.cs` | | New repository or adding to an existing one? | Ask | New → Path A (scaffold); Existing → Path B/C/D | | Operations needed? | Ask | Get / Create / Update / Delete / Query | | Using `Result` / ROP pipelines? | No | Yes → use `*WithResultAsync` + `Result` pipeline (Path D); either style works at the repository level — this is a per-project or per-service choice | @@ -28,6 +28,45 @@ Answer these questions before emitting any code. ## Path A — New Repository Class +### A0 — Branch on provider + +- **Relational (`SqlServer` / `Postgres`)** → follow A1–A4 exactly. +- **Cosmos** → use the Cosmos path below for the accessor and repository shape; there is no `DbContext` / `EfDb`. + +### A0a — Cosmos repository scaffold + +```csharp +[ScopedService] +public class CustomerRepository(CustomersCosmosDb cosmos) : ICustomerRepository +{ + private readonly CustomersCosmosDb _cosmos = cosmos.ThrowIfNull(); + + public Task GetAsync(string id, CancellationToken ct = default) + => _cosmos.Customers.GetAsync(CompositeKey.Create(id), ct); + + public Task> CreateAsync(Contracts.Customer value, CancellationToken ct = default) + => _cosmos.Customers.CreateAsync(value, ct); + + public Task> UpdateAsync(Contracts.Customer value, CancellationToken ct = default) + => _cosmos.Customers.UpdateAsync(value, ct); +} +``` + +The typed accessor lives on `*CosmosDb.cs`, not `*EfDb.cs`: + +```csharp +public class CustomersCosmosDb(CosmosClient client, string databaseId) : CosmosDb(client, databaseId, _options) +{ + private static readonly CosmosDbOptions _options = new CosmosDbOptions().Container("ref-data", c => c.WithReferenceDataOutboxEvent()); + + public CosmosDbMappedContainer Customers + => Container("customers").ToMappedModel(new CustomerMapper()); +} +``` + +Use `Container(...)` / `.ToMappedModel<...>()` per container. Query through `_cosmos.Customers.Container.Query(...)` and materialize from the returned `CosmosDbQuery` instance methods. + + ### A1 — Scaffold repository ```csharp @@ -50,7 +89,7 @@ public class {Name}Repository({Solution}EfDb ef) : I{Name}Repository `[ScopedService]` auto-registers via `AddDynamicServicesUsing()` — no manual DI wiring needed. -### A2 — Add EfDb accessor +### A2 — Add EfDb accessor (relational only) In `*EfDb.cs`, add a strongly-typed property for the new entity. Use `EfDbMappedModel` when the mapper already exists; `EfDbModel` for plain models (ref-data, read-only joins): @@ -107,7 +146,7 @@ public class {Name}Mapper : BiDirectionMapper o ## Path B — Add a CRUD Operation to an Existing Repository -Locate the repository class and add the method using the EfDb delegate shortcuts. No raw `DbContext` queries for simple CRUD: +Locate the repository class and add the method using the provider's typed accessors. For relational domains, use the EfDb delegate shortcuts below. For Cosmos domains, call the mapped container methods (`_cosmos.Customers.CreateAsync(...)`, `_cosmos.Customers.UpdateAsync(...)`, etc.) and use `CosmosDbArgs` where you need request options such as `IfMatchEtag`. No raw `DbContext` queries for simple CRUD, and never instantiate a `CosmosClient` inside the repository. ```csharp // Get by key — returns null when not found; service checks diff --git a/.github/skills/coreex-scaffold/README.md b/.github/skills/coreex-scaffold/README.md index d0ba75577..e5a599732 100644 --- a/.github/skills/coreex-scaffold/README.md +++ b/.github/skills/coreex-scaffold/README.md @@ -53,5 +53,3 @@ If the prompt file is not present, attach the skill file directly in Copilot Cha ## Reference - [SKILL.md](./SKILL.md) - main workflow guidance. - - diff --git a/.github/skills/coreex-scaffold/SKILL.md b/.github/skills/coreex-scaffold/SKILL.md index 0fb7a243e..3e046886b 100644 --- a/.github/skills/coreex-scaffold/SKILL.md +++ b/.github/skills/coreex-scaffold/SKILL.md @@ -1,6 +1,6 @@ --- name: coreex-scaffold -description: "Guide a developer through CoreEx solution shaping after bootstrap, using a short plain-English interview that turns user answers into safe dotnet new template inputs. USE FOR: bootstrap-only repos, deciding API-only vs API plus relay vs API plus subscriber, choosing SQL Server vs Postgres vs no database, choosing refdata/outbox/DDD/ROP options, installing CoreEx.Template, checking current solution shape, adding missing Api/Relay/Subscribe hosts to an existing repo, adding a .NET Aspire AppHost for local multi-host orchestration and dashboard visibility, and optionally preparing a first local runnable state with local dependency assets plus database/code-generation steps. DO NOT USE FOR: unrelated runtime debugging, bootstrap creation, or forcing root re-scaffolding over an existing solution. INVOKES: workspace inspection, ask-questions style interviews, dotnet new install/list, dry-run validation, solution wiring, optional local dependency asset creation, focused build/test validation, and either template generation or manual retrofit work depending on repo shape." +description: "Guide a developer through CoreEx solution shaping after bootstrap, using a short plain-English interview that turns user answers into safe dotnet new template inputs. USE FOR: bootstrap-only repos, deciding API-only vs API plus relay vs API plus subscriber, choosing SQL Server vs Postgres vs Cosmos vs no database, choosing refdata/outbox/DDD/ROP options, installing CoreEx.Template, checking current solution shape, adding missing Api/Relay/Subscribe hosts to an existing repo, adding a .NET Aspire AppHost for local multi-host orchestration and dashboard visibility, and optionally preparing a first local runnable state with local dependency assets plus database/code-generation steps. DO NOT USE FOR: unrelated runtime debugging, bootstrap creation, or forcing root re-scaffolding over an existing solution. INVOKES: workspace inspection, ask-questions style interviews, dotnet new install/list, dry-run validation, solution wiring, optional local dependency asset creation, focused build/test validation, and either template generation or manual retrofit work depending on repo shape." argument-hint: "Optional: base solution name, whether this is new or retrofit, required hosts, database choice, messaging needs, and whether to add an Aspire AppHost for local orchestration." tags: ["coreex", "scaffolding", "retrofit", "template", "hosts", "aspire", "apphost", "orchestration"] --- @@ -87,6 +87,7 @@ Set outbox to `true` only when the user chose owned persistence and reliable pub | "I need to consume events." | Add `coreex-subscribe`; use a messaging provider. | | "This service stores its own data in SQL Server." | `--data-provider SqlServer` | | "This service stores its own data in Postgres." | `--data-provider Postgres` | +| "This service stores its own data in Azure Cosmos DB." | `--data-provider Cosmos` | | "This is a facade. No local database." | `--data-provider None` | | "I need reference data." | `--refdata-enabled true` | | "I want a Domain layer." | Add `coreex-domain` addon after `coreex` | @@ -135,7 +136,7 @@ dotnet new coreex-relay -n Avanade.Product.Books.Relay --data-provider dotnet new coreex-subscribe -n Avanade.Product.Books.Subscribe --data-provider SqlServer --messaging-provider ServiceBus --refdata-enabled true # Step 2a (optional): Add an Aspire AppHost for local orchestration, once the hosts above exist. -dotnet new coreex-aspire -n Avanade.Product.Books.Aspire --has-api true --has-relay true --has-subscribe true +dotnet new coreex-aspire -n Avanade.Product.Books.Aspire --has-api true --has-relay true --has-subscribe true --data-provider SqlServer --messaging-provider ServiceBus # Step 3: Add host and test projects to the solution file. dotnet sln Avanade.Product.Books.slnx add src/Avanade.Product.Books.Api @@ -144,7 +145,9 @@ dotnet sln Avanade.Product.Books.slnx add src/Avanade.Product.Books.Relay dotnet sln Avanade.Product.Books.slnx add tests/Avanade.Product.Books.Test.Relay dotnet sln Avanade.Product.Books.slnx add src/Avanade.Product.Books.Subscribe dotnet sln Avanade.Product.Books.slnx add tests/Avanade.Product.Books.Test.Subscribe -dotnet sln Avanade.Product.Books.slnx add src/Avanade.Product.Books.Aspire +dotnet sln Avanade.Product.Books.slnx add aspire/Avanade.Product.Books.Aspire +dotnet sln Avanade.Product.Books.slnx add aspire/Avanade.Product.Books.Aspire.MockHost +dotnet sln Avanade.Product.Books.slnx add aspire/Avanade.Product.Books.Test.Aspire # Step 4: Validate the generated solution. dotnet build Avanade.Product.Books.slnx @@ -166,7 +169,7 @@ dotnet new coreex-aspire -n Company.Product.Books.Aspire --has-api ... --has- # Then add new projects to the solution file as in Step 3 above. ``` -**Adding an AppHost that already exists:** if `coreex-aspire` output already exists and a new host is being added, do not re-run `coreex-aspire --force` — it will overwrite any customisation already made to `AppHost.cs`/`Extensions.cs`. Instead, add the missing `` and `builder.AddProject<...>(...)` line to the existing files by hand. +**Adding an AppHost that already exists:** if `coreex-aspire` output already exists and a new host is being added, do not re-run `coreex-aspire --force` — it will overwrite any customisation already made to `AppHost.cs`. Instead, add the missing `` and `builder.AddProject<...>(...)` line to the existing files by hand. **Adding an API host after CodeGen already ran:** when `refdata-enabled` is `true`, the `*.CodeGen` tool only emits the reference-data controller into an `*.Api` project directory that exists at generation time — it silently skips it (a log warning, not an error) for any `coreex-api` added afterward. If `coreex-api` is added to a solution where CodeGen has already been run at least once, re-run `dotnet run --project tools/[solution].CodeGen` afterward so the new host gets its reference-data controller. @@ -188,14 +191,14 @@ dotnet new coreex-aspire -n Company.Product.Books.Aspire --has-api ... --has- - Always run `dotnet build` on the solution as the minimum end-to-end validation step. - Always run the unit test project when present. - Run API, relay, or subscriber test projects only when their required local dependencies are available; otherwise skip them and report the reason explicitly. -- Treat missing local infrastructure such as SQL Server, Postgres, Redis, or Service Bus as a validation skip, not as a scaffolding failure, unless the user explicitly asked for full local environment setup. +- Treat missing local infrastructure such as SQL Server, Postgres, Cosmos emulator, Redis, or Service Bus as a validation skip, not as a scaffolding failure, unless the user explicitly asked for full local environment setup. - Once the scaffolded test projects build and run their initial templated tests, hand off to `coreex-test-api`, `coreex-test-subscribe`, or `coreex-test-relay` for authoring any further domain-specific integration tests — this skill only validates the generated shape, not test content. - When the user explicitly wants a first local runnable state, create any missing local dependency assets before broader validation, using bundled templates where available. -- When `data-provider != None` and the user wants runnable local state, run the `*.Database` tool from its own project directory so `dbex.yaml` resolves correctly; prefer `dotnet run -- All` for first-run local setup. +- When `data-provider != None` and the user wants runnable local state, run the `*.Database` tool from its own project directory. For relational providers this ensures `dbex.yaml` resolves correctly; for Cosmos it provisions the declared containers and imports seed data. Prefer `dotnet run -- All` for first-run local setup. - When `refdata-enabled` is `true` and the user wants runnable local state, run the `*.CodeGen` tool from its own project directory so its config file resolves correctly. - If generated code fails to compile before environment or tool execution becomes relevant, treat that as a template or package-version defect first; prefer fixing the template pack rather than teaching the workflow to hand-patch generated output every time. - If `refdata-enabled` is `true`, call out the `*.CodeGen` tool as the next step or run it only when the user wants a fully generated local state. -- If `data-provider != None`, call out the `*.Database` tool as the next step or run it only when the user wants local schema setup as part of scaffolding. +- If `data-provider != None`, call out the `*.Database` tool as the next step or run it only when the user wants local persistence setup as part of scaffolding. Relational domains create schema/migrations; Cosmos domains provision containers and seed data. ## Shape Selection Guide @@ -225,5 +228,3 @@ dotnet new coreex-aspire -n Company.Product.Books.Aspire --has-api ... --has- | CoreEx Outbox Relay host | `coreex-relay` | Adds an outbox relay host | | CoreEx Subscriber host | `coreex-subscribe` | Adds an event subscriber host | | CoreEx Aspire AppHost | `coreex-aspire` | Adds an Aspire AppHost orchestrating this solution's own hosts — run after those hosts exist | - - diff --git a/.github/skills/coreex-scaffold/assets/README.md b/.github/skills/coreex-scaffold/assets/README.md index 88b55ebea..9a28498bd 100644 --- a/.github/skills/coreex-scaffold/assets/README.md +++ b/.github/skills/coreex-scaffold/assets/README.md @@ -36,7 +36,7 @@ changes were made to keep them consumer-safe and domain-agnostic: — per this repo's core principle that skills must not be tied to the sample domains. - Unlike `CoreEx.Core`'s `docker-compose.yml` (which conditionally includes only the services matching the selected `data-provider`/`messaging-provider`), these are an unconditional "everything" stack — - SQL Server, Postgres, Redis, Service Bus emulator, Aspire Dashboard, and the DTS emulator — so the + SQL Server, Postgres, Cosmos emulator, Redis, Service Bus emulator, Aspire Dashboard, and the DTS emulator — so the fallback works regardless of which options the retrofit repo actually chose. Trim unused services when applying them to a specific repo. diff --git a/.github/skills/coreex-scaffold/references/workflow.md b/.github/skills/coreex-scaffold/references/workflow.md index cf05fb7fa..bff58ade5 100644 --- a/.github/skills/coreex-scaffold/references/workflow.md +++ b/.github/skills/coreex-scaffold/references/workflow.md @@ -175,12 +175,14 @@ Recommended options: - `SQL Server` - `Postgres` +- `Cosmos` - `No local database` Interpretation: - `SQL Server` maps to `--data-provider SqlServer`. - `Postgres` maps to `--data-provider Postgres`. +- `Cosmos` maps to `--data-provider Cosmos`. - `No local database` maps to `--data-provider None`. Default: `No local database`. @@ -276,7 +278,7 @@ Recommended options: Interpretation: -- `Yes` means run `dotnet new coreex-aspire -n Company.Product.Books.Aspire` after every chosen host template, passing `--has-api`/`--has-relay`/`--has-subscribe` to match the hosts chosen in questions 3-5. +- `Yes` means run `dotnet new coreex-aspire -n Company.Product.Books.Aspire` after every chosen host template, passing `--has-api`/`--has-relay`/`--has-subscribe` to match the hosts chosen in questions 3-5, and `--data-provider`/`--messaging-provider` to match questions 6/7. - `No` means skip `coreex-aspire` entirely. Default: `No`. @@ -293,12 +295,13 @@ Build a structured decision summary before any command is run. | Consume events | `dotnet new coreex-subscribe -n Company.Product.Books.Subscribe ...` | | SQL Server | `--data-provider SqlServer` | | Postgres | `--data-provider Postgres` | +| Cosmos | `--data-provider Cosmos` | | No local database | `--data-provider None` | | Azure Service Bus | `--messaging-provider ServiceBus` | | Reference data needed | `--refdata-enabled true` | | Domain layer needed | `dotnet new coreex-domain -n Company.Product.Books.Domain` + `dotnet sln Company.Product.Books.slnx add src/Company.Product.Books.Domain` + `dotnet add src/Company.Product.Books.Application/Company.Product.Books.Application.csproj reference src/Company.Product.Books.Domain/Company.Product.Books.Domain.csproj` (Application must reference Domain directly -- nothing else wires this) | | Result/ROP style needed | `--rop-enabled true` | -| Aspire AppHost needed | `dotnet new coreex-aspire -n Company.Product.Books.Aspire --has-api --has-relay --has-subscribe `, where each `--has-*` flag mirrors whether that host was chosen in questions 3-5 -- run this last, after every chosen host template | +| Aspire AppHost needed | `dotnet new coreex-aspire -n Company.Product.Books.Aspire --has-api --has-relay --has-subscribe --data-provider --messaging-provider `, where each `--has-*` flag mirrors whether that host was chosen in questions 3-5 and the provider flags mirror questions 6-7 -- run this last, after every chosen host template | If the user is retrofitting an existing solution, only include flags that are required for missing projects or that the user explicitly asked to change. @@ -319,13 +322,13 @@ State that the workflow will finish by wiring projects into the solution and run 4. Stop if the dry-run shows nested root folders, incorrect host names, or a layout that conflicts with the existing repo. 5. For bootstrap-only repos, run `coreex` first (omit `-n` when the folder name matches), then add each needed host template with the **4-part name** (`base.Api`, `base.Relay`, `base.Subscribe`). 6. For retrofit work, add only the missing hosts unless the user explicitly asked for broader reshaping. -7. If an Aspire AppHost was requested, run `coreex-aspire` last, after every chosen host template has already been generated -- its `--has-*` flags and generated `Projects.*` references depend on those hosts already existing. +7. If an Aspire AppHost was requested, run `coreex-aspire` last, after every chosen host template has already been generated -- its `--has-*` flags and generated `Projects.*` references depend on those hosts already existing. Preserve its visible external resources using `CoreEx.UnitTesting`'s `AddExternalConnectionString`; do not replace them with parameter-only `AddConnectionString(name)` calls or duplicate the helper locally. Configure `ConnectionStrings` in the AppHost; Cosmos's resource URL is derived from `AccountEndpoint`, while Compose retains infrastructure lifecycle ownership. 8. After generating all host templates (and the AppHost, if requested), run `dotnet sln` to add every new host project and its test project to the `.slnx` solution file. 9. Verify that each expected generated host and test project exists on disk before moving to validation. ### Adding a host to a solution that already has an AppHost -If `coreex-aspire` has already been run and a new host is added afterward, do not re-run `coreex-aspire --force` -- it would overwrite any customisation already made to `AppHost.cs`/`Extensions.cs`. Instead, add the missing `` in the AppHost's `.csproj` and the matching `builder.AddProject<...>(...)` line in `AppHost.cs` by hand, following the pattern already used for the other hosts there. +If `coreex-aspire` has already been run and a new host is added afterward, do not re-run `coreex-aspire --force` -- it would overwrite any customisation already made to `AppHost.cs`. Instead, add the missing `` in the AppHost's `.csproj` and the matching `builder.AddProject<...>(...)` line in `AppHost.cs` by hand, following the pattern already used for the other hosts there. ### Adding an API host to a solution where CodeGen has already run @@ -363,15 +366,15 @@ For local development there are two validation modes: - `dotnet build` is the default minimum bar for a successful scaffold. - `dotnet test` for `tests/[solution].Test.Unit` should run by default when the project exists. -- API, relay, and subscriber tests may depend on local SQL Server or Postgres, Redis, Service Bus, or other host prerequisites. If those are not configured, skip those tests and say why. +- API, relay, and subscriber tests may depend on local SQL Server or Postgres, the Cosmos emulator, Redis, Service Bus, or other host prerequisites. If those are not configured, skip those tests and say why. - A missing local dependency is a deferred setup item, not necessarily a scaffolding failure. - If the user wants runnable local validation and local dependency files are missing, materialize them first from bundled templates or repo-standard equivalents, for example `.github/skills/coreex-scaffold/assets/docker-compose.local.yml` and `.github/skills/coreex-scaffold/assets/servicebus-config.template.json`. These are a static, options-agnostic fallback for use when it is unsafe to re-run the root scaffold — see `.github/skills/coreex-scaffold/assets/README.md` for why they exist and how they relate to the scaffold's own generated `docker-compose.yml`/`servicebus/Config.json`. -- If the generated shape includes SQL Server, Redis, or Service Bus and the repo lacks a root `docker-compose.yml`, create one before trying to run broader tests. +- If the generated shape includes SQL Server, Postgres, the Cosmos emulator, Redis, or Service Bus and the repo lacks a root `docker-compose.yml`, create one before trying to run broader tests. - If Service Bus emulator wiring is needed and the repo lacks `servicebus/Config.json`, create it before starting the dependency stack. -- For `tools/[solution].Database`, run from that directory so `dbex.yaml` is discovered correctly. Prefer `dotnet run -- All` for first-run local setup and `dotnet run -- CodeGen` or `dotnet run -- Database` for narrower follow-up work. +- For `tools/[solution].Database`, run from that directory. In relational domains this ensures `dbex.yaml` is discovered correctly; in Cosmos domains it keeps the provisioning console aligned with its embedded `Data/*` resources. Prefer `dotnet run -- All` for first-run local setup and `dotnet run -- CodeGen` or `dotnet run -- Database` for narrower relational follow-up work. - For `tools/[solution].CodeGen`, run from that directory so `ref-data.yaml` is discovered correctly. - If `refdata-enabled` is `true`, call out `tools/[solution].CodeGen` as the next step, or run it only when the user wants the repo left in a fully generated state. -- If `data-provider != None`, call out `tools/[solution].Database` as the next step, or run it only when the user wants local schema setup as part of scaffolding. +- If `data-provider != None`, call out `tools/[solution].Database` as the next step, or run it only when the user wants local persistence setup as part of scaffolding. Relational domains create schema/migrations; Cosmos domains provision containers and seed data. - If generated code fails compile due to placeholder types, missing project references, wrong package APIs, or other issues unrelated to missing local dependencies, classify that as a template defect and surface it explicitly. - If `dotnet new` reports duplicate CoreEx template identities, surface that warning and note which template source was used. @@ -430,7 +433,7 @@ dotnet new coreex --data-provider SqlServer --messaging-provider ServiceBus --re dotnet new coreex-api -n Avanade.Erp.Sales.Api --data-provider SqlServer --refdata-enabled true --outbox-enabled true dotnet new coreex-relay -n Avanade.Erp.Sales.Relay --data-provider SqlServer --messaging-provider ServiceBus dotnet new coreex-subscribe -n Avanade.Erp.Sales.Subscribe --data-provider SqlServer --messaging-provider ServiceBus --refdata-enabled true -dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true +dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true --data-provider SqlServer --messaging-provider ServiceBus dotnet sln Avanade.Erp.Sales.slnx add src/Avanade.Erp.Sales.Api dotnet sln Avanade.Erp.Sales.slnx add tests/Avanade.Erp.Sales.Test.Api @@ -438,7 +441,9 @@ dotnet sln Avanade.Erp.Sales.slnx add src/Avanade.Erp.Sales.Relay dotnet sln Avanade.Erp.Sales.slnx add tests/Avanade.Erp.Sales.Test.Relay dotnet sln Avanade.Erp.Sales.slnx add src/Avanade.Erp.Sales.Subscribe dotnet sln Avanade.Erp.Sales.slnx add tests/Avanade.Erp.Sales.Test.Subscribe -dotnet sln Avanade.Erp.Sales.slnx add src/Avanade.Erp.Sales.Aspire +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Aspire +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Aspire.MockHost +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Test.Aspire dotnet build Avanade.Erp.Sales.slnx dotnet test tests/Avanade.Erp.Sales.Test.Unit @@ -468,4 +473,3 @@ dotnet sln Avanade.Erp.Sales.slnx add tests/Avanade.Erp.Sales.Test.Subscribe dotnet build Avanade.Erp.Sales.slnx dotnet test tests/Avanade.Erp.Sales.Test.Unit ``` - diff --git a/.github/skills/coreex-subscriber-e2e/SKILL.md b/.github/skills/coreex-subscriber-e2e/SKILL.md index 5ffee9ff2..8cf2db133 100644 --- a/.github/skills/coreex-subscriber-e2e/SKILL.md +++ b/.github/skills/coreex-subscriber-e2e/SKILL.md @@ -1,6 +1,6 @@ --- name: coreex-subscriber-e2e -description: "Create a complete new event or command subscriber end-to-end in a single guided workflow: event/command DTO contract (if new), subscriber handler, optional application service and repository, and integration tests. USE FOR: adding a brand-new subscriber to an existing CoreEx Subscribe host for any scenario — command handling, event-data-sync replication, or event-driven business-process choreography. DO NOT USE FOR: modifying an existing subscriber (use coreex-subscriber directly), API endpoint work (use coreex-api-e2e), or setting up the Subscribe host itself (use coreex-scaffold)." +description: "Create a complete new event subscriber end-to-end in a single guided workflow: event DTO contract (if new), subscriber handler, optional application service and repository, and integration tests. USE FOR: adding a brand-new subscriber to an existing CoreEx Subscribe host for event-data-sync replication or event-driven business-process choreography. DO NOT USE FOR: command handling (use coreex-command-subscribe-e2e), sending a command (use coreex-command-publish-e2e), modifying an existing subscriber (use coreex-subscriber directly), API endpoint work (use coreex-api-e2e), or setting up the Subscribe host itself (use coreex-scaffold)." argument-hint: "Optional: event or command subject, subscriber scenario (command / event-data-sync / business-process)" tags: ["coreex", "subscriber", "event", "command", "end-to-end", "vertical-slice", "messaging"] --- @@ -21,7 +21,7 @@ Guides you through adding a complete new event or command subscriber in one sitt - Adding a brand-new subscriber to an existing CoreEx Subscribe host - You know the event or command subject and want the full slice — contract, handler, optional orchestration, and tests — in one workflow -- Covers all three subscriber scenarios: command handling, event-data-sync replication, and event-driven business-process choreography +- Covers the event-data-sync and business-process scenarios; for a **command** (queue-based, needs a dedicated receiver in the Subscribe host) prefer [`coreex-command-subscribe-e2e`](../coreex-command-subscribe-e2e/SKILL.md), which also covers topology and Aspire ## When Not to Use @@ -31,7 +31,7 @@ Guides you through adding a complete new event or command subscriber in one sitt ## Workflow Overview -1. **Read Feature Configuration** — resolve `messaging-provider`, `outbox-enabled`, `data-provider`, and `rop-enabled` from the solution-root `AGENTS.md` before asking anything. +1. **Read Feature Configuration** — resolve `messaging-provider`, `outbox-enabled`, `data-provider` (`SqlServer` / `Postgres` / `Cosmos` / `None`), and `rop-enabled` from the solution-root `AGENTS.md` before asking anything. 2. **Interview** — identify the subject, scenario, DTO needs, and whether state persistence is required; batch all questions into one turn. 3. **Execute L1 sequence** — invoke each applicable L1 skill in order, passing context resolved in steps 1–2; no repeated questions. 4. **Validate** — `dotnet build`; confirm the subscriber is registered and integration tests are present. diff --git a/.github/skills/coreex-subscriber-e2e/references/workflow.md b/.github/skills/coreex-subscriber-e2e/references/workflow.md index e28b0a70d..8a033d7ce 100644 --- a/.github/skills/coreex-subscriber-e2e/references/workflow.md +++ b/.github/skills/coreex-subscriber-e2e/references/workflow.md @@ -20,7 +20,7 @@ Before asking the user anything, read the solution-root `AGENTS.md` **Feature Co |---|---| | `messaging-provider` | Confirms the broker (ServiceBus) and subject naming conventions | | `outbox-enabled` | Passed to app-service step for mutation event publication | -| `data-provider` | `None` → Steps 2 and 3 (migration + repository) are never applicable | +| `data-provider` | `None` → Steps 2 and 3 (migration + repository) are never applicable; `Cosmos` still runs them, but via container provisioning and Cosmos repositories instead of DbEx/EF | | `rop-enabled` | Passed to app-service step to determine Result<T> pipeline shape | If any feature is absent from `AGENTS.md`, include it as a question in Phase 2 and record the answer before proceeding. @@ -36,10 +36,10 @@ Gather everything needed for all downstream L1 skills in a single conversation t | # | Question | Why needed | |---|---|---| | 1 | What is the event or command subject? (e.g. `products.product.modified`, `shopping.basket.checkout`) | Drives subscriber subject registration and test assertions | -| 2 | Which scenario does this subscriber implement? — **Command handler** (inbound command triggers a direct action), **Event-data-sync** (inbound event replicates data to a local store via `IXxxSyncAdapter`), or **Business-process** (inbound event triggers multi-step orchestration) | Determines which L1 steps run | +| 2 | Which scenario does this subscriber implement? — **Command handler** (inbound command triggers a direct action — for commands prefer [`coreex-command-subscribe-e2e`](../coreex-command-subscribe-e2e/SKILL.md), which also covers the dedicated queue receiver, topology and Aspire), **Event-data-sync** (inbound event replicates data to a local store via `IXxxSyncAdapter`), or **Business-process** (inbound event triggers multi-step orchestration) | Determines which L1 steps run | | 3 | Does a new DTO contract need to be created for the event/command payload, or does one already exist? | Gates Step 1 (`coreex-contract`) | -| 4 | (Business-process and command scenarios) Does the handler need to persist state to a new local entity table? | Gates Steps 2 and 3 (migration + repository) | -| 5 | (If `data-provider` was unrecorded) Does this solution have local persistence (SqlServer / Postgres)? | Required before Steps 2–3 can be evaluated | +| 4 | (Business-process and command scenarios) Does the handler need to persist state to a new local store shape (relational table or Cosmos container)? | Gates Steps 2 and 3 (migration + repository) | +| 5 | (If `data-provider` was unrecorded) Does this solution have local persistence (`SqlServer` / `Postgres` / `Cosmos`)? | Required before Steps 2–3 can be evaluated | **Do not ask** about `messaging-provider`, `outbox-enabled`, or `rop-enabled` if already recorded in Feature Configuration — restate the resolved values for confirmation instead. @@ -61,11 +61,11 @@ Skip this step if the event/command DTO type already exists in the Contracts pro --- -### Step 2 — `coreex-db-migration` _(business-process scenario only, when `data-provider ≠ None` and a new local entity table is needed)_ +### Step 2 — `coreex-db-migration` _(business-process scenario only, when `data-provider ≠ None` and a new local persistence shape is needed — relational table or Cosmos container)_ Invoke [`coreex-db-migration`](../coreex-db-migration/SKILL.md). -Pass: table name, columns derived from the entity properties. This step is only needed when the business-process scenario creates a new locally-owned entity that does not yet have a table. +Pass: relational table name + columns, or the Cosmos container/persistence-model shape derived from the entity properties. This step is only needed when the business-process scenario creates a new locally-owned entity that does not yet have a persistence home. --- diff --git a/.github/skills/coreex-subscriber/SKILL.md b/.github/skills/coreex-subscriber/SKILL.md index c17726b8c..74e9119a3 100644 --- a/.github/skills/coreex-subscriber/SKILL.md +++ b/.github/skills/coreex-subscriber/SKILL.md @@ -35,6 +35,7 @@ There are three distinct subscriber scenarios — determine which applies before ## When Not to Use +- The full command slice (contract, dedicated queue receiver, topology, tests, Aspire) — use `coreex-command-subscribe-e2e`, which invokes this skill for the subscriber step - HTTP API controllers — use `coreex-api` - Application services that the subscriber calls — use `coreex-app-service` - Replication adapter implementations (`IXxxSyncAdapter`) — use `coreex-adapter` diff --git a/.github/skills/coreex-subscriber/references/workflow.md b/.github/skills/coreex-subscriber/references/workflow.md index 9a2e4b5aa..1bc6cc86a 100644 --- a/.github/skills/coreex-subscriber/references/workflow.md +++ b/.github/skills/coreex-subscriber/references/workflow.md @@ -311,7 +311,7 @@ restores inventory. - **Subject version suffix (`.v{n}`) is payload-driven**, not event-type-driven — add it when the message carries a data schema; omit it for key-only messages - **Outbox assertion helper is database-specific** — use `UseExpectedSqlServerOutboxPublisher()` for - SQL Server domains; `UseExpectedPostgresOutboxPublisher()` for PostgreSQL; do not mix them + SQL Server domains; `UseExpectedPostgresOutboxPublisher()` for PostgreSQL; `UseExpectedCosmosDbOutboxPublisher()` for Cosmos; do not mix them - **Always specify seed files explicitly** — `read-data.seed.yaml` for read tests, `mutate-data.seed.yaml` for mutate/subscribe tests, `no-data.seed.yaml` for schema-only tests (health, relay); never rely on auto-discovery of YAML files in the test common assembly diff --git a/.github/skills/coreex-test-api/SKILL.md b/.github/skills/coreex-test-api/SKILL.md index 1cbb75fe3..d1a37f3a6 100644 --- a/.github/skills/coreex-test-api/SKILL.md +++ b/.github/skills/coreex-test-api/SKILL.md @@ -1,7 +1,7 @@ --- name: coreex-test-api description: "Write or update an integration test in a CoreEx *.Test.Api project. USE FOR: new XxxReadTests/XxxMutateTests partial classes, per-operation test files (Get/Query/Create/Update/Patch/Delete), OneTimeSetUp seeding + cache + outbox wiring, seed data (read-data.seed.yaml/mutate-data.seed.yaml), .res.json/.req.json resources, ETag/concurrency and soft-delete scenarios, outbox event assertions, inter-domain HTTP mocking. DO NOT USE FOR: Subscribe host tests (use coreex-test-subscribe), Outbox Relay host tests (use coreex-test-relay), pure unit tests with no infrastructure (validators/aggregates/adapters already cover their own Test.Unit guidance), the controller/endpoint implementation itself (use coreex-api)." -argument-hint: "Entity/endpoint name, operations to cover (Get/Query/Create/Update/Patch/Delete), database provider (PostgreSQL/SQL Server)" +argument-hint: "Entity/endpoint name, operations to cover (Get/Query/Create/Update/Patch/Delete), database provider (PostgreSQL/SQL Server/Cosmos)" tags: ["testing", "integration-tests", "api", "unittestex", "coreex"] --- @@ -35,7 +35,7 @@ setup shared with Subscribe host tests (`coreex-test-subscribe` links back here - Implementing the controller/endpoint itself — use `coreex-api` > **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` -> **Feature Configuration**: `data-provider` (PostgreSQL vs SQL Server) selects the migrate/seed and +> **Feature Configuration**: `data-provider` (PostgreSQL vs SQL Server vs Cosmos) selects the migrate/seed and > outbox helper family used throughout — never mix them; `outbox-enabled` and `messaging-provider` > determine whether outbox-event assertions (`ExpectXxxOutboxEvents`) apply at all. Only prompt for > what is unrecorded; re-state resolved values for confirmation. @@ -43,7 +43,7 @@ setup shared with Subscribe host tests (`coreex-test-subscribe` links back here ## Quick Reference - **One class pair per entity**: `XxxReadTests` (seeds `read-data.seed.yaml`) / `XxxMutateTests` (seeds `mutate-data.seed.yaml`) — each a `partial class`, one operation per sub-file: `Xxx{Read|Mutate}Tests.{Operation}.cs` -- **`[OneTimeSetUp]`** lives in the base partial file: migrate + seed (named-file overload) → `Test.ClearFusionCacheAsync()` → provider-specific `Test.UseExpected{Postgres|SqlServer}OutboxPublisher()` → HTTP mocks if the domain has adapters +- **`[OneTimeSetUp]`** lives in the base partial file: migrate + seed (named-file overload — `MigratePostgresDataAsync`, `MigrateSqlServerDataAsync`, or `MigrateCosmosDataAsync`) → `Test.ClearFusionCacheAsync()` → provider-specific `Test.UseExpected{Postgres|SqlServer|CosmosDb}OutboxPublisher()` → HTTP mocks if the domain has adapters (if an adapter caches via `IHybridCache`, also add a per-test `[SetUp] => Test.ClearFusionCacheAsync()` — Redis is shared across tests and runs) - **Seed → Tests → Resources**, in that order — seed known `^N` rows first, write tests referencing them, capture `.res.json`/`.req.json` from the actual run - **Seed the precondition state directly — don't chain API calls to reach it.** A test needing e.g. a *cancelled* booking should seed a row with that status, not `Create → Confirm → Cancel` through the API just to arrange it. Exceptions: the transition itself is under test (`Cancel_Success`), the value is only known at runtime (ETag — read via `GET` first), or the thing being asserted is a side effect of the action (an outbox event). See `coreex-tests.instructions.md` → "Prefer Seeded State Over API-Orchestrated Setup" - **`read-data.seed.yaml`/`mutate-data.seed.yaml` are the preferred default, not the only option** — the seed overload takes a list, so a class with data too distinct to fit cleanly (e.g. several precondition-state rows for one workflow) can add its own file alongside or instead of the shared pair @@ -53,7 +53,7 @@ setup shared with Subscribe host tests (`coreex-test-subscribe` links back here - **412 vs 409 vs 428**: stale ETag → `AssertPreconditionFailed()` (412); duplicate/business conflict → `AssertConflict()` (409); no ETag supplied → `428` (auto on PUT/PATCH, or POST/DELETE opted in via `ro.WithIfMatchRequired()`) — assert with `.Assert(HttpStatusCode.PreconditionRequired)`, no dedicated helper exists - **Delete is idempotent**: always `AssertNoContent()` (204), never `AssertNotFound()` on DELETE — the 404 belongs to the follow-up GET; only the first delete emits an outbox event - **Verify persistence, not just the echo**: every Create/Update/Patch `_Success` test explicitly asserts the specific changed property/properties on the mutation response (e.g. `updated.Salary.Should().Be(60000.00m)`), then ends with a follow-up `GET` + `AssertValue(...)` — the mutation response alone only proves the handler echoed the value back, not that the intended value was actually written -- **Outbox assertions are provider-specific** — `ExpectPostgresOutboxEvents`/`ExpectSqlServerOutboxEvents`; `.AssertWithValue(destination, subject)` for value-carrying events (Create/Update), `.AssertMetadata(destination, subject, key)` for no-value events (Delete) — `key` is optional (`AssertMetadata(destination, subject)` checks routing only). If the event's payload type differs from the API's returned type (e.g. a leaner projection), use `.AssertWithValue(() => expectedPayload, destination, subject)` instead of the plain form +- **Outbox assertions are provider-specific** — `ExpectPostgresOutboxEvents`/`ExpectSqlServerOutboxEvents`/`ExpectCosmosDbOutboxEvents`; `.AssertWithValue(destination, subject)` for value-carrying events (Create/Update), `.AssertMetadata(destination, subject, key)` for no-value events (Delete) — `key` is optional (`AssertMetadata(destination, subject)` checks routing only). If the event's payload type differs from the API's returned type (e.g. a leaner projection), use `.AssertWithValue(() => expectedPayload, destination, subject)` instead of the plain form - **Every event-publishing call needs an explicit expectation** — `UseExpectedXxxOutboxPublisher` defaults to asserting **zero** events when none is declared; omitting it on a call that legitimately publishes fails the test. Prefer `AssertMetadata`/`AssertWithValue` (confirms *which* event) over a bare `ExpectXxxOutboxEvents()` (confirms only that *some* event was published) - **Reference-data JSON name**: non-`Code` suffix (`gender`, not `genderCode`) in `.res.json`/`.req.json`/inline bodies - **HTTP client mocking**: `MockHttpClientFactory` + `MockHttpClientRequest` fields configured in `OneTimeSetUp`, per-test `.Respond.With(...)`/`.Respond.WithJsonResource(...)`, always `.Verify()` @@ -69,3 +69,4 @@ For full workflow, decision trees, and code patterns see [`references/workflow.m - Illustrative examples (CoreEx sample — not present in your project): - [Contoso.Products.Test.Api](https://github.com/Avanade/CoreEx/tree/main/samples/tests/Contoso.Products.Test.Api) — PostgreSQL domain example (outbox + HTTP mock adapter) - [Contoso.Shopping.Test.Api](https://github.com/Avanade/CoreEx/tree/main/samples/tests/Contoso.Shopping.Test.Api) — SQL Server domain example (outbox + ETag/concurrency) + - [Contoso.Customers.Test.Api](https://github.com/Avanade/CoreEx/tree/main/samples/tests/Contoso.Customers.Test.Api) — Cosmos domain example (`MigrateCosmosDataAsync`, `UseExpectedCosmosDbOutboxPublisher`) diff --git a/.github/skills/coreex-test-api/references/workflow.md b/.github/skills/coreex-test-api/references/workflow.md index 43e20d074..cec668331 100644 --- a/.github/skills/coreex-test-api/references/workflow.md +++ b/.github/skills/coreex-test-api/references/workflow.md @@ -12,7 +12,7 @@ Identify: entity name, which operations need coverage (Get/Query/Create/Update/Patch/Delete), and the domain's database provider (check `*.Database/Program.cs` or `appsettings.json` — PostgreSQL vs SQL -Server). The provider determines which outbox helper family to use throughout (never mix them). +Server vs Cosmos). The provider determines which migrate/seed and outbox helper family to use throughout (never mix them). ## Phase 1 — Seed Data First @@ -86,9 +86,9 @@ public partial class EmployeeMutateTests : WithApiTester [OneTimeSetUp] public async Task OneTimeSetUpAsync() { - await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // or MigratePostgresDataAsync(...) — provider-specific + await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // or MigratePostgresDataAsync(...) / MigrateCosmosDataAsync(..., DbMigration.ConfigureProvisionArgs) — provider-specific await Test.ClearFusionCacheAsync().ConfigureAwait(false); - Test.UseExpectedSqlServerOutboxPublisher(); // or UseExpectedPostgresOutboxPublisher() — provider-specific + Test.UseExpectedSqlServerOutboxPublisher(); // or UseExpectedPostgresOutboxPublisher() / UseExpectedCosmosDbOutboxPublisher() — provider-specific } } ``` @@ -427,7 +427,7 @@ values, include them; don't exclude them. The only volatile fields are `id`/`eta - [ ] Read/mutate split into separate classes with separate seed files - [ ] Named-file seed overload used (not the no-arg "load everything" overload) - [ ] One seed row per destructive test, not per operation -- [ ] Provider-correct outbox helpers used throughout (no Postgres/SQL Server mixing) +- [ ] Provider-correct outbox helpers used throughout (no Postgres/SQL Server/Cosmos mixing) - [ ] Delete tested as the idempotent 4-step flow; never `AssertNotFound()` on DELETE - [ ] Create/Update/Patch `_Success` tests assert the specific changed property/properties explicitly (`updated.Prop.Should().Be(expected)`), then re-GET and `AssertValue(...)` to verify the change persisted (not just that the response echoed it back) - [ ] ETag concurrency asserted as 412, not 409; 428 case understood for missing-ETag scenarios diff --git a/.github/skills/coreex-test-relay/SKILL.md b/.github/skills/coreex-test-relay/SKILL.md index 210b4dd8a..b517ea6a1 100644 --- a/.github/skills/coreex-test-relay/SKILL.md +++ b/.github/skills/coreex-test-relay/SKILL.md @@ -37,21 +37,25 @@ rarely need new scenarios per domain. > **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` > **Feature Configuration**: `messaging-provider` gates this skill (a Relay host exists only when a -> messaging provider is configured) and `data-provider` (PostgreSQL vs SQL Server) selects the outbox -> publisher family (`PostgresOutboxPublisher`/`SqlServerOutboxPublisher`) — never mix them. Only prompt +> messaging provider is configured) and `data-provider` (PostgreSQL vs SQL Server vs Cosmos) selects the outbox +> publisher family (`PostgresOutboxPublisher`/`SqlServerOutboxPublisher`/`CosmosDbEventPublisher`) — never mix them. Only prompt > for what is unrecorded; re-state resolved values for confirmation. ## Quick Reference - **Base class**: `WithApiTester<{Domain}.Relay.Program>` — relay hosts have **no** FusionCache; do not call `ClearFusionCacheAsync()` here -- **No DB/cache seeding needed for the core forwarding test** — it writes directly to the outbox via `Test.ScopedType` and a provider-specific outbox publisher (`PostgresOutboxPublisher`/`SqlServerOutboxPublisher`), then waits for the background relay service to forward it -- **Assert delivery** via `Test.GetAndClearAzureServiceBusAsync(...)` against the expected topic/subscription +- **No business-data seeding needed for relational forwarding tests** — write directly to the outbox via `Test.ScopedType` and a provider-specific outbox publisher (`PostgresOutboxPublisher`/`SqlServerOutboxPublisher`). Cosmos requires provisioned business/lease containers and an outbox event paired with a business mutation in the same transaction. +- **Assert delivery** via `Test.GetAndClearAzureServiceBusAsync(...)` against the expected topic/subscription — or, for a **command**, the target domain queue: `Test.GetAndClearAzureServiceBusAsync(ServiceBusSessionReceiverOptions.CreateForQueue("{destination}-{target}"))` (the queue must exist in Test.Common `ServiceBus.GetQueues()`; see `coreex-command-publish-e2e`) - **Hosted-service management endpoints** are also testable over plain HTTP — `/hosted-services/{name}/pause`, `/resume`, etc. -- **A domain rarely needs more than the templated `RelayTests.cs` + `OtherTests.Health.cs` + `OtherTests.HostedServices.cs`** — check what already exists before writing something new +- **Check existing coverage before extending it** — relational hosts usually need only `RelayTests.cs` + `OtherTests.Health.cs` + `OtherTests.HostedServices.cs`. Cosmos tests should cover every outbox-hosting container, cleanup, the first-start/checkpoint policy, and pause/resume. ```csharp public class RelayTests : WithApiTester { + [OneTimeSetUp] + public async Task OneTimeSetUpAsync() + => await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()); // Test.Common ServiceBus: the code-based topology. + [Test] public async Task Outbox_Relay() { @@ -59,7 +63,7 @@ public class RelayTests : WithApiTester { test.Run(async _ => { - var pub = ActivatorUtilities.GetServiceOrCreateInstance(test.Services); // or SqlServerOutboxPublisher — provider-specific + var pub = ActivatorUtilities.GetServiceOrCreateInstance(test.Services); // or SqlServerOutboxPublisher — provider-specific relational example pub.Add("{solution}", [ce1, ce2]); await pub.PublishAsync(); @@ -82,6 +86,8 @@ Test.Http() .Response.StatusCode.Should().Be(HttpStatusCode.Accepted); ``` +For **Cosmos** relay tests, publish via `CosmosDbEventPublisher` inside `CosmosDbUnitOfWork.TransactionAsync(...)`, paired with a real business mutation, and use bounded polling to assert broker delivery and outbox cleanup. Keep the relay's default `IEventPublisher` registered as its broker destination. Provision containers before accessing the tester: even `Test.Configuration` constructs the host and captures its first-start boundary. Use a test-owned observation subscription and do not reset containers beneath a running change-feed processor. The Customers sample below demonstrates both `PartitionKey.None` and explicit partition keys. + ## Troubleshooting — Service Bus Emulator "Entity Not Found" If a Relay (or any Service Bus) test fails with: @@ -92,8 +98,8 @@ The messaging entity 'sb://sbemulatorns.servicebus.onebox.windows-int.net/ **Resolve project-wide choices from state before asking.** Read the solution-root `AGENTS.md` > **Feature Configuration**: `messaging-provider` gates this skill (Subscribe host tests exist only when -> a messaging provider is configured); `data-provider` (PostgreSQL vs SQL Server) selects the +> a messaging provider is configured); `data-provider` (PostgreSQL vs SQL Server vs Cosmos) selects the > migrate/seed and outbox helper family used throughout — never mix them; `outbox-enabled` determines > whether outbox-event assertions apply. Only prompt for what is unrecorded; re-state resolved values. ## Quick Reference -- **Base class**: `WithApiTester<{Domain}.Subscribe.Program>` — same DB/cache/outbox `[OneTimeSetUp]` shape as API tests (migrate + seed via named-file overload → `ClearFusionCacheAsync()` → provider-specific `UseExpected{Postgres|SqlServer}OutboxPublisher()`); Subscribe hosts **do** have FusionCache (reference data, idempotency) +- **Base class**: `WithApiTester<{Domain}.Subscribe.Program>` — same DB/cache/outbox `[OneTimeSetUp]` shape as API tests (migrate + seed via named-file overload → `ClearFusionCacheAsync()` → provider-specific `UseExpected{Postgres|SqlServer|CosmosDb}OutboxPublisher()`); Subscribe hosts **do** have FusionCache (reference data, idempotency) - **Simulate receipt**: build an `EventData` → `Test.CreateCloudEventFrom(ed)` → `.ToServiceBusReceivedMessage()` → resolve `ServiceBusSubscribedSubscriber` from DI → `.ReceiveAsync(sbm)` +- **Key matters**: event subscribers resolve the key `"subscriber-events"`; **command** subscribers (dedicated queue receiver) resolve `"subscriber-commands"` — see `coreex-command-subscribe-e2e`. Resolving the wrong key throws or hits the wrong receiver - **One partial file per subscriber scenario** — `SubscriberTests.{Scenario}.cs` - **Match the test shape to the subscriber scenario**: command → assert outcome + outbox events published as a result; event-data-sync → assert local state/adapter reflects the payload; event-business-process → assert the downstream service ran and published its own events - **`ErrorHandler` outcomes are assertable** — a handled exception surfaces as `EventSubscriberHandledException` with `.ErrorHandling` and `.InnerException` to check diff --git a/.github/skills/coreex-test-subscribe/references/workflow.md b/.github/skills/coreex-test-subscribe/references/workflow.md index 073434f68..f7f08316f 100644 --- a/.github/skills/coreex-test-subscribe/references/workflow.md +++ b/.github/skills/coreex-test-subscribe/references/workflow.md @@ -38,10 +38,11 @@ public partial class SubscriberTests : WithApiTester<{Domain}.Subscribe.Program> { // Always specify the seed file explicitly — read-data.seed.yaml, mutate-data.seed.yaml, // or a schema-only no-data.seed.yaml for plumbing/health-only tests. - await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // or MigratePostgresDataAsync(...) — provider-specific + await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // or MigratePostgresDataAsync(...) / MigrateCosmosDataAsync(..., DbMigration.ConfigureProvisionArgs) — provider-specific await Test.ClearFusionCacheAsync().ConfigureAwait(false); + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); // Test.Common ServiceBus: the code-based topology. - Test.UseExpectedSqlServerOutboxPublisher(); // or UseExpectedPostgresOutboxPublisher() — provider-specific + Test.UseExpectedSqlServerOutboxPublisher(); // or UseExpectedPostgresOutboxPublisher() / UseExpectedCosmosDbOutboxPublisher() — provider-specific } } ``` @@ -51,7 +52,7 @@ One partial file per subscriber scenario under test: `SubscriberTests.{Scenario} ## Phase 2 — Simulating Message Receipt Build the `EventData` the subscriber expects, convert it to a `ServiceBusReceivedMessage`, resolve -`ServiceBusSubscribedSubscriber` from DI, and call `.ReceiveAsync(sbm)` — no live Service Bus connection +the keyed `ServiceBusSubscribedSubscriber` (key `"subscriber-events"`, as registered by the templated Subscribe host) from DI, and call `.ReceiveAsync(sbm)` — no live Service Bus connection needed. ```csharp @@ -64,7 +65,7 @@ var ed = EventData.CreateCommand("{domain}", "{entity}", "{action}").WithKey(ref var ce = Test.CreateCloudEventFrom(ed); var sbm = ce.ToServiceBusReceivedMessage(); -var sbs = test.Services.GetRequiredService(); +var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); ``` @@ -84,14 +85,14 @@ public void {Entity}{Action}_Success() => Test.Scoped(async test => items.Should().HaveCount(3); // Act — simulate the command message; assert any resulting outbox events. - test.ExpectSqlServerOutboxEvents(e => e.AssertCount(3)) // or ExpectPostgresOutboxEvents(...) — provider-specific + test.ExpectSqlServerOutboxEvents(e => e.AssertCount(3)) // or ExpectPostgresOutboxEvents(...) / ExpectCosmosDbOutboxEvents(...) — provider-specific .Run(async _ => { var ed = EventData.CreateCommand("{domain}", "{entity}", "{action}").WithKey(referenceId); var ce = Test.CreateCloudEventFrom(ed); var sbm = ce.ToServiceBusReceivedMessage(); - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); }).AssertSuccess(); @@ -117,7 +118,7 @@ public void {Entity}{Action}_NotFound() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); @@ -148,7 +149,7 @@ public void {Entity}Modify_Success() => Test.Scoped(async test => // Act. test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); }).AssertSuccess(); @@ -179,7 +180,7 @@ public void Unsubscribed_CompletesSilently() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); @@ -194,7 +195,7 @@ public void Unsubscribed_CompletesSilently() => Test.Scoped(test => - [ ] Test shape matches the subscriber scenario (command / event-data-sync / event-business-process) - [ ] `OneTimeSetUp` clears FusionCache — Subscribe hosts have it, don't skip it -- [ ] Provider-correct outbox helpers used (no Postgres/SQL Server mixing) +- [ ] Provider-correct outbox helpers used (no Postgres/SQL Server/Cosmos mixing) - [ ] `ErrorHandler` outcomes asserted where the subscriber defines one (handled exception → `.ErrorHandling` + `.InnerException`) - [ ] Event-data-sync tests assert via the adapter/local store, not the raw external contract - [ ] An "unsubscribed subject" test exists for the host diff --git a/.github/skills/coreex-validator/SKILL.md b/.github/skills/coreex-validator/SKILL.md index 95cab155c..f15a4c00f 100644 --- a/.github/skills/coreex-validator/SKILL.md +++ b/.github/skills/coreex-validator/SKILL.md @@ -28,7 +28,8 @@ Guides you through creating or modifying a CoreEx validator (`Application/Valida ## When Not to Use - Domain invariants (aggregates, entities, value objects) — those belong in the Domain layer -- Infrastructure-level data checks that are not accessed via an Application-layer interface +- External HTTP response-contract validation — use `coreex-adapter`; small client-only rules belong inline in the Infrastructure client, not `Application/Validators/` +- Other Infrastructure-level data checks that are not accessed via an Application-layer interface - `FluentValidation` NuGet package — `AbstractValidator` here is `CoreEx.Validation.AbstractValidator` ## Quick Reference diff --git a/.github/skills/coreex-validator/references/workflow.md b/.github/skills/coreex-validator/references/workflow.md index 07a7e24a9..2c6048940 100644 --- a/.github/skills/coreex-validator/references/workflow.md +++ b/.github/skills/coreex-validator/references/workflow.md @@ -10,6 +10,12 @@ Full workflow for creating or modifying a CoreEx validator in `Application/Validators/`. Follow the path that matches the request. +For external HTTP response-contract rules, use the +[coreex-adapter workflow](../../coreex-adapter/references/workflow.md#response-contract-validation) +instead. Prefer a private static readonly inline `Validator` in the Infrastructure client; +test it through client unit tests. This workflow's standalone Application-validator paths do not +apply to that boundary-specific pattern. + --- ## Phase 1 — Clarify Before Writing @@ -60,6 +66,7 @@ public class {Name}Validator : Validator | Run rule only when entity condition met | `.WhenEntity(e => e.StartDate.HasValue)` chained after the rule | | Run rule only when property value condition met | `.WhenValue(v => v != null)` chained after the rule (predicate on property value, not entity — use `WhenEntity` for entity conditions) | | Skip when dependent prop invalid | `.DependsOn(x => x.OtherProp)` at end of chain | +| Compare to another property on the same entity | `.CompareProperty(CompareOperator.GreaterThanOrEqualTo, x => x.StartsOn)` — compares only when both have a value and are valid; add `.Mandatory()` first to require it | | Runtime-computed threshold | `.LessThanOrEqualTo(_ => DateOnly.FromDateTime(Runtime.UtcNow.UtcDateTime.AddYears(-16)), _ => "the minimum age of 16")` | **Mandatory() on non-nullable value types:** @@ -315,7 +322,7 @@ var dv = Validator.Create() - Use `(jsonName, "Full expected message.")` tuples in `AssertErrors(...)`. - Use camelCase JSON property path; ref-data rules key on the navigation name (`"gender"`, not `"genderCode"`). - Use sentence-case labels in expected messages (`"First name is required."` not `"FirstName is required."`). -4. If a ref-data type is used in the test host that is not yet registered, add the corresponding case to `EntryPoint.ReferenceDataServiceDecorator.GetAsync`. +4. If a ref-data type is used in the test host that is not yet registered, add the corresponding case to `EntryPoint.ReferenceDataProviderDecorator.GetAsync`. --- diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 56fd8005e..bc2cccbde 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -92,6 +92,10 @@ jobs: # --- Samples --- + # The core Cosmos tests churn the emulator's databases; restart it (in-memory, so it comes back clean) while the samples build. + - name: Restart Cosmos emulator + run: docker compose restart cosmos-emulator + - name: Build samples run: dotnet build CoreEx.Samples.Build.slnf --no-restore --configuration Release @@ -103,6 +107,10 @@ jobs: working-directory: samples/src/Contoso.Shopping.CodeGen run: dotnet run refdata --no-build --configuration Release -f net10.0 --expect-no-changes + - name: CodeGen tool (samples/src/Contoso.Customers.CodeGen) + working-directory: samples/src/Contoso.Customers.CodeGen + run: dotnet run refdata --no-build --configuration Release -f net10.0 --expect-no-changes + - name: Database tool (samples/src/Contoso.Products.Database) working-directory: samples/src/Contoso.Products.Database run: dotnet run all --no-build --configuration Release -f net10.0 --expect-no-changes @@ -115,15 +123,40 @@ jobs: working-directory: samples/src/Contoso.Orders.Database run: dotnet run all --no-build --configuration Release -f net10.0 --expect-no-changes + - name: Wait for Cosmos emulator + timeout-minutes: 6 + run: | + # The vNext emulator reports readiness (gateway + explorer + backing store) on its health port (moved to 8083 in docker-compose.yml to avoid the DTS emulator on 8080). + until curl -sf http://localhost:8083/ready -o /dev/null; do + echo "Waiting for the Cosmos emulator..."; sleep 5 + done + + - name: Database tool (samples/src/Contoso.Customers.Database) + working-directory: samples/src/Contoso.Customers.Database + run: dotnet run all --no-build --configuration Release -f net10.0 + - name: Test samples run: dotnet test CoreEx.Samples.Test.slnf --no-build --configuration Release --logger trx --collect "XPlat Code Coverage" --results-directory coverage/samples -m:1 -p:TestTfmsInParallel=false -p:BuildInParallel=false + # The E2E hosts call each other over https using the ASP.NET Core dev certificate, which 'dotnet dev-certs https --trust' can't trust on Linux; add it to the system store instead. + - name: Trust ASP.NET Core dev certificate + run: | + dotnet dev-certs https + dotnet dev-certs https --export-path aspnetcore-dev.pem --format Pem --no-password + sudo cp aspnetcore-dev.pem /usr/local/share/ca-certificates/aspnetcore-dev.crt + sudo update-ca-certificates + rm aspnetcore-dev.pem + + # The Aspire end-to-end tests orchestrate every sample host together, so they run last (after all the intra-domain sample tests) and on .NET 10 only (the AppHost is single-TFM). + - name: Test samples (Aspire end-to-end, net10.0 only) + timeout-minutes: 20 + run: dotnet test CoreEx.Samples.Test.E2E.slnf --no-build --configuration Release --logger trx --results-directory coverage/samples-e2e -m:1 -p:TestTfmsInParallel=false -p:BuildInParallel=false - name: Upload samples test results if: always() uses: actions/upload-artifact@v4 with: name: samples-test-results - path: "coverage/samples/**/*.trx" + path: "coverage/samples*/**/*.trx" retention-days: 30 # --- Coverage --- diff --git a/AGENTS.md b/AGENTS.md index 2a392e632..82524cd53 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,7 +15,7 @@ reference data, validation, and data access — into a consistent, composable ba - ✅ Multi-domain .NET service topologies (microservices, event-driven, hexagonal architecture) - ✅ Teams that want opinionated, pattern-aligned scaffolding from day one - ✅ Organisations that need consistent HTTP/event behaviour across many services -- ✅ Projects using Entity Framework Core + SQL Server or PostgreSQL +- ✅ Projects using Entity Framework Core + SQL Server or PostgreSQL, or Azure Cosmos DB - ✅ Solutions that publish domain events with transactional guarantees (outbox pattern) - ⚠️ Not a general-purpose framework — CoreEx is intentionally opinionated; teams wanting full flexibility over error handling, HTTP response shaping, and event publishing will be constrained @@ -33,7 +33,7 @@ reference data, validation, and data access — into a consistent, composable ba | `CoreEx.EntityFrameworkCore` | EF Core integration, typed CRUD, `ValueConverter` bridges | | `CoreEx.RefData` | Typed reference data with hybrid-cache-backed orchestrator | | `CoreEx.Caching.FusionCache` | `IHybridCache` backed by ZiggyCreatures FusionCache (L1/L2 + Redis backplane) | -| `CoreEx.Cosmos` | **Preview — newly added; API surface may still change without following strict semver until it stabilizes.** Typed Azure Cosmos DB access: `CosmosDbContainer`/`CosmosDbMappedContainer` for CRUD + query with ETag/multi-tenancy/logical-delete support, a `TransactionalBatch`-based transactional outbox, and a Change Feed Processor-based outbox relay | +| `CoreEx.Cosmos` | Typed Azure Cosmos DB access: `CosmosDbContainer`/`CosmosDbMappedContainer` for CRUD + query with ETag/multi-tenancy/logical-delete support, a `TransactionalBatch`-based transactional outbox, and a Change Feed Processor-based outbox relay | | `CoreEx.Data` | OData-esque dynamic querying (`QueryArgs`/`PagingArgs`/`QueryArgsConfig`), `ItemsResult` | | `CoreEx.Data.GraphQL` | Transport-agnostic GraphQL-lite bridge (`IGraphQLEngine`) over `CoreEx.Data` querying + `JsonFilter` field projection; hosted via `CoreEx.AspNetCore`'s `MapCoreExGraphQLLite` | | `CoreEx.UnitTesting` | Fluent test toolkit: event assertions, outbox assertions, JSON seed data | @@ -69,8 +69,9 @@ Rules that are easy to violate and cause real breakage or wrong choices: - **`GlobalUsings.cs`** — every project has a single `GlobalUsings.cs` at the project root; all `using` statements go there, never in individual source files. The Roslyn code generator emits no `using` statements and depends on this. - **`AwesomeAssertions` not FluentAssertions** — tests use the `AwesomeAssertions` NuGet package. Do not reach for FluentAssertions. -- **Polyglot data** — Products uses PostgreSQL (`CoreEx.Database.Postgres`); Shopping uses SQL Server (`CoreEx.Database.SqlServer`). Do not assume SQL Server when working on Products, and do not mix outbox/publisher helpers across domains. +- **Polyglot data** — Products uses PostgreSQL (`CoreEx.Database.Postgres`); Shopping uses SQL Server (`CoreEx.Database.SqlServer`); Customers uses Azure Cosmos DB (`CoreEx.Cosmos`). Do not assume SQL Server when working on Products, do not assume EF Core/DbEx in Cosmos domains, and do not mix outbox/publisher helpers across domains. - **No AutoMapper** — do not introduce AutoMapper. All mapping is explicit via `Mapper<>` (application layer) or `BiDirectionMapper<>` (infrastructure layer). +- **Aspire external infrastructure** — use `CoreEx.UnitTesting`'s `AddExternalConnectionString` extension (`UnitTestEx` namespace) for visible graph nodes, preserving connection names and Compose lifecycle ownership. Plain `AddConnectionString(name)` creates parameter-only nodes excluded from Aspire 13.5.4's Graph/Table views. Cosmos's configured `AccountEndpoint` is exposed as a credential-free resource URL for trace matching. Node status reflects configuration resolution, not connectivity; hosts own readiness probes. See [Aspire guidance](./samples/docs/aspire.md). - **`.ConfigureAwait(false)`** — always use it in service and repository code. - **File-scoped namespaces** — `namespace Foo.Bar;` only; never block-scoped `namespace Foo.Bar { }`. - **Query materialization across data-access packages** — a new data-access package (e.g. `CoreEx.Cosmos`, a future `CoreEx.MongoDb`) should not materialize queries via `IQueryable` extension methods named with a bare, generic-sounding verb (`ToListAsync`, `ToItemsResultAsync`, `ToMappedItemsAsync`, etc.) if another CoreEx package (or a third-party one, e.g. `Microsoft.EntityFrameworkCore`) already defines an identically-shaped one on the same receiver type. C# extension-method resolution has no precedence rule between two equally-applicable candidates from different namespaces — it is a hard `CS0121` ambiguous-call compile error in any file that imports both namespaces, not merely a style clash. The preferred fix is a package-owned query-wrapper type (e.g. `CoreEx.Cosmos`'s `CosmosDbQuery`, returned from `CosmosDbContainer.Query(...)` instead of a bare `IQueryable`) so materializers are instance methods on that type — a different receiver type structurally cannot collide, so plain names (`ToListAsync`, `ToItemsResultAsync`, ...) are safe there; see [`CoreEx.Cosmos/AGENTS.md`](./src/CoreEx.Cosmos/AGENTS.md#do-not) for the worked example. Only fall back to prefixing with the provider name (`ToCosmosListAsync`, `ToCosmosItemsResultAsync`, ...) if a package genuinely must expose `IQueryable` extensions directly instead of owning a wrapper type. @@ -130,8 +131,8 @@ This installs: - `.github/skills/` — the CoreEx skill suite: `coreex-bootstrap`, `coreex-docs-sync`, `coreex-scaffold`, the L1 skills (`coreex-contract`, `coreex-refdata`, `coreex-db-migration`, `coreex-repository`, `coreex-adapter`, `coreex-app-service`, `coreex-validator`, `coreex-policy`, `coreex-aggregate`, `coreex-api`, `coreex-subscriber`, - `coreex-test-api`, `coreex-test-subscribe`, `coreex-test-relay`), and the L2 end-to-end skills - (`coreex-api-e2e`, `coreex-subscriber-e2e`) + `coreex-test-api`, `coreex-test-subscribe`, `coreex-test-relay`, `coreex-aspire`), and the L2 end-to-end skills + (`coreex-api-e2e`, `coreex-subscriber-e2e`, `coreex-command-publish-e2e`, `coreex-command-subscribe-e2e`) - `.github/agents/coreex-expert.agent.md` — architecture guidance agent - `.github/docs/coreex/` — the architecture docs + per-package guides cache, self-describing via `.github/docs/coreex/manifest.txt` (refresh later, version-pinned, with `/coreex-docs-sync`) diff --git a/CHANGELOG.md b/CHANGELOG.md index 58e8ef182..e8528441c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,34 @@ Represents the **NuGet** versions. +## v4.1.0 +- *Enhancement:* Added `NamedDestinationProvider` (`AddNamedDestinationProvider()`); events are sent to the shared destination (topic), whereas commands are sent to a per-domain queue named `{Destination}-{domain}` so that only the addressed domain receives them. A `null` or empty command domain is self-addressed to the `IHostSettings.DomainName`. +- *Enhancement:* Added keyed `WithReceiver`/`WithSessionReceiver` and `WithKeyedSubscribedSubscriber` support to `AzureServiceBusReceiving` to enable multiple receivers (e.g. events topic and commands queue) within a single Subscribe host. +- *Enhancement:* Updated `CoreEx.Template` hosts to use the `NamedDestinationProvider` and a keyed subscriber by default, with the Service Bus topology now code-based in the generated `Test.Common` `ServiceBus` class. +- *Enhancement:* Added/extended end-to-end (`WithAspireTester`) Aspire testing capabilities within `CoreEx.UnitTesting`: `DistributedApplication` helpers to migrate SQL Server/PostgreSQL databases, clear the Redis cache and reset the Azure Service Bus emulator (queues, topics and subscriptions) from code; `AddEndpoints`, `AddHostedServiceSupport` and `DisableHttpCertificateValidation` AppHost extensions; and a WireMock-based `MockHost` to stub third-party HTTP dependencies. The `CoreEx.Template` `coreex-aspire` template now generates the AppHost, `MockHost` and `Test.Aspire` projects. +- *Enhancement:* Added `coreex-command-publish-e2e`, `coreex-command-subscribe-e2e` and `coreex-aspire` AI skills, with associated instructions, prompts and documentation. +- *Enhancement:* Added ReferenceDataOrchestrator.TryInvalidateAsync (best-effort, awaited, failure logged and swallowed) for post-commit cache eviction; the generated reference data write service now uses it so a cache outage cannot fail an already-committed mutation. Also, ReferenceDataHybridCache.RemoveAsync now serializes with in-flight loads to prevent re-caching stale data. +- *Enhancement:* Samples migrated to command queues; Aspire end-to-end tests now included in CI (`net10.0` only). +- *Enhancement:* Updated `CoreEx.CodeGen` to support mutability of reference data (side-effect was a rename of the previously generated `ReferenceDataService` (manually remove existing) to `ReferenceDataProvider`). +- *Enhancement:* Added `CosmosDbReferenceData` (Cosmos counterpart of `EfDbReferenceData`), `CosmosDbReferenceDataModelBase` and `CosmosDbContainerOptions.WithReferenceDataOutboxEvent`; `CoreEx.CodeGen` reference-data `mutability` now supports `repository: Cosmos` (unique `/typeDiscriminator` + `/code` key for duplicate-code detection). +- *Enhancement:* Added a scoped change tracker to `CoreEx.Cosmos` (`ICosmosDb.ChangeTracker`, `CosmosDbArgs.ClearChangeTrackerAfterGet`): repeat point gets are served from a per-scope snapshot cache, and `UpdateAsync` now pre-reads to copy back server-managed values (change-log `Created*`, `ttl`, unmapped extension data) and skip the replace when nothing changed. `CosmosDbModelBase` implements the new `CoreEx.Json.IExtensionData`. +- *Enhancement:* Added `CoreEx.Cosmos.Provisioning` (`CosmosDbProvisioner`, `CosmosDbConsole`, `CosmosDbProvisionArgs`, `CosmosDbContainerDefinition`): a reusable DbEx-style Drop/Create/Reset/Data console for Cosmos DB (dev/test; as per DbEx `Reset` does not implicitly create the database, and `Drop`/`Reset` prompt for confirmation unless `--accept-prompts` is specified), plus `CosmosDbBatch.ImportDiscriminatedBatchAsync` at container level and `MigrateCosmosDataAsync` in `CoreEx.UnitTesting` (first call `Create | ResetAndData`, then `ResetAndData`, as per `MigratePostgresDataAsync`). The seed-data `JsonDataReaderOptions` are overridable per container and per data file (`AddDataResource(..., dataOptions)` per file, `Container(..., dataOptions:)`/`ReferenceDataContainer(..., dataOptions:)` per container; the resource wins). The Customers sample now has a `Contoso.Customers.Database` project. As per DbEx, a `$`-prefixed seed key (container, or ref-data type group) is merged (upserted, re-runnable) rather than inserted, and a `^`-prefixed type group generates the `id` where not specified. +- *Enhancement:* Added `RuntimeMetadata.AreEqual` overload that excludes root-level properties from the comparison. +- *Enhancement:* Added `[NonNullable]` support for OpenAPI schemas to describe mandatory DTO properties that remain nullable in C#. +- *Enhancement:* Added fluent typed HTTP response validation and cached-result helpers for adapters; invalid successful dependency responses are surfaced as dependency failures rather than accepted or cached. +- *Enhancement:* Relational outbox publishers and relay hosts now share `CoreEx:Host:Outbox:PartitionSize` (default `4`, range `1`–`256`); configure it consistently in every writer and relay host. The former `CoreEx:Host:Services:OutboxRelay:PartitionSize` key is removed and causes relay startup to fail with a migration message. Keyless relational outbox events now use the stable `$none` sentinel rather than a random partition. Service Bus session hashing is independently configurable through `CoreEx:Host:ServiceBus:SessionIdPartitionSize`, with per-destination strategy and bucket overrides under `CoreEx:Host:ServiceBus:Destinations`. +- *Fix:* EF `Created*` audit columns are no longer generated as updatable/overwritten on update, and the default EF update mapping now excludes the change-log `Created*` and tenant properties (regenerate `*DbContext.g.cs` via `*.Database`). +- *Fixed:* `TimerHostedServiceBase` race where stopping (or `OneOffTrigger`/pause/resume) could `Cancel()` an already-disposed per-interval `CancellationTokenSource`, throwing `ObjectDisposedException` from `StopAsync` (intermittent host/test teardown failures). Also fixed abandoned interval-wait semaphore waiters accumulating per interval (unbounded growth) and swallowing subsequent wake-up signals (e.g. a resume could leave the service stuck paused). +- *Fixed:* `Extensions.IsCanceled()` now detects an `OperationCanceledException` wrapped in nested `AggregateException`s (as produced by multiple `Result.ThrowOnError()` boundaries); previously only a single level was detected, so cancellations could be misreported as failures. +- *Fixed:* `DatabaseOutboxRelayBase` now releases (cancels) a failed batch claim using a separate bounded token (new `CancelBatchTimeout`, default 30 seconds) rather than the already-cancelled lease-duration token, which previously left the claimed events locked until the database lease expired. A lease-duration timeout is now logged as a warning and treated as a partition-level failure (continuing to the next partition) rather than being rethrown as a cancellation that hosting services interpret as a stop request; a caller-requested cancellation still flows. +- *Fixed:* `CoreEx.Cosmos` `CosmosDbOutboxRelay` now self-recovers where the monitored and/or lease container is deleted and recreated whilst running (e.g. a provisioning `Reset`); previously the Change Feed Processor remained bound to the deleted containers, endlessly reporting lost leases (412/404) and never relaying another event until the host was restarted. A new `CosmosDbOutboxRelayOptions.RecoveryDelay` (default 5 seconds) controls the delay before/between recovery attempts. +- *Fixed:* Cosmos relay fresh leases explicitly read from the beginning, preventing existing outbox events from being skipped on first startup or container-reset recovery by the SDK's default "from now" behavior. Existing checkpoints take precedence; an explicitly configured `StartTime` remains supported. +- *Fixed:* Cosmos change-tracker snapshot publication is version-checked across model types, and immediate writes evict on completion as well as before mutation. Reads overlapping writes, invalidation, or a whole-tracker clear can no longer repopulate the cache with stale snapshots. +- *Fixed:* `CoreEx.Template` Relay host (and the Contoso sample relays) `OutboxRelay:LeaseDuration` increased from 5 to 30 seconds; the previous value was shorter than a single Azure Service Bus throttling back-off (~10 seconds), resulting in sends being cancelled mid-retry and batches repeatedly abandoned under load. +- *Fixed:* Products sample `MovementRepository` now adjusts inventory with atomic PostgreSQL statements (upsert for increases/adjustments, guarded `UPDATE` for decreases) rather than a tracked read-modify-write, so concurrent reservations for the same product no longer fail with a `DbUpdateConcurrencyException` (HTTP 500) and can never over-reserve. +- *Fixed:* Added `Extensions.IsCanceledBy(Exception, CancellationToken)` (nested-`AggregateException` aware) and applied it consistently so a caller-requested cancellation (shutdown/client disconnect) is no longer misclassified, swallowed or logged as an error: event subscriber/`SubscribedManager` error classification, Service Bus receiver (no longer converted per `UnhandledErrorHandling`, e.g. dead-lettered, on shutdown), `WebApi` (unhandled-exception conversion; a disconnect while reading the request body is no longer a 400), `WithTotalCountAsync` (no longer swallowed), and Error-level logging in `Database.GetConnectionAsync`, `CosmosDbInvoker`, `ReferenceDataOrchestrator` and `CosmosDbOutboxRelayProcessor`. +- *Breaking:* `IReferenceDataCache` has a new required `RemoveAsync(Type, CancellationToken)` member (any custom implementation must add it; `ReferenceDataHybridCache` already does), and `IIdentifierGenerator.AssignIdentifierAsync` now constrains `TFor` to `class, IIdentifier`. Both are internal-plumbing extension points that are rarely implemented outside the framework, hence not treated as a major version. + ## v4.0.0 - This is a **major** version release; a re-imagine / re-invention of the existing capabilities to enable a more modern, flexible and maintainable codebase. - This release contains **significant breaking changes** - there is **no** upgrade path from the previous `v3.x` versions; however, the core capabilities and patterns remain largely consistent. diff --git a/CoreEx.Samples.Build.slnf b/CoreEx.Samples.Build.slnf index e8a141bf0..a16288d3b 100644 --- a/CoreEx.Samples.Build.slnf +++ b/CoreEx.Samples.Build.slnf @@ -2,12 +2,15 @@ "solution": { "path": "CoreEx.slnx", "projects": [ + "samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj", "samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj", + "samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj", "samples/src/Contoso.Customers.Api/Contoso.Customers.Api.csproj", "samples/src/Contoso.Customers.Application/Contoso.Customers.Application.csproj", "samples/src/Contoso.Customers.CodeGen/Contoso.Customers.CodeGen.csproj", "samples/src/Contoso.Customers.Contracts/Contoso.Customers.Contracts.csproj", "samples/src/Contoso.Customers.Infrastructure/Contoso.Customers.Infrastructure.csproj", + "samples/src/Contoso.Customers.Relay/Contoso.Customers.Relay.csproj", "samples/src/Contoso.Order.Workflow.Client/Contoso.Order.Workflow.Client.csproj", "samples/src/Contoso.Order.Workflow.Worker/Contoso.Order.Workflow.Worker.csproj", "samples/src/Contoso.Order.Workflow.Workflow/Contoso.Order.Workflow.Workflow.csproj", @@ -35,6 +38,7 @@ "samples/src/Contoso.Shopping.Subscribe/Contoso.Shopping.Subscribe.csproj", "samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj", "samples/tests/Contoso.Customers.Test.Common/Contoso.Customers.Test.Common.csproj", + "samples/tests/Contoso.Customers.Test.Relay/Contoso.Customers.Test.Relay.csproj", "samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj", "samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj", "samples/tests/Contoso.Orders.Test.Api/Contoso.Orders.Test.Api.csproj", @@ -52,4 +56,4 @@ "samples/tests/Contoso.Shopping.Test.Unit/Contoso.Shopping.Test.Unit.csproj" ] } -} \ No newline at end of file +} diff --git a/CoreEx.Samples.Test.E2E.slnf b/CoreEx.Samples.Test.E2E.slnf new file mode 100644 index 000000000..7b0c291ca --- /dev/null +++ b/CoreEx.Samples.Test.E2E.slnf @@ -0,0 +1,8 @@ +{ + "solution": { + "path": "CoreEx.slnx", + "projects": [ + "samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj" + ] + } +} diff --git a/CoreEx.Samples.Test.slnf b/CoreEx.Samples.Test.slnf index 31a2cc0d6..ea0e091f6 100644 --- a/CoreEx.Samples.Test.slnf +++ b/CoreEx.Samples.Test.slnf @@ -3,6 +3,7 @@ "path": "CoreEx.slnx", "projects": [ "samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj", + "samples/tests/Contoso.Customers.Test.Relay/Contoso.Customers.Test.Relay.csproj", "samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj", "samples/tests/Contoso.Orders.Test.Api/Contoso.Orders.Test.Api.csproj", "samples/tests/Contoso.Orders.Test.Unit/Contoso.Orders.Test.Unit.csproj", @@ -16,4 +17,4 @@ "samples/tests/Contoso.Shopping.Test.Unit/Contoso.Shopping.Test.Unit.csproj" ] } -} \ No newline at end of file +} diff --git a/CoreEx.sln b/CoreEx.sln index 5ae51acd9..fd1e3411a 100644 --- a/CoreEx.sln +++ b/CoreEx.sln @@ -281,6 +281,12 @@ Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Contoso.Customers.Test.Comm EndProject Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Contoso.Customers.Test.Api", "samples\tests\Contoso.Customers.Test.Api\Contoso.Customers.Test.Api.csproj", "{B9CFAE96-BF7B-4E5F-B59A-671C847FCDAA}" EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Contoso.Customers.Database", "samples\src\Contoso.Customers.Database\Contoso.Customers.Database.csproj", "{1D33A260-681C-4FAB-9971-EF230696AEC4}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Contoso.Customers.Relay", "samples\src\Contoso.Customers.Relay\Contoso.Customers.Relay.csproj", "{508DEFC9-0DA5-4BDD-8E45-2182926B7685}" +EndProject +Project("{FAE04EC0-301F-11D3-BF4B-00C04F79EFBC}") = "Contoso.Customers.Test.Relay", "samples\tests\Contoso.Customers.Test.Relay\Contoso.Customers.Test.Relay.csproj", "{2E8AFD2A-FCC8-4C13-8779-E04222405404}" +EndProject Global GlobalSection(SolutionConfigurationPlatforms) = preSolution Debug|Any CPU = Debug|Any CPU @@ -1335,6 +1341,42 @@ Global {B9CFAE96-BF7B-4E5F-B59A-671C847FCDAA}.Release|x64.Build.0 = Release|Any CPU {B9CFAE96-BF7B-4E5F-B59A-671C847FCDAA}.Release|x86.ActiveCfg = Release|Any CPU {B9CFAE96-BF7B-4E5F-B59A-671C847FCDAA}.Release|x86.Build.0 = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|Any CPU.Build.0 = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|x64.ActiveCfg = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|x64.Build.0 = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|x86.ActiveCfg = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Debug|x86.Build.0 = Debug|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|Any CPU.ActiveCfg = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|Any CPU.Build.0 = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|x64.ActiveCfg = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|x64.Build.0 = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|x86.ActiveCfg = Release|Any CPU + {1D33A260-681C-4FAB-9971-EF230696AEC4}.Release|x86.Build.0 = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|Any CPU.Build.0 = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|x64.ActiveCfg = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|x64.Build.0 = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|x86.ActiveCfg = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Debug|x86.Build.0 = Debug|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|Any CPU.ActiveCfg = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|Any CPU.Build.0 = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|x64.ActiveCfg = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|x64.Build.0 = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|x86.ActiveCfg = Release|Any CPU + {508DEFC9-0DA5-4BDD-8E45-2182926B7685}.Release|x86.Build.0 = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|Any CPU.ActiveCfg = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|Any CPU.Build.0 = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|x64.ActiveCfg = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|x64.Build.0 = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|x86.ActiveCfg = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Debug|x86.Build.0 = Debug|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|Any CPU.ActiveCfg = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|Any CPU.Build.0 = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|x64.ActiveCfg = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|x64.Build.0 = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|x86.ActiveCfg = Release|Any CPU + {2E8AFD2A-FCC8-4C13-8779-E04222405404}.Release|x86.Build.0 = Release|Any CPU EndGlobalSection GlobalSection(SolutionProperties) = preSolution HideSolutionNode = FALSE @@ -1443,11 +1485,14 @@ Global {E3CC4EC9-E1B5-43FC-ACAE-3860BD4E3266} = {3DE0C8B7-B7FF-A272-13CB-2C0B8C361F5F} {0EBCF41E-FE1D-C1B6-6D1C-8EB39D4921FC} = {B3AB6B39-2911-9174-0680-BA3663BCEA26} {AF3856DD-B11F-402E-9AD3-D0846EF2EFB2} = {0EBCF41E-FE1D-C1B6-6D1C-8EB39D4921FC} + {1D33A260-681C-4FAB-9971-EF230696AEC4} = {0EBCF41E-FE1D-C1B6-6D1C-8EB39D4921FC} {36A24C9E-2496-A023-30B1-0BC8D8B24E54} = {B3AB6B39-2911-9174-0680-BA3663BCEA26} {5A056D15-2E67-4689-976F-FFA184B0CF68} = {36A24C9E-2496-A023-30B1-0BC8D8B24E54} {6AE52727-2385-B6B6-F414-0FC24AF94819} = {B3AB6B39-2911-9174-0680-BA3663BCEA26} {1EC2BA33-781A-49D7-BCD1-CCFFBE9F2E03} = {6AE52727-2385-B6B6-F414-0FC24AF94819} {2197E3FB-BE43-495E-BDF6-0B9CDBDD356F} = {36A24C9E-2496-A023-30B1-0BC8D8B24E54} {B9CFAE96-BF7B-4E5F-B59A-671C847FCDAA} = {36A24C9E-2496-A023-30B1-0BC8D8B24E54} + {508DEFC9-0DA5-4BDD-8E45-2182926B7685} = {6AE52727-2385-B6B6-F414-0FC24AF94819} + {2E8AFD2A-FCC8-4C13-8779-E04222405404} = {36A24C9E-2496-A023-30B1-0BC8D8B24E54} EndGlobalSection EndGlobal diff --git a/CoreEx.slnx b/CoreEx.slnx index 15b570ccc..87696666d 100644 --- a/CoreEx.slnx +++ b/CoreEx.slnx @@ -12,12 +12,15 @@ + + + @@ -27,10 +30,12 @@ + + @@ -115,6 +120,7 @@ + diff --git a/Directory.Packages.props b/Directory.Packages.props index 16f6c58eb..b7889c434 100644 --- a/Directory.Packages.props +++ b/Directory.Packages.props @@ -18,6 +18,7 @@ + @@ -26,7 +27,6 @@ - @@ -69,15 +69,18 @@ - - + + - - + + + + + diff --git a/README.md b/README.md index 29ce8fee0..376b3cb7d 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ _CoreEx_ helps teams build enterprise-grade back-end services faster — without Built-in AI support already knows _CoreEx_ patterns, conventions, and architecture — powering purpose-built skills and an expert agent that guide delivery toward production-grade, best-practice outcomes, consistently, every time. -_CoreEx_ also addresses the recurring concerns that .NET leaves to each team to solve independently: shaping business rule failures into consistent HTTP responses, publishing and subscribing to domain events, implementing reference data with caching and validity, enforcing validation with structured error messages, and wiring data access across ADO.NET and Entity Framework Core with a shared unit-of-work model. +_CoreEx_ also addresses the recurring concerns that .NET leaves to each team to solve independently: shaping business rule failures into consistent HTTP responses, publishing and subscribing to domain events and commands, implementing reference data with caching and validity, enforcing validation with structured error messages, and wiring data access across ADO.NET, Entity Framework Core, and Azure Cosmos DB with a shared unit-of-work model. The solution is composed of focused, independently consumable packages — `CoreEx` provides the shared runtime primitives, and each additional package (`CoreEx.AspNetCore`, `CoreEx.Events`, `CoreEx.Database`, `CoreEx.Validation`, etc.) adds a specific capability layer. Teams adopt only what they need. @@ -44,10 +44,10 @@ The remainder of this README is organized into the following sections. _CoreEx_ is intended to be the foundational library for any business-service implementation — the motivation for adopting it comes down to four things: -- **Value** — focus on business-value delivery; stop re-inventing the same foundational plumbing on every project. -- **Velocity** — reduce cost and time-to-market by inheriting foundational capabilities rather than building them up-front. -- **Consistency** — encapsulate complexity behind proven patterns so every service is familiar to build and predictable to run. -- **Confidence** — improve quality and reduce delivery risk by leaning on capabilities that are already proven and tested. +- **Value** — focus on business-value delivery; stop re-inventing the same foundational plumbing every time. +- **Velocity** — reduce delivery time by pairing the framework with agentic AI that builds it for you. +- **Consistency** — every service looks, builds, and behaves the same way, no matter who is on the team. +- **Confidence** — improve quality and reduce delivery risk by leaning on capabilities that are proven and tested. ## Better Together @@ -85,22 +85,27 @@ Here is a high-level overview of some of the key capabilities provided by _CoreE **Web / API** - 🌐 **ASP.NET Core Integration** — `WebApi` execution helpers for MVC and Minimal API, `ExecutionContext` scoping middleware, idempotency key handling (`[IdempotencyKey]`), and exception-to-`ProblemDetails` translation with consistent error response shaping. -- 📋 **OpenAPI / NSwag** — `IOperationProcessor` that reads CoreEx MVC attributes (`[Paging]`, `[Query]`, `[Accepts]`, `[IdempotencyKey]`, `[ProducesNotFoundProblem]`) and injects the corresponding parameters, request bodies, and response entries into the generated OpenAPI specification. +- 📋 **OpenAPI / NSwag** — `IOperationProcessor` that reads CoreEx MVC attributes (`[Paging]`, `[Query]`, `[Accepts]`, `[IdempotencyKey]`, `[ProducesNotFoundProblem]`) and injects the corresponding parameters, request bodies, and response entries into the generated OpenAPI specification; `[NonNullable]` describes mandatory nullable DTO properties correctly. +- **Typed HTTP Clients** — Fluent response validation and cached-result helpers for domain adapters calling other services. +- **GraphQL-lite** — An opt-in query bridge over existing `QueryArgsConfig` and paging, with JSON field projection rather than a separate resolver/filter implementation. **Data** - ✅ **Validation Framework** — Fluent, property-centric validator with composable rules, conditional clauses, async predicates, `RuleSet` grouping, base-validator include, FluentValidation interop bridge, and deep `ValidationException` / `ProblemDetails` integration. - 📊 **Data Access & OData-style Querying** — `QueryArgsConfig`, `QueryFilterParser`, and `QueryOrderByParser` for safe, explicitly-configured `$filter` and `$orderby` LINQ translation; query arguments, paging, partitioning, logical deletion, and `IUnitOfWork` transactional orchestration. - 🗄️ **Database & ORM** — `IDatabase` / `DatabaseCommand` ADO.NET abstraction with multi-result-set support, transactional outbox relay infrastructure, SQL Server and PostgreSQL implementations with OpenTelemetry metrics, and an Entity Framework Core integration layer with typed CRUD operations and `ValueConverter` bridges. -- 📚 **Reference Data** — Typed `ReferenceData` / `ReferenceData` base classes, thread-safe `Id`- and `Code`-indexed collections, hybrid-cache-backed orchestrator with per-type semaphore loading, contextual date-validity checking, and a code-serialization collection for wire-compatible reference-data properties. +- **Azure Cosmos DB** — Typed CRUD/query access with ETags, multi-tenancy, logical deletion, scoped snapshot tracking, a transactional-batch outbox, and change-feed relaying with backlog replay on fresh leases. +- 📚 **Reference Data** — Typed `ReferenceData` / `ReferenceData` base classes, thread-safe collections, hybrid caching and date validity; opt-in generated mutation APIs backed by EF Core or Cosmos, with validation and post-commit cache invalidation. - ⚡ **Hybrid Caching** — `IHybridCache` abstraction with `FusionHybridCache` (ZiggyCreatures FusionCache) as the recommended implementation, providing L1/L2 hybrid storage and backplane-based cache invalidation. **Messaging** - 📨 **Event Publishing & Subscribing** — `EventData` ↔ CloudEvents formatting, a two-phase queue-then-publish pipeline, and configurable subscriber dispatch with per-subscriber structured error handling and dead-letter support. -- 🚌 **Azure Service Bus** — `ServiceBusPublisher` implementing `IEventPublisher`, subscriber bases wired to `EventSubscriberBase`, and receiver hosts with built-in resiliency, OpenTelemetry metrics, and session support. +- 🚌 **Azure Service Bus** — `ServiceBusPublisher` implementing `IEventPublisher`, named routing of commands to per-domain queues while events stay on a shared topic, keyed receivers, resiliency, OpenTelemetry metrics, and independently configurable per-destination session policies. +- **Partitioning Boundaries** — Shared partition sizing across relational outbox writers and relays, deterministic keyless routing, and separate Service Bus session grouping; consumer concurrency and broker partitions are distinct concerns. **Tooling** -- 🏗️ **Reference Data Code Generation** — Development-time `CoreEx.CodeGen` tooling that scaffolds the complete reference-data layer (contract, controller, service, repository interface, repository, and mapper) from a single schema-validated `ref-data.yaml` configuration file. -- 🧪 **Testing Toolkit** — `CoreEx.UnitTesting` provides fluent value expectations, event-capture and outbox assertions (SQL Server, PostgreSQL, Azure Service Bus), JSON/YAML seed-data loading with placeholder substitution, and UnitTestEx bridge extensions covering every CoreEx subsystem. +- 🏗️ **Reference Data Code Generation** — Development-time `CoreEx.CodeGen` tooling that scaffolds contracts, read providers/controllers, repositories and mappers, plus opt-in mutation services/controllers, from a single schema-validated `ref-data.yaml` configuration file. +- **Provisioning & Scaffolding** — Composable `CoreEx.Template` solution/host/AI templates, relational database lifecycle tooling, and a Cosmos provisioning console for declared containers and seed data. +- 🧪 **Testing Toolkit** — `CoreEx.UnitTesting` provides fluent expectations, event/outbox assertions (SQL Server, PostgreSQL, Cosmos, Azure Service Bus), JSON/YAML seeds, and Aspire E2E lifecycle helpers with real cross-domain hosts and a WireMock host for third-party HTTP. ## Patterns in Practice @@ -133,7 +138,7 @@ Project/Package | Description | Source `CoreEx.AspNetCore.NSwag`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.AspNetCore.NSwag)](https://www.nuget.org/packages/CoreEx.AspNetCore.NSwag/absoluteLatest) | Provides the NSwag `IOperationProcessor` integration that reads CoreEx MVC attributes (`[Paging]`, `[Query]`, `[Accepts]`, `[IdempotencyKey]`, `[ProducesNotFoundProblem]`) and injects the corresponding parameters, request bodies, and response entries into the generated OpenAPI specification. | [Link](./src/CoreEx.AspNetCore.NSwag) `CoreEx.Azure.Messaging.ServiceBus`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Azure.Messaging.ServiceBus)](https://www.nuget.org/packages/CoreEx.Azure.Messaging.ServiceBus/absoluteLatest) | Provides Azure Service Bus integration for CoreEx: a `ServiceBusPublisher` implementing `IEventPublisher`, subscriber bases wired to `EventSubscriberBase`, and receiver hosts with built-in resiliency, metrics, and session support. | [Link](./src/CoreEx.Azure.Messaging.ServiceBus) `CoreEx.Caching.FusionCache`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Caching.FusionCache)](https://www.nuget.org/packages/CoreEx.Caching.FusionCache/absoluteLatest) | Provides a `FusionHybridCache` implementation of `IHybridCache` backed by the ZiggyCreatures FusionCache library, bridging CoreEx caching contracts to FusionCache's L1/L2 hybrid and backplane capabilities. | [Link](./src/CoreEx.Caching.FusionCache) -`CoreEx.Cosmos`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Cosmos)](https://www.nuget.org/packages/CoreEx.Cosmos/absoluteLatest) | **Preview — newly added; API surface may still change without following strict semver until it stabilizes.** Provides the core Azure Cosmos DB access layer: `ICosmosDb`/`CosmosDb` as the CoreEx-Cosmos bridge, `CosmosDbContainer` and `CosmosDbMappedContainer` for typed CRUD + query operations with ETag/concurrency, multi-tenancy, logical-delete, and type-discriminator support, a `TransactionalBatch`-based transactional outbox (`CosmosDbUnitOfWork`), and a Change Feed Processor-based outbox relay. | [Link](./src/CoreEx.Cosmos) +`CoreEx.Cosmos`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Cosmos)](https://www.nuget.org/packages/CoreEx.Cosmos/absoluteLatest) | Provides the core Azure Cosmos DB access layer: `ICosmosDb`/`CosmosDb` as the CoreEx-Cosmos bridge, `CosmosDbContainer` and `CosmosDbMappedContainer` for typed CRUD + query operations with ETag/concurrency, multi-tenancy, logical-delete, and type-discriminator support, a `TransactionalBatch`-based transactional outbox (`CosmosDbUnitOfWork`), and a Change Feed Processor-based outbox relay. | [Link](./src/CoreEx.Cosmos) `CoreEx.Data`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Data)](https://www.nuget.org/packages/CoreEx.Data/absoluteLatest) | Provides the `IUnitOfWork` transactional orchestration contract, `DataResult` mutation outcome types, data model base classes, and the `QueryArgsConfig` / `QueryFilterParser` / `QueryOrderByParser` pipeline for safe, explicitly-configured OData-style `$filter` and `$orderby` LINQ query translation. | [Link](./src/CoreEx.Data) `CoreEx.Data.GraphQL`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Data.GraphQL)](https://www.nuget.org/packages/CoreEx.Data.GraphQL/absoluteLatest) | Provides a transport-agnostic "GraphQL-lite" bridge (`IGraphQLEngine`) over the existing `CoreEx.Data` querying pipeline: a single registered root field maps a GraphQL selection set to `QueryArgs`/`PagingArgs` and `JsonFilter` field projection, reusing each entity's existing `QueryArgsConfig` — no new per-entity resolver code required. | [Link](./src/CoreEx.Data.GraphQL) `CoreEx.Database`
 
[![NuGet version](https://img.shields.io/nuget/vpre/CoreEx.Database)](https://www.nuget.org/packages/CoreEx.Database/absoluteLatest) | Provides the `IDatabase` / `Database` ADO.NET abstraction, `DatabaseCommand` fluent query builder, `DatabaseRecord` row reader, multi-result-set support, convention-based column mapping, database wildcard translation, typed mapper contracts, and the transactional outbox relay infrastructure for publishing events from a relational database. | [Link](./src/CoreEx.Database) @@ -174,7 +179,7 @@ The repository includes [Contoso reference samples](./samples/README.md) that im > **AI agents:** see [AGENTS.md](./AGENTS.md) for a structured CoreEx summary and the consumer cold-start installation steps. -The repository includes an AI workflow set in [`.github/`](./.github/) that gives GitHub Copilot and Claude Code authoritative knowledge of CoreEx patterns, conventions, and architecture — no need to explain CoreEx to the tool each time. The artefacts can also be copied into a consuming project. +The repository includes an AI workflow set in [`.github/`](./.github/) that gives GitHub Copilot and Claude Code authoritative knowledge of CoreEx patterns, conventions, and architecture — no need to explain CoreEx to the tool each time. Install the consumer bundle with `dotnet new coreex-ai` from a `CoreEx.Template::` release matching the project's CoreEx packages; instructions, skills, commands, agent, and cached docs stay on one version. **Agent** — `coreex-expert` provides architecture guidance, pattern recommendations, and design reviews aligned to the sample implementations. See the [agent README](./.github/agents/README.md) for the resolution flowchart and local doc cache design. @@ -183,15 +188,17 @@ The repository includes an AI workflow set in [`.github/`](./.github/) that give | Command | What it does | Claude Code | GitHub Copilot Chat | |---------|-------------|-------------|---------------------| | [Expert guidance](./.github/agents/README.md) | Architecture, pattern, and design advice | `@coreex-expert` | Agent mode → **CoreEx Expert** | -| [CoreEx.Template](./src/CoreEx.Template/README.md) | Deterministic solution and host scaffolding via `dotnet new coreex*` templates | `dotnet new install CoreEx.Template` then `dotnet new coreex...` | Run the same `dotnet new` commands in the terminal | +| [CoreEx.Template](./src/CoreEx.Template/README.md) | Deterministic solution and host scaffolding via `dotnet new coreex*` templates | `dotnet new install CoreEx.Template::` then `dotnet new coreex...` | Run the same `dotnet new` commands in the terminal | | [`coreex-ai`](./src/CoreEx.Template/README.md#template-1----coreex-ai-ai-workflow-assets) | Installs this AI workflow set — agents, skills, prompts, instructions — into any repository | `dotnet new coreex-ai` | Run the same `dotnet new` command in the terminal | -**Skills — the AI-augmented building blocks** — sixteen skills codify CoreEx's architectural patterns into guided, correct-by-construction workflows: fourteen per-capability (L1) skills that add or modify a single building block, and two end-to-end (L2) skills that orchestrate a full vertical slice in one pass. Each is invoked as `/coreex-` (Claude Code) or via the matching `prompts/coreex-.prompt.md` (Copilot). +**Skills — the AI-augmented building blocks** — twenty skills codify CoreEx's architectural patterns into guided, correct-by-construction workflows: sixteen per-capability (L1) skills that add or modify a single building block, and four end-to-end (L2) skills that orchestrate a full vertical slice in one pass. Each is invoked as `/coreex-` (Claude Code) or via the matching `prompts/coreex-.prompt.md` (Copilot). | Group | Skill | What it does | |---|---|---| | **End-to-end (L2)** | `coreex-api-e2e` | New entity + full CRUD API — contract, migration, repository, validator, service, endpoint, and tests — in one guided workflow | -| | `coreex-subscriber-e2e` | New event/command subscriber — contract, handler, service, and tests — in one guided workflow | +| | `coreex-subscriber-e2e` | New event subscriber (data-sync or business-process) — contract, handler, service, and tests — in one guided workflow | +| | `coreex-command-publish-e2e` | Send a **command** to another domain — adapter, outbox/direct call site, queue routing, Service Bus topology, API + Relay tests, and Aspire E2E | +| | `coreex-command-subscribe-e2e` | Handle a **command** addressed to this domain — contract, subscriber, dedicated command-queue receiver, topology, Subscribe tests, and Aspire E2E | | | | | | Contracts | `coreex-contract` | Hand-authored DTO/entity contracts — root, subordinate, or extending an existing one | | | `coreex-refdata` | Reference data types — new entity, extra properties, seed rows, wiring into contracts | @@ -201,18 +208,22 @@ The repository includes an AI workflow set in [`.github/`](./.github/) that give | | `coreex-policy` | Application-layer policies enforcing I/O-dependent business guards | | Infrastructure | `coreex-repository` | Infrastructure-layer repositories — CRUD, custom queries, bi-directional mappers | | | `coreex-adapter` | Anti-corruption-layer adapters and typed HTTP clients for cross-domain or external calls | -| | `coreex-db-migration` | Database tables and schema changes for a domain | +| | `coreex-db-migration` | Relational tables/schema or Cosmos container provisioning changes for a domain | | Hosts | `coreex-api` | API controllers or Minimal API endpoints — CRUD and business actions | +| | `coreex-graphql` | GraphQL-lite query bridge (`CoreEx.Data.GraphQL`) on an API host | | | `coreex-subscriber` | Event/command subscribers in a Subscribe host | | Testing | `coreex-test-api` | Integration tests for an API host | | | `coreex-test-subscribe` | Integration tests for a Subscribe host | | | `coreex-test-relay` | Integration tests for an Outbox Relay host | +| | `coreex-aspire` | Aspire AppHost wiring, service-bus topology, MockHost stubs for third-party HTTP, and `Test.Aspire` E2E tests | See the [full skill catalog](./.github/coreex-ai-workflows.md#prompts-skills-and-templates) for full detail on each. -**Also available** — repo-maintenance and local-orchestration skills: [`/coreex-scaffold`](./.github/skills/coreex-scaffold/README.md) (greenfield solution scaffolding), [`/coreex-docs-sync`](./.github/skills/coreex-docs-sync/README.md) (refresh cached CoreEx docs), [`/acquire-codebase-knowledge`](./.github/skills/acquire-codebase-knowledge/README.md) (map an existing codebase), and [`/aspire`](./.github/skills/aspire/README.md) (orchestrate Aspire apps locally). +**Bootstrap and maintenance** — [`/coreex-bootstrap`](./.github/skills/coreex-bootstrap/SKILL.md) initializes a blank repository's root guidance and chains to [`/coreex-scaffold`](./.github/skills/coreex-scaffold/README.md) for solution scaffolding. [`/coreex-docs-sync`](./.github/skills/coreex-docs-sync/README.md) refreshes the entire version-pinned AI bundle, not just the docs cache. Together with the twenty L1/L2 workflows, these are **23 shipped CoreEx skills**. -**Instructions** — 11 scoped instruction files are injected automatically when editing matching file types (contracts, services, repositories, controllers, tests, etc.). No action required. +**Framework-repository-only skills** — [`/acquire-codebase-knowledge`](./.github/skills/acquire-codebase-knowledge/README.md) maps an existing codebase and [`/aspire`](./.github/skills/aspire/README.md) orchestrates Aspire apps locally. Neither is installed by `coreex-ai`. + +**Instructions** — 12 scoped instruction files are injected automatically when editing matching file types (contracts, services, repositories, controllers, tests, etc.). No action required. → **[Full AI workflow overview](./.github/coreex-ai-workflows.md)** diff --git a/Version.props b/Version.props index 3fe9b80e5..2bbd6a4d4 100644 --- a/Version.props +++ b/Version.props @@ -1,5 +1,5 @@ - 4.0.0 + 4.1.0 diff --git a/consumer-instructions/.github/copilot-instructions.md b/consumer-instructions/.github/copilot-instructions.md index 071c2af38..f4e23ae30 100644 --- a/consumer-instructions/.github/copilot-instructions.md +++ b/consumer-instructions/.github/copilot-instructions.md @@ -129,7 +129,7 @@ Every project has a single `GlobalUsing.cs` at its root where all namespace impo - Use exception-based flow for simpler CRUD-oriented services where pipeline composition adds no value. - Use `WebApi` helpers in controllers — never return typed `ActionResult` directly. - Use `[ScopedService]` on service and repository classes for automatic DI discovery — avoid manual `services.AddScoped<>()` registration. -- Use `AddDynamicServicesUsing(...)` in `Program.cs` to register `[ScopedService]`-decorated types — it scans whole **assemblies**. The hosts pass the assemblies via the stable `AssemblyMarker` types (`typeof({Solution}.Application.AssemblyMarker).Assembly`, `…Infrastructure…`), so there is **no compile-time dependency on CodeGen-generated types and no bootstrap/uncomment step** — ref-data wiring is correct as soon as CodeGen runs. Pair it with the **non-generic `AddReferenceDataOrchestrator()`** (binds `IReferenceDataProvider` from DI at runtime). Adding a new entity in an existing assembly does **not** change these calls; add another assembly only for a **new** project with `[ScopedService]` types (e.g. the Subscribe host adds `typeof(Program).Assembly` for its subscribers). Do **not** revert to `AddDynamicServicesUsing()` / `AddReferenceDataOrchestrator()`. +- Use `AddDynamicServicesUsing(...)` in `Program.cs` to register `[ScopedService]`-decorated types — it scans whole **assemblies**. The hosts pass the assemblies via the stable `AssemblyMarker` types (`typeof({Solution}.Application.AssemblyMarker).Assembly`, `…Infrastructure…`), so there is **no compile-time dependency on CodeGen-generated types and no bootstrap/uncomment step** — ref-data wiring is correct as soon as CodeGen runs. Pair it with the **non-generic `AddReferenceDataOrchestrator()`** (binds `IReferenceDataProvider` from DI at runtime). Adding a new entity in an existing assembly does **not** change these calls; add another assembly only for a **new** project with `[ScopedService]` types (e.g. the Subscribe host adds `typeof(Program).Assembly` for its subscribers). Do **not** revert to `AddDynamicServicesUsing()` / `AddReferenceDataOrchestrator()`. ### Mapping @@ -205,4 +205,4 @@ File-scoped instruction files provide detailed guidance when you edit matching f - [CoreEx on GitHub](https://github.com/Avanade/CoreEx) - [Capabilities Guide](https://github.com/Avanade/CoreEx/blob/main/docs/capabilities.md) — deep capability and pattern explanations - [Application Scaffolding Guide](https://github.com/Avanade/CoreEx/blob/main/docs/application-scaffolding-guide.md) — deciding what to scaffold for a new service -- [Sample Reference Architecture](https://github.com/Avanade/CoreEx/tree/main/samples) — complete Contoso domains demonstrating all patterns end-to-end \ No newline at end of file +- [Sample Reference Architecture](https://github.com/Avanade/CoreEx/tree/main/samples) — complete Contoso domains demonstrating all patterns end-to-end diff --git a/docker-compose.yml b/docker-compose.yml index a43976888..91d2f7946 100644 --- a/docker-compose.yml +++ b/docker-compose.yml @@ -51,22 +51,13 @@ services: - "8080:8080" - "8082:8082" - # Used by CoreEx.Cosmos.Test.Unit. Note: under rootless Podman this image can be flaky with the default bridge - # network (its TLS certificate is issued for the IP(s) it discovers at startup, which may not match the - # container's mapped/published address) - if `podman compose up -d cosmos-emulator` doesn't come up cleanly, - # try re-running with `--privileged` or host networking. Clients also need to accept the emulator's - # self-signed certificate (e.g. CosmosClientOptions with a permissive ServerCertificateCustomValidationCallback) - # rather than relying on the emulator's own IP-override/cert-export workarounds. + # Used by CoreEx.Cosmos.Test.Unit and the Customers domain. This is the Linux "vNext" emulator (the legacy + # image with its ~35 container cap and long-uptime wedging is no longer used). The .NET SDK only speaks HTTPS, and + # vNext defaults to HTTP, hence `--protocol https` (self-signed certificate; clients must accept it). + # The health probe defaults to 8080 which clashes with dts-emulator, so it is moved to 8083 (/ready). + # Host networking is used because rootless Podman on Windows (WSL) does not forward the TLS gateway port + # correctly via published ports (the TLS handshake is aborted); it also works as-is with Docker on Linux (CI). cosmos-emulator: - image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:latest - environment: - # Caps the TOTAL number of containers the emulator can host across the whole account (not "partitions per container") - each container the test suite creates consumes one. The test suite currently - # creates 24 distinct containers (see `grep -rn "ContainerId = \"\|const string containerId = \"" tests/CoreEx.Cosmos.Test.Unit`); when this cap is hit, container creation fails with a generic, - # misleading "high demand"/503 ServiceUnavailable that looks like transient load but is actually deterministic - confirmed by reading the container list directly and seeing exactly this many - # already exist. Kept with real headroom above the current count for future test additions, rather than raised just enough to limp past today's number. - AZURE_COSMOS_EMULATOR_PARTITION_COUNT: "35" - AZURE_COSMOS_EMULATOR_ENABLE_DATA_PERSISTENCE: "false" - AZURE_COSMOS_EMULATOR_IP_ADDRESS_OVERRIDE: "127.0.0.1" - ports: - - "8081:8081" - - "10251-10254:10251-10254" \ No newline at end of file + image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-latest + command: ["--protocol", "https", "--health-port", "8083"] + network_mode: host \ No newline at end of file diff --git a/docs/agent-interaction-guide.md b/docs/agent-interaction-guide.md index 9afc0f1d1..f13f700c3 100644 --- a/docs/agent-interaction-guide.md +++ b/docs/agent-interaction-guide.md @@ -113,6 +113,7 @@ The repo already exposes several entry points. Use them intentionally. | Plan or review a new domain shape before scaffolding. | `CoreEx Expert` | Best when you want sample-aligned advice on host shape, layering, and capability choices before running `dotnet new`. | | Add capabilities to an existing domain. | `CoreEx Expert` or normal chat with inspect-first prompts | Best for incremental retrofits such as relay, subscribe, and messaging alignment now that scaffolding is deterministic and capability work is manual. | | Start local dependencies or sample runtime. | `init`, `setup`, or Aspire tooling | Best for environment and sample execution workflows. | +| Send/receive a command, add an event subscriber, or add an Aspire E2E test. | `coreex-command-publish-e2e`, `coreex-command-subscribe-e2e`, `coreex-subscriber-e2e`, `coreex-aspire` | Guided end-to-end workflows covering code, Service Bus topology, tests and Aspire wiring. | ## A Good Question Usually Includes Four Things diff --git a/docs/agent-prompt-recipes.md b/docs/agent-prompt-recipes.md index 9ae12688a..a781e99d7 100644 --- a/docs/agent-prompt-recipes.md +++ b/docs/agent-prompt-recipes.md @@ -216,6 +216,29 @@ Then add the missing pieces required to consume . Keep subscriber logic thin and aligned to repo conventions. ``` +### Send a command to another domain + +```text +Use the coreex-command-publish-e2e skill. +This domain must instruct to (a command, not an event). +Inspect the existing adapters, outbox and NamedDestinationProvider wiring first, then add the adapter, call site, queue routing, tests and Aspire E2E. +``` + +### Handle a command addressed to this domain + +```text +Use the coreex-command-subscribe-e2e skill. +This domain must accept and act on the command sent by . +Add the command contract, application service, subscriber, keyed command-queue receiver, Subscribe tests and Aspire E2E. +``` + +### Add an Aspire end-to-end test + +```text +Use the coreex-aspire skill. +Wire into the AppHost and add a Test.Aspire test that exercises the cross-domain flow. +``` + ### Ask for a retrofit plan explicitly ```text diff --git a/docs/agent-skills-workflow-guide.md b/docs/agent-skills-workflow-guide.md index 7b4fe4282..c2f2f97f9 100644 --- a/docs/agent-skills-workflow-guide.md +++ b/docs/agent-skills-workflow-guide.md @@ -15,6 +15,14 @@ CoreEx now uses the `CoreEx.Template` `dotnet new` template pack for determinist | `CoreEx Expert` | Agent | Review architecture choices and recommend CoreEx-aligned patterns | Shape selection, design review, capability planning | | `/acquire-codebase-knowledge` | Skill | Document and understand an existing codebase | Onboarding and architecture discovery | | `/aspire` | Skill | Orchestrate distributed apps locally | Running and debugging Aspire apps | +| `/coreex-` | Skills (L1) | Add or change one building block (contract, repository, service, API, validator, etc.) | Guided, convention-aligned changes to an existing domain | +| `/coreex-*-e2e` | Skills (L2) | Deliver a whole slice across layers, tests and Aspire | New entity, event subscriber, or command send/receive | +| `/coreex-aspire` | Skill | Wire AppHost, `MockHost` and `Test.Aspire` end-to-end tests | Cross-domain E2E coverage | +| `/coreex-bootstrap` | Skill | Initialize blank-repository guidance and chain to scaffold | First-time repository setup after `coreex-ai` | +| `/coreex-scaffold` | Skill | Choose and execute the appropriate template commands | Guided greenfield scaffolding | +| `/coreex-docs-sync` | Skill | Refresh the whole version-pinned AI bundle | After a CoreEx package version change | + +The consumer bundle ships 23 CoreEx skills (16 L1, four L2, bootstrap, scaffold and docs-sync). Generic `/acquire-codebase-knowledge` and `/aspire` are available in this framework repository but are not installed by `coreex-ai`. See the [complete asset inventory](../.github/coreex-ai-workflows.md#command-catalog). --- @@ -35,6 +43,9 @@ CoreEx now uses the `CoreEx.Template` `dotnet new` template pack for determinist - `dotnet new coreex-api` adds an API host - `dotnet new coreex-relay` adds an outbox relay host - `dotnet new coreex-subscribe` adds a subscriber host +- `dotnet new coreex-domain` adds an optional DDD layer +- `dotnet new coreex-aspire` adds AppHost, MockHost and Test.Aspire projects +- `dotnet new coreex-ai` installs the version-pinned AI workflow and documentation bundle **Output characteristics:** - Deterministic and repeatable @@ -43,11 +54,14 @@ CoreEx now uses the `CoreEx.Template` `dotnet new` template pack for determinist **Example workflow:** ``` -dotnet new install CoreEx.Template +dotnet new install CoreEx.Template:: +dotnet new coreex-ai dotnet new coreex -n Avanade.Erp.Orders dotnet new coreex-api -n Avanade.Erp.Orders.Api ``` +Pin `` to the project's CoreEx package version, or explicitly resolve and record a stable release for first-time adoption. + --- ### 2. `CoreEx Expert` — Shape and Capability Guidance @@ -158,6 +172,32 @@ User: Ready to test Orders and Shopping domains together --- +### 5. `/coreex-*` Workflow Skills — Guided Capability Changes + +**Purpose:** Add or modify CoreEx building blocks in an existing solution by following the repo's instructions and sample patterns, rather than hand-editing from scratch. All L1/L2 skills have a Copilot prompt and a Claude command wrapper (`/coreex-`). The full catalog lives in [`.github/coreex-ai-workflows.md`](../.github/coreex-ai-workflows.md). + +**End-to-end (L2) skills and their supporting Aspire (L1) skill:** + +| Skill | Use when | +|---|---| +| `coreex-api-e2e` | Adding a brand-new entity with CRUD endpoints, from table to integration tests | +| `coreex-subscriber-e2e` | Adding an **event** subscriber (data sync or business-process choreography) | +| `coreex-command-publish-e2e` | This domain must **send a command** to another domain: adapter, outbox vs direct publish, `NamedDestinationProvider` queue routing, Service Bus topology, tests and Aspire E2E | +| `coreex-command-subscribe-e2e` | This domain must **receive a command** addressed to it: contract, app service, subscriber, dedicated keyed queue receiver, tests and Aspire E2E | +| `coreex-aspire` | Wiring a host, queue/topic, inter-domain dependency or third-party stub (`MockHost`) into the AppHost and `Test.Aspire` | + +**Events vs commands:** events go to the shared topic and any interested domain may subscribe; a command goes to the target domain's own queue (`{destination}-{domain}`) and is only ever consumed by that domain. Pick the skill accordingly. + +**Example workflow:** +``` +User: Shopping must tell Products to confirm a reservation after checkout. +→ /coreex-command-publish-e2e in Shopping (adapter + call site + queue routing) +→ /coreex-command-subscribe-e2e in Products (subscriber + keyed queue receiver) +→ /coreex-aspire verifies the cross-domain flow in Test.Aspire +``` + +--- + ## Typical Development Workflows ### Workflow A: Greenfield Domain from Scratch @@ -223,6 +263,12 @@ START: What do you need? ├─ I want to RUN the system locally or DEBUG multiple services │ └─ → Use /aspire │ +├─ I want to SEND or RECEIVE a command, or add an event subscriber +│ └─ → Use /coreex-command-publish-e2e, /coreex-command-subscribe-e2e or /coreex-subscriber-e2e +│ +├─ I want a cross-domain end-to-end (Aspire) test +│ └─ → Use /coreex-aspire +│ └─ I want to PLAN a refactor or understand dependencies └─ → Use /acquire-codebase-knowledge ``` @@ -349,4 +395,5 @@ If alignment is unclear, ask the CoreEx Expert agent. - [CoreEx Capabilities](capabilities.md) — Detailed feature guide - [Application Scaffolding Guide](application-scaffolding-guide.md) — Deep dive on domain structure - [Orchestration Guide](orchestration.md) — Aspire and distributed app patterns +- [Aspire sample guide](../samples/docs/aspire.md) — AppHost, `MockHost` and `Test.Aspire` end-to-end tests - [Agent Interaction Guide](agent-interaction-guide.md) — How to interact with CoreEx agents effectively diff --git a/docs/capabilities.md b/docs/capabilities.md index f77102ec6..1d66582a0 100644 --- a/docs/capabilities.md +++ b/docs/capabilities.md @@ -39,18 +39,22 @@ This document provides detailed explanations of CoreEx capabilities and common p - [SQL Server](#sql-server) - [PostgreSQL](#postgresql) - [ADO.NET Command & Parameter Extensions](#adonet-command--parameter-extensions) + - [Transient Failure Retry](#transient-failure-retry) - [Entity Framework Integration](#entity-framework-integration) - [Messaging & Events](#messaging--events) - [EventData Abstraction](#eventdata-abstraction) - [CloudEvent Interoperability](#cloudevent-interoperability) - [Publish + Subscribe Patterns](#publish--subscribe-patterns) - [Azure Service Bus Integration](#azure-service-bus-integration) + - [Event vs. Command Destination Routing](#event-vs-command-destination-routing) - [Outbox Relay](#outbox-relay-with-partitioning) - [Workflow Orchestration (Durable Task SDK + DTS)](#workflow-orchestration-durable-task-sdk--dts) - [Domain-Driven Design](#domain-driven-design) - [Aggregate & Entity Modeling](#aggregate--entity-modeling) - [Value Objects](#value-objects) - [Integration Events Only](#integration-events-only) +- [Testing](#testing) + - [End-to-End Testing with Aspire](#end-to-end-testing-with-aspire) - [Putting It All Together](#putting-it-all-together-a-typical-request-flow) - [Summary](#summary) @@ -776,6 +780,16 @@ var cmd = connection.CreateCommand("SELECT * FROM [Products] WHERE Id = @Id") Safer, more readable, less repetitive. +### Transient Failure Retry + +**Pattern:** Opt-in, per-invocation retry of database operations that fail with a transient error. + +`IDatabase.IsTransientException(Exception)` classifies whether a failure is worth retrying; it returns `false` by default. `SqlServerDatabase` overrides it to treat deadlocks, lock timeouts, stale-plan recompiles, and Azure SQL Database throttling/failover/connectivity error codes as transient. + +Retrying is **off by default** (blind retry is only safe where the operation is read-only or idempotent). Opt in per invocation by setting `RetryOnTransient = true` on the database args (`DatabaseArgsBase`). The default pipeline is 3 attempts with exponential backoff starting at 2 seconds. + +Only exceptions already classified as transient are retried; anything else is rethrown immediately. Supply `RetryResiliencePipeline` to control attempts/backoff; classification always stays with `IsTransientException`. + ### Entity Framework Integration **Pattern:** CoreEx works with EF Core repositories and unit-of-work patterns. @@ -888,6 +902,21 @@ Handles: In this repository, Azure Service Bus is the **default initial broker implementation** because the sample relay/subscriber hosts are wired around it. The surrounding event model is broader than that specific broker choice: `EventData` is transport-oriented rather than Service Bus-specific. +### Event vs. Command Destination Routing + +**Pattern:** Events fan out to every interested domain; commands are addressed to exactly one. + +`IDestinationProvider` resolves where a message goes. `FixedDestinationProvider` sends everything to one destination. `NamedDestinationProvider` (`AddNamedDestinationProvider()`) distinguishes the message type: + +| Message type | Destination | Example | +|---|---|---| +| Event | The shared destination (topic) | `contoso` | +| Command | A per-domain queue named `{Destination}-{domain}` | `contoso-products` | + +A command with a `null`/empty target domain is self-addressed to the host's own domain (`IHostSettings.DomainName`). The `Destination` defaults from the `CoreEx:Events:Destination` configuration key. + +On the receiving side a single Subscribe host can host multiple receivers (for example the events topic subscription plus its own command queue) using the keyed `WithReceiver`/`WithSessionReceiver` and `WithKeyedSubscribedSubscriber` support. A command is only ever consumed by the domain that owns it. + ### Outbox Relay (with Partitioning) **Pattern:** Dedicated host that reads from database outbox and publishes to broker. @@ -1062,6 +1091,25 @@ _mediator.Publish(new ProductCreatedDomainEvent(...)); // In-process --- +## Testing + +### End-to-End Testing with Aspire + +**Pattern:** Self-host the whole Aspire AppHost from an NUnit test and drive real cross-domain flows. + +`CoreEx.UnitTesting` builds on `UnitTestEx.Aspire` (`WithAspireTester`) so an E2E test starts every host itself, with no need for Aspire to already be running. Intra-domain tests still use real infrastructure with inter-domain calls mocked; the Aspire tests are where the domains genuinely talk to each other. + +`DistributedApplication` helpers reset state from code in `OnBeforeStartAsync`: +- `MigrateSqlServerDataAsync` / `MigratePostgresDataAsync` — migrate and seed the databases. +- `ClearRedisCacheAsync` — clear the Redis cache. +- `ResetAzureServiceBusAsync` — recreate the Service Bus emulator queues, topics and subscriptions. + +AppHost extensions: `AddEndpoints`, `AddHostedServiceSupport` and `DisableHttpCertificateValidation`. Third-party HTTP dependencies (e.g. SendGrid) are stubbed by a WireMock-based `MockHost` project rather than called for real. + +The `coreex-aspire` template generates the AppHost, `MockHost` and `Test.Aspire` projects. See [`samples/docs/aspire.md`](../samples/docs/aspire.md) for the Contoso implementation. + +--- + ## Putting It All Together: A Typical Request Flow To illustrate how these patterns work together, here's a typical request flow based on the **Contoso sample architecture in this repository**, not a mandatory flow for every CoreEx application. diff --git a/docs/codebase/ARCHITECTURE.md b/docs/codebase/ARCHITECTURE.md index e6e88bb57..ba33d3baa 100644 --- a/docs/codebase/ARCHITECTURE.md +++ b/docs/codebase/ARCHITECTURE.md @@ -20,8 +20,8 @@ Describe the flow in 4-6 steps using file-backed evidence. 2. The application service validates input, loads current state when needed, and coordinates a unit-of-work, for example ProductService and BasketService. 3. For Shopping mutations, domain behavior is applied on Basket and BasketItem before persistence. 4. Infrastructure repositories translate between domain/contracts and EF-backed persistence models, for example BasketRepository and ProductRepository. -5. Cross-service behavior happens through a typed HTTP client and adapter for real-time reservation, plus outbox messages or direct Service Bus publishing for async commands. -6. Relay and subscriber hosts move outbox records to Azure Service Bus and consume messages back into application services. +5. Cross-service behavior happens through a typed HTTP client and adapter for real-time reservation, plus outbox messages or direct Service Bus publishing for async commands. `NamedDestinationProvider` routes events to the shared topic and commands to a per-domain queue (`{topic}-{domain}`, e.g. `contoso-products`). +6. Relay and subscriber hosts move outbox records to Azure Service Bus and consume messages back into application services; Subscribe hosts use keyed receivers so the events topic subscription and the domain's own command queue can be consumed side by side. ### 3) Layer/Module Responsibilities diff --git a/docs/codebase/INTEGRATIONS.md b/docs/codebase/INTEGRATIONS.md index afd287150..f3668bc36 100644 --- a/docs/codebase/INTEGRATIONS.md +++ b/docs/codebase/INTEGRATIONS.md @@ -8,8 +8,9 @@ |--------|---------------------------|---------|------------|-------------|----------| | SQL Server | DB | Primary persistence for sample domains, outbox tables, and migration utilities | Connection string-based | High | docker-compose.yml; samples/src/Contoso.Products.Database/Program.cs; samples/src/Contoso.Products.Api/Program.cs | | Redis | Cache/backplane | L2 distributed cache and FusionCache backplane | Connection configured through Aspire/registered ConfigurationOptions | Medium | docker-compose.yml; samples/src/Contoso.Products.Api/Program.cs; samples/src/Contoso.Shopping.Api/Program.cs | -| Azure Service Bus | Queue/topic broker | Async event publishing, relay, and subscriber processing | Connection configured through Aspire/host config; emulator config committed for local use | High | servicebus/Config.json; samples/src/Contoso.Products.Relay/Program.cs; samples/src/Contoso.Products.Subscribe/Program.cs | +| Azure Service Bus | Queue/topic broker | Async event publishing (shared `contoso` topic), per-domain command queues (`contoso-products`), relay, and subscriber processing | Connection configured through Aspire/host config; emulator config committed for local use | High | servicebus/Config.json; samples/src/Contoso.Products.Relay/Program.cs; samples/src/Contoso.Products.Subscribe/Program.cs | | Products API from Shopping | Internal HTTP API | Real-time inventory reservation during checkout | [TODO] no explicit auth configuration was found in the inspected Shopping client code | High | samples/src/Contoso.Shopping.Infrastructure/Clients/ProductsHttpClient.cs; samples/src/Contoso.Shopping.Infrastructure/Adapters/ProductAdapter.cs | +| SendGrid | Third-party HTTP API | Checkout-confirmation email sent by the Shopping Subscribe host (`BasketCheckedOutSubscriber` via `INotificationAdapter`); stubbed by the Aspire `MockHost` in E2E tests | API key (`SendGridOptions.ApiKey`) applied as the Authorization header on the typed HttpClient | Low | samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridHttpClient.cs; samples/src/Contoso.Shopping.Infrastructure/Adapters/Notifications/NotificationAdapter.cs; samples/src/Contoso.Shopping.Subscribe/Subscribers/BasketCheckedOutSubscriber.cs; samples/aspire/Contoso.Aspire.MockHost/Program.cs | | OTLP / Aspire dashboard | Observability endpoint | Export traces from sample hosts and inspect them locally | No auth found in local compose config; dashboard is configured for anonymous local access | Medium | docker-compose.yml; samples/src/Contoso.Products.Api/Program.cs; samples/src/Contoso.Products.Relay/Program.cs | | Durable Task Scheduler | Workflow runtime | Order workflow worker orchestration sample | Connection string assembled from endpoint/task hub and auth mode | Medium | samples/src/Contoso.Order.Workflow.Worker/Program.cs | @@ -28,7 +29,7 @@ ### 4) Reliability and Failure Behavior -- Retry/backoff behavior: transactional outbox relays are implemented for Products and Shopping; [TODO] no explicit HTTP retry/backoff policy configuration was found in the inspected host files. +- Retry/backoff behavior: transactional outbox relays are implemented for Products and Shopping; database operations can opt in to retry on transient errors (`DatabaseArgsBase.RetryOnTransient`, default off; 3 attempts with exponential backoff, SQL Server classifies deadlocks/lock timeouts/Azure SQL throttling as transient); [TODO] no explicit HTTP retry/backoff policy configuration was found in the inspected host files. - Timeout policy: [TODO] no explicit timeout configuration was found in the inspected host or client files. - Circuit-breaker or fallback behavior: Shopping checkout falls back to direct broker publication for reservation cancellation if the transactional path fails; Service Bus subscriber sessions set MaxConcurrentSessions and emulator MaxDeliveryCount is configured. @@ -43,6 +44,7 @@ - docker-compose.yml - servicebus/Config.json - samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj +- samples/aspire/Contoso.Aspire.MockHost/Program.cs - samples/src/Contoso.Products.Api/Program.cs - samples/src/Contoso.Shopping.Api/Program.cs - samples/src/Contoso.Products.Relay/Program.cs diff --git a/docs/codebase/STACK.md b/docs/codebase/STACK.md index f681457bb..da30f3091 100644 --- a/docs/codebase/STACK.md +++ b/docs/codebase/STACK.md @@ -30,6 +30,7 @@ | CloudNative.CloudEvents.SystemTextJson | 2.8.0 | CloudEvent interoperability | Directory.Packages.props | | Microsoft.DurableTask.* | 1.17.1 | Order workflow sample orchestration and worker runtime | Directory.Packages.props; samples/src/Contoso.Order.Workflow.Worker/Program.cs; samples/src/Contoso.Order.Workflow.Workflow/Contoso.Order.Workflow.Workflow.csproj | | DbEx.SqlServer | 3.0.0-preview-3 | Database migration/data console utilities in samples/tests | Directory.Packages.props; samples/src/Contoso.Products.Database/Program.cs | +| UnitTestEx.Aspire + WireMock.Net | 5.12.0 / 2.18.0 | Aspire end-to-end testing (`WithAspireTester`) and the HTTP-stubbing `MockHost` | Directory.Packages.props; samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj; samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj | ### 3) Development Toolchain @@ -48,6 +49,7 @@ dotnet build CoreEx.sln dotnet test CoreEx.sln dotnet run --project samples/aspire/Contoso.Aspire +dotnet test CoreEx.Samples.Test.E2E.slnf # Aspire end-to-end tests (net10.0) docker compose up -d db-sql-server redis-cache servicebus-emulator aspire-dashboard dts-emulator ``` diff --git a/docs/codebase/STRUCTURE.md b/docs/codebase/STRUCTURE.md index 2a70396ce..97a63c0a9 100644 --- a/docs/codebase/STRUCTURE.md +++ b/docs/codebase/STRUCTURE.md @@ -12,17 +12,17 @@ List only meaningful top-level directories and files. | tests/ | Unit and API-style tests for CoreEx libraries | CoreEx.sln; tests/CoreEx.Test.Unit/CoreEx.Test.Unit.csproj | | samples/src/ | Contoso sample applications and hosts by domain/layer, including in-progress Orders and Order.Workflow sample areas | CoreEx.sln; samples/README.md | | samples/tests/ | Sample API, relay, subscriber, unit, and E2E test projects | CoreEx.sln; samples/README.md | -| samples/aspire/ | Aspire AppHost that references runnable sample services | samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj | +| samples/aspire/ | Aspire AppHost (`Contoso.Aspire`), WireMock-based `Contoso.Aspire.MockHost` for stubbing third-party HTTP dependencies, and the `Contoso.Test.Aspire` end-to-end test project | samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj; samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj; samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj | | gen/ | Roslyn source generator/analyzer project | CoreEx.sln; gen/CoreEx.Generator/CoreEx.Generator.csproj | | docs/ | Technical notes and generated codebase knowledge docs | README.md; docs/capabilities.md | | ref/ | Separate reference solution content (NDCOrderOrchestration.sln) | list_dir output; ref/NDCOrderOrchestration.sln | -| servicebus/ | Emulator configuration for local Azure Service Bus topics/subscriptions | CoreEx.sln; servicebus/Config.json | +| servicebus/ | Emulator configuration for local Azure Service Bus topic/subscriptions and the per-domain command queue (`contoso-products`), plus `Reset-ServiceBusEmulator.ps1` | CoreEx.sln; servicebus/Config.json; servicebus/Reset-ServiceBusEmulator.ps1 | | tools/ | Repo utility area | list_dir output | ### 2) Entry Points - Main runtime entry: there is no single root application entry; runnable entry points are sample host Program.cs files under samples/src/ plus the Aspire AppHost in samples/aspire/Contoso.Aspire. -- Secondary entry points (worker/cli/jobs): database console utilities in samples/src/*Database/Program.cs, outbox relays in samples/src/*.Relay/Program.cs, subscriber hosts in samples/src/*.Subscribe/Program.cs, and the order workflow worker in samples/src/Contoso.Order.Workflow.Worker/Program.cs. +- Secondary entry points (worker/cli/jobs): database console utilities in samples/src/*Database/Program.cs, outbox relays in samples/src/*.Relay/Program.cs, subscriber hosts in samples/src/*.Subscribe/Program.cs, the order workflow worker in samples/src/Contoso.Order.Workflow.Worker/Program.cs, and the Aspire `MockHost` in samples/aspire/Contoso.Aspire.MockHost/Program.cs. - How entry is selected (script/config): projects are selected explicitly via dotnet run --project ..., as shown in README.md and samples/README.md. ### 3) Module Boundaries @@ -48,6 +48,8 @@ List only meaningful top-level directories and files. - README.md - samples/README.md - samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj +- samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj +- samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj - samples/src/Contoso.Products.Api/Program.cs - samples/src/Contoso.Products.Database/Program.cs - samples/src/Contoso.Products.Relay/Program.cs diff --git a/docs/codebase/TESTING.md b/docs/codebase/TESTING.md index 3b178ff40..26d4cdf7b 100644 --- a/docs/codebase/TESTING.md +++ b/docs/codebase/TESTING.md @@ -5,13 +5,14 @@ ### 1) Test Stack and Commands - Primary test framework: NUnit 4.3.2. -- Assertion/mocking tools: AwesomeAssertions, UnitTestEx.NUnit, coverlet.collector, and sample-specific mock helpers such as MockHttpClientFactory described in samples/README.md. +- Assertion/mocking tools: AwesomeAssertions, UnitTestEx.NUnit, UnitTestEx.Aspire (Aspire E2E), WireMock.Net (Aspire MockHost), coverlet.collector, and sample-specific mock helpers such as MockHttpClientFactory described in samples/README.md. - Commands: ```bash dotnet test CoreEx.sln dotnet test tests/CoreEx.Test.Unit/CoreEx.Test.Unit.csproj dotnet test samples/tests/Contoso.Products.Test.Api/Contoso.Products.Test.Api.csproj +dotnet test CoreEx.Samples.Test.E2E.slnf # Aspire end-to-end (net10.0; requires the docker-compose infrastructure images) [TODO] no dedicated committed coverage command beyond standard dotnet test with coverlet.collector references was found. ``` @@ -27,7 +28,7 @@ dotnet test samples/tests/Contoso.Products.Test.Api/Contoso.Products.Test.Api.cs |-------|----------|----------------|-------| | Unit | yes | CoreEx library primitives and sample validators/domain behavior | tests/CoreEx.Test.Unit and Contoso.Products.Test.Unit are present | | Integration | yes | Sample APIs, subscriber hosts, and relay hosts | Contoso.Products.Test.Api, Contoso.Shopping.Test.Api, Contoso.Products.Test.Subscribe, and Contoso.Products.Test.Relay are present | -| E2E | yes | Cross-service sample scenarios against running APIs | Contoso.E2E.Runner is an interactive console runner | +| E2E | yes | Cross-service sample scenarios against running APIs | Contoso.E2E.Runner is an interactive console runner; Contoso.Test.Aspire is an automated NUnit test that self-hosts the Contoso.Aspire AppHost (`WithAspireTester`), resets databases/Redis/Service Bus from code, stubs SendGrid via the MockHost, and runs in CI on net10.0 only | ### 4) Mocking and Isolation Strategy @@ -50,4 +51,5 @@ dotnet test samples/tests/Contoso.Products.Test.Api/Contoso.Products.Test.Api.cs - samples/tests/Contoso.Products.Test.Api/ProductMutateTests.Create.cs - samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj - samples/tests/Contoso.E2E.Runner/appsettings.json +- samples/aspire/Contoso.Test.Aspire/E2ETest.cs - samples/README.md diff --git a/docs/getting-started.md b/docs/getting-started.md index ee4f038e3..b8ab50a80 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -80,11 +80,7 @@ git init ## 2. Install the template pack -```bash -dotnet new install CoreEx.Template -``` - -This installs the latest stable release. To target a specific version (e.g. to match an existing project's CoreEx NuGet reference), use `::`: +Resolve the version first: use the exact CoreEx NuGet version referenced by an existing project, or explicitly look up and record the latest stable release for first-time adoption. Always pin the install so the runtime packages and AI assets describe the same release: ```bash dotnet new install CoreEx.Template:: diff --git a/docs/presentations/coreex-agentic-scaffolding-slides.md b/docs/presentations/coreex-agentic-scaffolding-slides.md index 186f09289..36aedce70 100644 --- a/docs/presentations/coreex-agentic-scaffolding-slides.md +++ b/docs/presentations/coreex-agentic-scaffolding-slides.md @@ -348,6 +348,7 @@ This is where friction drops versus generic frameworks: CoreEx packages wire the - CloudEvent conversion and interoperability support. - Publish and subscribe patterns with per-message subscription behavior from stream. - Azure Service Bus integration patterns. +- Event vs command routing: events to a shared topic, commands to a per-domain queue. - Outbox relay support with partition-aware patterns. - Domain-driven modeling support for aggregates and entities. - ValueObject modeling using C# record class patterns. @@ -376,6 +377,7 @@ Roadmap: Aspire enabled (done): - Leverages component runtime libraries. - Sample uses console for logging, tracing, and metrics visualization. +- Automated cross-domain end-to-end tests self-host the AppHost, with a WireMock-based MockHost stubbing third-party HTTP dependencies. Speaker notes: This makes current maturity and future direction explicit for stakeholders. It also reinforces that platform portability is planned and active, not theoretical. diff --git a/samples/Directory.Build.props b/samples/Directory.Build.props index 252bed922..1ff9bb6b2 100644 --- a/samples/Directory.Build.props +++ b/samples/Directory.Build.props @@ -10,5 +10,7 @@ $(AZD_DOTNET_TARGET_FRAMEWORK) $(DOTNET_TARGET_FRAMEWORK) $(AzdPublishTargetFramework) + + \ No newline at end of file diff --git a/samples/README.md b/samples/README.md index b6ffa511e..d248aeebf 100644 --- a/samples/README.md +++ b/samples/README.md @@ -1,10 +1,10 @@ # Contoso Samples -The `samples` folder contains reference implementations of domain microservices built with CoreEx: **Products** (PostgreSQL) and **Shopping** (SQL Server) are complete; **Customers** (Azure Cosmos DB) demonstrates typed CRUD, reference data, and a transactional outbox against a schemaless store, but has no Outbox Relay/Subscribe host yet. A fourth, **Orders**, is a work in progress that will eventually demonstrate asynchronous workflow processing. +The `samples` folder contains reference implementations of domain microservices built with CoreEx: **Products** (PostgreSQL) and **Shopping** (SQL Server) are complete; **Customers** (Azure Cosmos DB) demonstrates typed CRUD, reference data, a transactional outbox, and a change-feed Outbox Relay against a schemaless store. Customers has no Subscribe host. A fourth, **Orders**, is a work in progress that will eventually demonstrate asynchronous workflow processing. ![Sample architecture interactions](../images/SampleArchitectureInteractions.png "Architecture") -Each of Products and Shopping is an independently deployable unit with an API host, an Outbox Relay host, and an Event Subscriber host, backed by an applicable data repository, and connected to other domains via synchronous HTTP and asynchronous messaging over Azure Service Bus. Customers is API-only for now — no Relay or Subscribe host — and is not wired into the inter-domain messaging shown below. +Each of Products and Shopping is an independently deployable unit with an API host, an Outbox Relay host, and an Event Subscriber host, backed by an applicable data repository, and connected to other domains via synchronous HTTP and asynchronous messaging over Azure Service Bus. Customers has API and Relay hosts: its relay forwards events from both Cosmos containers to the shared topic. Shopping still consumes Customers only via synchronous HTTP (see flow ⑧), not event-driven replication. > **Documentation** — detailed guides for layers, patterns, tooling, and testing are in [`samples/docs`](docs/). > @@ -40,18 +40,29 @@ graph TB PDB[("SQL Server\n[Products] schema\n─────────────\nProduct\nInventory\nMovement\nOutbox / OutboxLease\nRef data")] end + subgraph CUSTOMERS["Contoso.Customers Domain"] + direction TB + CAPI["Customers API\n─────────────────\nGET/POST/PUT/PATCH/DELETE /customers\nGET /refdata"] + CDB[("Cosmos DB\ncustomers / ref-data\nCo-located outbox events\n$outbox-leases")] + COUTBOX["Customers.Relay\nChange feed to Service Bus"] + end + subgraph SHOPPING["Contoso.Shopping Domain"] direction TB SAPI["Shopping API\n─────────────────\nPOST /customers/{id}/baskets\nPOST /{id}/checkout\nPUT /{id}/apply-discount\nPOST/PUT/DELETE /{id}/items\nGET /baskets"] SAPP["Shopping Application\n─────────────────\nBasketService\nBasketReadService"] SDOMAIN["Shopping Domain\n─────────────────\nBasket (Aggregate Root)\nBasketItem (Entity)\nItemPricing (Value Object)"] - SINFRA["Shopping Infrastructure\n─────────────────\nBasketRepository\nShoppingOutboxPublisher\nProductAdapter (ACL)\nProductsHttpClient\nProductSyncAdapter"] + SINFRA["Shopping Infrastructure\n─────────────────\nBasketRepository\nShoppingOutboxPublisher\nProductAdapter (ACL)\nProductsHttpClient\nProductSyncAdapter\nCustomerAdapter (ACL)\nCustomersHttpClient"] SSUBSCRIBE["Shopping.Subscribe\n─────────────────\nProductModifySubscriber\nProductDeleteSubscriber"] SOUTBOX["Shopping.Relay\n─────────────────\nOutbox to Service Bus\nPartitioned relay"] SDB[("SQL Server\n[Shopping] schema\n─────────────\nBasket\nBasketItem\nProduct (replica)\nOutbox / OutboxLease\nRef data")] end PAPI --> PAPP + CAPI --> CDB + CAPI -->|"L1/L2 Hybrid Cache + backplane"| REDIS + CDB -->|"Change feed"| COUTBOX + COUTBOX -->|"Publish customer and reference-data events"| ASB PAPP --> PINFRA PINFRA --> PDB POUTBOX -->|"Poll Outbox table"| PDB @@ -64,6 +75,7 @@ graph TB SINFRA -->|"L1/L2 Hybrid Cache"| REDIS SINFRA -->|"① HTTP POST /api/inventory/reserve\nReserve inventory at checkout"| PAPI + SINFRA -->|"⑧ HTTP GET /api/customers/{id}\nValidate customer + default address"| CAPI POUTBOX -->|"② Publish product.created/updated/deleted"| ASB ASB -->|"③ Consume product events (replication)"| SSUBSCRIBE @@ -88,6 +100,7 @@ graph TB | ① Inventory reservation | Shopping → Products | Synchronous HTTP — `ProductAdapter` calls `POST /api/inventory/reserve` at basket checkout. | | ② – ④ Product replication | Products → Shopping | Products Outbox → Relay → Service Bus → `Shopping.Subscribe` keeps a local product replica in sync. | | ⑤ – ⑦ Reservation commands | Shopping → Products | On checkout success the outbox enqueues `reservation.confirm`; on failure Shopping publishes `reservation.cancel` directly to Service Bus (the transaction has already rolled back). | +| ⑧ Customer lookup | Shopping → Customers | Synchronous HTTP — `CustomerAdapter` calls `GET /api/customers/{id}` when a basket is created (the `customerId` is validated and the shipping address defaulted from the customer) and when the checkout confirmation email is addressed. Unlike Products there is no replication: this is the alternate, real-time adapter pattern (with successful lookups held briefly in the `IHybridCache`). | See [Patterns](docs/patterns.md) for the full catalog of architectural patterns demonstrated across the samples. @@ -97,7 +110,7 @@ See [Patterns](docs/patterns.md) for the full catalog of architectural patterns |---|---| | `src/Contoso.Products.*` | Products domain — Contracts, Application, Infrastructure, API, Relay, Subscribe, CodeGen, Database | | `src/Contoso.Shopping.*` | Shopping domain — same layer split plus Domain aggregate | -| `src/Contoso.Customers.*` | Customers domain (Cosmos DB) — Contracts, Application, Infrastructure, API, CodeGen; no Relay/Subscribe/Database project (schemaless, code-first containers) | +| `src/Contoso.Customers.*` | Customers domain (Cosmos DB) — Contracts, Application, Infrastructure, API, Relay, CodeGen, Database (Cosmos provisioning console); no Subscribe project | | `src/Contoso.Orders.*` | Orders domain (work in progress) | | `aspire/Contoso.Aspire` | Aspire AppHost — orchestrates all hosts for local development and E2E validation | | `tests/Contoso.*.Test.*` | Unit, API, Relay, and Subscribe test projects per domain | @@ -141,7 +154,7 @@ dotnet run --project samples/src/Contoso.Shopping.Database -- All dotnet run --project samples/src/Contoso.Orders.Database -- All ``` -> Customers (Cosmos DB) has no `*.Database` project — it is schemaless, and its containers are created/reset code-first at test/run time via `ReplaceOrCreateContainerAsync` (see [`Contoso.Customers.Test.Api/DatabaseSetUp.cs`](tests/Contoso.Customers.Test.Api/DatabaseSetUp.cs)); no separate migration step is required. +> Customers (Cosmos DB) is schemaless, so `Contoso.Customers.Database` is not a schema migration tool: it declares the containers and seed data and runs them through the reusable `CoreEx.Cosmos.Provisioning` console (`dotnet run --project samples/src/Contoso.Customers.Database -- All`; use `DropAndAll` or `Create,ResetAndData` to start clean, noting `Drop`/`Reset` prompt for confirmation unless `--accept-prompts` is specified; `ResetAndData` alone fails where the database does not yet exist). Tests reuse the same definition via `Test.MigrateCosmosDataAsync(...)`. > The E2E runner's **Database Migration and Base Data Refresh** option can also apply pending migrations across all domains without restarting hosts. See [Aspire & E2E](docs/aspire.md) for details. @@ -165,9 +178,12 @@ dotnet test samples/tests/Contoso.Products.Test.Subscribe dotnet test samples/tests/Contoso.Shopping.Test.Api dotnet test samples/tests/Contoso.Customers.Test.Unit dotnet test samples/tests/Contoso.Customers.Test.Api +dotnet test samples/tests/Contoso.Customers.Test.Relay ``` -The required infrastructure (data store, Redis, Service Bus emulator) must be running for API, Relay, and Subscribe tests. Customers' API tests require the `cosmos-emulator` container instead of a SQL Server/Postgres store. +The required infrastructure (data store, Redis, Service Bus emulator) must be running for API, Relay, and Subscribe tests. Customers' API tests require `cosmos-emulator` and Redis instead of a SQL Server/Postgres store; its Relay tests require Cosmos and Service Bus. + +The Customers relay deliberately skips pending events older than its startup boundary when no checkpoint exists. Subsequent restarts resume saved checkpoints; delivery is at-least-once and retries can duplicate messages. Provision `$outbox-leases` with `Contoso.Customers.Database` before starting the relay. See [Hosts](docs/hosts-layer.md#customers-cosmos-relay) for configuration and recovery implications. See [Testing](docs/testing.md) for an explanation of test taxonomy, intra-domain vs inter-domain boundaries, data seeding, mock patterns, and the fluent assertion model. diff --git a/samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj b/samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj new file mode 100644 index 000000000..513fc7705 --- /dev/null +++ b/samples/aspire/Contoso.Aspire.MockHost/Contoso.Aspire.MockHost.csproj @@ -0,0 +1,22 @@ + + + + Exe + + net10.0 + enable + enable + + + + + + + + + + + + + + diff --git a/samples/aspire/Contoso.Aspire.MockHost/Program.cs b/samples/aspire/Contoso.Aspire.MockHost/Program.cs new file mode 100644 index 000000000..40bce4621 --- /dev/null +++ b/samples/aspire/Contoso.Aspire.MockHost/Program.cs @@ -0,0 +1,6 @@ +await WireMockConsole.RunAsync(settings => +{ + // Stubbed request/response logging is verbose under load; raise to LogLevel.Information when diagnosing mock matching. + settings.RequestResponseLogLevel = LogLevel.None; + return WireMockServer.Start(settings); +}); diff --git a/samples/aspire/Contoso.Aspire/AppHost.cs b/samples/aspire/Contoso.Aspire/AppHost.cs index 218fc65e5..19bd3865a 100644 --- a/samples/aspire/Contoso.Aspire/AppHost.cs +++ b/samples/aspire/Contoso.Aspire/AppHost.cs @@ -1,48 +1,68 @@ var builder = DistributedApplication.CreateBuilder(args); +// dotnet dev-certs https --trust is not fully supported on Linux, so the ASP.NET Core dev cert used by each +// project resource's https endpoint is not OS-trusted on Linux CI runners. Both the health check probes (via +// AddEndpoints) and AspireTesterBase's CreateHttpClient() resolve their HttpClient via this same DI container's +// IHttpClientFactory, so disabling certificate validation here (dev/test-only AppHost, never shipped) covers both. +builder.DisableHttpCertificateValidation(); + +// IsRunMode: never surface this test-only resource in a published manifest (per the self-hosted WireMock.Net housekeeping note - see AppHost.cs's header comment). +var mockhost = builder.AddMockHostProject("mock-host"); + +// Compose owns the infrastructure lifecycle; visible connection-string resources expose graph dependencies. +// The secret parameters retain the existing ConnectionStrings configuration and are not graph nodes. +var postgres = builder.AddExternalConnectionString("Postgres").WithIconName("DatabaseMultiple"); +var sqlServer = builder.AddExternalConnectionString("SqlServer").WithIconName("DatabaseMultiple"); +var redis = builder.AddExternalConnectionString("redis").WithIconName("Database"); +var serviceBus = builder.AddExternalConnectionString("ServiceBus").WithIconName("MailMultiple"); +var cosmos = builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint").WithIconName("DatabaseMultiple"); + +// Customers domain. +var customersApi = builder.AddProject("customers-api").WithReference(cosmos).WithReference(redis).AddEndpoints("/health/ready/detailed"); +builder.AddProject("customers-relay").WithReference(cosmos).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); + // Products domain. -builder.AddProject("products-api").AddEndpoints("/health/ready/detailed"); -builder.AddProject("products-relay").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); -builder.AddProject("products-subscribe").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); +var productsApi = builder.AddProject("products-api").WithReference(postgres).WithReference(redis).AddEndpoints("/health/ready/detailed"); +builder.AddProject("products-relay").WithReference(postgres).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); +builder.AddProject("products-subscribe").WithReference(postgres).WithReference(redis).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); // Shopping domain. -builder.AddProject("shopping-api").AddEndpoints("/health/ready/detailed"); -builder.AddProject("shopping-relay").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); -builder.AddProject("shopping-subscribe").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); +// Note: shopping-api and shopping-subscribe call Products and Customers synchronously (see ProductsHttpClient and CustomersHttpClient). The appsettings default is the standalone +// "https://localhost:7200" (Products) / "https://localhost:7320" (Customers); here the logical "https+http://products-api" / "https+http://customers-api" is injected instead, and WithReference populates the +// "Services:products-api:*" / "Services:customers-api:*" configuration used by CoreEx's service-discovery-aware AddTypedHttpClient, so the resolved address +// always matches whichever endpoint/port the target actually binds to (regardless of launch profile), rather than a static guess. +builder.AddProject("shopping-api") + .WithReference(sqlServer) + .WithReference(redis) + .WithReference(serviceBus) + .WithReference(productsApi) + .WithEnvironment("ProductsApi__BaseAddress", "https+http://products-api") + .WithReference(customersApi) + .WithEnvironment("CustomersApi__BaseAddress", "https+http://customers-api") + .AddEndpoints("/health/ready/detailed"); + +builder.AddProject("shopping-relay").WithReference(sqlServer).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); + +builder.AddProject("shopping-subscribe") + .WithReference(sqlServer) + .WithReference(redis) + .WithReference(serviceBus) + .WithReference(productsApi) + .WithEnvironment("ProductsApi__BaseAddress", "https+http://products-api") + .WithReference(customersApi) + .WithEnvironment("CustomersApi__BaseAddress", "https+http://customers-api") + .AddEndpoints("/health/ready/detailed") + .AddHostedServiceSupport() + .WithMockHostEnvironment("SendGrid__BaseAddress", mockhost, "http"); // Orders domain. var orderWorkflowWorker = builder.AddProject("order-workflow-worker").AddEndpoints("/health").WithUrlForEndpoint("https", ep => { ep.Url = "http://localhost:8082"; ep.DisplayText = "DTS Dashboard"; }); -builder.AddProject("orders-api").WaitFor(orderWorkflowWorker).AddEndpoints("/health/ready/detailed"); - -builder.Build().Run(); - - -internal static class Extensions -{ - public static IResourceBuilder AddEndpoints(this IResourceBuilder builder, params string[] urls) - { - var httpEndpoint = builder.GetEndpoint("http"); - foreach (var url in urls) - { - builder.WithAnnotation(new ResourceUrlAnnotation { Endpoint = httpEndpoint, Url = url }); - } - - return builder; - } - - // Icons: https://storybooks.fluentui.dev/react/?path=/docs/icons-catalog--docs - public static IResourceBuilder AddCommand(this IResourceBuilder builder, HttpMethod method, string path, string displayName, string? iconName) - => builder.WithHttpCommand( - path: path, - displayName: displayName, - commandOptions: new HttpCommandOptions() - { - Method = method, - IconName = iconName - }); - - public static IResourceBuilder AddHostedServiceSupport(this IResourceBuilder builder) - => builder.AddEndpoints("/hosted-services/all/status") - .AddCommand(HttpMethod.Post, "/hosted-services/all/pause", "Pause all services", "Pause") - .AddCommand(HttpMethod.Post, "/hosted-services/all/resume", "Resume all services", "PauseOff"); -} \ No newline at end of file +builder.AddProject("orders-api").WithReference(sqlServer).WithReference(redis).WaitFor(orderWorkflowWorker).AddEndpoints("/health/ready/detailed"); + +var app = builder.Build(); +await app.StartAsync(); + +await app.HttpMock("mock-host", "http") + .Request(HttpMethod.Post, "v3/mail/send").WithAnyBody().Respond.WithAsync(HttpStatusCode.Accepted); + +await app.WaitForShutdownAsync(); diff --git a/samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj b/samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj index 03100ae1f..dad3ae73e 100644 --- a/samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj +++ b/samples/aspire/Contoso.Aspire/Contoso.Aspire.csproj @@ -1,6 +1,20 @@ - + + + + net10.0 + + net10.0 Exe enable enable @@ -9,20 +23,40 @@ 6a58b94b-d2ff-413a-9e03-e9995a3de4dd + + + + - - - - - - - - + + + + + + + + + + + + + + + + + - \ No newline at end of file + diff --git a/samples/aspire/Contoso.Aspire/GlobalUsing.cs b/samples/aspire/Contoso.Aspire/GlobalUsing.cs new file mode 100644 index 000000000..717d1b02b --- /dev/null +++ b/samples/aspire/Contoso.Aspire/GlobalUsing.cs @@ -0,0 +1,2 @@ +global using Microsoft.Extensions.Hosting; +global using System.Net; diff --git a/samples/aspire/Contoso.Aspire/appsettings.Development.json b/samples/aspire/Contoso.Aspire/appsettings.Development.json index 1b2d3bafd..eae4d12c4 100644 --- a/samples/aspire/Contoso.Aspire/appsettings.Development.json +++ b/samples/aspire/Contoso.Aspire/appsettings.Development.json @@ -1,8 +1,15 @@ { "Logging": { "LogLevel": { - "Default": "Information", + "Default": "Warning", "Microsoft.AspNetCore": "Warning" } + }, + "ConnectionStrings": { + "Postgres": "Server=127.0.0.1;Database=contoso;Username=postgres;Password=yourStrong#!Password", + "SqlServer": "Data Source=127.0.0.1,1433;Initial Catalog=Contoso;User id=sa;Password=yourStrong(!)Password;TrustServerCertificate=true", + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==", + "redis": "localhost:6379", + "ServiceBus": "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;" } -} \ No newline at end of file +} diff --git a/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj b/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj new file mode 100644 index 000000000..ed3c0d2c9 --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.csproj @@ -0,0 +1,45 @@ + + + + + + net10.0 + false + $(MSBuildProjectDirectory)\Contoso.Test.Aspire.runsettings + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.runsettings b/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.runsettings new file mode 100644 index 000000000..a50f0eb8d --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/Contoso.Test.Aspire.runsettings @@ -0,0 +1,10 @@ + + + + + + net10.0 + + + diff --git a/samples/aspire/Contoso.Test.Aspire/E2ETest.CustomersRelay.cs b/samples/aspire/Contoso.Test.Aspire/E2ETest.CustomersRelay.cs new file mode 100644 index 000000000..0196dab05 --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/E2ETest.CustomersRelay.cs @@ -0,0 +1,73 @@ +namespace Contoso.Test.Aspire; + +public partial class E2ETest +{ + [Test] + public async Task Customers_Api_Outbox_Relay_PublishesEvents() + { + Test.Checkpoint("Create, update and delete a Customer; observe its real Cosmos outbox events on Service Bus."); + var customer = Test.Http(_customersApi) + .Run(HttpMethod.Post, "/api/customers", new Contoso.Customers.Contracts.Customer + { + FirstName = "Relay", + LastName = "Customer", + Email = "relay.customer@example.com", + CustomerTypeCode = "IND", + ContactMethodCode = "EM" + }, r => r.WithIdempotencyKey()) + .AssertCreated().Value ?? throw new InvalidOperationException("The Customers API returned no customer."); + + var customerId = customer.Id ?? throw new InvalidOperationException("The created customer has no identifier."); + customer.FirstName = "Updated"; + customer = Test.Http(_customersApi) + .Run(HttpMethod.Put, $"/api/customers/{customerId}", customer, r => r.WithIfMatch(customer.ETag)) + .AssertOK().Value ?? throw new InvalidOperationException("The Customers API returned no updated customer."); + Test.Http(_customersApi).Run(HttpMethod.Delete, $"/api/customers/{customerId}").AssertNoContent(); + + Test.Checkpoint("Create, update, activate, deactivate and delete reference data; its outbox uses the separate ref-data container."); + const string refDataUrl = "/api/refdata/customer-types"; + var refData = Test.Http(_customersApi) + .Run(HttpMethod.Post, refDataUrl, new Contoso.Customers.Contracts.CustomerType { Code = $"RELAY-{Runtime.NewId()[..8]}", Text = "Relay E2E" }) + .AssertCreated().Value ?? throw new InvalidOperationException("The Customers API returned no customer type."); + var refDataId = refData.Id ?? throw new InvalidOperationException("The customer type has no identifier."); + refData = Test.Http(_customersApi) + .Run(HttpMethod.Patch, $"{refDataUrl}/{refDataId}", new { text = "Relay E2E updated" }, r => r.WithIfMatch(refData.ETag).WithMergePatchJsonContentType()) + .AssertOK().Value ?? throw new InvalidOperationException("The Customers API returned no updated customer type."); + Test.Http(_customersApi).Run(HttpMethod.Post, $"{refDataUrl}/{refDataId}/activate").AssertOK(); + Test.Http(_customersApi).Run(HttpMethod.Post, $"{refDataUrl}/{refDataId}/deactivate").AssertOK(); + Test.Http(_customersApi).Run(HttpMethod.Delete, $"{refDataUrl}/{refDataId}").AssertNoContent(); + + (string Subject, string Type)[] expected = + [ + (customerId, "contoso.customers.customer.created.v1"), + (customerId, "contoso.customers.customer.updated.v1"), + (customerId, "contoso.customers.customer.deleted"), + (refDataId, "contoso.customers.customertype.created.v1"), + (refDataId, "contoso.customers.customertype.updated.v1"), + (refDataId, "contoso.customers.customertype.activated.v1"), + (refDataId, "contoso.customers.customertype.deactivated.v1"), + (refDataId, "contoso.customers.customertype.deleted") + ]; + + await using var client = new ServiceBusClient(_serviceBusConnectionString, new ServiceBusClientOptions + { + RetryOptions = new ServiceBusRetryOptions { MaxRetries = 0, TryTimeout = TimeSpan.FromSeconds(1) } + }); + var messages = await Contoso.Customers.Test.Common.ServiceBus.ReceiveEventsAsync(client, expected).ConfigureAwait(false); + foreach (var key in expected) + { + var message = messages[key]; + message.MessageId.Should().NotBeNullOrEmpty(); + message.SessionId.Should().Be(CoreEx.Data.PartitionKey.GetPartitionIdAsString(key.Subject)); + using var json = JsonDocument.Parse(message.Body); + json.RootElement.GetProperty("id").GetString().Should().Be(message.MessageId); + if (key.Type.EndsWith(".v1", StringComparison.Ordinal)) + json.RootElement.GetProperty("data").GetProperty("id").GetString().Should().Be(key.Subject); + } + + using var created = JsonDocument.Parse(messages[(customerId, "contoso.customers.customer.created.v1")].Body); + created.RootElement.GetProperty("data").GetProperty("firstName").GetString().Should().Be("Relay"); + using var updated = JsonDocument.Parse(messages[(customerId, "contoso.customers.customer.updated.v1")].Body); + updated.RootElement.GetProperty("data").GetProperty("firstName").GetString().Should().Be("Updated"); + } +} diff --git a/samples/aspire/Contoso.Test.Aspire/E2ETest.cs b/samples/aspire/Contoso.Test.Aspire/E2ETest.cs new file mode 100644 index 000000000..6bf3d6335 --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/E2ETest.cs @@ -0,0 +1,200 @@ +namespace Contoso.Test.Aspire; + +public partial class E2ETest : WithAspireTester +{ + private readonly string _productsApi = "products-api"; + private readonly string _shoppingApi = "shopping-api"; + private readonly string _customersApi = "customers-api"; + private string _serviceBusConnectionString = string.Empty; + + protected override async Task OnBeforeStartAsync(DistributedApplication app) + { + _serviceBusConnectionString = await AspireTesterBase.GetConnectionStringAsync(app, "ServiceBus").ConfigureAwait(false) + ?? throw new InvalidOperationException("The ServiceBus resource has no connection string."); + + // Migrate the Products, Shopping and Customers databases and seed each with test data. + await app.MigratePostgresDataAsync("Postgres", ["mutate-data.seed.yaml"], Contoso.Products.Database.Program.ConfigureMigrationArgs); + await app.MigrateSqlServerDataAsync("SqlServer", ["mutate-data.seed.yaml"], Contoso.Shopping.Database.Program.ConfigureMigrationArgs); + await app.MigrateCosmosDataAsync("Cosmos", "contoso", ["mutate-data.seed.yaml"], Contoso.Customers.Database.Program.ConfigureProvisionArgs); + + // Clear the Redis cache. + await app.ClearRedisCacheAsync("redis"); + + // Reset the Azure Service Bus queues and topics/subscriptions to an initial state. + await app.ResetAzureServiceBusAsync("servicebus", ServiceBus.GetQueues(), ServiceBus.GetTopicsAndSubscriptions()); + } + + protected override async Task OnAfterStartAsync(DistributedApplication app) + { + // Wait for every host used by the asynchronous checkout flow before running the tests. + await app.WaitForResourceAsync([_productsApi, _shoppingApi, _customersApi, "products-relay", "products-subscribe", "shopping-relay", "shopping-subscribe", "customers-relay"]); + + // Mock the SendGrid API so that the Shopping domain's Subscribe project can send emails without actually sending them. + await app.HttpMock("mock-host", "http").Request(HttpMethod.Post, "/v3/mail/send").WithAnyBody().Respond.WithAsync(HttpStatusCode.Accepted); + } + + [Test] + public async Task CreateOrderAndConfirmAsync() + { + Test.Checkpoint("Create and activate a new Product; this should sync to the Shopping domain via the Products domain's outbox and Service Bus."); + + var product = Test.Http(_productsApi) + .Run(HttpMethod.Post, "/api/products", new Product { Sku = "YETI-ASR-LR-XO-90", Text = "Yeti ASR LT XO/90", Price = 7900m, SubCategoryCode = "XC", UnitOfMeasureCode = "EA", Tags = ["cross-country", "full-suspension"]}, r => r.WithIdempotencyKey()) + .AssertCreated() + .Value!; + + product = Test.Http(_productsApi) + .Run(HttpMethod.Post, $"/api/products/{product.Id}/activate") + .AssertOK() + .Value!; + + Test.Checkpoint("Adjust the quantity-on-hand for the new Product so we can sell it."); + + Test.Http(_productsApi) + .Run(HttpMethod.Post, "/api/inventory/adjust", new MovementRequest + { + Id = Runtime.NewGuid().ToString(), + Products = new() + { + { product.Id!, new MovementRequestProduct { Quantity = 3, UnitOfMeasureCode = "EA" } }, + } + }) + .AssertOK(); + + Test.Checkpoint("Create a new Customer (with a shipping address) in the Customers domain."); + + var customer = Test.Http(_customersApi) + .Run(HttpMethod.Post, "/api/customers", new Contoso.Customers.Contracts.Customer + { + FirstName = "Gina", + LastName = "Garcia", + Email = "gina.garcia@example.com", + CustomerTypeCode = "IND", + ContactMethodCode = "EM", + ShippingAddress = new Contoso.Customers.Contracts.Address { Street1 = "42 Wallaby Way", City = "Sydney", State = "NSW", PostCode = "2000" } + }, r => r.WithIdempotencyKey()) + .AssertCreated() + .Value!; + + Test.Checkpoint("Create a new Basket for the Customer; the Customer is validated in real-time (via the Customers API) and the shipping address is defaulted from it."); + + var basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/customers/{customer.Id}/baskets", r => r.WithIdempotencyKey()) + .AssertCreated() + .Value!; + + basket.CustomerId.Should().Be(customer.Id); + basket.ShippingAddress.Should().NotBeNull(); + basket.ShippingAddress.Street1.Should().Be("42 Wallaby Way"); + basket.ShippingAddress.City.Should().Be("Sydney"); + basket.ShippingAddress.State.Should().Be("NSW"); + basket.ShippingAddress.PostCode.Should().Be("2000"); + + Test.Checkpoint("Creating a Basket for a Customer that does not exist should fail validation."); + + Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/customers/{Runtime.NewGuid()}/baskets", r => r.WithIdempotencyKey()) + .AssertBadRequest(); + + Test.Checkpoint("Add two existing Products to the Basket."); + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/baskets/{basket.Id}/items", new BasketItemAddRequest { ProductId = 28.ToGuid().ToString(), Quantity = 1m }, r => r.WithIdempotencyKey()) + .AssertOK() + .Value!; + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/baskets/{basket.Id}/items", new BasketItemAddRequest { ProductId = 32.ToGuid().ToString(), Quantity = 1.5m }, r => r.WithIdempotencyKey()) + .AssertOK() + .Value!; + + Test.Checkpoint("Apply a discount to the Basket"); + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Put, $"/api/baskets/{basket.Id}/apply-discount/SAVE10") + .AssertOK() + .Value!; + + basket.Pricing.Should().NotBeNull(); + basket.Pricing.DiscountPercentage.Should().Be(10m); + + Test.Checkpoint("Update the Basket's shipping address (overriding the address defaulted from the Customer)."); + + var address = new Address + { + Street1 = "123 Main St", + City = "Anytown", + State = "CA", + PostCode = "12345" + }; + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Put, $"/api/baskets/{basket.Id}/shipping-address", address) + .AssertOK() + .Value!; + + basket.ShippingAddress.Should().NotBeNull(); + basket.ShippingAddress.Street1.Should().Be("123 Main St"); + + Test.Checkpoint("Add the new Product to the Basket; should have sync'd by now."); + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/baskets/{basket.Id}/items", new BasketItemAddRequest { ProductId = product.Id, Quantity = 1m }, r => r.WithIdempotencyKey()) + .AssertOK() + .Value!; + + Test.Checkpoint("Checkout the Basket."); + + basket = Test.Http(_shoppingApi) + .Run(HttpMethod.Post, $"/api/baskets/{basket.Id}/checkout") + .AssertOK() + .Value!; + + basket.StatusCode.Should().Be(BasketStatus.CheckedOut); + + Test.Checkpoint("Confirm that the product was successfully reserved/confirmed."); + + var movements = Test.Http(_productsApi) + .Run(HttpMethod.Get, "/api/inventory/movements", r => r.WithQuery($"referenceid eq '{basket.Id}'")) + .AssertOK() + .Value!; + + movements.Should().HaveCount(2); // One of the items is not stocked, so only two movements should be created for the two items that were successfully reserved. + + const int maxConfirmationAttempts = 60; + var attempt = 0; + while (movements.Single(m => m.ProductId == product.Id).StatusCode != MovementStatus.Confirmed) + { + if (attempt == maxConfirmationAttempts) + Assert.Fail($"Inventory movement confirmation timed out after {maxConfirmationAttempts} polls: BasketId='{basket.Id}', ProductId='{product.Id}', StatusCode='{movements.Single(m => m.ProductId == product.Id).StatusCode}'. Expected path: Shopping outbox -> Shopping Relay -> contoso-products command queue -> Products Subscribe."); + + Test.Delay(1000, $"Waiting for the Product's inventory movement to be confirmed (via Shopping's outbox, Relay, and the Products command subscriber); iteration {attempt + 1} of {maxConfirmationAttempts}."); + movements = Test.Http(_productsApi) + .Run(HttpMethod.Get, "/api/inventory/movements", r => r.WithQuery($"referenceid eq '{basket.Id}'")) + .AssertOK() + .Value!; + + attempt++; + } + + Test.Checkpoint("The Product's inventory movement was successfully confirmed - you little beauty!"); + + Test.Checkpoint("Poll the stand-in SendGrid endpoint for a mail-send request addressed to the Customer's e-mail."); + + var email = customer.Email!; + var api = await Test.HttpMock("mock-host", "http").GetAdminApiAsync().ConfigureAwait(false); + for (attempt = 1; !await SendReceivedAsync(api, email).ConfigureAwait(false); attempt++) + { + if (attempt > 30) + Assert.Fail($"No SendGrid mail-send request addressed to '{email}' was received after 30 attempts; the Basket.CheckedOut event did not result in a send."); + + Test.Delay(1000, $"Waiting for the mail-send request; iteration {attempt}."); + } + } + + private static async Task SendReceivedAsync(WireMock.Client.IWireMockAdminApi api, string email) + { + var requests = await api.GetRequestsAsync().ConfigureAwait(false); + return requests.Any(r => r.Request?.Path == "/v3/mail/send" && r.Request.Body?.Contains(email, StringComparison.OrdinalIgnoreCase) == true); + } +} diff --git a/samples/aspire/Contoso.Test.Aspire/GlobalUsing.cs b/samples/aspire/Contoso.Test.Aspire/GlobalUsing.cs new file mode 100644 index 000000000..36af7f410 --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/GlobalUsing.cs @@ -0,0 +1,15 @@ +global using Aspire.Hosting; +global using Aspire.Hosting.ApplicationModel; +global using AwesomeAssertions; +global using Azure.Messaging.ServiceBus; +global using Azure.Messaging.ServiceBus.Administration; +global using Contoso.Products.Contracts; +global using Contoso.Shopping.Contracts; +global using CoreEx; +global using Microsoft.Extensions.Configuration; +global using NUnit.Framework; +global using System.Net; +global using System.Text.Json; +global using UnitTestEx; +global using UnitTestEx.Aspire; +global using UnitTestEx.Expectations; diff --git a/samples/aspire/Contoso.Test.Aspire/InfrastructureResourceTests.cs b/samples/aspire/Contoso.Test.Aspire/InfrastructureResourceTests.cs new file mode 100644 index 000000000..23829d65c --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/InfrastructureResourceTests.cs @@ -0,0 +1,54 @@ +namespace Contoso.Test.Aspire; + +public class InfrastructureResourceTests +{ + [TestCase("Postgres")] + [TestCase("SqlServer")] + [TestCase("redis")] + [TestCase("ServiceBus")] + [TestCase("Cosmos")] + public async Task ExternalConnectionString_IsVisibleAndPreservesConfiguration(string name) + { + var builder = DistributedApplication.CreateBuilder(new DistributedApplicationOptions { DisableDashboard = true }); + const string value = "configured-connection-string"; + builder.Configuration.AddInMemoryCollection(new Dictionary { [$"ConnectionStrings:{name}"] = value }); + + var resource = builder.AddExternalConnectionString(name); + + resource.Resource.Should().BeOfType(); + resource.Resource.Name.Should().Be(name); + (await resource.Resource.ConnectionStringExpression.GetValueAsync(default).ConfigureAwait(false)).Should().Be(value); + var parameter = builder.Resources.OfType().Single(); + parameter.Secret.Should().BeTrue(); + parameter.Name.Should().Be($"{name}-connection"); + } + + [Test] + public void Cosmos_ExposesConfiguredEndpointWithoutCredentials() + { + var builder = DistributedApplication.CreateBuilder(new DistributedApplicationOptions { DisableDashboard = true }); + builder.Configuration.AddInMemoryCollection(new Dictionary + { + ["ConnectionStrings:Cosmos"] = "AccountEndpoint=https://cosmos.example.test:18081/;AccountKey=test-key;" + }); + + var resource = builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint"); + + resource.Resource.Annotations.OfType().Single().Url.Should().Be("https://cosmos.example.test:18081"); + } + + [TestCase(null)] + [TestCase("AccountKey=test-key;")] + [TestCase("AccountEndpoint=not-a-url;AccountKey=test-key;")] + [TestCase("AccountEndpoint=ftp://cosmos.example.test/;AccountKey=test-key;")] + [TestCase("AccountEndpoint=https://user:password@cosmos.example.test/;AccountKey=test-key;")] + public void Cosmos_RejectsMissingOrInvalidEndpoint(string? connectionString) + { + var builder = DistributedApplication.CreateBuilder(new DistributedApplicationOptions { DisableDashboard = true }); + builder.Configuration.AddInMemoryCollection(new Dictionary { ["ConnectionStrings:Cosmos"] = connectionString }); + + var action = () => builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint"); + + action.Should().Throw(); + } +} diff --git a/samples/aspire/Contoso.Test.Aspire/ServiceBus.cs b/samples/aspire/Contoso.Test.Aspire/ServiceBus.cs new file mode 100644 index 000000000..ecb2a0ba5 --- /dev/null +++ b/samples/aspire/Contoso.Test.Aspire/ServiceBus.cs @@ -0,0 +1,19 @@ +namespace Contoso.Test.Aspire; + +public static class ServiceBus +{ + public static CreateQueueOptions[]? GetQueues() => + [ + new CreateQueueOptions("contoso-products") { RequiresSession = true } + ]; + + public static (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? GetTopicsAndSubscriptions() => + [ + (new CreateTopicOptions("contoso"), + [ + new CreateSubscriptionOptions("contoso", "shopping") { RequiresSession = true }, + new CreateSubscriptionOptions("contoso", "products") { RequiresSession = true }, + new CreateSubscriptionOptions("contoso", Contoso.Customers.Test.Common.ServiceBus.ObservationSubscription) { RequiresSession = true } + ]) + ]; +} diff --git a/samples/docs/application-layer.md b/samples/docs/application-layer.md index 39a551b2e..e659f33e7 100644 --- a/samples/docs/application-layer.md +++ b/samples/docs/application-layer.md @@ -172,6 +172,18 @@ A second interface, `IProductSyncAdapter`, handles the **event-driven data repli [`IProductSyncAdapter`](../src/Contoso.Shopping.Application/Adapters/Products/IProductSyncAdapter.cs) +`ICustomerAdapter` demonstrates the **alternate, real-time pattern**: Shopping does not replicate Customers data, it calls the Customers API live whenever it needs it. `BasketService.CreateAsync` uses it (via `CustomerPolicy`) to validate the `customerId` and to default the basket's shipping address from the customer's address when the customer has one; the Infrastructure `NotificationAdapter` uses it to address the checkout confirmation email. The adapter returns Shopping's own `Customer` contract (only the properties Shopping cares about), not the Customers domain's. + +```csharp +// samples/src/Contoso.Shopping.Application/Adapters/Customers/ICustomerAdapter.cs +public interface ICustomerAdapter +{ + Task> GetAsync(string id, CancellationToken cancellationToken = default); +} +``` + +[`ICustomerAdapter`](../src/Contoso.Shopping.Application/Adapters/Customers/ICustomerAdapter.cs) + --- ## Mapping (Application-level) @@ -213,3 +225,5 @@ public class ProductPolicy(IProductAdapter productAdapter) : r); } ``` + +`CustomerPolicy` (`EnsureExistsAsync`) follows the same shape for the basket's `customerId`, returning a `customerId` validation error ("Customer was not found.") when the Customers API responds 404. diff --git a/samples/docs/aspire.md b/samples/docs/aspire.md index 9291ce650..14abc43ca 100644 --- a/samples/docs/aspire.md +++ b/samples/docs/aspire.md @@ -10,10 +10,13 @@ Aspire is the required foundation for any activity that involves **cross-domain ## What Aspire orchestrates -The `Contoso.Aspire` AppHost (`samples/aspire/Contoso.Aspire/AppHost.cs`) registers all six production hosts and the Orders workflow worker: +The `Contoso.Aspire` AppHost (`samples/aspire/Contoso.Aspire/AppHost.cs`) registers all six Products/Shopping hosts, the Customers API and Cosmos relay, the Orders workflow worker, and a WireMock.Net stub host: | Resource name | Host project | Endpoints exposed | |---|---|---| +| `mock-host` | `Contoso.Aspire.MockHost` | WireMock.Net stub server (see [MockHost](#mockhost-stubbing-external-dependencies) below) | +| `customers-api` | `Contoso.Customers.Api` | HTTP + `/health/ready/detailed` (called by `shopping-api` and `shopping-subscribe`) | +| `customers-relay` | `Contoso.Customers.Relay` | Cosmos + Service Bus; HTTP + `/health/ready/detailed` + hosted-service controls | | `products-api` | `Contoso.Products.Api` | HTTP + `/health/ready/detailed` | | `products-relay` | `Contoso.Products.Relay` | HTTP + `/health/ready/detailed` + hosted-service controls | | `products-subscribe` | `Contoso.Products.Subscribe` | HTTP + `/health/ready/detailed` + hosted-service controls | @@ -23,8 +26,26 @@ The `Contoso.Aspire` AppHost (`samples/aspire/Contoso.Aspire/AppHost.cs`) regist | `order-workflow-worker` | `Contoso.Order.Workflow.Worker` | HTTP + `/health` + DTS Dashboard link | | `orders-api` | `Contoso.Orders.Api` | HTTP + `/health/ready/detailed` (waits for workflow worker) | +The `AddEndpoints(...)`/`AddHostedServiceSupport()`/`DisableHttpCertificateValidation()`/`AddMockHostProject<...>(...)` calls used to wire up the resources above are extension methods on `IDistributedApplicationBuilder`/`IResourceBuilder` provided by the `CoreEx.UnitTesting` package's Aspire support (`` in `Contoso.Aspire.csproj`) — there is no local `Extensions.cs`. + +`AddExternalConnectionString(...)` is also provided by `CoreEx.UnitTesting` (`UnitTestEx` namespace), in `UnitTestExExtensions.Aspire.cs`; AppHosts should not maintain their own copies. + Hosted-service resources (outbox relays and subscribers) also get **Pause all services** and **Resume all services** commands surfaced as buttons in the Aspire Dashboard, backed by the `/hosted-services/all/pause` and `/hosted-services/all/resume` management endpoints. This allows controlled simulation of relay downtime or subscriber lag without restarting the process. +Customers API uses Cosmos DB and Redis-backed FusionCache (L1 memory, L2 Redis, and a Redis backplane), matching the relational APIs' caching configuration. Its readiness check probes both Cosmos and Redis. The API enables `Azure.Experimental.EnableActivitySource` in its runtime configuration so Cosmos SDK operations appear as dependency spans; missing spans do not imply that readiness skipped the database check. Shopping and Orders use SQL Server, while Products uses PostgreSQL; dashboard peer labels are inferred from endpoint attributes, not from the domain's database provider. + +External infrastructure is represented by visible **connection-string resources** (`Postgres`, `SqlServer`, `redis`, `ServiceBus`, and `Cosmos`), so the Graph view includes these nodes and their host dependencies. Each wraps a secret parameter backed by the existing `ConnectionStrings` configuration; parameters alone are excluded from Aspire 13.5.4's Graph and Table views. Compose still owns the actual infrastructure processes, and host readiness checks remain responsible for probing them; a running connection-string resource only means its configuration resolved. Cosmos additionally exposes the configured `AccountEndpoint` as a resource URL because the dashboard's connection-string parser does not recognise that key, preventing correct HTTP peer attribution otherwise. + +### MockHost — stubbing external dependencies + +`Contoso.Aspire.MockHost` (`samples/aspire/Contoso.Aspire.MockHost/Program.cs`) is a plain console host that starts a [WireMock.Net](https://github.com/WireMock-Net/WireMock.Net) server and keeps it running for the lifetime of the AppHost: + +```csharp +await WireMockConsole.RunAsync(settings => WireMockServer.Start(settings)); +``` + +`WireMockConsole.RunAsync` (from `UnitTestEx.Aspire`) wires up graceful shutdown and console logging around the server; `WireMockServer.Start(settings)` is WireMock.Net's own entry point. It is registered in `AppHost.cs` via `builder.AddMockHostProject("mock-host")` so it shows up as a normal resource in the dashboard. It stubs *third-party* HTTP dependencies (something outside the AppHost, unlike Products/Shopping which are real intra-repo domains): `shopping-subscribe` is pointed at it for SendGrid via `.WithMockHostEnvironment("SendGrid__BaseAddress", mockhost, "http")`, and `Contoso.Test.Aspire` registers the routes in `OnAfterStartAsync` with `app.HttpMock("mock-host", "http").Request(...)`. Domains that *are* in the AppHost are called for real via `WithReference` — never stub those. + --- ## Starting Aspire @@ -42,6 +63,7 @@ And databases must have been migrated and seeded at least once (see [tooling.md ```bash dotnet run --project samples/src/Contoso.Products.Database -- all dotnet run --project samples/src/Contoso.Shopping.Database -- all +dotnet run --project samples/src/Contoso.Customers.Database -f net10.0 -- All ``` ### Start all hosts @@ -70,6 +92,18 @@ The hosted-service command buttons (Pause / Resume) are also surfaced here, maki --- +## Contoso.Test.Aspire — automated smoke test + +`Contoso.Test.Aspire` (`samples/aspire/Contoso.Test.Aspire/E2ETest.cs`) is an NUnit project that provides an **automated, CI-friendly** counterpart to the interactive E2E Runner described below. Unlike the E2E Runner, it does not require Aspire to already be running — its single test class derives from `WithAspireTester` (from `CoreEx.UnitTesting`'s Aspire support), which starts the whole `Contoso.Aspire` AppHost itself for the duration of the test run. + +Its `OnBeforeStartAsync` override migrates and seeds the Products (Postgres), Shopping (SQL Server) and Customers (Cosmos DB) databases, clears the Redis cache, and resets the Service Bus emulator's queues/topics/subscriptions to a known state (the topology is the code-based `ServiceBus` class), then `OnAfterStartAsync` waits for the three APIs, all three relays, and the Products and Shopping subscribers. The `CreateOrderAndConfirmAsync` test drives the same cross-domain flow as the E2E Runner's **Shopping Basket Lifecycle** scenario: checkout, async inventory confirmation, and the stubbed SendGrid email. Inventory confirmation uses up to 60 one-second polling delays to accommodate relay contention and lease recovery on slower CI runners; failure reports the basket, product, last movement status, and expected command path. + +`Customers_Api_Outbox_Relay_PublishesEvents` additionally drives customer create/update/delete and reference-data create/update/activate/deactivate/delete through the real Customers API. It receives the resulting events on the isolated, session-enabled `customers-relay` subscription using the Service Bus SDK and the AppHost's resolved connection string. Bounded polling correlates messages to this test's entity identifiers and event types; it does not drain Products/Shopping subscriptions or assume exactly-once delivery. No Customers subscriber or replication flow is introduced. + +Use `Contoso.Test.Aspire` when you want a single deterministic pass/fail signal (e.g. in CI, or as a quick local smoke test after a change) — `dotnet test samples/aspire/Contoso.Test.Aspire`. Use the E2E Runner (below) when you want to explore interactively, run load simulations, or watch traces build up live in the Aspire Dashboard against a long-running AppHost. + +--- + ## The Aspire + E2E pairing Aspire provides the running environment; the **E2E Runner** provides the workload. @@ -185,7 +219,8 @@ Press `ESC` to stop gracefully. Errors are written to `logs/load-simulation-erro |---|---|---| | Unit tests (`*.Test.Unit`) | No | No | | Intra-domain host tests (`*.Test.Api`, `*.Test.Subscribe`, `*.Test.Relay`) | No | No | -| Cross-domain functional validation | Yes | Yes | -| Load / concurrency simulation | Yes | Yes | +| Automated cross-domain smoke test (`Contoso.Test.Aspire`) | Self-hosted (`WithAspireTester`) | No | +| Cross-domain functional validation (interactive) | Yes (already running) | Yes | +| Load / concurrency simulation | Yes (already running) | Yes | -See [testing.md](testing.md) for the intra-domain testing guide. The E2E Runner is the complement to that guide — it covers the inter-domain surface that intra-domain tests deliberately leave mocked. +See [testing.md](testing.md) for the intra-domain testing guide. `Contoso.Test.Aspire` and the E2E Runner are both complements to that guide — they cover the inter-domain surface that intra-domain tests deliberately leave mocked, one as an automated pass/fail test and the other as an interactive/load-simulation tool. diff --git a/samples/docs/contracts-layer.md b/samples/docs/contracts-layer.md index 3a5d6fbea..236d9b374 100644 --- a/samples/docs/contracts-layer.md +++ b/samples/docs/contracts-layer.md @@ -57,6 +57,8 @@ Mutable resources typically compose three cross-cutting interfaces: Server-managed fields are decorated with `[ReadOnly(true)]` so they are excluded from inbound requests, by the likes of NSwag (Swagger), automatically. +Properties that are mandatory but declared nullable (`string?`, `int?`) for DTO flexibility are decorated with `[NonNullable]` so the generated OpenAPI schema reports them as `nullable: false`; the C# type and validation are unaffected. `Id`, `ETag` and `ChangeLog` are never marked (they are null on create); other `[ReadOnly]` properties are marked only when the type is never a request body (e.g. `BasketItem`). + ```csharp // samples/src/Contoso.Products.Contracts/Product.cs [Contract] diff --git a/samples/docs/domain-layer.md b/samples/docs/domain-layer.md index f90c9d9a0..551c8a1c3 100644 --- a/samples/docs/domain-layer.md +++ b/samples/docs/domain-layer.md @@ -21,9 +21,10 @@ An aggregate is a cluster of related entities treated as a single consistency bo // samples/src/Contoso.Shopping.Domain/Basket.cs public sealed class Basket : Aggregate { - public static Basket CreateNew(string customerId) => new Basket(Runtime.NewId()) + public static Basket CreateNew(string customerId, Address? shippingAddress = null) => new Basket(Runtime.NewId()) { CustomerId = customerId, + ShippingAddress = shippingAddress, Status = BasketStatus.Empty }.AsNew(); diff --git a/samples/docs/hosts-layer.md b/samples/docs/hosts-layer.md index 70e74ad60..456680e09 100644 --- a/samples/docs/hosts-layer.md +++ b/samples/docs/hosts-layer.md @@ -6,11 +6,11 @@ The host is the **composition root** of the application. It sits above all layer **Example projects** -| Host type | Products | Shopping | -|---|---|---| -| API | [`Contoso.Products.Api`](../src/Contoso.Products.Api) | [`Contoso.Shopping.Api`](../src/Contoso.Shopping.Api) | -| Outbox Relay | [`Contoso.Products.Relay`](../src/Contoso.Products.Relay) | [`Contoso.Shopping.Relay`](../src/Contoso.Shopping.Relay) | -| Subscribe | [`Contoso.Products.Subscribe`](../src/Contoso.Products.Subscribe) | [`Contoso.Shopping.Subscribe`](../src/Contoso.Shopping.Subscribe) | +| Host type | Products | Shopping | Customers | +|---|---|---|---| +| API | [`Contoso.Products.Api`](../src/Contoso.Products.Api) | [`Contoso.Shopping.Api`](../src/Contoso.Shopping.Api) | [`Contoso.Customers.Api`](../src/Contoso.Customers.Api) | +| Outbox Relay | [`Contoso.Products.Relay`](../src/Contoso.Products.Relay) | [`Contoso.Shopping.Relay`](../src/Contoso.Shopping.Relay) | [`Contoso.Customers.Relay`](../src/Contoso.Customers.Relay) | +| Subscribe | [`Contoso.Products.Subscribe`](../src/Contoso.Products.Subscribe) | [`Contoso.Shopping.Subscribe`](../src/Contoso.Shopping.Subscribe) | Not implemented | --- @@ -131,7 +131,7 @@ way as any other CoreEx invoker. `Program.cs` follows a predictable CoreEx shape and is the only file in the API host: 1. `builder.AddHostSettings()` — loads CoreEx host configuration. -2. Core services — `AddExecutionContext()`, `AddReferenceDataOrchestrator()`, `AddMvcWebApi()`, `AddHttpWebApi()`. +2. Core services — `AddExecutionContext()`, `AddReferenceDataOrchestrator()` (non-generic; binds the CodeGen-generated `IReferenceDataProvider` from DI at runtime), `AddMvcWebApi()`, `AddHttpWebApi()`. 3. Dynamic service registration — `AddDynamicServicesUsing()` auto-discovers all `[ScopedService]`-decorated types. 4. Infrastructure wiring — database, EF DbContext, outbox publisher, caching (L1 in-memory + L2 Redis + FusionCache backplane). 5. `PostConfigureAllHealthChecks()` — adds standard health-check tags. @@ -143,7 +143,7 @@ way as any other CoreEx invoker. // samples/src/Contoso.Products.Api/Program.cs (abridged) builder.Services .AddExecutionContext() - .AddReferenceDataOrchestrator() + .AddReferenceDataOrchestrator() .AddMvcWebApi() .AddHttpWebApi(); @@ -195,8 +195,32 @@ app.MapHealthChecks(); app.MapHostedServices(); // Exposes pause/resume management endpoints. ``` +Relational outbox writers and relay hosts must configure the same `CoreEx:Host:Outbox:PartitionSize` value (default `4`, valid range `1`–`256`). The relay's startup log and detailed health data report the resolved size. Service Bus session bucketing is a separate boundary and can be tuned independently with `CoreEx:Host:ServiceBus:SessionIdPartitionSize`; do not wire it to the outbox value, since a host such as Shopping API can publish directly to Service Bus without passing through the outbox. + +The Service Bus size is a publisher-wide default. When destinations need different session profiles, use `CoreEx:Host:ServiceBus:Destinations:{destination}` with the exact resolved name (for example, Shopping targets the `contoso-products` command queue, so its override belongs under `Destinations:contoso-products`). Apply the profile in every host that publishes to the destination, including an outbox Relay. The bucket count groups producer keys; it does not set broker partitions or consumer concurrency. Keep `MaxConcurrentSessions` on the receiving host independently tuned. + +#### Changing partition and session mappings safely + +Relational outbox partition IDs are persisted on each row; changing `PartitionSize` does not remap existing rows. Each relay scans only its configured partition range, so shrinking that range before the backlog drains can strand older rows indefinitely. Preserve the effective size when migrating from the removed `CoreEx:Host:Services:OutboxRelay:PartitionSize` key to `CoreEx:Host:Outbox:PartitionSize`. + +For an intentional mapping change, pause affected producers and drain their outboxes using the old relay configuration before deploying the new size consistently to every writer and relay. Resume only after the configuration is aligned. The deterministic `$none` mapping for keyless events also changes their placement relative to older random routing; do not assume mixed old/new placement preserves ordering. + +Service Bus session strategy or bucket-count changes are a separate rollout: the same key can move to a different session, and mixed producer policies can break per-key ordering. Pause every affected producer, including direct broker publishers, drain upstream outboxes and the affected broker queues/subscriptions and in-flight handlers, then switch destination policies consistently before resuming. Changing receiver concurrency alone does not change the producer's key-to-session mapping. + > The `Program.cs` for the Outbox Relay is intentionally minimal — no controllers, no OpenAPI document, no application-layer services. Its sole concern is shuttling committed outbox records to the broker reliably. +### Customers Cosmos relay + +[`Contoso.Customers.Relay`](../src/Contoso.Customers.Relay) uses `AddCosmosDbOutboxRelayHostedService` once for `customers` and once for `ref-data` in database `contoso`. Outbox documents are committed beside business documents in the same container/partition; there is no separate Cosmos outbox container. Both processor groups share the `$outbox-leases` container, provisioned by `Contoso.Customers.Database` with partition key `/id`. The relay never creates that container and fails startup if it is missing. + +`CoreEx:Host:Services:CosmosOutboxRelay:{containerId}` controls `ServicesCount`, `PollInterval`, and `BatchSize` independently for each container. The sample defaults to two processors per container, named `cosmos-outbox-relay-customers-00/01` and `cosmos-outbox-relay-ref-data-00/01`. Each exposes the standard status/pause/resume endpoints. Pausing retains pending events and resuming continues from the saved change-feed position. + +**First-start policy:** a single live UTC host-start boundary is supplied to both processor groups. With no existing checkpoint, older pending outbox documents are intentionally skipped and left to expire under their existing seven-day TTL. This is not a historical replay. On later restarts, saved checkpoints take precedence, so downtime does not reset the feed to the new startup time. Deleting the lease state resets this behavior and can skip pending events again. The boundary uses Cosmos change-feed timestamps, not the event's application timestamp. + +Service Bus is the relay's destination `IEventPublisher`, never the Cosmos write-side publisher. CloudEvent payloads and metadata are preserved, and partition keys map to session identifiers consistently with the other relays. Delivery is **at-least-once**: publish retries or replay from a checkpoint can duplicate messages; consumers must tolerate duplicates. Events expire if an outage exceeds their outbox TTL. + +Tracing follows the relational relay defaults: idle change-feed polling, lease maintenance, and cleanup SDK/HTTP spans are suppressed, while actual publishing spans and originating-trace relay markers remain enabled. Metrics and warning/error logs are retained. Set `CosmosDbOutboxRelayOptions.IsInstrumentationEnabledForPolling` to `true` through `configureOptions` before start/resume when investigating processor internals. The Customers relay additionally constructs its singleton Cosmos client under an instrumentation-suppression scope: the SDK starts account discovery and its five-minute account-refresh timer during client construction, before the change-feed processor's lifecycle scopes exist. Foreground health checks remain instrumented after the construction scope is disposed. + ### Distributed tracing: why the relay's own span is not the originating trace's parent/child A relayed event's outgoing message keeps the **original producer's** W3C `traceparent`/`tracestate` untouched — `IEventFormatter.AddTracing` is idempotent and skips an event that already carries trace context, so a Subscriber's span correlates directly back to the request that raised the event (e.g. an API `PUT`), never to the relay. This is intentional, not a gap: a single relay poll can pull a batch of events raised by many causally-unrelated originating traces, so there is no one valid "parent" for the relay's own batch-level span — reparenting it into a single trace only when a batch happens to contain one trace would make the relay's visibility a runtime accident (present for batch-of-one in dev, silently gone for real multi-event batches in production). @@ -281,15 +305,24 @@ builder.Services.AzureServiceBusReceiving() var o = ServiceBusSessionReceiverOptions.CreateForTopicSubscription(); o.SessionProcessorOptions.MaxConcurrentSessions = 4; return o; - }) - .WithSubscribedSubscriber() // Routes received messages through the SubscribedManager. - .WithHostedService() // Runs the receiver as a BackgroundService. + }, "receiver-events") // Keyed so more than one receiver can coexist. + .WithKeyedSubscribedSubscriber("subscriber-events") // Routes received messages through the SubscribedManager. + .WithHostedService("hosted-subscriber-events") // Runs the receiver as a BackgroundService. + .Build(); + +// A second receiver for the domain's command queue (commands are addressed to a single consuming domain by the NamedDestinationProvider). +builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForQueue("contoso-products"), "receiver-commands") + .WithKeyedSubscribedSubscriber("subscriber-commands") + .WithHostedService("hosted-subscriber-commands") .Build(); app.MapHealthChecks(); app.MapHostedServices(); // Exposes pause/resume management endpoints. ``` +A single host can consume both a topic subscription and a command queue; each receiver, subscriber and hosted service needs its own service key (the optional `serviceKey` on `WithReceiver`/`WithSessionReceiver`). Publishing hosts register `AddNamedDestinationProvider()` so events go to the shared topic and commands to `{topic}-{domain}` queues; relay hosts need no provider as the destination is persisted in the outbox. + `AddSubscribersUsing()` scans the assembly containing `T` and auto-registers every class decorated with `[Subscribe]`, so adding a new subscriber requires only creating the class — no `Program.cs` edits are needed. > **See also**: [`SubscribedBase`](../../src/CoreEx.Events/Subscribing/SubscribedBase.cs) · [`SubscribedBase`](../../src/CoreEx.Events/Subscribing/SubscribedBase.cs) · [`ErrorHandler`](../../src/CoreEx.Events/Subscribing/ErrorHandler.cs) · [`AddSubscribedManager`](../../src/CoreEx.Azure.Messaging.ServiceBus) · [Competing Consumers pattern](https://learn.microsoft.com/en-us/azure/architecture/patterns/competing-consumers) diff --git a/samples/docs/infrastructure-layer.md b/samples/docs/infrastructure-layer.md index 061a51c4f..ef39c399c 100644 --- a/samples/docs/infrastructure-layer.md +++ b/samples/docs/infrastructure-layer.md @@ -64,8 +64,6 @@ public sealed class ProductsEfDb(ProductsDbContext dbContext) : EfDb ⚠️ **Preview**: [`CoreEx.Cosmos`](../../src/CoreEx.Cosmos) is newly added in this release — treat its API surface as subject to change without following strict semver until it stabilizes. - Customers is schemaless and code-first — there is no `*.Database`/DbEx migration project and no EF Core `DbContext`. `ref-data.yaml`-driven CodeGen still produces the generated persistence models (`Infrastructure/Persistence/*.g.cs`), same as the relational domains. Instead, `CosmosDb` (from `CoreEx.Cosmos`) is sub-classed once per domain to declare its containers, each exposed as a typed `CosmosDbContainer` (same contract/model type) or `CosmosDbMappedContainer` (contract mapped to a distinct persistence model), analogous in role to an `EfDb` unit-of-work facade: ```csharp @@ -96,7 +94,7 @@ public class CustomerRepository(CustomersCosmosDb cosmos) : ICustomerRepository } ``` -Multi-document writes (e.g. an entity plus its outbox event) go through `CosmosDbUnitOfWork`, which uses `TransactionalBatch` to commit atomically within a partition — the Cosmos analogue of the EF Core `IUnitOfWork`/outbox pattern used by Products and Shopping. Because Customers has no Relay host yet, published events accumulate in the outbox container but are not currently forwarded to Service Bus. +Multi-document writes (e.g. an entity plus its outbox event) go through `CosmosDbUnitOfWork`, which uses `TransactionalBatch` to commit atomically within a partition — the Cosmos analogue of the EF Core `IUnitOfWork`/outbox pattern used by Products and Shopping. Outbox documents are co-located with their business mutations in `customers` or `ref-data`. `Contoso.Customers.Relay` monitors both change feeds, publishes to Service Bus, and deletes successfully relayed documents. Its first-start policy excludes older pending events; subsequent restarts resume saved checkpoints (see [Customers Cosmos relay](hosts-layer.md#customers-cosmos-relay)). > **See also**: [`CosmosDb`](../../src/CoreEx.Cosmos/CosmosDb.cs) · [`CosmosDbContainer`](../../src/CoreEx.Cosmos/CosmosDbContainer.cs) · [`CosmosDbMappedContainer`](../../src/CoreEx.Cosmos/CosmosDbMappedContainer.cs) · [`CosmosDbUnitOfWork`](../../src/CoreEx.Cosmos/CosmosDbUnitOfWork.cs) @@ -146,4 +144,15 @@ public class ProductAdapter(ShoppingEfDb ef, IEventPublisher eventPublisher, Pro This split — client for transport, adapter for orchestration — keeps each class focused on a single responsibility and makes the HTTP interaction independently testable. -> **See also**: [`ToResultAsync`](../../src/CoreEx.AspNetCore/HttpResponseMessageExtensions.cs) · [`JsonDefaults`](../../src/CoreEx/Json/JsonDefaults.cs) · [Strangler Fig / ACL patterns](https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig) +`CustomersHttpClient` / `CustomerAdapter` are the purely real-time variant: no local replica, the adapter delegates to the client (`GET api/customers/{id}`), mapping a 404 to `Result.NotFoundError()`. The lookup is wrapped in a single `IHybridCache.GetOrCreateWithResultAsync(...)` call: only successful results are cached (short, configurable expiry via `CoreEx:Caching:Customer:*`), failures such as not-found are never cached. The lookup is single-flight per key (per node) under FusionCache, so concurrent misses make one remote call; coalescing across nodes sharing Redis would need an opt-in FusionCache distributed locker, which isn't worth it for this cheap read. The client is registered in the Api and Subscribe hosts with `AddTypedHttpClient("CustomersApi")`, the `CustomersApi:BaseAddress` setting being overridden with service discovery under Aspire. + +Before returning a successful lookup, `CustomersHttpClient` uses +`response.WithValidator(_validator).ToResultAsync(ct)` to require an email with valid format. +The response-specific rules are defined once in a private static readonly `Validator` +inside the client using `Validator.Create().HasProperty(...)`, leaving other details unrestricted rather +than applying create/update rules to a read response. Customer identity is trusted, not validated +or compared with the requested identifier. Invalid successful responses become +`HttpRequestException` failures with structured validation diagnostics, so they are not cached +or mistaken for caller validation errors. The client disposes the HTTP response after consumption. + +> **See also**: [`ToResultAsync`](../../src/CoreEx/Extensions.HttpResponseMessage.cs) · [`JsonDefaults`](../../src/CoreEx/Json/JsonDefaults.cs) · [Strangler Fig / ACL patterns](https://learn.microsoft.com/en-us/azure/architecture/patterns/strangler-fig) diff --git a/samples/docs/layers.md b/samples/docs/layers.md index 86ea839de..09eef1337 100644 --- a/samples/docs/layers.md +++ b/samples/docs/layers.md @@ -1,8 +1,8 @@ # Layers -CoreEx promotes a clean, layered architecture for building enterprise APIs and distributed services. The sample domains — **Products** (Postgres), **Shopping** (SQL Server), and **Customers** (Azure Cosmos DB) — demonstrate these patterns in a polyglot setting, showing that the same architectural approach scales across different persistence technologies. Each domain is decomposed into business layers (**Contracts**, **Domain** (optional), **Application**, **Infrastructure**) and one or more host layers (**API**, **Outbox Relay**, **Subscribe**). The business layers enforce a strict inward dependency rule — `Domain` references only `Contracts`; `Application` references `Contracts` and `Domain`; `Infrastructure` references `Application` (and transitively `Domain`) — never the reverse; the host layers act as composition roots that wire everything together and delegate to Application logic. Customers currently has only an API host — no Relay or Subscribe — since it is a work-in-progress sample. +CoreEx promotes a clean, layered architecture for building enterprise APIs and distributed services. The sample domains — **Products** (Postgres), **Shopping** (SQL Server), and **Customers** (Azure Cosmos DB) — demonstrate these patterns in a polyglot setting, showing that the same architectural approach scales across different persistence technologies. Each domain is decomposed into business layers (**Contracts**, **Domain** (optional), **Application**, **Infrastructure**) and one or more host layers (**API**, **Outbox Relay**, **Subscribe**). The business layers enforce a strict inward dependency rule — `Domain` references only `Contracts`; `Application` references `Contracts` and `Domain`; `Infrastructure` references `Application` (and transitively `Domain`) — never the reverse; the host layers act as composition roots that wire everything together and delegate to Application logic. Customers has API and Cosmos Outbox Relay hosts, but no Subscribe host. -> **Polyglot data**: Products is backed by PostgreSQL via [`CoreEx.Database.Postgres`](../../src/CoreEx.Database.Postgres) and [`CoreEx.EntityFrameworkCore`](../../src/CoreEx.EntityFrameworkCore). Shopping uses SQL Server via [`CoreEx.Database.SqlServer`](../../src/CoreEx.Database.SqlServer) and the same EF Core integration. Customers is backed by Azure Cosmos DB via [`CoreEx.Cosmos`](../../src/CoreEx.Cosmos) (preview — newly added, API surface may still change) — a schemaless store accessed directly through the Cosmos SDK, with no EF Core and no `*.Database` migration project (containers are created code-first). The layers above the Infrastructure boundary are completely unaware of the underlying database technology. +> **Polyglot data**: Products is backed by PostgreSQL via [`CoreEx.Database.Postgres`](../../src/CoreEx.Database.Postgres) and [`CoreEx.EntityFrameworkCore`](../../src/CoreEx.EntityFrameworkCore). Shopping uses SQL Server via [`CoreEx.Database.SqlServer`](../../src/CoreEx.Database.SqlServer) and the same EF Core integration. Customers is backed by Azure Cosmos DB via [`CoreEx.Cosmos`](../../src/CoreEx.Cosmos) — a schemaless store accessed directly through the Cosmos SDK, with no EF Core and no `*.Database` migration project (containers are created code-first). The layers above the Infrastructure boundary are completely unaware of the underlying database technology. ``` caller / message broker @@ -58,4 +58,3 @@ These are design-time console projects that feed the layer stack but have no run Tests are organised by host boundary — each `*.Test.*` project tests one deployable unit in isolation. Intra-domain dependencies (database, cache, outbox) are real; inter-domain calls (HTTP to other domains, direct broker publishes) are mocked. See [testing.md](testing.md) for the full testing guide. For cross-domain end-to-end validation and load simulation, all hosts are run together under [Aspire](aspire.md), and the [E2E Runner](aspire.md#e2e-runner) drives real workloads across the full system. - diff --git a/samples/docs/local-dev.md b/samples/docs/local-dev.md index 4b9f353f0..e20571bc2 100644 --- a/samples/docs/local-dev.md +++ b/samples/docs/local-dev.md @@ -63,9 +63,13 @@ The emulator is pre-configured by `servicebus/Config.json`. Key values the sampl | Topic | `contoso` | | Subscription — Products | `products` (session-enabled) | | Subscription — Shopping | `shopping` (session-enabled) | +| Subscription — Customers relay tests | `customers-relay` (session-enabled observation subscription; no Customers Subscribe host) | +| Queue — Products commands | `contoso-products` (session-enabled; Shopping → Products `reservation.confirm`/`reservation.cancel` commands; Shopping publisher uses a destination-specific session bucket count of `2`) | | Unit test topics | `unit-test`, `unit-test-2` (used by integration tests) | -The `contoso` topic is shared across both domains. Session-enabled subscriptions ensure ordered, per-entity processing of events. +The `contoso` topic is shared across Products, Shopping, and Customers for **events**. The `customers-relay` subscription is test-owned observation, not a business consumer. Session-enabled subscriptions ensure ordered, per-entity processing of events. **Commands** are single-consumer: the `NamedDestinationProvider` routes each command to a per-target-domain queue (`contoso-{domain}`, e.g. `contoso-products`) so only the addressed domain sees it. Any new command target needs a matching queue in `servicebus/Config.json` (and in the tests' `ServiceBus.GetQueues()`), and the emulator container must be restarted to pick it up. + +Shopping's API, Subscribe, and Relay publisher configurations override `CoreEx:Host:ServiceBus:Destinations:contoso-products:SessionIdPartitionSize` to `2`; the publisher-wide default remains `4` for other destinations. This is a producer-side key-to-session bucket setting, not Service Bus's physical partition count or the Products receiver's `MaxConcurrentSessions`. --- @@ -115,7 +119,7 @@ All sample hosts use the Aspire component configuration key hierarchy in `appset ### Azure Service Bus emulator -All hosts that publish or subscribe add the same base connection string. Subscribe hosts additionally set `QueueOrTopicName` and `SubscriptionName`: +All hosts that publish or subscribe add the same base connection string. Subscribe hosts consuming a topic additionally set `QueueOrTopicName` and `SubscriptionName` (a command queue receiver, e.g. `contoso-products`, is named explicitly via `CreateForQueue("contoso-products")`): ```json "Aspire": { @@ -164,6 +168,11 @@ dotnet run --project samples/src/Contoso.Products.Api dotnet run --project samples/src/Contoso.Products.Relay dotnet run --project samples/src/Contoso.Products.Subscribe +# Customers (provision both business containers and $outbox-leases first) +dotnet run --project samples/src/Contoso.Customers.Database -f net10.0 -- All +dotnet run --project samples/src/Contoso.Customers.Api -f net10.0 +dotnet run --project samples/src/Contoso.Customers.Relay -f net10.0 + # Shopping dotnet run --project samples/src/Contoso.Shopping.Api dotnet run --project samples/src/Contoso.Shopping.Relay @@ -176,6 +185,8 @@ dotnet run --project samples/src/Contoso.Orders.Api Intra-domain host tests (`*.Test.Api`, `*.Test.Subscribe`, `*.Test.Relay`) start their own in-process test host — they do not require any host process to be running. Infrastructure containers must still be up. +Orders API uses HTTPS port `7330` and HTTP port `5330`; the E2E Runner targets `https://localhost:7330`. Keep fixed development ports outside Windows' default dynamic TCP range (`49152`-`65535`), where changing port reservations can prevent Aspire's proxy from binding. Restart the AppHost and runner after changing their endpoint configuration. + --- ## Running with Aspire (cross-domain E2E) diff --git a/samples/docs/patterns.md b/samples/docs/patterns.md index 926f0ffc2..a35e815af 100644 --- a/samples/docs/patterns.md +++ b/samples/docs/patterns.md @@ -57,4 +57,3 @@ The samples are built on two overarching architectural styles — **Domain-based | | 🔗 | **Inter-domain Mocking** | Replacing cross-domain HTTP calls and direct broker publishes with controlled fakes that assert the correct outbound request was made, decoupling test correctness from the availability or behavior of other domains. | [Testing](testing.md#intra-domain-vs-inter-domain-testing) | | | 🧹 | **Unit Testing** | Testing stateless components (validators, mappers) in a minimal DI container with no I/O, using real reference data loaded from the domain’s own seed file and mocking only intra-domain repository dependencies. | [Testing](testing.md#unit-tests-testunit) | | | ✨ | **Local Orchestration** | Running all domain hosts simultaneously as a single distributed application with unified health, log, trace, and metric visibility, enabling cross-domain interaction and end-to-end validation in a local environment. | [Aspire](aspire.md) | - diff --git a/samples/docs/testing.md b/samples/docs/testing.md index 167ad8088..34a94b8a6 100644 --- a/samples/docs/testing.md +++ b/samples/docs/testing.md @@ -26,11 +26,12 @@ Understanding this distinction is the key to understanding every test setup deci | Azure Service Bus (direct publish) | Shopping | **Inter** | Captured via `UseExpectedAzureServiceBusPublisher()` | | Azure Service Bus (relay) | Products Outbox Relay | **Inter** | Real — drained and asserted via `GetAndClearAzureServiceBusAsync` | | Customers Cosmos DB (containers) | Customers | **Intra** | Real — created/reset code-first via `ReplaceOrCreateContainerAsync` in `[OneTimeSetUp]` | -| Customers outbox (Cosmos DB) | Customers | **Intra** | Real — captured via `UseExpectedCosmosDbOutboxPublisher()`; asserted via `ExpectCosmosDbOutboxEvents(...)` / `ExpectNoCosmosDbOutboxEvents()`. No Relay host yet, so captured events are never actually forwarded to Service Bus. | +| Customers outbox (Cosmos DB) | Customers | **Intra** | Real — captured in API tests via `UseExpectedCosmosDbOutboxPublisher()` and `ExpectCosmosDbOutboxEvents(...)` / `ExpectNoCosmosDbOutboxEvents()`. | +| Azure Service Bus (Cosmos relay) | Customers Outbox Relay | **Inter** | Real — `Contoso.Customers.Test.Relay` verifies both containers, payload/session metadata, cleanup, first-start exclusion, and pause/resume using the isolated `customers-relay` subscription. Aspire additionally verifies API-to-broker lifecycle events. | The Shopping `Basket_Checkout_Save_Failure` test is the sharpest illustration of the boundary: when the outbox write fails mid-checkout, Shopping falls back to publishing a `reservation.cancel` command *directly* to Service Bus (bypassing the outbox, since the DB transaction has already rolled back). The test asserts that: - No outbox events are published (intra-domain write failed, as injected). -- One direct Service Bus event *is* published (inter-domain cancel, asserted via `ExpectAzureServiceBusEvents`). +- One direct Service Bus command *is* published (inter-domain cancel, asserted via `ExpectAzureServiceBusEvents`) - addressed to the `contoso-products` command queue (the `NamedDestinationProvider` routes events to the shared `contoso` topic and commands to `{topic}-{domain}` queues). - The basket remains `Active` (state was correctly rolled back). This single test exercises three layers of the intra/inter boundary in one shot. @@ -125,7 +126,7 @@ The `[OneTimeSetUp]` pattern for all API test classes: 1. **Migrate and seed** — `MigratePostgresDataAsync` / `MigrateSqlServerDataAsync` resets the domain schema to the contents of `Data/data.yaml` in `*.Test.Common`. A `DataResetFilterPredicate` scopes the reset to the domain's own schema so Products and Shopping test runs cannot affect each other. 2. **Clear cache** — `ClearFusionCacheAsync()` flushes L1 (in-process) and L2 (Redis) so tests start from a known state. 3. **Capture events** — `UseExpectedOutboxPublisher()` wraps the outbox publisher with a capture decorator; tests can then assert published events inline. -4. **Mock inter-domain HTTP** (Shopping only) — `MockHttpClientFactory` intercepts `POST api/inventory/reserve` so the Shopping API can be tested without a running Products API. +4. **Mock inter-domain HTTP** (Shopping only) — `MockHttpClientFactory` intercepts `POST api/inventory/reserve` (Products) and `GET api/customers/{id}` (Customers) so the Shopping API can be tested without running either API. Customer ids used in Shopping tests are the golden values from the Customers domain's `read-data.seed.yaml` (e.g. `^11` Alice with no address, `^16` Frank with an address). ```csharp // Contoso.Shopping.Test.Api — intra-domain real, inter-domain mocked @@ -162,7 +163,7 @@ var v = Test.Http() .ExpectChangeLogUpdated() .ExpectSqlServerOutboxEvents(e => e .AssertWithValue("contoso", "contoso.shopping.basket.checkedout.v1") - .AssertMetadata("contoso", "contoso.products.reservation.confirm", basket.Id)) + .AssertMetadata("contoso-products", "contoso.products.reservation.confirm", basket.Id)) .Run(HttpMethod.Post, $"/api/baskets/{basket.Id}/checkout") .AssertOK() .Value!; @@ -176,7 +177,7 @@ Test.Http() .OnEventPublish(SqlServerOutboxPublisher.DefaultServiceKey, () => throw new InvalidOperationException("Simulated outbox failure")) .ExpectNoSqlServerOutboxEvents() - .ExpectAzureServiceBusEvents(e => e.AssertMetadata("contoso", "contoso.products.reservation.cancel", id)) + .ExpectAzureServiceBusEvents(e => e.AssertMetadata("contoso-products", "contoso.products.reservation.cancel", id)) .Run(HttpMethod.Post, $"/api/baskets/{id}/checkout") .AssertInternalServerError(); ``` @@ -199,7 +200,7 @@ public partial class SubscriberTests : WithApiTester(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); }).AssertSuccess(); @@ -296,4 +297,6 @@ Each `*.Test.Api` and `*.Test.Relay` project has a `Resources/` folder containin `Contoso.E2E.Runner` is an interactive console runner for cross-domain scenarios. Unlike the host tests above it requires **all** infrastructure and **all** hosts to be running simultaneously — orchestrated via Aspire. It tests the complete inter-domain flow end-to-end: basket checkout triggers a real HTTP call to Products, which publishes a real event to Service Bus, which is consumed by the real subscriber. It also supports a parallel load-simulation mode for concurrency and performance validation. -See [aspire.md](aspire.md) for the full Aspire setup, E2E Runner usage, scenario descriptions, load-simulation configuration, and the recommended first-run order. +`Contoso.Test.Aspire` is the automated, CI-friendly counterpart — an NUnit test that self-hosts the whole `Contoso.Aspire` AppHost via `WithAspireTester<...>` (no need for Aspire to already be running) and asserts the same cross-domain flow as a single deterministic pass/fail test. + +See [aspire.md](aspire.md) for the full Aspire setup, `Contoso.Test.Aspire`, E2E Runner usage, scenario descriptions, load-simulation configuration, and the recommended first-run order. diff --git a/samples/src/Contoso.Customers.Api/Contoso.Customers.Api.csproj b/samples/src/Contoso.Customers.Api/Contoso.Customers.Api.csproj index 9dcd1a7f6..6ca4e048c 100644 --- a/samples/src/Contoso.Customers.Api/Contoso.Customers.Api.csproj +++ b/samples/src/Contoso.Customers.Api/Contoso.Customers.Api.csproj @@ -10,8 +10,13 @@ + + + + + diff --git a/samples/src/Contoso.Customers.Api/Controllers/CustomerTypeController.g.cs b/samples/src/Contoso.Customers.Api/Controllers/CustomerTypeController.g.cs new file mode 100644 index 000000000..7dd0462fd --- /dev/null +++ b/samples/src/Contoso.Customers.Api/Controllers/CustomerTypeController.g.cs @@ -0,0 +1,71 @@ + + +// + +/* + * This file is automatically generated by 'Contoso.Customers.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Customers.Api.Controllers; + +/// Represents the reference-data controller. +[ApiController, Route("/api/refdata/customer-types")] +[AllowAnonymous] +public partial class CustomerTypeController(CoreEx.AspNetCore.Mvc.WebApi webApi, IReferenceDataService service) : ControllerBase +{ + private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); + private readonly IReferenceDataService _service = service.ThrowIfNull(); + + [HttpGet("{id}"), HttpHead("{id}")] + [ProducesResponseType(typeof(CustomerType), 200)] + [ProducesNotFoundProblem()] + public Task GetAsync(string id, CancellationToken cancellationToken = default) + => _webApi.GetWithResultAsync(Request, (_, ct) => _service.GetCustomerTypeAsync(id.Required(), ct), cancellationToken: cancellationToken); + + [HttpPost] + [Accepts] + [ProducesResponseType(201)] + [IdempotencyKey] + public Task CreateAsync(CancellationToken cancellationToken = default) => _webApi.PostWithResultAsync(Request, (ro, ct) => + { + ro.WithLocationUri(v => new Uri($"/api/refdata/customer-types/{v.Id}", UriKind.Relative)); + return _service.CreateCustomerTypeAsync(ro.Value, ct); + }, cancellationToken: cancellationToken); + + [HttpPatch("{id}")] + [Accepts(HttpNames.MergePatchJsonMediaTypeName)] + [ProducesResponseType(typeof(CustomerType), 200)] + [ProducesNotFoundProblem()] + public Task PatchAsync(string id, CancellationToken cancellationToken = default) => _webApi.PatchWithResultAsync(Request, + get: (ro, ct) => _service.GetCustomerTypeAsync(id.Required(), ct), + put: (ro, ct) => _service.UpdateCustomerTypeAsync(id, ro.Value, ct), + cancellationToken: cancellationToken); + + [HttpPut("{id}")] + [Accepts] + [ProducesResponseType(typeof(CustomerType), 200)] + [ProducesNotFoundProblem()] + public Task PutAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PutWithResultAsync(Request, (ro, ct) => _service.UpdateCustomerTypeAsync(id.Required(), ro.Value, ct), cancellationToken: cancellationToken); + + [HttpPost("{id}/activate")] + [ProducesResponseType(typeof(CustomerType), 200)] + [ProducesNotFoundProblem()] + public Task ActivateAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync(Request, (_, ct) => _service.ActivateCustomerTypeAsync(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpPost("{id}/deactivate")] + [ProducesResponseType(typeof(CustomerType), 200)] + [ProducesNotFoundProblem()] + public Task DeactivateAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync(Request, (_, ct) => _service.DeactivateCustomerTypeAsync(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpDelete("{id}")] + [ProducesResponseType(204)] + public Task DeleteAsync(string id, CancellationToken cancellationToken = default) + => _webApi.DeleteWithResultAsync(Request, (_, ct) => _service.DeleteCustomerTypeAsync(id.Required(), ct), cancellationToken: cancellationToken); +} + +#nullable restore diff --git a/samples/src/Contoso.Customers.Api/Controllers/ReferenceDataController.g.cs b/samples/src/Contoso.Customers.Api/Controllers/ReferenceDataController.g.cs index 818d28dba..d72e6a032 100644 --- a/samples/src/Contoso.Customers.Api/Controllers/ReferenceDataController.g.cs +++ b/samples/src/Contoso.Customers.Api/Controllers/ReferenceDataController.g.cs @@ -14,22 +14,17 @@ public partial class ReferenceDataController(CoreEx.AspNetCore.Mvc.WebApi webApi { private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); - [HttpGet("customer-types"), HttpHead("customer-types")] + [HttpGet("customer-types")] [ProducesResponseType(typeof(CustomerType[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetCustomerTypesAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("contact-methods"), HttpHead("contact-methods")] + [HttpGet("contact-methods")] [ProducesResponseType(typeof(ContactMethod[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetContactMethodsAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - - [HttpGet] - [ProducesResponseType(typeof(ReferenceDataMultiDictionary), 200)] - public Task GetNamedAsync([FromQuery] string[] name, CancellationToken cancellationToken) - => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.GetNamedAsync(name, ro.IsIncludeInactive, ct), cancellationToken: cancellationToken); } #nullable restore diff --git a/samples/src/Contoso.Customers.Api/GlobalUsing.cs b/samples/src/Contoso.Customers.Api/GlobalUsing.cs index 47653192b..910c17de0 100644 --- a/samples/src/Contoso.Customers.Api/GlobalUsing.cs +++ b/samples/src/Contoso.Customers.Api/GlobalUsing.cs @@ -14,8 +14,15 @@ global using CoreEx.Json; global using CoreEx.RefData; global using CoreEx.Validation; +global using Microsoft.AspNetCore.Authorization; global using Microsoft.AspNetCore.Mvc; global using Microsoft.Azure.Cosmos; +global using Microsoft.Extensions.Options; global using NSwag.Annotations; +global using OpenTelemetry; +global using OpenTelemetry.Trace; +global using StackExchange.Redis; global using System.Net; global using System.Text.Json; +global using ZiggyCreatures.Caching.Fusion; +global using ZiggyCreatures.Caching.Fusion.Backplane.StackExchangeRedis; diff --git a/samples/src/Contoso.Customers.Api/Program.cs b/samples/src/Contoso.Customers.Api/Program.cs index 8d5ebb6a5..267c56a19 100644 --- a/samples/src/Contoso.Customers.Api/Program.cs +++ b/samples/src/Contoso.Customers.Api/Program.cs @@ -1,7 +1,3 @@ -using OpenTelemetry; -using OpenTelemetry.Trace; -using ZiggyCreatures.Caching.Fusion; - namespace Contoso.Customers.Api; public class Program @@ -23,12 +19,15 @@ private static void Main(string[] args) .AddHttpWebApi(); // Add all the dynamically registered services. - builder.Services.AddDynamicServicesUsing(); + builder.Services.AddDynamicServicesUsing(); - // Add caching services - in-memory (L1) only for this sample; no distributed (L2)/Redis, kept deliberately simple. + // Add L1/L2 caching services and the Redis backplane. builder.Services.AddMemoryCache(); + builder.AddRedisDistributedCache("redis"); builder.Services.AddFusionCache() .WithRegisteredMemoryCache() + .WithRegisteredDistributedCache() + .WithBackplane(sp => new RedisBackplane(new RedisBackplaneOptions { Configuration = sp.GetRequiredService>().Value.ToString() })) .WithSystemTextJsonSerializer(JsonDefaults.SerializerOptions); builder.Services @@ -52,6 +51,7 @@ private static void Main(string[] args) builder.Services.AddCosmosDb("contoso"); builder.Services .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddCosmosDbEventPublisher() // Adds the CosmosDbEventPublisher/IEventPublisher .AddCosmosDbUnitOfWork() // Adds the CosmosDbUnitOfWork/IUnitOfWork, matching AddPostgresUnitOfWork/AddSqlServerUnitOfWork's multi-register shape. .AddCosmosDbHealthCheck(); // Adds the CosmosDbHealthCheck - Aspire's own AddAzureCosmosClient does not register one itself, unlike its Npgsql/SqlClient counterparts. diff --git a/samples/src/Contoso.Customers.Api/appsettings.Development.json b/samples/src/Contoso.Customers.Api/appsettings.Development.json index 093f80cd4..c0df19d2e 100644 --- a/samples/src/Contoso.Customers.Api/appsettings.Development.json +++ b/samples/src/Contoso.Customers.Api/appsettings.Development.json @@ -3,10 +3,13 @@ "LogLevel": { "Default": "Information", "Azure": "Warning", - "Microsoft": "Warning" + "Microsoft": "Warning", + "ZiggyCreature": "Warning", + "StackExchange": "Warning" } }, "ConnectionStrings": { + "redis": "localhost:6379", "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==" } } diff --git a/samples/src/Contoso.Customers.Application/Interfaces/IReferenceDataService.g.cs b/samples/src/Contoso.Customers.Application/Interfaces/IReferenceDataService.g.cs new file mode 100644 index 000000000..6252ab531 --- /dev/null +++ b/samples/src/Contoso.Customers.Application/Interfaces/IReferenceDataService.g.cs @@ -0,0 +1,37 @@ +// + +/* + * This file is automatically generated by 'Contoso.Customers.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Customers.Application; + +/// Enables the mutable services. +public partial interface IReferenceDataService +{ + #region CustomerType + + /// Gets the specified . + Task> GetCustomerTypeAsync(string id, CancellationToken cancellationToken); + + /// Creates the specified . + Task> CreateCustomerTypeAsync(CustomerType value, CancellationToken cancellationToken); + + /// Updates the specified . + Task> UpdateCustomerTypeAsync(string id, CustomerType value, CancellationToken cancellationToken); + + /// Activates the specified . + Task> ActivateCustomerTypeAsync(string id, CancellationToken cancellationToken); + + /// Deactivates the specified . + Task> DeactivateCustomerTypeAsync(string id, CancellationToken cancellationToken); + + /// Deletes the specified . + Task DeleteCustomerTypeAsync(string id, CancellationToken cancellationToken); + + #endregion +} + +#nullable restore diff --git a/samples/src/Contoso.Customers.Application/ReferenceDataProvider.g.cs b/samples/src/Contoso.Customers.Application/ReferenceDataProvider.g.cs new file mode 100644 index 000000000..feda79528 --- /dev/null +++ b/samples/src/Contoso.Customers.Application/ReferenceDataProvider.g.cs @@ -0,0 +1,40 @@ +// + +/* + * This file is automatically generated by 'Contoso.Customers.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Customers.Application; + +/// Provides the implementation. +[ScopedService] +public partial class ReferenceDataProvider(IReferenceDataRepository repository) : IReferenceDataProvider +{ + private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + + /// + public IEnumerable<(Type, Type)> Types => + [ + (typeof(CustomerType), typeof(CustomerTypeCollection)), + (typeof(ContactMethod), typeof(ContactMethodCollection)), + ]; + + /// + public IEnumerable<(string, Type)> AlternateNames => + [ + ( "customer-types", typeof(CustomerType) ), + ( "contact-methods", typeof(ContactMethod) ) + ]; + + /// + public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch + { + _ when type == typeof(CustomerType) => await _repository.GetAllCustomerTypesAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(ContactMethod) => await _repository.GetAllContactMethodsAsync(cancellationToken).ConfigureAwait(false), + _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") + }; +} + +#nullable restore diff --git a/samples/src/Contoso.Customers.Application/ReferenceDataService.g.cs b/samples/src/Contoso.Customers.Application/ReferenceDataService.g.cs index 687dc2526..17cd36568 100644 --- a/samples/src/Contoso.Customers.Application/ReferenceDataService.g.cs +++ b/samples/src/Contoso.Customers.Application/ReferenceDataService.g.cs @@ -8,33 +8,212 @@ namespace Contoso.Customers.Application; -/// Provides the implementation. -[ScopedService] -public partial class ReferenceDataService(IReferenceDataRepository repository) : IReferenceDataProvider +/// Provides the mutable implementation. +[ScopedService] +public partial class ReferenceDataService : IReferenceDataService { - private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + private readonly IUnitOfWork _unitOfWork; + private readonly IReferenceDataRepository _repository; + + /// + /// Initializes a new instance of the class. + /// + /// The . + /// The . + public ReferenceDataService(IUnitOfWork unitOfWork, IReferenceDataRepository repository) + { + _unitOfWork = unitOfWork.ThrowIfNull(); + _repository = repository.ThrowIfNull(); + OnInitialization(); + } + + /// + /// Provides the with an opportunity to perform additional initialization. + /// + partial void OnInitialization(); + + /// + /// Gets or sets the pre-check function that is invoked prior to the activate, deactivate or delete. + /// + /// This is intended to be used for any additional validation or business rules that need to be applied prior to the action being performed. + /// The passed is the reference data entity being mutated and the indicates the resulting action (being one of: + /// , , or ). + private Func> PreCheckAsync { get; set => field = value.ThrowIfNull(); } = (_, _, _) => Result.SuccessTask; + + #region CustomerType + + /// + public Task> GetCustomerTypeAsync(string id, CancellationToken cancellationToken) => GetAsync(id, _repository.GetCustomerTypeAsync, cancellationToken); + + /// + public Task> CreateCustomerTypeAsync(CustomerType value, CancellationToken cancellationToken) => CreateAsync>(value, _repository.CreateCustomerTypeAsync, cancellationToken, refresh: (v, ct) => _repository.GetCustomerTypeAsync(v.Id, ct)); /// - public IEnumerable<(Type, Type)> Types => - [ - (typeof(CustomerType), typeof(CustomerTypeCollection)), - (typeof(ContactMethod), typeof(ContactMethodCollection)), - ]; + public Task> UpdateCustomerTypeAsync(string id, CustomerType value, CancellationToken cancellationToken) => UpdateAsync>(id, value, _repository.UpdateCustomerTypeAsync, cancellationToken, refresh: (v, ct) => _repository.GetCustomerTypeAsync(v.Id, ct)); /// - public IEnumerable<(string, Type)> AlternateNames => - [ - ( "customer-types", typeof(CustomerType) ), - ( "contact-methods", typeof(ContactMethod) ) - ]; + public Task> ActivateCustomerTypeAsync(string id, CancellationToken cancellationToken) => ActivateAsync(id, _repository.GetCustomerTypeAsync, _repository.ActivateCustomerTypeAsync, cancellationToken, refresh: (v, ct) => _repository.GetCustomerTypeAsync(v.Id, ct)); /// - public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch + public Task> DeactivateCustomerTypeAsync(string id, CancellationToken cancellationToken) => DeactivateAsync(id, _repository.GetCustomerTypeAsync, _repository.DeactivateCustomerTypeAsync, cancellationToken, refresh: (v, ct) => _repository.GetCustomerTypeAsync(v.Id, ct)); + + /// + public Task DeleteCustomerTypeAsync(string id, CancellationToken cancellationToken) => DeleteAsync(id, _repository.GetCustomerTypeAsync, _repository.DeleteCustomerTypeAsync, cancellationToken); + + #endregion + + #region Internal + + /// + /// Gets the reference data of type by the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The cancellation token. + /// The result. + private Task> GetAsync(TId id, Func>> getFunc, CancellationToken cancellationToken) + where TRef : class, IReferenceData + => Result.GoAsync(() => getFunc(id, cancellationToken)); + + /// + /// Creates the reference data of type using the specified . + /// + /// The reference data . + /// The validator . + /// The reference data value. + /// The function to create the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> CreateAsync(TRef value, Func>>> create, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => create(value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Created)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Updates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The validator . + /// The identifier. + /// The reference data value. + /// The function to update the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> UpdateAsync(TId id, TRef value, Func>>> update, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => update(id, value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Updated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Activates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to activate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> ActivateAsync(TId id, Func>> get, Func>>> activate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Activated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => activate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Activated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Deactivates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to deactivate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> DeactivateAsync(TId id, Func>> get, Func>>> deactivate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Deactivated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => deactivate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Deactivated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Re-gets the mutated reference data using the function (where specified) so that the final persisted is returned. + /// + /// This occurs after the mutation has been committed, so it is deliberately not cancellable and a refresh failure is swallowed; i.e. it must never turn a committed success into an error, which would encourage a retry of a completed operation (the ETag is then simply as per the mutation). + private static async Task>> RefreshAsync(DataResult dr, Func>>? refresh) + where TRef : class, IReferenceData + { + if (refresh is null || !dr.WasMutated) + return dr; + + try + { + var r = await refresh(dr.Value, CancellationToken.None).ConfigureAwait(false); + return r.IsSuccess ? new DataResult(r.Value, true) : dr; + } + catch (Exception) + { + return dr; + } + } + /// + /// Deletes the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to delete the reference data. + /// The cancellation token. + /// The result. + private async Task DeleteAsync(TId id, Func>> get, Func>> delete, CancellationToken cancellationToken) + where TRef : class, IReferenceData { - _ when type == typeof(CustomerType) => await _repository.GetAllCustomerTypesAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(ContactMethod) => await _repository.GetAllContactMethodsAsync(cancellationToken).ConfigureAwait(false), - _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") - }; + var r = await get(id, cancellationToken).ConfigureAwait(false); + if (r.IsFailure) + return r.IsNotFoundError ? Result.Success : r.AsResult(); // Delete is idempotent, so if not found then return success. + + return await Result.GoAsync(() => PreCheckAsync(r.Value, EventAction.Deleted, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => delete(id, ct)) + .Then(dr => dr.WhereMutatedAnd(() => _unitOfWork.Events.Add(EventData.CreateEvent(EventAction.Deleted).WithKey(CompositeKey.Create(id))))); + }, cancellationToken)) + .ThenAsAsync(dr => dr.WhereMutatedAsync(ct => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)).ConfigureAwait(false); + } + + #endregion } #nullable restore diff --git a/samples/src/Contoso.Customers.Application/Repositories/IReferenceDataRepository.g.cs b/samples/src/Contoso.Customers.Application/Repositories/IReferenceDataRepository.g.cs index 0d2a5d10c..5fb5aa32c 100644 --- a/samples/src/Contoso.Customers.Application/Repositories/IReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Customers.Application/Repositories/IReferenceDataRepository.g.cs @@ -12,12 +12,32 @@ namespace Contoso.Customers.Application.Repositories; public partial interface IReferenceDataRepository { /// Gets all items. - /// The . Task GetAllCustomerTypesAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllContactMethodsAsync(CancellationToken cancellationToken = default); + + #region CustomerType + + /// Gets the specified . + Task> GetCustomerTypeAsync(string id, CancellationToken cancellationToken = default); + + /// Creates the . + Task>> CreateCustomerTypeAsync(Contracts.CustomerType value, CancellationToken cancellationToken = default); + + /// Updates the . + Task>> UpdateCustomerTypeAsync(string id, Contracts.CustomerType value, CancellationToken cancellationToken = default); + + /// Activates the . + Task>> ActivateCustomerTypeAsync(string id, CancellationToken cancellationToken = default); + + /// Deactivates the . + Task>> DeactivateCustomerTypeAsync(string id, CancellationToken cancellationToken = default); + + /// Deletes the specified . + Task> DeleteCustomerTypeAsync(string id, CancellationToken cancellationToken = default); + + #endregion } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Customers.CodeGen/ref-data.yaml b/samples/src/Contoso.Customers.CodeGen/ref-data.yaml index 8616c8d6b..880b3288b 100644 --- a/samples/src/Contoso.Customers.CodeGen/ref-data.yaml +++ b/samples/src/Contoso.Customers.CodeGen/ref-data.yaml @@ -3,4 +3,6 @@ collectionSortOrder: Code repository: Cosmos entities: - name: CustomerType + mutability: CreateUpdateDelete + mutableAttribute: '[AllowAnonymous]' # This sample API has no authentication; production APIs should retain the default [Authorize]. - name: ContactMethod diff --git a/samples/src/Contoso.Customers.Contracts/Address.cs b/samples/src/Contoso.Customers.Contracts/Address.cs index cb2523582..2db66fec0 100644 --- a/samples/src/Contoso.Customers.Contracts/Address.cs +++ b/samples/src/Contoso.Customers.Contracts/Address.cs @@ -3,9 +3,16 @@ namespace Contoso.Customers.Contracts; [Contract] public partial class Address { + [NonNullable] public string? Street1 { get; set; } + public string? Street2 { get; set; } + [NonNullable] + public string? City { get; set; } + [NonNullable] public string? PostCode { get; set; } + + [NonNullable] public string? State { get; set; } } diff --git a/samples/src/Contoso.Customers.Contracts/CustomerBase.cs b/samples/src/Contoso.Customers.Contracts/CustomerBase.cs index 70aff5505..5f3c48021 100644 --- a/samples/src/Contoso.Customers.Contracts/CustomerBase.cs +++ b/samples/src/Contoso.Customers.Contracts/CustomerBase.cs @@ -6,10 +6,13 @@ public abstract partial class CustomerBase : IIdentifier [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? FirstName { get; set; } + [NonNullable] public string? LastName { get; set; } + [NonNullable] public string? Email { get; set; } public string? Phone { get; set; } @@ -17,9 +20,11 @@ public abstract partial class CustomerBase : IIdentifier public Address? ShippingAddress { get; set; } [ReferenceData] + [NonNullable] public partial string? CustomerTypeCode { get; set; } [ReferenceData] + [NonNullable] public partial string? ContactMethodCode { get; set; } [ReadOnly(true)] diff --git a/samples/src/Contoso.Customers.Contracts/CustomerLite.cs b/samples/src/Contoso.Customers.Contracts/CustomerLite.cs index 3013b3c97..33deb7580 100644 --- a/samples/src/Contoso.Customers.Contracts/CustomerLite.cs +++ b/samples/src/Contoso.Customers.Contracts/CustomerLite.cs @@ -6,13 +6,17 @@ public partial class CustomerLite : IIdentifier [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? FirstName { get; set; } + [NonNullable] public string? LastName { get; set; } + [NonNullable] public string? Email { get; set; } [ReferenceData] + [NonNullable] public partial string? CustomerTypeCode { get; set; } [ReadOnly(true)] diff --git a/samples/src/Contoso.Customers.Database/Contoso.Customers.Database.csproj b/samples/src/Contoso.Customers.Database/Contoso.Customers.Database.csproj new file mode 100644 index 000000000..16e9ed841 --- /dev/null +++ b/samples/src/Contoso.Customers.Database/Contoso.Customers.Database.csproj @@ -0,0 +1,18 @@ + + + + Exe + + true + + + + + + + + + + + diff --git a/samples/tests/Contoso.Customers.Test.Common/Data/ref-data.seed.yaml b/samples/src/Contoso.Customers.Database/Data/ref-data.seed.yaml similarity index 100% rename from samples/tests/Contoso.Customers.Test.Common/Data/ref-data.seed.yaml rename to samples/src/Contoso.Customers.Database/Data/ref-data.seed.yaml diff --git a/samples/src/Contoso.Customers.Database/GlobalUsing.cs b/samples/src/Contoso.Customers.Database/GlobalUsing.cs new file mode 100644 index 000000000..eb24d23d9 --- /dev/null +++ b/samples/src/Contoso.Customers.Database/GlobalUsing.cs @@ -0,0 +1,2 @@ +global using CoreEx.Cosmos.Provisioning; +global using Microsoft.Azure.Cosmos; diff --git a/samples/src/Contoso.Customers.Database/Program.cs b/samples/src/Contoso.Customers.Database/Program.cs new file mode 100644 index 000000000..a6ac8b3d6 --- /dev/null +++ b/samples/src/Contoso.Customers.Database/Program.cs @@ -0,0 +1,49 @@ +namespace Contoso.Customers.Database; + +/// +/// Provisions the Customers Cosmos DB database and containers, and imports the seed data; the Cosmos DB equivalent of the relational domains' *.Database (DbEx) projects. +/// +/// Run using dotnet run -- All (or DropAndAll; ResetAndData requires the database to already exist) (see --help for all commands). The container identifiers declared here must match those used by CustomersCosmosDb. +public class Program +{ + /// + /// The default (local emulator) connection string; override using -cs. + /// + public const string DefaultConnectionString = "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="; + + /// + /// The default database identifier; override using -d. + /// + public const string DefaultDatabaseId = "contoso"; + + /// + /// The program entry point. + /// + public static Task Main(string[] args) => CosmosDbConsole.Create(DefaultConnectionString, DefaultDatabaseId).Configure(c => ConfigureProvisionArgs(c.Args)).RunAsync(args); + + /// + /// Declares the containers (and embedded Data seed resources); shared by the console and the tests so both provision identically. + /// + /// The . + /// The . + public static CosmosDbProvisionArgs ConfigureProvisionArgs(CosmosDbProvisionArgs args) => args + .AddAssembly() + .OutboxLeaseContainer() + .Container("customers", configure: cp => + { + // CustomerQueryArgsConfig's "LastName" order-by field is configured WithAlwaysInclude() (always appended, in its own default ascending direction, regardless of what the caller actually + // requested) - so ordering by "FirstName" always produces a two-property ORDER BY (e.g. "firstName DESC, lastName ASC"), which Cosmos DB rejects outright unless a matching composite index + // exists. Both direction combinations actually reachable via the API ($orderby=firstname[,desc]) are indexed. + cp.IndexingPolicy.CompositeIndexes.Add( + [ + new() { Path = "/firstName", Order = CompositePathSortOrder.Ascending }, + new() { Path = "/lastName", Order = CompositePathSortOrder.Ascending } + ]); + cp.IndexingPolicy.CompositeIndexes.Add( + [ + new() { Path = "/firstName", Order = CompositePathSortOrder.Descending }, + new() { Path = "/lastName", Order = CompositePathSortOrder.Ascending } + ]); + }) + .ReferenceDataContainer("ref-data"); +} diff --git a/samples/src/Contoso.Customers.Infrastructure/GlobalUsing.cs b/samples/src/Contoso.Customers.Infrastructure/GlobalUsing.cs index 541666ce3..84b321dc2 100644 --- a/samples/src/Contoso.Customers.Infrastructure/GlobalUsing.cs +++ b/samples/src/Contoso.Customers.Infrastructure/GlobalUsing.cs @@ -11,6 +11,7 @@ global using CoreEx.Events.Publishing; global using CoreEx.Mapping; global using CoreEx.RefData; +global using CoreEx.Results; global using Microsoft.Azure.Cosmos; global using Microsoft.Extensions.Logging; global using System.Text.Json; diff --git a/samples/src/Contoso.Customers.Infrastructure/Mapping/ContactMethodMapper.g.cs b/samples/src/Contoso.Customers.Infrastructure/Mapping/ContactMethodMapper.g.cs index 844e119f1..a7277d5b2 100644 --- a/samples/src/Contoso.Customers.Infrastructure/Mapping/ContactMethodMapper.g.cs +++ b/samples/src/Contoso.Customers.Infrastructure/Mapping/ContactMethodMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Customers.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class ContactMethodMapper : Mapper +/// Provides mapping from to . +internal partial class ContactMethodMapper : BiDirectionMapper { /// protected override Contracts.ContactMethod OnMap(Persistence.ContactMethod source) @@ -31,8 +31,11 @@ protected override Contracts.ContactMethod OnMap(Persistence.ContactMethod sourc return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.ContactMethod source, Contracts.ContactMethod destination); + + /// + protected override Persistence.ContactMethod OnMap(Contracts.ContactMethod destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Customers.Infrastructure/Mapping/CustomerTypeMapper.g.cs b/samples/src/Contoso.Customers.Infrastructure/Mapping/CustomerTypeMapper.g.cs index b845905c2..976238637 100644 --- a/samples/src/Contoso.Customers.Infrastructure/Mapping/CustomerTypeMapper.g.cs +++ b/samples/src/Contoso.Customers.Infrastructure/Mapping/CustomerTypeMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Customers.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class CustomerTypeMapper : Mapper +/// Provides mapping from to . +internal partial class CustomerTypeMapper : BiDirectionMapper { /// protected override Contracts.CustomerType OnMap(Persistence.CustomerType source) @@ -31,8 +31,30 @@ protected override Contracts.CustomerType OnMap(Persistence.CustomerType source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.CustomerType source, Contracts.CustomerType destination); + + /// + protected override Persistence.CustomerType OnMap(Contracts.CustomerType destination) + { + var source = new Persistence.CustomerType + { + Code = destination.Code!, + Text = destination.Text, + Description = destination.Description, + SortOrder = destination.SortOrder, + IsActive = !destination.IsInactive, + StartsOn = destination.StartsOn, + EndsOn = destination.EndsOn, + ETag = destination.ETag + }; + + OnMapExtend(destination, source); + return source; + } + + /// Provides the opportunity to extend the From mapping. + partial void OnMapExtend(Contracts.CustomerType destination, Persistence.CustomerType source); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomerQueryArgsConfig.cs b/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomerQueryArgsConfig.cs index fd50175dd..dcd821e07 100644 --- a/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomerQueryArgsConfig.cs +++ b/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomerQueryArgsConfig.cs @@ -10,7 +10,7 @@ public CustomerQueryArgsConfig() WithFilter(filter => filter .AddField(nameof(Contracts.CustomerBase.FirstName), c => c.WithOperators(QueryFilterOperator.StringFunctions).AsUpperCase()) .AddField(nameof(Contracts.CustomerBase.LastName), c => c.WithOperators(QueryFilterOperator.StringFunctions).AsUpperCase()) - .AddField(nameof(Contracts.CustomerBase.Email), c => c.WithOperators(QueryFilterOperator.EqualityOperators).AsUpperCase()) + .AddField(nameof(Contracts.CustomerBase.Email), c => c.WithOperators(QueryFilterOperator.EqualityOperators | QueryFilterOperator.StringFunctions).AsUpperCase()) .AddReferenceDataField(nameof(Contracts.CustomerBase.CustomerType), "CustomerTypeCode")); WithOrderBy(orderby => orderby diff --git a/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomersCosmosDb.cs b/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomersCosmosDb.cs index 4d358680a..d52fc6546 100644 --- a/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomersCosmosDb.cs +++ b/samples/src/Contoso.Customers.Infrastructure/Repositories/CustomersCosmosDb.cs @@ -5,7 +5,7 @@ public class CustomersCosmosDb(CosmosClient client, string databaseId) : CosmosD private const string RefDataContainerId = "ref-data"; private const string CustomersContainerId = "customers"; - private static readonly CosmosDbOptions _options = new(); + private static readonly CosmosDbOptions _options = new CosmosDbOptions().Container(RefDataContainerId, c => c.WithReferenceDataOutboxEvent()); public CosmosDbContainer ContactMethods => Container(RefDataContainerId, o => o.WithTypeDiscriminator()); diff --git a/samples/src/Contoso.Customers.Infrastructure/Repositories/ReferenceDataRepository.g.cs b/samples/src/Contoso.Customers.Infrastructure/Repositories/ReferenceDataRepository.g.cs index 4b39ffe13..11693f5af 100644 --- a/samples/src/Contoso.Customers.Infrastructure/Repositories/ReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Customers.Infrastructure/Repositories/ReferenceDataRepository.g.cs @@ -14,11 +14,39 @@ public partial class ReferenceDataRepository : IReferenceDataRepository { /// public Task GetAllCustomerTypesAsync(CancellationToken cancellationToken = default) - => _cosmos.CustomerTypes.Query().ToMappedItemsAsync(CustomerTypeMapper.Map, cancellationToken); + => _cosmos.CustomerTypes.Query().ToMappedItemsAsync(CustomerTypeMapper.From, cancellationToken); /// public Task GetAllContactMethodsAsync(CancellationToken cancellationToken = default) - => _cosmos.ContactMethods.Query().ToMappedItemsAsync(ContactMethodMapper.Map, cancellationToken); + => _cosmos.ContactMethods.Query().ToMappedItemsAsync(ContactMethodMapper.From, cancellationToken); + + #region CustomerType + + /// + public Task> GetCustomerTypeAsync(string id, CancellationToken cancellationToken = default) + => _cosmos.ThrowIfNull().CustomerTypes.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ThenAs(model => CustomerTypeMapper.From.Map(model)); + + /// + public Task>> CreateCustomerTypeAsync(Contracts.CustomerType value, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.CreateAsync(_cosmos.ThrowIfNull().CustomerTypes, value, CustomerTypeMapper.Default, cancellationToken); + + /// + public Task>> UpdateCustomerTypeAsync(string id, Contracts.CustomerType value, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.UpdateAsync(_cosmos.ThrowIfNull().CustomerTypes, id, value, CustomerTypeMapper.Default, cancellationToken); + + /// + public Task>> ActivateCustomerTypeAsync(string id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.ActivateAsync(_cosmos.ThrowIfNull().CustomerTypes, id, CustomerTypeMapper.Default, cancellationToken); + + /// + public Task>> DeactivateCustomerTypeAsync(string id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.DeactivateAsync(_cosmos.ThrowIfNull().CustomerTypes, id, CustomerTypeMapper.Default, cancellationToken); + + /// + public Task> DeleteCustomerTypeAsync(string id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.DeleteAsync(_cosmos.ThrowIfNull().CustomerTypes, id, cancellationToken); + + #endregion } #nullable restore diff --git a/samples/src/Contoso.Customers.Relay/Contoso.Customers.Relay.csproj b/samples/src/Contoso.Customers.Relay/Contoso.Customers.Relay.csproj new file mode 100644 index 000000000..a1b58ca58 --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/Contoso.Customers.Relay.csproj @@ -0,0 +1,14 @@ + + + + + + + + + + + + + + diff --git a/samples/src/Contoso.Customers.Relay/CosmosClientInstrumentationExtensions.cs b/samples/src/Contoso.Customers.Relay/CosmosClientInstrumentationExtensions.cs new file mode 100644 index 000000000..7cb833737 --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/CosmosClientInstrumentationExtensions.cs @@ -0,0 +1,28 @@ +namespace Contoso.Customers.Relay; + +/// +/// Configures instrumentation for the relay's shared Cosmos client. +/// +public static class CosmosClientInstrumentationExtensions +{ + /// + /// Suppresses SDK background work started during client construction without suppressing subsequent foreground operations. + /// + /// The services containing Aspire's singleton Cosmos client factory. + public static void SuppressCosmosClientBackgroundInstrumentation(this IServiceCollection services) + { + var registration = services.LastOrDefault(s => s.ServiceType == typeof(CosmosClient) && !s.IsKeyedService) + ?? throw new InvalidOperationException("Register the Cosmos client before configuring background instrumentation."); + var factory = registration.ImplementationFactory + ?? throw new InvalidOperationException("The Cosmos client must be registered using a factory before configuring background instrumentation."); + if (registration.Lifetime != ServiceLifetime.Singleton) + throw new InvalidOperationException("The Cosmos client must be registered as a singleton."); + + // Account refresh starts at client construction, before the change-feed processor's suppression scope. + services[services.IndexOf(registration)] = ServiceDescriptor.Singleton(sp => + { + using var instrumentation = SuppressInstrumentationScope.Begin(); + return (CosmosClient)factory(sp); + }); + } +} diff --git a/samples/src/Contoso.Customers.Relay/GlobalUsing.cs b/samples/src/Contoso.Customers.Relay/GlobalUsing.cs new file mode 100644 index 000000000..4e0d8cd2a --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/GlobalUsing.cs @@ -0,0 +1,7 @@ +global using CoreEx; +global using CoreEx.Azure.Messaging.ServiceBus; +global using Microsoft.Azure.Cosmos; +global using Microsoft.Extensions.DependencyInjection; +global using OpenTelemetry; +global using OpenTelemetry.Trace; +global using System.Text.Json; diff --git a/samples/src/Contoso.Customers.Relay/Program.cs b/samples/src/Contoso.Customers.Relay/Program.cs new file mode 100644 index 000000000..6ac2ad5aa --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/Program.cs @@ -0,0 +1,65 @@ +namespace Contoso.Customers.Relay; + +/// +/// Relays the Customers domain's committed Cosmos outbox events to Azure Service Bus. +/// +public class Program +{ + /// + /// Configures and runs the relay host. + /// + private static void Main(string[] args) + { + // Use the live host clock, not an ambient request's frozen Runtime timestamp; existing checkpoints still take precedence. + var startTime = TimeProvider.System.GetUtcNow().UtcDateTime; + var builder = WebApplication.CreateBuilder(args); + builder.AddHostSettings(); + + builder.Services + .AddPrecisionTimeProvider() + .AddExecutionContext() + .AddMvcWebApi() + .AddHttpWebApi() + .AddHostedServiceManager(); + + builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => + { + o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; + if (builder.Environment.IsDevelopment()) + { + o.ConnectionMode = ConnectionMode.Gateway; + o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); + } + }); + builder.Services.SuppressCosmosClientBackgroundInstrumentation(); + + builder.Services.AddCosmosDb("contoso").AddCosmosDbHealthCheck(); + foreach (var containerId in new[] { "customers", "ref-data" }) + { + builder.AddCosmosDbOutboxRelayHostedService(containerId, configureOptions: (_, o) => + { + o.StartTime = startTime; + o.PollInterval = builder.Configuration.GetValue($"CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:PollInterval"); + o.BatchSize = builder.Configuration.GetValue($"CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:BatchSize"); + }); + } + + builder.AddAzureServiceBusClient("ServiceBus"); + builder.Services.AddAzureServiceBusPublisher((_, o) => o.SessionIdStrategy = ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId); + builder.Services.PostConfigureAllHealthChecks(); + +#if NET8_0 + // The combined telemetry patterns exceed .NET 8's default regex automata cap. + AppContext.SetData("REGEX_NONBACKTRACKING_MAX_AUTOMATA_SIZE", 10_000); +#endif + builder.WithCoreExTelemetry().WithCoreExCosmosDbTelemetry().WithCoreExServiceBusTelemetry().UseOtlpExporter(); + + var app = builder.Build(); + app.UseCoreExExceptionHandler(); + app.UseHttpsRedirection(); + app.UseExecutionContext(); + app.MapHealthChecks(); + app.MapHostedServices(/* groupConfigure: g => g.RequireAuthorization() */); + app.Run(); + } +} diff --git a/samples/src/Contoso.Customers.Relay/Properties/launchSettings.json b/samples/src/Contoso.Customers.Relay/Properties/launchSettings.json new file mode 100644 index 000000000..5144b8c97 --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/Properties/launchSettings.json @@ -0,0 +1,23 @@ +{ + "$schema": "https://json.schemastore.org/launchsettings.json", + "profiles": { + "http": { + "commandName": "Project", + "dotnetRunMessages": true, + "launchBrowser": false, + "applicationUrl": "http://localhost:5321", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development" + } + }, + "https": { + "commandName": "Project", + "dotnetRunMessages": true, + "launchBrowser": false, + "applicationUrl": "https://localhost:7321;http://localhost:5321", + "environmentVariables": { + "ASPNETCORE_ENVIRONMENT": "Development" + } + } + } +} diff --git a/samples/src/Contoso.Customers.Relay/appsettings.Development.json b/samples/src/Contoso.Customers.Relay/appsettings.Development.json new file mode 100644 index 000000000..f62cc6c1a --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/appsettings.Development.json @@ -0,0 +1,13 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Azure": "Warning", + "Microsoft": "Warning" + } + }, + "ConnectionStrings": { + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==", + "ServiceBus": "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;" + } +} diff --git a/samples/src/Contoso.Customers.Relay/appsettings.json b/samples/src/Contoso.Customers.Relay/appsettings.json new file mode 100644 index 000000000..b4f86e75a --- /dev/null +++ b/samples/src/Contoso.Customers.Relay/appsettings.json @@ -0,0 +1,32 @@ +{ + "CoreEx": { + "Host": { + "SolutionName": "Contoso", + "DomainName": "Customers", + "Services": { + "CosmosOutboxRelay": { + "customers": { + "ServicesCount": 2, + "PollInterval": "00:00:01", + "BatchSize": 10 + }, + "ref-data": { + "ServicesCount": 2, + "PollInterval": "00:00:01", + "BatchSize": 10 + } + } + }, + "ServiceBus": { + "SessionIdPartitionSize": 4 + } + } + }, + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning" + } + }, + "AllowedHosts": "*" +} diff --git a/samples/src/Contoso.Orders.Api/Program.cs b/samples/src/Contoso.Orders.Api/Program.cs index 141373c6a..91ed52d53 100644 --- a/samples/src/Contoso.Orders.Api/Program.cs +++ b/samples/src/Contoso.Orders.Api/Program.cs @@ -46,6 +46,7 @@ private static void Main(string[] args) .AddSqlServerDatabase() .AddSqlServerUnitOfWork() .AddEventFormatter() + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddSqlServerOutboxPublisher() .AddDbContext() .AddEfDb(); diff --git a/samples/src/Contoso.Orders.Api/Properties/launchSettings.json b/samples/src/Contoso.Orders.Api/Properties/launchSettings.json index 0898c92e9..805a1e348 100644 --- a/samples/src/Contoso.Orders.Api/Properties/launchSettings.json +++ b/samples/src/Contoso.Orders.Api/Properties/launchSettings.json @@ -2,12 +2,12 @@ "profiles": { "Contoso.Orders.Api": { "commandName": "Project", - "launchUrl": "https://localhost:62023/swagger", + "launchUrl": "https://localhost:7330/swagger", "environmentVariables": { "ASPNETCORE_ENVIRONMENT": "Development" }, "dotnetRunMessages": true, - "applicationUrl": "https://localhost:62023;http://localhost:62024" + "applicationUrl": "https://localhost:7330;http://localhost:5330" } } } \ No newline at end of file diff --git a/samples/src/Contoso.Orders.Contracts/OrderBase.cs b/samples/src/Contoso.Orders.Contracts/OrderBase.cs index 98ec1a6cf..7f57a9570 100644 --- a/samples/src/Contoso.Orders.Contracts/OrderBase.cs +++ b/samples/src/Contoso.Orders.Contracts/OrderBase.cs @@ -6,10 +6,12 @@ public abstract partial class OrderBase : IIdentifier [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? CustomerId { get; set; } [ReferenceData] [Localization("Order status")] + [NonNullable] public partial string? StatusCode { get; set; } public List? Items { get; set; } diff --git a/samples/src/Contoso.Orders.Contracts/OrderItem.cs b/samples/src/Contoso.Orders.Contracts/OrderItem.cs index c835fc758..7df7a69cc 100644 --- a/samples/src/Contoso.Orders.Contracts/OrderItem.cs +++ b/samples/src/Contoso.Orders.Contracts/OrderItem.cs @@ -3,8 +3,10 @@ namespace Contoso.Orders.Contracts; [Contract] public partial class OrderItem : IIdentifier { + [NonNullable] public string? Id { get; set; } + [NonNullable] public string? ProductId { get; set; } public decimal Quantity { get; set; } diff --git a/samples/src/Contoso.Orders.Contracts/OrderLite.cs b/samples/src/Contoso.Orders.Contracts/OrderLite.cs index ee1bdca35..ab774440b 100644 --- a/samples/src/Contoso.Orders.Contracts/OrderLite.cs +++ b/samples/src/Contoso.Orders.Contracts/OrderLite.cs @@ -6,10 +6,12 @@ public partial class OrderLite : IIdentifier [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? CustomerId { get; set; } [ReferenceData] [Localization("Order status")] + [NonNullable] public partial string? StatusCode { get; set; } [ReadOnly(true)] diff --git a/samples/src/Contoso.Orders.Infrastructure/Repositories/OrdersDbContext.g.cs b/samples/src/Contoso.Orders.Infrastructure/Repositories/OrdersDbContext.g.cs index 8c856553a..bcf6c6afe 100644 --- a/samples/src/Contoso.Orders.Infrastructure/Repositories/OrdersDbContext.g.cs +++ b/samples/src/Contoso.Orders.Infrastructure/Repositories/OrdersDbContext.g.cs @@ -24,10 +24,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("Text").HasColumnType("NVARCHAR(250)"); e.Property(p => p.IsActive).HasColumnName("IsActive").HasColumnType("BIT"); e.Property(p => p.SortOrder).HasColumnName("SortOrder").HasColumnType("INT"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -40,10 +40,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Id).HasColumnName("OrderId").HasColumnType("NVARCHAR(50)"); e.Property(p => p.CustomerId).HasColumnName("CustomerId").HasColumnType("NVARCHAR(100)"); e.Property(p => p.StatusCode).HasColumnName("StatusCode").HasColumnType("NVARCHAR(50)"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); }); @@ -57,13 +57,13 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.ProductId).HasColumnName("ProductId").HasColumnType("NVARCHAR(100)"); e.Property(p => p.Quantity).HasColumnName("Quantity").HasColumnType("DECIMAL(18, 4)"); e.Property(p => p.UnitPrice).HasColumnName("UnitPrice").HasColumnType("DECIMAL(18, 4)"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Ignore(p => p.ETag); }); } } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Api/Controllers/BrandController.g.cs b/samples/src/Contoso.Products.Api/Controllers/BrandController.g.cs new file mode 100644 index 000000000..b053dce2c --- /dev/null +++ b/samples/src/Contoso.Products.Api/Controllers/BrandController.g.cs @@ -0,0 +1,71 @@ + + +// + +/* + * This file is automatically generated by 'Contoso.Products.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Products.Api.Controllers; + +/// Represents the reference-data controller. +[ApiController, Route("/api/refdata/brands")] +[AllowAnonymous] +public partial class BrandController(CoreEx.AspNetCore.Mvc.WebApi webApi, IReferenceDataService service) : ControllerBase +{ + private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); + private readonly IReferenceDataService _service = service.ThrowIfNull(); + + [HttpGet("{id}"), HttpHead("{id}")] + [ProducesResponseType(typeof(Brand), 200)] + [ProducesNotFoundProblem()] + public Task GetAsync(string id, CancellationToken cancellationToken = default) + => _webApi.GetWithResultAsync(Request, (_, ct) => _service.GetBrandAsync(id.Required(), ct), cancellationToken: cancellationToken); + + [HttpPost] + [Accepts] + [ProducesResponseType(201)] + [IdempotencyKey] + public Task CreateAsync(CancellationToken cancellationToken = default) => _webApi.PostWithResultAsync(Request, (ro, ct) => + { + ro.WithLocationUri(v => new Uri($"/api/refdata/brands/{v.Id}", UriKind.Relative)); + return _service.CreateBrandAsync(ro.Value, ct); + }, cancellationToken: cancellationToken); + + [HttpPatch("{id}")] + [Accepts(HttpNames.MergePatchJsonMediaTypeName)] + [ProducesResponseType(typeof(Brand), 200)] + [ProducesNotFoundProblem()] + public Task PatchAsync(string id, CancellationToken cancellationToken = default) => _webApi.PatchWithResultAsync(Request, + get: (ro, ct) => _service.GetBrandAsync(id.Required(), ct), + put: (ro, ct) => _service.UpdateBrandAsync(id, ro.Value, ct), + cancellationToken: cancellationToken); + + [HttpPut("{id}")] + [Accepts] + [ProducesResponseType(typeof(Brand), 200)] + [ProducesNotFoundProblem()] + public Task PutAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PutWithResultAsync(Request, (ro, ct) => _service.UpdateBrandAsync(id.Required(), ro.Value, ct), cancellationToken: cancellationToken); + + [HttpPost("{id}/activate")] + [ProducesResponseType(typeof(Brand), 200)] + [ProducesNotFoundProblem()] + public Task ActivateAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync(Request, (_, ct) => _service.ActivateBrandAsync(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpPost("{id}/deactivate")] + [ProducesResponseType(typeof(Brand), 200)] + [ProducesNotFoundProblem()] + public Task DeactivateAsync(string id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync(Request, (_, ct) => _service.DeactivateBrandAsync(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpDelete("{id}")] + [ProducesResponseType(204)] + public Task DeleteAsync(string id, CancellationToken cancellationToken = default) + => _webApi.DeleteWithResultAsync(Request, (_, ct) => _service.DeleteBrandAsync(id.Required(), ct), cancellationToken: cancellationToken); +} + +#nullable restore diff --git a/samples/src/Contoso.Products.Api/Controllers/ProductController.cs b/samples/src/Contoso.Products.Api/Controllers/ProductController.cs index 119972dbc..31ecfc536 100644 --- a/samples/src/Contoso.Products.Api/Controllers/ProductController.cs +++ b/samples/src/Contoso.Products.Api/Controllers/ProductController.cs @@ -36,4 +36,9 @@ public Task PatchAsync(string id, CancellationToken cancellationT [ProducesResponseType(204)] public Task DeleteAsync(string id, CancellationToken cancellationToken = default) => _webApi.DeleteAsync(Request, (_, ct) => _service.DeleteAsync(id.Required(), ct), cancellationToken: cancellationToken); -} \ No newline at end of file + + [HttpPost("{id}/activate")] + [ProducesResponseType(typeof(Product), 200)] + public Task ActivateAsync(string id, CancellationToken cancellationToken = default) => _webApi.PostAsync(Request, (_, ct) + => _service.ActivateAsync(id.Required(), ct), HttpStatusCode.OK, cancellationToken: cancellationToken); +} diff --git a/samples/src/Contoso.Products.Api/Controllers/ReferenceDataController.g.cs b/samples/src/Contoso.Products.Api/Controllers/ReferenceDataController.g.cs index 375e880f0..dd85352cf 100644 --- a/samples/src/Contoso.Products.Api/Controllers/ReferenceDataController.g.cs +++ b/samples/src/Contoso.Products.Api/Controllers/ReferenceDataController.g.cs @@ -14,37 +14,37 @@ public partial class ReferenceDataController(CoreEx.AspNetCore.Mvc.WebApi webApi { private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); - [HttpGet("brands"), HttpHead("brands")] + [HttpGet("brands")] [ProducesResponseType(typeof(Brand[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetBrandsAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("categories"), HttpHead("categories")] + [HttpGet("categories")] [ProducesResponseType(typeof(Category[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetCategoriesAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("movement-kinds"), HttpHead("movement-kinds")] + [HttpGet("movement-kinds")] [ProducesResponseType(typeof(MovementKind[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetMovementKindsAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("movement-statuses"), HttpHead("movement-statuses")] + [HttpGet("movement-statuses")] [ProducesResponseType(typeof(MovementStatus[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetMovementStatusesAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("sub-categories"), HttpHead("sub-categories")] + [HttpGet("sub-categories")] [ProducesResponseType(typeof(SubCategory[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetSubCategoriesAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("units-of-measure"), HttpHead("units-of-measure")] + [HttpGet("units-of-measure")] [ProducesResponseType(typeof(UnitOfMeasure[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetUnitsOfMeasureAsync(CancellationToken cancellationToken) diff --git a/samples/src/Contoso.Products.Api/GlobalUsing.cs b/samples/src/Contoso.Products.Api/GlobalUsing.cs index 4abd099c2..551e5bf37 100644 --- a/samples/src/Contoso.Products.Api/GlobalUsing.cs +++ b/samples/src/Contoso.Products.Api/GlobalUsing.cs @@ -11,7 +11,8 @@ global using CoreEx.Json; global using CoreEx.RefData; global using CoreEx.Validation; +global using Microsoft.AspNetCore.Authorization; global using Microsoft.AspNetCore.Mvc; global using NSwag.Annotations; global using System.Net; -global using System.Text.Json; \ No newline at end of file +global using System.Text.Json; diff --git a/samples/src/Contoso.Products.Api/Program.cs b/samples/src/Contoso.Products.Api/Program.cs index bda4a8140..923cb436e 100644 --- a/samples/src/Contoso.Products.Api/Program.cs +++ b/samples/src/Contoso.Products.Api/Program.cs @@ -52,6 +52,7 @@ private static void Main(string[] args) .AddPostgresDatabase() // Adds the PostgresDatabase. .AddPostgresUnitOfWork() // Adds the PostgresUnitOfWork for the PostgresDatabase. .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddPostgresOutboxPublisher() // Adds the PostgresOutboxPublisher/IEventPublisher. .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. diff --git a/samples/src/Contoso.Products.Api/appsettings.json b/samples/src/Contoso.Products.Api/appsettings.json index 7052c07ab..eccaba3d1 100644 --- a/samples/src/Contoso.Products.Api/appsettings.json +++ b/samples/src/Contoso.Products.Api/appsettings.json @@ -2,7 +2,10 @@ "CoreEx": { "Host": { "SolutionName": "Contoso", - "DomainName": "Products" + "DomainName": "Products", + "Outbox": { + "PartitionSize": 4 + } }, "Events": { "Destination": "contoso" // Topic/queue name. @@ -17,4 +20,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Products.Application/Interfaces/IReferenceDataService.g.cs b/samples/src/Contoso.Products.Application/Interfaces/IReferenceDataService.g.cs new file mode 100644 index 000000000..57e396a73 --- /dev/null +++ b/samples/src/Contoso.Products.Application/Interfaces/IReferenceDataService.g.cs @@ -0,0 +1,37 @@ +// + +/* + * This file is automatically generated by 'Contoso.Products.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Products.Application; + +/// Enables the mutable services. +public partial interface IReferenceDataService +{ + #region Brand + + /// Gets the specified . + Task> GetBrandAsync(string id, CancellationToken cancellationToken); + + /// Creates the specified . + Task> CreateBrandAsync(Brand value, CancellationToken cancellationToken); + + /// Updates the specified . + Task> UpdateBrandAsync(string id, Brand value, CancellationToken cancellationToken); + + /// Activates the specified . + Task> ActivateBrandAsync(string id, CancellationToken cancellationToken); + + /// Deactivates the specified . + Task> DeactivateBrandAsync(string id, CancellationToken cancellationToken); + + /// Deletes the specified . + Task DeleteBrandAsync(string id, CancellationToken cancellationToken); + + #endregion +} + +#nullable restore diff --git a/samples/src/Contoso.Products.Application/ReferenceDataProvider.g.cs b/samples/src/Contoso.Products.Application/ReferenceDataProvider.g.cs new file mode 100644 index 000000000..798f54002 --- /dev/null +++ b/samples/src/Contoso.Products.Application/ReferenceDataProvider.g.cs @@ -0,0 +1,52 @@ +// + +/* + * This file is automatically generated by 'Contoso.Products.CodeGen'; any changes will be lost. + */ + +#nullable enable + +namespace Contoso.Products.Application; + +/// Provides the implementation. +[ScopedService] +public partial class ReferenceDataProvider(IReferenceDataRepository repository) : IReferenceDataProvider +{ + private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + + /// + public IEnumerable<(Type, Type)> Types => + [ + (typeof(Brand), typeof(BrandCollection)), + (typeof(Category), typeof(CategoryCollection)), + (typeof(MovementKind), typeof(MovementKindCollection)), + (typeof(MovementStatus), typeof(MovementStatusCollection)), + (typeof(SubCategory), typeof(SubCategoryCollection)), + (typeof(UnitOfMeasure), typeof(UnitOfMeasureCollection)), + ]; + + /// + public IEnumerable<(string, Type)> AlternateNames => + [ + ( "brands", typeof(Brand) ), + ( "categories", typeof(Category) ), + ( "movement-kinds", typeof(MovementKind) ), + ( "movement-statuses", typeof(MovementStatus) ), + ( "sub-categories", typeof(SubCategory) ), + ( "units-of-measure", typeof(UnitOfMeasure) ) + ]; + + /// + public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch + { + _ when type == typeof(Brand) => await _repository.GetAllBrandsAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(Category) => await _repository.GetAllCategoriesAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(MovementKind) => await _repository.GetAllMovementKindsAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(MovementStatus) => await _repository.GetAllMovementStatusesAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(SubCategory) => await _repository.GetAllSubCategoriesAsync(cancellationToken).ConfigureAwait(false), + _ when type == typeof(UnitOfMeasure) => await _repository.GetAllUnitsOfMeasureAsync(cancellationToken).ConfigureAwait(false), + _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") + }; +} + +#nullable restore diff --git a/samples/src/Contoso.Products.Application/ReferenceDataService.cs b/samples/src/Contoso.Products.Application/ReferenceDataService.cs new file mode 100644 index 000000000..fd7fd500d --- /dev/null +++ b/samples/src/Contoso.Products.Application/ReferenceDataService.cs @@ -0,0 +1,12 @@ +namespace Contoso.Products.Application; + +public partial class ReferenceDataService +{ + partial void OnInitialization() => PreCheckAsync = (value, action, cancellationToken) => + { + if (value is Brand b && b.Code == "YETI" && action == EventAction.Deactivated) + return Result.BusinessError("YETI brand cannot be deactivated as it is awesome.", c => c.WithErrorCode("yeti-cannot-be-deactivated")).AsTask(); + + return Result.SuccessTask; + }; +} diff --git a/samples/src/Contoso.Products.Application/ReferenceDataService.g.cs b/samples/src/Contoso.Products.Application/ReferenceDataService.g.cs index d62866ea2..c611edaf8 100644 --- a/samples/src/Contoso.Products.Application/ReferenceDataService.g.cs +++ b/samples/src/Contoso.Products.Application/ReferenceDataService.g.cs @@ -8,45 +8,212 @@ namespace Contoso.Products.Application; -/// Provides the implementation. -[ScopedService] -public partial class ReferenceDataService(IReferenceDataRepository repository) : IReferenceDataProvider +/// Provides the mutable implementation. +[ScopedService] +public partial class ReferenceDataService : IReferenceDataService { - private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + private readonly IUnitOfWork _unitOfWork; + private readonly IReferenceDataRepository _repository; + + /// + /// Initializes a new instance of the class. + /// + /// The . + /// The . + public ReferenceDataService(IUnitOfWork unitOfWork, IReferenceDataRepository repository) + { + _unitOfWork = unitOfWork.ThrowIfNull(); + _repository = repository.ThrowIfNull(); + OnInitialization(); + } + + /// + /// Provides the with an opportunity to perform additional initialization. + /// + partial void OnInitialization(); + + /// + /// Gets or sets the pre-check function that is invoked prior to the activate, deactivate or delete. + /// + /// This is intended to be used for any additional validation or business rules that need to be applied prior to the action being performed. + /// The passed is the reference data entity being mutated and the indicates the resulting action (being one of: + /// , , or ). + private Func> PreCheckAsync { get; set => field = value.ThrowIfNull(); } = (_, _, _) => Result.SuccessTask; + + #region Brand + + /// + public Task> GetBrandAsync(string id, CancellationToken cancellationToken) => GetAsync(id, _repository.GetBrandAsync, cancellationToken); + + /// + public Task> CreateBrandAsync(Brand value, CancellationToken cancellationToken) => CreateAsync>(value, _repository.CreateBrandAsync, cancellationToken); /// - public IEnumerable<(Type, Type)> Types => - [ - (typeof(Brand), typeof(BrandCollection)), - (typeof(Category), typeof(CategoryCollection)), - (typeof(MovementKind), typeof(MovementKindCollection)), - (typeof(MovementStatus), typeof(MovementStatusCollection)), - (typeof(SubCategory), typeof(SubCategoryCollection)), - (typeof(UnitOfMeasure), typeof(UnitOfMeasureCollection)), - ]; + public Task> UpdateBrandAsync(string id, Brand value, CancellationToken cancellationToken) => UpdateAsync>(id, value, _repository.UpdateBrandAsync, cancellationToken); /// - public IEnumerable<(string, Type)> AlternateNames => - [ - ( "brands", typeof(Brand) ), - ( "categories", typeof(Category) ), - ( "movement-kinds", typeof(MovementKind) ), - ( "movement-statuses", typeof(MovementStatus) ), - ( "sub-categories", typeof(SubCategory) ), - ( "units-of-measure", typeof(UnitOfMeasure) ) - ]; + public Task> ActivateBrandAsync(string id, CancellationToken cancellationToken) => ActivateAsync(id, _repository.GetBrandAsync, _repository.ActivateBrandAsync, cancellationToken); /// - public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch + public Task> DeactivateBrandAsync(string id, CancellationToken cancellationToken) => DeactivateAsync(id, _repository.GetBrandAsync, _repository.DeactivateBrandAsync, cancellationToken); + + /// + public Task DeleteBrandAsync(string id, CancellationToken cancellationToken) => DeleteAsync(id, _repository.GetBrandAsync, _repository.DeleteBrandAsync, cancellationToken); + + #endregion + + #region Internal + + /// + /// Gets the reference data of type by the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The cancellation token. + /// The result. + private Task> GetAsync(TId id, Func>> getFunc, CancellationToken cancellationToken) + where TRef : class, IReferenceData + => Result.GoAsync(() => getFunc(id, cancellationToken)); + + /// + /// Creates the reference data of type using the specified . + /// + /// The reference data . + /// The validator . + /// The reference data value. + /// The function to create the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> CreateAsync(TRef value, Func>>> create, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => create(value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Created)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Updates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The validator . + /// The identifier. + /// The reference data value. + /// The function to update the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> UpdateAsync(TId id, TRef value, Func>>> update, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => update(id, value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Updated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Activates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to activate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> ActivateAsync(TId id, Func>> get, Func>>> activate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Activated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => activate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Activated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Deactivates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to deactivate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> DeactivateAsync(TId id, Func>> get, Func>>> deactivate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Deactivated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => deactivate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Deactivated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Re-gets the mutated reference data using the function (where specified) so that the final persisted is returned. + /// + /// This occurs after the mutation has been committed, so it is deliberately not cancellable and a refresh failure is swallowed; i.e. it must never turn a committed success into an error, which would encourage a retry of a completed operation (the ETag is then simply as per the mutation). + private static async Task>> RefreshAsync(DataResult dr, Func>>? refresh) + where TRef : class, IReferenceData + { + if (refresh is null || !dr.WasMutated) + return dr; + + try + { + var r = await refresh(dr.Value, CancellationToken.None).ConfigureAwait(false); + return r.IsSuccess ? new DataResult(r.Value, true) : dr; + } + catch (Exception) + { + return dr; + } + } + /// + /// Deletes the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to delete the reference data. + /// The cancellation token. + /// The result. + private async Task DeleteAsync(TId id, Func>> get, Func>> delete, CancellationToken cancellationToken) + where TRef : class, IReferenceData { - _ when type == typeof(Brand) => await _repository.GetAllBrandsAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(Category) => await _repository.GetAllCategoriesAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(MovementKind) => await _repository.GetAllMovementKindsAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(MovementStatus) => await _repository.GetAllMovementStatusesAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(SubCategory) => await _repository.GetAllSubCategoriesAsync(cancellationToken).ConfigureAwait(false), - _ when type == typeof(UnitOfMeasure) => await _repository.GetAllUnitsOfMeasureAsync(cancellationToken).ConfigureAwait(false), - _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") - }; + var r = await get(id, cancellationToken).ConfigureAwait(false); + if (r.IsFailure) + return r.IsNotFoundError ? Result.Success : r.AsResult(); // Delete is idempotent, so if not found then return success. + + return await Result.GoAsync(() => PreCheckAsync(r.Value, EventAction.Deleted, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => delete(id, ct)) + .Then(dr => dr.WhereMutatedAnd(() => _unitOfWork.Events.Add(EventData.CreateEvent(EventAction.Deleted).WithKey(CompositeKey.Create(id))))); + }, cancellationToken)) + .ThenAsAsync(dr => dr.WhereMutatedAsync(ct => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)).ConfigureAwait(false); + } + + #endregion } #nullable restore diff --git a/samples/src/Contoso.Products.Application/Repositories/IReferenceDataRepository.g.cs b/samples/src/Contoso.Products.Application/Repositories/IReferenceDataRepository.g.cs index d0021b38b..384f0ed0a 100644 --- a/samples/src/Contoso.Products.Application/Repositories/IReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Products.Application/Repositories/IReferenceDataRepository.g.cs @@ -12,28 +12,44 @@ namespace Contoso.Products.Application.Repositories; public partial interface IReferenceDataRepository { /// Gets all items. - /// The . Task GetAllBrandsAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllCategoriesAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllMovementKindsAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllMovementStatusesAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllSubCategoriesAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllUnitsOfMeasureAsync(CancellationToken cancellationToken = default); + + #region Brand + + /// Gets the specified . + Task> GetBrandAsync(string id, CancellationToken cancellationToken = default); + + /// Creates the . + Task>> CreateBrandAsync(Contracts.Brand value, CancellationToken cancellationToken = default); + + /// Updates the . + Task>> UpdateBrandAsync(string id, Contracts.Brand value, CancellationToken cancellationToken = default); + + /// Activates the . + Task>> ActivateBrandAsync(string id, CancellationToken cancellationToken = default); + + /// Deactivates the . + Task>> DeactivateBrandAsync(string id, CancellationToken cancellationToken = default); + + /// Deletes the specified . + Task> DeleteBrandAsync(string id, CancellationToken cancellationToken = default); + + #endregion } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.CodeGen/ref-data.yaml b/samples/src/Contoso.Products.CodeGen/ref-data.yaml index 0da59dc8f..32d7c9efc 100644 --- a/samples/src/Contoso.Products.CodeGen/ref-data.yaml +++ b/samples/src/Contoso.Products.CodeGen/ref-data.yaml @@ -1,8 +1,11 @@ # yaml-language-server: $schema=https://raw.githubusercontent.com/Avanade/CoreEx/refs/heads/main/schema/coreex-refdata.json collectionSortOrder: Code repository: EntityFramework +getNamed: true entities: - name: Brand + mutability: CreateUpdateDelete + mutableAttribute: '[AllowAnonymous]' # This sample API has no authentication; production APIs should retain the default [Authorize]. - name: Category - name: MovementKind - name: MovementStatus diff --git a/samples/src/Contoso.Products.Contracts/Movement.cs b/samples/src/Contoso.Products.Contracts/Movement.cs index 90f77c44a..2e8f5bb1f 100644 --- a/samples/src/Contoso.Products.Contracts/Movement.cs +++ b/samples/src/Contoso.Products.Contracts/Movement.cs @@ -6,19 +6,24 @@ public partial class Movement : IIdentifier, IETag, IChangeLog [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? ReferenceId { get; set; } [ReferenceData] + [NonNullable] public partial string? KindCode { get; set; } [ReferenceData] + [NonNullable] public partial string? StatusCode { get; set; } + [NonNullable] public string? ProductId { get; set; } public decimal Quantity { get; set; } [ReferenceData] + [NonNullable] public partial string? UnitOfMeasureCode { get; set; } [ReadOnly(true)] diff --git a/samples/src/Contoso.Products.Contracts/MovementRequest.cs b/samples/src/Contoso.Products.Contracts/MovementRequest.cs index 56732a39a..21165cb62 100644 --- a/samples/src/Contoso.Products.Contracts/MovementRequest.cs +++ b/samples/src/Contoso.Products.Contracts/MovementRequest.cs @@ -3,7 +3,9 @@ namespace Contoso.Products.Contracts; [Contract] public partial class MovementRequest : IIdentifier { + [NonNullable] public string? Id { get; set; } + [NonNullable] public DataMap? Products { get; set; } } \ No newline at end of file diff --git a/samples/src/Contoso.Products.Contracts/MovementRequestProduct.cs b/samples/src/Contoso.Products.Contracts/MovementRequestProduct.cs index 500298bcb..23a43540f 100644 --- a/samples/src/Contoso.Products.Contracts/MovementRequestProduct.cs +++ b/samples/src/Contoso.Products.Contracts/MovementRequestProduct.cs @@ -8,5 +8,6 @@ public partial class MovementRequestProduct [ReferenceData] [Localization("Unit-of-measure")] + [NonNullable] public partial string? UnitOfMeasureCode { get; set; } } \ No newline at end of file diff --git a/samples/src/Contoso.Products.Contracts/ProductBase.cs b/samples/src/Contoso.Products.Contracts/ProductBase.cs index 65ab1a73a..6f5e50665 100644 --- a/samples/src/Contoso.Products.Contracts/ProductBase.cs +++ b/samples/src/Contoso.Products.Contracts/ProductBase.cs @@ -6,8 +6,10 @@ public abstract partial class ProductBase : IIdentifier [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? Sku { get; set => field = value?.ToUpper(); } + [NonNullable] public string? Text { get; set; } [ReadOnly(true)] @@ -16,10 +18,12 @@ public abstract partial class ProductBase : IIdentifier [ReferenceData] [Localization("Sub-category")] + [NonNullable] public partial string? SubCategoryCode { get; set; } [ReferenceData()] [Localization("Unit-of-measure")] + [NonNullable] public partial string? UnitOfMeasureCode { get; set; } [ReferenceData()] diff --git a/samples/src/Contoso.Products.Infrastructure/GlobalUsing.cs b/samples/src/Contoso.Products.Infrastructure/GlobalUsing.cs index 76e2b971b..3ac6741e7 100644 --- a/samples/src/Contoso.Products.Infrastructure/GlobalUsing.cs +++ b/samples/src/Contoso.Products.Infrastructure/GlobalUsing.cs @@ -14,6 +14,8 @@ global using CoreEx.Events; global using CoreEx.Events.Publishing; global using CoreEx.Mapping; +global using CoreEx.Results; global using Microsoft.EntityFrameworkCore; global using Microsoft.Extensions.Logging; -global using System.Text.Json; \ No newline at end of file +global using System.Text.Json; +global using ExecutionContext = CoreEx.ExecutionContext; diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/BrandMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/BrandMapper.g.cs index c9061b513..d806bbf11 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/BrandMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/BrandMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class BrandMapper : Mapper +/// Provides mapping from to . +internal partial class BrandMapper : BiDirectionMapper { /// protected override Contracts.Brand OnMap(Persistence.Brand source) @@ -31,8 +31,30 @@ protected override Contracts.Brand OnMap(Persistence.Brand source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.Brand source, Contracts.Brand destination); + + /// + protected override Persistence.Brand OnMap(Contracts.Brand destination) + { + var source = new Persistence.Brand + { + Code = destination.Code!, + Text = destination.Text, + Description = destination.Description, + SortOrder = destination.SortOrder, + IsActive = !destination.IsInactive, + StartsOn = destination.StartsOn, + EndsOn = destination.EndsOn, + ETag = destination.ETag + }; + + OnMapExtend(destination, source); + return source; + } + + /// Provides the opportunity to extend the From mapping. + partial void OnMapExtend(Contracts.Brand destination, Persistence.Brand source); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/CategoryMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/CategoryMapper.g.cs index 48f90d0b5..2c4512a22 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/CategoryMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/CategoryMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class CategoryMapper : Mapper +/// Provides mapping from to . +internal partial class CategoryMapper : BiDirectionMapper { /// protected override Contracts.Category OnMap(Persistence.Category source) @@ -31,8 +31,11 @@ protected override Contracts.Category OnMap(Persistence.Category source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.Category source, Contracts.Category destination); + + /// + protected override Persistence.Category OnMap(Contracts.Category destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/MovementKindMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/MovementKindMapper.g.cs index bfed492ab..b1b3c4f28 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/MovementKindMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/MovementKindMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class MovementKindMapper : Mapper +/// Provides mapping from to . +internal partial class MovementKindMapper : BiDirectionMapper { /// protected override Contracts.MovementKind OnMap(Persistence.MovementKind source) @@ -31,8 +31,11 @@ protected override Contracts.MovementKind OnMap(Persistence.MovementKind source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.MovementKind source, Contracts.MovementKind destination); + + /// + protected override Persistence.MovementKind OnMap(Contracts.MovementKind destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/MovementStatusMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/MovementStatusMapper.g.cs index cabf9f72f..b835b9d22 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/MovementStatusMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/MovementStatusMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class MovementStatusMapper : Mapper +/// Provides mapping from to . +internal partial class MovementStatusMapper : BiDirectionMapper { /// protected override Contracts.MovementStatus OnMap(Persistence.MovementStatus source) @@ -31,8 +31,11 @@ protected override Contracts.MovementStatus OnMap(Persistence.MovementStatus sou return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.MovementStatus source, Contracts.MovementStatus destination); + + /// + protected override Persistence.MovementStatus OnMap(Contracts.MovementStatus destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/SubCategoryMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/SubCategoryMapper.g.cs index 765540f1f..a3f00612e 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/SubCategoryMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/SubCategoryMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class SubCategoryMapper : Mapper +/// Provides mapping from to . +internal partial class SubCategoryMapper : BiDirectionMapper { /// protected override Contracts.SubCategory OnMap(Persistence.SubCategory source) @@ -32,8 +32,11 @@ protected override Contracts.SubCategory OnMap(Persistence.SubCategory source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.SubCategory source, Contracts.SubCategory destination); + + /// + protected override Persistence.SubCategory OnMap(Contracts.SubCategory destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs b/samples/src/Contoso.Products.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs index a837a0387..5747180c2 100644 --- a/samples/src/Contoso.Products.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Products.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class UnitOfMeasureMapper : Mapper +/// Provides mapping from to . +internal partial class UnitOfMeasureMapper : BiDirectionMapper { /// protected override Contracts.UnitOfMeasure OnMap(Persistence.UnitOfMeasure source) @@ -32,8 +32,11 @@ protected override Contracts.UnitOfMeasure OnMap(Persistence.UnitOfMeasure sourc return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.UnitOfMeasure source, Contracts.UnitOfMeasure destination); + + /// + protected override Persistence.UnitOfMeasure OnMap(Contracts.UnitOfMeasure destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Infrastructure/Repositories/MovementRepository.cs b/samples/src/Contoso.Products.Infrastructure/Repositories/MovementRepository.cs index 8094e38d5..280143bf3 100644 --- a/samples/src/Contoso.Products.Infrastructure/Repositories/MovementRepository.cs +++ b/samples/src/Contoso.Products.Infrastructure/Repositories/MovementRepository.cs @@ -12,17 +12,12 @@ public class MovementRepository(ProductsEfDb ef) : IMovementRepository .ThrowWhen(movements => movements.Any(m => !m.IsQuantityValidForKind), "One or more movements have invalid quantities for their kind."); // Process the movements and related inventory items. - var ids = movements.Select(m => m.ProductId).Distinct(); - var inventoryItems = await _ef.Inventory.QueryTracked().Where(i => ids.Contains(i.Id)).ToDictionaryAsync(i => i.Id!, i => i, ct).ConfigureAwait(false); var args = new EfDbArgs { SaveChanges = false }; foreach (var movement in movements) { - movement.Id = Runtime.NewId(); - var movementModel = MovementMapper.To.Map(movement); - // Adjust the inventory item for the movement. - await InventoryAdjustAsync(args, inventoryItems, movementModel, ct).ConfigureAwait(false); + await InventoryAdjustAsync(movement.ProductId!, movement.KindCode, movement.Quantity, ct).ConfigureAwait(false); // Create the movement. movement.Id = Runtime.NewId(); @@ -34,36 +29,37 @@ public class MovementRepository(ProductsEfDb ef) : IMovementRepository } /// - /// Adjust the inventory item for the movement. + /// Adjusts the inventory item for the movement using a single atomic statement. /// - private async Task InventoryAdjustAsync(EfDbArgs args, Dictionary inventoryItems, Persistence.Movement movement, CancellationToken ct) + /// A tracked read-modify-write is deliberately avoided: concurrent movements for the same product would otherwise race on the row version + /// (xmin) and fail with a concurrency error. The database applies the delta atomically (row-locked within the enclosing transaction), and the + /// sufficient-quantity guard is evaluated against the latest committed value. + private async Task InventoryAdjustAsync(string productId, string? kind, decimal quantity, CancellationToken ct) { - // Create/adjust the inventory item for the movement. - if (inventoryItems.TryGetValue(movement.ProductId!, out var inventoryItem)) - { - if (movement.MovementKindCode == Contracts.MovementKind.Adjust) - inventoryItem.QtyOnHand = movement.Quantity; - else - inventoryItem.QtyOnHand += movement.Quantity; - - if (inventoryItem.QtyOnHand < 0) - throw new BusinessException($"Product '{movement.ProductId}' does not have sufficient quantity on hand.").WithErrorCode("insufficient-quantity").WithKey(movement.ProductId); - - await _ef.Inventory.UpdateAsync(args, inventoryItem, ct).ConfigureAwait(false); - } - else + var db = _ef.DbContext.Database; + var rows = kind switch { - if (movement.MovementKindCode == Contracts.MovementKind.Issue) - throw new BusinessException($"Product '{movement.ProductId}' does not have sufficient quantity on hand.").WithErrorCode("insufficient-quantity").WithKey(movement.ProductId); - - var newInventoryItem = new Persistence.Inventory - { - Id = movement.ProductId, - QtyOnHand = movement.Quantity - }; - - await _ef.Inventory.CreateAsync(args, newInventoryItem, ct).ConfigureAwait(false); - } + // Set the absolute quantity on hand; create where it does not yet exist. + Contracts.MovementKind.Adjust => await db.ExecuteSqlInterpolatedAsync($""" + INSERT INTO "products"."inventory" ("inventory_id", "qty_on_hand") VALUES ({productId}, {quantity}) + ON CONFLICT ("inventory_id") DO UPDATE SET "qty_on_hand" = EXCLUDED."qty_on_hand" + """, ct).ConfigureAwait(false), + + // Increase the quantity on hand; create where it does not yet exist. + _ when quantity >= 0 => await db.ExecuteSqlInterpolatedAsync($""" + INSERT INTO "products"."inventory" ("inventory_id", "qty_on_hand") VALUES ({productId}, {quantity}) + ON CONFLICT ("inventory_id") DO UPDATE SET "qty_on_hand" = "inventory"."qty_on_hand" + EXCLUDED."qty_on_hand" + """, ct).ConfigureAwait(false), + + // Decrease the quantity on hand only where sufficient; no row affected means insufficient (or non-existent) inventory. + _ => await db.ExecuteSqlInterpolatedAsync($""" + UPDATE "products"."inventory" SET "qty_on_hand" = "qty_on_hand" + {quantity} + WHERE "inventory_id" = {productId} AND "qty_on_hand" + {quantity} >= 0 + """, ct).ConfigureAwait(false) + }; + + if (rows == 0) + throw new BusinessException($"Product '{productId}' does not have sufficient quantity on hand.").WithErrorCode("insufficient-quantity").WithKey(productId); } /// @@ -94,24 +90,12 @@ private async Task InventoryAdjustAsync(EfDbArgs args, Dictionary m.ProductId).Distinct(); - var inventoryItems = await _ef.Inventory.QueryTracked().Where(i => ids.Contains(i.Id)).ToDictionaryAsync(i => i.Id!, i => i, ct).ConfigureAwait(false); - // Update the movement status to cancelled and adjust inventory back for all movements. var args = new EfDbArgs { SaveChanges = false }; foreach (var movement in movements) { - // Create a fake movement with opposite quantity to adjust the inventory back. - var fakeMovement = new Persistence.Movement - { - ProductId = movement.ProductId, - Quantity = -movement.Quantity, - MovementKindCode = CreateReversalMovementKind(movement.MovementKindCode) - }; - - // Reverse (adjust back) the inventory for the movement. - await InventoryAdjustAsync(args, inventoryItems, fakeMovement, ct).ConfigureAwait(false); + // Reverse (adjust back) the inventory for the movement using the opposite kind and quantity. + await InventoryAdjustAsync(movement.ProductId!, CreateReversalMovementKind(movement.MovementKindCode), -movement.Quantity, ct).ConfigureAwait(false); // Update movement status to cancelled. movement.MovementStatusCode = Contracts.MovementStatus.Canceled; diff --git a/samples/src/Contoso.Products.Infrastructure/Repositories/ProductsDbContext.g.cs b/samples/src/Contoso.Products.Infrastructure/Repositories/ProductsDbContext.g.cs index ccd71c9b8..67c215ae0 100644 --- a/samples/src/Contoso.Products.Infrastructure/Repositories/ProductsDbContext.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Repositories/ProductsDbContext.g.cs @@ -24,10 +24,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("text").HasColumnType("CHARACTER VARYING(250)"); e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -42,10 +42,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("text").HasColumnType("CHARACTER VARYING(250)"); e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -60,10 +60,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("text").HasColumnType("CHARACTER VARYING(250)"); e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -78,10 +78,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("text").HasColumnType("CHARACTER VARYING(250)"); e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -97,10 +97,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); e.Property(p => p.CategoryCode).HasColumnName("category_code").HasColumnType("CHARACTER VARYING(50)"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -116,10 +116,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.IsActive).HasColumnName("is_active").HasColumnType("BOOLEAN"); e.Property(p => p.SortOrder).HasColumnName("sort_order").HasColumnType("INTEGER"); e.Property(p => p.Scale).HasColumnName("scale").HasColumnType("INTEGER"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -147,10 +147,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.ProductId).HasColumnName("product_id").HasColumnType("CHARACTER VARYING(50)"); e.Property(p => p.Quantity).HasColumnName("quantity").HasColumnType("NUMERIC(18, 2)"); e.Property(p => p.UnitOfMeasureCode).HasColumnName("unit_of_measure_code").HasColumnType("CHARACTER VARYING(50)"); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); }); @@ -169,10 +169,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.IsInactive).HasColumnName("is_inactive").HasColumnType("BOOLEAN"); e.Property(p => p.IsNonStocked).HasColumnName("is_non_stocked").HasColumnType("BOOLEAN"); e.Property(p => p.Tags).HasColumnName("tags_json").HasColumnType("JSONB").HasConversion(TypeToJsonStringEfConverter?>.Default, TypeToJsonStringEfComparer?>.Default); - e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); - e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE"); + e.Property(p => p.CreatedBy).HasColumnName("created_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("created_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("updated_by").HasColumnType("CHARACTER VARYING(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("updated_on").HasColumnType("TIMESTAMP WITH TIME ZONE").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("xmin").HasColumnType("XID").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Property(p => p.IsDeleted).HasColumnName("is_deleted").HasColumnType("BOOLEAN"); }); diff --git a/samples/src/Contoso.Products.Infrastructure/Repositories/ReferenceDataRepository.g.cs b/samples/src/Contoso.Products.Infrastructure/Repositories/ReferenceDataRepository.g.cs index 133e1f1c4..d8fe746db 100644 --- a/samples/src/Contoso.Products.Infrastructure/Repositories/ReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Products.Infrastructure/Repositories/ReferenceDataRepository.g.cs @@ -14,27 +14,55 @@ public partial class ReferenceDataRepository : IReferenceDataRepository { /// public Task GetAllBrandsAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(BrandMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(BrandMapper.From, cancellationToken); /// public Task GetAllCategoriesAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(CategoryMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(CategoryMapper.From, cancellationToken); /// public Task GetAllMovementKindsAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(MovementKindMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(MovementKindMapper.From, cancellationToken); /// public Task GetAllMovementStatusesAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(MovementStatusMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(MovementStatusMapper.From, cancellationToken); /// public Task GetAllSubCategoriesAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(SubCategoryMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(SubCategoryMapper.From, cancellationToken); /// public Task GetAllUnitsOfMeasureAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(UnitOfMeasureMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(UnitOfMeasureMapper.From, cancellationToken); + + #region Brand + + /// + public Task> GetBrandAsync(string id, CancellationToken cancellationToken = default) + => _ef.ThrowIfNull().Model().GetWithResultAsync(id, cancellationToken).ThenAs(model => BrandMapper.From.Map(model)); + + /// + public Task>> CreateBrandAsync(Contracts.Brand value, CancellationToken cancellationToken = default) + => EfDbReferenceData.CreateAsync(_ef.ThrowIfNull().Model(), value, BrandMapper.Default, cancellationToken); + + /// + public Task>> UpdateBrandAsync(string id, Contracts.Brand value, CancellationToken cancellationToken = default) + => EfDbReferenceData.UpdateAsync(_ef.ThrowIfNull().Model(), id, value, BrandMapper.Default, cancellationToken); + + /// + public Task>> ActivateBrandAsync(string id, CancellationToken cancellationToken = default) + => EfDbReferenceData.ActivateAsync(_ef.ThrowIfNull().Model(), id, BrandMapper.Default, cancellationToken); + + /// + public Task>> DeactivateBrandAsync(string id, CancellationToken cancellationToken = default) + => EfDbReferenceData.DeactivateAsync(_ef.ThrowIfNull().Model(), id, BrandMapper.Default, cancellationToken); + + /// + public Task> DeleteBrandAsync(string id, CancellationToken cancellationToken = default) + => EfDbReferenceData.DeleteAsync(_ef.ThrowIfNull().Model(), id, cancellationToken); + + #endregion } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Products.Relay/appsettings.json b/samples/src/Contoso.Products.Relay/appsettings.json index 9cda3b361..3cdc2507e 100644 --- a/samples/src/Contoso.Products.Relay/appsettings.json +++ b/samples/src/Contoso.Products.Relay/appsettings.json @@ -8,10 +8,16 @@ "OutboxRelay": { "BatchSize": 10, "PerWorkerPartitionCount": 2, - "LeaseDuration": "00:00:05", + "LeaseDuration": "00:00:30", "BackoffDuration": "00:00:05", "ServicesCount": 4 } + }, + "Outbox": { + "PartitionSize": 4 + }, + "ServiceBus": { + "SessionIdPartitionSize": 4 } } }, @@ -22,4 +28,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Products.Subscribe/Program.cs b/samples/src/Contoso.Products.Subscribe/Program.cs index 93e74b56d..48b68c424 100644 --- a/samples/src/Contoso.Products.Subscribe/Program.cs +++ b/samples/src/Contoso.Products.Subscribe/Program.cs @@ -56,6 +56,7 @@ private static void Main(string[] args) .AddPostgresDatabase() // Adds the PostgresDatabase. .AddPostgresUnitOfWork() // Adds the PostgresUnitOfWork for the PostgresDatabase. .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddPostgresOutboxPublisher() // Adds the ProductsOutboxPublisher as the PostgresOutboxPublisher/IEventPublisher. .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. @@ -66,18 +67,31 @@ private static void Main(string[] args) // Add event formatter and subscribed-manager. builder.Services.AddSubscribedManager((_, c) => c.AddSubscribersUsing()); // Adds the SubscribedManager and dynamically links to the individual Subscribers. - // Builds and creates the Azure Service Bus receiving services. + // Builds and creates the Azure Service Bus receiving services; each receiver (and subscriber/hosted service) requires its own service key. + // Receiver 1: the shared 'contoso' topic 'products' subscription (events). builder.Services.AzureServiceBusReceiving() .WithSessionReceiver(_ => { // Adds the service bus receiver to pump messages to the subscriber. var o = ServiceBusSessionReceiverOptions.CreateForTopicSubscription(); // Set the topic and subscription from configuration. o.SessionProcessorOptions.MaxConcurrentSessions = 4; // Set the maximum number of concurrent sessions to process. return o; - }) - .WithSubscribedSubscriber() // Adds the service bus subscriber using the ^ SubscribedManager. - .WithHostedService() // Adds the ^ service bus receiver as a hosted service. + }, "receiver-events") + .WithKeyedSubscribedSubscriber("subscriber-events") // Adds the service bus subscriber using the ^ SubscribedManager. + .WithHostedService("hosted-subscriber-events") // Adds the ^ service bus receiver as a hosted service. .Build(); // Builds all the ^ services and adds to the service collection. + // Receiver 2: the 'contoso-products' command queue (commands are addressed to a single consuming domain via the NamedDestinationProvider). + builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => + { + var o = ServiceBusSessionReceiverOptions.CreateForQueue("contoso-products"); // Set the queue name; this must match the NamedDestinationProvider command destination. + o.SessionProcessorOptions.MaxConcurrentSessions = 4; + return o; + }, "receiver-commands") + .WithKeyedSubscribedSubscriber("subscriber-commands") + .WithHostedService("hosted-subscriber-commands") + .Build(); + // Post-configure all health-checks; adds the standard tags. builder.Services.PostConfigureAllHealthChecks(); diff --git a/samples/src/Contoso.Products.Subscribe/appsettings.json b/samples/src/Contoso.Products.Subscribe/appsettings.json index 66eb72d39..6f0c33598 100644 --- a/samples/src/Contoso.Products.Subscribe/appsettings.json +++ b/samples/src/Contoso.Products.Subscribe/appsettings.json @@ -5,6 +5,12 @@ "DomainName": "Products", "Services": { "Interval": "00:00:00.500" + }, + "Outbox": { + "PartitionSize": 4 + }, + "ServiceBus": { + "SessionIdPartitionSize": 4 } }, "Events": { @@ -18,4 +24,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Api/Controllers/BasketReadController.cs b/samples/src/Contoso.Shopping.Api/Controllers/BasketReadController.cs index 010626181..c87943e76 100644 --- a/samples/src/Contoso.Shopping.Api/Controllers/BasketReadController.cs +++ b/samples/src/Contoso.Shopping.Api/Controllers/BasketReadController.cs @@ -10,4 +10,14 @@ public class BasketReadController(WebApi webApi, IBasketReadService service) : C [ProducesResponseType(typeof(Basket), 200)] [ProducesNotFoundProblem()] public Task GetAsync(string basketId, CancellationToken cancellationToken = default) => _webApi.GetWithResultAsync(Request, (_, ct) => _service.GetAsync(basketId.Required(), ct), cancellationToken: cancellationToken); + + [HttpGet("/api/customers/{customerId}/baskets")] + [ProducesResponseType(200)] + [Query, Paging(supportsCount: true)] + public Task QueryAsync(string customerId, CancellationToken cancellationToken = default) => _webApi.GetAsync(Request, (ro, ct) + => _service.QueryAsync(customerId.Required(), ro.QueryArgs, ro.PagingArgs, ct), HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpGet("/api/customers/{customerId}/baskets/$query")] + [ProducesResponseType(typeof(JsonElement), 200)] + public Task QuerySchemaAsync(string customerId, CancellationToken cancellationToken = default) => _webApi.GetAsync(Request, (ro, ct) => _service.QuerySchemaAsync(ct), cancellationToken: cancellationToken); } \ No newline at end of file diff --git a/samples/src/Contoso.Shopping.Api/Controllers/ReferenceDataController.g.cs b/samples/src/Contoso.Shopping.Api/Controllers/ReferenceDataController.g.cs index 4eabd3f9f..acd34de08 100644 --- a/samples/src/Contoso.Shopping.Api/Controllers/ReferenceDataController.g.cs +++ b/samples/src/Contoso.Shopping.Api/Controllers/ReferenceDataController.g.cs @@ -14,28 +14,23 @@ public partial class ReferenceDataController(CoreEx.AspNetCore.Mvc.WebApi webApi { private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); - [HttpGet("basket-statuses"), HttpHead("basket-statuses")] + [HttpGet("basket-statuses")] [ProducesResponseType(typeof(BasketStatus[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetBasketStatusesAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("discount-coupons"), HttpHead("discount-coupons")] + [HttpGet("discount-coupons")] [ProducesResponseType(typeof(DiscountCoupon[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetDiscountCouponsAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - [HttpGet("units-of-measure"), HttpHead("units-of-measure")] + [HttpGet("units-of-measure")] [ProducesResponseType(typeof(UnitOfMeasure[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task GetUnitsOfMeasureAsync(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); - - [HttpGet] - [ProducesResponseType(typeof(ReferenceDataMultiDictionary), 200)] - public Task GetNamedAsync([FromQuery] string[] name, CancellationToken cancellationToken) - => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.GetNamedAsync(name, ro.IsIncludeInactive, ct), cancellationToken: cancellationToken); } #nullable restore diff --git a/samples/src/Contoso.Shopping.Api/GlobalUsing.cs b/samples/src/Contoso.Shopping.Api/GlobalUsing.cs index 8b035da6f..c5c496f73 100644 --- a/samples/src/Contoso.Shopping.Api/GlobalUsing.cs +++ b/samples/src/Contoso.Shopping.Api/GlobalUsing.cs @@ -3,6 +3,7 @@ global using Contoso.Shopping.Contracts; global using CoreEx; global using CoreEx.AspNetCore.Mvc; +global using CoreEx.Data; global using CoreEx.Entities; global using CoreEx.Http; global using CoreEx.Json; diff --git a/samples/src/Contoso.Shopping.Api/Program.cs b/samples/src/Contoso.Shopping.Api/Program.cs index 3a28106b7..e2159cf07 100644 --- a/samples/src/Contoso.Shopping.Api/Program.cs +++ b/samples/src/Contoso.Shopping.Api/Program.cs @@ -1,4 +1,6 @@ +using Contoso.Shopping.Infrastructure.Clients.Customers; using Contoso.Shopping.Infrastructure.Clients.Products; +using Contoso.Shopping.Infrastructure.Clients.SendGrid; using Contoso.Shopping.Infrastructure.Repositories; using CoreEx.Azure.Messaging.ServiceBus; using CoreEx.Database.SqlServer.Outbox; @@ -7,6 +9,7 @@ using OpenTelemetry; using OpenTelemetry.Trace; using StackExchange.Redis; +using System.Net.Http.Headers; using ZiggyCreatures.Caching.Fusion; using ZiggyCreatures.Caching.Fusion.Backplane.StackExchangeRedis; @@ -31,7 +34,7 @@ private static void Main(string[] args) .AddHttpWebApi(); // Add all the dynamically registered services. - builder.Services.AddDynamicServicesUsing(); + builder.Services.AddDynamicServicesUsing(); // Add L1/L2 caching services. builder.Services.AddMemoryCache(); // Adds the in-memory cache - L1. @@ -56,6 +59,7 @@ private static void Main(string[] args) .AddSqlServerDatabase() // Adds the SqlServerDatabase. .AddSqlServerUnitOfWork() // Adds the SqlServerUnitOfWork for the SqlServerDatabase. .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddSqlServerOutboxPublisher() // Adds the SqlServerOutboxPublisher/IEventPublisher. .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. @@ -69,6 +73,9 @@ private static void Main(string[] args) // Add external API services. builder.AddTypedHttpClient("ProductsApi"); + builder.AddTypedHttpClient("CustomersApi"); + builder.AddTypedHttpClient("SendGrid", client => client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", builder.Configuration["SendGrid:ApiKey"])); + builder.Services.Configure(builder.Configuration.GetSection("SendGrid")); // Post-configure all health-checks; adds the standard tags. builder.Services.PostConfigureAllHealthChecks(); diff --git a/samples/src/Contoso.Shopping.Api/appsettings.Development.json b/samples/src/Contoso.Shopping.Api/appsettings.Development.json index 72654558b..5e0abb9a2 100644 --- a/samples/src/Contoso.Shopping.Api/appsettings.Development.json +++ b/samples/src/Contoso.Shopping.Api/appsettings.Development.json @@ -37,5 +37,8 @@ }, "ProductsApi": { "BaseAddress": "https://localhost:7200" + }, + "CustomersApi": { + "BaseAddress": "https://localhost:7320" } -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Api/appsettings.json b/samples/src/Contoso.Shopping.Api/appsettings.json index 6c943a3ce..9a85e06e3 100644 --- a/samples/src/Contoso.Shopping.Api/appsettings.json +++ b/samples/src/Contoso.Shopping.Api/appsettings.json @@ -2,7 +2,18 @@ "CoreEx": { "Host": { "SolutionName": "Contoso", - "DomainName": "Shopping" + "DomainName": "Shopping", + "Outbox": { + "PartitionSize": 4 + }, + "ServiceBus": { + "SessionIdPartitionSize": 4, + "Destinations": { + "contoso-products": { + "SessionIdPartitionSize": 2 + } + } + } }, "Events": { "Destination": "contoso" // Topic/queue name. @@ -15,4 +26,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Application/Adapters/Customers/Customer.cs b/samples/src/Contoso.Shopping.Application/Adapters/Customers/Customer.cs new file mode 100644 index 000000000..8c0728540 --- /dev/null +++ b/samples/src/Contoso.Shopping.Application/Adapters/Customers/Customer.cs @@ -0,0 +1,18 @@ +namespace Contoso.Shopping.Application.Adapters.Customers; + +/// +/// Represents the Shopping domain's view of a Customer (from the Customers domain); only the data required by Shopping is included. +/// +[Contract] +public partial class Customer : IIdentifier +{ + public string? Id { get; set; } + + public string? FirstName { get; set; } + + public string? LastName { get; set; } + + public string? Email { get; set; } + + public Address? ShippingAddress { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Application/Adapters/Customers/ICustomerAdapter.cs b/samples/src/Contoso.Shopping.Application/Adapters/Customers/ICustomerAdapter.cs new file mode 100644 index 000000000..b36bf1a11 --- /dev/null +++ b/samples/src/Contoso.Shopping.Application/Adapters/Customers/ICustomerAdapter.cs @@ -0,0 +1,13 @@ +namespace Contoso.Shopping.Application.Adapters.Customers; + +/// +/// Enables the Customers domain integration, serving as the external dependency boundary (anti-corruption layer) for customer-related operations. +/// +/// Unlike the (which reads from an event-replicated local store), this adapter makes real-time calls to the Customers domain. +public interface ICustomerAdapter +{ + /// + /// Gets the customer; will result in a where it does not exist. + /// + Task> GetAsync(string id, CancellationToken ct = default); +} diff --git a/samples/src/Contoso.Shopping.Application/Adapters/Notifications/INotificationAdapter.cs b/samples/src/Contoso.Shopping.Application/Adapters/Notifications/INotificationAdapter.cs new file mode 100644 index 000000000..9d7f6a239 --- /dev/null +++ b/samples/src/Contoso.Shopping.Application/Adapters/Notifications/INotificationAdapter.cs @@ -0,0 +1,12 @@ +namespace Contoso.Shopping.Application.Adapters.Notifications; + +/// +/// Enables outbound customer notifications, serving as the external dependency boundary (anti-corruption layer) for notification-related operations. +/// +public interface INotificationAdapter +{ + /// + /// Sends a confirmation notification for the specified checked-out . + /// + Task SendBasketCheckedOutConfirmationAsync(Contracts.Basket basket, CancellationToken ct = default); +} diff --git a/samples/src/Contoso.Shopping.Application/BasketReadService.cs b/samples/src/Contoso.Shopping.Application/BasketReadService.cs index fdfea1884..85e31b892 100644 --- a/samples/src/Contoso.Shopping.Application/BasketReadService.cs +++ b/samples/src/Contoso.Shopping.Application/BasketReadService.cs @@ -9,4 +9,11 @@ public class BasketReadService(IBasketRepository repository) : IBasketReadServic public Task> GetAsync(string basketId, CancellationToken ct = default) => Result.GoAsync(() => _repository.GetAsync(basketId, ct)) .ThenAs(b => BasketMapper.Map(b)); + + /// + public Task QuerySchemaAsync(CancellationToken ct = default) => _repository.QuerySchemaAsync(ct); + + /// + public Task> QueryAsync(string customerId, QueryArgs? query, PagingArgs? paging, CancellationToken ct = default) + => _repository.QueryAsync(customerId, query, paging, ct); } \ No newline at end of file diff --git a/samples/src/Contoso.Shopping.Application/BasketService.cs b/samples/src/Contoso.Shopping.Application/BasketService.cs index f79ae4d9f..6db82209b 100644 --- a/samples/src/Contoso.Shopping.Application/BasketService.cs +++ b/samples/src/Contoso.Shopping.Application/BasketService.cs @@ -1,23 +1,27 @@ namespace Contoso.Shopping.Application; [ScopedService] -public class BasketService(IUnitOfWork unitOfWork, IBasketRepository repository, IProductAdapter productAdapter, ILogger logger) : IBasketService +public class BasketService(IUnitOfWork unitOfWork, IBasketRepository repository, IProductAdapter productAdapter, ICustomerAdapter customerAdapter, ILogger logger) : IBasketService { private readonly IUnitOfWork _unitOfWork = unitOfWork.ThrowIfNull(); private readonly IBasketRepository _repository = repository.ThrowIfNull(); private readonly IProductAdapter _productAdapter = productAdapter.ThrowIfNull(); + private readonly ICustomerAdapter _customerAdapter = customerAdapter.ThrowIfNull(); private readonly ILogger _logger = logger.ThrowIfNull(); /// - public Task> CreateAsync(string customerId, CancellationToken ct = default) + public async Task> CreateAsync(string customerId, CancellationToken ct = default) { - // No validation for customerId is performed here, but could be added as required (e.g. check for valid format, or existence of the customer in the system). + // Ensure the customer exists; this is a real-time call to the Customers domain (via the adapter). + var cr = await new CustomerPolicy(_customerAdapter).EnsureExistsAsync(customerId.ThrowIfNullOrEmpty(), ct).ConfigureAwait(false); + if (cr.IsFailure) + return cr.AsResult(); - // Create the aggregate representation. - var aggregate = Domain.Basket.CreateNew(customerId.ThrowIfNullOrEmpty()); + // Default the shipping address from the customer where one exists (a null address maps to null); the basket address can subsequently be overridden. + var aggregate = Domain.Basket.CreateNew(customerId, AddressMapper.From.Map(cr.Value.ShippingAddress)); // Orchestrate the creation of the basket within a transaction, ensuring that any events are only published if the transaction is successful. - return _unitOfWork.TransactionAsync(async tct => + return await _unitOfWork.TransactionAsync(async tct => { // Create the aggregate in the repository, which will return the created aggregate with any updates (e.g. Id). var br = await _repository.CreateAsync(aggregate, tct).ConfigureAwait(false); @@ -31,7 +35,7 @@ public Task> CreateAsync(string customerId, CancellationToken ct return contract; }); - }, ct); + }, ct).ConfigureAwait(false); } /// diff --git a/samples/src/Contoso.Shopping.Application/GlobalUsing.cs b/samples/src/Contoso.Shopping.Application/GlobalUsing.cs index 83ea53fb0..e8c9d1de0 100644 --- a/samples/src/Contoso.Shopping.Application/GlobalUsing.cs +++ b/samples/src/Contoso.Shopping.Application/GlobalUsing.cs @@ -1,3 +1,4 @@ +global using Contoso.Shopping.Application.Adapters.Customers; global using Contoso.Shopping.Application.Adapters.Products; global using Contoso.Shopping.Application.Interfaces; global using Contoso.Shopping.Application.Mapping; @@ -16,4 +17,5 @@ global using CoreEx.RefData.Abstractions; global using CoreEx.Results; global using CoreEx.Validation; -global using Microsoft.Extensions.Logging; \ No newline at end of file +global using Microsoft.Extensions.Logging; +global using System.Text.Json; \ No newline at end of file diff --git a/samples/src/Contoso.Shopping.Application/Interfaces/IBasketReadService.cs b/samples/src/Contoso.Shopping.Application/Interfaces/IBasketReadService.cs index bb9cb502d..1e0b75e9e 100644 --- a/samples/src/Contoso.Shopping.Application/Interfaces/IBasketReadService.cs +++ b/samples/src/Contoso.Shopping.Application/Interfaces/IBasketReadService.cs @@ -6,4 +6,18 @@ public interface IBasketReadService /// Get the for the specified /// Task> GetAsync(string basketId, CancellationToken ct = default); + + /// + /// Gets the schema. + /// + Task QuerySchemaAsync(CancellationToken ct = default); + + /// + /// Queries the s (summary only; excludes the items and shipping address) for the specified . + /// + /// The customer identifier; always applied as a filter. + /// The . + /// The . + /// The resulting baskets. + Task> QueryAsync(string customerId, QueryArgs? query, PagingArgs? paging, CancellationToken ct = default); } \ No newline at end of file diff --git a/samples/src/Contoso.Shopping.Application/Policies/CustomerPolicy.cs b/samples/src/Contoso.Shopping.Application/Policies/CustomerPolicy.cs new file mode 100644 index 000000000..924198718 --- /dev/null +++ b/samples/src/Contoso.Shopping.Application/Policies/CustomerPolicy.cs @@ -0,0 +1,11 @@ +namespace Contoso.Shopping.Application.Policies; + +public class CustomerPolicy(ICustomerAdapter customerAdapter) +{ + private static readonly LText _customerText = new("Customer"); + private readonly ICustomerAdapter _customerAdapter = customerAdapter.ThrowIfNull(); + + public Task> EnsureExistsAsync(string customerId, CancellationToken ct = default) => Result + .GoAsync(() => _customerAdapter.GetAsync(customerId, ct)) + .OnFailure(r => r.IsNotFoundError ? Result.ValidationError(MessageItem.CreateErrorMessage(nameof(customerId), "{0} was not found.", _customerText)) : r); +} diff --git a/samples/src/Contoso.Shopping.Application/ReferenceDataService.g.cs b/samples/src/Contoso.Shopping.Application/ReferenceDataProvider.g.cs similarity index 94% rename from samples/src/Contoso.Shopping.Application/ReferenceDataService.g.cs rename to samples/src/Contoso.Shopping.Application/ReferenceDataProvider.g.cs index 5b6bc1b42..eb9e3197d 100644 --- a/samples/src/Contoso.Shopping.Application/ReferenceDataService.g.cs +++ b/samples/src/Contoso.Shopping.Application/ReferenceDataProvider.g.cs @@ -10,7 +10,7 @@ namespace Contoso.Shopping.Application; /// Provides the implementation. [ScopedService] -public partial class ReferenceDataService(IReferenceDataRepository repository) : IReferenceDataProvider +public partial class ReferenceDataProvider(IReferenceDataRepository repository) : IReferenceDataProvider { private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); diff --git a/samples/src/Contoso.Shopping.Application/Repositories/IBasketRepository.cs b/samples/src/Contoso.Shopping.Application/Repositories/IBasketRepository.cs index 258017816..e812f0578 100644 --- a/samples/src/Contoso.Shopping.Application/Repositories/IBasketRepository.cs +++ b/samples/src/Contoso.Shopping.Application/Repositories/IBasketRepository.cs @@ -7,4 +7,18 @@ public interface IBasketRepository Task> CreateAsync(Domain.Basket basket, CancellationToken ct = default); Task> UpdateAsync(Domain.Basket basket, CancellationToken ct = default); + + /// + /// Gets the schema. + /// + Task QuerySchemaAsync(CancellationToken ct = default); + + /// + /// Queries the baskets (summary only; excludes the items and shipping address) for the specified . + /// + /// The customer identifier; always applied as a filter. + /// The . + /// The . + /// The resulting baskets. + Task> QueryAsync(string customerId, QueryArgs? query, PagingArgs? paging, CancellationToken ct = default); } \ No newline at end of file diff --git a/samples/src/Contoso.Shopping.Application/Repositories/IReferenceDataRepository.g.cs b/samples/src/Contoso.Shopping.Application/Repositories/IReferenceDataRepository.g.cs index b7cad4ac9..a06bc28c7 100644 --- a/samples/src/Contoso.Shopping.Application/Repositories/IReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Shopping.Application/Repositories/IReferenceDataRepository.g.cs @@ -12,16 +12,13 @@ namespace Contoso.Shopping.Application.Repositories; public partial interface IReferenceDataRepository { /// Gets all items. - /// The . Task GetAllBasketStatusesAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllDiscountCouponsAsync(CancellationToken cancellationToken = default); /// Gets all items. - /// The . Task GetAllUnitsOfMeasureAsync(CancellationToken cancellationToken = default); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Shopping.Contracts/Address.cs b/samples/src/Contoso.Shopping.Contracts/Address.cs index 4e6e7921a..48fd28551 100644 --- a/samples/src/Contoso.Shopping.Contracts/Address.cs +++ b/samples/src/Contoso.Shopping.Contracts/Address.cs @@ -3,9 +3,17 @@ namespace Contoso.Shopping.Contracts; [Contract] public partial class Address { + [NonNullable] public string? Street1 { get; set; } + public string? Street2 { get; set; } + + [NonNullable] public string? City { get; set; } + + [NonNullable] public string? PostCode { get; set; } + + [NonNullable] public string? State { get; set; } } diff --git a/samples/src/Contoso.Shopping.Contracts/Basket.cs b/samples/src/Contoso.Shopping.Contracts/Basket.cs index 45dfb248b..d37d07b94 100644 --- a/samples/src/Contoso.Shopping.Contracts/Basket.cs +++ b/samples/src/Contoso.Shopping.Contracts/Basket.cs @@ -6,9 +6,11 @@ public partial class Basket : IIdentifier, IChangeLog, IETag [ReadOnly(true)] public string? Id { get; set; } + [NonNullable] public string? CustomerId { get; set; } [ReferenceData] + [NonNullable] public partial string? StatusCode { get; set; } [ReadOnly(true)] diff --git a/samples/src/Contoso.Shopping.Contracts/BasketItem.cs b/samples/src/Contoso.Shopping.Contracts/BasketItem.cs index 7eb344310..38f84185d 100644 --- a/samples/src/Contoso.Shopping.Contracts/BasketItem.cs +++ b/samples/src/Contoso.Shopping.Contracts/BasketItem.cs @@ -7,26 +7,30 @@ public partial class BasketItem : IIdentifier, IETag public string? Id { get; set; } [ReadOnly(true)] + [NonNullable] public string? ProductId { get; set; } [ReadOnly(true)] + [NonNullable] public string? Sku { get; set; } [ReadOnly(true)] + [NonNullable] public string? Text { get; set; } - public decimal? Quantity { get; set; } + public decimal Quantity { get; set; } [ReadOnly(true)] + [NonNullable] [ReferenceData] public partial string? UnitOfMeasureCode { get; set; } [ReadOnly(true)] - public decimal? UnitPrice { get; set; } + public decimal UnitPrice { get; set; } [ReadOnly(true)] - public decimal? Total { get; set; } + public decimal Total { get; set; } [ReadOnly(true)] public string? ETag { get; set; } -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Contracts/BasketItemAddRequest.cs b/samples/src/Contoso.Shopping.Contracts/BasketItemAddRequest.cs index 5978b9790..9abe4f425 100644 --- a/samples/src/Contoso.Shopping.Contracts/BasketItemAddRequest.cs +++ b/samples/src/Contoso.Shopping.Contracts/BasketItemAddRequest.cs @@ -2,6 +2,7 @@ namespace Contoso.Shopping.Contracts; public class BasketItemAddRequest { + [NonNullable] public string? ProductId { get; set; } public decimal Quantity { get; set; } diff --git a/samples/src/Contoso.Shopping.Domain/Basket.cs b/samples/src/Contoso.Shopping.Domain/Basket.cs index b20e22c34..1ebae7692 100644 --- a/samples/src/Contoso.Shopping.Domain/Basket.cs +++ b/samples/src/Contoso.Shopping.Domain/Basket.cs @@ -4,10 +4,11 @@ public sealed class Basket : Aggregate { private List _items = []; - public static Basket CreateNew(string customerId) => new Basket(Runtime.NewId()) + public static Basket CreateNew(string customerId, Address? shippingAddress = null) => new Basket(Runtime.NewId()) { CustomerId = customerId, - Status = Contracts.BasketStatus.Empty + Status = Contracts.BasketStatus.Empty, + ShippingAddress = shippingAddress }.AsNew(); public static Basket CreateFrom(string id, string customerId, Contracts.BasketStatus status, Contracts.DiscountCoupon? discountCoupon, Address? shippingAddress, IEnumerable? items, ChangeLog? changeLog, string? etag) => new Basket(id) diff --git a/samples/src/Contoso.Shopping.Infrastructure/Adapters/Customers/CustomerAdapter.cs b/samples/src/Contoso.Shopping.Infrastructure/Adapters/Customers/CustomerAdapter.cs new file mode 100644 index 000000000..56488a98d --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Adapters/Customers/CustomerAdapter.cs @@ -0,0 +1,16 @@ +namespace Contoso.Shopping.Infrastructure.Adapters.Customers; + +[ScopedService] +public class CustomerAdapter(CustomersHttpClient client, IHybridCache cache) : ICustomerAdapter +{ + private readonly CustomersHttpClient _client = client.ThrowIfNull(); + private readonly IHybridCache _cache = cache.ThrowIfNull(); + + // Customer data is owned by another domain and can change at any time, so keep the expirations short; overridable via configuration (CoreEx:Caching:Customer:*). + private readonly HybridCacheEntryOptions _cacheOptions = HybridCacheEntryOptions.CreateFor(localExpiration: TimeSpan.FromMinutes(1), distributedExpiration: TimeSpan.FromMinutes(5)); + + /// + /// Invokes the Customers API in real-time (there is no local replicated copy of customer data); the result is cached to avoid repeated cross-domain calls. Only a successful result is cached, + /// so a customer that does not exist (yet) is never remembered as such. + public Task> GetAsync(string id, CancellationToken ct = default) => _cache.GetOrCreateWithResultAsync(id, c => _client.GetAsync(id, c), _cacheOptions, ct); +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Adapters/Notifications/NotificationAdapter.cs b/samples/src/Contoso.Shopping.Infrastructure/Adapters/Notifications/NotificationAdapter.cs new file mode 100644 index 000000000..9a528561c --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Adapters/Notifications/NotificationAdapter.cs @@ -0,0 +1,42 @@ +namespace Contoso.Shopping.Infrastructure.Adapters.Notifications; + +[ScopedService] +public class NotificationAdapter(SendGridHttpClient client, ICustomerAdapter customerAdapter, IOptions options) : INotificationAdapter +{ + private readonly SendGridHttpClient _client = client.ThrowIfNull(); + private readonly ICustomerAdapter _customerAdapter = customerAdapter.ThrowIfNull(); + private readonly SendGridOptions _options = options.ThrowIfNull().Value; + + /// + /// The basket has no email address of its own (only a ); the recipient is therefore resolved in real-time via the . + public async Task SendBasketCheckedOutConfirmationAsync(Contracts.Basket basket, CancellationToken ct = default) + { + var cr = await _customerAdapter.GetAsync(basket.CustomerId!, ct).ConfigureAwait(false); + if (cr.IsFailure) + return cr.AsResult(); + + var customer = cr.Value; + var request = new SendGridMailRequest + { + From = new SendGridEmailAddress { Email = _options.FromEmail, Name = _options.FromName }, + Personalizations = + [ + new SendGridPersonalization + { + To = [new SendGridEmailAddress { Email = customer.Email!, Name = $"{customer.FirstName} {customer.LastName}".Trim() }], + Subject = $"Your Contoso order {basket.Id} is confirmed" + } + ], + Content = + [ + new SendGridContent + { + Type = "text/plain", + Value = $"Hi {customer.FirstName}, thanks for shopping with Contoso! Your basket {basket.Id} has been checked out and is now confirmed. Order total: {basket.Pricing?.Total:C}." + } + ] + }; + + return await _client.SendMailAsync(request, ct).ConfigureAwait(false); + } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/Customers/CustomersHttpClient.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/Customers/CustomersHttpClient.cs new file mode 100644 index 000000000..ac1dc8dee --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/Customers/CustomersHttpClient.cs @@ -0,0 +1,26 @@ +namespace Contoso.Shopping.Infrastructure.Clients.Customers; + +/// +/// Provides the HTTP facade for interacting with the external Customers API. +/// +/// The . +public class CustomersHttpClient(HttpClient httpClient) +{ + private static readonly Validator _validator = Validator.Create() + .HasProperty(x => x.Email, c => c.Mandatory().Email()); + + private readonly HttpClient _httpClient = httpClient.ThrowIfNull(); + + /// + /// Gets the customer, or returns a not-found result when the customer does not exist. + /// + /// The customer identifier. + public async Task> GetAsync(string id, CancellationToken ct = default) + { + using var response = await _httpClient.GetAsync($"api/customers/{Uri.EscapeDataString(id)}", ct).ConfigureAwait(false); + if (response.StatusCode == System.Net.HttpStatusCode.NotFound) + return Result.NotFoundError(); + + return await response.WithValidator(_validator).ToResultAsync(ct).ConfigureAwait(false); + } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridContent.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridContent.cs new file mode 100644 index 000000000..6bb15b1a1 --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridContent.cs @@ -0,0 +1,17 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Represents a SendGrid v3 content object (see ). +/// +public class SendGridContent +{ + /// + /// Gets or sets the MIME type of the content (e.g. text/plain or text/html). + /// + public string? Type { get; set; } + + /// + /// Gets or sets the actual content. + /// + public string? Value { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridEmailAddress.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridEmailAddress.cs new file mode 100644 index 000000000..508ee7b60 --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridEmailAddress.cs @@ -0,0 +1,17 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Represents a SendGrid v3 email object (see ). +/// +public class SendGridEmailAddress +{ + /// + /// Gets or sets the email address. + /// + public string? Email { get; set; } + + /// + /// Gets or sets the display name. + /// + public string? Name { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridHttpClient.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridHttpClient.cs new file mode 100644 index 000000000..649fa693e --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridHttpClient.cs @@ -0,0 +1,21 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Provides the HTTP facade for interacting with the external SendGrid v3 Mail Send API (see ). +/// +/// The . +public class SendGridHttpClient(HttpClient httpClient) +{ + private readonly HttpClient _httpClient = httpClient.ThrowIfNull(); + + /// + /// Sends the specified mail . + /// + /// The . + /// A successful send returns (202) with no response body. + public async Task SendMailAsync(SendGridMailRequest request, CancellationToken ct = default) + { + var response = await _httpClient.PostAsJsonAsync("v3/mail/send", request, JsonDefaults.SerializerOptions, ct).ConfigureAwait(false); + return await response.ToResultAsync(ct).ConfigureAwait(false); // Handles the response and returns errors/exceptions as expected. + } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridMailRequest.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridMailRequest.cs new file mode 100644 index 000000000..4892c1a2e --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridMailRequest.cs @@ -0,0 +1,22 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Represents the SendGrid v3 POST /v3/mail/send request body (see ). +/// +public class SendGridMailRequest +{ + /// + /// Gets or sets the personalization(s) - i.e. the recipient(s) and per-recipient subject. + /// + public List? Personalizations { get; set; } + + /// + /// Gets or sets the sender. + /// + public SendGridEmailAddress? From { get; set; } + + /// + /// Gets or sets the content(s) of the email. + /// + public List? Content { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridOptions.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridOptions.cs new file mode 100644 index 000000000..a6231473f --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridOptions.cs @@ -0,0 +1,23 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Represents the configuration options for sending mail via the SendGrid API. +/// +public class SendGridOptions +{ + /// + /// Gets or sets the sending API key. + /// + /// Not used directly by ; applied as the Authorization header when the underlying is registered. + public string? ApiKey { get; set; } + + /// + /// Gets or sets the "from" email address used for all outgoing mail. + /// + public string? FromEmail { get; set; } + + /// + /// Gets or sets the "from" display name used for all outgoing mail. + /// + public string? FromName { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridPersonalization.cs b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridPersonalization.cs new file mode 100644 index 000000000..8f81098cd --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Clients/SendGrid/SendGridPersonalization.cs @@ -0,0 +1,17 @@ +namespace Contoso.Shopping.Infrastructure.Clients.SendGrid; + +/// +/// Represents a SendGrid v3 personalization object (see ). +/// +public class SendGridPersonalization +{ + /// + /// Gets or sets the recipient(s) of the email. + /// + public List? To { get; set; } + + /// + /// Gets or sets the subject of the email. + /// + public string? Subject { get; set; } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/GlobalUsing.cs b/samples/src/Contoso.Shopping.Infrastructure/GlobalUsing.cs index 03d721ccf..57b3df860 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/GlobalUsing.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/GlobalUsing.cs @@ -1,11 +1,18 @@ +global using Contoso.Shopping.Application.Adapters.Customers; +global using Contoso.Shopping.Application.Adapters.Notifications; global using Contoso.Shopping.Application.Adapters.Products; global using Contoso.Shopping.Application.Repositories; global using Contoso.Shopping.Infrastructure.Adapters.Products; +global using Contoso.Shopping.Infrastructure.Clients.Customers; global using Contoso.Shopping.Infrastructure.Clients.Products; +global using Contoso.Shopping.Infrastructure.Clients.SendGrid; global using Contoso.Shopping.Infrastructure.Mapping; global using Contoso.Shopping.Infrastructure.Repositories; global using CoreEx; +global using CoreEx.Caching; +global using CoreEx.Data; global using CoreEx.Data.Models; +global using CoreEx.Data.Querying; global using CoreEx.Database; global using CoreEx.Database.SqlServer; global using CoreEx.Database.SqlServer.Outbox; @@ -19,8 +26,11 @@ global using CoreEx.Json; global using CoreEx.Mapping; global using CoreEx.Results; +global using CoreEx.Validation; global using Microsoft.EntityFrameworkCore; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Logging; +global using Microsoft.Extensions.Options; global using System.Net.Http.Json; +global using System.Text.Json; global using System.Text.Json.Serialization; diff --git a/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketStatusMapper.g.cs b/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketStatusMapper.g.cs index 99995c412..a04debb53 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketStatusMapper.g.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketStatusMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Shopping.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class BasketStatusMapper : Mapper +/// Provides mapping from to . +internal partial class BasketStatusMapper : BiDirectionMapper { /// protected override Contracts.BasketStatus OnMap(Persistence.BasketStatus source) @@ -31,8 +31,11 @@ protected override Contracts.BasketStatus OnMap(Persistence.BasketStatus source) return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.BasketStatus source, Contracts.BasketStatus destination); + + /// + protected override Persistence.BasketStatus OnMap(Contracts.BasketStatus destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketSummaryMapper.cs b/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketSummaryMapper.cs new file mode 100644 index 000000000..5b420a45f --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Mapping/BasketSummaryMapper.cs @@ -0,0 +1,23 @@ +namespace Contoso.Shopping.Infrastructure.Mapping; + +/// +/// Maps the to a summary (list-oriented) ; i.e. excludes the and . +/// +public class BasketSummaryMapper : Mapper +{ + protected override Contracts.Basket OnMap(Persistence.Basket source) => new() + { + Id = source.Id, + CustomerId = source.CustomerId, + StatusCode = source.BasketStatusCode, + Pricing = new Contracts.BasketPricing + { + SubTotal = source.SubTotal, + DiscountCouponCode = source.DiscountCouponCode, + DiscountAmount = source.DiscountAmount, + Total = source.Total + }, + ChangeLog = ChangeLog.CreateFrom(source), + ETag = source.ETag + }; +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Mapping/DiscountCouponMapper.g.cs b/samples/src/Contoso.Shopping.Infrastructure/Mapping/DiscountCouponMapper.g.cs index 1469338f5..a1366e3bf 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Mapping/DiscountCouponMapper.g.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Mapping/DiscountCouponMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Shopping.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class DiscountCouponMapper : Mapper +/// Provides mapping from to . +internal partial class DiscountCouponMapper : BiDirectionMapper { /// protected override Contracts.DiscountCoupon OnMap(Persistence.DiscountCoupon source) @@ -32,8 +32,11 @@ protected override Contracts.DiscountCoupon OnMap(Persistence.DiscountCoupon sou return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.DiscountCoupon source, Contracts.DiscountCoupon destination); + + /// + protected override Persistence.DiscountCoupon OnMap(Contracts.DiscountCoupon destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Shopping.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs b/samples/src/Contoso.Shopping.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs index 83329cddb..627dc4f26 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Mapping/UnitOfMeasureMapper.g.cs @@ -8,8 +8,8 @@ namespace Contoso.Shopping.Infrastructure.Mapping; -/// Provides mapping from to . -internal partial class UnitOfMeasureMapper : Mapper +/// Provides mapping from to . +internal partial class UnitOfMeasureMapper : BiDirectionMapper { /// protected override Contracts.UnitOfMeasure OnMap(Persistence.UnitOfMeasure source) @@ -32,8 +32,11 @@ protected override Contracts.UnitOfMeasure OnMap(Persistence.UnitOfMeasure sourc return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend(Persistence.UnitOfMeasure source, Contracts.UnitOfMeasure destination); + + /// + protected override Persistence.UnitOfMeasure OnMap(Contracts.UnitOfMeasure destination) => throw new NotSupportedException(); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketQueryArgsConfig.cs b/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketQueryArgsConfig.cs new file mode 100644 index 000000000..5045ae9a5 --- /dev/null +++ b/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketQueryArgsConfig.cs @@ -0,0 +1,24 @@ +namespace Contoso.Shopping.Infrastructure.Repositories; + +/// +/// Provides the configuration for . +/// +/// The is not a query field; it is always applied by the . +public class BasketQueryArgsConfig : QueryArgsConfig +{ + private const string _pricingTotal = $"{nameof(Contracts.Basket.Pricing)}.{nameof(Contracts.BasketPricing.Total)}"; + + public BasketQueryArgsConfig() + { + // Configure the query arguments for filtering baskets. + WithFilter(filter => filter + .AddReferenceDataField("Status", nameof(Persistence.Basket.BasketStatusCode)) + .AddField(_pricingTotal, nameof(Persistence.Basket.Total), c => c.WithOperators(QueryFilterOperator.ComparisonOperators))); + + // Configure the query arguments for ordering baskets; CreatedOn is always the final order-by to ensure consistent sequencing. + WithOrderBy(orderby => orderby + .AddField(nameof(Contracts.Basket.Status), nameof(Persistence.Basket.BasketStatusCode)) + .AddField(_pricingTotal, nameof(Persistence.Basket.Total)) + .AddField(nameof(Persistence.Basket.CreatedOn), c => c.WithDefault().WithAlwaysInclude())); + } +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketRepository.cs b/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketRepository.cs index 2d824ef74..9d4f2e4f7 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketRepository.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Repositories/BasketRepository.cs @@ -29,6 +29,17 @@ public class BasketRepository(ShoppingEfDb ef) : IBasketRepository .ThenAsAsync(model => _ef.Baskets.UpdateWithResultAsync(model, ct)) .ThenAs(basket => BasketMapper.Map(basket)); + /// + public Task QuerySchemaAsync(CancellationToken ct = default) => Task.FromResult(BasketQueryArgsConfig.Default.ToJsonSchema()); + + /// + public async Task> QueryAsync(string customerId, QueryArgs? query, PagingArgs? paging, CancellationToken ct = default) + { + var parsed = BasketQueryArgsConfig.Default.Parse(query).ThrowOnError(); + var baskets = _ef.Baskets.Query().IgnoreAutoIncludes().Where(b => b.CustomerId == customerId); + return await baskets.Where(parsed).OrderBy(parsed).ToMappedItemsResultAsync(b => BasketSummaryMapper.Map(b), paging, cancellationToken: ct).ConfigureAwait(false); + } + /// /// Synchronize the items between the domain and model, ensuring the appropriate EntityState is set for each item based on its PersistenceState. /// @@ -68,4 +79,4 @@ public class BasketRepository(ShoppingEfDb ef) : IBasketRepository return model; } -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Infrastructure/Repositories/ReferenceDataRepository.g.cs b/samples/src/Contoso.Shopping.Infrastructure/Repositories/ReferenceDataRepository.g.cs index 09fd8fea2..13636f0e2 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Repositories/ReferenceDataRepository.g.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Repositories/ReferenceDataRepository.g.cs @@ -14,15 +14,15 @@ public partial class ReferenceDataRepository : IReferenceDataRepository { /// public Task GetAllBasketStatusesAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(BasketStatusMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(BasketStatusMapper.From, cancellationToken); /// public Task GetAllDiscountCouponsAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(DiscountCouponMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(DiscountCouponMapper.From, cancellationToken); /// public Task GetAllUnitsOfMeasureAsync(CancellationToken cancellationToken = default) - => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(UnitOfMeasureMapper.Map, cancellationToken); + => _ef.ThrowIfNull().Model().Query().ToMappedItemsAsync(UnitOfMeasureMapper.From, cancellationToken); } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/samples/src/Contoso.Shopping.Infrastructure/Repositories/ShoppingDbContext.g.cs b/samples/src/Contoso.Shopping.Infrastructure/Repositories/ShoppingDbContext.g.cs index 07b13bcb7..79aef8d95 100644 --- a/samples/src/Contoso.Shopping.Infrastructure/Repositories/ShoppingDbContext.g.cs +++ b/samples/src/Contoso.Shopping.Infrastructure/Repositories/ShoppingDbContext.g.cs @@ -24,10 +24,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.Text).HasColumnName("Text").HasColumnType("NVARCHAR(250)"); e.Property(p => p.IsActive).HasColumnName("IsActive").HasColumnType("BIT"); e.Property(p => p.SortOrder).HasColumnName("SortOrder").HasColumnType("INT"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -45,10 +45,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.DiscountPercentage).HasColumnName("DiscountPercentage").HasColumnType("DECIMAL(18, 2)"); e.Property(p => p.StartsOn).HasColumnName("StartsOn").HasColumnType("DATETIMEOFFSET"); e.Property(p => p.EndsOn).HasColumnName("EndsOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description); }); @@ -64,10 +64,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.IsActive).HasColumnName("IsActive").HasColumnType("BIT"); e.Property(p => p.SortOrder).HasColumnName("SortOrder").HasColumnType("INT"); e.Property(p => p.Scale).HasColumnName("Scale").HasColumnType("INT"); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); e.Ignore(p => p.Description).Ignore(p => p.StartsOn).Ignore(p => p.EndsOn); }); @@ -85,10 +85,10 @@ partial void AddGeneratedModels(Microsoft.EntityFrameworkCore.ModelBuilder model e.Property(p => p.DiscountAmount).HasColumnName("DiscountAmount").HasColumnType("DECIMAL(18, 2)"); e.Property(p => p.Total).HasColumnName("Total").HasColumnType("DECIMAL(18, 2)"); e.Property(p => p.ShippingAddress).HasColumnName("ShippingAddressJson").HasColumnType("NVARCHAR(2000)").HasConversion(TypeToJsonStringEfConverter.Default, TypeToJsonStringEfComparer.Default); - e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET"); - e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)"); - e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET"); + e.Property(p => p.CreatedBy).HasColumnName("CreatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnUpdate(); + e.Property(p => p.CreatedOn).HasColumnName("CreatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnUpdate(); + e.Property(p => p.UpdatedBy).HasColumnName("UpdatedBy").HasColumnType("NVARCHAR(250)").ValueGeneratedOnAdd(); + e.Property(p => p.UpdatedOn).HasColumnName("UpdatedOn").HasColumnType("DATETIMEOFFSET").ValueGeneratedOnAdd(); e.Property(p => p.ETag).HasColumnName("RowVersion").HasColumnType("TIMESTAMP").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); }); diff --git a/samples/src/Contoso.Shopping.Relay/appsettings.json b/samples/src/Contoso.Shopping.Relay/appsettings.json index d73f602d7..c5aaf76fc 100644 --- a/samples/src/Contoso.Shopping.Relay/appsettings.json +++ b/samples/src/Contoso.Shopping.Relay/appsettings.json @@ -8,10 +8,21 @@ "OutboxRelay": { "BatchSize": 10, "PerWorkerPartitionCount": 2, - "LeaseDuration": "00:00:05", + "LeaseDuration": "00:00:30", "BackoffDuration": "00:00:05", "ServicesCount": 4 } + }, + "Outbox": { + "PartitionSize": 4 + }, + "ServiceBus": { + "SessionIdPartitionSize": 4, + "Destinations": { + "contoso-products": { + "SessionIdPartitionSize": 2 + } + } } } }, @@ -22,4 +33,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Subscribe/GlobalUsing.cs b/samples/src/Contoso.Shopping.Subscribe/GlobalUsing.cs index 08ab16b21..e0b05c8a2 100644 --- a/samples/src/Contoso.Shopping.Subscribe/GlobalUsing.cs +++ b/samples/src/Contoso.Shopping.Subscribe/GlobalUsing.cs @@ -1,5 +1,7 @@ +global using Contoso.Shopping.Application.Adapters.Notifications; global using Contoso.Shopping.Application.Adapters.Products; global using Contoso.Shopping.Application.Validators; +global using Contoso.Shopping.Contracts; global using CoreEx; global using CoreEx.DependencyInjection; global using CoreEx.Events; diff --git a/samples/src/Contoso.Shopping.Subscribe/Program.cs b/samples/src/Contoso.Shopping.Subscribe/Program.cs index 717dfc168..e3b824174 100644 --- a/samples/src/Contoso.Shopping.Subscribe/Program.cs +++ b/samples/src/Contoso.Shopping.Subscribe/Program.cs @@ -1,5 +1,7 @@ using Contoso.Shopping.Application; +using Contoso.Shopping.Infrastructure.Clients.Customers; using Contoso.Shopping.Infrastructure.Clients.Products; +using Contoso.Shopping.Infrastructure.Clients.SendGrid; using Contoso.Shopping.Infrastructure.Repositories; using Contoso.Shopping.Subscribe.Subscribers; using CoreEx.Azure.Messaging.ServiceBus; @@ -7,6 +9,7 @@ using OpenTelemetry; using OpenTelemetry.Trace; using StackExchange.Redis; +using System.Net.Http.Headers; using ZiggyCreatures.Caching.Fusion; using ZiggyCreatures.Caching.Fusion.Backplane.StackExchangeRedis; @@ -32,7 +35,7 @@ private static void Main(string[] args) .AddHostedServiceManager(); // Add all the dynamically registered services. - builder.Services.AddDynamicServicesUsing(); + builder.Services.AddDynamicServicesUsing(); // Add L1/L2 caching services. builder.Services.AddMemoryCache(); // Adds the in-memory cache - L1. @@ -70,6 +73,7 @@ private static void Main(string[] args) // Add event formatter and subscribed-manager. builder.Services .AddEventFormatter() // Adds the EventFormatter to enable message parsing. + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider; events to the shared topic, commands to per-domain queues. .AddSubscribedManager((_, c) => c.AddSubscribersUsing()); // Adds the SubscribedManager and dynamically links to the individual Subscribers. // Creates the Azure Service Bus receiving services builder. @@ -86,6 +90,9 @@ private static void Main(string[] args) // Add external API services. builder.AddTypedHttpClient("ProductsApi"); + builder.AddTypedHttpClient("CustomersApi"); + builder.AddTypedHttpClient("SendGrid", client => client.DefaultRequestHeaders.Authorization = new AuthenticationHeaderValue("Bearer", builder.Configuration["SendGrid:ApiKey"])); + builder.Services.Configure(builder.Configuration.GetSection("SendGrid")); // Post-configure all health-checks; adds the standard tags. builder.Services.PostConfigureAllHealthChecks(); diff --git a/samples/src/Contoso.Shopping.Subscribe/Subscribers/BasketCheckedOutSubscriber.cs b/samples/src/Contoso.Shopping.Subscribe/Subscribers/BasketCheckedOutSubscriber.cs new file mode 100644 index 000000000..78861892c --- /dev/null +++ b/samples/src/Contoso.Shopping.Subscribe/Subscribers/BasketCheckedOutSubscriber.cs @@ -0,0 +1,11 @@ +namespace Contoso.Shopping.Subscribe.Subscribers; + +[ScopedService] +[Subscribe("contoso.shopping.basket.checkedout.v1")] +public class BasketCheckedOutSubscriber(INotificationAdapter adapter) : SubscribedBase +{ + private readonly INotificationAdapter _adapter = adapter.ThrowIfNull(); + + protected override Task OnReceiveAsync(Basket value, EventData @event, EventSubscriberArgs args, CancellationToken cancellationToken = default) + => _adapter.SendBasketCheckedOutConfirmationAsync(value, cancellationToken); +} diff --git a/samples/src/Contoso.Shopping.Subscribe/Subscribers/ProductModifySubscriber.cs b/samples/src/Contoso.Shopping.Subscribe/Subscribers/ProductModifySubscriber.cs index 070019f5c..3ec627d96 100644 --- a/samples/src/Contoso.Shopping.Subscribe/Subscribers/ProductModifySubscriber.cs +++ b/samples/src/Contoso.Shopping.Subscribe/Subscribers/ProductModifySubscriber.cs @@ -1,8 +1,7 @@ namespace Contoso.Shopping.Subscribe.Subscribers; [ScopedService] -[Subscribe("contoso.products.product.created.v1")] -[Subscribe("contoso.products.product.updated.v1")] +[Subscribe("contoso.products.product.*.v1")] public class ProductModifySubscriber(IProductSyncAdapter adapter) : SubscribedBase { private readonly IProductSyncAdapter _adapter = adapter.ThrowIfNull(); @@ -11,4 +10,4 @@ public class ProductModifySubscriber(IProductSyncAdapter adapter) : SubscribedBa protected override Task OnReceiveAsync(Product value, EventData @event, EventSubscriberArgs args, CancellationToken cancellationToken = default) => _adapter.ModifyAsync(value); -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Subscribe/appsettings.Development.json b/samples/src/Contoso.Shopping.Subscribe/appsettings.Development.json index 690b25e57..19908408a 100644 --- a/samples/src/Contoso.Shopping.Subscribe/appsettings.Development.json +++ b/samples/src/Contoso.Shopping.Subscribe/appsettings.Development.json @@ -7,7 +7,9 @@ "ZiggyCreature": "Warning", "StackExchange": "Warning", "Azure.Messaging.ServiceBus": "Critical", - "CoreEx.Azure.Messaging.ServiceBus": "Warning" + "CoreEx.Azure.Messaging.ServiceBus": "Warning", + "System.Net.Http.HttpClient": "Warning", + "Polly": "Warning" } }, "Aspire": { @@ -39,5 +41,8 @@ }, "ProductsApi": { "BaseAddress": "https://localhost:7200" + }, + "CustomersApi": { + "BaseAddress": "https://localhost:7320" } -} \ No newline at end of file +} diff --git a/samples/src/Contoso.Shopping.Subscribe/appsettings.json b/samples/src/Contoso.Shopping.Subscribe/appsettings.json index 6c943a3ce..00a07dc2c 100644 --- a/samples/src/Contoso.Shopping.Subscribe/appsettings.json +++ b/samples/src/Contoso.Shopping.Subscribe/appsettings.json @@ -2,12 +2,29 @@ "CoreEx": { "Host": { "SolutionName": "Contoso", - "DomainName": "Shopping" + "DomainName": "Shopping", + "Outbox": { + "PartitionSize": 4 + }, + "ServiceBus": { + "SessionIdPartitionSize": 4, + "Destinations": { + "contoso-products": { + "SessionIdPartitionSize": 2 + } + } + } }, "Events": { "Destination": "contoso" // Topic/queue name. } }, + "SendGrid": { + "BaseAddress": "https://api.sendgrid.com/", + "FromEmail": "orders@contoso.com", + "FromName": "Contoso Shopping" + // "ApiKey" is intentionally not set here; supply via user-secrets or an environment variable in each environment. + }, "Logging": { "LogLevel": { "Default": "Information", @@ -15,4 +32,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj b/samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj index 248c889cb..713ca2f60 100644 --- a/samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj +++ b/samples/tests/Contoso.Customers.Test.Api/Contoso.Customers.Test.Api.csproj @@ -20,9 +20,11 @@ + + PreserveNewest diff --git a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Patch.cs b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Patch.cs index b577225a4..22ffbe153 100644 --- a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Patch.cs +++ b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Patch.cs @@ -22,7 +22,7 @@ public void Patch_Concurrency() // Act/Assert. Test.Http() - .Run(HttpMethod.Patch, $"/api/customers/{c.Id}", new { lastName = "Updated" }, requestModifier: r => r.WithIfMatch("AAAAAAAA").WithMergePatchJsonContentType()) + .Run(HttpMethod.Patch, $"/api/customers/{c.Id}", new { lastName = "Updated" }, requestModifier: r => r.WithIfMatch("00000000-0000-0000-0000-000000000000").WithMergePatchJsonContentType()) .AssertPreconditionFailed(); } diff --git a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Update.cs b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Update.cs index d7c1c6cc5..9ea92ca96 100644 --- a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Update.cs +++ b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.Update.cs @@ -28,9 +28,9 @@ public void Update_Concurrency() c.LastName += " Updated"; - // Act/Assert. + // Act/Assert. The stale ETag must be GUID-shaped: the vNext emulator ignores a malformed If-Match inside a TransactionalBatch. Test.Http() - .Run(HttpMethod.Put, $"/api/customers/{c.Id}", c, requestModifier: r => r.WithIfMatch("AAAAAAAA")) + .Run(HttpMethod.Put, $"/api/customers/{c.Id}", c, requestModifier: r => r.WithIfMatch("00000000-0000-0000-0000-000000000000")) .AssertPreconditionFailed(); } diff --git a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.cs b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.cs index 7bd802fd5..545cb8766 100644 --- a/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.cs +++ b/samples/tests/Contoso.Customers.Test.Api/CustomerMutateTests.cs @@ -5,7 +5,7 @@ public partial class CustomerMutateTests : WithApiTester(["mutate-data.seed.yaml"], Contoso.Customers.Database.Program.ConfigureProvisionArgs).ConfigureAwait(false); await Test.ClearFusionCacheAsync().ConfigureAwait(false); Test.UseExpectedCosmosDbOutboxPublisher(); diff --git a/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.Query.cs b/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.Query.cs index ac09c1e7d..b71dfcb90 100644 --- a/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.Query.cs +++ b/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.Query.cs @@ -117,7 +117,7 @@ public void Query_OrderBy_LastName_Desc() [Test] public void Query_OrderBy_FirstName() { - // LastName is configured WithAlwaysInclude(), so this is a two-property "ORDER BY firstName, lastName" under the hood - requires the composite index DatabaseSetUp configures on "customers". + // LastName is configured WithAlwaysInclude(), so this is a two-property "ORDER BY firstName, lastName" under the hood - requires the composite index Contoso.Customers.Database configures on "customers". var r = Test.Http() .Run(HttpMethod.Get, "/api/customers?$orderby=firstname") .AssertOK() diff --git a/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.cs b/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.cs index af7972677..53500cfce 100644 --- a/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.cs +++ b/samples/tests/Contoso.Customers.Test.Api/CustomerReadTests.cs @@ -5,7 +5,7 @@ public partial class CustomerReadTests : WithApiTester(["read-data.seed.yaml"], Contoso.Customers.Database.Program.ConfigureProvisionArgs).ConfigureAwait(false); await Test.ClearFusionCacheAsync().ConfigureAwait(false); } } diff --git a/samples/tests/Contoso.Customers.Test.Api/DatabaseSetUp.cs b/samples/tests/Contoso.Customers.Test.Api/DatabaseSetUp.cs deleted file mode 100644 index dce061480..000000000 --- a/samples/tests/Contoso.Customers.Test.Api/DatabaseSetUp.cs +++ /dev/null @@ -1,48 +0,0 @@ -namespace Contoso.Customers.Test.Api; - -internal static class DatabaseSetUp -{ - /// The - the is resolved from its running host's own registration (see - /// ), guaranteeing this always seeds the exact database/containers the API host itself reads from. - /// Zero or more embedded YAML resource names (see ) imported via - /// . - public static async Task DatabaseSetUpAsync(this TesterBase tester, params string[] resourceFileNames) - { - var database = await tester.GetCosmosDatabaseAsync().ConfigureAwait(false); - - // Replace or create "customers" container used by the API. The partition key is assumed to be "/partitionKey" for all containers, which is a common pattern for Cosmos DB. - // CustomerQueryArgsConfig's "LastName" order-by field is configured WithAlwaysInclude() (always appended, in its own default ascending direction, regardless of what the caller actually - // requested) - so ordering by "FirstName" always produces a two-property ORDER BY (e.g. "firstName DESC, lastName ASC"), which Cosmos DB rejects outright unless a matching composite index - // exists. Both direction combinations actually reachable via the API ($orderby=firstname[,desc]) are indexed here so CustomerReadTests' query tests can exercise both. - var customersContainerProperties = new ContainerProperties("customers", "/partitionKey"); - customersContainerProperties.IndexingPolicy.CompositeIndexes.Add( - [ - new() { Path = "/firstName", Order = CompositePathSortOrder.Ascending }, - new() { Path = "/lastName", Order = CompositePathSortOrder.Ascending } - ]); - customersContainerProperties.IndexingPolicy.CompositeIndexes.Add( - [ - new() { Path = "/firstName", Order = CompositePathSortOrder.Descending }, - new() { Path = "/lastName", Order = CompositePathSortOrder.Ascending } - ]); - - await database.ReplaceOrCreateContainerAsync(customersContainerProperties).ConfigureAwait(false); - - // Seed any known precondition rows (e.g. mutate-data.seed.yaml) directly into the freshly reset container - no TModel typing involved, raw JSON straight through. - foreach (var fileName in resourceFileNames) - { - var customerJdr = JsonDataReader.ParseYaml(fileName, new JsonDataReaderOptions(JsonPropertyNamingConvention.CamelCase)); - await CosmosDbBatch.ImportBatchAsync(database, customerJdr).ConfigureAwait(false); - } - - // Replace or create "ref-data" container used by the API. Reuse the "test" configured reference data and import. A unique key policy on "/typeDiscriminator" and "/code" enforces - // (per logical partition) that no two documents share the same discriminator/code combination - the closest Cosmos DB equivalent to a unique index. - var refDataContainerProperties = new ContainerProperties("ref-data", "/partitionKey"); - refDataContainerProperties.UniqueKeyPolicy.UniqueKeys.Add(new UniqueKey { Paths = { "/typeDiscriminator", "/code" } }); - - await database.ReplaceOrCreateContainerAsync(refDataContainerProperties).ConfigureAwait(false); - - var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.CamelCase)); - await CosmosDbBatch.ImportDiscriminatedBatchAsync(database, jdr).ConfigureAwait(false); - } -} diff --git a/samples/tests/Contoso.Customers.Test.Api/GlobalUsing.cs b/samples/tests/Contoso.Customers.Test.Api/GlobalUsing.cs index fddb1732f..335e4ad5c 100644 --- a/samples/tests/Contoso.Customers.Test.Api/GlobalUsing.cs +++ b/samples/tests/Contoso.Customers.Test.Api/GlobalUsing.cs @@ -1,16 +1,20 @@ +global using AwesomeAssertions; global using Contoso.Customers.Contracts; global using CoreEx; +global using CoreEx.Caching; +global using CoreEx.Cosmos.Extended; global using CoreEx.Data.Json; global using CoreEx.Http.Abstractions; -global using CoreEx.Cosmos.Extended; -global using AwesomeAssertions; global using Microsoft.Azure.Cosmos; +global using Microsoft.Extensions.DependencyInjection; global using NUnit.Framework; +global using System.Collections.Concurrent; global using System.Collections.ObjectModel; +global using System.Diagnostics; global using System.Net; global using System.Net.Http; global using System.Text.Json; +global using TestData = Contoso.Customers.Test.Common.TestData; global using UnitTestEx; global using UnitTestEx.Abstractions; global using UnitTestEx.Expectations; -global using TestData = Contoso.Customers.Test.Common.TestData; diff --git a/samples/tests/Contoso.Customers.Test.Api/HostTests.cs b/samples/tests/Contoso.Customers.Test.Api/HostTests.cs index 08c7fcdfe..589b5560d 100644 --- a/samples/tests/Contoso.Customers.Test.Api/HostTests.cs +++ b/samples/tests/Contoso.Customers.Test.Api/HostTests.cs @@ -3,7 +3,11 @@ namespace Contoso.Customers.Test.Api; public partial class HostTests : WithApiTester { [OneTimeSetUp] - public Task OneTimeSetUpAsync() => Test.DatabaseSetUpAsync(); + public async Task OneTimeSetUpAsync() + { + await Test.MigrateCosmosDataAsync(configure: Contoso.Customers.Database.Program.ConfigureProvisionArgs).ConfigureAwait(false); + await Test.ClearFusionCacheAsync().ConfigureAwait(false); + } [Test] public void Swagger_UI() @@ -48,14 +52,14 @@ public void Health(string path) [TestCase("/health/ready/detailed", false)] public void Health_Detailed(string path, bool minimal) { - // Both registered against HealthCheckTags.StartUpAndReadyOnly (not Live), matching AddHostedService's own convention - liveness reflects the process, not downstream dependencies. - string[] paths = ["$.entries.reference-data-orchestrator", "$.entries.cosmos-database"]; + // Liveness reflects the process, not downstream dependencies. + string[] paths = ["$.entries.reference-data-orchestrator", "$.entries.cosmos-database", "$.entries['stackExchange.Redis']"]; var r = Test.Http() .Run(HttpMethod.Get, path) .AssertContentTypeJson(); - r.Response.StatusCode.Should().BeOneOf(HttpStatusCode.OK, HttpStatusCode.ServiceUnavailable); + r.Response.StatusCode.Should().Be(HttpStatusCode.OK); var json = r.GetContent().Should().BeJson(); if (minimal) @@ -63,4 +67,42 @@ public void Health_Detailed(string path, bool minimal) else json.ContainAll(paths); } + + [Test] + public void Redis_DistributedCache_RoundTrip() + { + Test.ScopedType(async test => + { + var cache = test.Services.GetRequiredService(); + var key = $"redis-host-test-{Runtime.NewId()}"; + var options = new HybridCacheEntryOptions { Strategy = CacheStrategy.Distributed, DistributedExpiration = TimeSpan.FromMinutes(1) }; + try + { + await cache.SetByKeyAsync(key, "Customers Redis", options).ConfigureAwait(false); + var (exists, value) = await cache.TryGetByKeyAsync(key, options).ConfigureAwait(false); + exists.Should().BeTrue(); + value.Should().Be("Customers Redis"); + } + finally + { + await cache.RemoveByKeyAsync(key, options).ConfigureAwait(false); + } + }); + } + + [Test] + public void Health_Ready_EmitsCosmosDependencySpan() + { + var activities = new ConcurrentQueue(); + using var listener = new ActivityListener + { + ShouldListenTo = source => source.Name == "Azure.Cosmos.Operation", + Sample = (ref ActivityCreationOptions _) => ActivitySamplingResult.AllDataAndRecorded, + ActivityStopped = activity => activities.Enqueue(activity) + }; + ActivitySource.AddActivityListener(listener); + + Test.Http().Run(HttpMethod.Get, "/health/ready").AssertOK(); + activities.Should().Contain(a => a.DisplayName.StartsWith("read_database", StringComparison.Ordinal)); + } } diff --git a/samples/tests/Contoso.Customers.Test.Api/RefDataMutateTests.cs b/samples/tests/Contoso.Customers.Test.Api/RefDataMutateTests.cs new file mode 100644 index 000000000..88282d6b6 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Api/RefDataMutateTests.cs @@ -0,0 +1,529 @@ +namespace Contoso.Customers.Test.Api; + +public class RefDataMutateTests : WithApiTester +{ + private const string _url = "/api/refdata/customer-types"; + + [OneTimeSetUp] + public async Task OneTimeSetUpAsync() + { + await Test.MigrateCosmosDataAsync(["mutate-data.seed.yaml"], Contoso.Customers.Database.Program.ConfigureProvisionArgs).ConfigureAwait(false); + await Test.ClearFusionCacheAsync().ConfigureAwait(false); + + Test.UseExpectedCosmosDbOutboxPublisher(); + } + + /// + /// Creates a new (inactive) customer type with the specified . + /// + private CustomerType CreateCustomerType(string code) + => Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Post, _url, new CustomerType { Code = code, Text = $"Customer type {code}" }) + .AssertCreated() + .Value!; + + /// + /// Creates a new customer type with the specified and activates it. + /// + private CustomerType CreateActiveCustomerType(string code) + { + var b = CreateCustomerType(code); + return Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Post, $"{_url}/{b.Id}/activate") + .AssertOK() + .Value!; + } + + private CustomerType[] GetCustomerTypes(bool includeInactive = false) + => Test.Http() + .Run(HttpMethod.Get, includeInactive ? $"{_url}?$inactive=true" : _url) + .AssertOK() + .Value!; + + #region Get + + [Test] + public void Get_NotFound() + { + Test.Http() + .Run(HttpMethod.Get, $"{_url}/404") + .AssertNotFound(); + } + + [Test] + public void Get_Success() + { + // Arrange - the seeded identifier is not assumed; get via the list. + var b = GetCustomerTypes().Single(x => x.Code == "IND"); + + // Act/Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertOK() + .AssertValue(b); + } + + #endregion + + #region Create + + [Test] + public void Create_Empty() + { + Test.Http() + .Run(HttpMethod.Post, _url, new CustomerType()) + .AssertBadRequest() + .AssertErrors( + "Code is required.", + "Text is required." + ); + } + + [Test] + public void Create_Bad_Data() + { + var b = new CustomerType + { + Code = new string('X', CoreEx.Validation.ReferenceDataValidator.CodeMaximumLength + 1), + Text = new string('X', CoreEx.Validation.ReferenceDataValidator.TextMaximumLength + 1), + Description = new string('X', CoreEx.Validation.ReferenceDataValidator.DescriptionMaximumLength + 1), + }; + + Test.Http() + .Run(HttpMethod.Post, _url, b) + .AssertBadRequest() + .AssertErrors( + $"Code must not exceed {CoreEx.Validation.ReferenceDataValidator.CodeMaximumLength} character(s) in length.", + $"Text must not exceed {CoreEx.Validation.ReferenceDataValidator.TextMaximumLength} character(s) in length.", + $"Description must not exceed {CoreEx.Validation.ReferenceDataValidator.DescriptionMaximumLength} character(s) in length." + ); + } + + [Test] + public void Create_Duplicate() + { + Test.Http() + .Run(HttpMethod.Post, _url, new CustomerType { Code = "IND", Text = "Yeti Again" }) + .AssertConflict(); + } + + [Test] + public void Create_Success() + { + // Act/Assert - a newly created reference data value is always inactive until explicitly activated. + var b = Test.Http() + .ExpectIdentifier() + .ExpectETag() + .ExpectCosmosDbOutboxEvents(e => e.AssertWithValue("contoso", "contoso.customers.customertype.created.v1")) + .Run(HttpMethod.Post, _url, new CustomerType { Code = "CREATE-OK", Text = "Create OK", SortOrder = 5 }) + .AssertCreated() + .AssertLocationHeader(r => new Uri($"{_url}/{r!.Id}", UriKind.Relative)) + .Value!; + + b.Code.Should().Be("CREATE-OK"); + b.Text.Should().Be("Create OK"); + b.SortOrder.Should().Be(5); + b.IsInactive.Should().BeTrue(); + + // Assert - get by identifier. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertOK() + .AssertValue(b); + + // Assert - not within the (default) active list, but is within the inactive-inclusive list. + GetCustomerTypes().Should().NotContain(x => x.Code == "CREATE-OK"); + GetCustomerTypes(includeInactive: true).Should().Contain(x => x.Code == "CREATE-OK"); + } + + [Test] + public void Create_IdempotencyKey() + { + var b = new CustomerType { Code = "CREATE-IDEM", Text = "Create Idempotent" }; + var ik = Guid.NewGuid().ToString(); + + // Act/Assert - first request creates and emits an event. + var v1 = Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Post, _url, b, requestModifier: r => r.WithIdempotencyKey(ik)) + .AssertCreated() + .Value!; + + // Assert - repeat with the same idempotency key; same result and no further event emitted. + var v2 = Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Post, _url, b, requestModifier: r => r.WithIdempotencyKey(ik)) + .AssertCreated() + .Value!; + + ObjectComparer.Assert(v1, v2); + } + + #endregion + + #region Patch + + [Test] + public void Patch_NotFound() + { + Test.Http() + .Run(HttpMethod.Patch, $"{_url}/404", new { text = "abc" }, requestModifier: r => r.WithMergePatchJsonContentType()) + .AssertNotFound(); + } + + [Test] + public void Patch_Concurrency() + { + var b = CreateCustomerType("PATCH-CONC"); + + Test.Http() + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { text = "Changed" }, requestModifier: r => r.WithIfMatch("00000000-0000-0000-0000-000000000000").WithMergePatchJsonContentType()) + .AssertPreconditionFailed(); + } + + [Test] + public void Patch_Validation() + { + var b = CreateCustomerType("PATCH-VAL"); + + Test.Http() + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { text = string.Empty }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertBadRequest() + .AssertErrors("Text is required."); + } + + [Test] + public void Patch_Success() + { + var b = CreateCustomerType("PATCH-OK"); + + // Act/Assert. + var u = Test.Http() + .ExpectCosmosDbOutboxEvents(e => e.AssertWithValue("contoso", "contoso.customers.customertype.updated.v1")) + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { text = "Patched Text", sortOrder = 9 }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.Id.Should().Be(b.Id); + u.Code.Should().Be(b.Code); + u.Text.Should().Be("Patched Text"); + u.SortOrder.Should().Be(9); + u.IsInactive.Should().BeTrue(); + u.ETag.Should().NotBe(b.ETag); + + // Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertOK() + .AssertValue(u); + } + + [Test] + public void Patch_Code_Immutable() + { + var b = CreateCustomerType("PATCH-CODE"); + + // Act/Assert - the code is silently retained; only the text is updated. + var u = Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { code = "PATCH-CODE-CHANGED", text = "Code Retained" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.Code.Should().Be("PATCH-CODE"); + u.Text.Should().Be("Code Retained"); + } + + [Test] + public void Patch_IsInactive_Ignored() + { + // The active state can only be changed via activate/deactivate. + var b = CreateCustomerType("PATCH-ACTIVE"); + + var u = Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { isInactive = false, text = "Still Inactive" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.IsInactive.Should().BeTrue(); + u.Text.Should().Be("Still Inactive"); + } + + [Test] + public void Patch_NoChanges() + { + var b = CreateCustomerType("PATCH-NONE"); + + // Act/Assert - nothing changed so no event and the etag remains. + var u = Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Patch, $"{_url}/{b.Id}", new { }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.ETag.Should().Be(b.ETag); + u.Text.Should().Be(b.Text); + } + + #endregion + + #region Put + + [Test] + public void Put_NotFound() + { + Test.Http() + .Run(HttpMethod.Put, $"{_url}/404", new { code = "PUT-404", text = "abc" }, requestModifier: r => r.WithIfMatch("abc")) + .AssertNotFound(); + } + + [Test] + public void Put_MissingETag() + { + Test.Http() + .Run(HttpMethod.Put, $"{_url}/404", new { code = "PUT-404", text = "abc" }) + .Assert(System.Net.HttpStatusCode.PreconditionRequired); + } + + [Test] + public void Put_Concurrency() + { + var b = CreateCustomerType("PUT-CONC"); + + Test.Http() + .Run(HttpMethod.Put, $"{_url}/{b.Id}", new { code = b.Code, text = "Changed" }, requestModifier: r => r.WithIfMatch("00000000-0000-0000-0000-000000000000")) + .AssertPreconditionFailed(); + } + + [Test] + public void Put_Validation() + { + var b = CreateCustomerType("PUT-VAL"); + + Test.Http() + .Run(HttpMethod.Put, $"{_url}/{b.Id}", new { code = b.Code, text = string.Empty }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertBadRequest() + .AssertErrors("Text is required."); + } + + [Test] + public void Put_Success() + { + var b = CreateCustomerType("PUT-OK"); + + // Act/Assert. + var u = Test.Http() + .ExpectCosmosDbOutboxEvents(e => e.AssertWithValue("contoso", "contoso.customers.customertype.updated.v1")) + .Run(HttpMethod.Put, $"{_url}/{b.Id}", new { code = b.Code, text = "Put Text", sortOrder = 9 }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.Id.Should().Be(b.Id); + u.Code.Should().Be(b.Code); + u.Text.Should().Be("Put Text"); + u.SortOrder.Should().Be(9); + u.IsInactive.Should().BeTrue(); + u.ETag.Should().NotBe(b.ETag); + + // Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertOK() + .AssertValue(u); + } + + [Test] + public void Put_Code_And_IsInactive_Ignored() + { + // The code is immutable and the active state can only be changed via activate/deactivate. + var b = CreateCustomerType("PUT-IGNORED"); + + var u = Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Put, $"{_url}/{b.Id}", new { code = "PUT-IGNORED-CHANGED", text = "Retained", isInactive = false }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.Code.Should().Be("PUT-IGNORED"); + u.IsInactive.Should().BeTrue(); + u.Text.Should().Be("Retained"); + } + + [Test] + public void Put_NoChanges() + { + var b = CreateCustomerType("PUT-NONE"); + + // Act/Assert - nothing changed so no event and the etag remains. + var u = Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Put, $"{_url}/{b.Id}", new { code = b.Code, text = b.Text, sortOrder = b.SortOrder }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.ETag.Should().Be(b.ETag); + } + + #endregion + + #region Activate + + [Test] + public void Activate_NotFound() + { + Test.Http() + .Run(HttpMethod.Post, $"{_url}/404/activate") + .AssertNotFound(); + } + + [Test] + public void Activate_Success() + { + var b = CreateCustomerType("ACTIVATE-OK"); + + // Act/Assert. + var a = Test.Http() + .ExpectCosmosDbOutboxEvents(e => e.AssertWithValue("contoso", "contoso.customers.customertype.activated.v1")) + .Run(HttpMethod.Post, $"{_url}/{b.Id}/activate") + .AssertOK() + .Value!; + + a.IsInactive.Should().BeFalse(); + a.ETag.Should().NotBe(b.ETag); + + // Assert - now within the (default) active list. + GetCustomerTypes().Should().Contain(x => x.Code == "ACTIVATE-OK"); + } + + [Test] + public void Activate_AlreadyActive() + { + var b = CreateActiveCustomerType("ACTIVATE-AGAIN"); + + // Act/Assert - already active so no change and no event. + var a = Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Post, $"{_url}/{b.Id}/activate") + .AssertOK() + .Value!; + + a.IsInactive.Should().BeFalse(); + a.ETag.Should().Be(b.ETag); + } + + #endregion + + #region Deactivate + + [Test] + public void Deactivate_NotFound() + { + Test.Http() + .Run(HttpMethod.Post, $"{_url}/404/deactivate") + .AssertNotFound(); + } + + [Test] + public void Deactivate_Success() + { + var b = CreateActiveCustomerType("DEACTIVATE-OK"); + GetCustomerTypes().Should().Contain(x => x.Code == "DEACTIVATE-OK"); + + // Act/Assert. + var d = Test.Http() + .ExpectCosmosDbOutboxEvents(e => e.AssertWithValue("contoso", "contoso.customers.customertype.deactivated.v1")) + .Run(HttpMethod.Post, $"{_url}/{b.Id}/deactivate") + .AssertOK() + .Value!; + + d.IsInactive.Should().BeTrue(); + d.ETag.Should().NotBe(b.ETag); + + // Assert - no longer within the (default) active list, but still exists. + GetCustomerTypes().Should().NotContain(x => x.Code == "DEACTIVATE-OK"); + GetCustomerTypes(includeInactive: true).Should().Contain(x => x.Code == "DEACTIVATE-OK"); + } + + [Test] + public void Deactivate_AlreadyInactive() + { + var b = CreateCustomerType("DEACTIVATE-AGAIN"); + + // Act/Assert - already inactive so no change and no event. + var d = Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Post, $"{_url}/{b.Id}/deactivate") + .AssertOK() + .Value!; + + d.IsInactive.Should().BeTrue(); + d.ETag.Should().Be(b.ETag); + } + + #endregion + + #region Delete + + [Test] + public void Delete_NotFound() + { + // Delete is idempotent. + Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Delete, $"{_url}/404") + .AssertNoContent(); + } + + [Test] + public void Delete_IsActive() + { + var b = CreateActiveCustomerType("DELETE-ACTIVE"); + + Test.Http() + .Run(HttpMethod.Delete, $"{_url}/{b.Id}") + .AssertBadRequest() + .AssertProblemDetails(p => p.Title.Should().Be("An active reference data value cannot be deleted.")); + + // Assert - still exists. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertOK(); + } + + [Test] + public void Delete_Success() + { + var b = CreateActiveCustomerType("DELETE-OK"); + GetCustomerTypes().Should().Contain(x => x.Code == "DELETE-OK"); + + // Arrange - must be deactivated before it can be deleted. + Test.Http() + .ExpectCosmosDbOutboxEvents() + .Run(HttpMethod.Post, $"{_url}/{b.Id}/deactivate") + .AssertOK(); + + // Act. + Test.Http() + .ExpectCosmosDbOutboxEvents(e => e.AssertMetadata("contoso", "contoso.customers.customertype.deleted", b.Id!)) + .Run(HttpMethod.Delete, $"{_url}/{b.Id}") + .AssertNoContent(); + + // Assert - idempotent. + Test.Http() + .ExpectNoCosmosDbOutboxEvents() + .Run(HttpMethod.Delete, $"{_url}/{b.Id}") + .AssertNoContent(); + + // Assert - gone, including from the cached reference data. + Test.Http() + .Run(HttpMethod.Get, $"{_url}/{b.Id}") + .AssertNotFound(); + + GetCustomerTypes(includeInactive: true).Should().NotContain(x => x.Code == "DELETE-OK"); + } + + #endregion +} diff --git a/samples/tests/Contoso.Customers.Test.Common/Contoso.Customers.Test.Common.csproj b/samples/tests/Contoso.Customers.Test.Common/Contoso.Customers.Test.Common.csproj index ab86e2019..f2f60a85b 100644 --- a/samples/tests/Contoso.Customers.Test.Common/Contoso.Customers.Test.Common.csproj +++ b/samples/tests/Contoso.Customers.Test.Common/Contoso.Customers.Test.Common.csproj @@ -1,16 +1,10 @@ - - - true - - diff --git a/samples/tests/Contoso.Customers.Test.Common/GlobalUsing.cs b/samples/tests/Contoso.Customers.Test.Common/GlobalUsing.cs new file mode 100644 index 000000000..b845e92f1 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Common/GlobalUsing.cs @@ -0,0 +1,3 @@ +global using Azure.Messaging.ServiceBus; +global using Azure.Messaging.ServiceBus.Administration; +global using System.Text.Json; diff --git a/samples/tests/Contoso.Customers.Test.Common/ServiceBus.cs b/samples/tests/Contoso.Customers.Test.Common/ServiceBus.cs new file mode 100644 index 000000000..627323c35 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Common/ServiceBus.cs @@ -0,0 +1,66 @@ +namespace Contoso.Customers.Test.Common; + +public static class ServiceBus +{ + public const string ObservationSubscription = "customers-relay"; + + public static (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[] GetTopicsAndSubscriptions() => + [ + (new CreateTopicOptions("contoso"), + [ + new CreateSubscriptionOptions("contoso", "shopping") { RequiresSession = true }, + new CreateSubscriptionOptions("contoso", "products") { RequiresSession = true }, + new CreateSubscriptionOptions("contoso", ObservationSubscription) { RequiresSession = true } + ]) + ]; + + public static async Task> ReceiveEventsAsync(ServiceBusClient client, IReadOnlyCollection<(string Subject, string Type)> expected, CancellationToken cancellationToken = default) + { + var messages = new Dictionary<(string Subject, string Type), ServiceBusReceivedMessage>(); + using var timeout = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); + timeout.CancelAfter(TimeSpan.FromSeconds(90)); + + try + { + while (expected.Any(key => !messages.ContainsKey(key))) + { + ServiceBusSessionReceiver session; + try + { + session = await client.AcceptNextSessionAsync("contoso", ObservationSubscription, cancellationToken: timeout.Token).ConfigureAwait(false); + } + catch (ServiceBusException ex) when (ex.Reason == ServiceBusFailureReason.ServiceTimeout) + { + continue; + } + + await using (session) + { + while (true) + { + var batch = await session.ReceiveMessagesAsync(50, TimeSpan.FromSeconds(1), timeout.Token).ConfigureAwait(false); + if (batch.Count == 0) + break; + + foreach (var message in batch) + { + using var json = JsonDocument.Parse(message.Body); + var subject = json.RootElement.GetProperty("subject").GetString() ?? throw new InvalidOperationException("A relayed event has no subject."); + var type = json.RootElement.GetProperty("type").GetString() ?? throw new InvalidOperationException("A relayed event has no type."); + var key = (subject, type); + messages.TryAdd(key, message); + + await session.CompleteMessageAsync(message, timeout.Token).ConfigureAwait(false); + } + } + } + } + } + catch (OperationCanceledException ex) when (!cancellationToken.IsCancellationRequested && timeout.IsCancellationRequested) + { + throw new TimeoutException($"Customers relay did not deliver: {string.Join(", ", expected.Where(key => !messages.ContainsKey(key)))}.", ex); + } + + return messages; + } +} diff --git a/samples/tests/Contoso.Customers.Test.Relay/ClientInstrumentationTests.cs b/samples/tests/Contoso.Customers.Test.Relay/ClientInstrumentationTests.cs new file mode 100644 index 000000000..8b7ef5953 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Relay/ClientInstrumentationTests.cs @@ -0,0 +1,69 @@ +namespace Contoso.Customers.Test.Relay; + +public class ClientInstrumentationTests +{ + [Test] + public async Task ClientBackgroundRefresh_IsSuppressed_WhileForegroundReadIsNot() + { + var configuration = new ConfigurationBuilder().SetBasePath(AppContext.BaseDirectory) + .AddJsonFile("appsettings.Development.json").AddEnvironmentVariables().Build(); + var connectionString = configuration.GetConnectionString("Cosmos") ?? throw new InvalidOperationException("The Cosmos connection string is missing."); + var requests = new ConcurrentQueue<(string Path, bool Suppressed)>(); + var spans = new ConcurrentQueue(); + using var telemetry = Sdk.CreateTracerProviderBuilder() + .AddHttpClientInstrumentation() + .AddProcessor(new SimpleActivityExportProcessor(new CapturingExporter(spans))) + .Build(); + var services = new ServiceCollection(); + services.AddSingleton(_ => CosmosDbClientFactory.Create(connectionString, options => + { + options.HttpClientFactory = () => new HttpClient(new RecordingHandler(requests)); + })); + services.SuppressCosmosClientBackgroundInstrumentation(); + using var provider = services.BuildServiceProvider(); + + var client = provider.GetRequiredService(); + Sdk.SuppressInstrumentation.Should().BeFalse(); + await client.GetDatabase("contoso").ReadAsync().ConfigureAwait(false); + requests.Should().Contain(r => r.Path == "/dbs/contoso" && !r.Suppressed); + requests.Where(r => r.Path == "/").Should().NotBeEmpty().And.OnlyContain(r => r.Suppressed); + spans.Should().NotBeEmpty(); + + // Exercise the SDK's real five-minute timer, not only its initial account discovery. + var initialAccountReads = requests.Count(r => r.Path == "/"); + var initialSpanCount = spans.Count; + var timeout = Stopwatch.StartNew(); + while (requests.Count(r => r.Path == "/") == initialAccountReads && timeout.Elapsed < TimeSpan.FromSeconds(320)) + await Task.Delay(TimeSpan.FromSeconds(1)).ConfigureAwait(false); + + requests.Count(r => r.Path == "/").Should().BeGreaterThan(initialAccountReads); + requests.Where(r => r.Path == "/").Should().OnlyContain(r => r.Suppressed); + spans.Count.Should().Be(initialSpanCount); + } + + private sealed class RecordingHandler(ConcurrentQueue<(string Path, bool Suppressed)> requests) : DelegatingHandler(new HttpClientHandler + { + ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator + }) + { + protected override async Task SendAsync(HttpRequestMessage request, CancellationToken cancellationToken) + { + var path = request.RequestUri?.AbsolutePath ?? throw new InvalidOperationException("The request URL is missing."); + var suppressed = Sdk.SuppressInstrumentation; + var response = await base.SendAsync(request, cancellationToken).ConfigureAwait(false); + requests.Enqueue((path, suppressed)); + return response; + } + } + + private sealed class CapturingExporter(ConcurrentQueue spans) : BaseExporter + { + public override ExportResult Export(in Batch batch) + { + foreach (var span in batch) + spans.Enqueue(span); + + return ExportResult.Success; + } + } +} diff --git a/samples/tests/Contoso.Customers.Test.Relay/Contoso.Customers.Test.Relay.csproj b/samples/tests/Contoso.Customers.Test.Relay/Contoso.Customers.Test.Relay.csproj new file mode 100644 index 000000000..92a222f44 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Relay/Contoso.Customers.Test.Relay.csproj @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + + + + + + PreserveNewest + + + diff --git a/samples/tests/Contoso.Customers.Test.Relay/GlobalUsing.cs b/samples/tests/Contoso.Customers.Test.Relay/GlobalUsing.cs new file mode 100644 index 000000000..0eb89f26c --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Relay/GlobalUsing.cs @@ -0,0 +1,29 @@ +global using AwesomeAssertions; +global using Azure.Messaging.ServiceBus; +global using Broker = Contoso.Customers.Test.Common.ServiceBus; +global using CloudNative.CloudEvents; +global using CloudNative.CloudEvents.Extensions; +global using Contoso.Customers.Infrastructure.Repositories; +global using Contoso.Customers.Relay; +global using CoreEx; +global using CoreEx.Cosmos; +global using CoreEx.Cosmos.Outbox; +global using CoreEx.Cosmos.Provisioning; +global using CoreEx.Entities; +global using CoreEx.Events; +global using CoreEx.Events.Publishing; +global using CoreEx.Json; +global using CoreEx.UnitTesting; +global using Microsoft.Azure.Cosmos; +global using Microsoft.Azure.Cosmos.Linq; +global using Microsoft.Extensions.Configuration; +global using Microsoft.Extensions.DependencyInjection; +global using NUnit.Framework; +global using OpenTelemetry; +global using OpenTelemetry.Trace; +global using PartitionKey = Microsoft.Azure.Cosmos.PartitionKey; +global using System.Collections.Concurrent; +global using System.Diagnostics; +global using System.Net; +global using System.Text.Json; +global using UnitTestEx; diff --git a/samples/tests/Contoso.Customers.Test.Relay/RelayTests.cs b/samples/tests/Contoso.Customers.Test.Relay/RelayTests.cs new file mode 100644 index 000000000..7494ff4b7 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Relay/RelayTests.cs @@ -0,0 +1,228 @@ +namespace Contoso.Customers.Test.Relay; + +public class RelayTests : WithApiTester +{ + private CosmosClient _client = null!; // Initialized by OneTimeSetUp before any test executes. + private readonly List _backlog = []; + private static readonly string[] _services = ["cosmos-outbox-relay-customers-00", "cosmos-outbox-relay-ref-data-00"]; + + [OneTimeSetUp] + public async Task OneTimeSetUpAsync() + { + // Even Test.Configuration constructs Program and captures its start boundary; provision before accessing the tester. + var configuration = new ConfigurationBuilder().SetBasePath(AppContext.BaseDirectory) + .AddJsonFile("appsettings.Development.json").AddJsonFile("appsettings.unittest.json").AddEnvironmentVariables().Build(); + var connectionString = configuration.GetConnectionString("Cosmos") ?? throw new InvalidOperationException("The Cosmos connection string is missing."); + _client = CosmosDbClientFactory.Create(connectionString); + var args = Contoso.Customers.Database.Program.ConfigureProvisionArgs(new CosmosDbProvisionArgs { DatabaseId = "contoso" }); + await new CosmosDbProvisioner(_client, args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData).ConfigureAwait(false); + + var serviceBusConnection = configuration.GetConnectionString("ServiceBus") ?? throw new InvalidOperationException("The Service Bus connection string is missing."); + await UnitTestExExtensions.ResetAzureServiceBusAsync(UnitTestExExtensions.CreateAzureServiceBusAdminConnectionString(serviceBusConnection), topicsAndSubscriptions: Broker.GetTopicsAndSubscriptions()).ConfigureAwait(false); + + _backlog.Add(await PublishAsync("customers").ConfigureAwait(false)); + _backlog.Add(await PublishAsync("ref-data").ConfigureAwait(false)); + // Cosmos change-feed timestamps have second precision; make the pre-start writes unambiguously older. + await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false); + + foreach (var service in _services) + await WaitForStatusAsync(service, "Running").ConfigureAwait(false); + } + + [OneTimeTearDown] + public void OneTimeTearDown() => _client?.Dispose(); + + [Test] + public async Task Outbox_Relay_BothContainers_And_FirstStartBoundary() + { + await SetPausedAsync(true).ConfigureAwait(false); + PublishedEvents customers; + PublishedEvents refData; + try + { + customers = await PublishAsync("customers").ConfigureAwait(false); + refData = await PublishAsync("ref-data").ConfigureAwait(false); + await AssertDocumentsAsync(customers, deleted: false).ConfigureAwait(false); + await AssertDocumentsAsync(refData, deleted: false).ConfigureAwait(false); + } + finally + { + await SetPausedAsync(false).ConfigureAwait(false); + } + + var expected = customers.Events.Concat(refData.Events).ToArray(); + var received = await Broker.ReceiveEventsAsync(Test.Services.GetRequiredService(), expected.Select(EventKey).ToArray()).ConfigureAwait(false); + foreach (var e in expected) + { + var message = received[EventKey(e)]; + message.MessageId.Should().Be(e.Id); + message.SessionId.Should().Be(CoreEx.Data.PartitionKey.GetPartitionIdAsString(e.GetPartitionKey() ?? throw new InvalidOperationException("The test event has no partition key."))); + message.PartitionKey.Should().Be(message.SessionId); + ObjectComparer.AssertJson(e.EncodeToJsonElement().ToString(), message.Body.ToString()); + } + + await AssertDocumentsAsync(customers, deleted: true).ConfigureAwait(false); + await AssertDocumentsAsync(refData, deleted: true).ConfigureAwait(false); + foreach (var backlog in _backlog) + { + received.Keys.Should().NotContain(key => key.Subject == backlog.BusinessId); + await AssertDocumentsAsync(backlog, deleted: false).ConfigureAwait(false); + } + } + + [Test] + public async Task Outbox_Relay_ExplicitPartitionKey() + { + await SetPausedAsync(true).ConfigureAwait(false); + PublishedEvents events; + try + { + events = await PublishAsync("customers", Runtime.NewId()).ConfigureAwait(false); + } + finally + { + await SetPausedAsync(false).ConfigureAwait(false); + } + + var received = await Broker.ReceiveEventsAsync(Test.Services.GetRequiredService(), events.Events.Select(EventKey).ToArray()).ConfigureAwait(false); + foreach (var e in events.Events) + { + var message = received[EventKey(e)]; + message.MessageId.Should().Be(e.Id); + message.SessionId.Should().Be(CoreEx.Data.PartitionKey.GetPartitionIdAsString(events.PartitionKey ?? throw new InvalidOperationException())); + ObjectComparer.AssertJson(e.EncodeToJsonElement().ToString(), message.Body.ToString()); + } + + await AssertDocumentsAsync(events, deleted: true).ConfigureAwait(false); + } + + [Test] + public async Task Health_And_HostedServices() + { + foreach (var path in new[] { "/health/live", "/health/startup", "/health/ready" }) + { + var watch = Stopwatch.StartNew(); + while (Test.Http().Run(HttpMethod.Get, path).Response.StatusCode != HttpStatusCode.OK) + { + if (watch.Elapsed > TimeSpan.FromSeconds(30)) + Assert.Fail($"Relay health endpoint '{path}' did not become healthy."); + await Task.Delay(200).ConfigureAwait(false); + } + } + + Test.Http().Run(HttpMethod.Get, "/health/ready/detailed").AssertOK().GetContent() + .Should().ContainAll(["cosmos-database", .. _services]); + + await SetPausedAsync(true).ConfigureAwait(false); + try + { + using var health = JsonDocument.Parse(Test.Http().Run(HttpMethod.Get, "/health/ready/detailed").AssertOK().GetContent() ?? throw new InvalidOperationException("The health response is empty.")); + foreach (var service in _services) + health.RootElement.GetProperty("entries").GetProperty(service).GetProperty("status").GetString().Should().Be("Degraded"); + } + finally + { + await SetPausedAsync(false).ConfigureAwait(false); + } + } + + private async Task SetPausedAsync(bool paused) + { + foreach (var service in _services) + { + Test.Http().Run(HttpMethod.Post, $"/hosted-services/{service}/{(paused ? "pause" : "resume")}").Response.StatusCode.Should().Be(HttpStatusCode.Accepted); + await WaitForStatusAsync(service, paused ? "Paused" : "Running").ConfigureAwait(false); + } + } + + private async Task WaitForStatusAsync(string service, string status) + { + var watch = Stopwatch.StartNew(); + while (true) + { + var actual = Test.Http().Run(HttpMethod.Get, $"/hosted-services/{service}/status").AssertOK().Value; + if (actual == status) + return; + if (watch.Elapsed > TimeSpan.FromSeconds(30)) + Assert.Fail($"Relay service '{service}' stayed '{actual}', expected '{status}'."); + await Task.Delay(200).ConfigureAwait(false); + } + } + + private async Task PublishAsync(string containerId, string? partitionKey = null) + { + var db = new CustomersCosmosDb(_client, "contoso"); + var publisher = new CosmosDbEventPublisher(db); + var unitOfWork = new CosmosDbUnitOfWork(db, publisher); + var id = Runtime.NewId(); + var type = containerId == "customers" ? "customer" : "customertype"; + CloudEvent[] events = [CreateEvent("created"), CreateEvent("updated")]; + + CloudEvent CreateEvent(string action) + { + var e = new CloudEvent + { + Id = Runtime.NewId(), + Source = new Uri("https://contoso/customers"), + Type = $"contoso.customers.{type}.{action}.v1", + Subject = id, + Time = Runtime.UtcNow, + DataContentType = "application/json", + Data = JsonSerializer.SerializeToElement(new { id, text = "Relay test" }) + }; + e.SetPartitionKey(partitionKey ?? id); + return e; + } + + await unitOfWork.TransactionAsync(async ct => + { + if (containerId == "customers") + { + var model = new Infrastructure.Persistence.Customer { Id = id, PartitionKey = partitionKey, FirstName = "Relay", LastName = "Test", Email = "relay@example.com" }; + var container = db.Container(containerId, o => o.WithPartitionKey(m => m.PartitionKey)); + await container.CreateAsync(model, ct).ConfigureAwait(false); + } + else + { + await db.CustomerTypes.CreateAsync(new Infrastructure.Persistence.CustomerType { Id = id, Code = id, Text = "Relay test", TypeDiscriminator = "CustomerType" }, ct).ConfigureAwait(false); + } + + publisher.Add("contoso", events); + }).ConfigureAwait(false); + + var documents = new List(); + using var iterator = db.GetContainer(containerId).GetItemLinqQueryable().Where(e => e.Id.StartsWith(CosmosDbOutboxEvent.OutboxKeyPrefix)).ToFeedIterator(); + while (iterator.HasMoreResults) + documents.AddRange(await iterator.ReadNextAsync().ConfigureAwait(false)); + var matching = documents.Where(d => d.Event.GetProperty("subject").GetString() == id).ToArray(); + matching.Should().HaveCount(events.Length); + return new PublishedEvents(containerId, id, partitionKey, events, matching); + } + + private async Task AssertDocumentsAsync(PublishedEvents events, bool deleted) + { + var container = _client.GetDatabase("contoso").GetContainer(events.ContainerId); + var partitionKey = events.PartitionKey is null ? PartitionKey.None : new PartitionKey(events.PartitionKey); + var watch = Stopwatch.StartNew(); + foreach (var document in events.Documents) + { + while (true) + { + using var response = await container.ReadItemStreamAsync(document.Id, partitionKey).ConfigureAwait(false); + if (response.StatusCode == (deleted ? HttpStatusCode.NotFound : HttpStatusCode.OK)) + break; + if (!deleted || watch.Elapsed > TimeSpan.FromSeconds(30)) + Assert.Fail($"Outbox document '{document.Id}' returned {response.StatusCode}; expected {(deleted ? "cleanup deletion" : "retention")}."); + await Task.Delay(100).ConfigureAwait(false); + } + } + + using var business = await container.ReadItemStreamAsync(events.BusinessId, partitionKey).ConfigureAwait(false); + business.StatusCode.Should().Be(HttpStatusCode.OK); + } + + private static (string Subject, string Type) EventKey(CloudEvent e) + => (e.Subject ?? throw new InvalidOperationException("The test event has no subject."), e.Type ?? throw new InvalidOperationException("The test event has no type.")); + + private sealed record PublishedEvents(string ContainerId, string BusinessId, string? PartitionKey, CloudEvent[] Events, CosmosDbOutboxEvent[] Documents); +} diff --git a/samples/tests/Contoso.Customers.Test.Relay/appsettings.unittest.json b/samples/tests/Contoso.Customers.Test.Relay/appsettings.unittest.json new file mode 100644 index 000000000..580c0bad3 --- /dev/null +++ b/samples/tests/Contoso.Customers.Test.Relay/appsettings.unittest.json @@ -0,0 +1,38 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "CoreEx.Cosmos.Outbox.CosmosDbOutboxRelayHostedService": "Warning", + "Microsoft": "Warning", + "Microsoft.Extensions.Diagnostics.HealthChecks": "Error", + "Azure": "Warning" + } + }, + "CoreEx.AspNetCore.HealthChecks": { + "AreDetailedEndpointsEnabled": true + }, + "CoreEx": { + "Host": { + "Services": { + "CosmosOutboxRelay": { + "customers": { "ServicesCount": 1 }, + "ref-data": { "ServicesCount": 1 } + } + } + } + }, + "Aspire": { + "Azure": { + "Messaging": { + "ServiceBus": { + "ClientOptions": { + "RetryOptions": { + "MaxRetries": 0, + "TryTimeout": "00:00:01" + } + } + } + } + } + } +} diff --git a/samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj b/samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj index 5686d3e28..997ec2ae4 100644 --- a/samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj +++ b/samples/tests/Contoso.Customers.Test.Unit/Contoso.Customers.Test.Unit.csproj @@ -15,6 +15,7 @@ + diff --git a/samples/tests/Contoso.Customers.Test.Unit/EntryPoint.cs b/samples/tests/Contoso.Customers.Test.Unit/EntryPoint.cs index 6f07ef9db..0d27feb26 100644 --- a/samples/tests/Contoso.Customers.Test.Unit/EntryPoint.cs +++ b/samples/tests/Contoso.Customers.Test.Unit/EntryPoint.cs @@ -9,16 +9,16 @@ public static void ConfigureApplication(IHostApplicationBuilder builder) // Configure the minimum services required for the execution context and reference data orchestrator; caching will be in-memory for the unit tests. builder.Services.AddExecutionContext(); builder.Services.AddMemoryCache(); - builder.Services.AddReferenceDataOrchestrator(); + builder.Services.AddReferenceDataOrchestrator(); // Reuse the "test" configured reference data. - var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.CamelCase)); - builder.Services.AddSingleton(new ReferenceDataServiceDecorator(jdr)); + var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.CamelCase)); + builder.Services.AddSingleton(new ReferenceDataProviderDecorator(jdr)); } // TODO: temporary hard-coded stand-in values only, pending the real Cosmos-backed seed data fixture (see CustomersCosmosDb/ReferenceDataRepository) - replace once that seed data exists. - public class ReferenceDataServiceDecorator(JsonDataReader jdr) : ReferenceDataService(Mock.Of()) + public class ReferenceDataProviderDecorator(JsonDataReader jdr) : ReferenceDataProvider(Mock.Of()) { public override Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch { diff --git a/samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj b/samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj index 8172b1b1c..ee3a474cc 100644 --- a/samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj +++ b/samples/tests/Contoso.E2E.Runner/Contoso.E2E.Runner.csproj @@ -5,6 +5,8 @@ enable enable preview + + true @@ -19,6 +21,9 @@ + + + diff --git a/samples/tests/Contoso.E2E.Runner/GlobalUsing.cs b/samples/tests/Contoso.E2E.Runner/GlobalUsing.cs index 26a0d4456..5a45b36da 100644 --- a/samples/tests/Contoso.E2E.Runner/GlobalUsing.cs +++ b/samples/tests/Contoso.E2E.Runner/GlobalUsing.cs @@ -2,7 +2,10 @@ global using Contoso.E2E.Runner.Scenarios; global using Contoso.Products.Contracts; global using Contoso.Shopping.Contracts; +global using Contoso.Customers.Contracts; global using CoreEx; +global using CoreEx.Cosmos; +global using CoreEx.Cosmos.Provisioning; global using CoreEx.Json; global using DbEx; global using DbEx.Migration; @@ -14,4 +17,4 @@ global using System.Diagnostics; global using System.Net.Http.Json; global using System.Reflection; -global using System.Text; \ No newline at end of file +global using System.Text; diff --git a/samples/tests/Contoso.E2E.Runner/Infrastructure/LoadSimulationRunner.cs b/samples/tests/Contoso.E2E.Runner/Infrastructure/LoadSimulationRunner.cs index 49db237fa..555ad7e65 100644 --- a/samples/tests/Contoso.E2E.Runner/Infrastructure/LoadSimulationRunner.cs +++ b/samples/tests/Contoso.E2E.Runner/Infrastructure/LoadSimulationRunner.cs @@ -227,7 +227,7 @@ private Panel CreateDisplay(string spinnerFrame) foreach (var scenario in _context.Scenarios.Keys) { var scenarioStatistics = _scenarioStatistics[scenario]; - var workers = _config.Simulations[scenario].Parallelism; + var workers = _config.Simulations.GetValueOrDefault(scenario, new LoadSimulationSimulatorConfig()).Parallelism; totalWorkers += workers; table.AddRow( diff --git a/samples/tests/Contoso.E2E.Runner/Infrastructure/ScenarioRunner.cs b/samples/tests/Contoso.E2E.Runner/Infrastructure/ScenarioRunner.cs index 697097ec7..ee6bd1ad6 100644 --- a/samples/tests/Contoso.E2E.Runner/Infrastructure/ScenarioRunner.cs +++ b/samples/tests/Contoso.E2E.Runner/Infrastructure/ScenarioRunner.cs @@ -47,9 +47,18 @@ private static void DisplayResults(string title, List results) .AddColumn(new TableColumn("[bold]Duration[/]").RightAligned()) .AddColumn(new TableColumn("[bold]Details[/]").LeftAligned()); + var hasErrorLinks = false; + foreach (var result in results) { var status = result.Success ? "[green]✓ PASS[/]" : "[red]✗ FAIL[/]"; + + if (!result.Success && TryWriteErrorFile(title, result) is { } errorUri) + { + status = $"[red link={errorUri.EscapeMarkup()}]✗ FAIL[/]"; + hasErrorLinks = true; + } + var duration = $"{result.Duration.TotalMilliseconds:F0}ms"; var details = result.Details.Length > 200 ? result.Details[..197] + "..." : result.Details; @@ -63,6 +72,9 @@ private static void DisplayResults(string title, List results) AnsiConsole.Write(table); + if (hasErrorLinks) + AnsiConsole.MarkupLine("[grey]Ctrl+click a FAIL to view the error details.[/]"); + var successCount = results.Count(r => r.Success); var totalCount = results.Count; var successRate = totalCount > 0 ? (double)successCount / totalCount * 100 : 0; @@ -74,4 +86,33 @@ private static void DisplayResults(string title, List results) AnsiConsole.Write(rule); AnsiConsole.WriteLine(); } + + /// + /// Writes the full error detail of a failed step to a temp file and returns its file:// URI (or where there is nothing to write or the write fails). + /// + private static string? TryWriteErrorFile(string title, StepResult result) + { + try + { + var dir = Path.Combine(Path.GetTempPath(), "Contoso.E2E.Runner", "errors"); + Directory.CreateDirectory(dir); + + var name = string.Concat(result.Name.Select(c => char.IsLetterOrDigit(c) ? c : '-')).Trim('-'); + var path = Path.Combine(dir, $"{DateTime.Now:yyyyMMdd-HHmmss-fff}-{(name.Length > 60 ? name[..60] : name)}.txt"); + + var content = new StringBuilder() + .AppendLine($"Scenario: {title}") + .AppendLine($"Step: {result.Name}") + .AppendLine($"Duration: {result.Duration.TotalMilliseconds:F0}ms") + .AppendLine() + .AppendLine(result.Exception?.ToString() ?? result.Details); + + File.WriteAllText(path, content.ToString()); + return new Uri(path).AbsoluteUri; + } + catch (Exception) + { + return null; + } + } } \ No newline at end of file diff --git a/samples/tests/Contoso.E2E.Runner/Infrastructure/TestContext.cs b/samples/tests/Contoso.E2E.Runner/Infrastructure/TestContext.cs index 24873c2dc..3c50c82c1 100644 --- a/samples/tests/Contoso.E2E.Runner/Infrastructure/TestContext.cs +++ b/samples/tests/Contoso.E2E.Runner/Infrastructure/TestContext.cs @@ -17,6 +17,7 @@ public TestContext(IConfiguration config) ProductsHttpClient = new HttpClient { BaseAddress = new Uri(config["E2E:Products:BaseAddress"] ?? throw new InvalidOperationException("E2E:Products:BaseAddress configuration value is missing.")), Timeout = TimeSpan.FromSeconds(30) }; ShoppingHttpClient = new HttpClient { BaseAddress = new Uri(config["E2E:Shopping:BaseAddress"] ?? throw new InvalidOperationException("E2E:Shopping:BaseAddress configuration value is missing.")), Timeout = TimeSpan.FromSeconds(30) }; + CustomersHttpClient = new HttpClient { BaseAddress = new Uri(config["E2E:Customers:BaseAddress"] ?? throw new InvalidOperationException("E2E:Customers:BaseAddress configuration value is missing.")), Timeout = TimeSpan.FromSeconds(30) }; OrdersHttpClient = new HttpClient { BaseAddress = new Uri(config["E2E:Orders:BaseAddress"] ?? throw new InvalidOperationException("E2E:Orders:BaseAddress configuration value is missing.")), Timeout = TimeSpan.FromSeconds(30) }; PerStepMinDelayMilliseconds = config.GetValue("E2E:PerStepMinDelayMilliseconds"); @@ -41,6 +42,11 @@ public TestContext(IConfiguration config) /// public HttpClient ShoppingHttpClient { get; } + /// + /// Gets the "Customers" domain HTTP client configured with the base address and timeout specified in the configuration. + /// + public HttpClient CustomersHttpClient { get; } + /// /// Gets the "Orders" domain HTTP client configured with the base address and timeout specified in the configuration. /// @@ -81,4 +87,4 @@ public static async Task HealthCheckAsync(HttpClient client) return false; } } -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.E2E.Runner/Program.cs b/samples/tests/Contoso.E2E.Runner/Program.cs index e465dca45..f12e861ae 100644 --- a/samples/tests/Contoso.E2E.Runner/Program.cs +++ b/samples/tests/Contoso.E2E.Runner/Program.cs @@ -9,7 +9,7 @@ .Build(); var context = new TestContext(config); -(bool ProductApiOk, bool ShoppingApiOk, bool OrdersApiOk) status = (false, false, false); +(bool ProductApiOk, bool ShoppingApiOk, bool CustomersApiOk, bool OrdersApiOk) status = (false, false, false, false); /***** Main program and choice pump. *****/ @@ -67,6 +67,7 @@ void DisplayBannerAndConfig() new Panel( new Markup($"{(status.ProductApiOk ? "[green]:check_mark:[/] " : "[red]:cross_mark:[/]")} [grey]Products API:[/] {context.ProductsHttpClient.BaseAddress?.ToString().EscapeMarkup()}\n" + $"{(status.ShoppingApiOk ? "[green]:check_mark:[/] " : "[red]:cross_mark:[/]")} [grey]Shopping API:[/] {context.ShoppingHttpClient.BaseAddress?.ToString().EscapeMarkup()}\n" + + $"{(status.CustomersApiOk ? "[green]:check_mark:[/] " : "[red]:cross_mark:[/]")} [grey]Customers API:[/] {context.CustomersHttpClient.BaseAddress?.ToString().EscapeMarkup()}\n" + $"{(status.OrdersApiOk ? "[green]:check_mark:[/] " : "[red]:cross_mark:[/]")} [grey]Orders API:[/] {context.OrdersHttpClient.BaseAddress?.ToString().EscapeMarkup()}")) .Header("[bold]API status:[/]") .BorderColor(Color.Grey) @@ -90,16 +91,17 @@ async Task CheckApiStatusAsync() status = await AnsiConsole.Status() .Spinner(Spinner.Known.Dots) - .StartAsync<(bool ProductApiOk, bool ShoppingApiOk, bool OrdersApiOk)>("[grey]Checking API status... (press [yellow]ESC[/] to cancel)...[/]", async _ => + .StartAsync<(bool ProductApiOk, bool ShoppingApiOk, bool CustomersApiOk, bool OrdersApiOk)>("[grey]Checking API status... (press [yellow]ESC[/] to cancel)...[/]", async _ => { var healthCheckTask = Task.Run(async () => { var productApi = TestContext.HealthCheckAsync(context.ProductsHttpClient); var shoppingApi = TestContext.HealthCheckAsync(context.ShoppingHttpClient); + var customersApi = TestContext.HealthCheckAsync(context.CustomersHttpClient); var ordersApi = TestContext.HealthCheckAsync(context.OrdersHttpClient); - await Task.WhenAll(productApi, shoppingApi, ordersApi); - return (productApi.Result, shoppingApi.Result, ordersApi.Result); + await Task.WhenAll(productApi, shoppingApi, customersApi, ordersApi); + return (productApi.Result, shoppingApi.Result, customersApi.Result, ordersApi.Result); }); // Wait for health check or ESC key @@ -116,12 +118,12 @@ async Task CheckApiStatusAsync() } if (wasCancelled) - return (false, false, false); + return (false, false, false, false); else return await healthCheckTask; }); - if (!status.ProductApiOk || !status.ShoppingApiOk || !status.OrdersApiOk) + if (!status.ProductApiOk || !status.ShoppingApiOk || !status.CustomersApiOk || !status.OrdersApiOk) { DisplayBannerAndConfig(); AnsiConsole.MarkupLine("API(s) are not available :no_bicycles: - please start the Contoso.Aspire application [yellow]:oncoming_fist:[/]"); @@ -129,4 +131,4 @@ async Task CheckApiStatusAsync() } DisplayBannerAndConfig(); -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerQueryScenario.cs b/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerQueryScenario.cs new file mode 100644 index 000000000..2d26748d8 --- /dev/null +++ b/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerQueryScenario.cs @@ -0,0 +1,45 @@ +namespace Contoso.E2E.Runner.Scenarios; + +/// +/// Scenario: Product query lifecycle. +/// +[Scenario("Customers-Query", "Customer Query Lifecycle", 1)] +public class CustomerQueryScenario : IScenario +{ + /// + public async Task RunAsync(ScenarioContext context) + { + var sb = new StringBuilder(); + + var val = Random.Shared.Next(0, 4); + if (val == 0) + { + var customerTypes = await context.StepAsync($"Get all customer types.", async () => + { + var response = await context.TestContext.CustomersHttpClient.GetAsync($"/api/refdata/customer-types"); + return await response.GetValueAsync(); + }, result => "Customer types retrieved successfully."); + + var customerType = customerTypes![Random.Shared.Next(0, customerTypes.Length)]; + sb.Append($"customertype eq '{customerType.Code}'"); + + await ScenarioContext.RandomizedDelayAsync(context); + } + + val = Random.Shared.Next(0, 3); + if (val == 1 || val == 2) + { + if (sb.Length > 0) + sb.Append(" and "); + + sb.Append($"contains(email, 'example')"); + } + + await context.StepAsync($"Query {sb}", async () => + { + var response = await context.TestContext.CustomersHttpClient.GetAsync($"/api/customers?$filter={sb}"); + var customers = await response.GetValueAsync(); + return customers; + }, result => $"Query returned {result!.Length} customers."); + } +} diff --git a/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerUpdateScenario.cs b/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerUpdateScenario.cs new file mode 100644 index 000000000..c9734f0ac --- /dev/null +++ b/samples/tests/Contoso.E2E.Runner/Scenarios/CustomerUpdateScenario.cs @@ -0,0 +1,68 @@ +namespace Contoso.E2E.Runner.Scenarios; + +/// +/// Scenario: Customer update lifecycle. +/// +[Scenario("Customers-Update", "Customer Update Lifecycle", 1)] +public class CustomerUpdateScenario : IScenario +{ + private static readonly SemaphoreSlim _semaphore = new(1); + private CustomerLite[]? _customers; + + /// + /// Gets all the customers (limited to 100) for use in the scenario. + /// + public static async Task GetAllCustomersAsync(ScenarioContext context) + { + var response = await context.TestContext.CustomersHttpClient.GetAsync("/api/customers?$take=100"); + var customers = await response.GetValueAsync(); + return customers ?? []; + } + + /// + public async Task RunAsync(ScenarioContext context) + { + // Step 1: Find all the customers (first time only). + _semaphore.Wait(); + try + { + if (_customers is null) + { + _customers = await context.StepAsync("Find all customers.", async () => + { + return await GetAllCustomersAsync(context); + }, result => $"{result!.Length} customer(s) found."); + + await ScenarioContext.RandomizedDelayAsync(context); + } + } + finally + { + _semaphore.Release(); + } + + // Step 2: Select a random customer and full get. + var index = Random.Shared.Next(0, _customers!.Length - 1); + var c = _customers[index]; + + var customer = await context.StepAsync($"Get: '{c.Id}'.", async () => + { + var response = await context.TestContext.CustomersHttpClient.GetAsync($"/api/customers/{c.Id}"); + return await response.GetValueAsync(); + }, result => "Customer retrieved successfully."); + + await ScenarioContext.RandomizedDelayAsync(context); + + // Step 3: Update customer details. + if (customer!.LastName!.EndsWith(" (updated)")) + customer.LastName = customer.LastName[..^10]; + else + customer.LastName += " (updated)"; + + await context.StepAsync($"Update: '{customer.Id}'.", async () => + { + var response = await context.TestContext.CustomersHttpClient.PutAsJsonAsync($"/api/customers/{customer.Id}", customer, JsonDefaults.SerializerOptions); + return await response.GetValueAsync(); + }, c => $"Customer {c!.Id} updated."); + } +} diff --git a/samples/tests/Contoso.E2E.Runner/Scenarios/DatabaseMigrationSetup.cs b/samples/tests/Contoso.E2E.Runner/Scenarios/DatabaseMigrationSetup.cs index fd874f2a1..88c84b992 100644 --- a/samples/tests/Contoso.E2E.Runner/Scenarios/DatabaseMigrationSetup.cs +++ b/samples/tests/Contoso.E2E.Runner/Scenarios/DatabaseMigrationSetup.cs @@ -1,7 +1,7 @@ namespace Contoso.E2E.Runner.Scenarios; /// -/// Provides scenario setup for performing database migrations and refreshing base data for the Products, Shopping and Orders databases. +/// Provides scenario setup for performing database migrations (or Cosmos DB provisioning) and refreshing base data for the Products, Shopping, Customers and Orders databases. /// [ScenarioSetUp("Database-Migration", "Database Migration and Base Data Refresh", 1, false)] public sealed class DatabaseMigrationSetup : IScenario @@ -13,9 +13,9 @@ public async Task RunAsync(ScenarioContext context) await context.StepAsync("Products database migration.", async () => { var cs = context.TestContext.Config.GetValue("E2E:Products:ConnectionString") ?? throw new InvalidOperationException("E2E:Products:ConnectionString configuration value is missing."); - var ma = new MigrationArgs(MigrationCommand.All | MigrationCommand.ResetAndData, cs); + var ma = new MigrationArgs(MigrationCommand.ResetAndDatabase, cs); Contoso.Products.Database.Program.ConfigureMigrationArgs(ma); - ma.AddAssembly(); + ma.AddAssembly().DataParserArgs.AddNamed("mutate-data.seed.yaml"); using var m = new PostgresMigration(ma); var (Success, Output) = await m.MigrateAndLogAsync().ConfigureAwait(false); @@ -27,9 +27,9 @@ await context.StepAsync("Products database migration.", async () => await context.StepAsync("Shopping database migration.", async () => { var cs = context.TestContext.Config.GetValue("E2E:Shopping:ConnectionString") ?? throw new InvalidOperationException("E2E:Shopping:ConnectionString configuration value is missing."); - var ma = new MigrationArgs(MigrationCommand.All | MigrationCommand.ResetAndData, cs); + var ma = new MigrationArgs(MigrationCommand.ResetAndDatabase, cs); Contoso.Shopping.Database.Program.ConfigureMigrationArgs(ma); - ma.AddAssembly(); + ma.AddAssembly().DataParserArgs.AddNamed("mutate-data.seed.yaml"); using var m = new SqlServerMigration(ma); var (Success, Output) = await m.MigrateAndLogAsync().ConfigureAwait(false); @@ -37,13 +37,28 @@ await context.StepAsync("Shopping database migration.", async () => throw new Exception("Database migration failed:" + Environment.NewLine + Output); }, "Successfully migrated; base data refreshed.").ConfigureAwait(false); - // Step 3: Orders database migration. + // Step 3: Customers database migration. + await context.StepAsync("Customers database migration.", async () => + { + var cs = context.TestContext.Config.GetValue("E2E:Customers:ConnectionString") ?? throw new InvalidOperationException("E2E:Customers:ConnectionString configuration value is missing."); + var databaseId = context.TestContext.Config.GetValue("E2E:Customers:DatabaseId") ?? Contoso.Customers.Database.Program.DefaultDatabaseId; + + var args = new CosmosDbProvisionArgs { DatabaseId = databaseId }; + Contoso.Customers.Database.Program.ConfigureProvisionArgs(args); + args.AddDataResource("read-data.seed.yaml"); + + using var client = CosmosDbClientFactory.Create(cs); + var (Success, Output) = await new CosmosDbProvisioner(client, args).RunAndLogAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData).ConfigureAwait(false); + if (!Success) + throw new Exception("Database provisioning failed:" + Environment.NewLine + Output); + }, "Successfully provisioned; base data refreshed.").ConfigureAwait(false); + + // Step 4: Orders database migration. await context.StepAsync("Orders database migration.", async () => { var cs = context.TestContext.Config.GetValue("E2E:Orders:ConnectionString") ?? throw new InvalidOperationException("E2E:Orders:ConnectionString configuration value is missing."); var ma = new MigrationArgs(MigrationCommand.All | MigrationCommand.ResetAndData, cs); Contoso.Orders.Database.Program.ConfigureMigrationArgs(ma); - ma.AddAssembly(); using var m = new SqlServerMigration(ma); var (Success, Output) = await m.MigrateAndLogAsync().ConfigureAwait(false); @@ -51,4 +66,4 @@ await context.StepAsync("Orders database migration.", async () => throw new Exception("Database migration failed:" + Environment.NewLine + Output); }, "Successfully migrated; base data refreshed.").ConfigureAwait(false); } -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.E2E.Runner/Scenarios/ShoppingBasketScenario.cs b/samples/tests/Contoso.E2E.Runner/Scenarios/ShoppingBasketScenario.cs index aa56fc8f1..8ce841a7e 100644 --- a/samples/tests/Contoso.E2E.Runner/Scenarios/ShoppingBasketScenario.cs +++ b/samples/tests/Contoso.E2E.Runner/Scenarios/ShoppingBasketScenario.cs @@ -8,11 +8,13 @@ public class ShoppingBasketScenario : IScenario { private static readonly SemaphoreSlim _semaphore = new(1); private ProductLite[]? _products; + private CustomerLite[]? _customers; + /// public async Task RunAsync(ScenarioContext context) { - // Step 1: Find all the products (first time only). + // Step 1a: Find all the products (first time only). _semaphore.Wait(); try { @@ -31,10 +33,28 @@ public async Task RunAsync(ScenarioContext context) _semaphore.Release(); } - // Step 2: Create a new basket + // Step 1b: Find all the Customers (first time only). + _semaphore.Wait(); + try + { + if (_customers is null) + { + _customers = await context.StepAsync("Find all customers.", async () => + { + return await CustomerUpdateScenario.GetAllCustomersAsync(context); + }, result => $"{result!.Length} customer(s) found."); + await ScenarioContext.RandomizedDelayAsync(context); + } + } + finally + { + _semaphore.Release(); + } + + // Step 2: Create a new basket (for the Customers domain's golden customer ^16 - Frank Foster, who has a shipping address that is defaulted). var basket = await context.StepAsync("Create new basket.", async () => { - var response = await context.TestContext.ShoppingHttpClient.PostAsync($"/api/customers/test/baskets", null); + var response = await context.TestContext.ShoppingHttpClient.PostAsync($"/api/customers/{16.ToGuid()}/baskets", null); return await response.GetValueAsync(); }, b => $"Basket '{b!.Id}' created."); @@ -77,7 +97,7 @@ public async Task RunAsync(ScenarioContext context) // Step 5: Update the shipping address. basket = await context.StepAsync("Update shipping address.", async () => { - var address = new Address + var address = new Shopping.Contracts.Address { Street1 = "123 Main St", City = "Anytown", @@ -101,7 +121,7 @@ public async Task RunAsync(ScenarioContext context) basket = await context.StepAsync("Get checked-out basket.", async () => { var response = await context.TestContext.ShoppingHttpClient.GetAsync($"/api/baskets/{basket!.Id}"); - return await response.GetValueAsync() ?? throw new NotFoundException(); + return await response.GetValueAsync(); }, b => $"Basket retrieved."); } } diff --git a/samples/tests/Contoso.E2E.Runner/appsettings.json b/samples/tests/Contoso.E2E.Runner/appsettings.json index fda37ec2c..bf71ffdae 100644 --- a/samples/tests/Contoso.E2E.Runner/appsettings.json +++ b/samples/tests/Contoso.E2E.Runner/appsettings.json @@ -8,8 +8,13 @@ "BaseAddress": "https://localhost:7219", "ConnectionString": "Data Source=127.0.0.1,1433;Initial Catalog=Contoso;User id=sa;Password=yourStrong(!)Password;TrustServerCertificate=true" }, + "Customers": { + "BaseAddress": "https://localhost:7320", + "ConnectionString": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==", + "DatabaseId": "contoso" + }, "Orders": { - "BaseAddress": "https://localhost:62023", + "BaseAddress": "https://localhost:7330", "ConnectionString": "Data Source=127.0.0.1,1433;Initial Catalog=Contoso;User id=sa;Password=yourStrong(!)Password;TrustServerCertificate=true" }, "RecentEventsDisplayCount": 10, @@ -31,9 +36,19 @@ "MinDelayMilliseconds": 200, "MaxDelayMilliseconds": 300 }, + "Customers-Query": { + "Parallelism": 2, + "MinDelayMilliseconds": 1000, + "MaxDelayMilliseconds": 5000 + }, + "Customers-Update": { + "Parallelism": 2, + "MinDelayMilliseconds": 1000, + "MaxDelayMilliseconds": 3000 + }, "Shopping-Basket": { "Parallelism": 3, - "MinDelayMilliseconds": 250, + "MinDelayMilliseconds": 150, "MaxDelayMilliseconds": 750 } } @@ -44,4 +59,4 @@ "System.Net.Http": "Warning" } } -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.Products.Test.Api/GlobalUsing.cs b/samples/tests/Contoso.Products.Test.Api/GlobalUsing.cs index 6ebb089da..41ac99fa2 100644 --- a/samples/tests/Contoso.Products.Test.Api/GlobalUsing.cs +++ b/samples/tests/Contoso.Products.Test.Api/GlobalUsing.cs @@ -1,11 +1,15 @@ +global using AwesomeAssertions; global using Contoso.Products.Contracts; global using CoreEx; global using CoreEx.Http.Abstractions; -global using AwesomeAssertions; +global using CoreEx.Json; +global using DbMigration = Contoso.Products.Database.Program; +global using Microsoft.EntityFrameworkCore; +global using Microsoft.Extensions.DependencyInjection; global using NUnit.Framework; global using System.Net; +global using System.Net.Http.Json; global using System.Text.Json; +global using TestData = Contoso.Products.Test.Common.TestData; global using UnitTestEx; global using UnitTestEx.Expectations; -global using DbMigration = Contoso.Products.Database.Program; -global using TestData = Contoso.Products.Test.Common.TestData; \ No newline at end of file diff --git a/samples/tests/Contoso.Products.Test.Api/MovementMutateTests.Reserve.cs b/samples/tests/Contoso.Products.Test.Api/MovementMutateTests.Reserve.cs index 4ec239965..915304538 100644 --- a/samples/tests/Contoso.Products.Test.Api/MovementMutateTests.Reserve.cs +++ b/samples/tests/Contoso.Products.Test.Api/MovementMutateTests.Reserve.cs @@ -42,6 +42,34 @@ public void Reserve_InsufficientStock() .AssertProblemDetails(p => p.Title.Should().Be($"Product '{p2}' does not have sufficient quantity on hand.")); } + [Test] + public async Task Reserve_Concurrent() + { + // Arrange - set a known on-hand quantity for an otherwise unused product (also exercises the first-time inventory insert). + var p = 29.ToGuid().ToString(); + var adj = new MovementRequest { Id = 671954.ToGuid().ToString(), Products = new() { { p, new MovementRequestProduct { Quantity = 10, UnitOfMeasureCode = "EA" } } } }; + + Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertCount(1)) + .Run(HttpMethod.Post, "/api/inventory/adjust", adj) + .AssertOK(); + + // Act - fire more concurrent single-unit reservations than there is stock for. + using var client = Test.GetTestServer().CreateClient(); + var responses = await Task.WhenAll(Enumerable.Range(0, 15).Select(i => client.PostAsJsonAsync("/api/inventory/reserve", + new MovementRequest { Id = (680000 + i).ToGuid().ToString(), Products = new() { { p, new MovementRequestProduct { Quantity = 1, UnitOfMeasureCode = "EA" } } } }, + JsonDefaults.SerializerOptions))); + + // Assert - exactly the available stock is reserved; the remainder fail with insufficient quantity (never a 500 concurrency failure). + responses.Select(r => r.StatusCode).Should().AllSatisfy(sc => sc.Should().BeOneOf(HttpStatusCode.OK, HttpStatusCode.BadRequest)); + responses.Count(r => r.StatusCode == HttpStatusCode.OK).Should().Be(10); + + Test.Http() + .Run(HttpMethod.Get, $"/api/products/{p}/on-hand") + .AssertOK() + .Value.Should().Be(0); + } + [Test] public void Reserve_Success() { diff --git a/samples/tests/Contoso.Products.Test.Api/ProductMutateTests.Activate.cs b/samples/tests/Contoso.Products.Test.Api/ProductMutateTests.Activate.cs new file mode 100644 index 000000000..be9e9ccd4 --- /dev/null +++ b/samples/tests/Contoso.Products.Test.Api/ProductMutateTests.Activate.cs @@ -0,0 +1,63 @@ +namespace Contoso.Products.Test.Api; + +public partial class ProductMutateTests : WithApiTester +{ + [Test] + public void Activate_NotFound() + { + // Arrange/Act/Assert. + Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, "/api/products/404/activate") + .AssertNotFound(); + } + + [Test] + public void Activate_AlreadyActive() + { + var id = 20.ToGuid().ToString(); + + // Arrange/Act/Assert - no-op, so unchanged and no event. + var p = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, $"/api/products/{id}/activate") + .AssertOK() + .Value!; + + p.IsInactive.Should().BeFalse(); + } + + [Test] + public void Activate_Success() + { + var id = 27.ToGuid().ToString(); + + // Arrange. + Test.Http() + .Run(HttpMethod.Get, $"/api/products/{id}") + .AssertOK() + .Value!.IsInactive.Should().BeTrue(); + + // Act. + var p = Test.Http() + .ExpectPostgresOutboxEvents(c => c.AssertMetadata("contoso", "contoso.products.product.activated.v1", id)) + .Run(HttpMethod.Post, $"/api/products/{id}/activate") + .AssertOK() + .Value!; + + p.IsInactive.Should().BeFalse(); + + // Assert persisted. + Test.Http() + .Run(HttpMethod.Get, $"/api/products/{id}") + .AssertOK() + .Value!.IsInactive.Should().BeFalse(); + + // Assert idempotent - already active, so no event. + Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, $"/api/products/{id}/activate") + .AssertOK() + .Value!.IsInactive.Should().BeFalse(); + } +} diff --git a/samples/tests/Contoso.Products.Test.Api/RefDataMutateTests.cs b/samples/tests/Contoso.Products.Test.Api/RefDataMutateTests.cs new file mode 100644 index 000000000..b78763915 --- /dev/null +++ b/samples/tests/Contoso.Products.Test.Api/RefDataMutateTests.cs @@ -0,0 +1,591 @@ +namespace Contoso.Products.Test.Api; + +public class RefDataMutateTests : WithApiTester +{ + private const string _brandsUrl = "/api/refdata/brands"; + + [OneTimeSetUp] + public async Task OneTimeSetUpAsync() + { + await Test.MigratePostgresDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); + await Test.ClearFusionCacheAsync().ConfigureAwait(false); + + Test.UseExpectedPostgresOutboxPublisher(); + } + + /// + /// Creates a new (inactive) brand with the specified . + /// + private Brand CreateBrand(string code) + => Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Post, _brandsUrl, new Brand { Code = code, Text = $"Brand {code}" }) + .AssertCreated() + .Value!; + + /// + /// Creates a new brand with the specified and activates it. + /// + private Brand CreateActiveBrand(string code) + { + var b = CreateBrand(code); + return Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/activate") + .AssertOK() + .Value!; + } + + private Brand[] GetBrands(bool includeInactive = false) + => Test.Http() + .Run(HttpMethod.Get, includeInactive ? $"{_brandsUrl}?$inactive=true" : _brandsUrl) + .AssertOK() + .Value!; + + #region Get + + [Test] + public void Get_NotFound() + { + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/404") + .AssertNotFound(); + } + + [Test] + public void Get_Success() + { + // Arrange - the seeded identifier is not assumed; get via the list. + var b = GetBrands().Single(x => x.Code == "YETI"); + + // Act/Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertOK() + .AssertValue(b); + } + + #endregion + + #region Create + + [Test] + public void Create_Empty() + { + Test.Http() + .Run(HttpMethod.Post, _brandsUrl, new Brand()) + .AssertBadRequest() + .AssertErrors( + "Code is required.", + "Text is required." + ); + } + + [Test] + public void Create_Bad_Data() + { + var b = new Brand + { + Code = new string('X', CoreEx.Validation.ReferenceDataValidator.CodeMaximumLength + 1), + Text = new string('X', CoreEx.Validation.ReferenceDataValidator.TextMaximumLength + 1), + Description = new string('X', CoreEx.Validation.ReferenceDataValidator.DescriptionMaximumLength + 1), + }; + + Test.Http() + .Run(HttpMethod.Post, _brandsUrl, b) + .AssertBadRequest() + .AssertErrors( + $"Code must not exceed {CoreEx.Validation.ReferenceDataValidator.CodeMaximumLength} character(s) in length.", + $"Text must not exceed {CoreEx.Validation.ReferenceDataValidator.TextMaximumLength} character(s) in length.", + $"Description must not exceed {CoreEx.Validation.ReferenceDataValidator.DescriptionMaximumLength} character(s) in length." + ); + } + + [Test] + public void Create_Duplicate() + { + Test.Http() + .Run(HttpMethod.Post, _brandsUrl, new Brand { Code = "YETI", Text = "Yeti Again" }) + .AssertConflict(); + } + + [Test] + public void Create_Success() + { + // Act/Assert - a newly created reference data value is always inactive until explicitly activated. + var b = Test.Http() + .ExpectIdentifier() + .ExpectETag() + .ExpectPostgresOutboxEvents(e => e.AssertWithValue("contoso", "contoso.products.brand.created.v1")) + .Run(HttpMethod.Post, _brandsUrl, new Brand { Code = "CREATE-OK", Text = "Create OK", SortOrder = 5 }) + .AssertCreated() + .AssertLocationHeader(r => new Uri($"{_brandsUrl}/{r!.Id}", UriKind.Relative)) + .Value!; + + b.Code.Should().Be("CREATE-OK"); + b.Text.Should().Be("Create OK"); + b.SortOrder.Should().Be(5); + b.IsInactive.Should().BeTrue(); + + // Assert - get by identifier. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertOK() + .AssertValue(b); + + // Assert - not within the (default) active list, but is within the inactive-inclusive list. + GetBrands().Should().NotContain(x => x.Code == "CREATE-OK"); + GetBrands(includeInactive: true).Should().Contain(x => x.Code == "CREATE-OK"); + } + + [Test] + public void Create_IdempotencyKey() + { + var b = new Brand { Code = "CREATE-IDEM", Text = "Create Idempotent" }; + var ik = Guid.NewGuid().ToString(); + + // Act/Assert - first request creates and emits an event. + var v1 = Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Post, _brandsUrl, b, requestModifier: r => r.WithIdempotencyKey(ik)) + .AssertCreated() + .Value!; + + // Assert - repeat with the same idempotency key; same result and no further event emitted. + var v2 = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, _brandsUrl, b, requestModifier: r => r.WithIdempotencyKey(ik)) + .AssertCreated() + .Value!; + + ObjectComparer.Assert(v1, v2); + } + + #endregion + + #region Patch + + [Test] + public void Patch_NotFound() + { + Test.Http() + .Run(HttpMethod.Patch, $"{_brandsUrl}/404", new { text = "abc" }, requestModifier: r => r.WithMergePatchJsonContentType()) + .AssertNotFound(); + } + + [Test] + public void Patch_Concurrency() + { + var b = CreateBrand("PATCH-CONC"); + + Test.Http() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { text = "Changed" }, requestModifier: r => r.WithIfMatch("AAAAAAAA").WithMergePatchJsonContentType()) + .AssertPreconditionFailed(); + } + + [Test] + public void Patch_Validation() + { + var b = CreateBrand("PATCH-VAL"); + + Test.Http() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { text = string.Empty }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertBadRequest() + .AssertErrors("Text is required."); + } + + [Test] + public void Patch_NotSupported() + { + var b = CreateBrand("PATCH-NS"); + + Test.Http() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { startsOn = "2024-01-01" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertBadRequest() + .AssertProblemDetails(pd => pd.Title.Should().Be("Starts on is not currently supported and as such cannot be set.")); + } + + [Test] + public void Patch_Success() + { + var b = CreateBrand("PATCH-OK"); + + // Act/Assert. + var u = Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertWithValue("contoso", "contoso.products.brand.updated.v1")) + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { text = "Patched Text", sortOrder = 9 }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.Id.Should().Be(b.Id); + u.Code.Should().Be(b.Code); + u.Text.Should().Be("Patched Text"); + u.SortOrder.Should().Be(9); + u.IsInactive.Should().BeTrue(); + u.ETag.Should().NotBe(b.ETag); + + // Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertOK() + .AssertValue(u); + } + + [Test] + public void Patch_Success_PreservesCreatedAudit() + { + // Created audit is not exposed by the contract, so the database row is checked directly; the update maps a new model so these columns must be protected (ValueGeneratedOnUpdate). + var b = CreateBrand("PATCH-AUDIT"); + var (createdBy, createdOn, _, _) = GetAudit(b.Code!); + createdBy.Should().NotBeNullOrEmpty(); + createdOn.Should().NotBeNull(); + + Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { text = "Patched Text" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK(); + + var (cb, co, ub, uo) = GetAudit(b.Code!); + cb.Should().Be(createdBy); + co.Should().Be(createdOn); + ub.Should().NotBeNullOrEmpty(); + uo.Should().NotBeNull(); + } + + private (string? CreatedBy, DateTimeOffset? CreatedOn, string? UpdatedBy, DateTimeOffset? UpdatedOn) GetAudit(string code) + { + using var scope = Test.Services.CreateScope(); + var db = scope.ServiceProvider.GetRequiredService(); + using var cmd = db.Database.GetDbConnection().CreateCommand(); + db.Database.OpenConnection(); + cmd.CommandText = "select created_by, created_on, updated_by, updated_on from products.brand where code = @code"; + var p = cmd.CreateParameter(); + p.ParameterName = "code"; + p.Value = code; + cmd.Parameters.Add(p); + + using var r = cmd.ExecuteReader(); + r.Read().Should().BeTrue(); + return (r.IsDBNull(0) ? null : r.GetString(0), r.IsDBNull(1) ? null : r.GetFieldValue(1), r.IsDBNull(2) ? null : r.GetString(2), r.IsDBNull(3) ? null : r.GetFieldValue(3)); + } + + [Test] + public void Patch_Code_Immutable() + { + var b = CreateBrand("PATCH-CODE"); + + // Act/Assert - the code is silently retained; only the text is updated. + var u = Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { code = "PATCH-CODE-CHANGED", text = "Code Retained" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.Code.Should().Be("PATCH-CODE"); + u.Text.Should().Be("Code Retained"); + } + + [Test] + public void Patch_IsInactive_Ignored() + { + // The active state can only be changed via activate/deactivate. + var b = CreateBrand("PATCH-ACTIVE"); + + var u = Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { isInactive = false, text = "Still Inactive" }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.IsInactive.Should().BeTrue(); + u.Text.Should().Be("Still Inactive"); + } + + [Test] + public void Patch_NoChanges() + { + var b = CreateBrand("PATCH-NONE"); + + // Act/Assert - nothing changed so no event and the etag remains. + var u = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Patch, $"{_brandsUrl}/{b.Id}", new { }, requestModifier: r => r.WithIfMatch(b.ETag).WithMergePatchJsonContentType()) + .AssertOK() + .Value!; + + u.ETag.Should().Be(b.ETag); + u.Text.Should().Be(b.Text); + } + + #endregion + + #region Put + + [Test] + public void Put_NotFound() + { + Test.Http() + .Run(HttpMethod.Put, $"{_brandsUrl}/404", new { code = "PUT-404", text = "abc" }, requestModifier: r => r.WithIfMatch("abc")) + .AssertNotFound(); + } + + [Test] + public void Put_MissingETag() + { + Test.Http() + .Run(HttpMethod.Put, $"{_brandsUrl}/404", new { code = "PUT-404", text = "abc" }) + .Assert(System.Net.HttpStatusCode.PreconditionRequired); + } + + [Test] + public void Put_Concurrency() + { + var b = CreateBrand("PUT-CONC"); + + Test.Http() + .Run(HttpMethod.Put, $"{_brandsUrl}/{b.Id}", new { code = b.Code, text = "Changed" }, requestModifier: r => r.WithIfMatch("AAAAAAAA")) + .AssertPreconditionFailed(); + } + + [Test] + public void Put_Validation() + { + var b = CreateBrand("PUT-VAL"); + + Test.Http() + .Run(HttpMethod.Put, $"{_brandsUrl}/{b.Id}", new { code = b.Code, text = string.Empty }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertBadRequest() + .AssertErrors("Text is required."); + } + + [Test] + public void Put_Success() + { + var b = CreateBrand("PUT-OK"); + + // Act/Assert. + var u = Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertWithValue("contoso", "contoso.products.brand.updated.v1")) + .Run(HttpMethod.Put, $"{_brandsUrl}/{b.Id}", new { code = b.Code, text = "Put Text", sortOrder = 9 }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.Id.Should().Be(b.Id); + u.Code.Should().Be(b.Code); + u.Text.Should().Be("Put Text"); + u.SortOrder.Should().Be(9); + u.IsInactive.Should().BeTrue(); + u.ETag.Should().NotBe(b.ETag); + + // Assert. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertOK() + .AssertValue(u); + } + + [Test] + public void Put_Code_And_IsInactive_Ignored() + { + // The code is immutable and the active state can only be changed via activate/deactivate. + var b = CreateBrand("PUT-IGNORED"); + + var u = Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Put, $"{_brandsUrl}/{b.Id}", new { code = "PUT-IGNORED-CHANGED", text = "Retained", isInactive = false }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.Code.Should().Be("PUT-IGNORED"); + u.IsInactive.Should().BeTrue(); + u.Text.Should().Be("Retained"); + } + + [Test] + public void Put_NoChanges() + { + var b = CreateBrand("PUT-NONE"); + + // Act/Assert - nothing changed so no event and the etag remains. + var u = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Put, $"{_brandsUrl}/{b.Id}", new { code = b.Code, text = b.Text, sortOrder = b.SortOrder }, requestModifier: r => r.WithIfMatch(b.ETag)) + .AssertOK() + .Value!; + + u.ETag.Should().Be(b.ETag); + } + + #endregion + + #region Activate + + [Test] + public void Activate_NotFound() + { + Test.Http() + .Run(HttpMethod.Post, $"{_brandsUrl}/404/activate") + .AssertNotFound(); + } + + [Test] + public void Activate_Success() + { + var b = CreateBrand("ACTIVATE-OK"); + + // Act/Assert. + var a = Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertWithValue("contoso", "contoso.products.brand.activated.v1")) + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/activate") + .AssertOK() + .Value!; + + a.IsInactive.Should().BeFalse(); + a.ETag.Should().NotBe(b.ETag); + + // Assert - now within the (default) active list. + GetBrands().Should().Contain(x => x.Code == "ACTIVATE-OK"); + } + + [Test] + public void Activate_AlreadyActive() + { + var b = CreateActiveBrand("ACTIVATE-AGAIN"); + + // Act/Assert - already active so no change and no event. + var a = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/activate") + .AssertOK() + .Value!; + + a.IsInactive.Should().BeFalse(); + a.ETag.Should().Be(b.ETag); + } + + #endregion + + #region Deactivate + + [Test] + public void Deactivate_NotFound() + { + Test.Http() + .Run(HttpMethod.Post, $"{_brandsUrl}/404/deactivate") + .AssertNotFound(); + } + + [Test] + public void Deactivate_Success() + { + var b = CreateActiveBrand("DEACTIVATE-OK"); + GetBrands().Should().Contain(x => x.Code == "DEACTIVATE-OK"); + + // Act/Assert. + var d = Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertWithValue("contoso", "contoso.products.brand.deactivated.v1")) + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/deactivate") + .AssertOK() + .Value!; + + d.IsInactive.Should().BeTrue(); + d.ETag.Should().NotBe(b.ETag); + + // Assert - no longer within the (default) active list, but still exists. + GetBrands().Should().NotContain(x => x.Code == "DEACTIVATE-OK"); + GetBrands(includeInactive: true).Should().Contain(x => x.Code == "DEACTIVATE-OK"); + } + + [Test] + public void Deactivate_AlreadyInactive() + { + var b = CreateBrand("DEACTIVATE-AGAIN"); + + // Act/Assert - already inactive so no change and no event. + var d = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/deactivate") + .AssertOK() + .Value!; + + d.IsInactive.Should().BeTrue(); + d.ETag.Should().Be(b.ETag); + } + + [Test] + public void Deactivate_PreCheck() + { + var b = GetBrands().Where(x => x.Code == "YETI").Single(); + + // Act/Assert - pre-check should prevent deactivation. + var d = Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/deactivate") + .AssertBadRequest() + .AssertProblemDetails(p => p.Title.Should().Be("YETI brand cannot be deactivated as it is awesome.")); + } + + #endregion + + #region Delete + + [Test] + public void Delete_NotFound() + { + // Delete is idempotent. + Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Delete, $"{_brandsUrl}/404") + .AssertNoContent(); + } + + [Test] + public void Delete_IsActive() + { + var b = CreateActiveBrand("DELETE-ACTIVE"); + + Test.Http() + .Run(HttpMethod.Delete, $"{_brandsUrl}/{b.Id}") + .AssertBadRequest() + .AssertProblemDetails(p => p.Title.Should().Be("An active reference data value cannot be deleted.")); + + // Assert - still exists. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertOK(); + } + + [Test] + public void Delete_Success() + { + var b = CreateActiveBrand("DELETE-OK"); + GetBrands().Should().Contain(x => x.Code == "DELETE-OK"); + + // Arrange - must be deactivated before it can be deleted. + Test.Http() + .ExpectPostgresOutboxEvents() + .Run(HttpMethod.Post, $"{_brandsUrl}/{b.Id}/deactivate") + .AssertOK(); + + // Act. + Test.Http() + .ExpectPostgresOutboxEvents(e => e.AssertMetadata("contoso", "contoso.products.brand.deleted", b.Id!)) + .Run(HttpMethod.Delete, $"{_brandsUrl}/{b.Id}") + .AssertNoContent(); + + // Assert - idempotent. + Test.Http() + .ExpectNoPostgresOutboxEvents() + .Run(HttpMethod.Delete, $"{_brandsUrl}/{b.Id}") + .AssertNoContent(); + + // Assert - gone, including from the cached reference data. + Test.Http() + .Run(HttpMethod.Get, $"{_brandsUrl}/{b.Id}") + .AssertNotFound(); + + GetBrands(includeInactive: true).Should().NotContain(x => x.Code == "DELETE-OK"); + } + + #endregion +} diff --git a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationCancel.cs b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationCancel.cs index 6b0c6da9c..624605964 100644 --- a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationCancel.cs +++ b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationCancel.cs @@ -11,7 +11,7 @@ public void ReservationCancel_NotFound() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); @@ -43,7 +43,7 @@ public void ReservationCancel_Success() => Test.Scoped(async test => var ce = Test.CreateCloudEventFrom(ed); var sbm = ce.ToServiceBusReceivedMessage(); - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); }).AssertSuccess(); diff --git a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationConfirm.cs b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationConfirm.cs index cc33369b7..ffa6d3988 100644 --- a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationConfirm.cs +++ b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.ReservationConfirm.cs @@ -11,7 +11,7 @@ public void ReservationConfirm_NotFound() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); @@ -39,7 +39,7 @@ public void ReservationConfirm_Success() => Test.Scoped(async test => var ce = Test.CreateCloudEventFrom(ed); var sbm = ce.ToServiceBusReceivedMessage(); - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsSuccess.Should().BeTrue(); }).AssertSuccess(); diff --git a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.cs b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.cs index aaf8ad004..185ed2a8c 100644 --- a/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.cs +++ b/samples/tests/Contoso.Products.Test.Subscribe/SubscriberTests.cs @@ -23,7 +23,7 @@ public void Unsubscribed_Error() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-commands"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); diff --git a/samples/tests/Contoso.Products.Test.Unit/EntryPoint.cs b/samples/tests/Contoso.Products.Test.Unit/EntryPoint.cs index c6e2ae360..4660122fb 100644 --- a/samples/tests/Contoso.Products.Test.Unit/EntryPoint.cs +++ b/samples/tests/Contoso.Products.Test.Unit/EntryPoint.cs @@ -7,14 +7,14 @@ public static void ConfigureApplication(IHostApplicationBuilder builder) // Configure the minimum services required for the execution context and reference data orchestrator; caching will be in-memory for the unit tests. builder.Services.AddExecutionContext(); builder.Services.AddMemoryCache(); - builder.Services.AddReferenceDataOrchestrator(); + builder.Services.AddReferenceDataOrchestrator(); // Reuse the "real" database configured reference data. var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.SnakeCase)); - builder.Services.AddSingleton(new ReferenceDataServiceDecorator(jdr)); + builder.Services.AddSingleton(new ReferenceDataProviderDecorator(jdr)); } - public class ReferenceDataServiceDecorator(JsonDataReader jdr) : ReferenceDataService(Mock.Of()) + public class ReferenceDataProviderDecorator(JsonDataReader jdr) : ReferenceDataProvider(Mock.Of()) { public override Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch { @@ -27,4 +27,4 @@ public class ReferenceDataServiceDecorator(JsonDataReader jdr) : ReferenceDataSe _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") }; } -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.Shopping.Test.Api/MutateTests.Basket.cs b/samples/tests/Contoso.Shopping.Test.Api/MutateTests.Basket.cs index 19bac2af6..57e3a00e0 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/MutateTests.Basket.cs +++ b/samples/tests/Contoso.Shopping.Test.Api/MutateTests.Basket.cs @@ -5,23 +5,83 @@ public partial class MutateTests [Test] public void Basket_Create() { + MockCustomerWithAddress(); + var basket = Test.Http() .ExpectIdentifier() .ExpectChangeLogCreated() .ExpectETag() .ExpectJsonFromResource("Basket_Create.res.json", _pathsToIgnore) .ExpectSqlServerOutboxEvents(e => e.AssertWithValue("contoso", "contoso.shopping.basket.created.v1")) - .Run(HttpMethod.Post, $"/api/customers/{1004.ToGuid()}/baskets") + .Run(HttpMethod.Post, $"/api/customers/{CustomerWithAddressId}/baskets") .AssertCreated() .AssertLocationHeader(b => new Uri($"/api/baskets/{b!.Id}", UriKind.Relative)) .Value!; + _mockHttpCustomerWithAddress.Verify(); + + // The shipping address is defaulted from the customer. + basket.ShippingAddress.Should().BeEquivalentTo(new Address { Street1 = "1 Test Street", City = "Springfield", PostCode = "49007", State = "IL" }); + Test.Http() .Run(HttpMethod.Get, $"/api/baskets/{basket.Id}") .AssertOK() .AssertValue(basket); } + [Test] + public void Basket_Create_NoCustomerAddress() + { + MockCustomerNoAddress(); + + var basket = Test.Http() + .ExpectSqlServerOutboxEvents(e => e.AssertWithValue("contoso", "contoso.shopping.basket.created.v1")) + .Run(HttpMethod.Post, $"/api/customers/{CustomerNoAddressId}/baskets") + .AssertCreated() + .Value!; + + _mockHttpCustomerNoAddress.Verify(); + + basket.CustomerId.Should().Be(CustomerNoAddressId); + basket.ShippingAddress.Should().BeNull(); + } + + [Test] + public void Basket_Create_CustomerCached() + { + // The first create invokes the Customers API; the customer is then cached. + MockCustomerNoAddress(); + + Test.Http() + .ExpectSqlServerOutboxEvents() + .Run(HttpMethod.Post, $"/api/customers/{CustomerNoAddressId}/baskets") + .AssertCreated(); + + _mockHttpCustomerNoAddress.Verify(); + + // The customer is no longer available remotely; the second create must succeed by using the cached customer. + _mockHttpCustomerNoAddress.Respond.With(HttpStatusCode.NotFound); + + Test.Http() + .ExpectSqlServerOutboxEvents() + .Run(HttpMethod.Post, $"/api/customers/{CustomerNoAddressId}/baskets") + .AssertCreated(); + } + + [Test] + public void Basket_Create_CustomerNotFound() + { + _mockHttpCustomerNotFound.Respond.With(HttpStatusCode.NotFound); + + Test.Http() + .ExpectNoSqlServerOutboxEvents() + .Run(HttpMethod.Post, $"/api/customers/{CustomerNotFoundId}/baskets") + .AssertBadRequest() + .AssertErrors(new ApiError("customerId", "Customer was not found.")); + + _mockHttpCustomerNotFound.Verify(); + } + [Test] public void Basket_ApplyDiscount_NotFound() { @@ -83,9 +143,11 @@ public void Basket_ApplyDiscount_Success() [Test] public void Basket_Update_ShippingAddress() { + MockCustomerNoAddress(); + var v = Test.Http() .ExpectSqlServerOutboxEvents() - .Run(HttpMethod.Post, $"/api/customers/{1004.ToGuid()}/baskets") + .Run(HttpMethod.Post, $"/api/customers/{CustomerNoAddressId}/baskets") .AssertCreated() .Value!; @@ -156,7 +218,7 @@ public void Basket_Checkout_Success() v = Test.Http() .ExpectChangeLogUpdated() .ExpectSqlServerOutboxEvents(e => e.AssertWithValue("contoso", "contoso.shopping.basket.checkedout.v1") - .AssertMetadata("contoso", "contoso.products.reservation.confirm", v.Id)) + .AssertMetadata("contoso-products", "contoso.products.reservation.confirm", v.Id)) .Run(HttpMethod.Post, $"/api/baskets/{v.Id}/checkout") .AssertOK() .Value!; @@ -219,7 +281,7 @@ public void Basket_Checkout_Save_Failure() Test.Http() .OnEventPublish(SqlServerOutboxPublisher.DefaultServiceKey, () => throw new InvalidOperationException("Simulated failure during save; oh no, we all ready reserved inventory!")) .ExpectNoSqlServerOutboxEvents() - .ExpectAzureServiceBusEvents(e => e.AssertMetadata("contoso", "contoso.products.reservation.cancel", id)) + .ExpectAzureServiceBusEvents(e => e.AssertMetadata("contoso-products", "contoso.products.reservation.cancel", id)) .Run(HttpMethod.Post, $"/api/baskets/{id}/checkout") .AssertInternalServerError(); diff --git a/samples/tests/Contoso.Shopping.Test.Api/MutateTests.cs b/samples/tests/Contoso.Shopping.Test.Api/MutateTests.cs index e48618d27..b16d88dae 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/MutateTests.cs +++ b/samples/tests/Contoso.Shopping.Test.Api/MutateTests.cs @@ -5,6 +5,12 @@ public partial class MutateTests : WithApiTester private static readonly string[] _pathsToIgnore = ["items.etag"]; private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpReserveRequest = null!; + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpCustomerWithAddress = null!; + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpCustomerNoAddress = null!; + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpCustomerNotFound = null!; + + [SetUp] + public Task SetUpAsync() => Test.ClearFusionCacheAsync(); // Customers are cached by the adapter; ensure each test starts clean so the Customers mocks are invoked as expected. [OneTimeSetUp] public async Task OneTimeSetUpAsync() @@ -12,14 +18,52 @@ public async Task OneTimeSetUpAsync() // Migrate the database and seed it with test data before starting the test server. await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); await Test.ClearFusionCacheAsync().ConfigureAwait(false); + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); // Use the expected SQL Server Outbox & Azure Service Bus publishers for the tests. Test.UseExpectedSqlServerOutboxPublisher(); Test.UseExpectedAzureServiceBusPublisher(); - // Mock the HTTP client. + // Mock the HTTP clients; the Customers domain is external, so is always mocked (the ids align with the Customers 'read-data.seed.yaml' golden values). var mcf = MockHttpClientFactory.Create(); _mockHttpReserveRequest = mcf.CreateClient("ProductsApi").Request(HttpMethod.Post, "api/inventory/reserve"); + + var customers = mcf.CreateClient("CustomersApi"); + _mockHttpCustomerWithAddress = customers.Request(HttpMethod.Get, $"api/customers/{CustomerWithAddressId}"); + _mockHttpCustomerNoAddress = customers.Request(HttpMethod.Get, $"api/customers/{CustomerNoAddressId}"); + _mockHttpCustomerNotFound = customers.Request(HttpMethod.Get, $"api/customers/{CustomerNotFoundId}"); Test.ReplaceHttpClientFactory(mcf); } -} \ No newline at end of file + + /// Frank Foster (^16) has a complete shipping address. + private static readonly string CustomerWithAddressId = 16.ToGuid().ToString(); + + /// Alice Anderson (^11) has no shipping address. + private static readonly string CustomerNoAddressId = 11.ToGuid().ToString(); + + private static readonly string CustomerNotFoundId = 404.ToGuid().ToString(); + + /// Mocks the Customers API response for the golden (^16) customer with a complete shipping address. + private void MockCustomerWithAddress() => _mockHttpCustomerWithAddress.Respond.WithJson(new + { + id = CustomerWithAddressId, + firstName = "Frank", + lastName = "Foster", + email = "frank.foster@example.com", + phone = "555-0100", + shippingAddress = new { street1 = "1 Test Street", city = "Springfield", postCode = "49007", state = "IL" }, + customerTypeCode = "IND", + contactMethodCode = "EM" + }); + + /// Mocks the Customers API response for the golden (^11) customer with no shipping address. + private void MockCustomerNoAddress() => _mockHttpCustomerNoAddress.Respond.WithJson(new + { + id = CustomerNoAddressId, + firstName = "Alice", + lastName = "Anderson", + email = "alice.anderson@example.com", + customerTypeCode = "IND", + contactMethodCode = "EM" + }); +} diff --git a/samples/tests/Contoso.Shopping.Test.Api/ReadTests.BasketQuery.cs b/samples/tests/Contoso.Shopping.Test.Api/ReadTests.BasketQuery.cs new file mode 100644 index 000000000..f16f36b7e --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Api/ReadTests.BasketQuery.cs @@ -0,0 +1,108 @@ +namespace Contoso.Shopping.Test.Api; + +public partial class ReadTests +{ + private static string BasketsUrl(int customer, string? query = null) => $"/api/customers/{customer.ToGuid()}/baskets" + (query is null ? null : $"?{query}"); + + private static string[] BasketIds(Basket[]? baskets) => [.. baskets!.Select(b => b.Id!)]; + + [Test] + public void Basket_Query_CustomerScoped() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(11)).AssertOK().Value; + BasketIds(r).Should().BeEquivalentTo([3001.ToGuid().ToString(), 3003.ToGuid().ToString()]); + r!.Should().OnlyContain(b => b.CustomerId == 11.ToGuid().ToString()); + } + + [Test] + public void Basket_Query_UnknownCustomer() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(404)).AssertOK().Value; + r.Should().NotBeNull().And.BeEmpty(); + } + + [Test] + public void Basket_Query_Summary_ExcludesItemsAndShippingAddress() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(12)).AssertOK().Value; + r.Should().ContainSingle(); + r![0].Id.Should().Be(3002.ToGuid().ToString()); + r[0].Items.Should().BeNull(); + r[0].ShippingAddress.Should().BeNull(); + r[0].Pricing!.Total.Should().Be(9533.81m); + } + + [Test] + public void Basket_Query_Filter_Status() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(13, "$filter=status eq 'B'")).AssertOK().Value; + BasketIds(r).Should().Equal(3004.ToGuid().ToString()); + } + + [Test] + public void Basket_Query_Filter_Total() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$filter=pricing.total gt 100")).AssertOK().Value; + BasketIds(r).Should().Equal(3003.ToGuid().ToString()); + } + + [Test] + public void Basket_Query_Filter_NeverCrossesCustomer() + { + // Basket 3002 belongs to customer 12; filtering on its status for customer 11 must not return it. + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$filter=status eq 'C'")).AssertOK().Value; + r.Should().BeEmpty(); + } + + [Test] + public void Basket_Query_OrderBy_Total() + { + var asc = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$orderby=pricing.total")).AssertOK().Value; + BasketIds(asc).Should().Equal(3001.ToGuid().ToString(), 3003.ToGuid().ToString()); + + var desc = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$orderby=pricing.total desc")).AssertOK().Value; + BasketIds(desc).Should().Equal(3003.ToGuid().ToString(), 3001.ToGuid().ToString()); + } + + [Test] + public void Basket_Query_OrderBy_Status() + { + var asc = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$orderby=status")).AssertOK().Value; + BasketIds(asc).Should().Equal(3003.ToGuid().ToString(), 3001.ToGuid().ToString()); // A, E. + + var desc = Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$orderby=status desc")).AssertOK().Value; + BasketIds(desc).Should().Equal(3001.ToGuid().ToString(), 3003.ToGuid().ToString()); + } + + [Test] + public void Basket_Query_OrderBy_CreatedOn_FinalTieBreak() + { + // All of customer 13's baskets have the same total (98); CreatedOn must deterministically sequence them, and repeated calls must agree. + var r1 = Test.Http().Run(HttpMethod.Get, BasketsUrl(13, "$orderby=pricing.total")).AssertOK().Value; + var r2 = Test.Http().Run(HttpMethod.Get, BasketsUrl(13, "$orderby=pricing.total")).AssertOK().Value; + r1.Should().HaveCount(5); + BasketIds(r1).Should().Equal(BasketIds(r2)); + r1!.Select(b => b.ChangeLog!.CreatedOn).Should().BeInAscendingOrder(); + } + + [Test] + public void Basket_Query_Paging() + { + var r = Test.Http().Run(HttpMethod.Get, BasketsUrl(13, "$orderby=pricing.total&$take=2&$skip=1")).AssertOK().Value; + r.Should().HaveCount(2); + } + + [Test] + public void Basket_Query_InvalidField() + { + Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$filter=unknown eq 'X'")).AssertBadRequest(); + Test.Http().Run(HttpMethod.Get, BasketsUrl(11, "$orderby=unknown")).AssertBadRequest(); + } + + [Test] + public void Basket_Query_Schema() + { + var r = Test.Http().Run(HttpMethod.Get, $"/api/customers/{11.ToGuid()}/baskets/$query").AssertOK().Response.Content.ReadAsStringAsync().GetAwaiter().GetResult(); + r.Should().Contain("pricing.total").And.Contain("status").And.Contain("createdon"); + } +} diff --git a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Create.res.json b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Create.res.json index d3febb87e..3828fd2ca 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Create.res.json +++ b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Create.res.json @@ -1,12 +1,18 @@ { "id": "eda1e75a-199a-44f1-b927-93844608d4e2", - "customerId": "000003ec-0000-0000-0000-000000000000", + "customerId": "00000010-0000-0000-0000-000000000000", "status": "E", "items": [], "pricing": { "subTotal": 0, "total": 0 }, + "shippingAddress": { + "street1": "1 Test Street", + "city": "Springfield", + "postCode": "49007", + "state": "IL" + }, "changeLog": { "createdBy": "DOMAIN-CORP\\eric.sibly", "createdOn": "2026-02-21T19:13:24.7027106\u002B00:00" diff --git a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Get_Found.res.json b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Get_Found.res.json index d67fbd373..756531b42 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Get_Found.res.json +++ b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Get_Found.res.json @@ -1,6 +1,6 @@ { "id": "00000bba-0000-0000-0000-000000000000", - "customerId": "000003ea-0000-0000-0000-000000000000", + "customerId": "0000000c-0000-0000-0000-000000000000", "status": "C", "items": [ { diff --git a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_Existing.res.json b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_Existing.res.json index 66e362364..9856f53a0 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_Existing.res.json +++ b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_Existing.res.json @@ -1,6 +1,6 @@ { "id": "00000bbd-0000-0000-0000-000000000000", - "customerId": "000003eb-0000-0000-0000-000000000000", + "customerId": "0000000d-0000-0000-0000-000000000000", "status": "A", "items": [ { diff --git a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_New.res.json b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_New.res.json index 5b9312952..4c60bb905 100644 --- a/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_New.res.json +++ b/samples/tests/Contoso.Shopping.Test.Api/Resources/Basket_Item_Add_New.res.json @@ -1,6 +1,6 @@ { "id": "00000bbb-0000-0000-0000-000000000000", - "customerId": "000003e9-0000-0000-0000-000000000000", + "customerId": "0000000b-0000-0000-0000-000000000000", "status": "A", "items": [ { diff --git a/samples/tests/Contoso.Shopping.Test.Common/Data/mutate-data.seed.yaml b/samples/tests/Contoso.Shopping.Test.Common/Data/mutate-data.seed.yaml index 1850548dc..24b62c959 100644 --- a/samples/tests/Contoso.Shopping.Test.Common/Data/mutate-data.seed.yaml +++ b/samples/tests/Contoso.Shopping.Test.Common/Data/mutate-data.seed.yaml @@ -32,14 +32,14 @@ Shopping: - { ProductId: ^31, Sku: MUCOFF-DRIVETRAIN-CLEANER, Text: Muc-Off Drivetrain Cleaner, UnitOfMeasureCode: EA, Price: 25.00 } - { ProductId: ^32, Sku: LABOR, Text: Labor, UnitOfMeasureCode: HR, Price: 80.00, IsNonStocked: true } - Basket: - - { BasketId: ^3001, CustomerId: ^1001, BasketStatusCode: E, SubTotal: 0, Total: 0 } - - { BasketId: ^3002, CustomerId: ^1002, BasketStatusCode: C, SubTotal: 9679, DiscountCouponCode: XMAS2025, DiscountAmount: 145.19, Total: 9533.81 } - - { BasketId: ^3003, CustomerId: ^1001, BasketStatusCode: A, SubTotal: 9999, Total: 9999 } - - { BasketId: ^3004, CustomerId: ^1003, BasketStatusCode: B, SubTotal: 98, Total: 98 } - - { BasketId: ^3005, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } - - { BasketId: ^3006, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } - - { BasketId: ^3007, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } - - { BasketId: ^3008, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } + - { BasketId: ^3001, CustomerId: ^11, BasketStatusCode: E, SubTotal: 0, Total: 0 } + - { BasketId: ^3002, CustomerId: ^12, BasketStatusCode: C, SubTotal: 9679, DiscountCouponCode: XMAS2025, DiscountAmount: 145.19, Total: 9533.81 } + - { BasketId: ^3003, CustomerId: ^11, BasketStatusCode: A, SubTotal: 9999, Total: 9999 } + - { BasketId: ^3004, CustomerId: ^13, BasketStatusCode: B, SubTotal: 98, Total: 98 } + - { BasketId: ^3005, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3006, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3007, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } + - { BasketId: ^3008, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } - BasketItem: - { BasketItemId: ^4001, BasketId: ^3006, ProductId: ^27, Sku: ONEUP-COMPOSITE-PEDALS, Text: OneUp Composite Pedals, UnitOfMeasureCode: PR, Quantity: 2, UnitPrice: 49 } - { BasketItemId: ^4002, BasketId: ^3001, ProductId: ^1, Sku: YETI-ASR-C2-2025, Text: Yeti ASR C2, UnitOfMeasureCode: EA, Quantity: 2, UnitPrice: 129 } @@ -54,4 +54,3 @@ Shopping: - { BasketItemId: ^4011, BasketId: ^3008, ProductId: ^1, Sku: YETI-ASR-C2-2025, Text: Yeti ASR C2, UnitOfMeasureCode: EA, Quantity: 2, UnitPrice: 9850 } - { BasketItemId: ^4012, BasketId: ^3008, ProductId: ^20, Sku: SHIMANO-SLX-M7100-CASSETTE, Text: Shimano SLX M7100 12-speed Cassette, UnitOfMeasureCode: EA, Quantity: 1, UnitPrice: 99 } - { BasketItemId: ^4013, BasketId: ^3007, ProductId: ^32, Sku: LABOR, Text: Labor, UnitOfMeasureCode: HR, Quantity: 1.5, UnitPrice: 80 } - diff --git a/samples/tests/Contoso.Shopping.Test.Common/Data/read-data.seed.yaml b/samples/tests/Contoso.Shopping.Test.Common/Data/read-data.seed.yaml index 7c66157ed..70c5c18f5 100644 --- a/samples/tests/Contoso.Shopping.Test.Common/Data/read-data.seed.yaml +++ b/samples/tests/Contoso.Shopping.Test.Common/Data/read-data.seed.yaml @@ -32,14 +32,14 @@ Shopping: - { ProductId: ^31, Sku: MUCOFF-DRIVETRAIN-CLEANER, Text: Muc-Off Drivetrain Cleaner, UnitOfMeasureCode: EA, Price: 25.00 } - { ProductId: ^32, Sku: LABOR, Text: Labor, UnitOfMeasureCode: HR, Price: 80.00, IsNonStocked: true } - Basket: - - { BasketId: ^3001, CustomerId: ^1001, BasketStatusCode: E, SubTotal: 0, Total: 0 } - - { BasketId: ^3002, CustomerId: ^1002, BasketStatusCode: C, SubTotal: 9679, DiscountCouponCode: XMAS2025, DiscountAmount: 145.19, Total: 9533.81, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } - - { BasketId: ^3003, CustomerId: ^1001, BasketStatusCode: A, SubTotal: 9999, Total: 9999 } - - { BasketId: ^3004, CustomerId: ^1003, BasketStatusCode: B, SubTotal: 98, Total: 98 } - - { BasketId: ^3005, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } - - { BasketId: ^3006, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } - - { BasketId: ^3007, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } - - { BasketId: ^3008, CustomerId: ^1003, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3001, CustomerId: ^11, BasketStatusCode: E, SubTotal: 0, Total: 0 } + - { BasketId: ^3002, CustomerId: ^12, BasketStatusCode: C, SubTotal: 9679, DiscountCouponCode: XMAS2025, DiscountAmount: 145.19, Total: 9533.81, ShippingAddressJson: { street1: "123 Main St", city: "Anytown", state: "CA", postCode: "12345" } } + - { BasketId: ^3003, CustomerId: ^11, BasketStatusCode: A, SubTotal: 9999, Total: 9999 } + - { BasketId: ^3004, CustomerId: ^13, BasketStatusCode: B, SubTotal: 98, Total: 98 } + - { BasketId: ^3005, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3006, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3007, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } + - { BasketId: ^3008, CustomerId: ^13, BasketStatusCode: A, SubTotal: 98, Total: 98 } - BasketItem: - { BasketItemId: ^4001, BasketId: ^3006, ProductId: ^27, Sku: ONEUP-COMPOSITE-PEDALS, Text: OneUp Composite Pedals, UnitOfMeasureCode: PR, Quantity: 2, UnitPrice: 49 } - { BasketItemId: ^4002, BasketId: ^3001, ProductId: ^1, Sku: YETI-ASR-C2-2025, Text: Yeti ASR C2, UnitOfMeasureCode: EA, Quantity: 2, UnitPrice: 129 } @@ -54,4 +54,3 @@ Shopping: - { BasketItemId: ^4011, BasketId: ^3008, ProductId: ^1, Sku: YETI-ASR-C2-2025, Text: Yeti ASR C2, UnitOfMeasureCode: EA, Quantity: 2, UnitPrice: 9850 } - { BasketItemId: ^4012, BasketId: ^3008, ProductId: ^20, Sku: SHIMANO-SLX-M7100-CASSETTE, Text: Shimano SLX M7100 12-speed Cassette, UnitOfMeasureCode: EA, Quantity: 1, UnitPrice: 99 } - { BasketItemId: ^4013, BasketId: ^3007, ProductId: ^32, Sku: LABOR, Text: Labor, UnitOfMeasureCode: HR, Quantity: 1.5, UnitPrice: 80 } - diff --git a/samples/tests/Contoso.Shopping.Test.Common/ServiceBus.cs b/samples/tests/Contoso.Shopping.Test.Common/ServiceBus.cs new file mode 100644 index 000000000..fd2e4fce7 --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Common/ServiceBus.cs @@ -0,0 +1,20 @@ +using Azure.Messaging.ServiceBus.Administration; + +namespace Contoso.Shopping.Test.Common; + +public static class ServiceBus +{ + public static CreateQueueOptions[]? GetQueues() => + [ + new CreateQueueOptions("contoso-products") { RequiresSession = true } + ]; + + public static (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? GetTopicsAndSubscriptions() => + [ + (new CreateTopicOptions("contoso"), + [ + new CreateSubscriptionOptions("contoso", "shopping") { RequiresSession = true }, + new CreateSubscriptionOptions("contoso", "products") { RequiresSession = true } + ]) + ]; +} diff --git a/samples/tests/Contoso.Shopping.Test.Relay/RelayTests.cs b/samples/tests/Contoso.Shopping.Test.Relay/RelayTests.cs index dbb9a7cd6..3685b352e 100644 --- a/samples/tests/Contoso.Shopping.Test.Relay/RelayTests.cs +++ b/samples/tests/Contoso.Shopping.Test.Relay/RelayTests.cs @@ -8,7 +8,7 @@ public class RelayTests : WithApiTester public async Task OneTimeSetUpAsync() { await Test.MigrateSqlServerDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); - await Test.GetAndClearAzureServiceBusAsync(ServiceBusSessionReceiverOptions.CreateForTopicSubscription("contoso", "shopping")); + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); } [Test] @@ -43,4 +43,35 @@ public void Outbox_Relay() }).AssertSuccess(); }); } + + [Test] + public void Outbox_Relay_Command_Queue() + { + // Arrange the two commands to publish and relay to the per-domain command queue. + var ce1 = Test.CreateCloudEventFromJsonResource("BasketCreatedCloudEvent.json"); + var ce2 = Test.CreateCloudEventFromJsonResource("BasketUpdatedCloudEvent.json"); + + Test.ScopedType(test => + { + test.Run(async _ => + { + // Publish to the queue destination (as resolved by the NamedDestinationProvider for a command to the products domain). + var pub = ActivatorUtilities.GetServiceOrCreateInstance(test.Services); + pub.Add("contoso-products", [ce1, ce2]); + await pub.PublishAsync(); + + // Poll (rather than sleep a fixed time) as the hosted-service relay latency varies; receive the messages from the queue and assert. The relay uses the persisted destination so needs no destination provider. + var list = new List(); + for (int i = 0; i < 30 && list.Count < 2; i++) + { + await Task.Delay(TimeSpan.FromSeconds(1)); + list.AddRange(await Test.GetAndClearAzureServiceBusAsync(ServiceBusSessionReceiverOptions.CreateForQueue("contoso-products"))); + } + + list.Should().NotBeNull().And.HaveCount(2); + list.Should().ContainSingle(x => x.MessageId == ce1.Id); + list.Should().ContainSingle(x => x.MessageId == ce2.Id); + }).AssertSuccess(); + }); + } } diff --git a/samples/tests/Contoso.Shopping.Test.Subscribe/GlobalUsing.cs b/samples/tests/Contoso.Shopping.Test.Subscribe/GlobalUsing.cs index b186acfb5..6845b1937 100644 --- a/samples/tests/Contoso.Shopping.Test.Subscribe/GlobalUsing.cs +++ b/samples/tests/Contoso.Shopping.Test.Subscribe/GlobalUsing.cs @@ -1,5 +1,6 @@ global using Contoso.Shopping.Application.Adapters.Products; global using Contoso.Shopping.Contracts; +global using Contoso.Shopping.Infrastructure.Clients.SendGrid; global using CoreEx; global using CoreEx.Azure.Messaging.ServiceBus; global using CoreEx.Database.SqlServer.Outbox; @@ -10,7 +11,10 @@ global using AwesomeAssertions; global using Microsoft.Extensions.DependencyInjection; global using NUnit.Framework; +global using System.Net; +global using System.Net.Http; global using UnitTestEx; global using UnitTestEx.Expectations; +global using UnitTestEx.Mocking; global using DbMigration = Contoso.Shopping.Database.Program; global using TestData = Contoso.Shopping.Test.Common.TestData; diff --git a/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.BasketCheckedOut.cs b/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.BasketCheckedOut.cs new file mode 100644 index 000000000..12cfd8b8a --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.BasketCheckedOut.cs @@ -0,0 +1,39 @@ +namespace Contoso.Shopping.Test.Subscribe; + +public partial class SubscriberTests +{ + [Test] + public void BasketCheckedOut_SendsConfirmationEmail() => Test.Scoped(test => + { + // Arrange - simulate a basket that has just been checked out. + var basket = new Basket + { + Id = "basket-1", + CustomerId = 16.ToGuid().ToString(), + Pricing = new BasketPricing { SubTotal = 100m, Total = 100m } + }; + + var ed = new EventData().WithTitle("contoso.shopping.basket.checkedout.v1").WithValue(basket); + var ce = Test.CreateCloudEventFrom(ed); + var sbm = ce.ToServiceBusReceivedMessage(); + + // The customer's email is resolved in real-time from the Customers API (golden value Frank Foster, ^16). + _mockHttpGetCustomerRequest.Respond.WithJson(new { id = 16.ToGuid().ToString(), firstName = "Frank", lastName = "Foster", email = "frank.foster@example.com" }); + + _mockHttpSendMailRequest.WithAnyBody() + .Respond.With(HttpStatusCode.Accepted); + + // Act - receive the event which should trigger the confirmation email send. + var r = test.Run(async _ => + { + var sbs = test.Services.GetRequiredService(); + return await sbs.ReceiveAsync(sbm); + }).AssertSuccess(); + + r.Value.IsSuccess.Should().BeTrue(); + + // Assert - the Customers API and SendGrid API were invoked to send the confirmation email. + _mockHttpGetCustomerRequest.Verify(); + _mockHttpSendMailRequest.Verify(); + }); +} diff --git a/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.cs b/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.cs index 31c539430..8aabfaaed 100644 --- a/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.cs +++ b/samples/tests/Contoso.Shopping.Test.Subscribe/SubscriberTests.cs @@ -5,13 +5,27 @@ namespace Contoso.Shopping.Test.Subscribe; /// public partial class SubscriberTests : WithApiTester { + private MockHttpClientRequest _mockHttpSendMailRequest = null!; + private MockHttpClientRequest _mockHttpGetCustomerRequest = null!; + + [SetUp] + public Task SetUpAsync() => Test.ClearFusionCacheAsync(); // Customers are cached by the adapter; ensure each test starts clean so the Customers mock is invoked as expected. + [OneTimeSetUp] public async Task OneTimeSetUpAsync() { await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); await Test.ClearFusionCacheAsync().ConfigureAwait(false); + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); Test.UseExpectedSqlServerOutboxPublisher(); + + // Mock the HTTP clients (SendGrid and CustomersApi are exercised in tests; ProductsApi is registered so IProductAdapter can still be resolved by other tests in this suite). + var mcf = UnitTestEx.MockHttpClientFactory.Create(); + mcf.CreateClient("ProductsApi"); + _mockHttpGetCustomerRequest = mcf.CreateClient("CustomersApi").Request(HttpMethod.Get, $"api/customers/{16.ToGuid()}"); + _mockHttpSendMailRequest = mcf.CreateClient("SendGrid").Request(HttpMethod.Post, "v3/mail/send"); + Test.ReplaceHttpClientFactory(mcf); } [Test] @@ -33,4 +47,4 @@ public void Unsubscribed_Error() => Test.Scoped(test => e.InnerException.Message.Should().Be("No subscriber matched the event."); }).AssertSuccess(); }); -} \ No newline at end of file +} diff --git a/samples/tests/Contoso.Shopping.Test.Unit/Adapters/Customers/CustomerAdapterTests.cs b/samples/tests/Contoso.Shopping.Test.Unit/Adapters/Customers/CustomerAdapterTests.cs new file mode 100644 index 000000000..54ede6fb4 --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Unit/Adapters/Customers/CustomerAdapterTests.cs @@ -0,0 +1,81 @@ +namespace Contoso.Shopping.Test.Unit.Adapters.Customers; + +public class CustomerAdapterTests : WithGenericTester +{ + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpFoundRequest = null!; + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpNotFoundRequest = null!; + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpInvalidRequest = null!; + + [OneTimeSetUp] + public void OneTimeSetUp() + { + var mcf = UnitTestEx.MockHttpClientFactory.Create(); + var client = mcf.CreateClient("CustomersApi"); + _mockHttpFoundRequest = client.Request(HttpMethod.Get, "api/customers/cache-found"); + _mockHttpNotFoundRequest = client.Request(HttpMethod.Get, "api/customers/cache-notfound"); + _mockHttpInvalidRequest = client.Request(HttpMethod.Get, "api/customers/cache-invalid"); + Test.ReplaceHttpClientFactory(mcf); + } + + private static CustomerAdapter CreateAdapter() => new(ExecutionContext.GetRequiredService(), ExecutionContext.GetRequiredService()); + + [Test] + public void GetAsync_InvalidResponse_IsNotCached() => Test.Scoped(test => + { + _mockHttpInvalidRequest.Respond.WithJson(new { id = "cache-invalid", email = "not-an-email" }); + + test.Run(async _ => + { + var adapter = CreateAdapter(); + var invalid = await adapter.GetAsync("cache-invalid").ConfigureAwait(false); + invalid.IsFailure.Should().BeTrue(); + invalid.Error.Should().BeOfType().Subject.InnerException.Should().BeOfType(); + + _mockHttpInvalidRequest.Respond.WithJson(new { id = "cache-invalid", email = "new.customer@example.com" }); + var valid = await adapter.GetAsync("cache-invalid").ConfigureAwait(false); + valid.IsSuccess.Should().BeTrue(); + valid.Value.Email.Should().Be("new.customer@example.com"); + }).AssertSuccess(); + }); + + [Test] + public void GetAsync_Found_IsCached() => Test.Scoped(test => + { + _mockHttpFoundRequest.Respond.WithJson(new { id = "cache-found", firstName = "Frank", lastName = "Foster", email = "frank.foster@example.com" }); + + test.Run(async _ => + { + var adapter = CreateAdapter(); + var r1 = await adapter.GetAsync("cache-found").ConfigureAwait(false); + r1.IsSuccess.Should().BeTrue(); + r1.Value.Email.Should().Be("frank.foster@example.com"); + + // The remote customer is now gone; the second call must be served from the cache and not call the Customers API. + _mockHttpFoundRequest.Respond.With(HttpStatusCode.NotFound); + + var r2 = await adapter.GetAsync("cache-found").ConfigureAwait(false); + r2.IsSuccess.Should().BeTrue(); + r2.Value.Email.Should().Be("frank.foster@example.com"); + }).AssertSuccess(); + }); + + [Test] + public void GetAsync_NotFound_IsNotCached() => Test.Scoped(test => + { + _mockHttpNotFoundRequest.Respond.With(HttpStatusCode.NotFound); + + test.Run(async _ => + { + var adapter = CreateAdapter(); + var r1 = await adapter.GetAsync("cache-notfound").ConfigureAwait(false); + r1.IsNotFoundError.Should().BeTrue(); + + // The customer now exists; a not-found must never be cached, so this call must reach the Customers API. + _mockHttpNotFoundRequest.Respond.WithJson(new { id = "cache-notfound", firstName = "New", lastName = "Customer", email = "new.customer@example.com" }); + + var r2 = await adapter.GetAsync("cache-notfound").ConfigureAwait(false); + r2.IsSuccess.Should().BeTrue(); + r2.Value.Email.Should().Be("new.customer@example.com"); + }).AssertSuccess(); + }); +} diff --git a/samples/tests/Contoso.Shopping.Test.Unit/Clients/Customers/CustomersHttpClientTests.cs b/samples/tests/Contoso.Shopping.Test.Unit/Clients/Customers/CustomersHttpClientTests.cs new file mode 100644 index 000000000..e65986840 --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Unit/Clients/Customers/CustomersHttpClientTests.cs @@ -0,0 +1,135 @@ +namespace Contoso.Shopping.Test.Unit.Clients.Customers; + +public class CustomersHttpClientTests : WithGenericTester +{ + private UnitTestEx.Mocking.MockHttpClientRequest _mockHttpGetRequest = null!; + + [OneTimeSetUp] + public void OneTimeSetUp() + { + var mcf = UnitTestEx.MockHttpClientFactory.Create(); + _mockHttpGetRequest = mcf.CreateClient("CustomersApi").Request(HttpMethod.Get, "api/customers/cust-1"); + Test.ReplaceHttpClientFactory(mcf); + } + + [Test] + public void GetAsync_Found_ReturnsCustomer() => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.WithJson(new + { + id = "cust-1", + firstName = "Frank", + lastName = "Foster", + email = "frank.foster@example.com", + shippingAddress = new { street1 = "1 Test Street", city = "Springfield", postCode = "49007", state = "IL" }, + customerTypeCode = "IND" + }); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + result.IsSuccess.Should().BeTrue(); + result.Value.Id.Should().Be("cust-1"); + result.Value.Email.Should().Be("frank.foster@example.com"); + result.Value.ShippingAddress.Should().NotBeNull(); + result.Value.ShippingAddress!.City.Should().Be("Springfield"); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); + + [TestCase(null, "Email is required.")] + [TestCase("", "Email is required.")] + [TestCase("not-an-email", "Email is an invalid e-mail address.")] + public void GetAsync_InvalidResponse_ReturnsDependencyFailure(string? email, string message) => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.WithJson(new { id = "cust-1", email }); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + result.IsFailure.Should().BeTrue(); + var error = result.Error.Should().BeOfType().Subject; + error.StatusCode.Should().Be(HttpStatusCode.OK); + var messages = error.InnerException.Should().BeOfType().Which.Messages; + messages.Should().ContainSingle().Which.Property.Should().Be("email"); + messages.Should().ContainSingle().Which.Text.ToString().Should().Be(message); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); + + [Test] + public void GetAsync_EmptyObject_ReportsRequiredEmail() => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.WithJson(new { }); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + var messages = result.Error.Should().BeOfType().Subject.InnerException.Should() + .BeOfType().Which.Messages; + messages.Should().ContainSingle().Which.Property.Should().Be("email"); + messages.Should().ContainSingle().Which.Text.ToString().Should().Be("Email is required."); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); + + [TestCase(null)] + [TestCase("")] + [TestCase("cust-1")] + [TestCase("another-customer")] + public void GetAsync_ValidEmail_DoesNotValidateIdentityOrOptionalDetails(string? id) => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.WithJson(new { id, email = "frank.foster@example.com" }); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + result.IsSuccess.Should().BeTrue(); + result.Value.Id.Should().Be(id); + result.Value.Email.Should().Be("frank.foster@example.com"); + result.Value.FirstName.Should().BeNull(); + result.Value.LastName.Should().BeNull(); + result.Value.ShippingAddress.Should().BeNull(); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); + + [Test] + public void GetAsync_NotFound_ReturnsNotFoundError() => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.With(HttpStatusCode.NotFound); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + result.IsNotFoundError.Should().BeTrue(); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); + + [Test] + public void GetAsync_ServerError_ReturnsFailure() => Test.Scoped(test => + { + _mockHttpGetRequest.Respond.With(HttpStatusCode.InternalServerError); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.GetAsync("cust-1").ConfigureAwait(false); + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + }).AssertSuccess(); + + _mockHttpGetRequest.Verify(); + }); +} diff --git a/samples/tests/Contoso.Shopping.Test.Unit/Clients/SendGrid/SendGridHttpClientTests.cs b/samples/tests/Contoso.Shopping.Test.Unit/Clients/SendGrid/SendGridHttpClientTests.cs new file mode 100644 index 000000000..8b144bc12 --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Unit/Clients/SendGrid/SendGridHttpClientTests.cs @@ -0,0 +1,73 @@ +namespace Contoso.Shopping.Test.Unit.Clients.SendGrid; + +public class SendGridHttpClientTests : WithGenericTester +{ + private MockHttpClientRequest _mockHttpSendMailRequest = null!; + + [OneTimeSetUp] + public void OneTimeSetUp() + { + var mcf = UnitTestEx.MockHttpClientFactory.Create(); + _mockHttpSendMailRequest = mcf.CreateClient("SendGrid").Request(HttpMethod.Post, "v3/mail/send"); + Test.ReplaceHttpClientFactory(mcf); + } + + [Test] + public void SendMailAsync_Success_ReturnsSuccess() => Test.Scoped(test => + { + _mockHttpSendMailRequest.WithAnyBody() + .Respond.With(HttpStatusCode.Accepted); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.SendMailAsync(CreateRequest()).ConfigureAwait(false); + result.IsSuccess.Should().BeTrue(); + }).AssertSuccess(); + + _mockHttpSendMailRequest.Verify(); + }); + + [Test] + public void SendMailAsync_ServerError_ReturnsFailure() => Test.Scoped(test => + { + _mockHttpSendMailRequest.WithAnyBody() + .Respond.With(HttpStatusCode.InternalServerError); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.SendMailAsync(CreateRequest()).ConfigureAwait(false); + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + }).AssertSuccess(); + + _mockHttpSendMailRequest.Verify(); + }); + + [Test] + public void SendMailAsync_UnprocessableEntityProblemDetails_ReturnsMappedFailure() => Test.Scoped(test => + { + _mockHttpSendMailRequest.WithAnyBody() + .Respond.With(new StringContent("{\"type\":\"https://tools.ietf.org/html/rfc4918#section-11.2\",\"title\":\"Unprocessable Entity\",\"status\":422,\"detail\":\"The from address does not match a verified Sender Identity.\"}", System.Text.Encoding.UTF8, "application/problem+json"), HttpStatusCode.UnprocessableEntity); + + test.Run(async _ => + { + var client = ExecutionContext.GetRequiredService(); + var result = await client.SendMailAsync(CreateRequest()).ConfigureAwait(false); + result.IsFailure.Should().BeTrue(); + var pde = result.Error.Should().BeOfType().Subject; + pde.ProblemDetails.Status.Should().Be(422); + pde.ProblemDetails.Detail.Should().Be("The from address does not match a verified Sender Identity."); + }).AssertSuccess(); + + _mockHttpSendMailRequest.Verify(); + }); + + private static SendGridMailRequest CreateRequest() => new() + { + From = new() { Email = "orders@contoso.com", Name = "Contoso Shopping" }, + Personalizations = [new() { To = [new() { Email = "customer-1@customer.contoso.local" }], Subject = "Your Contoso order is confirmed" }], + Content = [new() { Type = "text/plain", Value = "Thanks for shopping with Contoso!" }] + }; +} diff --git a/samples/tests/Contoso.Shopping.Test.Unit/Domains/BasketTests.cs b/samples/tests/Contoso.Shopping.Test.Unit/Domains/BasketTests.cs index 52a355e23..898d94646 100644 --- a/samples/tests/Contoso.Shopping.Test.Unit/Domains/BasketTests.cs +++ b/samples/tests/Contoso.Shopping.Test.Unit/Domains/BasketTests.cs @@ -36,6 +36,19 @@ public void Basket_ApplyDiscount_Invalid_Status() => Test.Scoped(test => basket.Total.Should().Be(100m); }); + [Test] + public void Basket_CreateNew_WithShippingAddress() => Test.Scoped(test => + { + var address = new Domain.ValueObjects.Address { Street1 = "1 Main St", City = "Sydney", PostCode = "2000", State = "NSW" }; + + var basket = Domain.Basket.CreateNew("customer-id", address); + + basket.ShippingAddress.Should().Be(address); + basket.Status.Should().Be(BasketStatus.Empty); + basket.HasChanges.Should().BeTrue(); + Domain.Basket.CreateNew("customer-id").ShippingAddress.Should().BeNull(); + }); + [Test] public void Basket_UpdateShippingAddress_Success() => Test.Scoped(test => { diff --git a/samples/tests/Contoso.Shopping.Test.Unit/EntryPoint.cs b/samples/tests/Contoso.Shopping.Test.Unit/EntryPoint.cs index b89f448ef..1b3488e81 100644 --- a/samples/tests/Contoso.Shopping.Test.Unit/EntryPoint.cs +++ b/samples/tests/Contoso.Shopping.Test.Unit/EntryPoint.cs @@ -7,17 +7,25 @@ public static void ConfigureApplication(IHostApplicationBuilder builder) // Configure the minimum services required for the execution context and reference data orchestrator; caching will be in-memory for the unit tests. builder.Services.AddExecutionContext(); builder.Services.AddMemoryCache(); - builder.Services.AddReferenceDataOrchestrator(); + builder.Services.AddMemoryOnlyHybridCache(); + builder.Services.AddReferenceDataOrchestrator(); // Configure the products http client. builder.AddTypedHttpClient("ProductsApi"); + // Configure the customers http client. + builder.AddTypedHttpClient("CustomersApi"); + + // Configure the SendGrid http client. + builder.AddTypedHttpClient("SendGrid"); + builder.Services.Configure(builder.Configuration.GetSection("SendGrid")); + // Reuse the "real" database configured reference data. var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.SnakeCase)); - builder.Services.AddSingleton(new ReferenceDataServiceDecorator(jdr)); + builder.Services.AddSingleton(new ReferenceDataProviderDecorator(jdr)); } - public class ReferenceDataServiceDecorator(JsonDataReader jdr) : ReferenceDataService(Mock.Of()) + public class ReferenceDataProviderDecorator(JsonDataReader jdr) : ReferenceDataProvider(Mock.Of()) { public override Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch { diff --git a/samples/tests/Contoso.Shopping.Test.Unit/GlobalUsing.cs b/samples/tests/Contoso.Shopping.Test.Unit/GlobalUsing.cs index bddfaba38..38717e64f 100644 --- a/samples/tests/Contoso.Shopping.Test.Unit/GlobalUsing.cs +++ b/samples/tests/Contoso.Shopping.Test.Unit/GlobalUsing.cs @@ -1,17 +1,22 @@ global using AwesomeAssertions; global using Contoso.Shopping.Application; +global using Contoso.Shopping.Application.Adapters.Customers; global using Contoso.Shopping.Application.Adapters.Products; global using Contoso.Shopping.Application.Policies; global using Contoso.Shopping.Application.Repositories; global using Contoso.Shopping.Application.Validators; global using Contoso.Shopping.Contracts; +global using Contoso.Shopping.Infrastructure.Adapters.Customers; +global using Contoso.Shopping.Infrastructure.Clients.Customers; global using Contoso.Shopping.Infrastructure.Clients.Products; +global using Contoso.Shopping.Infrastructure.Clients.SendGrid; global using CoreEx; global using CoreEx.RefData; global using CoreEx.RefData.Abstractions; global using CoreEx.Results; global using CoreEx.UnitTesting; global using CoreEx.Data.Json; +global using CoreEx.Http; global using CoreEx.Validation; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Hosting; diff --git a/samples/tests/Contoso.Shopping.Test.Unit/Policies/CustomerPolicyTests.cs b/samples/tests/Contoso.Shopping.Test.Unit/Policies/CustomerPolicyTests.cs new file mode 100644 index 000000000..3f7319384 --- /dev/null +++ b/samples/tests/Contoso.Shopping.Test.Unit/Policies/CustomerPolicyTests.cs @@ -0,0 +1,35 @@ +namespace Contoso.Shopping.Test.Unit.Policies; + +public class CustomerPolicyTests : WithGenericTester +{ + private readonly Mock _customerAdapterMock = new(); + + [OneTimeSetUp] + public void OneTimeSetUp() + { + _customerAdapterMock.Setup(x => x.GetAsync("existing-customer-id", It.IsAny())) + .ReturnsAsync(Result.Go(new Customer { Id = "existing-customer-id", FirstName = "Frank", LastName = "Foster", Email = "frank.foster@example.com" })); + _customerAdapterMock.Setup(x => x.GetAsync("nonexistent-customer-id", It.IsAny())) + .ReturnsAsync(Result.NotFoundError()); + } + + [Test] + public void CustomerPolicy_EnsureExistsAsync_CustomerNotFound_ReturnsValidationError() => Test.Scoped(async test => + { + var policy = new CustomerPolicy(_customerAdapterMock.Object); + var result = await policy.EnsureExistsAsync("nonexistent-customer-id"); + result.IsFailure.Should().BeTrue(); + result.IsValidationError.Should().BeTrue(); + + result.Error.As().AssertErrors(new ApiError("customerId", "Customer was not found.")); + }); + + [Test] + public void CustomerPolicy_EnsureExistsAsync_CustomerExists_ReturnsSuccess() => Test.Scoped(async test => + { + var policy = new CustomerPolicy(_customerAdapterMock.Object); + var result = await policy.EnsureExistsAsync("existing-customer-id"); + result.IsSuccess.Should().BeTrue(); + result.Value.Email.Should().Be("frank.foster@example.com"); + }); +} diff --git a/schema/coreex-refdata.json b/schema/coreex-refdata.json index 178ff6d49..938ff1fd3 100644 --- a/schema/coreex-refdata.json +++ b/schema/coreex-refdata.json @@ -50,6 +50,16 @@ "Lowercase" ] }, + "attribute": { + "type": "string", + "title": "The optional API controller class attribute.", + "description": "This is the attribute applied as-is to the generated \u0060ReferenceDataController\u0060 class. This is useful for adding the likes of \u0060[Authorize]\u0060." + }, + "getNamed": { + "type": "boolean", + "title": "Indicates whether the API controller emits the \u0060GetNamed\u0060 operation.", + "description": "Defaults to \u0060false\u0060." + }, "repository": { "type": "string", "title": "The default repository implementation.", @@ -170,6 +180,11 @@ "title": "The route suffix.", "description": "Defaults to \u0060{Plural}\u0060 and root \u0060{RouteConvention}\u0060 configuration." }, + "attribute": { + "type": "string", + "title": "The optional API controller operation attribute.", + "description": "This is the attribute applied as-is to the generated \u0060ReferenceDataController\u0060 operation. This is useful for adding the likes of \u0060[Authorize]\u0060." + }, "repository": { "type": "string", "title": "The repository implementation.", @@ -205,6 +220,26 @@ "title": "The mapper name.", "description": "This is the .NET mapper name used within the generated code. Defaults to root \u0060{Name}Mapper\u0060." }, + "mutability": { + "type": "string", + "title": "The reference-data entity mutability.", + "description": "Defaults to \u0060None\u0060. Requires a \u0060Repository\u0060 of \u0060EntityFramework\u0060 or \u0060Cosmos\u0060.", + "enum": [ + "None", + "CreateUpdate", + "CreateUpdateDelete" + ] + }, + "validator": { + "type": "string", + "title": "The validator type name used during mutability operations.", + "description": "Defaults to \u0060ReferenceDataValidator\u003C{Name}\u003E\u0060. Must have a default constructor." + }, + "mutableAttribute": { + "type": "string", + "title": "The mutable API controller class attribute.", + "description": "Defaults to \u0060[Authorize]\u0060. A configured value replaces the default attribute applied as-is to the generated \u0060{Name}Controller\u0060 class." + }, "excludeApi": { "type": "boolean", "title": "Indicates whether to exclude the generation of the API.", diff --git a/servicebus/Config.json b/servicebus/Config.json index 47a1a331c..51f485172 100644 --- a/servicebus/Config.json +++ b/servicebus/Config.json @@ -49,6 +49,16 @@ "RequiresDuplicateDetection": false }, "Subscriptions": [ + { + "Name": "customers-relay", + "Properties": { + "DefaultMessageTimeToLive": "PT1H", + "LockDuration": "PT1M", + "MaxDeliveryCount": 10, + "DeadLetteringOnMessageExpiration": false, + "RequiresSession": true + } + }, { "Name": "products", "Properties": { @@ -71,6 +81,19 @@ } ] } + ], + "Queues": [ + { + "Name": "contoso-products", + "Properties": { + "DefaultMessageTimeToLive": "PT1H", + "LockDuration": "PT1M", + "MaxDeliveryCount": 10, + "DeadLetteringOnMessageExpiration": false, + "RequiresDuplicateDetection": false, + "RequiresSession": true + } + } ] } ], @@ -78,4 +101,4 @@ "Type": "Console" } } -} \ No newline at end of file +} diff --git a/servicebus/Reset-ServiceBusEmulator.ps1 b/servicebus/Reset-ServiceBusEmulator.ps1 new file mode 100644 index 000000000..c7899a462 --- /dev/null +++ b/servicebus/Reset-ServiceBusEmulator.ps1 @@ -0,0 +1,86 @@ +<# +.SYNOPSIS + Recovers the local Service Bus emulator after a crash-loop. + +.DESCRIPTION + The 'servicebus-emulator' container occasionally crash-loops (exit code 139 / SIGSEGV) because it + depends on 'db-sql-server' for its backing message-container databases (e.g. SbMessageContainerDatabase00001) + and neither container has a persistent volume configured. If 'db-sql-server' is recreated (or its data + directory otherwise resets) while stale/mismatched database files remain, or the emulator's expected + databases get out of sync with what's actually in SQL Server, the emulator's startup 'ALTER DATABASE' + step fails and it crashes hard on every subsequent start. + + The fix is to recreate BOTH containers together - never just the emulator alone - so they reinitialize + from a consistent, empty state. + +.NOTES + Run this from anywhere; it locates the repo root (parent of this script's folder) to find docker-compose.yml. + Prefers podman compose; falls back to docker compose if podman is not available. +#> + +[CmdletBinding()] +param( + [switch]$UseDocker +) + +$ErrorActionPreference = 'Stop' + +$repoRoot = Split-Path -Parent $PSScriptRoot +$composeFile = Join-Path $repoRoot 'docker-compose.yml' + +if (-not (Test-Path $composeFile)) { + throw "Could not find docker-compose.yml at '$composeFile'." +} + +$composeCmd = if ($UseDocker) { 'docker' } else { 'podman' } + +if (-not (Get-Command $composeCmd -ErrorAction SilentlyContinue)) { + Write-Warning "'$composeCmd' not found on PATH; falling back to 'docker'." + $composeCmd = 'docker' +} + +function Invoke-Compose { + param([string[]]$Arguments) + Write-Host "> $composeCmd compose -f `"$composeFile`" $($Arguments -join ' ')" -ForegroundColor Cyan + & $composeCmd compose -f $composeFile @Arguments + if ($LASTEXITCODE -ne 0) { + throw "Command failed with exit code $LASTEXITCODE" + } +} + +Write-Host "== Stopping db-sql-server and servicebus-emulator ==" -ForegroundColor Yellow +Invoke-Compose @('stop', 'db-sql-server', 'servicebus-emulator') + +Write-Host "== Removing db-sql-server and servicebus-emulator containers ==" -ForegroundColor Yellow +Invoke-Compose @('rm', '-f', 'db-sql-server', 'servicebus-emulator') + +Write-Host "== Recreating both containers together ==" -ForegroundColor Yellow +Invoke-Compose @('up', '-d', 'db-sql-server', 'servicebus-emulator') + +Write-Host "== Waiting for SQL Server + Service Bus emulator init (up to ~60s) ==" -ForegroundColor Yellow +$healthy = $false +$status = $null +for ($i = 0; $i -lt 12; $i++) { + Start-Sleep -Seconds 5 + # Scope to this compose project only (name=servicebus-emulator alone would also match other unrelated repos' containers on the same host). + $status = & $composeCmd compose -f $composeFile ps servicebus-emulator --format '{{.Status}}' 2>$null + if ($status -match '^Up') { + $healthy = $true + break + } + if ($status -match 'Exited') { + Write-Host " Still initializing/crash-looping ($status), waiting..." -ForegroundColor DarkYellow + } +} + +Write-Host "" +if ($healthy) { + Write-Host "servicebus-emulator is up: $status" -ForegroundColor Green +} else { + Write-Warning "servicebus-emulator did not report 'Up' within the wait window. Check logs with:" + Write-Warning " $composeCmd logs coreex-servicebus-emulator-1 --tail 60" +} + +Write-Host "" +Write-Host "== Current container status ==" -ForegroundColor Yellow +& $composeCmd compose -f $composeFile ps diff --git a/src/CoreEx.AspNetCore.NSwag/AGENTS.md b/src/CoreEx.AspNetCore.NSwag/AGENTS.md index af43b88ed..525617a86 100644 --- a/src/CoreEx.AspNetCore.NSwag/AGENTS.md +++ b/src/CoreEx.AspNetCore.NSwag/AGENTS.md @@ -28,6 +28,9 @@ app.UseSwaggerUi(); | `[IdempotencyKey]` | `Idempotency-Key` header parameter | | `[ProducesNotFoundProblem]` | `404 application/problem+json` response entry | | `[Accepts(typeof(T))]` | Request body content type and JSON schema | +| `[NonNullable]` (on a DTO property) | Schema property emitted as `nullable: false`, even if declared `string?` / `int?` | + +Non-nullability is applied recursively to request and response types, including generic wrapper arguments such as `DataMap` dictionary values and nested collections. Unannotated nullable properties retain their declared nullability; the attribute changes the schema, not runtime validation. ## Do Not diff --git a/src/CoreEx.AspNetCore.NSwag/GlobalUsing.cs b/src/CoreEx.AspNetCore.NSwag/GlobalUsing.cs index b78fc2ba2..082a1d52c 100644 --- a/src/CoreEx.AspNetCore.NSwag/GlobalUsing.cs +++ b/src/CoreEx.AspNetCore.NSwag/GlobalUsing.cs @@ -4,6 +4,7 @@ global using CoreEx.AspNetCore.Mvc; global using CoreEx.AspNetCore.NSwag; global using CoreEx.Data; +global using CoreEx.Entities; global using CoreEx.Http; global using CoreEx.Json; global using CoreEx.Results; @@ -17,4 +18,6 @@ global using NSwag.Generation.Processors.Contexts; global using System.Net; global using System.Net.Mime; -global using System.Text.Json; \ No newline at end of file +global using System.Reflection; +global using System.Text.Json; +global using System.Text.Json.Serialization; \ No newline at end of file diff --git a/src/CoreEx.AspNetCore.NSwag/NSwagOpenApiOperationProcessor.cs b/src/CoreEx.AspNetCore.NSwag/NSwagOpenApiOperationProcessor.cs index c6eca6cf9..54bd1bd3a 100644 --- a/src/CoreEx.AspNetCore.NSwag/NSwagOpenApiOperationProcessor.cs +++ b/src/CoreEx.AspNetCore.NSwag/NSwagOpenApiOperationProcessor.cs @@ -22,6 +22,7 @@ public bool Process(OperationProcessorContext context) HandleIdempotencyKeyAttribute(ctx); HandleProducesNotFoundProblemAttribute(ctx); HandleResponseProblemDetails(ctx); + HandleNonNullableAttribute(ctx); if (Options.IncludeFieldsRequestHeaders) HandleRequestFieldQueryString(ctx); @@ -160,6 +161,77 @@ private void HandleResponseProblemDetails(AspNetCoreOperationProcessorContext co } } + /// + /// Handles the by marking the corresponding schema properties as non-nullable, regardless of the nullability declared in code. + /// + /// Walks the types reachable from the operation request and response bodies; the walk is idempotent so it is safe for types shared across operations. + private void HandleNonNullableAttribute(AspNetCoreOperationProcessorContext context) + { + var visited = new HashSet(); + var api = context.ApiDescription; + + foreach (var p in api.ParameterDescriptions) + ApplyNotNull(context, p.Type, visited); + + foreach (var r in api.SupportedResponseTypes) + ApplyNotNull(context, r.Type, visited); + + if (api.ActionDescriptor.EndpointMetadata.FirstOrDefault(x => x is AcceptsAttribute) is AcceptsAttribute accepts) + ApplyNotNull(context, accepts.BodyType, visited); + } + + /// + /// Applies the to the schema for the and recursively to its generic arguments and property types. + /// + private void ApplyNotNull(AspNetCoreOperationProcessorContext context, Type? type, HashSet visited) + { + if (type is null || type == typeof(void)) + return; + + if (type.IsArray) + { + ApplyNotNull(context, type.GetElementType(), visited); + return; + } + + if (Nullable.GetUnderlyingType(type) is Type underlying) + { + ApplyNotNull(context, underlying, visited); + return; + } + + if (!visited.Add(type)) + return; + + if (type.IsGenericType) + { + foreach (var arg in type.GetGenericArguments()) + ApplyNotNull(context, arg, visited); + } + + // Framework types have nothing to mark up beyond their generic arguments. + if (type.IsEnum || type.IsPrimitive || type.Namespace is null || IsFrameworkNamespace(type.Namespace, "System") || IsFrameworkNamespace(type.Namespace, "Microsoft")) + return; + + var schema = context.SchemaGenerator.Generate(type, context.SchemaResolver).ActualSchema; + var jso = (context.SchemaGenerator.Settings as SystemTextJsonSchemaGeneratorSettings)?.SerializerOptions ?? Options.JsonSerializerOptions; + + foreach (var pi in type.GetProperties(BindingFlags.Public | BindingFlags.Instance)) + { + var name = pi.GetCustomAttribute()?.Name ?? jso?.PropertyNamingPolicy?.ConvertName(pi.Name) ?? pi.Name; + if (!schema.ActualProperties.TryGetValue(name, out var property)) + continue; + + if (pi.IsDefined(typeof(NonNullableAttribute), true)) + { + property.IsNullableRaw = false; + property.Type &= ~JsonObjectType.Null; + } + + ApplyNotNull(context, pi.PropertyType, visited); + } + } + /// /// Gets or generates the schema for the specified . /// @@ -210,4 +282,9 @@ private void HandleRequestFieldQueryString(AspNetCoreOperationProcessorContext c Kind = OpenApiParameterKind.Query, Schema = new JsonSchema { Type = type } }; + + /// + /// Determines whether the is the framework namespace or one of its dotted sub-namespaces (e.g. System.Collections, but not Systematic). + /// + private static bool IsFrameworkNamespace(string ns, string root) => ns.StartsWith(root, StringComparison.Ordinal) && (ns.Length == root.Length || ns[root.Length] == '.'); } \ No newline at end of file diff --git a/src/CoreEx.AspNetCore.NSwag/README.md b/src/CoreEx.AspNetCore.NSwag/README.md index 3b74d6065..fb82ea490 100644 --- a/src/CoreEx.AspNetCore.NSwag/README.md +++ b/src/CoreEx.AspNetCore.NSwag/README.md @@ -16,6 +16,7 @@ - 🔍 **Query parameters**: Reads `[QueryAttribute]` and adds `$filter` and/or `$orderby` query-string parameters when `SupportsFilter` / `SupportsOrderBy` are set. - 📦 **Request body content types**: Reads `[AcceptsAttribute]` and populates the operation `RequestBody` with the declared content type(s) and NSwag-inferred JSON schema for the body type. - 🔑 **Idempotency-key header**: Reads `[IdempotencyKeyAttribute]` and adds an `Idempotency-Key` header parameter to the operation. +- ✅ **Non-nullable properties**: Walks the request/response types of each operation and marks any property decorated with `CoreEx.Entities.NonNullableAttribute` as `nullable: false` in the schema, regardless of whether it is declared nullable in code (e.g. `string?`, `int?`). - 🚫 **Not-found response**: Reads `[ProducesNotFoundProblemAttribute]` and adds a `404 application/problem+json` response entry. - ⚠️ **ProblemDetails responses**: Optionally injects `application/problem+json` response entries for the HTTP status codes configured via `OpenApiOptions.IncludeProblemDetailsHttpStatusCodes` / `OpenApiOptions.IncludeValidationProblemDetailsHttpStatusCodes` and their corresponding status-code lists; `500` is only included when it is present in the configured list, not by default via a separate `IncludeStandardProblemDetailsResponses` option. - 📡 **Fields query string**: When `OpenApiOptions.IncludeFieldsRequestHeaders` is set, adds the `$fields` query-string parameter for response field projection. diff --git a/src/CoreEx.AspNetCore/Abstractions/WebApi.MergePatch.cs b/src/CoreEx.AspNetCore/Abstractions/WebApi.MergePatch.cs index acec6d708..e94778591 100644 --- a/src/CoreEx.AspNetCore/Abstractions/WebApi.MergePatch.cs +++ b/src/CoreEx.AspNetCore/Abstractions/WebApi.MergePatch.cs @@ -14,7 +14,11 @@ public abstract partial class WebApi /// The resulting . public Task PatchAsync(HttpRequest request, Func, CancellationToken, Task> get, Func, CancellationToken, Task> put, HttpStatusCode statusCode = HttpStatusCode.OK, CancellationToken cancellationToken = default) => PatchWithResultAsync(request, - async (ro, ct) => Result.Ok(await get(ro, ct).ConfigureAwait(false)), + async (ro, ct) => + { + var gv = await get(ro, ct).ConfigureAwait(false); + return gv is null ? Result.NotFoundError() : Result.Ok(gv); + }, async (ro, ct) => Result.Ok(await put(ro, ct).ConfigureAwait(false)), statusCode, cancellationToken); @@ -28,7 +32,7 @@ public Task PatchAsync(HttpRequest request, FuncThe where result is not . /// The . /// The resulting . - public async Task PatchWithResultAsync(HttpRequest request, Func, CancellationToken, Task>> get, Func, CancellationToken, Task>> put, HttpStatusCode statusCode = HttpStatusCode.OK, CancellationToken cancellationToken = default) + public async Task PatchWithResultAsync(HttpRequest request, Func, CancellationToken, Task>> get, Func, CancellationToken, Task>> put, HttpStatusCode statusCode = HttpStatusCode.OK, CancellationToken cancellationToken = default) { CheckRequest(request, [HttpMethods.Patch]); get.ThrowIfNull(); @@ -63,9 +67,8 @@ public async Task PatchWithResultAsync(HttpRequest request, Fun var mpr = await JsonMergePatch.MergeWithResultAsync(content, async ct => { - // Perform the get operation to retrieve the current value. return Result.Go(await _invoker.InvokeAsync(this, async (_, cancellationToken) => await get(gro, cancellationToken).ConfigureAwait(false), cancellationToken, $"{nameof(PatchAsync)}::{nameof(get)}").ConfigureAwait(false)) - .Then(gv => + .Then(gv => { if (gv is null || gv is not IReadOnlyETag etag) return gv; @@ -103,4 +106,4 @@ public async Task PatchWithResultAsync(HttpRequest request, Fun return CreateResult(CreateContentForValue(pro, value)); }, cancellationToken, nameof(PatchAsync)).ConfigureAwait(false); } -} \ No newline at end of file +} diff --git a/src/CoreEx.AspNetCore/Abstractions/WebApi.cs b/src/CoreEx.AspNetCore/Abstractions/WebApi.cs index 4075fd02b..3302eb845 100644 --- a/src/CoreEx.AspNetCore/Abstractions/WebApi.cs +++ b/src/CoreEx.AspNetCore/Abstractions/WebApi.cs @@ -37,7 +37,7 @@ protected async Task InvokeAsync(HttpRequest request, Func(request.HttpContext.Response) { Exception = ex }); } @@ -165,7 +165,7 @@ private WebApiResult CreateContentForValue(WebApiOptionsBase options { return await request.ReadFromJsonAsync(JsonSerializerOptions, cancellationToken).ConfigureAwait(false); } - catch (Exception ex) + catch (Exception ex) when (!ex.IsCanceledBy(cancellationToken)) // A client disconnect is not an invalid request body. { return new ValidationException(RequestBodyInvalidText.WithArgs(ex.Message)).WithErrorType(RequestBodyErrorType); } diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/AGENTS.md b/src/CoreEx.Azure.Messaging.ServiceBus/AGENTS.md index 22dc72288..b715b474c 100644 --- a/src/CoreEx.Azure.Messaging.ServiceBus/AGENTS.md +++ b/src/CoreEx.Azure.Messaging.ServiceBus/AGENTS.md @@ -20,6 +20,10 @@ builder.Services.AddAzureServiceBusPublisher((_, c) => }); ``` +`SessionIdPartitionSize` can be configured independently with `CoreEx:Host:ServiceBus:SessionIdPartitionSize` (default: `PartitionKey.DefaultPartitionSize`). Exact destination names can override `SessionIdStrategy` and/or `SessionIdPartitionSize` under `CoreEx:Host:ServiceBus:Destinations:{destination}`. Prefer stable entity keys and tune bounded bucketing only where the ordering/concurrency profile warrants it. Do not reuse the relational outbox setting: direct Service Bus publishing can bypass the outbox, and the two sizes control separate ordering and concurrency boundaries. + +Changing a destination's session strategy or bucket count can move the same key to a different session. A rolling deployment with mixed producer policies can therefore break per-key ordering. Pause all affected producers (including direct publishers), drain upstream outboxes and affected broker queues/subscriptions and in-flight handlers, then switch policies consistently before resuming. Changing receiver concurrency alone does not change the producer's key-to-session mapping. + ## Subscribe Host Wiring ```csharp @@ -42,6 +46,24 @@ builder.Services.AzureServiceBusReceiving() .Build(); ``` +To consume more than one entity from a single host (e.g. the shared event topic subscription **and** the domain's command queue, as routed by `NamedDestinationProvider`), give each receiver its own service keys so they do not collide. The hosted-service key is also the health-check and hosted-service management name, so make it descriptive: + +```csharp +builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForTopicSubscription(), "receiver-events") + .WithKeyedSubscribedSubscriber("subscriber-events") + .WithHostedService("hosted-subscriber-events") + .Build(); + +builder.Services.AzureServiceBusReceiving() + .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForQueue("contoso-products"), "receiver-commands") + .WithKeyedSubscribedSubscriber("subscriber-commands") + .WithHostedService("hosted-subscriber-commands") + .Build(); +``` + +Keyed subscribers are resolved in tests via `GetRequiredKeyedService("subscriber-events")`. + ## Subscriber Classes Subscribers are decorated with `[Subscribe("subject")]` and extend `SubscribedBase` (untyped) or `SubscribedBase` (typed payload). Register with `[ScopedService]`. diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/Abstractions/ServiceBusReceiverBaseT.cs b/src/CoreEx.Azure.Messaging.ServiceBus/Abstractions/ServiceBusReceiverBaseT.cs index a8919027f..fc6af8de3 100644 --- a/src/CoreEx.Azure.Messaging.ServiceBus/Abstractions/ServiceBusReceiverBaseT.cs +++ b/src/CoreEx.Azure.Messaging.ServiceBus/Abstractions/ServiceBusReceiverBaseT.cs @@ -52,6 +52,10 @@ protected sealed async override Task OnProcessMessageAsync(ServiceBusRec result = Result.Fail(ex); } + // A cancellation attributable to this receive's own token (e.g. shutdown) must bubble up unclassified; it must not be converted (and logged) as per the configured UnhandledErrorHandling (e.g. DeadLetter). + if (result.IsFailure && result.Error.IsCanceledBy(ctx.CancellationToken)) + return result; + // Determine where unhandled what the final error handling will be; otherwise, success. return MessageUnhandledErrorDetermination(result, state.Owner.Options, state.Owner.Logger); }, ctx, (subscriber, message, esa, this)); @@ -66,7 +70,7 @@ protected sealed async override Task OnProcessMessageAsync(ServiceBusRec // A cancellation attributable to *this* receive's own cancellationToken (e.g. a host/processor shutdown) is intended to bubble up unclassified (see CoreEx.Events.Subscribing.EventSubscriberBase); // do not log it as an unhandled error nor attempt any message action using the very token that just got cancelled (which the Azure SDK would likely reject immediately) - simply let the message be // left for natural redelivery once its lock expires. - if (result.Error is OperationCanceledException oce && oce.CancellationToken == cancellationToken) + if (result.Error.IsCanceledBy(cancellationToken)) return result; // Handle the error accordingly; a) retry error conversion, b) invoke message action, then c) pause where critical. diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/CoreExServiceBusExtensions.DependencyInjection.cs b/src/CoreEx.Azure.Messaging.ServiceBus/CoreExServiceBusExtensions.DependencyInjection.cs index c92827a85..111e0172e 100644 --- a/src/CoreEx.Azure.Messaging.ServiceBus/CoreExServiceBusExtensions.DependencyInjection.cs +++ b/src/CoreEx.Azure.Messaging.ServiceBus/CoreExServiceBusExtensions.DependencyInjection.cs @@ -22,9 +22,45 @@ public static IServiceCollection AddAzureServiceBusPublisher(this IServiceCollec { var sbp = ActivatorUtilities.CreateInstance(sp); configure?.Invoke(sp, sbp); + var partitionSize = CoreEx.Abstractions.Internal.GetConfigurationValue(ServiceBusPublisher.SessionIdPartitionSizeConfigurationKey, null, sp.GetService()); + if (partitionSize is not null) + sbp.SessionIdPartitionSize = PartitionKey.ValidatePartitionSize(partitionSize.Value); + + if (sp.GetService() is { } configuration) + ConfigureDestinationSettings(sbp, configuration); + return sbp; }, addAsDefaultIEventPublisher); + /// + /// Applies destination-specific session settings from configuration. + /// + private static void ConfigureDestinationSettings(ServiceBusPublisher publisher, IConfiguration configuration) + { + foreach (var section in configuration.GetSection("CoreEx:Host:ServiceBus:Destinations").GetChildren()) + { + ServiceBusSessionStrategy? strategy = null; + if (section["SessionIdStrategy"] is { } strategyValue) + { + if (!Enum.TryParse(strategyValue, true, out var parsedStrategy) || !Enum.IsDefined(parsedStrategy)) + throw new InvalidOperationException($"The Service Bus destination '{section.Key}' has an invalid SessionIdStrategy value '{strategyValue}'."); + + strategy = parsedStrategy; + } + + var size = CoreEx.Abstractions.Internal.GetConfigurationValue($"{section.Path}:SessionIdPartitionSize", null, configuration); + if (size is not null) + size = PartitionKey.ValidatePartitionSize(size.Value); + + if (!publisher.DestinationSettings.TryGetValue(section.Key, out var settings)) + settings = new ServiceBusDestinationSettings(); + + settings.SessionIdStrategy = strategy ?? settings.SessionIdStrategy; + settings.SessionIdPartitionSize = size ?? settings.SessionIdPartitionSize; + publisher.DestinationSettings[section.Key] = settings; + } + } + /// /// Adds a singleton Azure service. /// @@ -66,15 +102,17 @@ public sealed class AzureServiceBusReceiveServiceBuilder /// Adds a singleton Azure service enabling ongoing fluent-style method-chaining registration. /// /// The factory to create the required . + /// The optional service key for the receiver; required to be unique where more than one receiver of the same type is registered (e.g. a topic subscription and a queue). /// The for fluent-style method-chaining. - public AzureServiceBusReceiverService WithReceiver(Func optionsFactory) => new(new(_services, null, optionsFactory)); + public AzureServiceBusReceiverService WithReceiver(Func optionsFactory, object? serviceKey = null) => new(new(_services, serviceKey, optionsFactory)); /// /// Adds a singleton Azure service enabling ongoing fluent-style method-chaining registration. /// /// The factory to create the required . + /// The optional service key for the receiver; required to be unique where more than one receiver of the same type is registered (e.g. a topic subscription and a queue). /// The for fluent-style method-chaining. - public AzureServiceBusSessionReceiverService WithSessionReceiver(Func optionsFactory) => new(new(_services, null, optionsFactory)); + public AzureServiceBusSessionReceiverService WithSessionReceiver(Func optionsFactory, object? serviceKey = null) => new(new(_services, serviceKey, optionsFactory)); /// /// Holds the state (, service key and options factory) shared by both the queue/topic and session receiver-service registration builders, so it is captured once rather diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/README.md b/src/CoreEx.Azure.Messaging.ServiceBus/README.md index 736713c00..a2585100c 100644 --- a/src/CoreEx.Azure.Messaging.ServiceBus/README.md +++ b/src/CoreEx.Azure.Messaging.ServiceBus/README.md @@ -8,6 +8,33 @@ **Publishing** is handled by `ServiceBusPublisher`, which inherits `EventPublisherBase` and implements `OnPublishAsync` using the SDK's safe-batch send API. Each `DestinationEvent` is converted from a `CloudEvent` to a `ServiceBusMessage` (structured or binary content mode), with CloudEvents attributes optionally written to message application properties. Session-ID assignment is controlled by the `ServiceBusSessionStrategy` enum. +When `UsePartitionKeyConvertedToAnId` is selected, `CoreEx:Host:ServiceBus:SessionIdPartitionSize` controls the publisher-wide session bucket count (default `4`). Destination-specific settings can override the strategy and/or bucket count when destinations have different ordering or throughput needs: + +```json +{ + "CoreEx": { + "Host": { + "ServiceBus": { + "SessionIdPartitionSize": 8, + "Destinations": { + "contoso": { + "SessionIdStrategy": "UsePartitionKeyAsIs" + }, + "contoso-products": { + "SessionIdStrategy": "UsePartitionKeyConvertedToAnId", + "SessionIdPartitionSize": 2 + } + } + } + } + } +} +``` + +The destination keys are the exact Service Bus entity names. Unspecified destination values inherit the publisher-wide strategy or bucket count. This setting is intentionally separate from relational outbox partition sizing because hosts may publish directly to Service Bus as well as through an outbox relay. + +Prefer a stable business/aggregate key with `UsePartitionKeyAsIs` when ordering is required per entity; Service Bus then serializes each session while different sessions can be processed concurrently. Use `UsePartitionKeyConvertedToAnId` only when bounding session cardinality is useful, accepting that unrelated keys may collide and serialize together. More buckets do not themselves increase throughput: the receiving host must also allow enough concurrent sessions. A command destination's lower message volume does not inherently require a smaller count; tune it only when measurement or an explicit ordering/concurrency requirement justifies it. Avoid changing a destination's bucketing profile while relevant messages are queued or in flight, because a changed count may route the same key to another session. + **Subscribing** is layered: `ServiceBusSubscriberBase` extends `EventSubscriberBase` to accept a raw `ServiceBusReceivedMessage`, converting it to a `CloudEvent` before delegating upward. `ServiceBusSubscribedSubscriber` adds `SubscribedManager` dispatch so that `[Subscribe]`-decorated handlers are resolved automatically from the message subject and source. The `ServiceBusReceiver` and `ServiceBusSessionReceiver` classes wrap the SDK `ServiceBusProcessor` / `ServiceBusSessionProcessor` lifetimes, and `ServiceBusReceiverHostedService` integrates them with the .NET hosted-service model including pause/resume and health-check support. Resiliency is provided out-of-the-box: `ServiceBusReceiverResiliency` supplies factory methods for a receiver-level circuit breaker and a per-message retry pipeline (via Polly), both of which are applied by default in `ServiceBusReceiverOptionsBase`. Both factories are now thin wrappers over `CoreEx.Hosting`'s generic `CircuitBreakerResiliency`/`RetryResiliency` (promoted there so `CoreEx.Cosmos`'s Change Feed Processor-based outbox relay shares the exact same self-pause/self-resume behaviour) - this package supplies only the Service-Bus-specific pause-reason/dead-letter-exclusion/retry-classification wiring. @@ -28,6 +55,7 @@ Resiliency is provided out-of-the-box: `ServiceBusReceiverResiliency` supplies f | Type | Description | |------|-------------| | **[`ServiceBusPublisher`](./ServiceBusPublisher.cs)** | `IEventPublisher` implementation for Azure Service Bus; sends batched `ServiceBusMessage` objects, supports structured/binary CloudEvents content mode and session-ID strategies. | +| **[`ServiceBusDestinationSettings`](./ServiceBusDestinationSettings.cs)** | Optional per-destination overrides for the session-ID strategy and hashed session bucket count. | | _[`ServiceBusSubscriberBase`](./ServiceBusSubscriberBase.cs)_ | Abstract `EventSubscriberBase` for Service Bus; accepts a raw `ServiceBusReceivedMessage`, converts it to a `CloudEvent`, and provides an `OnBeforeReceiveAsync` hook. | | **[`ServiceBusSubscribedSubscriber`](./ServiceBusSubscribedSubscriber.cs)** | Concrete `ServiceBusSubscriberBase` that uses `SubscribedManager` to dispatch each message to the matching `[Subscribe]`-decorated handler. | | **[`ServiceBusReceiver`](./ServiceBusReceiver.cs)** | Manages a `ServiceBusProcessor` lifetime (start/pause/resume/stop) and delegates each received message to the scoped `TSubscriber`. | @@ -57,4 +85,4 @@ Resiliency is provided out-of-the-box: `ServiceBusReceiverResiliency` supplies f ## AI Usage Guide -An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. \ No newline at end of file +An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusDestinationSettings.cs b/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusDestinationSettings.cs new file mode 100644 index 000000000..543686864 --- /dev/null +++ b/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusDestinationSettings.cs @@ -0,0 +1,18 @@ +namespace CoreEx.Azure.Messaging.ServiceBus; + +/// +/// Provides optional destination-specific Service Bus session settings. +/// +public sealed class ServiceBusDestinationSettings +{ + /// + /// Gets or sets the destination-specific session strategy, or to use the publisher default. + /// + public ServiceBusSessionStrategy? SessionIdStrategy { get; set; } + + /// + /// Gets or sets the destination-specific session bucket count when is + /// , or to use the publisher default. + /// + public int? SessionIdPartitionSize { get; set; } +} diff --git a/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusPublisher.cs b/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusPublisher.cs index 511fbab52..9b0117705 100644 --- a/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusPublisher.cs +++ b/src/CoreEx.Azure.Messaging.ServiceBus/ServiceBusPublisher.cs @@ -46,13 +46,24 @@ public sealed class ServiceBusPublisher(ServiceBusClient serviceBusClient, IDest /// /// Gets or sets the size of the partition used for when the is . /// - /// Where not specified the is used. + /// Where not specified the is used. The optional configuration key is and is independent of relational outbox partition sizing. public int? SessionIdPartitionSize { get; set; } + /// + /// Gets the session settings keyed by the exact Service Bus destination name. + /// + /// Destination settings override the corresponding publisher-wide setting; unspecified values inherit from the publisher. + public IDictionary DestinationSettings { get; } = new Dictionary(StringComparer.Ordinal); + + /// + /// Gets the configuration key for Service Bus session partition sizing. + /// + public const string SessionIdPartitionSizeConfigurationKey = "CoreEx:Host:ServiceBus:SessionIdPartitionSize"; + /// /// Gets the default value: '$none'. /// - public const string DefaultNoPartitionKeySessionId = "$none"; + public const string DefaultNoPartitionKeySessionId = PartitionKey.DefaultNoPartitionKey; /// /// Gets or sets the fixed value used to derive the for an event with no , regardless of ; @@ -110,12 +121,12 @@ private async Task SendBatchAsync(int totalEventCount, string destination, Queue using var batch = await sender.CreateMessageBatchAsync(cancellationToken).ConfigureAwait(false); // Add first message to the batch. - if (batch.TryAddMessage(SetSessionId(events.Peek().Event.ToServiceBusMessage(ContentMode, IncludeCloudEventAttributes)))) + if (batch.TryAddMessage(SetSessionId(events.Peek().Event.ToServiceBusMessage(ContentMode, IncludeCloudEventAttributes), destination))) { events.Dequeue(); // Keep adding messages until we run out of messages or batch is full. - while (events.Count > 0 && batch.TryAddMessage(SetSessionId(events.Peek().Event.ToServiceBusMessage(ContentMode, IncludeCloudEventAttributes)))) + while (events.Count > 0 && batch.TryAddMessage(SetSessionId(events.Peek().Event.ToServiceBusMessage(ContentMode, IncludeCloudEventAttributes), destination))) { events.Dequeue(); } @@ -191,14 +202,17 @@ await Invoker.InvokeAsync(this, async (tracer, cancellationToken) => /// /// Sets the based on the configured . /// - private ServiceBusMessage SetSessionId(ServiceBusMessage message) + private ServiceBusMessage SetSessionId(ServiceBusMessage message, string destination) { message.ThrowIfNull(); + var settings = DestinationSettings.TryGetValue(destination, out var destinationSettings) ? destinationSettings : null; + var strategy = settings?.SessionIdStrategy ?? SessionIdStrategy; + var partitionSize = settings?.SessionIdPartitionSize ?? SessionIdPartitionSize ?? PartitionKey.DefaultPartitionSize; - return SessionIdStrategy switch + return strategy switch { - ServiceBusSessionStrategy.UsePartitionKeyAsIs => message.Adjust(message => message.SessionId = message.PartitionKey ?? LogNoPartitionKeyFallback(message, NoPartitionKeySessionId)), - ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId => message.Adjust(message => message.SessionId = message.PartitionKey = PartitionKey.GetPartitionIdAsString(message.PartitionKey ?? LogNoPartitionKeyFallback(message, NoPartitionKeySessionId), SessionIdPartitionSize ?? PartitionKey.DefaultPartitionSize)), + ServiceBusSessionStrategy.UsePartitionKeyAsIs => message.Adjust(message => message.SessionId = message.PartitionKey ?? LogNoPartitionKeyFallback(message, NoPartitionKeySessionId, strategy)), + ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId => message.Adjust(message => message.SessionId = message.PartitionKey = PartitionKey.GetPartitionIdAsString(message.PartitionKey ?? LogNoPartitionKeyFallback(message, NoPartitionKeySessionId, strategy), PartitionKey.ValidatePartitionSize(partitionSize))), _ => message }; } @@ -206,10 +220,10 @@ private ServiceBusMessage SetSessionId(ServiceBusMessage message) /// /// Logs (at ) that an event with no is falling back to the specified for session assignment. /// - private string LogNoPartitionKeyFallback(ServiceBusMessage message, string fallbackValue) + private string LogNoPartitionKeyFallback(ServiceBusMessage message, string fallbackValue, ServiceBusSessionStrategy strategy) { if (Logger?.IsEnabled(LogLevel.Debug) ?? false) - Logger.LogDebug("Event (Id={MessageId}) has no PartitionKey; falling back to '{FallbackValue}' for {SessionIdStrategy} session assignment.", message.MessageId, fallbackValue, SessionIdStrategy); + Logger.LogDebug("Event (Id={MessageId}) has no PartitionKey; falling back to '{FallbackValue}' for {SessionIdStrategy} session assignment.", message.MessageId, fallbackValue, strategy); return fallbackValue; } diff --git a/src/CoreEx.Caching.FusionCache/AGENTS.md b/src/CoreEx.Caching.FusionCache/AGENTS.md index abf984a01..a8387fbb8 100644 --- a/src/CoreEx.Caching.FusionCache/AGENTS.md +++ b/src/CoreEx.Caching.FusionCache/AGENTS.md @@ -39,6 +39,8 @@ public class ReferenceDataService(IHybridCache cache) } ``` +For a `Result`-returning factory (e.g. a real-time adapter call) use `GetOrCreateWithResultAsync` / `GetOrCreateByKeyWithResultAsync`: only successes are cached, and `FusionHybridCache` makes concurrent misses invoke the factory once per key per node (cross-node needs an opt-in FusionCache distributed locker). + ## Do Not - Do not inject `IFusionCache` directly into application or domain services — use `IHybridCache`. diff --git a/src/CoreEx.Caching.FusionCache/FusionHybridCache.cs b/src/CoreEx.Caching.FusionCache/FusionHybridCache.cs index 2f9273a7a..2fc9d8fbe 100644 --- a/src/CoreEx.Caching.FusionCache/FusionHybridCache.cs +++ b/src/CoreEx.Caching.FusionCache/FusionHybridCache.cs @@ -28,6 +28,29 @@ public class FusionHybridCache(IFusionCache fusionCache, ICacheKeyProvider cache public async Task GetOrCreateByKeyAsync(string key, Func> factory, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => await _fusionCache.GetOrSetAsync(KeyProvider.GetFullyQualifiedCacheKey(key), async ct => await factory(ct).ConfigureAwait(false), ConfigureEntryOptions(options), QualifyTags(options?.Tags), cancellationToken).ConfigureAwait(false); + /// + /// Concurrent misses for the same key invoke the one at a time (per node by default, courtesy of FusionCache's memory locker; cross-node coalescing requires configuring a FusionCache distributed locker), with later callers served from the cache where the first succeeded. + /// A failure is not cached by skipping the memory and distributed cache writes via the adaptive factory context (rather than throwing, which would be logged as a factory error). + public async Task> GetOrCreateByKeyWithResultAsync(string key, Func>> factory, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) + { + factory.ThrowIfNull(); + + Result? failure = null; + var value = await _fusionCache.GetOrSetAsync(KeyProvider.GetFullyQualifiedCacheKey(key), async (ctx, ct) => + { + var result = await factory(ct).ConfigureAwait(false); + if (result.IsSuccess) + return result.Value; + + failure = result; + ctx.Options.SkipMemoryCacheWrite = true; + ctx.Options.SkipDistributedCacheWrite = true; + return default!; + }, default, ConfigureEntryOptions(options), QualifyTags(options?.Tags), cancellationToken).ConfigureAwait(false); + + return failure ?? value; + } + /// public async Task SetByKeyAsync(string key, T value, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => await _fusionCache.SetAsync(KeyProvider.GetFullyQualifiedCacheKey(key), value, ConfigureEntryOptions(options), QualifyTags(options?.Tags), cancellationToken).ConfigureAwait(false); diff --git a/src/CoreEx.Caching.FusionCache/GlobalUsings.cs b/src/CoreEx.Caching.FusionCache/GlobalUsings.cs index 67e83b15c..ef6ee96f0 100644 --- a/src/CoreEx.Caching.FusionCache/GlobalUsings.cs +++ b/src/CoreEx.Caching.FusionCache/GlobalUsings.cs @@ -1,3 +1,4 @@ global using CoreEx; global using CoreEx.Caching; +global using CoreEx.Results; global using ZiggyCreatures.Caching.Fusion; \ No newline at end of file diff --git a/src/CoreEx.Caching.FusionCache/README.md b/src/CoreEx.Caching.FusionCache/README.md index 18039a97b..dac6abe3e 100644 --- a/src/CoreEx.Caching.FusionCache/README.md +++ b/src/CoreEx.Caching.FusionCache/README.md @@ -10,7 +10,7 @@ The package is deliberately thin: `FusionHybridCache` delegates every cache oper ## Key capabilities -- 🧩 **`IHybridCache` implementation**: `FusionHybridCache` fulfils all `IHybridCache` contract methods — `TryGetByKeyAsync`, `GetOrDefaultByKeyAsync`, `GetOrCreateByKeyAsync`, `SetByKeyAsync`, `RemoveByKeyAsync`, and `RemoveByTagAsync` — against an `IFusionCache` instance. +- 🧩 **`IHybridCache` implementation**: `FusionHybridCache` fulfils all `IHybridCache` contract methods — `TryGetByKeyAsync`, `GetOrDefaultByKeyAsync`, `GetOrCreateByKeyAsync`, `SetByKeyAsync`, `RemoveByKeyAsync`, and `RemoveByTagAsync` — against an `IFusionCache` instance. It also overrides `GetOrCreateByKeyWithResultAsync`, which caches **only successful** `Result` values and uses FusionCache's single-flight, so concurrent misses invoke the factory once per key per node; coalescing across nodes needs an opt-in FusionCache distributed locker (`.WithDistributedLocker(...)`). - 🔄 **Options translation**: `FusionCacheExtensions.ToFusionCacheEntryOptions` maps `HybridCacheEntryOptions` (`LocalExpiration`, `DistributedExpiration`, `CacheStrategy`, `Tags`) directly to a `FusionCacheEntryOptions` instance, including `SkipDistributedCache*` and `SkipMemoryCache*` flags derived from `CacheStrategy`. - 📦 **L1/L2 and backplane support**: relies on FusionCache's own backplane packages (`ZiggyCreatures.FusionCache.Backplane.StackExchangeRedis`, `ZiggyCreatures.FusionCache.Backplane.Memory`) and serializer (`ZiggyCreatures.FusionCache.Serialization.SystemTextJson`) that are bundled as transitive dependencies. - 🏷 **Tag-based invalidation**: `RemoveByTagAsync` delegates to FusionCache's native tag-eviction API, allowing entire categories of cache entries to be invalidated by a single call. diff --git a/src/CoreEx.CodeGen/AGENTS.md b/src/CoreEx.CodeGen/AGENTS.md index fd87effa8..d982598ca 100644 --- a/src/CoreEx.CodeGen/AGENTS.md +++ b/src/CoreEx.CodeGen/AGENTS.md @@ -1,6 +1,6 @@ # CoreEx.CodeGen — AI Usage Guide -`CoreEx.CodeGen` is a **development-time** code generation tool — it is never deployed at runtime. It reads a `ref-data.yaml` file and generates the complete reference-data layer (contract, controller, service, repository, mapper) as `.g.cs` files. +`CoreEx.CodeGen` is a **development-time** code generation tool — it is never deployed at runtime. It reads a `ref-data.yaml` file and generates the complete reference-data layer (contract, controller, provider, repository, mapper — plus service and write endpoints for mutable entities) as `.g.cs` files. ## Setup @@ -35,16 +35,53 @@ Run the project (`dotnet run`) to regenerate all `.g.cs` files after changing th |---|---|---| | `{Name}.g.cs` (Contracts project root) | Contracts | `ref-data.yaml` entity/property config | | `Controllers/ReferenceDataController.g.cs` | Api | `ref-data.yaml` route/entity config | -| `ReferenceDataService.g.cs` (Application project root) | Application | `ref-data.yaml` entity config | +| `ReferenceDataProvider.g.cs` (Application project root) — `IReferenceDataProvider` for the orchestrator | Application | `ref-data.yaml` entity config | +| `Controllers/{Name}Controller.g.cs` — **mutable entities only** | Api | `mutability` / `mutableAttribute` | +| `Interfaces/IReferenceDataService.g.cs` + `ReferenceDataService.g.cs` — **only if any entity is mutable** | Application | `mutability` / `validator` | | `Repositories/IReferenceDataRepository.g.cs` | Application | `ref-data.yaml` entity/repository config | | `Repositories/ReferenceDataRepository.g.cs` | Infrastructure | `ref-data.yaml` repository config | | `Mapping/{Name}Mapper.g.cs` | Infrastructure | `ref-data.yaml` property/mapping config | `Repositories`/`Mapping` are the default `dataRepositoriesPath`/`dataMappingPath` folder names — both configurable in `ref-data.yaml`. +## Readonly vs Mutation + +Entities are **read-only by default** (`mutability: None`). Set `mutability` per entity to opt in: + +```yaml +repository: EntityFramework +getNamed: true # Root; default false — emits GetNamedAsync on ReferenceDataController. +entities: +- name: Brand + mutability: CreateUpdateDelete # None (default) | CreateUpdate | CreateUpdateDelete + validator: BrandValidator # Optional; default ReferenceDataValidator; needs a default constructor. + mutableAttribute: '[Authorize(Policy = "ReferenceDataAdmin")]' # Replaces the default [Authorize]. +- name: Category # Read-only. +``` + +| | `None` | `CreateUpdate` | `CreateUpdateDelete` | +|---|:-:|:-:|:-:| +| List via `ReferenceDataController` (cached orchestrator) | ✅ | ✅ | ✅ | +| `{Name}Controller`: GET `{id}`, POST (inactive), PUT, PATCH, POST `{id}/activate`, POST `{id}/deactivate` | — | ✅ | ✅ | +| DELETE `{id}` (204; active values rejected with 400 — deactivate first) | — | — | ✅ | +| `IReferenceDataService`/`ReferenceDataService` methods, repository write ops, bidirectional mapper | — | ✅ | ✅ (+ delete) | +| Events (`created/updated/activated/deactivated` `.v1`, `deleted`) + orchestrator cache invalidation | — | ✅ | ✅ | + +- Service and service interface are generated **only when at least one entity is mutable**. +- A mutable entity **requires `repository: EntityFramework` or `Cosmos`** — codegen throws otherwise (`None` is read-only only). `Cosmos` uses `CosmosDbReferenceData` (duplicate codes rely on a `/typeDiscriminator` + `/code` unique key on the container, which must also be registered with `CosmosDbContainerOptions.WithReferenceDataOutboxEvent()` so the co-located outbox events — which carry neither path — do not collide with each other) and the generated service re-gets the item after the transaction so the returned `ETag` is final. +- `code` is immutable after create; create always yields an inactive item; activate/deactivate are no-ops (no event) if already in that state. +- Mutable controllers default to `[Authorize]`. Set `mutableAttribute` to a project-specific authorization policy/role, or to `[AllowAnonymous]` only when the write API is intentionally public; this replaces the default attribute. Ensure the API project's `GlobalUsing.cs` imports `Microsoft.AspNetCore.Authorization`. +- `attribute` (root/entity) decorates only the read-only `ReferenceDataController`; use `mutableAttribute` for the write endpoints. +- The host needs `IUnitOfWork` registered; the generated service wraps each write and its outbox event in one transaction. +- **No usage/cascade check on delete or deactivate.** Other tables usually store the code (e.g. Product.BrandCode), so deleting or deactivating a value still in use leaves that data silently invalid. This is the consumer's responsibility — use the PreCheckAsync hook on the generated ReferenceDataService to veto the operation. +- PreCheckAsync is a private Func> set from a hand-written partial class ReferenceDataService via partial void OnInitialization(). It is a lightweight allow/deny veto only — no side effects, cascades or other writes; if more is needed, hand-write the data/repository logic instead of stretching the hook. It runs for Activated/Deactivated/Deleted only (not create/update), before the transaction (so not atomic, by design), and one delegate serves all mutable entities — switch on type/action. See the README for an example. +- If the table lacks `Description`/`StartsOn`/`EndsOn` columns (EF `Ignore`d), a non-null value for them is rejected on create/update with a 400 (`not-supported`) — add the columns if needed. + ## Do Not - Do not edit `*.g.cs` files — they are overwritten on every generation run. Edit `ref-data.yaml` or the Handlebars templates in the `CoreEx.CodeGen` package instead. +- Do not set `mutability` on an entity whose `repository` is not `EntityFramework` or `Cosmos`, and do not hand-write mutation endpoints/services that duplicate the generated ones — turn on `mutability` instead. +- Do not assume delete/deactivate protects referenced data — add the checks yourself in `PreCheckAsync` (never in the `.g.cs`). - Do not add `CoreEx.CodeGen` as a runtime dependency — it is a development tool only. ## Further Reading diff --git a/src/CoreEx.CodeGen/Counting/CodeGenCounter.cs b/src/CoreEx.CodeGen/Counting/CodeGenCounter.cs index db6a7d35e..cb2e2d375 100644 --- a/src/CoreEx.CodeGen/Counting/CodeGenCounter.cs +++ b/src/CoreEx.CodeGen/Counting/CodeGenCounter.cs @@ -6,18 +6,24 @@ internal class CodeGenCounter(CodeGeneratorArgs args) public Task CountAsync() { - if (args is null || args.OutputDirectory?.Parent is null || args.Logger is null) + if (args is null || args.OutputDirectory is null || args.Logger is null) throw new ArgumentNullException(nameof(args), "Arguments, and its OutputDirectory, and Logger cannot be null."); if (!args.Logger.IsEnabled(LogLevel.Information)) throw new ArgumentException("Logger must be enabled for Information level.", nameof(args)); - args.Logger.LogInformation("{Content}", $"Counting: {args.OutputDirectory.Parent.FullName}"); + if (!args.OutputDirectory.Exists) + throw new DirectoryNotFoundException($"The specified output directory '{args.OutputDirectory.FullName}' does not exist."); + + var countDir = args.OutputDirectory.Parent ?? args.OutputDirectory; + countDir = countDir.Parent ?? countDir; + + args.Logger.LogInformation("{Content}", $"Counting: {countDir.FullName}"); args.Logger.LogInformation("{Content}", $"Include: {string.Join(", ", _countExtensions)}"); args.Logger.LogInformation("{Content}", string.Empty); var sw = Stopwatch.StartNew(); - var dcs = new DirectoryCountStatistics(args.OutputDirectory.Parent); + var dcs = new DirectoryCountStatistics(countDir); CountDirectoryAndItsChildren(dcs); var columnLength = Math.Max(dcs.TotalLineCount.ToString().Length, 5); @@ -59,4 +65,4 @@ private static void CountDirectoryAndItsChildren(DirectoryCountStatistics dcs) dcs.IncrementFileCount(isGenerated); } } -} \ No newline at end of file +} diff --git a/src/CoreEx.CodeGen/README.md b/src/CoreEx.CodeGen/README.md index 17d6200e7..7de507eec 100644 --- a/src/CoreEx.CodeGen/README.md +++ b/src/CoreEx.CodeGen/README.md @@ -1,6 +1,6 @@ # CoreEx.CodeGen -> Provides the CoreEx development-time code-generation tooling: a deterministic, schema-driven pipeline that scaffolds the full reference-data implementation — contract, controller, service, repository interface, repository, and mapper — from a single YAML configuration file. +> Provides the CoreEx development-time code-generation tooling: a deterministic, schema-driven pipeline that scaffolds the full reference-data implementation — contract, controller, service, repository interface, repository, and mapper — from a single YAML configuration file — read-only by default, with opt-in per-entity mutation (create, update, activate/deactivate, delete). ## Overview @@ -19,11 +19,12 @@ The package exposes `CodeGenConsole`, a thin wrapper around `OnRamp.Console.Code ## Key capabilities -- 📄 **Schema-validated YAML input**: `ref-data.yaml` is validated against `schema/coreex-refdata.json`; the schema covers root settings (`domain`, `idType`, `collectionSortOrder`, `route`, `routeConvention`, `repository`) and per-entity / per-property overrides. -- 🔧 **Script-driven generation**: `ref-data-script.yaml` (embedded) defines the ordered generation steps — contract, controller, service, repository interface, repository, and mapper — each bound to a generator class and a Handlebars template. -- 📐 **Handlebars templates**: six `.hbs` templates (embedded in `RefData/Templates/`) produce idiomatic CoreEx C# across all target layers; output files carry a `.g.cs` suffix to clearly distinguish generated from hand-authored code. +- 📄 **Schema-validated YAML input**: `ref-data.yaml` is validated against `schema/coreex-refdata.json`; the schema covers root settings (`domain`, `idType`, `collectionSortOrder`, `route`, `routeConvention`, `repository`) and per-entity / per-property overrides, including `getNamed`, `attribute`, `mutability`, `validator` and `mutableAttribute`. +- 🔧 **Script-driven generation**: `ref-data-script.yaml` (embedded) defines the ordered generation steps — contract, controller, provider, repository interface, repository, and mapper, plus service interface, service and per-entity controller for mutable entities — each bound to a generator class and a Handlebars template. +- 📐 **Handlebars templates**: nine `.hbs` templates (embedded in `RefData/Templates/`) produce idiomatic CoreEx C# across all target layers; output files carry a `.g.cs` suffix to clearly distinguish generated from hand-authored code. - 🏗️ **Multi-layer output**: a single run generates artefacts across four project directories — Contracts, Api, Application, and Infrastructure — all resolved automatically from the CodeGen project's location by convention. - 🗺️ **EF Core mapper generation**: the `MapperGenerator` emits a typed mapper per entity, with per-property mapping directives derived from the `PropertyConfig`; entities can opt out via `excludeMapper: true`. +- ✏️ **Opt-in mutation**: per-entity `mutability` (`None` / `CreateUpdate` / `CreateUpdateDelete`) additionally generates a service, service interface, per-entity controller and repository write operations — see [Readonly vs Mutation](#readonly-vs-mutation). - 📊 **Code-count reporting**: the `Count` command walks the solution output directories and reports total vs. generated file and line counts per directory, helping track the proportion of the codebase that is generated. ## Usage @@ -79,6 +80,110 @@ Configuration details for each of the above are as follows: - [`Entity`](./docs/Entity.md) - [`Property`](./docs/Property.md) +## Readonly vs Mutation + +Reference data is **read-only by default**: entities are loaded through the `ReferenceDataOrchestrator` (cached) and exposed through the single `ReferenceDataController`. Mutation is **opt-in per entity** via `mutability`, and adds a transactional write path (service → repository → database, with events and cache invalidation) alongside the read path. Nothing about the read path changes for a mutable entity. + +### Options + +| Option | Level | Default | Description | +|---|---|---|---| +| `mutability` | Entity | `None` | `None` (read-only), `CreateUpdate` (create, update, activate, deactivate) or `CreateUpdateDelete` (adds delete). | +| `validator` | Entity | `ReferenceDataValidator<{Name}>` | Validator type used on create/update. Must have a default constructor. The default enforces `code` (mandatory, max 50), `text` (mandatory, max 250), `description` (max 1000) and `endsOn >= startsOn`. | +| `mutableAttribute` | Entity | `[Authorize]` | Replaces the default attribute on the generated `{Name}Controller`; use a policy/role (e.g. `'[Authorize(Roles = "Admin")]'`) or `[AllowAnonymous]` only when intentionally public. | +| `attribute` | Root / Entity | _none_ | Attribute applied to the read-only `ReferenceDataController` (root) or that entity's read operation (entity). Does **not** apply to mutable endpoints. | +| `getNamed` | Root | `false` | Emits the read-only `GetNamedAsync` endpoint. Applies to read-only and mutable entities alike. | +| `repository` | Root / Entity | root value | **Must be `EntityFramework` or `Cosmos` for a mutable entity** — generation fails fast otherwise. `None` is read-only only. `Cosmos` uses `CosmosDbReferenceData`; duplicate codes are rejected by a `/typeDiscriminator` + `/code` unique key on the container, which must also use `CosmosDbContainerOptions.WithReferenceDataOutboxEvent()` so co-located outbox events do not collide. | + +```yaml +collectionSortOrder: Code +repository: EntityFramework +entities: +- name: Brand + mutability: CreateUpdateDelete + mutableAttribute: '[Authorize(Roles = "Admin")]' # Replaces the default [Authorize]. +- name: Category # Read-only (default). +``` + +Mutable controllers require authorization by default. Set `mutableAttribute` to a project-specific policy or role as needed; the value replaces `[Authorize]`. For intentionally public sample/dev APIs, set it to `'[AllowAnonymous]'` (as in the Products and Customers sample `ref-data.yaml` files). The API project's `GlobalUsing.cs` must include `Microsoft.AspNetCore.Authorization` when using these attributes; the CoreEx API template includes it. + +### Outputs by mode + +| Output | Layer | Read-only | Mutable | +|---|---|:-:|:-:| +| `{Name}.g.cs` contract | Contracts | ✅ | ✅ | +| `Controllers/ReferenceDataController.g.cs` — list (and optional `GetNamed`) | Api | ✅ | ✅ | +| `Controllers/{Name}Controller.g.cs` — per-entity write endpoints | Api | — | ✅ | +| `ReferenceDataProvider.g.cs` — `IReferenceDataProvider` for the orchestrator | Application | ✅ | ✅ | +| `Interfaces/IReferenceDataService.g.cs` | Application | — | ✅ | +| `ReferenceDataService.g.cs` — validation, unit of work, events, cache invalidation | Application | — | ✅ | +| `Repositories/IReferenceDataRepository.g.cs` | Application | read ops | + write ops | +| `Repositories/ReferenceDataRepository.g.cs` | Infrastructure | read ops | + write ops via `EfDbReferenceData` | +| `Mapping/{Name}Mapper.g.cs` | Infrastructure | forward only (reverse throws `NotSupportedException`) | bidirectional | + +The service interface and service are generated **only if at least one entity is mutable**; a read-only solution produces neither. `ReferenceDataProvider` replaced the provider role formerly held by `ReferenceDataService` — a mutable solution's `ReferenceDataService` is now a different class entirely (the write service). + +### Endpoints + +Generated per mutable entity into `{Name}Controller`, routed at the root `route` (default `/api/refdata`) plus the entity `route` (default: pluralised name per `routeConvention`) — e.g. `/api/refdata/brands`. The `{id}` below is that entity's `idType` (`String`, `Guid`, `Int32` or `Int64`). + +| Verb & route | Mutability | Success | Notes | +|---|---|---|---| +| `GET {id}` | any mutable | 200 | Includes inactive items. | +| `POST` | any mutable | 201 + `Location` | `[IdempotencyKey]`; generated `id`, always created **inactive**. | +| `PUT {id}` | any mutable | 200 | Full replace; `If-Match` required (428 if absent); `id`, `code` and `isActive` in the body are ignored. | +| `PATCH {id}` | any mutable | 200 | `application/merge-patch+json`; ETag-checked. | +| `POST {id}/activate` | any mutable | 200 | No-op (no event, no cache invalidation) if already active. | +| `POST {id}/deactivate` | any mutable | 200 | No-op if already inactive. | +| `DELETE {id}` | `CreateUpdateDelete` | 204 | Idempotent (missing → 204). Deleting an **active** value is a 400 (`cannot-delete-active`) — deactivate first. | + +The list endpoint continues to exclude inactive items unless `$inactive=true` is supplied. + +### Runtime behaviour + +- **`code` is immutable** — update (PUT or PATCH) keeps the existing `id`, `code` and active state; use activate/deactivate to change the latter. +- **Transactional** — each operation runs inside `IUnitOfWork.TransactionAsync`; the change and its outbox event commit atomically. +- **Events** — added to the unit of work's outbox only when the operation actually changed something: `{domain}.{entity}.created.v1`, `.updated.v1`, `.activated.v1`, `.deactivated.v1` (carrying the value) and `.deleted` (key only). +- **Cache** — after a successful mutation the service calls `ReferenceDataOrchestrator.Current.TryInvalidateAsync()` (post-commit), so the next read reloads the collection (and, with a distributed cache, the shared entry is removed too). The try variant is awaited but best-effort: a cache failure is logged as a warning and does not fail the already-committed write; the stale entry then expires per its TTL. Use the strict `InvalidateAsync()` only where an eviction failure must surface, and never invoke either inside the transaction. +- **Validation** — failures surface as a standard 400 validation `ProblemDetails`. + +### Referential integrity and pre-checks + +**CodeGen does not check whether a reference-data value is in use.** Delete, deactivate (and the `code` that other tables store) carry no cascade or usage check, and consumers typically hold the *code* (e.g. `Product.BrandCode`), usually without a database foreign key. Deleting a value, or deactivating one that is still referenced, therefore leaves existing data pointing at a missing or inactive value — it does not fail, and the data is now quietly invalid. Whether that is acceptable, and what to do about it, is **the consumer's responsibility**. The only built-in guard is that an *active* value cannot be deleted (`cannot-delete-active`), which merely forces deactivate-then-delete; it does nothing about usage. + +Where a rule is needed, the generated `ReferenceDataService` provides a `PreCheckAsync` hook. **Its intent is deliberately narrow: a lightweight veto (allow/deny) before the mutation — nothing more.** It is not a place for side effects, cascades or other data changes. If the operation needs more than a veto (e.g. reassigning or deactivating dependents atomically with the change, locking reads, or any extra writes), do not stretch the hook: implement that data/repository logic yourself (a hand-written service or repository method inside your own unit of work) rather than using the generated mutation for that entity. The hook is a `Func>` (default is a no-op success) exposed as a **private** property, so set it from the hand-written side of the `partial` class via the `OnInitialization()` partial method: + +```csharp +public partial class ReferenceDataService +{ + partial void OnInitialization() => PreCheckAsync = (value, action, cancellationToken) => + { + if (value is Brand b && b.Code == "YETI" && action == EventAction.Deactivated) + return Result.BusinessError("YETI brand cannot be deactivated as it is awesome.", c => c.WithErrorCode("yeti-cannot-be-deactivated")).AsTask(); + + return Result.SuccessTask; + }; +} +``` + +(This is the Products sample, `ReferenceDataService.cs`.) For a real in-use check, add a method to `IReferenceDataRepository` and the matching `partial` repository class (both are `partial`) that queries the referencing table by code, and call it from the hook when `action` is `Deactivated` or `Deleted`; return a `Result` failure (e.g. `Result.BusinessError(...)`) to veto the operation. + +How the hook behaves: + +| | | +|---|---| +| **Invoked for** | `Activated`, `Deactivated`, `Deleted` only — **not** create or update. | +| **Single hook** | One delegate for all mutable entities; switch on the value's type (and `action`) to target specific entities. | +| **Ordering** | Runs after the value is loaded and **before** the transaction opens. Deactivate/activate run it even when the value is already in that state. Delete runs it only if the value exists (a missing value is an idempotent 204), and before the `cannot-delete-active` check. | +| **Not atomic (by design)** | The check and the write are not in the same transaction, so a concurrent insert can still slip in between. Back it with a database constraint if you need a hard guarantee; if you need the check and the change to be atomic, hand-write that logic instead. | +| **Failure** | The returned failure surfaces as a normal error response (a `BusinessError` becomes a 400) and no event or cache invalidation occurs. | +### Prerequisites and gotchas + +- The host must register `IUnitOfWork` (and the outbox, where events are required) — the generated service takes `IUnitOfWork` and `IReferenceDataRepository`. +- The generated reverse mapper writes `Code`, `Text`, `Description`, `SortOrder`, `IsActive`, `StartsOn`, `EndsOn`, `ETag` and any extra properties. If the table has no `Description`/`StartsOn`/`EndsOn` columns (the DbEx generated `DbContext` then `Ignore`s them, as for the sample `Brand`), create/update **rejects** any non-null value for them with a 400 (`not-supported`, e.g. "Starts on is not currently supported and as such cannot be set.") — add the columns if you need them. +- Authorization is required by default on mutable controllers via `[Authorize]`. Per-entity policies/roles are not inferred; configure them through `mutableAttribute` or a hand-written partial. Do not override with `[AllowAnonymous]` unless anonymous writes are an explicit requirement. +- Never edit the `.g.cs` outputs — change `ref-data.yaml` or the templates and regenerate. + ## Key types | Type | Description | @@ -88,7 +193,9 @@ Configuration details for each of the above are as follows: | **[`CodeGenConfig`](./RefData/Config/CodeGenConfig.cs)** | Root configuration model for reference-data generation; maps directly to the top-level YAML and resolves all project directory paths, namespace defaults, and entity collection preparation. | | **[`EntityConfig`](./RefData/Config/EntityConfig.cs)** | Per-entity configuration: name, plural, text, `idType`, `collectionSortOrder`, route, repository mode, model name, mapper name, `excludeMapper`, and the property collection. | | **[`PropertyConfig`](./RefData/Config/PropertyConfig.cs)** | Per-property configuration: name, type (with `^`-prefix for reference-data and `?`-suffix for nullable), text, data model property name, and exclude flags for contract and mapping generation. | -| **[`RootGenerator`](./RefData/Generators/RootGenerator.cs)** | `CodeGeneratorBase` implementation that selects `CodeGenConfig` as its single generation target; used for the service, repository interface, and repository templates. | +| **[`RootGenerator`](./RefData/Generators/RootGenerator.cs)** | `CodeGeneratorBase` implementation that selects `CodeGenConfig` as its single generation target; used for the provider, repository interface, and repository templates. | +| **[`RootMutableGenerator`](./RefData/Generators/RootMutableGenerator.cs)** | Selects `CodeGenConfig` only when at least one entity is mutable; used for the service interface and service templates (skipped entirely for read-only solutions). | +| **[`MutableApiGenerator`](./RefData/Generators/MutableApiGenerator.cs)** | Selects each mutable `EntityConfig` (only when the Api project directory exists); produces one `{Name}Controller.g.cs` per mutable entity. | | **[`ApiGenerator`](./RefData/Generators/ApiGenerator.cs)** | `CodeGeneratorBase` implementation that selects `CodeGenConfig` only when the Api project directory exists; used for the controller template (API generation is skipped entirely when the Api project isn't found). | | **[`ContractGenerator`](./RefData/Generators/ContractGenerator.cs)** | `CodeGeneratorBase` implementation that iterates all `EntityConfig` entries to produce one contract file per entity. | | **[`MapperGenerator`](./RefData/Generators/MapperGenerator.cs)** | `CodeGeneratorBase` implementation that iterates `EntityConfig` entries where `excludeMapper` is not `true`, producing one mapper file per entity. | @@ -109,4 +216,4 @@ Configuration details for each of the above are as follows: ## AI Usage Guide -An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. \ No newline at end of file +An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. diff --git a/src/CoreEx.CodeGen/RefData/Config/CodeGenConfig.cs b/src/CoreEx.CodeGen/RefData/Config/CodeGenConfig.cs index 7ca7a83b8..4ca14c76a 100644 --- a/src/CoreEx.CodeGen/RefData/Config/CodeGenConfig.cs +++ b/src/CoreEx.CodeGen/RefData/Config/CodeGenConfig.cs @@ -54,6 +54,21 @@ public class CodeGenConfig : ConfigRootBase [JsonPropertyName("routeConvention")] [CodeGenProperty("API", Title = "The route naming convention where not directly specified.", IsImportant = true, Options = ["KebabCase", "SnakeCase", "CamelCase", "Lowercase"], Description = "Defaults to `KebabCase`.")] public string? RouteConvention { get; set; } + + /// + /// Gets or sets the optional attribute. + /// + [JsonPropertyName("attribute")] + [CodeGenProperty("API", Title = "The optional API controller class attribute.", Description = "This is the attribute applied as-is to the generated `ReferenceDataController` class. This is useful for adding the likes of `[Authorize]`.")] + public string? Attribute { get; set; } + + + /// + /// Gets or sets a value indicating whether the API controller emits the `GetNamed` operation. + /// + [JsonPropertyName("getNamed")] + [CodeGenProperty("API", Title = "Indicates whether the API controller emits the `GetNamed` operation.", Description = "Defaults to `false`.")] + public bool? GetNamed { get; set; } #endregion @@ -166,6 +181,16 @@ public class CodeGenConfig : ConfigRootBase /// public List? CosmosPersistenceModels => Entities?.Where(x => x.CosmosPersistenceModel ?? false).ToList(); + /// + /// Gets the list of configured entities that are mutable (i.e. not read-only). + /// + public List? EntitiesThatAreMutable => Entities?.Where(x => x.IsMutable).ToList(); + + /// + /// Gets the list of configured entities that are mutable (i.e. not read-only) and have a repository implementation specified (i.e. not 'None'). + /// + public List? EntitiesThatAreMutableWithRepository => Entities?.Where(x => x.IsMutable && x.Repository != "None").ToList(); + #endregion /// @@ -223,6 +248,11 @@ public class CodeGenConfig : ConfigRootBase /// public string? DataModelsNamespace { get; set; } + /// + /// Indicates whether there are any configured entities that are mutable (i.e. not read-only). + /// + public bool HasMutableEntities => EntitiesThatAreMutable?.Count > 0; + /// protected override async Task PrepareAsync() { @@ -299,6 +329,7 @@ protected override async Task PrepareAsync() CollectionSortOrder = DefaultWhereNull(CollectionSortOrder, () => "Code"); Route = DefaultWhereNull(Route, () => "/api/refdata"); RouteConvention = DefaultWhereNull(RouteConvention, () => "KebabCase"); + GetNamed = DefaultWhereNull(GetNamed, () => false); // Load the entities configuration. Entities = await PrepareCollectionAsync(Entities).ConfigureAwait(false); diff --git a/src/CoreEx.CodeGen/RefData/Config/EntityConfig.cs b/src/CoreEx.CodeGen/RefData/Config/EntityConfig.cs index a178175f9..da747ce90 100644 --- a/src/CoreEx.CodeGen/RefData/Config/EntityConfig.cs +++ b/src/CoreEx.CodeGen/RefData/Config/EntityConfig.cs @@ -8,6 +8,7 @@ namespace CoreEx.CodeGen.RefData.Config; [CodeGenCategory("API", Title = "Provides the configuration for the generated API code.")] [CodeGenCategory("Repository", Title = "Provides the configuration for the generated repository code.")] [CodeGenCategory("Mapping", Title = "Provides the configuration for the generated mapping code.")] +[CodeGenCategory("Mutable", Title = "Provides the configuration for the generated mutability code.")] [CodeGenCategory("Exclude", Title = "Provides the configuration for code generation exclusion.")] [CodeGenCategory("Collections", Title = "Provides the collections configuration.")] public class EntityConfig : ConfigBase @@ -37,14 +38,14 @@ public class EntityConfig : ConfigBase /// Gets or sets the identifier type. /// [JsonPropertyName("idType")] - [CodeGenProperty("Primary", Title = "The reference-data identifier type.", IsImportant = true, Options = ["String", "Guid", "Int32", "Int64"], Description = "Defaults to root `{IdType}`.")] + [CodeGenProperty("Primary", Title = "The reference-data identifier type.", Options = ["String", "Guid", "Int32", "Int64"], Description = "Defaults to root `{IdType}`.")] public string? IdType { get; set; } /// /// Gets or sets the default collection sort order. /// [JsonPropertyName("collectionSortOrder")] - [CodeGenProperty("Primary", Title = "The collection sort order.", IsImportant = true, Options = ["Code", "Id", "Text", "SortOrder"], Description = "This is the collection sort order. Defaults to root `{CollectionSortOrder}`.")] + [CodeGenProperty("Primary", Title = "The collection sort order.", Options = ["Code", "Id", "Text", "SortOrder"], Description = "This is the collection sort order. Defaults to root `{CollectionSortOrder}`.")] public string? CollectionSortOrder { get; set; } #region API @@ -56,6 +57,13 @@ public class EntityConfig : ConfigBase [CodeGenProperty("API", Title = "The route suffix.", IsImportant = true, Description = "Defaults to `{Plural}` and root `{RouteConvention}` configuration.")] public string? Route { get; set; } + /// + /// Gets or sets the optional attribute. + /// + [JsonPropertyName("attribute")] + [CodeGenProperty("API", Title = "The optional API controller operation attribute.", Description = "This is the attribute applied as-is to the generated `ReferenceDataController` operation. This is useful for adding the likes of `[Authorize]`.")] + public string? Attribute { get; set; } + #endregion #region Repository @@ -108,6 +116,31 @@ public class EntityConfig : ConfigBase #endregion + #region Mutable + + /// + /// Gets or sets the reference-data entity mutability. + /// + [JsonPropertyName("mutability")] + [CodeGenProperty("Mutable", Title = "The reference-data entity mutability.", Options = ["None", "CreateUpdate", "CreateUpdateDelete"], Description = "Defaults to `None`. Requires a `Repository` of `EntityFramework` or `Cosmos`.")] + public string? Mutability { get; set; } + + /// + /// Gets or sets the validator type name. + /// + [JsonPropertyName("validator")] + [CodeGenProperty("Mutable", Title = "The validator type name used during mutability operations.", Description = "Defaults to `ReferenceDataValidator<{Name}>`. Must have a default constructor.")] + public string? Validator { get; set; } + + /// + /// Gets or sets the optional attribute. + /// + [JsonPropertyName("mutableAttribute")] + [CodeGenProperty("Mutable", Title = "The mutable API controller class attribute.", Description = "Defaults to `[Authorize]`. A configured value replaces the default attribute applied as-is to the generated `{Name}Controller` class.")] + public string? MutableAttribute { get; set; } + + #endregion + #region Exclude /// @@ -147,6 +180,11 @@ public class EntityConfig : ConfigBase #endregion + /// + /// Gets or sets the .NET type of the identifier. + /// + public string? IdDotNetType { get; set; } + /// /// Gets or sets the contract inherits base class name. /// @@ -169,6 +207,16 @@ public class EntityConfig : ConfigBase /// persistence model's Id column type is expected to already agree with ) is a direct assignment. public string? MapperIdExpression { get; set; } + /// + /// Indicates whether the entity is mutable; see . + /// + public bool IsMutable => Mutability switch + { + "CreateUpdate" => true, + "CreateUpdateDelete" => true, + _ => false + }; + /// protected override async Task PrepareAsync() { @@ -177,9 +225,14 @@ protected override async Task PrepareAsync() IdType = DefaultWhereNull(IdType, () => Root?.IdType); CollectionSortOrder = DefaultWhereNull(CollectionSortOrder, () => Root!.CollectionSortOrder); Repository = DefaultWhereNull(Repository, () => Root!.Repository); - + Mutability = DefaultWhereNull(Mutability, () => "None"); Mapper = DefaultWhereNull(Mapper, () => $"{Name}Mapper"); ExcludeMapper = DefaultWhereNull(ExcludeMapper, () => false); + Validator = DefaultWhereNull(Validator, () => $"ReferenceDataValidator<{Name}>"); + MutableAttribute = DefaultWhereNull(MutableAttribute, () => "[Authorize]"); + + if (IsMutable && Repository is not ("EntityFramework" or "Cosmos")) + throw new CodeGenException(this, nameof(Mutability), $"Mutability '{Mutability}' requires a '{nameof(Repository)}' of 'EntityFramework' or 'Cosmos'; '{Repository}' is not supported."); Plural = DefaultWhereNull(Plural, () => { @@ -216,6 +269,14 @@ protected override async Task PrepareAsync() _ => Plural!.ToLower(), }); + IdDotNetType = IdType switch + { + "Int32" => "int", + "Int64" => "long", + "Guid" => "Guid", + _ => "string" + }; + Inherits = IdType switch { "Int32" => $"ReferenceData", diff --git a/src/CoreEx.CodeGen/RefData/Generators/MutableApiGenerator.cs b/src/CoreEx.CodeGen/RefData/Generators/MutableApiGenerator.cs new file mode 100644 index 000000000..1590b9660 --- /dev/null +++ b/src/CoreEx.CodeGen/RefData/Generators/MutableApiGenerator.cs @@ -0,0 +1,10 @@ +namespace CoreEx.CodeGen.RefData.Generators; + +/// +/// Provides the API mutable code-generator. +/// +public class MutableApiGenerator : CodeGeneratorBase +{ + /// + protected override IEnumerable SelectGenConfig(CodeGenConfig config) => (config.ApiDirectory?.Exists ?? false) ? config.EntitiesThatAreMutable?.Where(x => !(x.ExcludeApi ?? false)) ?? [] : []; +} diff --git a/src/CoreEx.CodeGen/RefData/Generators/RootMutableGenerator.cs b/src/CoreEx.CodeGen/RefData/Generators/RootMutableGenerator.cs new file mode 100644 index 000000000..f79291d41 --- /dev/null +++ b/src/CoreEx.CodeGen/RefData/Generators/RootMutableGenerator.cs @@ -0,0 +1,10 @@ +namespace CoreEx.CodeGen.RefData.Generators; + +/// +/// Provides the mutability code-generator. +/// +public class RootMutableGenerator : CodeGeneratorBase +{ + /// + protected override IEnumerable SelectGenConfig(CodeGenConfig config) => config.HasMutableEntities ? [config] : []; +} diff --git a/src/CoreEx.CodeGen/RefData/README.md b/src/CoreEx.CodeGen/RefData/README.md index 249a5aa3b..296fc85b9 100644 --- a/src/CoreEx.CodeGen/RefData/README.md +++ b/src/CoreEx.CodeGen/RefData/README.md @@ -4,15 +4,16 @@ ## Overview -`CoreEx.CodeGen.RefData` contains all the moving parts of the reference-data generation workflow. The `Config` sub-namespace holds the three typed configuration models (`CodeGenConfig`, `EntityConfig`, `PropertyConfig`) that OnRamp deserialises from `ref-data.yaml` and enriches with defaults before passing to the generators. The `Generators` sub-namespace holds the four `CodeGeneratorBase` implementations that select the appropriate configuration nodes for each template pass. The `Templates` folder contains the embedded Handlebars (`.hbs`) templates, one per generated artefact type. +`CoreEx.CodeGen.RefData` contains all the moving parts of the reference-data generation workflow. The `Config` sub-namespace holds the three typed configuration models (`CodeGenConfig`, `EntityConfig`, `PropertyConfig`) that OnRamp deserialises from `ref-data.yaml` and enriches with defaults before passing to the generators. The `Generators` sub-namespace holds the six `CodeGeneratorBase` implementations that select the appropriate configuration nodes for each template pass. The `Templates` folder contains the embedded Handlebars (`.hbs`) templates, one per generated artefact type. -OnRamp drives the overall orchestration: `ref-data-script.yaml` (also embedded in the package) lists the generation steps in order — contract, controller, service, repository interface, repository, mapper — binding each step to its generator class, template, output file name pattern, and output directory. The `RootGenerator` runs once against the whole `CodeGenConfig` for the single-file artefacts (service, repository interface, repository); `ApiGenerator` similarly runs once for the controller template, but only when the Api project directory exists. `ContractGenerator` and `MapperGenerator` each iterate over the entity collection to produce one file per entity. +OnRamp drives the overall orchestration: `ref-data-script.yaml` (also embedded in the package) lists the generation steps in order — contract, controller, provider, repository interface, repository, mapper, then (mutable entities only) service interface, service and per-entity controller — binding each step to its generator class, template, output file name pattern, and output directory. The `RootGenerator` runs once against the whole `CodeGenConfig` for the single-file artefacts (provider, repository interface, repository); `ApiGenerator` similarly runs once for the controller template, but only when the Api project directory exists. `ContractGenerator` and `MapperGenerator` each iterate over the entity collection to produce one file per entity. ## Key capabilities - 🗂️ **Typed configuration hierarchy**: `CodeGenConfig` → `EntityConfig` → `PropertyConfig` mirrors the YAML structure exactly; all defaults are resolved during the `PrepareAsync` phase before any template is evaluated. -- ⚙️ **Four-generator pattern**: `RootGenerator` (single-file artefacts), `ApiGenerator` (single-file controller, conditional on the Api project existing), `ContractGenerator` (per-entity contracts), and `MapperGenerator` (per-entity mappers, with opt-out support) cleanly separate the generation concerns. -- 📐 **Embedded templates**: `Contract_cs.hbs`, `Controller_cs.hbs`, `Service_cs.hbs`, `IRepository_cs.hbs`, `Repository_cs.hbs`, `Mapper_cs.hbs` — all embedded in the package so consuming projects carry no template files. +- ⚙️ **Six-generator pattern**: `RootGenerator` (single-file artefacts), `ApiGenerator` (single-file controller, conditional on the Api project existing), `ContractGenerator` (per-entity contracts), `MapperGenerator` (per-entity mappers, with opt-out support), `RootMutableGenerator` (service interface and service, only when any entity is mutable) and `MutableApiGenerator` (per-mutable-entity controller) cleanly separate the generation concerns. +- ✏️ **Readonly vs mutation**: `mutability` is opt-in per entity (`None` / `CreateUpdate` / `CreateUpdateDelete`); see [Readonly vs Mutation](../README.md#readonly-vs-mutation). +- 📐 **Embedded templates**: `Contract_cs.hbs`, `Controller_cs.hbs`, `MutableController_cs.hbs`, `Provider_cs.hbs`, `IService_cs.hbs`, `Service_cs.hbs`, `IRepository_cs.hbs`, `Repository_cs.hbs`, `Mapper_cs.hbs` — all embedded in the package so consuming projects carry no template files. - 🔀 **Property-level control**: `PropertyConfig` supports `^`-prefixed reference-data types, `?`-suffixed nullable types, per-property model name overrides, and per-property exclusion from contract or mapping generation. ## Key types @@ -20,9 +21,11 @@ OnRamp drives the overall orchestration: `ref-data-script.yaml` (also embedded i | Type | Description | |------|-------------| | **[`CodeGenConfig`](./Config/CodeGenConfig.cs)** | Root configuration model; deserialised from `ref-data.yaml` and enriched with defaults for domain name, project paths, namespace strings, `idType`, `collectionSortOrder`, `route`, `routeConvention`, and the entity collection. | -| **[`EntityConfig`](./Config/EntityConfig.cs)** | Per-entity configuration: `name`, `plural`, `text`, `idType`, `collectionSortOrder`, `route`, repository mode and parameter name, EF model name, mapper name, `excludeMapper`, and the `properties` collection. | +| **[`EntityConfig`](./Config/EntityConfig.cs)** | Per-entity configuration: `name`, `plural`, `text`, `idType`, `collectionSortOrder`, `route`, repository mode and parameter name, EF model name, mapper name, `excludeMapper`, `mutability`, `validator`, `mutableAttribute`, and the `properties` collection. | | **[`PropertyConfig`](./Config/PropertyConfig.cs)** | Per-property configuration: `name`, `type` (with `^`/`?` conventions), `text`, data model property name, `excludeContract`, and `excludeMapping`. | -| **[`RootGenerator`](./Generators/RootGenerator.cs)** | Selects `CodeGenConfig` itself as the generation target; used by the service, repository interface, and repository templates to produce one file for the entire entity set. | +| **[`RootMutableGenerator`](./Generators/RootMutableGenerator.cs)** | Selects `CodeGenConfig` only when at least one entity is mutable; used by the service interface and service templates. | +| **[`MutableApiGenerator`](./Generators/MutableApiGenerator.cs)** | Selects each mutable `EntityConfig` (when the Api project exists); produces one `{Name}Controller.g.cs` per mutable entity. | +| **[`RootGenerator`](./Generators/RootGenerator.cs)** | Selects `CodeGenConfig` itself as the generation target; used by the provider, repository interface, and repository templates to produce one file for the entire entity set. | | **[`ApiGenerator`](./Generators/ApiGenerator.cs)** | Selects `CodeGenConfig` only when the Api project directory exists; used by the controller template — API generation is skipped entirely (with a warning) when the Api project isn't found relative to the CodeGen project. | | **[`ContractGenerator`](./Generators/ContractGenerator.cs)** | Selects each `EntityConfig` in turn; produces one contract `.g.cs` file per reference-data entity. | | **[`MapperGenerator`](./Generators/MapperGenerator.cs)** | Selects each `EntityConfig` where `excludeMapper` is not `true`; produces one mapper `.g.cs` file per entity. | @@ -30,4 +33,4 @@ OnRamp drives the overall orchestration: `ref-data-script.yaml` (also embedded i ## Related namespaces - **[`CoreEx.CodeGen`](../README.md)** - Root package: `CodeGenConsole` entry point, `CommandType` enum, and the embedded `ref-data-script.yaml` that drives this pipeline. -- **[`CoreEx.CodeGen.Counting`](../Counting/README.md)** - Companion utility for reporting generated vs. hand-authored file and line counts. \ No newline at end of file +- **[`CoreEx.CodeGen.Counting`](../Counting/README.md)** - Companion utility for reporting generated vs. hand-authored file and line counts. diff --git a/src/CoreEx.CodeGen/RefData/Templates/Controller_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/Controller_cs.hbs index d5c79320c..2e60753aa 100644 --- a/src/CoreEx.CodeGen/RefData/Templates/Controller_cs.hbs +++ b/src/CoreEx.CodeGen/RefData/Templates/Controller_cs.hbs @@ -1,31 +1,39 @@ // /* - * This file is automatically generated by '{{Root.CodeGenName}}'; any changes will be lost. + * This file is automatically generated by '{{CodeGenName}}'; any changes will be lost. */ #nullable enable -namespace {{Root.ApiNamespace}}.Controllers; +namespace {{ApiNamespace}}.Controllers; /// Represents the reference-data controller. [ApiController, Route("{{Route}}")] +{{#ifval Attribute}} +{{Attribute}} +{{/ifval}} public partial class ReferenceDataController(CoreEx.AspNetCore.Mvc.WebApi webApi) : ControllerBase { private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); {{#each EntitiesWithApi}} - [HttpGet("{{Route}}"), HttpHead("{{Route}}")] + [HttpGet("{{Route}}")] + {{#ifval Attribute}} + {{Attribute}} + {{/ifval}} [ProducesResponseType(typeof({{Name}}[]), 200)] [Query(supportsOrderBy: true), Paging(supportsCount: true)] public Task Get{{Plural}}Async(CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.QueryAsync<{{Name}}>(ro.QueryArgs, ro.PagingArgs, ct), cancellationToken: cancellationToken); {{/each}} +{{#if GetNamed}} [HttpGet] [ProducesResponseType(typeof(ReferenceDataMultiDictionary), 200)] public Task GetNamedAsync([FromQuery] string[] name, CancellationToken cancellationToken) => _webApi.GetAsync(Request, (ro, ct) => ReferenceDataOrchestrator.Current.GetNamedAsync(name, ro.IsIncludeInactive, ct), cancellationToken: cancellationToken); +{{/if}} } #nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/IRepository_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/IRepository_cs.hbs index fd1834b7b..a662a97dd 100644 --- a/src/CoreEx.CodeGen/RefData/Templates/IRepository_cs.hbs +++ b/src/CoreEx.CodeGen/RefData/Templates/IRepository_cs.hbs @@ -16,9 +16,34 @@ public partial interface IReferenceDataRepository {{/unless}} /// Gets all items. - /// The . Task<{{Name}}Collection> GetAll{{Plural}}Async(CancellationToken cancellationToken = default); {{/each}} +{{#each EntitiesThatAreMutable}} + + #region {{Name}} + + /// Gets the specified . + Task> Get{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default); + + /// Creates the . + Task>> Create{{Name}}Async(Contracts.{{Name}} value, CancellationToken cancellationToken = default); + + /// Updates the . + Task>> Update{{Name}}Async({{IdDotNetType}} id, Contracts.{{Name}} value, CancellationToken cancellationToken = default); + + /// Activates the . + Task>> Activate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default); + + /// Deactivates the . + Task>> Deactivate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default); + {{#ifeq Mutability 'CreateUpdateDelete'}} + + /// Deletes the specified . + Task> Delete{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default); + {{/ifeq}} + + #endregion +{{/each}} } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/IService_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/IService_cs.hbs new file mode 100644 index 000000000..b372448e6 --- /dev/null +++ b/src/CoreEx.CodeGen/RefData/Templates/IService_cs.hbs @@ -0,0 +1,44 @@ +// + +/* + * This file is automatically generated by '{{Root.CodeGenName}}'; any changes will be lost. + */ + +#nullable enable + +namespace {{Root.ApplicationNamespace}}; + +/// Enables the mutable services. +public partial interface IReferenceDataService +{ +{{#each EntitiesThatAreMutable}} + {{#unless @first}} + + {{/unless}} + #region {{Name}} + + /// Gets the specified . + Task> Get{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken); + + /// Creates the specified . + Task> Create{{Name}}Async({{Name}} value, CancellationToken cancellationToken); + + /// Updates the specified . + Task> Update{{Name}}Async({{IdDotNetType}} id, {{Name}} value, CancellationToken cancellationToken); + + /// Activates the specified . + Task> Activate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken); + + /// Deactivates the specified . + Task> Deactivate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken); + {{#ifeq Mutability 'CreateUpdateDelete'}} + + /// Deletes the specified . + Task Delete{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken); + {{/ifeq}} + + #endregion +{{/each}} +} + +#nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/Mapper_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/Mapper_cs.hbs index 45bec937b..13190b5b5 100644 --- a/src/CoreEx.CodeGen/RefData/Templates/Mapper_cs.hbs +++ b/src/CoreEx.CodeGen/RefData/Templates/Mapper_cs.hbs @@ -8,8 +8,8 @@ namespace {{Root.DataMappingNamespace}}; -/// Provides mapping from to . -internal partial class {{Mapper}} : Mapper<{{Root.DataModelsPath}}.{{Model}}, Contracts.{{Name}}, {{Mapper}}> +/// Provides mapping from to . +internal partial class {{Mapper}} : BiDirectionMapper { /// protected override Contracts.{{Name}} OnMap({{Root.DataModelsPath}}.{{Model}} source) @@ -34,8 +34,37 @@ internal partial class {{Mapper}} : Mapper<{{Root.DataModelsPath}}.{{Model}}, Co return destination; } - /// Provides the opportunity to extend the method. + /// Provides the opportunity to extend the To mapping. partial void OnMapExtend({{Root.DataModelsPath}}.{{Model}} source, Contracts.{{Name}} destination); + + /// + {{#unless IsMutable}} + protected override {{Root.DataModelsPath}}.{{Model}} OnMap(Contracts.{{Name}} destination) => throw new NotSupportedException(); + {{else}} + protected override {{Root.DataModelsPath}}.{{Model}} OnMap(Contracts.{{Name}} destination) + { + var source = new {{Root.DataModelsPath}}.{{Model}} + { + Code = destination.Code!, + Text = destination.Text, + Description = destination.Description, + SortOrder = destination.SortOrder, + IsActive = !destination.IsInactive, + StartsOn = destination.StartsOn, + EndsOn = destination.EndsOn, + ETag = destination.ETag{{#ifne MappingProperties.Count 0}},{{/ifne}} + {{#each MappingProperties}} + {{Model}} = destination.{{Name}}{{#unless @last}},{{/unless}} + {{/each}} + }; + + OnMapExtend(destination, source); + return source; + } + + /// Provides the opportunity to extend the From mapping. + partial void OnMapExtend(Contracts.{{Name}} destination, {{Root.DataModelsPath}}.{{Model}} source); + {{/unless}} } -#nullable restore \ No newline at end of file +#nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/MutableController_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/MutableController_cs.hbs new file mode 100644 index 000000000..13e43148b --- /dev/null +++ b/src/CoreEx.CodeGen/RefData/Templates/MutableController_cs.hbs @@ -0,0 +1,75 @@ + + +// + +/* + * This file is automatically generated by '{{Root.CodeGenName}}'; any changes will be lost. + */ + +#nullable enable + +namespace {{Root.ApiNamespace}}.Controllers; + +/// Represents the reference-data controller. +[ApiController, Route("{{Root.Route}}/{{Route}}")] +{{#ifval MutableAttribute}} +{{MutableAttribute}} +{{/ifval}} +public partial class {{Name}}Controller(CoreEx.AspNetCore.Mvc.WebApi webApi, IReferenceDataService service) : ControllerBase +{ + private readonly CoreEx.AspNetCore.Mvc.WebApi _webApi = webApi.ThrowIfNull(); + private readonly IReferenceDataService _service = service.ThrowIfNull(); + + [HttpGet("{id}"), HttpHead("{id}")] + [ProducesResponseType(typeof({{Name}}), 200)] + [ProducesNotFoundProblem()] + public Task GetAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => _webApi.GetWithResultAsync(Request, (_, ct) => _service.Get{{Name}}Async(id.Required(), ct), cancellationToken: cancellationToken); + + [HttpPost] + [Accepts<{{Name}}>] + [ProducesResponseType<{{Name}}>(201)] + [IdempotencyKey] + public Task CreateAsync(CancellationToken cancellationToken = default) => _webApi.PostWithResultAsync<{{Name}}, {{Name}}>(Request, (ro, ct) => + { + ro.WithLocationUri(v => new Uri($"{{Root.Route}}/{{Route}}/{v.Id}", UriKind.Relative)); + return _service.Create{{Name}}Async(ro.Value, ct); + }, cancellationToken: cancellationToken); + + [HttpPatch("{id}")] + [Accepts<{{Name}}>(HttpNames.MergePatchJsonMediaTypeName)] + [ProducesResponseType(typeof({{Name}}), 200)] + [ProducesNotFoundProblem()] + public Task PatchAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) => _webApi.PatchWithResultAsync<{{Name}}>(Request, + get: (ro, ct) => _service.Get{{Name}}Async(id.Required(), ct), + put: (ro, ct) => _service.Update{{Name}}Async(id, ro.Value, ct), + cancellationToken: cancellationToken); + + [HttpPut("{id}")] + [Accepts<{{Name}}>] + [ProducesResponseType(typeof({{Name}}), 200)] + [ProducesNotFoundProblem()] + public Task PutAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => _webApi.PutWithResultAsync<{{Name}}, {{Name}}>(Request, (ro, ct) => _service.Update{{Name}}Async(id.Required(), ro.Value, ct), cancellationToken: cancellationToken); + + [HttpPost("{id}/activate")] + [ProducesResponseType(typeof({{Name}}), 200)] + [ProducesNotFoundProblem()] + public Task ActivateAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync<{{Name}}>(Request, (_, ct) => _service.Activate{{Name}}Async(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); + + [HttpPost("{id}/deactivate")] + [ProducesResponseType(typeof({{Name}}), 200)] + [ProducesNotFoundProblem()] + public Task DeactivateAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => _webApi.PostWithResultAsync<{{Name}}>(Request, (_, ct) => _service.Deactivate{{Name}}Async(id.Required(), ct), System.Net.HttpStatusCode.OK, cancellationToken: cancellationToken); +{{#ifeq Mutability 'CreateUpdateDelete'}} + + [HttpDelete("{id}")] + [ProducesResponseType(204)] + public Task DeleteAsync({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => _webApi.DeleteWithResultAsync(Request, (_, ct) => _service.Delete{{Name}}Async(id.Required(), ct), cancellationToken: cancellationToken); +{{/ifeq}} +} + +#nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/Provider_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/Provider_cs.hbs new file mode 100644 index 000000000..ad869183f --- /dev/null +++ b/src/CoreEx.CodeGen/RefData/Templates/Provider_cs.hbs @@ -0,0 +1,43 @@ +// + +/* + * This file is automatically generated by '{{Root.CodeGenName}}'; any changes will be lost. + */ + +#nullable enable + +namespace {{Root.ApplicationNamespace}}; + +/// Provides the implementation. +[ScopedService] +public partial class ReferenceDataProvider(IReferenceDataRepository repository) : IReferenceDataProvider +{ + private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + + /// + public IEnumerable<(Type, Type)> Types => + [ +{{#each Entities}} + (typeof({{Name}}), typeof({{Name}}Collection)), +{{/each}} + ]; + + /// + public IEnumerable<(string, Type)> AlternateNames => + [ +{{#each Entities}} + ( "{{Route}}", typeof({{Name}}) ){{#unless @last}},{{/unless}} +{{/each}} + ]; + + /// + public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch + { +{{#each Entities}} + _ when type == typeof({{Name}}) => await _repository.GetAll{{Plural}}Async(cancellationToken).ConfigureAwait(false), +{{/each}} + _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") + }; +} + +#nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/Repository_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/Repository_cs.hbs index d0d940ee5..d2b45779c 100644 --- a/src/CoreEx.CodeGen/RefData/Templates/Repository_cs.hbs +++ b/src/CoreEx.CodeGen/RefData/Templates/Repository_cs.hbs @@ -19,12 +19,74 @@ public partial class ReferenceDataRepository : IReferenceDataRepository /// public Task GetAll{{Plural}}Async(CancellationToken cancellationToken = default) {{#ifeq Repository 'EntityFramework'}} - => {{RepositoryName}}.ThrowIfNull().Model<{{Root.DataModelsPath}}.{{Name}}>().Query().ToMappedItemsAsync<{{Root.DataModelsPath}}.{{Name}}, Contracts.{{Name}}Collection, Contracts.{{Name}}>({{Mapper}}.Map, cancellationToken); + => {{RepositoryName}}.ThrowIfNull().Model<{{Root.DataModelsPath}}.{{Name}}>().Query().ToMappedItemsAsync<{{Root.DataModelsPath}}.{{Name}}, Contracts.{{Name}}Collection, Contracts.{{Name}}>({{Mapper}}.From, cancellationToken); {{/ifeq}} {{#ifeq Repository 'Cosmos'}} - => {{RepositoryName}}.{{ModelPlural}}.Query().ToMappedItemsAsync({{Mapper}}.Map, cancellationToken); + => {{RepositoryName}}.{{ModelPlural}}.Query().ToMappedItemsAsync({{Mapper}}.From, cancellationToken); {{/ifeq}} {{/each}} +{{#each EntitiesThatAreMutableWithRepository}} + + #region {{Name}} + + {{#ifeq Repository 'EntityFramework'}} + /// + public Task> Get{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => {{RepositoryName}}.ThrowIfNull().Model().GetWithResultAsync(id, cancellationToken).ThenAs(model => {{Name}}Mapper.From.Map(model)); + + /// + public Task>> Create{{Name}}Async(Contracts.{{Name}} value, CancellationToken cancellationToken = default) + => EfDbReferenceData.CreateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().Model(), value, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Update{{Name}}Async({{IdDotNetType}} id, Contracts.{{Name}} value, CancellationToken cancellationToken = default) + => EfDbReferenceData.UpdateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().Model(), id, value, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Activate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => EfDbReferenceData.ActivateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().Model(), id, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Deactivate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => EfDbReferenceData.DeactivateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().Model(), id, {{Name}}Mapper.Default, cancellationToken); + + {{#ifeq Mutability 'CreateUpdateDelete'}} + /// + public Task> Delete{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => EfDbReferenceData.DeleteAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}>({{RepositoryName}}.ThrowIfNull().Model(), id, cancellationToken); + + {{/ifeq}} + {{/ifeq}} + {{#ifeq Repository 'Cosmos'}} + /// + public Task> Get{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => {{RepositoryName}}.ThrowIfNull().{{ModelPlural}}.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ThenAs(model => {{Name}}Mapper.From.Map(model)); + + /// + public Task>> Create{{Name}}Async(Contracts.{{Name}} value, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.CreateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().{{ModelPlural}}, value, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Update{{Name}}Async({{IdDotNetType}} id, Contracts.{{Name}} value, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.UpdateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().{{ModelPlural}}, id, value, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Activate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.ActivateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().{{ModelPlural}}, id, {{Name}}Mapper.Default, cancellationToken); + + /// + public Task>> Deactivate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.DeactivateAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}, {{Name}}Mapper>({{RepositoryName}}.ThrowIfNull().{{ModelPlural}}, id, {{Name}}Mapper.Default, cancellationToken); + + {{#ifeq Mutability 'CreateUpdateDelete'}} + /// + public Task> Delete{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken = default) + => CosmosDbReferenceData.DeleteAsync<{{IdDotNetType}}, Contracts.{{Name}}, Persistence.{{Name}}>({{RepositoryName}}.ThrowIfNull().{{ModelPlural}}, id, cancellationToken); + + {{/ifeq}} + {{/ifeq}} + #endregion +{{/each}} } #nullable restore diff --git a/src/CoreEx.CodeGen/RefData/Templates/Service_cs.hbs b/src/CoreEx.CodeGen/RefData/Templates/Service_cs.hbs index 014110c69..96f2be58d 100644 --- a/src/CoreEx.CodeGen/RefData/Templates/Service_cs.hbs +++ b/src/CoreEx.CodeGen/RefData/Templates/Service_cs.hbs @@ -8,36 +8,216 @@ namespace {{Root.ApplicationNamespace}}; -/// Provides the implementation. -[ScopedService] -public partial class ReferenceDataService(IReferenceDataRepository repository) : IReferenceDataProvider +/// Provides the mutable implementation. +[ScopedService] +public partial class ReferenceDataService : IReferenceDataService { - private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); + private readonly IUnitOfWork _unitOfWork; + private readonly IReferenceDataRepository _repository; + + /// + /// Initializes a new instance of the class. + /// + /// The . + /// The . + public ReferenceDataService(IUnitOfWork unitOfWork, IReferenceDataRepository repository) + { + _unitOfWork = unitOfWork.ThrowIfNull(); + _repository = repository.ThrowIfNull(); + OnInitialization(); + } + + /// + /// Provides the with an opportunity to perform additional initialization. + /// + partial void OnInitialization(); + + /// + /// Gets or sets the pre-check function that is invoked prior to the activate, deactivate or delete. + /// + /// This is intended to be used for any additional validation or business rules that need to be applied prior to the action being performed. + /// The passed is the reference data entity being mutated and the indicates the resulting action (being one of: + /// , , or ). + private Func> PreCheckAsync { get; set => field = value.ThrowIfNull(); } = (_, _, _) => Result.SuccessTask; + +{{#each EntitiesThatAreMutable}} + #region {{Name}} /// - public IEnumerable<(Type, Type)> Types => - [ -{{#each Entities}} - (typeof({{Name}}), typeof({{Name}}Collection)), -{{/each}} - ]; + public Task> Get{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken) => GetAsync(id, _repository.Get{{Name}}Async, cancellationToken); /// - public IEnumerable<(string, Type)> AlternateNames => - [ -{{#each Entities}} - ( "{{Route}}", typeof({{Name}}) ){{#unless @last}},{{/unless}} -{{/each}} - ]; + public Task> Create{{Name}}Async({{Name}} value, CancellationToken cancellationToken) => CreateAsync<{{Name}}, {{Validator}}>(value, _repository.Create{{Name}}Async, cancellationToken{{#ifeq Repository 'Cosmos'}}, refresh: (v, ct) => _repository.Get{{Name}}Async(v.Id, ct){{/ifeq}}); /// - public virtual async Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch - { -{{#each Entities}} - _ when type == typeof({{Name}}) => await _repository.GetAll{{Plural}}Async(cancellationToken).ConfigureAwait(false), + public Task> Update{{Name}}Async({{IdDotNetType}} id, {{Name}} value, CancellationToken cancellationToken) => UpdateAsync<{{IdDotNetType}}, {{Name}}, {{Validator}}>(id, value, _repository.Update{{Name}}Async, cancellationToken{{#ifeq Repository 'Cosmos'}}, refresh: (v, ct) => _repository.Get{{Name}}Async(v.Id, ct){{/ifeq}}); + + /// + public Task> Activate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken) => ActivateAsync(id, _repository.Get{{Name}}Async, _repository.Activate{{Name}}Async, cancellationToken{{#ifeq Repository 'Cosmos'}}, refresh: (v, ct) => _repository.Get{{Name}}Async(v.Id, ct){{/ifeq}}); + + /// + public Task> Deactivate{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken) => DeactivateAsync(id, _repository.Get{{Name}}Async, _repository.Deactivate{{Name}}Async, cancellationToken{{#ifeq Repository 'Cosmos'}}, refresh: (v, ct) => _repository.Get{{Name}}Async(v.Id, ct){{/ifeq}}); + {{#ifeq Mutability 'CreateUpdateDelete'}} + + /// + public Task Delete{{Name}}Async({{IdDotNetType}} id, CancellationToken cancellationToken) => DeleteAsync<{{IdDotNetType}}, {{Name}}>(id, _repository.Get{{Name}}Async, _repository.Delete{{Name}}Async, cancellationToken); + {{/ifeq}} + + #endregion + {{/each}} - _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") - }; + #region Internal + + /// + /// Gets the reference data of type by the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The cancellation token. + /// The result. + private Task> GetAsync(TId id, Func>> getFunc, CancellationToken cancellationToken) + where TRef : class, IReferenceData + => Result.GoAsync(() => getFunc(id, cancellationToken)); + + /// + /// Creates the reference data of type using the specified . + /// + /// The reference data . + /// The validator . + /// The reference data value. + /// The function to create the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> CreateAsync(TRef value, Func>>> create, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => create(value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Created)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Updates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The validator . + /// The identifier. + /// The reference data value. + /// The function to update the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> UpdateAsync(TId id, TRef value, Func>>> update, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData where TValidator : IValidator, new() + => Result.GoAsync(async () => (await new TValidator().ValidateAsync(value, cancellationToken).ConfigureAwait(false)).ToResult()) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => update(id, value, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Updated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Activates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to activate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> ActivateAsync(TId id, Func>> get, Func>>> activate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Activated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => activate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Activated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Deactivates the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to deactivate the reference data. + /// The cancellation token. + /// The optional function to re-get the reference data after a mutation; required where the underlying data store only finalizes the once the unit of work has completed. + /// The result. + private Task> DeactivateAsync(TId id, Func>> get, Func>>> deactivate, CancellationToken cancellationToken, Func>>? refresh = null) + where TRef : class, IReferenceData + => Result.GoAsync(() => get(id, cancellationToken)) + .ThenAsAsync(v => PreCheckAsync(v, EventAction.Deactivated, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => deactivate(id, ct)) + .ThenAs(dr => dr.WhereMutatedAnd(v => _unitOfWork.Events.Add(EventData.CreateEventWith(v, EventAction.Deactivated)))); + }, cancellationToken)) + .ThenAsAsync(dr => RefreshAsync(dr, refresh)) + .ThenAsAsync(dr => dr.WhereMutatedAsync((v, ct) => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)); + + /// + /// Re-gets the mutated reference data using the function (where specified) so that the final persisted is returned. + /// + /// This occurs after the mutation has been committed, so it is deliberately not cancellable and a refresh failure is swallowed; i.e. it must never turn a committed success into an error, which would encourage a retry of a completed operation (the ETag is then simply as per the mutation). + private static async Task>> RefreshAsync(DataResult dr, Func>>? refresh) + where TRef : class, IReferenceData + { + if (refresh is null || !dr.WasMutated) + return dr; + + try + { + var r = await refresh(dr.Value, CancellationToken.None).ConfigureAwait(false); + return r.IsSuccess ? new DataResult(r.Value, true) : dr; + } + catch (Exception) + { + return dr; + } + } + /// + /// Deletes the reference data of type using the specified . + /// + /// The identifier . + /// The reference data . + /// The identifier. + /// The function to get the reference data. + /// The function to delete the reference data. + /// The cancellation token. + /// The result. + private async Task DeleteAsync(TId id, Func>> get, Func>> delete, CancellationToken cancellationToken) + where TRef : class, IReferenceData + { + var r = await get(id, cancellationToken).ConfigureAwait(false); + if (r.IsFailure) + return r.IsNotFoundError ? Result.Success : r.AsResult(); // Delete is idempotent, so if not found then return success. + + return await Result.GoAsync(() => PreCheckAsync(r.Value, EventAction.Deleted, cancellationToken)) + .ThenAsAsync(() => _unitOfWork.TransactionAsync(ct => + { + return Result.GoAsync(() => delete(id, ct)) + .Then(dr => dr.WhereMutatedAnd(() => _unitOfWork.Events.Add(EventData.CreateEvent(EventAction.Deleted).WithKey(CompositeKey.Create(id))))); + }, cancellationToken)) + .ThenAsAsync(dr => dr.WhereMutatedAsync(ct => ReferenceDataOrchestrator.Current.TryInvalidateAsync(ct), CancellationToken.None)).ConfigureAwait(false); + } + + #endregion } #nullable restore diff --git a/src/CoreEx.CodeGen/Scripts/ref-data-script.yaml b/src/CoreEx.CodeGen/Scripts/ref-data-script.yaml index b0b876007..c59a12955 100644 --- a/src/CoreEx.CodeGen/Scripts/ref-data-script.yaml +++ b/src/CoreEx.CodeGen/Scripts/ref-data-script.yaml @@ -1,9 +1,13 @@ configType: CoreEx.CodeGen.RefData.Config.CodeGenConfig, CoreEx.CodeGen generators: -- { type: 'CoreEx.CodeGen.RefData.Generators.ContractGenerator, CoreEx.CodeGen', template: 'Contract_cs', file: '{{Name}}.g.cs', directory: '{{Root.ContractsDirectory.FullName}}', text: Contract(s) generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.ApiGenerator, CoreEx.CodeGen', template: 'Controller_cs', file: 'ReferenceDataController.g.cs', directory: '{{Root.ApiDirectory.FullName}}/Controllers', text: ReferenceDataController generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'Service_cs', file: 'ReferenceDataService.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}', text: ReferenceDataService generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'IRepository_cs', file: 'IReferenceDataRepository.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}/{{Root.DataRepositoriesPath}}', text: IReferenceDataRepository generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'Repository_cs', file: 'ReferenceDataRepository.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataRepositoriesPath}}', text: ReferenceDataRepository generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.MapperGenerator, CoreEx.CodeGen', template: 'Mapper_cs', file: '{{Name}}Mapper.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataMappingPath}}', text: Mapper(s) generation } -- { type: 'CoreEx.CodeGen.RefData.Generators.CosmosPersistenceModelGenerator, CoreEx.CodeGen', template: 'CosmosPersistenceModel_cs', file: '{{Name}}.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataModelsPath}}', text: Cosmos persistence model(s) generation } +- { type: 'CoreEx.CodeGen.RefData.Generators.ContractGenerator, CoreEx.CodeGen', template: 'Contract_cs', file: '{{Name}}.g.cs', directory: '{{Root.ContractsDirectory.FullName}}', text: Contract(s) } +- { type: 'CoreEx.CodeGen.RefData.Generators.ApiGenerator, CoreEx.CodeGen', template: 'Controller_cs', file: 'ReferenceDataController.g.cs', directory: '{{Root.ApiDirectory.FullName}}/Controllers', text: ReferenceDataController } +- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'Provider_cs', file: 'ReferenceDataProvider.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}', text: ReferenceDataProvider } +- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'IRepository_cs', file: 'IReferenceDataRepository.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}/{{Root.DataRepositoriesPath}}', text: IReferenceDataRepository } +- { type: 'CoreEx.CodeGen.RefData.Generators.RootGenerator, CoreEx.CodeGen', template: 'Repository_cs', file: 'ReferenceDataRepository.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataRepositoriesPath}}', text: ReferenceDataRepository } +- { type: 'CoreEx.CodeGen.RefData.Generators.MapperGenerator, CoreEx.CodeGen', template: 'Mapper_cs', file: '{{Name}}Mapper.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataMappingPath}}', text: Mapper(s) } +- { type: 'CoreEx.CodeGen.RefData.Generators.CosmosPersistenceModelGenerator, CoreEx.CodeGen', template: 'CosmosPersistenceModel_cs', file: '{{Name}}.g.cs', directory: '{{Root.DataDirectory.FullName}}/{{Root.DataModelsPath}}', text: Cosmos persistence model(s) } + +- { type: 'CoreEx.CodeGen.RefData.Generators.MutableApiGenerator, CoreEx.CodeGen', template: 'MutableController_cs', file: '{{Name}}Controller.g.cs', directory: '{{Root.ApiDirectory.FullName}}/Controllers', text: Mutable Controller(s) } +- { type: 'CoreEx.CodeGen.RefData.Generators.RootMutableGenerator, CoreEx.CodeGen', template: 'IService_cs', file: 'IReferenceDataService.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}/Interfaces', text: Mutable IReferenceDataService } +- { type: 'CoreEx.CodeGen.RefData.Generators.RootMutableGenerator, CoreEx.CodeGen', template: 'Service_cs', file: 'ReferenceDataService.g.cs', directory: '{{Root.ApplicationDirectory.FullName}}', text: Mutable ReferenceDataService } diff --git a/src/CoreEx.CodeGen/docs/CodeGeneration.md b/src/CoreEx.CodeGen/docs/CodeGeneration.md index f32905a63..6894990e9 100644 --- a/src/CoreEx.CodeGen/docs/CodeGeneration.md +++ b/src/CoreEx.CodeGen/docs/CodeGeneration.md @@ -31,6 +31,8 @@ Property | Description -|- **`route`** | The route prefix.
† Defaults to `/api/refdata`. **`routeConvention`** | The route naming convention where not directly specified. Valid options are: `KebabCase`, `SnakeCase`, `CamelCase`, `Lowercase`.
† Defaults to `KebabCase`. +`attribute` | The optional API controller class attribute.
† This is the attribute applied as-is to the generated `ReferenceDataController` class. This is useful for adding the likes of `[Authorize]`. +`getNamed` | Indicates whether the API controller emits the `GetNamed` operation.
† Defaults to `false`. ## Repository Provides the configuration for the generated repository code. diff --git a/src/CoreEx.CodeGen/docs/Entity.md b/src/CoreEx.CodeGen/docs/Entity.md index 3ac84426b..8031a09af 100644 --- a/src/CoreEx.CodeGen/docs/Entity.md +++ b/src/CoreEx.CodeGen/docs/Entity.md @@ -11,6 +11,7 @@ Category | Description [`API`](#API) | Provides the configuration for the generated API code. [`Repository`](#Repository) | Provides the configuration for the generated repository code. [`Mapping`](#Mapping) | Provides the configuration for the generated mapping code. +[`Mutable`](#Mutable) | Provides the configuration for the generated mutability code. [`Exclude`](#Exclude) | Provides the configuration for code generation exclusion. [`Collections`](#Collections) | Provides the collections configuration. @@ -24,8 +25,8 @@ Property | Description **`name`** | The reference-data entity (contract) name. [Mandatory] **`plural`** | The pluralized reference-data entity (contract) name.
† Defaults to `{Name}` with the last word pluralized. `text` | The reference-data entity friendly text.
† Defaults to `{Name}` converted to sentence case. This is primarily used in generated code comments. -**`idType`** | The reference-data identifier type. Valid options are: `String`, `Guid`, `Int32`, `Int64`.
† Defaults to root `{IdType}`. -**`collectionSortOrder`** | The collection sort order. Valid options are: `Code`, `Id`, `Text`, `SortOrder`.
† This is the collection sort order. Defaults to root `{CollectionSortOrder}`. +`idType` | The reference-data identifier type. Valid options are: `String`, `Guid`, `Int32`, `Int64`.
† Defaults to root `{IdType}`. +`collectionSortOrder` | The collection sort order. Valid options are: `Code`, `Id`, `Text`, `SortOrder`.
† This is the collection sort order. Defaults to root `{CollectionSortOrder}`. ## API Provides the configuration for the generated API code. @@ -33,6 +34,7 @@ Provides the configuration for the generated API code. Property | Description -|- **`route`** | The route suffix.
† Defaults to `{Plural}` and root `{RouteConvention}` configuration. +`attribute` | The optional API controller operation attribute.
† This is the attribute applied as-is to the generated `ReferenceDataController` operation. This is useful for adding the likes of `[Authorize]`. ## Repository Provides the configuration for the generated repository code. @@ -52,6 +54,15 @@ Property | Description -|- **`mapper`** | The mapper name.
† This is the .NET mapper name used within the generated code. Defaults to root `{Name}Mapper`. +## Mutable +Provides the configuration for the generated mutability code. + +Property | Description +-|- +`mutability` | The reference-data entity mutability. Valid options are: `None`, `CreateUpdate`, `CreateUpdateDelete`.
† Defaults to `None`. Requires a `Repository` of `EntityFramework` or `Cosmos`. +`validator` | The validator type name used during mutability operations.
† Defaults to `ReferenceDataValidator<{Name}>`. Must have a default constructor. +`mutableAttribute` | The mutable API controller class attribute.
† Defaults to `[Authorize]`. A configured value replaces the default attribute applied as-is to the generated `{Name}Controller` class. + ## Exclude Provides the configuration for code generation exclusion. @@ -66,4 +77,3 @@ Provides the collections configuration. Property | Description -|- `properties` | The corresponding [`Property`](Property.md) collection. - diff --git a/src/CoreEx.Cosmos/AGENTS.md b/src/CoreEx.Cosmos/AGENTS.md index 2dca85143..d1721886c 100644 --- a/src/CoreEx.Cosmos/AGENTS.md +++ b/src/CoreEx.Cosmos/AGENTS.md @@ -1,7 +1,5 @@ # CoreEx.Cosmos — AI Usage Guide -> 🚧 **Preview**: newly added in this release. The API surface may still change in a future release without following strict semver until it stabilizes. - Azure Cosmos DB implementation of the CoreEx core CRUD + query access layer pattern (model-direct and contract-to-model), structurally mirroring `CoreEx.EntityFrameworkCore`'s `EfDb`/`EfDbModel`/`EfDbMappedModel` shape, plus a `TransactionalBatch`-based transactional outbox and a Change Feed Processor-based outbox relay. ## Registration @@ -46,6 +44,17 @@ public class OrderRepository(ICosmosDb cosmosDb) `UpdateAsync` maps the model's `IETag.ETag` into `ItemRequestOptions.IfMatchEtag` when `CosmosDbArgs.AutoMapETag` is `true` (the default) **and** the caller has not already supplied their own `ItemRequestOptions` — Cosmos DB enforces the check server-side and returns a `412`, which converts to `ConcurrencyException`/`Result.ConcurrencyError` automatically via the table above. +## Change tracker (scoped snapshot cache) + +`ICosmosDb.ChangeTracker` (`CosmosDbChangeTracker`, one per `CosmosDb` scope — i.e. per request) snapshots the as-read document of every point `GetAsync`; a repeat Get of the same container/partition/id in the same scope is served from the snapshot (a **fresh deserialized instance** each time — it is a snapshot, not an identity map, so mutating a returned instance never corrupts the cache) and costs no RUs. Mirrors the EF change tracker's lifetime. + +- **Requires** `CosmosClientOptions.UseSystemTextJsonSerializerWithOptions`; otherwise tracking is silently inert (the Update behaviours below still work). +- `CosmosDbArgs.ClearChangeTrackerAfterGet` (default `false`) bypasses and does not populate the cache for that call (name kept for parity with `EfDbArgs`). 404s are never cached. +- Create/Update/Upsert/Delete **evict** the affected entry (no write-through); `Delete`'s forced pre-read always bypasses the cache. The whole tracker is cleared when the **root** `CosmosDbUnitOfWork.TransactionAsync` ends (success or failure) — there is no read-your-writes inside a batch, and the batch may have caused server-side effects. +- Snapshot publication is version-checked per container/partition/id across all model types: a read started before eviction or a whole-tracker clear cannot repopulate the cache afterward. Immediate writes also evict on completion, preventing reads that overlap the write from leaving a stale snapshot. +- **`UpdateAsync` always pre-reads** (cache-assisted) the persisted document, then: copies back server-managed values the candidate lacks — `IChangeLog`/`IChangeLogEx` `Created*` and the `CosmosDbModelBase.ExtensionData` bag (unmapped properties survive; the candidate's own keys win; Cosmos system `_rid/_self/_etag/_attachments/_ts` are skipped); and **detects a no-op** (candidate equals persisted, ignoring `ETag`, change-log/audit and the bag's system keys) returning `DataResult.WasMutated == false` without a write. A candidate carrying a non-null `ETag` that differs from the persisted one is *never* treated as a no-op — it goes to the server and yields the `412`. As with EF, a no-op decision against a stale cache hit can report success; use `ClearChangeTrackerAfterGet` where that matters. +- `ttl`: where `WithTimeToLive` is not configured and the candidate's `TimeToLive` is `null`, the persisted `ttl` is carried forward (an explicit value, e.g. `-1` for never-expires, overrides); where configured it is always recomputed. `ttl` is not excluded from the no-op compare. +- Caveat: a replace drops any persisted property that is neither mapped nor in the bag, so models that don't derive from `CosmosDbModelBase` (or implement `IExtensionData`) can still lose unmapped data. ## Multi-type containers (type discriminator) ```csharp @@ -131,12 +140,25 @@ builder.AddCosmosDbOutboxRelayHostedService("orders"); // one call per outbox- builder.AddCosmosDbOutboxRelayHostedService("customers", servicesCount: 1); // lower-volume container, fewer concurrent instances ``` +The relay **never creates** its lease container (container creation is a control-plane operation a production identity is not permitted): declare it once in the Database project with `.OutboxLeaseContainer()` (single shared container `$outbox-leases`, partition key `/id`; or provision it via IaC) - `StartAsync` fails fast with an `InvalidOperationException` if it is missing. A single lease container is safely shared by all relays (each Change Feed Processor is named per monitored container); pass `leaseContainerId` to isolate one. + +**Fresh leases replay the existing outbox backlog:** the relay explicitly starts from the beginning unless `CosmosDbOutboxRelayOptions.StartTime` is supplied. The SDK's unconfigured default is "from now", which would skip pending events on first startup or after container-reset recovery. Existing lease checkpoints take precedence; an explicit start time intentionally excludes earlier events on a fresh lease. Replay remains at-least-once: successfully relayed outbox documents are deleted. + +**Tracing defaults match the relational relays:** change-feed polling, lease maintenance, and cleanup SDK/HTTP spans are suppressed; actual publishing spans and per-event originating-trace relay markers remain enabled. Metrics and warning/error logs are unaffected. For diagnostics, set `CosmosDbOutboxRelayOptions.IsInstrumentationEnabledForPolling = true` via `configureOptions` before starting/resuming the processor. Publishing can independently be disabled via `CosmosDbOutboxRelayProcessor.IsInstrumentationEnabledForPublishing` (default `true`). + `CosmosDbEventPublisher` must never be registered as this destination publisher - it is the outbox *write-side* publisher and can only publish inside an active `CosmosDbUnitOfWork.TransactionAsync` scope, which the relay's per-batch scope never has. `CosmosDbOutboxRelayProcessor.ProcessBatchAsync` detects this misconfiguration up front and throws a clear, actionable `InvalidOperationException` naming the offending container, rather than surfacing `CosmosDbEventPublisher`'s deeper "no active transaction" error. Poison-message/dead-letter handling is **not yet implemented** - a permanently-failing outbox document is redelivered forever by the Change Feed Processor's own native backoff (confirmed empirically against the emulator), with no built-in give-up, and can starve delivery of other, unrelated outbox documents sharing the same physical partition-key-range/lease. Since a Change Feed Processor lease checkpoints strictly in order, this isn't a bounded delay - it blocks every later change in that lease indefinitely, until either the underlying cause is fixed (so the same, already-captured change eventually succeeds) or an operator intervenes at the lease/checkpoint level; deleting the live document does not help, since the change feed record being retried is an immutable snapshot, not a live read. To be designed as one shared pattern across the SQL Server/Postgres/Cosmos relays, not Cosmos-specific. **Detecting a stuck lease:** `cosmos.outbox.enqueue` continuing to climb while `cosmos.outbox.relay.publish` stays flat for the same container is the signal - the write side is unaffected by a stuck relay, so a sustained divergence between the two indicates a lease is blocked. `cosmos.outbox.relay.oldest_lag`/`newest_lag` are recorded on both a successful and a failed publish attempt, so they keep climbing (rather than going silent) for as long as a batch keeps failing - alert on a sustained rise in `cosmos.outbox.relay.oldest_lag`, not just on `cosmos.outbox.relay.publish.failed`, since a low failure count can still mean one lease has been stuck for a long time. +## Reference data + +`CosmosDbReferenceData` (static; Cosmos counterpart of `EfDbReferenceData`) provides the mutation primitives for reference data stored in a type-discriminated container: `CreateAsync`, `UpdateAsync`, `ActivateAsync`, `DeactivateAsync` and `DeleteAsync`, each returning `Result>` (`WasMutated` indicates whether anything changed). The model must derive from `CosmosDbReferenceDataModelBase`; a duplicate code surfaces as `DuplicateException` (the 409 from the unique key) - atomically, with no pre-query. This **requires** the container to be created with a `/typeDiscriminator` + `/code` unique key **and** registered with `new CosmosDbOptions().Container("", c => c.WithReferenceDataOutboxEvent())`: outbox events share the container/partition but carry neither path, Cosmos treats a missing path as null, so without it every second outbox event collides on `(null, null)` and fails with a 409. `WithReferenceDataOutboxEvent` gives each outbox document a unique `code` (its own id, via `CosmosDbOutboxEvent.ExtensionData`) so the tuple is unique; the property name follows the Cosmos client serializer's naming policy. A container with the unique key but without the option fails fast: `CosmosDbEventPublisher` checks (once per container) that outbox events populate at least one path of every unique key and throws a clear `InvalidOperationException` otherwise (instead of a misleading `DuplicateException` on the second create). Without the unique key, duplicates are silently accepted - that is a consumer configuration responsibility. Caveats: a container's unique key policy is **immutable** after creation (adding it to an existing container means recreating and migrating it); unique keys are **case-sensitive** (unlike a typical SQL Server collation, so normalise codes if `ABC`/`abc` must clash); uniqueness is per logical partition (a Cosmos constraint - which is why reference data should use a single/fixed partition; its limited volume makes partitioning overkill); and the container should be dedicated to reference data (any other document with a `typeDiscriminator` but no `code` would collide on `(type, null)`). Custom outbox-event shaping is available (and composable) via `CosmosDbContainerOptions.WithOutboxEventUpdater`. + +- Ids are generated on create (string); create is always inactive; `Code` and `IsActive` are ignored on update; delete of an active item fails (`cannot-delete-active`). +- Inside `CosmosDbUnitOfWork.TransactionAsync` writes are batched, so the returned `ETag` is not final until commit; `IReferenceData` is not `IETag` so `SynchronizeETag` cannot be used - the CodeGen-generated service re-gets the item after the transaction instead. + ## Multi-set queries ```csharp @@ -194,6 +216,33 @@ await database.ImportBatchAsync(jdr); `ImportBatchAsync` calls `Container.CreateItemAsync` directly per item (no `CosmosDbContainer` involved), so none of the usual cross-cutting pipeline runs - no ETag/tenant/logical-delete/type-discriminator handling, no `Model.PrepareCreate` stamping, no outbox enlistment. The caller's JSON must already carry the correct partition-key property value and (where relevant) type-discriminator value. +### Provisioning engine & console (`CoreEx.Cosmos.Provisioning`) + +`CosmosDbProvisioner` + `CosmosDbConsole` are the Cosmos counterpart to DbEx's `*.Database` console - dev/test only (production provisioning is IaC's job): + +```csharp +// Program.cs of a *.Database console project. +public static Task Main(string[] args) => CosmosDbConsole.Create(connectionString, "contoso").Configure(c => ConfigureProvisionArgs(c.Args)).RunAsync(args); + +public static CosmosDbProvisionArgs ConfigureProvisionArgs(CosmosDbProvisionArgs args) => args + .AddAssembly() + .Container("customers", configure: cp => { /* indexing policy etc. */ }) + .ReferenceDataContainer("ref-data") + .OutboxLeaseContainer(); // Only where a Relay host runs; never auto-created at runtime. +``` + +- Commands (comma-separable): `Drop`, `Create`, `Reset`, `Data`, `All` (Create+Data), `ResetAndData`, `DropAndAll` - run in the order Drop → Create → Reset → Data. As per DbEx, `Reset` (replaces - deletes and recreates - the declared containers, so all data *and* container settings are lost) does **not** implicitly create the database and fails if it does not exist; creation must be explicitly requested (`All`, or `Create,ResetAndData`) - `ResetAndData` alone is for an already-created database. Options: `-cs|--connection-string`, `-cv|--connection-varname` (environment variable holding the connection string), `-d|--database`, `-p|--param Name=Value` (repeatable; feeds `CosmosDbProvisionArgs.Parameters`, referenced in seed data as `^Name`/`(^Name)` and overriding same-named options parameters), `--accept-prompts`. `Drop` and `Reset` prompt for confirmation (before any connection is made) unless `--accept-prompts` is specified - CI/scripts running them must pass it. +- **Data convention:** embedded resources in `Args.Assemblies` whose name contains `.Data.` and ends `.yaml|.yml|.json` are imported in ordinal order. Each top-level key **is the container id** and must match a declared container (otherwise it throws). +- **Console output (DbEx parity):** `CosmosDbConsole` writes a banner and settings block (command, endpoint - never the key, database, containers, assemblies) then `CosmosDbProvisioner.RunAsync` writes a titled, timed section per stage (`DATABASE DROP`, `DATABASE CREATE`, `CONTAINER RESET`, `DATABASE DATA`) separated by an 80-character rule, with created/already-exists lines and per-container (or per type-discriminator group) `Result:` counts (`N item(s) (X created, Y replaced)` for a merge). Everything after the banner goes through `CosmosDbProvisionArgs.Output`, so tests can capture it. `RunAndLogAsync` is the DbEx-style equivalent returning `(Success, Output)`: it captures the run output, converts failures into `Success = false` plus a `Failed: ...` line (cancellation still throws), and echoes the output to the original writer. +- **Insert vs merge (DbEx parity):** a top-level container key (or, for a ref-data container, a type-discriminator group key) prefixed with `$` is **merged** (upserted) and so is re-runnable; without `$` it is **inserted** (a re-run conflicts with a 409). A group key prefixed with `^` generates the `id` where not specified (the options' `id` default, `^id`); without it each item must carry its own. Merging ref-data reuses the existing `id` found by `typeDiscriminator` + `code` (seed ids are regenerated on each parse). A `$` on a ref-data container key itself has no effect - put it on the group key (`$^CustomerType`). +- `OutboxLeaseContainer()` declares the Change Feed Processor lease container (default id `$outbox-leases`, partition key `/id`) shared by the outbox relay(s); it is created/reset with the other containers and logged as an "outbox lease container". +- `ReferenceDataContainer` adds a unique key on `/typeDiscriminator` + `/code` and imports through `ImportDiscriminatedBatchAsync` (discriminator derived from the YAML key). +- **Seed-data reader options are overridable** (`JsonDataReaderOptions`: standard properties, `TenantId`, parameters, `RootNodePreProcessor`, ...). Resolution per container per file: the data resource's factory (`AddDataResource(..., dataOptions)`; receives `CosmosDbDataContext`: resource name + container + naming convention) → the container's own `dataOptions` factory (`Container(..., dataOptions:)`/`ReferenceDataContainer(..., dataOptions:)`) → default (`CreateForReferenceData` for ref-data; otherwise a plain reader). A factory returning `null` defers to the next level. Factories must return a **new** instance each call (options are mutable/stateful). There is deliberately no args-level override (last-wins is hard to reason about across `Program.cs` and tests) and no `idGenerator`/`includeTenantId` shortcuts (use `JsonDataReaderOptions.CreateForReferenceData(...)` in a factory). Adding the same resource twice throws; a resource that is also found by the assembly `Data` scan is imported once using its options. +- Loopback endpoints (the emulator) use Gateway mode with the certificate check relaxed; remote endpoints are untouched. +- Container creation retries on 503/`HttpRequestException`. The legacy emulator image had a ~35 container cap that also surfaced as a 503; the Linux **vNext** emulator used by `docker-compose.yml` has no such cap. +- **Emulator (vNext):** `docker-compose.yml` runs `azure-cosmos-emulator:vnext-latest` with `--protocol https` (the .NET SDK needs HTTPS; vNext defaults to HTTP), the health probe moved to port 8083 (`/ready`; default 8080 clashes with the DTS emulator) and host networking (published ports do not forward the TLS gateway under rootless Podman on WSL). Known vNext conformance quirk: a `TransactionalBatch` `ReplaceItem` whose `If-Match` is not GUID-shaped is silently ignored (the batch commits everything) instead of failing with 412 - real Cosmos DB returns 412, and real ETags are always well-formed, so tests that simulate a stale ETag must use a GUID-shaped value (e.g. `00000000-0000-0000-0000-000000000000`). +- **Stale cache:** a `CosmosClient` that has cached a container/database that is then replaced or dropped+recreated can throw 404/1003 "stale cache" errors. Use a fresh client (and, in tests, a unique database id) after a reset; `UnitTestEx` glue `MigrateCosmosDataAsync(...)` (CoreEx.UnitTesting) runs the provisioner using the host's own client; as per `MigratePostgresDataAsync` the first call per account endpoint + database is `Create | ResetAndData` and subsequent calls are `ResetAndData` (the `DistributedApplication` overload always runs `Create | ResetAndData`). There is no cheap data-only reset in Cosmos DB, so each call still replaces the containers. + ## Do Not - Do not construct `CosmosClient` directly in application code — resolve it from DI (Aspire's `AddAzureCosmosClient`). diff --git a/src/CoreEx.Cosmos/CoreEx.Cosmos.csproj b/src/CoreEx.Cosmos/CoreEx.Cosmos.csproj index a85e4b3db..c94daa575 100644 --- a/src/CoreEx.Cosmos/CoreEx.Cosmos.csproj +++ b/src/CoreEx.Cosmos/CoreEx.Cosmos.csproj @@ -6,6 +6,7 @@ + diff --git a/src/CoreEx.Cosmos/CosmosDb.cs b/src/CoreEx.Cosmos/CosmosDb.cs index 96c56b05b..a082a5133 100644 --- a/src/CoreEx.Cosmos/CosmosDb.cs +++ b/src/CoreEx.Cosmos/CosmosDb.cs @@ -61,6 +61,9 @@ public CosmosDb(CosmosClient client, string databaseId, CosmosDbOptions? options /// protected ILogger? Logger { get; } + /// + public CosmosDbChangeTracker ChangeTracker { get; } = new(); + /// public CosmosDbTransaction? CurrentTransaction { get; private set; } diff --git a/src/CoreEx.Cosmos/CosmosDbArgs.cs b/src/CoreEx.Cosmos/CosmosDbArgs.cs index 8753c6b7d..c615e1fca 100644 --- a/src/CoreEx.Cosmos/CosmosDbArgs.cs +++ b/src/CoreEx.Cosmos/CosmosDbArgs.cs @@ -28,6 +28,13 @@ public record class CosmosDbArgs : IDataArgs /// rarely required; it is provided primarily for structural/behavioral parity with other CoreEx data access layers. public bool Refresh { get; init; } = false; + /// + /// Indicates whether the is bypassed (not read from, not stored to, and any existing entry evicted) for a Get operation. + /// + /// Defaults to ; i.e. the persisted document read is snapshotted so that a subsequent Get within the same scope is served from the snapshot (a fresh instance each time) + /// and an Update can use it. Set to to always read from Cosmos DB. Named for parity with the Entity Framework equivalent. + public bool ClearChangeTrackerAfterGet { get; init; } = false; + /// /// Gets or sets the applied to point operations (Get/Create/Update/Delete). /// diff --git a/src/CoreEx.Cosmos/CosmosDbChangeTracker.cs b/src/CoreEx.Cosmos/CosmosDbChangeTracker.cs new file mode 100644 index 000000000..49ef7cd07 --- /dev/null +++ b/src/CoreEx.Cosmos/CosmosDbChangeTracker.cs @@ -0,0 +1,101 @@ +namespace CoreEx.Cosmos; + +/// +/// Provides the scoped, per- change tracker; a snapshot cache of the persisted documents read by point Get operations. +/// +/// The snapshot is the as-read serialized document; every cache hit deserializes a fresh TModel instance, so a caller mutating a returned instance can never corrupt the snapshot (nor +/// another caller's instance) - this is deliberately a snapshot, not an identity map. The lifetime is that of the owning (typically DI-scoped, i.e. per-request) , mirroring the Entity Framework +/// change tracker. +/// The snapshot is used to: avoid repeat reads (RU cost) of the same document within a scope; let Update perform its own pre-read in order to copy back server-managed values (change-log creation, unmapped +/// additional properties) and to detect a no-op. Every write (Create/Update/Upsert/Delete) evicts the affected entry, and the whole tracker is cleared when the root transaction ends (as there +/// may be any number of server-side effects). A single opts a call out. +/// Tracking is only enabled where the underlying is configured with ; otherwise this is silently inert. +public sealed class CosmosDbChangeTracker +{ + private readonly ConcurrentDictionary<(string ContainerId, Type ModelType, string PartitionKey, string Id), byte[]> _snapshots = new(); +#if NET8_0 + private readonly object _lock = new(); +#else + private readonly Lock _lock = new(); +#endif + private readonly Dictionary<(string ContainerId, string PartitionKey, string Id), object> _versions = new(); + + /// + /// Gets the number of tracked (snapshotted) documents. + /// + public int Count => _snapshots.Count; + + /// + /// Clears (evicts) all tracked documents. + /// + public void Clear() + { + lock (_lock) + { + _snapshots.Clear(); + _versions.Clear(); + } + } + + /// + /// Captures the document version before a read starts. + /// + internal object CaptureVersion(string containerId, PartitionKey partitionKey, string id) + { + lock (_lock) + { + var key = (containerId, partitionKey.ToString(), id); + if (!_versions.TryGetValue(key, out var version)) + _versions.Add(key, version = new object()); + + return version; + } + } + + /// + /// Attempts to get a fresh copy of the snapshotted model. + /// + internal bool TryGet(JsonSerializerOptions options, string containerId, PartitionKey partitionKey, string id, [NotNullWhen(true)] out TModel? model) where TModel : class + { + if (_snapshots.TryGetValue((containerId, typeof(TModel), partitionKey.ToString(), id), out var bytes)) + { + model = JsonSerializer.Deserialize(bytes, options); + return model is not null; + } + + model = null; + return false; + } + + /// + /// Snapshots the . + /// + /// The captured version must still be current; eviction or clearing invalidates in-flight reads. + internal void Set(JsonSerializerOptions options, string containerId, PartitionKey partitionKey, string id, TModel model, object version) where TModel : class + { + var bytes = JsonSerializer.SerializeToUtf8Bytes(model, options); + var pk = partitionKey.ToString(); + lock (_lock) + { + if (_versions.TryGetValue((containerId, pk, id), out var current) && ReferenceEquals(version, current)) + _snapshots[(containerId, typeof(TModel), pk, id)] = bytes; + } + } + + /// + /// Evicts the document (for all model types) from the tracker. + /// + internal void Remove(string containerId, PartitionKey partitionKey, string id) + { + var pk = partitionKey.ToString(); + lock (_lock) + { + _versions.Remove((containerId, pk, id)); + foreach (var key in _snapshots.Keys) + { + if (key.ContainerId == containerId && key.PartitionKey == pk && key.Id == id) + _snapshots.TryRemove(key, out _); + } + } + } +} diff --git a/src/CoreEx.Cosmos/CosmosDbContainer.Create.cs b/src/CoreEx.Cosmos/CosmosDbContainer.Create.cs index d2cddc573..4f781ab9a 100644 --- a/src/CoreEx.Cosmos/CosmosDbContainer.Create.cs +++ b/src/CoreEx.Cosmos/CosmosDbContainer.Create.cs @@ -65,6 +65,7 @@ private async Task>> CreateWithResultInternalAsync(Cos var partitionKeyValue = Options.GetPartitionKeyValue(model); var partitionKey = CosmosDbModelOptions.ToPartitionKey(partitionKeyValue); + EvictFromChangeTracker(partitionKey, Options.FormatIdentifier(Options.GetKeyFromModel(model))); // Where an ambient CosmosDbUnitOfWork transaction is active, enlist (queue) rather than execute immediately - see CosmosDbUnitOfWork for the full deferred-execution/atomicity model. The model's // ETag is not yet final at this point (the batch has not executed) - see IUnitOfWork.SynchronizeETag for how a caller resolves the true, persisted ETag once the unit-of-work has committed. @@ -75,7 +76,15 @@ private async Task>> CreateWithResultInternalAsync(Cos return Result.Ok(new DataResult(model, true)); } - var response = await Container.CreateItemAsync(model, partitionKey, BuildItemRequestOptions(args), cancellationToken).ConfigureAwait(false); + ItemResponse response; + try + { + response = await Container.CreateItemAsync(model, partitionKey, BuildItemRequestOptions(args), cancellationToken).ConfigureAwait(false); + } + finally + { + EvictFromChangeTracker(partitionKey, Options.FormatIdentifier(Options.GetKeyFromModel(model))); + } // Refresh as required (rarely needed given the SDK already returns the persisted resource). var pr = await RefreshPostMutationAsync(args, response.Resource, partitionKey, memberName, cancellationToken).ConfigureAwait(false); diff --git a/src/CoreEx.Cosmos/CosmosDbContainer.Delete.cs b/src/CoreEx.Cosmos/CosmosDbContainer.Delete.cs index 8c0c9b021..aed957ac6 100644 --- a/src/CoreEx.Cosmos/CosmosDbContainer.Delete.cs +++ b/src/CoreEx.Cosmos/CosmosDbContainer.Delete.cs @@ -114,6 +114,7 @@ private async Task> DeleteWithResultInternalAsync(CosmosDbArg throw new InvalidOperationException($"The model implements {nameof(IReadOnlyLogicallyDeleted)} which is ambiguous for a delete operation; the model must implement {nameof(ILogicallyDeleted)} not {nameof(IReadOnlyLogicallyDeleted)}."); var id = Options.FormatIdentifier(key); + EvictFromChangeTracker(partitionKey, id); async Task> PhysicalDeleteAsync() { @@ -141,6 +142,10 @@ async Task> PhysicalDeleteAsync() // A delete is considered idempotent; a 'not found' is not an error. return Result.Ok(DataResult.False); } + finally + { + EvictFromChangeTracker(partitionKey, id); + } } // Fast path: nothing is configured for CheckModel to check (no ITenantId/IReadOnlyTenantId support, no logical delete, no WithFilter registrations, no WithTypeDiscriminator configuration) AND @@ -156,7 +161,8 @@ async Task> PhysicalDeleteAsync() // Fetch first (via CheckModel) so tenant ownership and any configured WithFilter checks are enforced consistently with Get/Update for both a physical and a logical delete - a physical // DeleteItemAsync/ReplaceItemAsync call has no other opportunity to apply them, as Cosmos DB deletes/replaces purely by id + partition key with no awareness of our tenant/filter concerns. - var gr = await GetWithResultInternalAsync(args, key, partitionKey, memberName, treatNullAsNotFound: false, cancellationToken).ConfigureAwait(false); + // The read always bypasses the change tracker - a stale "exists" would otherwise fail a whole TransactionalBatch (or a logical-delete replace) with a 404/412. + var gr = await GetWithResultInternalAsync(args with { ClearChangeTrackerAfterGet = true }, key, partitionKey, memberName, treatNullAsNotFound: false, cancellationToken).ConfigureAwait(false); if (gr.IsFailure) return gr.Bind(); diff --git a/src/CoreEx.Cosmos/CosmosDbContainer.Get.cs b/src/CoreEx.Cosmos/CosmosDbContainer.Get.cs index 2f785526d..9c1515e10 100644 --- a/src/CoreEx.Cosmos/CosmosDbContainer.Get.cs +++ b/src/CoreEx.Cosmos/CosmosDbContainer.Get.cs @@ -93,10 +93,23 @@ public async Task> GetWithResultAsync(CosmosDbArgs args, Composit => await CosmosDb.Invoker.InvokeAsync(CosmosDb, args.ThrowIfNull(), async (_, args, cancellationToken) => { var id = Options.FormatIdentifier(key); + var serializerOptions = ChangeTrackerSerializerOptions; + var tracker = CosmosDb.ChangeTracker; + // Serve from the change tracker snapshot (a fresh instance each time) unless bypassed (refresh, or caller-supplied ItemRequestOptions such as a conditional read or consistency/session option, which must reach Cosmos DB); CheckModel is always re-run as the args (e.g. filters) can differ per call. + if (serializerOptions is not null && !args.ClearChangeTrackerAfterGet && args.ItemRequestOptions is null && tracker.TryGet(serializerOptions, Container.Id, partitionKey, id, out var tracked)) + return CheckModel(args, tracked, OperationType.Get, treatNullAsNotFound); + + var version = serializerOptions is not null && !args.ClearChangeTrackerAfterGet ? tracker.CaptureVersion(Container.Id, partitionKey, id) : null; try { var response = await Container.ReadItemAsync(id, partitionKey, BuildItemRequestOptions(args), cancellationToken).ConfigureAwait(false); + + if (args.ClearChangeTrackerAfterGet) + tracker.Remove(Container.Id, partitionKey, id); + else if (serializerOptions is not null && response.Resource is not null && version is not null) + tracker.Set(serializerOptions, Container.Id, partitionKey, id, response.Resource, version); + return CheckModel(args, response.Resource, OperationType.Get, treatNullAsNotFound); } catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) diff --git a/src/CoreEx.Cosmos/CosmosDbContainer.Update.cs b/src/CoreEx.Cosmos/CosmosDbContainer.Update.cs index 340358789..a4433bde2 100644 --- a/src/CoreEx.Cosmos/CosmosDbContainer.Update.cs +++ b/src/CoreEx.Cosmos/CosmosDbContainer.Update.cs @@ -67,19 +67,20 @@ private async Task>> UpdateWithResultInternalAsync(Cos var partitionKey = CosmosDbModelOptions.ToPartitionKey(partitionKeyValue); var id = Options.FormatIdentifier(Options.GetKeyFromModel(model)); - // The CheckModel call above only validates the incoming model, which is always self-consistent - Model.PrepareUpdate stamps its ITenantId/ITypeDiscriminator from the caller's own execution - // context/type before the check runs - so it cannot detect that the PERSISTED document at this id/partition actually belongs to a different tenant, is logically deleted, fails an additive - // WithFilter authorization rule, or belongs to a different configured type. Unless none of those are configured (mirrors the equivalent Delete fast-path condition exactly), the existing - // document must be read and validated first via CheckModel(OperationType.Get) - a blind ReplaceItemAsync/batch enlistment has no other way to enforce them, since Cosmos DB replaces purely by - // id + partition key with no awareness of these concerns. A missing document surfaces as the same Result.NotFoundError() a subsequent ReplaceItemAsync 404 would have produced anyway (unlike - // Delete, a missing document is not treated as an idempotent no-op for Update). Note this only reads to validate isolation - the request's own ETag (captured below from the incoming model, - // not this pre-read) is what still governs optimistic concurrency for the replace itself. - if (!Options.LogicalDeleteSupport.IsNone || Options.TenantSupport.IsSupported || Options.HasFilters || Options.IsTypeDiscriminatorFilterEnabled) - { - var er = await GetWithResultInternalAsync(args, Options.GetKeyFromModel(model), partitionKey, memberName, treatNullAsNotFound: true, cancellationToken).ConfigureAwait(false); - if (er.IsFailure) - return er.Bind(); - } + // The existing (persisted) document is always read (a change tracker hit where already read within this scope) for the following reasons: + // - The CheckModel call above only validates the incoming model, which is always self-consistent (Model.PrepareUpdate stamps its ITenantId/ITypeDiscriminator from the caller's own execution + // context/type before the check runs), so it cannot detect that the PERSISTED document at this id/partition actually belongs to a different tenant, is logically deleted, fails an additive + // WithFilter authorization rule, or belongs to a different configured type. + // - The server-managed values (change-log created, additional properties not known to the model) must be copied back as a Cosmos DB replace overwrites the whole document. + // - A no-op update (nothing but server-managed values differ) can be detected and skipped. + // A missing document surfaces as the same Result.NotFoundError() a subsequent ReplaceItemAsync 404 would have produced anyway (unlike Delete, a missing document is not treated as an idempotent no-op + // for Update). The request's own ETag (captured below from the incoming model, not this pre-read) is what still governs optimistic concurrency for the replace itself. + var er = await GetWithResultInternalAsync(args, Options.GetKeyFromModel(model), partitionKey, memberName, treatNullAsNotFound: true, cancellationToken).ConfigureAwait(false); + if (er.IsFailure) + return er.Bind(); + + var existing = er.Value!; + CopyServerManagedValues(existing, model); // Cosmos DB's native If-Match optimistic concurrency is enforced server-side (returns a 412 directly), unlike a relational/EF detached-entity comparison; the CosmosDbInvoker maps a 412 to a // ConcurrencyException/Result.ConcurrencyError automatically. Note: AutoMapETag only synthesizes an ItemRequestOptions when the caller has not already supplied one (args.ItemRequestOptions is @@ -88,6 +89,13 @@ private async Task>> UpdateWithResultInternalAsync(Cos if (options is null && args.AutoMapETag && model is IReadOnlyETag etag && !string.IsNullOrEmpty(etag.ETag)) options = new ItemRequestOptions { IfMatchEtag = etag.ETag }; + // Skip the replace where nothing has changed - unless an ETag is being asserted that differs from the persisted one, in which case the server must be allowed to fail it (412) as it otherwise would. The ETags are compared + // normalized (quote-bookends stripped) as the persisted value is Cosmos DB's native quote-wrapped "_etag" whereas one arriving via an If-Match header has already been parsed. + if (AreEquivalent(existing, model) && (string.IsNullOrEmpty(options?.IfMatchEtag) || ETag.ParseETag(options.IfMatchEtag) == ETag.ParseETag((existing as IReadOnlyETag)?.ETag ?? string.Empty))) + return Result.Ok(new DataResult(existing, false)); + + EvictFromChangeTracker(partitionKey, id); + // Where an ambient CosmosDbUnitOfWork transaction is active, enlist (queue) rather than execute immediately - see CosmosDbUnitOfWork for the full deferred-execution/atomicity model. The model's // ETag is not yet final at this point (the batch has not executed) - see IUnitOfWork.SynchronizeETag for how a caller resolves the true, persisted ETag once the unit-of-work has committed. var txn = CosmosDb.CurrentTransaction; @@ -98,7 +106,15 @@ private async Task>> UpdateWithResultInternalAsync(Cos return Result.Ok(new DataResult(model, true)); } - var response = await Container.ReplaceItemAsync(model, id, partitionKey, options, cancellationToken).ConfigureAwait(false); + ItemResponse response; + try + { + response = await Container.ReplaceItemAsync(model, id, partitionKey, options, cancellationToken).ConfigureAwait(false); + } + finally + { + EvictFromChangeTracker(partitionKey, id); + } // Refresh as required (rarely needed given the SDK already returns the persisted resource). var pr = await RefreshPostMutationAsync(args, response.Resource, partitionKey, memberName, cancellationToken).ConfigureAwait(false); diff --git a/src/CoreEx.Cosmos/CosmosDbContainer.cs b/src/CoreEx.Cosmos/CosmosDbContainer.cs index a0c9eb5ab..ae5bf3c03 100644 --- a/src/CoreEx.Cosmos/CosmosDbContainer.cs +++ b/src/CoreEx.Cosmos/CosmosDbContainer.cs @@ -80,6 +80,74 @@ internal CosmosDbContainer(ICosmosDb cosmosDb, Container container, CosmosDbMode return Options.CheckFilters(args, model, operationType); } + // Top-level properties that are server-managed (or carried separately) and therefore never considered when determining whether an Update is a no-op. + private static readonly string[] _noOpExcludedProperties = [nameof(IETag.ETag), nameof(IChangeLog.ChangeLog), nameof(IChangeLogEx.CreatedBy), nameof(IChangeLogEx.CreatedOn), nameof(IChangeLogEx.UpdatedBy), nameof(IChangeLogEx.UpdatedOn), nameof(IExtensionData.ExtensionData)]; + + // Cosmos DB system properties that surface in the additional properties bag; these are server-owned and never forwarded/compared. + private static readonly string[] _cosmosSystemProperties = ["_rid", "_self", "_etag", "_attachments", "_ts"]; + + /// + /// Gets the used to snapshot to/from the ; where tracking is not enabled (no System.Text.Json serializer configured). + /// + private JsonSerializerOptions? ChangeTrackerSerializerOptions => CosmosDb.Client.ClientOptions.UseSystemTextJsonSerializerWithOptions; + + /// + /// Evicts the specified document from the . + /// + private void EvictFromChangeTracker(PartitionKey partitionKey, string id) => CosmosDb.ChangeTracker.Remove(Container.Id, partitionKey, id); + + /// + /// Copies the server-managed values from the (persisted) model onto the (candidate) being updated so they are not lost by the replace. + /// + /// The is deliberately never copied; the incoming value drives the server-side If-Match optimistic concurrency check. + /// The persisted is only carried forward where the candidate has none and is not configured (where configured the value is + /// always recomputed); otherwise a replace would silently remove the expiry. An explicit value (e.g. -1 for never-expires) overrides. + private void CopyServerManagedValues(TModel existing, TModel model) + { + if (!Options.HasTimeToLive && model is ITimeToLive ttl && ttl.TimeToLive is null && existing is IReadOnlyTimeToLive ettl) + ttl.TimeToLive = ettl.TimeToLive; + + if (model is IChangeLog cl) + { + var ecl = (existing as IReadOnlyChangeLog)?.ChangeLog; + cl.ChangeLog = (cl.ChangeLog ?? new ChangeLog()) with { CreatedBy = ecl?.CreatedBy, CreatedOn = ecl?.CreatedOn }; + } + else if (model is IChangeLogEx cle && existing is IReadOnlyChangeLogEx ecle) + { + cle.CreatedBy = ecle.CreatedBy; + cle.CreatedOn = ecle.CreatedOn; + } + + // Forward any persisted additional properties the candidate does not itself supply (the candidate wins on a key conflict) as a replace otherwise drops anything the model is unaware of. + if (model is IExtensionData map && existing is IExtensionData eap && eap.ExtensionData is not null) + { + foreach (var kvp in eap.ExtensionData) + { + if (_cosmosSystemProperties.Contains(kvp.Key)) + continue; + + (map.ExtensionData ??= []).TryAdd(kvp.Key, kvp.Value); + } + } + } + + /// + /// Determines whether the (candidate) is equivalent to the (persisted) model, ignoring the server-managed values (see ). + /// + private static bool AreEquivalent(TModel existing, TModel model) + { + if (!RuntimeMetadata.AreEqual(existing, model, _noOpExcludedProperties)) + return false; + + if (existing is IExtensionData eap && model is IExtensionData map) + return RuntimeMetadata.AreEqual(WithoutSystemProperties(eap.ExtensionData), WithoutSystemProperties(map.ExtensionData)); + + return true; + } + + private static Dictionary WithoutSystemProperties(Dictionary? bag) + => bag is null ? [] : bag.Where(kvp => !_cosmosSystemProperties.Contains(kvp.Key)).ToDictionary(kvp => kvp.Key, kvp => kvp.Value); + /// /// Builds the for a point operation from the specified . /// diff --git a/src/CoreEx.Cosmos/CosmosDbContainerOptions.cs b/src/CoreEx.Cosmos/CosmosDbContainerOptions.cs new file mode 100644 index 000000000..1e864fa2f --- /dev/null +++ b/src/CoreEx.Cosmos/CosmosDbContainerOptions.cs @@ -0,0 +1,64 @@ +namespace CoreEx.Cosmos; + +/// +/// Provides options for the . +/// +public class CosmosDbContainerOptions +{ + private readonly List> _outboxEventUpdaters = []; + + /// + /// Adds a updater action to be invoked when an outbox event is being created. + /// + /// The action to update the outbox event. + /// The to support fluent-style method-chaining. + /// This can be used to customize the outbox events when they are created; for example, setting an additional property to avoid a duplicate (see ). + /// Multiple updaters are supported and are invoked in the order added. + public CosmosDbContainerOptions WithOutboxEventUpdater(Action outboxEventUpdater) + { + outboxEventUpdater.ThrowIfNull(); + _outboxEventUpdaters.Add((e, _) => outboxEventUpdater(e)); + return this; + } + + /// + /// Adds a updater action to be invoked when an outbox event is being created, which is also provided the used by the . + /// + /// The action to update the outbox event; the will be where the does not specify any. + /// The to support fluent-style method-chaining. + /// Required where the property name emitted must honor the serializer's , as keys are written as-is. + public CosmosDbContainerOptions WithOutboxEventUpdater(Action outboxEventUpdater) + { + _outboxEventUpdaters.Add(outboxEventUpdater.ThrowIfNull()); + return this; + } + + /// + /// Adds a updater action to be invoked when an outbox event is being created for a reference data item to uniquely set the + /// as an additional property (see ) to avoid a duplicate. + /// + /// The to support fluent-style method-chaining. + /// Outbox events share the container (and partition) with the business documents, so a unique key policy (e.g. /typeDiscriminator and /code) would otherwise treat every outbox event as the same + /// (, ) key and reject all but the first. The property name honors the of the (falling back + /// to ), so that it matches that of the business documents. + public CosmosDbContainerOptions WithReferenceDataOutboxEvent() + => WithOutboxEventUpdater((e, jso) => + { + var name = (jso ?? Json.JsonDefaults.SerializerOptions).PropertyNamingPolicy?.ConvertName(nameof(RefData.Abstractions.IReferenceData.Code)) ?? nameof(RefData.Abstractions.IReferenceData.Code); + + e.ExtensionData ??= []; + e.ExtensionData.TryAdd(name, e.Id); + }); + + /// + /// Applies the updater actions to the specified . + /// + /// The outbox event to apply the updater actions to. + /// The used by the , where specified. + public void ApplyOutboxEventUpdater(CosmosDbOutboxEvent outboxEvent, JsonSerializerOptions? serializerOptions = null) + { + outboxEvent.ThrowIfNull(); + foreach (var updater in _outboxEventUpdaters) + updater(outboxEvent, serializerOptions); + } +} \ No newline at end of file diff --git a/src/CoreEx.Cosmos/CosmosDbModelBase.cs b/src/CoreEx.Cosmos/CosmosDbModelBase.cs index 044593447..f586cc93d 100644 --- a/src/CoreEx.Cosmos/CosmosDbModelBase.cs +++ b/src/CoreEx.Cosmos/CosmosDbModelBase.cs @@ -11,7 +11,7 @@ namespace CoreEx.Cosmos; /// time, so implement directly (rather than deriving from this base class) where a different property name is required. /// lives in core CoreEx.Data (alongside /), not CoreEx.Cosmos, since a future non-Cosmos NoSQL data-access /// package (e.g. MongoDB, which has its own distinct TTL-index mechanism) can reuse the same storage-agnostic contract. -public abstract class CosmosDbModelBase : IIdentifier, IChangeLog, IETag, IPartitionKey, ITimeToLive +public abstract class CosmosDbModelBase : IIdentifier, IChangeLog, IETag, IPartitionKey, ITimeToLive, IExtensionData { /// [JsonPropertyName("id")] @@ -46,4 +46,8 @@ public abstract class CosmosDbModelBase : IIdentifier, IChangeLog, IETag [JsonPropertyOrder(100002)] [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public int? TimeToLive { get; set; } + + /// + [JsonExtensionData] + public Dictionary? ExtensionData { get; set; } } diff --git a/src/CoreEx.Cosmos/CosmosDbModelOptions.cs b/src/CoreEx.Cosmos/CosmosDbModelOptions.cs index c4fc1da10..bcede5f2c 100644 --- a/src/CoreEx.Cosmos/CosmosDbModelOptions.cs +++ b/src/CoreEx.Cosmos/CosmosDbModelOptions.cs @@ -6,6 +6,9 @@ namespace CoreEx.Cosmos; /// The model . public class CosmosDbModelOptions where TModel : class, IEntityKey, new() { + private static readonly MethodInfo _startsWithMethod = typeof(string).GetMethod(nameof(string.StartsWith), [typeof(string)])!; + private static readonly Lazy>?> _outboxIdExclusion = new(ResolveOutboxIdExclusion); + private readonly List<(Func, IQueryable> Filter, Func? NonQueryResult, bool AllowFilterBypass)> _filters = []; private Func _getKey = m => m.EntityKey; private Func _formatIdentifier = key => key.ToString() ?? string.Empty; @@ -80,14 +83,10 @@ namespace CoreEx.Cosmos; var parameter = Expression.Parameter(typeof(TModel), "m"); var idAccess = Expression.Property(parameter, property); - var startsWith = Expression.Call(idAccess, StartsWithMethod, Expression.Constant(CosmosDbOutboxEvent.OutboxKeyPrefix)); + var startsWith = Expression.Call(idAccess, _startsWithMethod, Expression.Constant(CosmosDbOutboxEvent.OutboxKeyPrefix)); return Expression.Lambda>(Expression.Not(startsWith), parameter); } - private static readonly MethodInfo StartsWithMethod = typeof(string).GetMethod(nameof(string.StartsWith), [typeof(string)])!; - - private static readonly Lazy>?> _outboxIdExclusion = new(ResolveOutboxIdExclusion); - /// /// Gets the default . /// @@ -302,6 +301,11 @@ public CosmosDbModelOptions WithTimeToLive(Func getTimeToL return this; } + /// + /// Indicates whether has been configured. + /// + public bool HasTimeToLive => _getTimeToLive is not null; + /// /// Applies the -computed time-to-live (where configured) to the . /// diff --git a/src/CoreEx.Cosmos/CosmosDbOptions.cs b/src/CoreEx.Cosmos/CosmosDbOptions.cs index c3fffea36..847e2ccc4 100644 --- a/src/CoreEx.Cosmos/CosmosDbOptions.cs +++ b/src/CoreEx.Cosmos/CosmosDbOptions.cs @@ -10,6 +10,8 @@ public class CosmosDbOptions // sharing that containerId throwing InvalidCastException when it tries to cast the cached entry back to its own CosmosDbModelOptions. private readonly ConcurrentDictionary<(string ContainerId, Type ModelType), object> _models = new(); + private readonly ConcurrentDictionary _containers = new(); + /// /// Gets the default . /// @@ -66,4 +68,56 @@ public CosmosDbOptions WithArgs(CosmosDbArgs args) modelOptions = null; return false; } + + /// + /// Gets or adds the for the specified container . + /// + /// The identifier. + /// An optional action to configure the . + /// The for the specified container. + /// is invoked only the first time a is created for this - deliberately, + /// since this is typically a long-lived singleton shared across every instance (e.g. one per request/scope) that calls + /// for the same . + public CosmosDbContainerOptions GetOrAddContainerOptions(string containerId, Action? configure = null) + { + return _containers.GetOrAdd(containerId.ThrowIfNull(), _ => + { + var options = new CosmosDbContainerOptions(); + configure?.Invoke(options); + return options; + }); + } + + /// + /// Tries to get the for the specified container . + /// + /// The identifier. + /// The where found. + /// where the was found; otherwise, . + public bool TryGetContainerOptions(string containerId, [NotNullWhen(true)] out CosmosDbContainerOptions? containerOptions) + { + if (_containers.TryGetValue(containerId.ThrowIfNull(), out var co)) + { + containerOptions = co; + return true; + } + + containerOptions = null; + return false; + } + + /// + /// Adds the specified to the with optional configuration. + /// + /// The identifier. + /// An optional action to configure the . + /// The to support fluent-style method-chaining. + /// is invoked only the first time a is created for this - deliberately, + /// since this is typically a long-lived singleton shared across every instance (e.g. one per request/scope) that calls + /// for the same . + public CosmosDbOptions Container(string containerId, Action? configure = null) + { + GetOrAddContainerOptions(containerId, configure); + return this; + } } diff --git a/src/CoreEx.Cosmos/CosmosDbReferenceData.cs b/src/CoreEx.Cosmos/CosmosDbReferenceData.cs new file mode 100644 index 000000000..4491fb9de --- /dev/null +++ b/src/CoreEx.Cosmos/CosmosDbReferenceData.cs @@ -0,0 +1,157 @@ +namespace CoreEx.Cosmos; + +/// +/// Provides standardized operations for Cosmos DB. +/// +/// This encapsulates common CRUD operations enabling consistent handling of reference data; the Cosmos DB equivalent of EfDbReferenceData. +/// The reference data is assumed to be stored within a container (typically shared by all reference data types, differentiated by ) using the default (no explicit) +/// partition key, as the identifier alone is used to get, update and delete the underlying document. Where an ambient transaction is active the mutations are enlisted +/// and only executed when the unit-of-work completes; in which case the resulting is not final until then (see ). +public static class CosmosDbReferenceData +{ + /// + /// Creates the specified in the container. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The . + /// The value to create. + /// The mapper. + /// The cancellation token. + /// The containing the created value. + /// The identifier is generated using and stored as a . A newly created value is always inactive. A duplicate Code (for the same type discriminator) results in a , but only where the container has a unique key policy covering /typeDiscriminator and /code; the container must also use (see ) so that the co-located outbox events do not themselves violate that unique key. + public static async Task>> CreateAsync(CosmosDbContainer container, TRef value, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + where TMapper : class, IBiDirectionMapper + { + container.ThrowIfNull(); + var model = mapper.To.Map(value)!; + + model.Id = (await Runtime.GenerateIdentifierAsync().ConfigureAwait(false))?.ToString() ?? throw new InvalidOperationException("A generated identifier must not be null."); + model.IsActive = false; + + var result = await container.CreateWithResultAsync(model, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Updates the specified in the container. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The . + /// The identifier of the value to update. + /// The value to update. + /// The mapper. + /// The cancellation token. + /// The containing the updated value. + /// The and active state are retained from the existing value. The optimistic concurrency check is performed using the of the . + public static async Task>> UpdateAsync(CosmosDbContainer container, TId id, TRef value, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + where TMapper : class, IBiDirectionMapper + { + container.ThrowIfNull(); + var existing = await container.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.AsResult(); + + var model = mapper.To.Map(value)!; + model.Id = existing.Value.Id; + model.Code = existing.Value.Code; // Code is immutable and cannot be updated. + model.IsActive = existing.Value.IsActive; // IsActive can only be updated via specific methods. + + var result = await container.UpdateWithResultAsync(model, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Activates the specified in the container. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The . + /// The identifier of the value to activate. + /// The mapper. + /// The cancellation token. + /// The containing the activated value. + public static Task>> ActivateAsync(CosmosDbContainer container, TId id, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + where TMapper : class, IBiDirectionMapper + => SetActiveAsync(container, id, true, mapper, cancellationToken); + + /// + /// Deactivates the specified in the container. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The . + /// The identifier of the value to deactivate. + /// The mapper. + /// The cancellation token. + /// The containing the deactivated value. + public static Task>> DeactivateAsync(CosmosDbContainer container, TId id, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + where TMapper : class, IBiDirectionMapper + => SetActiveAsync(container, id, false, mapper, cancellationToken); + + /// + /// Sets the active state of the specified ; no update is performed where already in the requested state. + /// + private static async Task>> SetActiveAsync(CosmosDbContainer container, TId id, bool isActive, TMapper mapper, CancellationToken cancellationToken) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + where TMapper : class, IBiDirectionMapper + { + container.ThrowIfNull(); + var existing = await container.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.AsResult(); + + if (existing.Value.IsActive == isActive) + return new DataResult(mapper.From.Map(existing.Value)!, false); + + existing.Value.IsActive = isActive; + var result = await container.UpdateWithResultAsync(existing.Value, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Deletes the specified from the container. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The identifier of the value to delete. + /// The cancellation token. + /// A + /// A delete is considered idempotent and as such no will be thrown. The returning is informational only. + public static async Task> DeleteAsync(CosmosDbContainer container, TId id, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : CosmosDbReferenceDataModelBase, IEntityKey, new() + { + container.ThrowIfNull(); + var existing = await container.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.IsNotFoundError ? DataResult.False : existing.AsResult(); + + if (existing.Value.IsActive) + return Result.Fail("An active reference data value cannot be deleted.", c => c.WithErrorCode("cannot-delete-active")); + + // Carry the ETag of the read above so that a concurrent change (e.g. activation) results in a precondition failure rather than deleting an item that is no longer inactive. + var args = container.Args with { ItemRequestOptions = new ItemRequestOptions { IfMatchEtag = existing.Value.ETag } }; + return await container.DeleteWithResultAsync(args, CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + } +} diff --git a/src/CoreEx.Cosmos/Extended/CosmosDbBatch.cs b/src/CoreEx.Cosmos/Extended/CosmosDbBatch.cs index 6659595c7..49125b6c2 100644 --- a/src/CoreEx.Cosmos/Extended/CosmosDbBatch.cs +++ b/src/CoreEx.Cosmos/Extended/CosmosDbBatch.cs @@ -18,22 +18,109 @@ public static class CosmosDbBatch /// Indicates whether the items are created sequentially (order-based and slower) rather than in parallel (no order guarantees, faster); defaults to . /// The . /// Each item is created individually and is not transactional - a partial failure part-way through leaves the already-created items in place. - public static async Task ImportBatchAsync(this Container container, JsonArray items, bool sequential = false, CancellationToken cancellationToken = default) + public static Task ImportBatchAsync(this Container container, JsonArray items, bool sequential = false, CancellationToken cancellationToken = default) { container.ThrowIfNull(); items.ThrowIfNull(); - var work = items.Where(n => n is not null).Select(n => container.CreateItemAsync(n, cancellationToken: cancellationToken)); + return CreateBatchAsync(container, items, sequential, cancellationToken); + } + + /// + /// Imports (creates) a batch of raw JSON , returning the counts (all created). + /// + internal static Task<(int Created, int Replaced)> ImportBatchCountAsync(this Container container, JsonArray items, CancellationToken cancellationToken) + => CreateBatchAsync(container, items, false, cancellationToken); + + /// + /// Creates each item, returning the counts (all created). + /// + private static Task<(int Created, int Replaced)> CreateBatchAsync(Container container, JsonArray items, bool sequential, CancellationToken cancellationToken) + => RunAsync(items, sequential, async n => + { + await container.CreateItemAsync(n, cancellationToken: cancellationToken).ConfigureAwait(false); + return true; + }); + + /// + /// Runs the for each non-null item; items are started lazily so is genuinely one-at-a-time. The returns where + /// the item was created (versus replaced). + /// + private static async Task<(int Created, int Replaced)> RunAsync(JsonArray items, bool sequential, Func> operation) + { + var work = items.Where(n => n is not null).Select(n => operation(n!)); + var results = new List(); if (sequential) { foreach (var task in work) { - await task.ConfigureAwait(false); + results.Add(await task.ConfigureAwait(false)); } } else - await Task.WhenAll(work).ConfigureAwait(false); + results.AddRange(await Task.WhenAll(work).ConfigureAwait(false)); + + var created = results.Count(x => x); + return (created, results.Count - created); + } + + /// + /// Merges (upserts) a batch of raw JSON into the ; unlike this is re-runnable + /// where each item has a stable id. + /// + internal static Task<(int Created, int Replaced)> MergeBatchAsync(this Container container, JsonArray items, bool sequential = false, CancellationToken cancellationToken = default) + => RunAsync(items, sequential, async n => (await container.UpsertItemAsync(n, cancellationToken: cancellationToken).ConfigureAwait(false)).StatusCode == HttpStatusCode.Created); + + /// + /// Merges (upserts) a batch of named items from the into the ; see . + /// + internal static async Task MergeBatchAsync(this Container container, JsonDataReader jsonDataReader, string path, bool sequential = false, CancellationToken cancellationToken = default) + { + if (!jsonDataReader.ThrowIfNull().TryCreateData(path.ThrowIfNullOrEmpty(), out var node) || node is not JsonArray array) + return false; + + await MergeBatchAsync(container, array, sequential, cancellationToken).ConfigureAwait(false); + return true; + } + + /// + /// Merges (upserts) a discriminated group of items. Seed items generate a new id on every parse so, to be re-runnable, an item's id is replaced by that of the existing item having the same + /// type discriminator and code (where one exists) before being upserted - this honors the unique key convention of a reference data container and keeps identifiers stable across runs. + /// + private static Task<(int Created, int Replaced)> MergeDiscriminatedItemsAsync(Container container, JsonDataReaderOptions options, JsonArray array, bool sequential, CancellationToken cancellationToken) + { + var typeProperty = options.ConvertPropertyName(nameof(ITypeDiscriminator.TypeDiscriminator))!; + var codeProperty = options.ConvertPropertyName(nameof(RefData.Abstractions.IReferenceData.Code))!; + var idProperty = options.ConvertPropertyName(nameof(IIdentifier.Id))!; + + return RunAsync(array, sequential, async n => + { + if (n is JsonObject jo && jo[typeProperty]?.GetValue() is string type && jo[codeProperty]?.GetValue() is string code + && await FindIdAsync(container, idProperty, typeProperty, codeProperty, type, code, cancellationToken).ConfigureAwait(false) is string existingId) + jo[idProperty] = existingId; + + return (await container.UpsertItemAsync(n, cancellationToken: cancellationToken).ConfigureAwait(false)).StatusCode == HttpStatusCode.Created; + }); + } + + /// + /// Finds the identifier of the existing item (across partitions) with the specified type discriminator and code. + /// + private static async Task FindIdAsync(Container container, string idProperty, string typeProperty, string codeProperty, string type, string code, CancellationToken cancellationToken) + { + // The property names originate from the naming convention (never external input) so are safe to embed; the values are parameterized. + var query = new QueryDefinition($"SELECT VALUE c[\"{idProperty}\"] FROM c WHERE c[\"{typeProperty}\"] = @type AND c[\"{codeProperty}\"] = @code").WithParameter("@type", type).WithParameter("@code", code); + using var iterator = container.GetItemQueryIterator(query); + while (iterator.HasMoreResults) + { + foreach (var id in await iterator.ReadNextAsync(cancellationToken).ConfigureAwait(false)) + { + return id; + } + } + + return null; } /// @@ -101,38 +188,86 @@ public static async Task ImportDiscriminatedBatchAsync(this Microsoft.Azure.Cosm database.ThrowIfNull(); jsonDataReader.ThrowIfNull(); - // RootNode is the raw, unsubstituted tree - only used here to discover the top-level container-id keys (and their child type-discriminator keys). Each is then re-resolved via - // TryCreateData so dynamic parameters (e.g. '^guid', '^1') are substituted the same way the explicit-path overload already does - walking RootNode's children directly would skip - // substitution entirely. + // RootNode is the raw, unsubstituted tree - only used here to discover the top-level container-id keys; each is then re-resolved (with substitution) by the per-container overload. if (jsonDataReader.RootNode is not JsonObject root) return; + foreach (var containerId in root.Select(kvp => kvp.Key).ToList()) + { + await ImportDiscriminatedBatchAsync(database.GetContainer(containerId), jsonDataReader, containerId, sequential, cancellationToken).ConfigureAwait(false); + } + } + + /// + /// Imports (creates) the discriminated data found at the top-level of the into the , treating each child + /// property name as an value. A child property name prefixed with '$' (the DbEx convention) is instead merged (upserted), making that group re-runnable; see remarks. + /// + /// The . + /// The . + /// The top-level property name whose array value is shaped as [{ Person: [...] }, { Organization: [...] }]. + /// Indicates whether the items are created sequentially rather than in parallel; defaults to . + /// The . + /// indicates that the was found and processed; otherwise, . + /// Each item is created individually and is not transactional - a partial failure part-way through leaves the already-created items in place. + /// A group name may be prefixed with '$' (merge) and/or '^' (generate the id where not specified), as per DbEx; a group without '^' does not generate an id so each item must specify its own. + /// For a '$'-prefixed group an item's id is replaced by that of the existing item with the same typeDiscriminator and code (where one exists) before being upserted, as seed identifiers are generated on each parse. + public static Task ImportDiscriminatedBatchAsync(this Container container, JsonDataReader jsonDataReader, string path, bool sequential = false, CancellationToken cancellationToken = default) + => ImportDiscriminatedBatchAsync(container, jsonDataReader, path, null, sequential, cancellationToken); + + /// + /// As but reports each group (type discriminator, merged, created count, replaced count) to the action. + /// + internal static Task ImportDiscriminatedBatchAsync(Container container, JsonDataReader jsonDataReader, string path, Action? report, bool sequential, CancellationToken cancellationToken) + => ImportDiscriminatedAsync(container, jsonDataReader, path, async (name, array, merge) => + { + var (created, replaced) = merge ? await MergeDiscriminatedItemsAsync(container, jsonDataReader.Options, array, sequential, cancellationToken).ConfigureAwait(false) : await CreateBatchAsync(container, array, sequential, cancellationToken).ConfigureAwait(false); + report?.Invoke(name, merge, created, replaced); + }, cancellationToken); + + /// + /// Stamps and resolves each discriminated group at then passes it (with its type discriminator name and whether to merge) to . + /// + private static async Task ImportDiscriminatedAsync(Container container, JsonDataReader jsonDataReader, string path, Func import, CancellationToken cancellationToken) + { + container.ThrowIfNull(); + jsonDataReader.ThrowIfNull(); + path.ThrowIfNullOrEmpty(); + + // RootNode is the raw, unsubstituted tree - only used here to discover the child type-discriminator keys. Each is then re-resolved via TryCreateData so dynamic parameters + // (e.g. '^guid', '^1') are substituted the same way the explicit-path overload already does - walking RootNode's children directly would skip substitution entirely. + if (jsonDataReader.RootNode is not JsonObject root || root[path] is not JsonArray containerArray) + return false; + var typeDiscriminatorProperty = jsonDataReader.Options.ConvertPropertyName(nameof(ITypeDiscriminator.TypeDiscriminator))!; var hadExistingTypeDiscriminatorProperty = jsonDataReader.Options.Properties.TryGetValue(typeDiscriminatorProperty, out var existingTypeDiscriminatorValue); + var idProperty = jsonDataReader.Options.ConvertPropertyName(nameof(IIdentifier.Id))!; + var hadExistingIdProperty = jsonDataReader.Options.Properties.TryGetValue(idProperty, out var existingIdValue); + try { - foreach (var containerId in root.Select(kvp => kvp.Key).ToList()) - { - if (root[containerId] is not JsonArray containerArray) - continue; - - var container = database.GetContainer(containerId); + // Only single-key objects (see also RootNodePreProcessor's identical convention for the '{ code: text }' shorthand) are treated as '$^TypeName'-style discriminator group markers - any + // other shape found in the same array (e.g. a flat, already-fully-formed document) is ignored here rather than silently misread as a bogus discriminator. + var discriminators = containerArray.OfType().Where(jo => jo.Count == 1).SelectMany(jo => jo.Select(kvp => kvp.Key)).Distinct().ToList(); - // Only single-key objects (see also RootNodePreProcessor's identical convention for the '{ code: text }' shorthand) are treated as '$^TypeName'-style discriminator group markers - any - // other shape found in the same array (e.g. a flat, already-fully-formed document) is ignored here rather than silently misread as a bogus discriminator. - var discriminators = containerArray.OfType().Where(jo => jo.Count == 1).SelectMany(jo => jo.Select(kvp => kvp.Key)).Distinct().ToList(); + foreach (var discriminator in discriminators) + { + // The discriminator key may be prefixed with '$' and/or '^' (as per DbEx) to signify additional behaviors; '$' indicates merge rather than insert, and '^' indicates that an identifier is generated where not + // specified. These are stripped before use as the actual property value in the resulting document. + var name = discriminator.TrimStart('$', '^'); + var prefix = discriminator[..^name.Length]; + jsonDataReader.Options.Properties[typeDiscriminatorProperty] = name; - foreach (var discriminator in discriminators) - { - // The discriminator key may be prefixed with '$' and/or '^' to signify additional behaviors as a JSON property name; strip these before use as the actual property value in the resulting document. - jsonDataReader.Options.Properties[typeDiscriminatorProperty] = discriminator.TrimStart('$', '^'); + // Without '^' no identifier is generated (even where the options would otherwise default one), so each item must specify its own; with '^' the options' identifier generation is used, else the default. + if (!prefix.Contains('^')) + jsonDataReader.Options.Properties.Remove(idProperty); + else if (!jsonDataReader.Options.Properties.ContainsKey(idProperty)) + jsonDataReader.Options.Properties[idProperty] = hadExistingIdProperty ? existingIdValue : "^id"; - if (!jsonDataReader.TryCreateData($"{containerId}.{discriminator}", out var node) || node is not JsonArray array) - continue; + if (!jsonDataReader.TryCreateData($"{path}.{discriminator}", out var node) || node is not JsonArray array) + continue; - await ImportBatchAsync(container, array, sequential, cancellationToken).ConfigureAwait(false); - } + await import(name, array, prefix.Contains('$')).ConfigureAwait(false); } } finally @@ -143,6 +278,13 @@ public static async Task ImportDiscriminatedBatchAsync(this Microsoft.Azure.Cosm jsonDataReader.Options.Properties[typeDiscriminatorProperty] = existingTypeDiscriminatorValue; else jsonDataReader.Options.Properties.Remove(typeDiscriminatorProperty); + + if (hadExistingIdProperty) + jsonDataReader.Options.Properties[idProperty] = existingIdValue; + else + jsonDataReader.Options.Properties.Remove(idProperty); } + + return true; } } diff --git a/src/CoreEx.Cosmos/Extended/CosmosDbInvoker.cs b/src/CoreEx.Cosmos/Extended/CosmosDbInvoker.cs index 78d17d14c..8346c4096 100644 --- a/src/CoreEx.Cosmos/Extended/CosmosDbInvoker.cs +++ b/src/CoreEx.Cosmos/Extended/CosmosDbInvoker.cs @@ -178,7 +178,7 @@ async Task DiscardAsync() { await DiscardAsync().ConfigureAwait(false); - if (tracer.Logger is not null && tracer.Logger.IsEnabled(LogLevel.Error)) + if (!(ex.IsCanceled() && cancellationToken.IsCancellationRequested) && tracer.Logger is not null && tracer.Logger.IsEnabled(LogLevel.Error)) tracer.Logger.LogError(ex, "Unit-of-work transaction discarded due to an unexpected error: {Error}", ex.Message); if (ExtendedException.TryConvertExceptionToResult(ex, out var result)) @@ -193,6 +193,9 @@ async Task DiscardAsync() // Retained (independent of the ambient scope, which is always cleared here) so IUnitOfWork.SynchronizeETag can resolve against it after this call returns. unitOfWork.LastTransaction = txn; unitOfWork.CosmosDb.UseTransaction(null); + + // The batch may have caused any number of server-side effects, so no snapshot read within this unit-of-work can be trusted afterwards. + unitOfWork.CosmosDb.ChangeTracker.Clear(); } } } diff --git a/src/CoreEx.Cosmos/GlobalUsing.cs b/src/CoreEx.Cosmos/GlobalUsing.cs index 246fb3e98..fd4a973d2 100644 --- a/src/CoreEx.Cosmos/GlobalUsing.cs +++ b/src/CoreEx.Cosmos/GlobalUsing.cs @@ -13,19 +13,23 @@ global using CoreEx.HealthChecks; global using CoreEx.Hosting; global using CoreEx.Invokers; +global using CoreEx.Json; global using CoreEx.Mapping; +global using CoreEx.Metadata; +global using CoreEx.RefData; global using CoreEx.Results; global using CoreEx.Results.Abstractions; global using CoreEx.Schemas; -global using Microsoft.Azure.Cosmos; -global using Microsoft.Azure.Cosmos.Linq; -// CoreEx.Data also declares a (differently-purposed, hash-partitioning) 'PartitionKey' type; alias to disambiguate in favour of the Cosmos SDK's struct throughout this package. -global using PartitionKey = Microsoft.Azure.Cosmos.PartitionKey; // CoreEx.Data also declares the shared base 'IMultiSetArgs' type; alias to disambiguate in favour of this package's discriminator-keyed variant throughout. global using IMultiSetArgs = CoreEx.Cosmos.Extended.IMultiSetArgs; +global using Microsoft.Azure.Cosmos; +global using Microsoft.Azure.Cosmos.Linq; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Diagnostics.HealthChecks; global using Microsoft.Extensions.Logging; +global using OpenTelemetry; +// CoreEx.Data also declares a (differently-purposed, hash-partitioning) 'PartitionKey' type; alias to disambiguate in favour of the Cosmos SDK's struct throughout this package. +global using PartitionKey = Microsoft.Azure.Cosmos.PartitionKey; global using Polly; global using System.Collections.Concurrent; global using System.Diagnostics.CodeAnalysis; diff --git a/src/CoreEx.Cosmos/ICosmosDb.cs b/src/CoreEx.Cosmos/ICosmosDb.cs index f939839e8..9b833215b 100644 --- a/src/CoreEx.Cosmos/ICosmosDb.cs +++ b/src/CoreEx.Cosmos/ICosmosDb.cs @@ -35,6 +35,13 @@ public interface ICosmosDb /// CosmosDbOptions Options { get; } + /// + /// Gets the scoped (snapshot cache of persisted documents read by point Get operations). + /// + /// Cleared automatically when the root transaction ends; use to do so explicitly (e.g. long-lived scopes or tests), or + /// to opt a single call out. + CosmosDbChangeTracker ChangeTracker { get; } + /// /// Gets the ambient ("current") for an active , where one is in scope; otherwise, . /// diff --git a/src/CoreEx.Cosmos/Outbox/CoreExCosmosExtensions.Outbox.DependencyInjection.cs b/src/CoreEx.Cosmos/Outbox/CoreExCosmosExtensions.Outbox.DependencyInjection.cs index 9be19d570..fff7d6771 100644 --- a/src/CoreEx.Cosmos/Outbox/CoreExCosmosExtensions.Outbox.DependencyInjection.cs +++ b/src/CoreEx.Cosmos/Outbox/CoreExCosmosExtensions.Outbox.DependencyInjection.cs @@ -17,7 +17,8 @@ public static class CoreExCosmosOutboxExtensions /// The number of hosted services to start to enable concurrency of processing for this (each gets its own Change Feed Processor instance, sharing /// the same lease container for coordination). Where not specified, attempts to get the value from configuration using 'CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:ServicesCount' as the /// key (namespaced by so different containers can have independent concurrency); otherwise, defaults to '4'. - /// The lease identifier; where not specified, defaults to "{containerId}-leases". + /// The lease identifier; where not specified, defaults to (a single container shared by all relays). The relay never creates it - it must + /// already exist (see CosmosDbProvisionArgs.OutboxLeaseContainer), otherwise starting the relay fails. /// The keyed singleton and health check key prefix, and the basis for each instance's distinct Change Feed Processor instance name; where not specified, defaults to /// "cosmos-outbox-relay-{containerId}-". /// An optional action to configure each instance before its is built. @@ -41,7 +42,7 @@ public static IHostApplicationBuilder AddCosmosDbOutboxRelayHostedService(this I servicesCount ??= CoreEx.Abstractions.Internal.GetConfigurationValue($"CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:ServicesCount", 4, builder.Configuration); servicesCount.ThrowWhen(c => c <= 0 || c > 32); - leaseContainerId ??= $"{containerId}-leases"; + leaseContainerId ??= CosmosDbOutboxRelayOptions.DefaultLeaseContainerId; serviceKeyPrefix ??= $"cosmos-outbox-relay-{containerId}-"; for (var i = 0; i < servicesCount; i++) diff --git a/src/CoreEx.Cosmos/Outbox/CosmosDbEventPublisher.cs b/src/CoreEx.Cosmos/Outbox/CosmosDbEventPublisher.cs index 45372fb60..d078fce2a 100644 --- a/src/CoreEx.Cosmos/Outbox/CosmosDbEventPublisher.cs +++ b/src/CoreEx.Cosmos/Outbox/CosmosDbEventPublisher.cs @@ -46,9 +46,13 @@ public class CosmosDbEventPublisher(ICosmosDb cosmosDb, IDestinationProvider? de // directly with a different [JsonPropertyName] to match a container whose real path is "/tenantId" would silently produce an outbox document with no value at that path, and TransactionalBatch // (which requires every enlisted operation to agree on the exact same partition key) would then fail with an undiagnosable BadRequest. Rather than let that happen silently, the first outbox publish // against a given container validates (and thereafter caches, since a container's partition-key path is immutable for its lifetime) that its actual path is "/partitionKey", failing fast with a clear, - // actionable exception otherwise. Keyed by (Database.Id, Container.Id) rather than the CosmosDb instance, since this reflects a physical, permanent fact about the container itself, safely shared - // process-wide regardless of how many CosmosDb/CosmosDbEventPublisher instances (e.g. one per request) ever touch it. - private static readonly ConcurrentDictionary<(string DatabaseId, string ContainerId), bool> _validatedOutboxContainers = new(); + // actionable exception otherwise. Keyed by (AccountEndpoint, Database.Id, Container.Id) rather than the CosmosDb instance, since this reflects a physical, permanent fact about the container itself, safely shared + // process-wide regardless of how many CosmosDb/CosmosDbEventPublisher instances (e.g. one per request) ever touch it. The key includes the account endpoint because database and container + // identifiers are only unique within a Cosmos account. + // The container's unique key policy (also immutable) is cached alongside, as the paths of each unique key, to support the one-off check that outbox events populate it (see ValidateUniqueKeys). + private static readonly ConcurrentDictionary<(string AccountEndpoint, string DatabaseId, string ContainerId), string[][]> _validatedOutboxContainers = new(); + + private static readonly ConcurrentDictionary<(string AccountEndpoint, string DatabaseId, string ContainerId), bool> _validatedUniqueKeys = new(); /// /// Thrown where there is no active @@ -65,7 +69,10 @@ protected override async Task OnPublishAsync(DestinationEvent[] events, Cancella throw new InvalidOperationException($"{nameof(CosmosDbEventPublisher)} requires at least one business mutation to already be enlisted in the current unit-of-work; an outbox event document has no container/partition key to bind to otherwise."); var container = txn.BoundContainer!; - await EnsureOutboxPartitionKeyPathAsync(container, cancellationToken).ConfigureAwait(false); + var accountEndpoint = CosmosDb.Client.Endpoint.AbsoluteUri; + var uniqueKeys = await EnsureOutboxPartitionKeyPathAsync(container, accountEndpoint, cancellationToken).ConfigureAwait(false); + + CosmosDb.Options.TryGetContainerOptions(container.Id, out var containerOptions); // BoundPartitionKeyValue is null where the enlisted business mutation's own partition key resolved to PartitionKey.None - a real, valid single logical partition (not an error; the simplest // possible container shape), so the outbox event document is co-located there too, exactly the same as any other partition key value. @@ -83,6 +90,10 @@ protected override async Task OnPublishAsync(DestinationEvent[] events, Cancella TimeToLive = OutboxTimeToLiveSeconds }; + var serializerOptions = CosmosDb.Client.ClientOptions.UseSystemTextJsonSerializerWithOptions; + containerOptions?.ApplyOutboxEventUpdater(outboxEvent, serializerOptions); + ValidateUniqueKeys(container, accountEndpoint, uniqueKeys, outboxEvent, serializerOptions); + txn.Enlist(container, partitionKey, partitionKeyValue, CompositeKey.Create(outboxEvent.Id), b => b.CreateItem(outboxEvent)); } } @@ -91,11 +102,11 @@ protected override async Task OnPublishAsync(DestinationEvent[] events, Cancella /// Validates (once per container, then caches the result for the remaining process lifetime) that 's actual, physical partition-key path is /partitionKey - /// see the remarks on / for why this matters and why caching is safe. /// - private static async Task EnsureOutboxPartitionKeyPathAsync(Container container, CancellationToken cancellationToken) + private static async Task EnsureOutboxPartitionKeyPathAsync(Container container, string accountEndpoint, CancellationToken cancellationToken) { - var key = (container.Database.Id, container.Id); - if (_validatedOutboxContainers.ContainsKey(key)) - return; + var key = (accountEndpoint, container.Database.Id, container.Id); + if (_validatedOutboxContainers.TryGetValue(key, out var cached)) + return cached; var response = await container.ReadContainerAsync(cancellationToken: cancellationToken).ConfigureAwait(false); var paths = response.Resource.PartitionKeyPaths; @@ -109,6 +120,46 @@ private static async Task EnsureOutboxPartitionKeyPathAsync(Container container, $"partition key path, or do not use {nameof(CosmosDbEventPublisher)}/{nameof(CosmosDbUnitOfWork)} against this container."); } - _validatedOutboxContainers[key] = true; + var uniqueKeys = response.Resource.UniqueKeyPolicy?.UniqueKeys.Select(uk => uk.Paths.ToArray()).ToArray() ?? []; + _validatedOutboxContainers[key] = uniqueKeys; + return uniqueKeys; + } + + /// + /// Validates (once per container) that the outbox event populates at least one path of every unique key defined on the container. Cosmos DB treats a missing unique key path as null, so an + /// outbox event populating none of them would share the (null, ...) tuple with every other outbox event, and the second would fail the whole TransactionalBatch with a 409 that surfaces + /// as a (misleading) on the business mutation. Failing fast here is far easier to diagnose. See . + /// + private static void ValidateUniqueKeys(Container container, string accountEndpoint, string[][] uniqueKeys, CosmosDbOutboxEvent outboxEvent, JsonSerializerOptions? serializerOptions) + { + if (uniqueKeys.Length == 0) + return; + + var key = (accountEndpoint, container.Database.Id, container.Id); + if (_validatedUniqueKeys.ContainsKey(key)) + return; + + var json = JsonSerializer.SerializeToElement(outboxEvent, serializerOptions ?? Json.JsonDefaults.SerializerOptions); + foreach (var paths in uniqueKeys) + { + if (!paths.Any(path => HasValue(json, path))) + throw new InvalidOperationException( + $"Container '{container.Id}' has a unique key policy ({string.Join(", ", paths)}) that no outbox event document populates. Cosmos DB treats a missing unique key path as null, so every outbox event " + + $"would collide on the same key and the second would fail the unit-of-work with a duplicate (409) error. Configure the container using {nameof(CosmosDbOptions)}.{nameof(CosmosDbOptions.Container)}(...) with " + + $"{nameof(CosmosDbContainerOptions.WithReferenceDataOutboxEvent)} (or {nameof(CosmosDbContainerOptions.WithOutboxEventUpdater)}) so that each outbox event carries a unique value for at least one of the paths."); + } + + _validatedUniqueKeys[key] = true; + } + + private static bool HasValue(JsonElement element, string path) + { + foreach (var segment in path.Split('/', StringSplitOptions.RemoveEmptyEntries)) + { + if (element.ValueKind != JsonValueKind.Object || !element.TryGetProperty(segment, out element)) + return false; + } + + return element.ValueKind is not (JsonValueKind.Null or JsonValueKind.Undefined); } } diff --git a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxEvent.cs b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxEvent.cs index 6f66c7cb2..3bfcee119 100644 --- a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxEvent.cs +++ b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxEvent.cs @@ -46,4 +46,10 @@ public sealed class CosmosDbOutboxEvent : IIdentifier, IPartitionKey, IT /// [JsonPropertyName("ttl")] public int? TimeToLive { get; set; } + + /// + /// Gets or sets the additional properties bag. + /// + [JsonExtensionData] + public Dictionary? ExtensionData { get; set; } } diff --git a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelay.cs b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelay.cs index c8cfca2bb..d8de90995 100644 --- a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelay.cs +++ b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelay.cs @@ -9,11 +9,10 @@ namespace CoreEx.Cosmos.Outbox; /// Constructed with the raw SDK rather than deliberately - is registered scoped, and an instance of this class /// is built once and lives for the process lifetime, so capturing a scoped service here would be a captive-dependency bug. The proxy, like a /// or , is stable and safe to hold long-term. -/// The lease container () is auto-provisioned (via , -/// partitioned on /id - the Change Feed Processor's own lease-document convention) the first time is called; a fresh Cosmos DB database would otherwise have no -/// lease container yet, and throws attempting to acquire leases against a container that does not exist. Concurrent hosted-service instances sharing the same -/// (see 's servicesCount) each call this -/// independently at their own startup; CreateContainerIfNotExistsAsync is idempotent/safe for this, so no additional coordination is required. +/// The lease container (, partitioned on /id - the Change Feed Processor's own lease-document convention) is never created by the relay: creating +/// a container is a control-plane operation that a production host identity (e.g. Entra ID data-plane RBAC) is not expected to be permitted. It must be provisioned up front (see CosmosDbProvisionArgs.OutboxLeaseContainer, +/// or infrastructure-as-code); fails fast with an where it does not exist (reading its metadata is permitted by data-plane roles). +/// Concurrent hosted-service instances (see 's servicesCount) share the same lease container for coordination. public sealed class CosmosDbOutboxRelay : IAsyncDisposable { #if NET9_0_OR_GREATER @@ -22,8 +21,10 @@ public sealed class CosmosDbOutboxRelay : IAsyncDisposable private readonly object _syncLock = new(); #endif private readonly SemaphoreSlim _semaphore = new(1, 1); + private readonly CancellationTokenSource _recoveryCts = new(); private readonly Database _database; - private readonly ChangeFeedProcessor _processor; + private ChangeFeedProcessor _processor; + private int _recovering; private bool _disposed; /// @@ -41,24 +42,29 @@ public CosmosDbOutboxRelay(Database database, CosmosDbOutboxRelayOptions options Resiliency = options.Resiliency ?? CosmosDbOutboxRelayResiliency.CreateRelayCircuitBreakerResiliency(); _database = database.ThrowIfNull(); - var container = database.GetContainer(options.ContainerId); - var leaseContainer = database.GetContainer(options.LeaseContainerId); + _processor = BuildProcessor(); + } - var builder = container.GetChangeFeedProcessorBuilder($"outbox-relay-{options.ContainerId}", OnChangesAsync) - .WithInstanceName(options.InstanceName) - .WithLeaseContainer(leaseContainer) + /// + /// Builds a new from the . + /// + private ChangeFeedProcessor BuildProcessor() + { + var builder = _database.GetContainer(Options.ContainerId).GetChangeFeedProcessorBuilder($"outbox-relay-{Options.ContainerId}", OnChangesAsync) + .WithInstanceName(Options.InstanceName) + .WithLeaseContainer(_database.GetContainer(Options.LeaseContainerId)) .WithErrorNotification(OnErrorNotificationAsync); - if (options.PollInterval is not null) - builder = builder.WithPollInterval(options.PollInterval.Value); + if (Options.PollInterval is not null) + builder = builder.WithPollInterval(Options.PollInterval.Value); - if (options.BatchSize is not null) - builder = builder.WithMaxItems(options.BatchSize.Value); + if (Options.BatchSize is not null) + builder = builder.WithMaxItems(Options.BatchSize.Value); - if (options.StartTime is not null) - builder = builder.WithStartTime(options.StartTime.Value); + // The SDK defaults fresh leases to "now", which would skip an existing outbox backlog. + builder = builder.WithStartTime(Options.StartTime ?? DateTime.SpecifyKind(DateTime.MinValue, DateTimeKind.Utc)); - _processor = builder.Build(); + return builder.Build(); } /// @@ -95,7 +101,7 @@ public CosmosDbOutboxRelay(Database database, CosmosDbOutboxRelayOptions options /// Starts the underlying . /// /// The . - /// Provisions (partitioned on /id) first, where it does not already exist - see this type's own remarks. + /// The container does not exist - see this type's own remarks. public async Task StartAsync(CancellationToken cancellationToken = default) { await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); @@ -104,8 +110,11 @@ public async Task StartAsync(CancellationToken cancellationToken = default) if (!Status.CanStart) return; + // SDK background tasks inherit this scope; publishing explicitly restores instrumentation in the batch processor. + using var instrumentation = SuppressInstrumentationScope.Begin(!Options.IsInstrumentationEnabledForPolling); + await EnsureLeaseContainerExistsAsync(cancellationToken).ConfigureAwait(false); + LogStatusChange(Status = ServiceStatus.Starting); - await _database.CreateContainerIfNotExistsAsync(Options.LeaseContainerId, "/id", Options.LeaseContainerThroughput, cancellationToken: cancellationToken).ConfigureAwait(false); await _processor.StartAsync().ConfigureAwait(false); LogStatusChange(Status = ServiceStatus.Running); } @@ -115,6 +124,22 @@ public async Task StartAsync(CancellationToken cancellationToken = default) } } + /// + /// Verifies that the lease container exists (a metadata read; the relay never creates it). + /// + private async Task EnsureLeaseContainerExistsAsync(CancellationToken cancellationToken) + { + try + { + await _database.GetContainer(Options.LeaseContainerId).ReadContainerAsync(cancellationToken: cancellationToken).ConfigureAwait(false); + } + catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) + { + throw new InvalidOperationException($"The outbox relay lease container '{Options.LeaseContainerId}' does not exist in database '{_database.Id}'; the relay never creates it. " + + $"Provision it up front (partition key '/id') - e.g. 'CosmosDbProvisionArgs.OutboxLeaseContainer()' in the Database project, or infrastructure-as-code.", cex); + } + } + /// /// Pauses the underlying (via - there is no dedicated pause API; validated empirically that stopping then later starting the /// same processor instance resumes correctly from its last checkpoint). @@ -129,6 +154,7 @@ public async Task PauseAsync(string reason, CancellationToken cancellationToken if (!Status.CanPause) return; + using var instrumentation = SuppressInstrumentationScope.Begin(!Options.IsInstrumentationEnabledForPolling); StatusReason = reason; LogStatusChange(Status = ServiceStatus.Pausing); await _processor.StopAsync().ConfigureAwait(false); @@ -152,6 +178,7 @@ public async Task ResumeAsync(CancellationToken cancellationToken = default) if (!Status.CanResume) return; + using var instrumentation = SuppressInstrumentationScope.Begin(!Options.IsInstrumentationEnabledForPolling); LogStatusChange(Status = ServiceStatus.Resuming); await _processor.StartAsync().ConfigureAwait(false); LogStatusChange(Status = ServiceStatus.Running); @@ -171,6 +198,7 @@ public async Task StopAsync(CancellationToken cancellationToken = default) await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); try { + using var instrumentation = SuppressInstrumentationScope.Begin(!Options.IsInstrumentationEnabledForPolling); var wasInitializing = Status.IsInitializing; LogStatusChange(Status = ServiceStatus.Stopping); @@ -220,14 +248,108 @@ private async Task OnChangesAsync(IReadOnlyCollection chang /// /// Handles a Change Feed Processor infrastructure-level error notification (e.g. lease acquisition issues) - distinct from a batch exception, which is handled by . /// + /// Where the error indicates the monitored and/or lease container no longer exists (a anywhere in the exception chain - e.g. a provisioning reset deleted and + /// recreated the containers underneath the running relay), the processor is bound to the deleted container resource identifiers and would never recover on its own; a background recovery is therefore + /// triggered (see ). private Task OnErrorNotificationAsync(string leaseToken, Exception error) { if (Logger.IsEnabled(LogLevel.Warning)) Logger.LogWarning(error, "Cosmos DB change feed processor error for container '{ContainerId}', lease '{LeaseToken}': {Error}", Options.ContainerId, leaseToken, error.Message); + // Never stop/rebuild the processor from within its own callback; recover on a separate task (at most one at a time). + if (IsContainerNotFound(error) && !_recoveryCts.IsCancellationRequested && Interlocked.CompareExchange(ref _recovering, 1, 0) == 0) + _ = Task.Run(RecoverAsync); + return Task.CompletedTask; } + /// + /// Determines whether the chain contains a with a status code. + /// + private static bool IsContainerNotFound(Exception? error) + { + while (error is not null) + { + if (error is CosmosException { StatusCode: HttpStatusCode.NotFound }) + return true; + + if (error is AggregateException aex && aex.InnerExceptions.Any(IsContainerNotFound)) + return true; + + error = error.InnerException; + } + + return false; + } + + /// + /// Recovers from the monitored and/or lease container having been deleted and recreated by stopping the existing (stale) and starting a newly built one; retried (after + /// ) until successful, or until the relay is no longer running or is stopped. + /// + /// The new processor starts against a new (empty) lease set and as such (unless is specified) reads the change feed from the beginning; this is + /// safe as the latest-version change feed only returns items that still exist, and relayed outbox documents are deleted (at-least-once delivery semantics are unaffected). + private async Task RecoverAsync() + { + var cancellationToken = _recoveryCts.Token; + try + { + while (!cancellationToken.IsCancellationRequested) + { + await Task.Delay(Options.RecoveryDelay, cancellationToken).ConfigureAwait(false); + + // Acquire per attempt (never across the delay) so a concurrent stop/dispose is not blocked. + await _semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + if (Status != ServiceStatus.Running) + return; + + using var instrumentation = SuppressInstrumentationScope.Begin(!Options.IsInstrumentationEnabledForPolling); + + try + { + await _processor.StopAsync().ConfigureAwait(false); + } + catch (Exception ex) when (!ex.IsCanceledBy(cancellationToken)) + { + if (Logger.IsEnabled(LogLevel.Debug)) + Logger.LogDebug(ex, "Cosmos DB outbox relay for container '{ContainerId}': stopping the stale change feed processor failed (ignored): {Error}", Options.ContainerId, ex.Message); + } + + try + { + await EnsureLeaseContainerExistsAsync(cancellationToken).ConfigureAwait(false); + await _database.GetContainer(Options.ContainerId).ReadContainerAsync(cancellationToken: cancellationToken).ConfigureAwait(false); + + var processor = BuildProcessor(); + await processor.StartAsync().ConfigureAwait(false); + _processor = processor; + + if (Logger.IsEnabled(LogLevel.Warning)) + Logger.LogWarning("Cosmos DB outbox relay for container '{ContainerId}' recovered: the monitored and/or lease container was recreated whilst running; the change feed processor has been restarted.", Options.ContainerId); + + return; + } + catch (Exception ex) when (!ex.IsCanceledBy(cancellationToken)) + { + if (Logger.IsEnabled(LogLevel.Warning)) + Logger.LogWarning(ex, "Cosmos DB outbox relay for container '{ContainerId}' recovery failed; will retry in {RecoveryDelay}: {Error}", Options.ContainerId, Options.RecoveryDelay, ex.Message); + } + } + finally + { + _semaphore.Release(); + } + } + } + catch (Exception ex) when (ex.IsCanceledBy(cancellationToken)) { } + catch (ObjectDisposedException) { } + finally + { + Interlocked.Exchange(ref _recovering, 0); + } + } + /// /// Logs the status change. /// @@ -254,8 +376,10 @@ public async ValueTask DisposeAsync() return; _disposed = true; + _recoveryCts.Cancel(); await StopAsync().ConfigureAwait(false); _semaphore.Dispose(); + _recoveryCts.Dispose(); GC.SuppressFinalize(this); } } diff --git a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayOptions.cs b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayOptions.cs index e344a941c..4067dc7b9 100644 --- a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayOptions.cs +++ b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayOptions.cs @@ -13,14 +13,13 @@ public sealed class CosmosDbOutboxRelayOptions /// /// Gets or sets the lease identifier. /// + /// The container must already exist (partitioned on /id); the relay never creates it. See CosmosDbProvisionArgs.OutboxLeaseContainer. public required string LeaseContainerId { get; init; } /// - /// Gets or sets the manually provisioned throughput (RU/s) to request when the container does not already exist and is auto-provisioned by ; - /// where not specified, no throughput is requested (the account/database default applies - e.g. serverless, or a database-level shared throughput). + /// The default lease identifier ($outbox-leases); a single container shared by all outbox relays (processors are uniquely named per monitored container). /// - /// Only consulted the first time a given is auto-provisioned - see 's constructor remarks. - public int? LeaseContainerThroughput { get; set; } + public const string DefaultLeaseContainerId = "$outbox-leases"; /// /// Gets or sets the Change Feed Processor instance name; must be distinct per concurrently-running instance for the same / pair. @@ -32,6 +31,12 @@ public sealed class CosmosDbOutboxRelayOptions /// public TimeSpan? PollInterval { get; set; } + /// + /// Gets or sets whether instrumentation is enabled for change-feed polling and lease maintenance. + /// + /// Defaults to to suppress frequent background SDK and HTTP spans, matching the relational outbox relays. Applied when the processor starts or resumes. + public bool IsInstrumentationEnabledForPolling { get; set; } + /// /// Gets or sets the maximum number of items returned per batch; where not specified, the Change Feed Processor default applies. /// @@ -40,11 +45,17 @@ public sealed class CosmosDbOutboxRelayOptions public int? BatchSize { get; set; } /// - /// Gets or sets the start time; where not specified, the Change Feed Processor's own default applies (confirmed empirically to mean "from the beginning" for a brand-new lease with no prior checkpoint, so - /// a first-ever relay startup does not silently skip a pre-existing outbox backlog). + /// Gets or sets the start time; where not specified, the relay explicitly reads from the beginning for a brand-new lease with no prior checkpoint so a first-ever startup or container-reset recovery + /// does not skip a pre-existing outbox backlog. Existing lease checkpoints take precedence; an explicit start time intentionally excludes earlier events on a fresh lease. /// public DateTime? StartTime { get; set; } + /// + /// Gets or sets the delay before (and between) attempts to recover the Change Feed Processor where the monitored and/or lease container was deleted and recreated whilst the relay was running (e.g. a + /// provisioning reset). Defaults to 5 seconds. + /// + public TimeSpan RecoveryDelay { get; set; } = TimeSpan.FromSeconds(5); + /// /// Gets or sets the used to protect batch execution with a self-pausing/self-resuming circuit breaker; where not specified, defaults to /// 's own defaults. diff --git a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayProcessor.cs b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayProcessor.cs index 0d557203d..854724285 100644 --- a/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayProcessor.cs +++ b/src/CoreEx.Cosmos/Outbox/CosmosDbOutboxRelayProcessor.cs @@ -34,6 +34,12 @@ public class CosmosDbOutboxRelayProcessor(IServiceProvider serviceProvider, stri /// protected CosmosDbOutboxRelayInvoker Invoker { get; } = invoker ?? CosmosDbOutboxRelayInvoker.Default; + /// + /// Gets or sets whether instrumentation is enabled for publishing. + /// + /// Defaults to to retain delivery spans and originating-trace relay markers, matching the relational outbox relays. + public bool IsInstrumentationEnabledForPublishing { get; set; } = true; + /// /// Processes a single batch of changes as delivered by the Change Feed Processor. /// @@ -74,6 +80,7 @@ public virtual async Task ProcessBatchAsync(IReadOnlyCollection { if (tracer.Activity is not null) @@ -98,7 +105,13 @@ await Invoker.InvokeAsync(this, async (tracer, ct) => { CosmosMetrics.OutboxRelayPublishFailed.Add(outboxDocs.Count, tag); RecordLagMetrics(eventPublisher); - if (Logger.IsEnabled(LogLevel.Error)) + // A cancellation requested by the caller (e.g. the Change Feed Processor stopping) is expected; the batch is redelivered, so it is not logged as an error. + if (ex.IsCanceledBy(cancellationToken)) + { + if (Logger.IsEnabled(LogLevel.Debug)) + Logger.LogDebug("Publish of {Count} outbox event(s) for container '{ContainerId}' was canceled; the batch will be redelivered.", outboxDocs.Count, ContainerId); + } + else if (Logger.IsEnabled(LogLevel.Error)) Logger.LogError(ex, "Failed to publish {Count} outbox event(s) for container '{ContainerId}': {Error}", outboxDocs.Count, ContainerId, ex.Message); throw; diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbClientFactory.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbClientFactory.cs new file mode 100644 index 000000000..3b267ff2e --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbClientFactory.cs @@ -0,0 +1,44 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Provides a factory suitable for provisioning/tooling and testing, particularly against the local Cosmos DB emulator. +/// +public static class CosmosDbClientFactory +{ + /// + /// Creates a from the using the System.Text.Json serializer (camelCase), as required by CoreEx.Cosmos. + /// + /// The connection string. + /// An optional action to further configure the . + /// The . + /// Where the connection string targets a local endpoint (i.e. the emulator; localhost, 127.0.0.1 or [::1]) is used and the emulator's self-signed + /// certificate is accepted; this is never applied to a non-local endpoint. + public static CosmosClient Create(string connectionString, Action? configure = null) + { + var options = new CosmosClientOptions + { + UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase } + }; + + if (IsLocalEndpoint(connectionString.ThrowIfNullOrEmpty())) + { + options.ConnectionMode = ConnectionMode.Gateway; + options.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); + } + + configure?.Invoke(options); + return new CosmosClient(connectionString, options); + } + + /// + /// Determines whether the connection string AccountEndpoint is a local endpoint. + /// + private static bool IsLocalEndpoint(string connectionString) + { + var endpoint = connectionString.Split(';', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries) + .Select(x => x.Split('=', 2)) + .FirstOrDefault(x => x.Length == 2 && string.Equals(x[0].Trim(), "AccountEndpoint", StringComparison.OrdinalIgnoreCase))?[1]; + + return Uri.TryCreate(endpoint, UriKind.Absolute, out var uri) && (uri.IsLoopback || string.Equals(uri.Host, "localhost", StringComparison.OrdinalIgnoreCase)); + } +} diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbConsole.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbConsole.cs new file mode 100644 index 000000000..0668ea85b --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbConsole.cs @@ -0,0 +1,308 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Represents the Cosmos DB provisioning console; the Cosmos DB equivalent of the DbEx migration console (e.g. PostgresMigrationConsole). +/// +/// Usage: dotnet run -- [options] <command> where command is one (or a comma-separated combination) of the names; use --help to list the options. +/// The destructive and commands require confirmation unless --accept-prompts is specified. +public sealed class CosmosDbConsole +{ + private CosmosDbConsole(string connectionString, string databaseId, Assembly assembly) + { + ConnectionString = connectionString.ThrowIfNullOrEmpty(); + Args = new CosmosDbProvisionArgs { DatabaseId = databaseId.ThrowIfNullOrEmpty() }.AddAssembly(assembly); + } + + /// + /// Creates a new . + /// + /// The used to infer the containing the embedded Data resources. + /// The default connection string; can be overridden using -cs or -cv. + /// The default ; can be overridden using -d. + public static CosmosDbConsole Create(string connectionString, string databaseId) => new(connectionString, databaseId, typeof(TProgram).Assembly); + + /// + /// Gets or sets the connection string. + /// + public string ConnectionString { get; set; } + + /// + /// Gets the . + /// + public CosmosDbProvisionArgs Args { get; } + + /// + /// Configures the . + /// + /// The configuration action. + /// The to support fluent-style method-chaining. + public CosmosDbConsole Configure(Action configure) + { + configure.ThrowIfNull()(this); + return this; + } + + /// + /// Runs the console using the command-line . + /// + /// The command-line arguments. + /// The status code whereby zero indicates success. + public async Task RunAsync(string[] args) + { + var name = Assembly.GetEntryAssembly()?.GetName().Name ?? "CosmosDb"; + var command = CosmosDbProvisionCommand.None; + + for (var i = 0; i < args.Length; i++) + { + var arg = args[i]; + switch (arg.ToLowerInvariant()) + { + case "-?" or "-h" or "--help": + WriteHelp(name); + return 0; + + case "-cs" or "--connection-string" when i + 1 < args.Length: + ConnectionString = args[++i]; + break; + + case "-cv" or "--connection-varname" when i + 1 < args.Length: + var varName = args[++i]; + var value = Environment.GetEnvironmentVariable(varName); + if (string.IsNullOrEmpty(value)) + return Fail($"Environment variable '{varName}' (--connection-varname) does not exist or has no value."); + + ConnectionString = value; + break; + + case "-d" or "--database" when i + 1 < args.Length: + Args.DatabaseId = args[++i]; + break; + + case "-p" or "--param" when i + 1 < args.Length: + var param = args[++i]; + var index = param.IndexOf('='); + if (index <= 0) + return Fail($"Parameter '{param}' is invalid; must be expressed as a 'Name=Value' pair."); + + Args.Parameters[param[..index].Trim()] = param[(index + 1)..]; + break; + + case "--accept-prompts": + Args.AcceptPrompts = true; + break; + + default: + if (arg.Length > 0 && !arg.StartsWith('-') && !char.IsDigit(arg[0]) && Enum.TryParse(arg, true, out var cmd)) + command |= cmd; + else + return Fail($"Unrecognized or incomplete argument: {arg}", name); + + break; + } + } + + if (command == CosmosDbProvisionCommand.None) + { + WriteHelp(name); + return 1; + } + + Args.Output = Console.Out; + var sw = System.Diagnostics.Stopwatch.StartNew(); + + try + { + WriteBanner(name, command); + + if (!Confirm(command)) + return 1; + + using var client = CosmosDbClientFactory.Create(ConnectionString); + await new CosmosDbProvisioner(client, Args).RunAsync(command).ConfigureAwait(false); + Console.Out.WriteLine($"{name} Complete. [{sw.Elapsed.TotalMilliseconds:0.0}ms]"); + Console.Out.WriteLine(); + return 0; + } + catch (Exception ex) + { + Console.Error.WriteLine($"Failed: {ex.Message}"); + return 1; + } + } + + /// + /// Writes the error and (optionally) the help; returns the failure status code. + /// + private static int Fail(string message, string? helpName = null) + { + Console.Error.WriteLine(message); + if (helpName is not null) + WriteHelp(helpName); + + return 1; + } + + /// + /// Confirms (unless ) the destructive commands. + /// + private bool Confirm(CosmosDbProvisionCommand command) + { + if (Args.AcceptPrompts) + return true; + + if (command.HasFlag(CosmosDbProvisionCommand.Drop)) + { + if (!GetYesNo("DROP: Confirm that where the specified database already exists it should be dropped?")) + { + Console.Error.WriteLine("Database drop was not confirmed; no execution occurred."); + return false; + } + } + else if (command.HasFlag(CosmosDbProvisionCommand.Reset)) + { + if (!GetYesNo("RESET: Confirm that where the declared containers already exist they should be deleted and recreated (all existing data and container settings will be lost)?")) + { + Console.Error.WriteLine("Container reset was not confirmed; no execution occurred."); + return false; + } + } + + return true; + } + + /// + /// Prompts for a yes/no response; defaults to no. + /// + private static bool GetYesNo(string prompt) + { + var color = Console.ForegroundColor; + Console.ForegroundColor = ConsoleColor.Yellow; + Console.Out.Write($"{prompt} [y/N] "); + Console.ForegroundColor = color; + + var response = Console.In.ReadLine()?.Trim(); + Console.Out.WriteLine(); + return response is not null && (response.Equals("y", StringComparison.OrdinalIgnoreCase) || response.Equals("yes", StringComparison.OrdinalIgnoreCase)); + } + + /// + /// Writes the logo. + /// + private static void WriteLogo() + { + Console.Out.WriteLine("╔═╗┌─┐┬─┐┌─┐╔═╗─┐ ┬ ╔═╗┌─┐┌─┐┌┬┐┌─┐┌─┐ ╔╦╗╔╗ "); + Console.Out.WriteLine("║ │ │├┬┘├┤ ║╣ ┌┴┬┘ ║ │ │└─┐││││ │└─┐ ║║╠╩╗"); + Console.Out.WriteLine("╚═╝└─┘┴└─└─┘╚═╝┴ └─ ╚═╝└─┘└─┘┴ ┴└─┘└─┘ ═╩╝╚═╝"); + Console.Out.WriteLine(); + } + + /// + /// Writes the banner and settings; the connection string is never written as it contains the account key. + /// + private void WriteBanner(string name, CosmosDbProvisionCommand command) + { + var o = Console.Out; + WriteLogo(); + o.WriteLine($"{name} Database Tool. [Azure Cosmos DB]"); + o.WriteLine(); + o.WriteLine($"Command = {command}"); + o.WriteLine($"Endpoint = {GetEndpoint()}"); + o.WriteLine($"Database = {Args.DatabaseId}"); + + if (Args.Parameters.Count > 0) + { + o.WriteLine("Parameters:"); + foreach (var p in Args.Parameters) + { + o.WriteLine($" {p.Key} = {p.Value}"); + } + } + + if (Args.Containers.Count > 0) + { + o.WriteLine("Containers:"); + foreach (var c in Args.Containers) + { + o.WriteLine($" {c.Id} (partition key: {c.PartitionKeyPath}{(c.IsReferenceData ? ", reference data" : string.Empty)}{(c.IsOutboxLease ? ", outbox lease" : string.Empty)})"); + } + } + + if (Args.Assemblies.Count > 0) + { + o.WriteLine("Assemblies:"); + foreach (var a in Args.Assemblies) + { + o.WriteLine($" {a.FullName}"); + } + } + + o.WriteLine(); + } + + /// + /// Gets the account endpoint from the connection string (without the key). + /// + private string GetEndpoint() + { + foreach (var part in ConnectionString.Split(';', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)) + { + if (part.StartsWith("AccountEndpoint=", StringComparison.OrdinalIgnoreCase)) + return part["AccountEndpoint=".Length..]; + } + + return "(unknown)"; + } + + /// + /// Writes the help. + /// + private static void WriteHelp(string name) + { + var o = Console.Out; + WriteLogo(); + o.WriteLine($"{name} Database Tool. [Azure Cosmos DB]"); + o.WriteLine(); + o.WriteLine($"Usage: {name} [options] "); + o.WriteLine(); + o.WriteLine("Arguments:"); + WriteItem("command", $"Database provisioning command (a comma-separated combination is allowed). Allowed values are: {string.Join(", ", Enum.GetNames())}."); + o.WriteLine(); + o.WriteLine("Options:"); + WriteItem("-?|-h|--help", "Show help information."); + WriteItem("-cs|--connection-string", "Cosmos DB connection string."); + WriteItem("-cv|--connection-varname", "Cosmos DB connection string environment variable name."); + WriteItem("-d|--database", "Cosmos DB database identifier."); + WriteItem("-p|--param", "Parameter expressed as a 'Name=Value' pair (multiple can be specified); referenced in the seed data as '^Name' (or '(^Name)' where embedded within a value)."); + WriteItem("--accept-prompts", "Accept prompts; command should _not_ stop and wait for user confirmation (DROP or RESET commands)."); + o.WriteLine(); + } + + /// + /// Writes a help item, wrapping the description within a fixed-width column. + /// + private static void WriteItem(string item, string description) + { + const int column = 30; + const int width = 130 - column; + + var lines = new List(); + var line = string.Empty; + foreach (var word in description.Split(' ')) + { + if (line.Length > 0 && line.Length + 1 + word.Length > width) + { + lines.Add(line); + line = word; + } + else + line = line.Length == 0 ? word : $"{line} {word}"; + } + + lines.Add(line); + Console.Out.WriteLine($" {item,-(column - 2)}{lines[0]}"); + foreach (var l in lines.Skip(1)) + { + Console.Out.WriteLine($"{new string(' ', column)}{l}"); + } + } +} diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbContainerDefinition.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbContainerDefinition.cs new file mode 100644 index 000000000..1c73b3163 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbContainerDefinition.cs @@ -0,0 +1,55 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Defines a Cosmos DB container to be provisioned by the . +/// +/// The . +/// The partition key path. +/// Indicates whether the container hosts type-discriminated reference data (seed data is imported using ). +/// An optional action to further configure the (e.g. indexing policy, unique keys, time-to-live). +/// An optional factory for the used to import this container's seed data; a new instance must be returned on each invocation (the options are mutable and may be stateful). +/// Returning uses the default ( for ; otherwise, a plain ). +/// Indicates that this container is the Change Feed Processor lease container shared by the outbox relay(s) (see ). +public sealed class CosmosDbContainerDefinition(string id, string partitionKeyPath, bool isReferenceData = false, Action? configure = null, Func? dataOptions = null, bool isOutboxLease = false) +{ + /// + /// Gets the . + /// + public string Id { get; } = id.ThrowIfNullOrEmpty(); + + /// + /// Gets the partition key path. + /// + public string PartitionKeyPath { get; } = partitionKeyPath.ThrowIfNullOrEmpty(); + + /// + /// Indicates whether the container hosts type-discriminated reference data. + /// + public bool IsReferenceData { get; } = isReferenceData; + + /// + /// Indicates whether this container is the outbox relay lease container. + /// + public bool IsOutboxLease { get; } = isOutboxLease; + + /// + /// Gets the optional factory. + /// + public Func? DataOptions { get; } = dataOptions; + + /// + /// Creates the for this definition. + /// + public ContainerProperties CreateProperties() + { + var cp = new ContainerProperties(Id, PartitionKeyPath); + + // Unique keys are enforced per logical partition, are case-sensitive, and are immutable once the container is created. The outbox events co-located in this container must carry a unique 'code' + // (see CosmosDbContainerOptions.WithReferenceDataOutboxEvent) or they would collide on the (null, null) tuple. + if (IsReferenceData) + cp.UniqueKeyPolicy.UniqueKeys.Add(new UniqueKey { Paths = { "/typeDiscriminator", "/code" } }); + + configure?.Invoke(cp); + return cp; + } +} diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbDataContext.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbDataContext.cs new file mode 100644 index 000000000..f9ad8c277 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbDataContext.cs @@ -0,0 +1,9 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Provides the context for resolving the used to import a single container's seed data. +/// +/// The data resource name (file) being imported. +/// The target . +/// The . +public sealed record CosmosDbDataContext(string ResourceName, CosmosDbContainerDefinition Container, JsonPropertyNamingConvention NamingConvention); diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbDataResource.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbDataResource.cs new file mode 100644 index 000000000..dec90f8f2 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbDataResource.cs @@ -0,0 +1,10 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Represents a named embedded seed data resource, with an optional factory specific to it. +/// +/// The containing the resource. +/// The resource name (matched to the end of the fully qualified resource name; see ). +/// The optional factory for the used to import each container within the resource; a new instance must be returned on each invocation (the options are mutable and may be stateful). +/// Returning defers to the and then the default. +public sealed record CosmosDbDataResource(Assembly Assembly, string ResourceName, Func? DataOptions = null); \ No newline at end of file diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionArgs.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionArgs.cs new file mode 100644 index 000000000..b1ccdcf49 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionArgs.cs @@ -0,0 +1,184 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Provides the arguments for the ; the Cosmos DB equivalent of the DbEx MigrationArgs. +/// +/// Seed data is YAML (.yaml/.yml) or JSON embedded resources where each top-level key is a and its array value the documents to import. Resources located +/// in a Data folder of any are always imported (in resource-name order); additional named resources (optionally with their own ) are added via and are imported afterwards. +/// Seeding creates documents directly (see ), so the JSON must already carry the partition key and, for a container, the +/// $^TypeName grouped shorthand is used ($ merges (upserts) rather than inserts, making the import re-runnable; ^ generates the identifier where not specified) (see ). +public class CosmosDbProvisionArgs +{ + private readonly List _containers = []; + private readonly List _dataResources = []; + + /// + /// Gets or sets the . + /// + public string? DatabaseId { get; set; } + + /// + /// Gets or sets the manually provisioned throughput (RU/s) used when creating each container; defaults to (the account default). + /// + public int? Throughput { get; set; } + + /// + /// Gets or sets the default property naming convention of the seed data; defaults to . + /// + public JsonPropertyNamingConvention NamingConvention { get; set; } = JsonPropertyNamingConvention.CamelCase; + + /// + /// Gets or sets the that progress is written to; defaults to . + /// + public TextWriter Output { get; set; } = TextWriter.Null; + + /// + /// Indicates whether confirmation prompts (for the destructive and commands) are accepted automatically; used by the . + /// + public bool AcceptPrompts { get; set; } + + /// + /// Gets the name/value parameters made available to the seed data parser (referenced as ^Name, or (^Name) where embedded within a value); these override any same-named parameter configured by the . + /// + public Dictionary Parameters { get; } = new(StringComparer.OrdinalIgnoreCase); + + /// + /// Gets the list that is scanned for embedded Data resources. + /// + public List Assemblies { get; } = []; + + /// + /// Gets the declared list. + /// + public IReadOnlyList Containers => _containers; + + /// + /// Gets the additional named data resources. + /// + public IReadOnlyList DataResources => _dataResources; + + /// + /// Adds the (inferred from ) to . + /// + /// The to infer the . + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs AddAssembly() => AddAssembly(typeof(TAssembly).Assembly); + + /// + /// Adds the to (where not already present). + /// + /// The array. + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs AddAssembly(params Assembly[] assemblies) + { + foreach (var a in assemblies.ThrowIfNull()) + { + if (!Assemblies.Contains(a)) + Assemblies.Add(a); + } + + return this; + } + + /// + /// Adds named embedded data resources from the inferred from . + /// + /// The to infer the . + /// The resource names (matched to the end of the fully qualified resource name; see ). + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs AddDataResource(params string[] resourceNames) => AddDataResource(typeof(TAssembly).Assembly, resourceNames); + + /// + /// Adds a named embedded data resource from the inferred from with resource-specific . + /// + /// The to infer the . + /// The resource name (matched to the end of the fully qualified resource name; see ). + /// The factory for the (see ). + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs AddDataResource(string resourceName, Func dataOptions) => AddDataResource(typeof(TAssembly).Assembly, resourceName, dataOptions); + + /// + /// Adds named embedded data resources from the . + /// + /// The . + /// The resource names (matched to the end of the fully qualified resource name; see ). + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs AddDataResource(Assembly assembly, params string[] resourceNames) + { + foreach (var rn in resourceNames.ThrowIfNull()) + { + AddDataResource(assembly, rn, (Func?)null); + } + + return this; + } + + /// + /// Adds a named embedded data resource from the with resource-specific . + /// + /// The . + /// The resource name (matched to the end of the fully qualified resource name; see ). + /// The optional factory for the (see ). + /// The to support fluent-style method-chaining. + /// Where the resource is also located in a Data folder of any then it is imported once, in its default position, using the ; otherwise, it is imported afterwards. + /// Adding the same resource more than once is not permitted. + public CosmosDbProvisionArgs AddDataResource(Assembly assembly, string resourceName, Func? dataOptions) + { + assembly.ThrowIfNull(); + resourceName.ThrowIfNullOrEmpty(); + + if (_dataResources.Any(x => x.Assembly == assembly && string.Equals(x.ResourceName, resourceName, StringComparison.OrdinalIgnoreCase))) + throw new InvalidOperationException($"Data resource '{resourceName}' has already been added."); + + _dataResources.Add(new CosmosDbDataResource(assembly, resourceName, dataOptions)); + return this; + } + + /// + /// Declares a container. + /// + /// The . + /// The partition key path; defaults to /partitionKey. + /// An optional action to further configure the (e.g. indexing policy, unique keys, time-to-live). + /// An optional factory for the seed data (see ). + /// The to support fluent-style method-chaining. + public CosmosDbProvisionArgs Container(string id, string partitionKeyPath = "/partitionKey", Action? configure = null, Func? dataOptions = null) + => Add(new CosmosDbContainerDefinition(id, partitionKeyPath, false, configure, dataOptions)); + + /// + /// Declares a type-discriminated reference data container, including the /typeDiscriminator and /code unique key that rejects duplicate codes (see ). + /// + /// The . + /// The partition key path; defaults to /partitionKey. + /// An optional action to further configure the . + /// An optional factory for the seed data (see ); use as the starting point (e.g. for a custom identifier generator or tenant). + /// The to support fluent-style method-chaining. + /// The consuming must register the same container using . + public CosmosDbProvisionArgs ReferenceDataContainer(string id, string partitionKeyPath = "/partitionKey", Action? configure = null, Func? dataOptions = null) + => Add(new CosmosDbContainerDefinition(id, partitionKeyPath, true, configure, dataOptions)); + + /// + /// Declares the Change Feed Processor lease container shared by the outbox relay(s) (see AddCosmosDbOutboxRelayHostedService). + /// + /// The lease ; where not specified, defaults to (the same default used by the relay). + /// An optional action to further configure the . + /// The to support fluent-style method-chaining. + /// The outbox relay never creates its lease container; it fails fast where it does not exist. Creating a container is a control-plane operation that a production host identity (e.g. Entra ID data-plane RBAC) + /// is not expected to be permitted, so it must be provisioned up front - here (for dev/test/CI and admin-run migrations) or via infrastructure-as-code. The lease container is partitioned on /id (the Change Feed Processor's + /// own lease-document convention), and is created, reset and dropped (as part of the database) with the other declared containers. A single lease container is safely shared by all relays (each Change Feed Processor + /// is uniquely named per monitored container, and leases are further scoped to the monitored container's resource identity). + public CosmosDbProvisionArgs OutboxLeaseContainer(string? leaseContainerId = null, Action? configure = null) + => Add(new CosmosDbContainerDefinition(leaseContainerId ?? CosmosDbOutboxRelayOptions.DefaultLeaseContainerId, "/id", false, configure, null, true)); + + /// + /// Adds the to the declared . + /// + private CosmosDbProvisionArgs Add(CosmosDbContainerDefinition definition) + { + if (_containers.Any(x => x.Id == definition.Id)) + throw new InvalidOperationException($"Container '{definition.Id}' has already been declared."); + + _containers.Add(definition); + return this; + } +} diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionCommand.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionCommand.cs new file mode 100644 index 000000000..005435165 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisionCommand.cs @@ -0,0 +1,48 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Defines the provisioning commands (); the Cosmos DB equivalent of the DbEx MigrationCommand. +/// +[Flags] +public enum CosmosDbProvisionCommand +{ + /// + /// No command. + /// + None = 0, + + /// + /// Deletes the database (and therefore all containers and data); a non-existent database is not an error. + /// + Drop = 1, + + /// + /// Creates the database and any declared containers that do not already exist; existing containers (and their data) are left untouched. + /// + Create = 2, + + /// + /// Replaces (deletes and recreates, so is therefore empty) every declared container. The database must already exist (it is not implicitly created; as per DbEx the creation must be specifically requested, e.g. Create | Reset); runs after . + /// + Reset = 4, + + /// + /// Imports the seed data; a $-prefixed key is merged (upserted, re-runnable), otherwise inserted (a re-run conflicts). + /// + Data = 8, + + /// + /// and . + /// + All = Create | Data, + + /// + /// and . + /// + ResetAndData = Reset | Data, + + /// + /// and . + /// + DropAndAll = Drop | All +} diff --git a/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisioner.cs b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisioner.cs new file mode 100644 index 000000000..01c51ac06 --- /dev/null +++ b/src/CoreEx.Cosmos/Provisioning/CosmosDbProvisioner.cs @@ -0,0 +1,323 @@ +namespace CoreEx.Cosmos.Provisioning; + +/// +/// Provisions a Cosmos DB database and its containers, and imports seed data; the Cosmos DB equivalent of the DbEx migration engine. +/// +/// The . +/// The . +/// Intended for development, test and CI (e.g. the emulator); production provisioning remains the responsibility of infrastructure-as-code. A container's unique key policy and partition key path are +/// immutable once created, so will not alter an existing container; use (or ) instead. +public class CosmosDbProvisioner(CosmosClient client, CosmosDbProvisionArgs args) +{ + private readonly CosmosClient _client = client.ThrowIfNull(); + private readonly CosmosDbProvisionArgs _args = args.ThrowIfNull(); + + /// + /// Gets the . + /// + public CosmosDbProvisionArgs Args => _args; + + /// + /// The horizontal rule written between sections. + /// + internal static readonly string Rule = new('-', 80); + + private string DatabaseId => _args.DatabaseId.ThrowIfNullOrEmpty(nameof(CosmosDbProvisionArgs.DatabaseId)); + + /// + /// Runs the in the order , , , then . + /// + /// The . + /// The . + public async Task RunAsync(CosmosDbProvisionCommand command, CancellationToken cancellationToken = default) + { + _args.Output.WriteLine(Rule); + _args.Output.WriteLine(); + + if (command.HasFlag(CosmosDbProvisionCommand.Drop)) + await SectionAsync("DATABASE DROP: Dropping the database where found...", () => DropAsync(cancellationToken)).ConfigureAwait(false); + + if (command.HasFlag(CosmosDbProvisionCommand.Create)) + await SectionAsync("DATABASE CREATE: Checking database and container existence and creating where not found...", () => CreateAsync(cancellationToken)).ConfigureAwait(false); + + if (command.HasFlag(CosmosDbProvisionCommand.Reset)) + await SectionAsync("CONTAINER RESET: Replacing (emptying) the declared containers...", () => ResetAsync(cancellationToken)).ConfigureAwait(false); + + if (command.HasFlag(CosmosDbProvisionCommand.Data)) + await SectionAsync("DATABASE DATA: Insert or merge the embedded data [yaml|json]..", () => DataAsync(cancellationToken)).ConfigureAwait(false); + } + + /// + /// Runs the (see ) capturing the output, and reports the outcome rather than throwing; the Cosmos DB equivalent of the DbEx MigrateAndLogAsync. + /// + /// The . + /// The . + /// The Success indicator and the captured Output; where unsuccessful the Output is appended with the failure message. + /// The is replaced for the duration of the run (so the instance must not be run concurrently) and then restored, with the captured output also written to it. A is not + /// treated as a failure and is therefore thrown. + public async Task<(bool Success, string Output)> RunAndLogAsync(CosmosDbProvisionCommand command, CancellationToken cancellationToken = default) + { + var original = _args.Output; + var captured = new StringWriter(); + _args.Output = captured; + + try + { + await RunAsync(command, cancellationToken).ConfigureAwait(false); + return (true, captured.ToString()); + } + catch (Exception ex) when (ex is not OperationCanceledException) + { + captured.WriteLine($"Failed: {ex.Message}"); + return (false, captured.ToString()); + } + finally + { + _args.Output = original; + original.Write(captured.ToString()); + original.Flush(); + } + } + + /// + /// Runs the as a titled, timed section followed by a rule. + /// + private async Task SectionAsync(string title, Func action) + { + var sw = System.Diagnostics.Stopwatch.StartNew(); + _args.Output.WriteLine(title); + await action().ConfigureAwait(false); + _args.Output.WriteLine(); + _args.Output.WriteLine($"Complete. [{sw.Elapsed.TotalMilliseconds:0.0}ms]"); + _args.Output.WriteLine(); + _args.Output.WriteLine(Rule); + _args.Output.WriteLine(); + } + + /// + /// Deletes the database; a non-existent database is not an error. + /// + public async Task DropAsync(CancellationToken cancellationToken = default) + { + _args.Output.WriteLine(" Drop database..."); + + try + { + await _client.GetDatabase(DatabaseId).DeleteAsync(cancellationToken: cancellationToken).ConfigureAwait(false); + _args.Output.WriteLine($" Database '{DatabaseId}' dropped."); + } + catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) + { + _args.Output.WriteLine($" Database '{DatabaseId}' does not exist and therefore not dropped."); + } + } + + /// + /// Creates the database and any declared containers that do not already exist. + /// + public async Task CreateAsync(CancellationToken cancellationToken = default) + { + var database = await CreateDatabaseAsync(cancellationToken).ConfigureAwait(false); + + foreach (var c in _args.Containers) + { + var label = Label(c); + _args.Output.WriteLine($" Create {label}..."); + var response = await RetryAsync(() => database.CreateContainerIfNotExistsAsync(c.CreateProperties(), _args.Throughput, cancellationToken: cancellationToken), cancellationToken).ConfigureAwait(false); + _args.Output.WriteLine(response.StatusCode == HttpStatusCode.Created ? $" {Capitalize(label)} created." : $" {Capitalize(label)} already exists and therefore not created."); + } + } + + /// + /// Replaces (deletes and recreates; therefore empty) every declared container; the database must already exist (as per DbEx, a reset does not implicitly create it - include ). + /// + /// The database does not exist. + public async Task ResetAsync(CancellationToken cancellationToken = default) + { + var database = _client.GetDatabase(DatabaseId); + + try + { + await RetryAsync(() => database.ReadAsync(cancellationToken: cancellationToken), cancellationToken).ConfigureAwait(false); + } + catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) + { + throw new InvalidOperationException($"Database '{DatabaseId}' does not exist; a reset does not create it (use the Create command, e.g. 'All' or 'Create,ResetAndData').", cex); + } + + foreach (var c in _args.Containers) + { + var label = Label(c); + _args.Output.WriteLine($" Reset {label}..."); + await RetryAsync(() => database.ReplaceOrCreateContainerAsync(c.CreateProperties(), _args.Throughput, cancellationToken), cancellationToken).ConfigureAwait(false); + _args.Output.WriteLine($" {Capitalize(label)} replaced (empty)."); + } + } + + /// + /// Gets the output label for the container (identifying the outbox relay lease container). + /// + private static string Label(CosmosDbContainerDefinition c) => c.IsOutboxLease ? $"outbox lease container '{c.Id}'" : $"container '{c.Id}'"; + + /// + /// Capitalizes the first character. + /// + private static string Capitalize(string text) => char.ToUpperInvariant(text[0]) + text[1..]; + + /// + /// Imports the seed data; the Data resources first, followed by any remaining . + /// + public async Task DataAsync(CancellationToken cancellationToken = default) + { + var database = _client.GetDatabase(DatabaseId); + _args.Output.WriteLine($" Probing for embedded resources: {string.Join(", ", _args.Assemblies.Select(a => $"{a.GetName().Name}.Data.*"))}"); + + foreach (var (name, content, resourceOptions) in GetDataResources()) + { + _args.Output.WriteLine(); + _args.Output.WriteLine($"** Parsing and executing: {name}"); + var isJson = name.EndsWith(".json", StringComparison.OrdinalIgnoreCase); + + // A plain reader is used only to discover the top-level container keys; each container is then imported using its own resolved options. + if ((isJson ? JsonDataReader.ParseJson(content, new JsonDataReaderOptions(_args.NamingConvention)) : JsonDataReader.ParseYaml(content, new JsonDataReaderOptions(_args.NamingConvention))).RootNode is not JsonObject root) + continue; + + // As per DbEx, a '$' prefix indicates merge (upsert) rather than insert; for a reference data container it is the type discriminator groups that carry the prefix instead. A leading '$' is not + // addressable in a JSON path (it denotes the root) so the prefix is removed from the top-level keys, noting which were merges, before the data is re-parsed. + var merges = root.Where(kvp => kvp.Key.StartsWith('$')).Select(kvp => kvp.Key.TrimStart('$')).ToHashSet(); + var normalized = new JsonObject(); + foreach (var kvp in root) + { + normalized[kvp.Key.TrimStart('$')] = kvp.Value?.DeepClone(); + } + + var json = normalized.ToJsonString(); + JsonDataReader Parse(JsonDataReaderOptions options) => JsonDataReader.ParseJson(json, options); + + foreach (var containerId in normalized.Select(kvp => kvp.Key).ToList()) + { + var merge = merges.Contains(containerId); + var definition = _args.Containers.FirstOrDefault(c => c.Id == containerId) + ?? throw new InvalidOperationException($"Data '{name}' top-level key '{containerId}' does not match a declared container ({string.Join(", ", _args.Containers.Select(c => $"'{c.Id}'"))})."); + + var container = database.GetContainer(definition.Id); + var jdr = Parse(ResolveDataOptions(name, resourceOptions, definition)); + var verb = merge ? "Merging" : "Inserting"; + + if (definition.IsReferenceData) + { + await CosmosDbBatch.ImportDiscriminatedBatchAsync(container, jdr, containerId, (group, groupMerge, created, replaced) => WriteResult($"{(groupMerge ? "Merging" : "Inserting")} '{containerId}' / {group}", groupMerge, created, replaced), false, cancellationToken).ConfigureAwait(false); + continue; + } + + if (!jdr.TryCreateData(containerId, out var node) || node is not JsonArray array) + continue; + + if (merge) + { + var (created, replaced) = await container.MergeBatchAsync(array, cancellationToken: cancellationToken).ConfigureAwait(false); + WriteResult($"{verb} '{containerId}'", true, created, replaced); + } + else + { + var (created, replaced) = await container.ImportBatchCountAsync(array, cancellationToken).ConfigureAwait(false); + WriteResult($"{verb} '{containerId}'", false, created, replaced); + } + } + } + } + + /// + /// Writes the result of an import or merge. + /// + private void WriteResult(string title, bool merge, int created, int replaced) + { + _args.Output.WriteLine(); + _args.Output.WriteLine($"---- {title}:"); + _args.Output.WriteLine(merge ? $"Result: {created + replaced} item(s) ({created} created, {replaced} replaced)." : $"Result: {created} item(s) created."); + } + + /// + /// Resolves the ; the resource first, then the , then the default. + /// + private JsonDataReaderOptions ResolveDataOptions(string resourceName, Func? resourceOptions, CosmosDbContainerDefinition definition) + { + var context = new CosmosDbDataContext(resourceName, definition, _args.NamingConvention); + var options = resourceOptions?.Invoke(context) + ?? definition.DataOptions?.Invoke(context) + ?? (definition.IsReferenceData ? JsonDataReaderOptions.CreateForReferenceData(_args.NamingConvention) : new JsonDataReaderOptions(_args.NamingConvention)); + + foreach (var (name, value) in _args.Parameters) + { + options.Parameters[name] = _ => value; + } + + return options; + } + + /// + /// Gets the data resources (name, content and optional resource-specific options) in import order. + /// + private IEnumerable<(string Name, string Content, Func? DataOptions)> GetDataResources() + { + var explicitly = _args.DataResources.ToList(); + var matched = new HashSet(); + + foreach (var assembly in _args.Assemblies) + { + foreach (var name in assembly.GetManifestResourceNames().Where(IsDataResource).OrderBy(x => x, StringComparer.Ordinal)) + { + // An explicitly added resource that is also a scanned one is imported once, here, using its options. + var resource = explicitly.FirstOrDefault(x => x.Assembly == assembly && name.EndsWith(x.ResourceName, StringComparison.OrdinalIgnoreCase)); + if (resource is not null) + matched.Add(resource); + + using var sr = new StreamReader(assembly.GetManifestResourceStream(name)!); + yield return (name, sr.ReadToEnd(), resource?.DataOptions); + } + } + + foreach (var resource in explicitly.Where(x => !matched.Contains(x))) + { + using var sr = new StreamReader(Resource.GetStream(resource.ResourceName, resource.Assembly)); + yield return (resource.ResourceName, sr.ReadToEnd(), resource.DataOptions); + } + } + + /// + /// Determines whether the manifest resource name is a Data folder YAML or JSON file. + /// + private static bool IsDataResource(string name) + => name.Contains(".Data.", StringComparison.OrdinalIgnoreCase) && (name.EndsWith(".yaml", StringComparison.OrdinalIgnoreCase) || name.EndsWith(".yml", StringComparison.OrdinalIgnoreCase) || name.EndsWith(".json", StringComparison.OrdinalIgnoreCase)); + + /// + /// Creates the database where not existing. + /// + private async Task CreateDatabaseAsync(CancellationToken cancellationToken) + { + _args.Output.WriteLine(" Create database..."); + var response = await RetryAsync(() => _client.CreateDatabaseIfNotExistsAsync(DatabaseId, cancellationToken: cancellationToken), cancellationToken).ConfigureAwait(false); + _args.Output.WriteLine(response.StatusCode == HttpStatusCode.Created ? $" Database '{DatabaseId}' created." : $" Database '{DatabaseId}' already exists and therefore not created."); + return response.Database; + } + + /// + /// The local emulator occasionally responds with a transient 503 ServiceUnavailable ("high demand") when several containers are created in quick succession, and intermittently drops the TLS handshake. + /// + /// Note: the legacy emulator image also returned a 503 once its container-count cap was exhausted; that was deterministic and retrying would not resolve it (the vNext emulator has no such cap). + private static async Task RetryAsync(Func> action, CancellationToken cancellationToken) + { + const int maxAttempts = 5; + for (var attempt = 1; ; attempt++) + { + try + { + return await action().ConfigureAwait(false); + } + catch (Exception ex) when (attempt < maxAttempts && (ex is CosmosException { StatusCode: HttpStatusCode.ServiceUnavailable } || ex is HttpRequestException || ex.InnerException is HttpRequestException)) + { + await Task.Delay(TimeSpan.FromSeconds(attempt * 2), cancellationToken).ConfigureAwait(false); + } + } + } +} diff --git a/src/CoreEx.Cosmos/README.md b/src/CoreEx.Cosmos/README.md index f94ec1b0d..16c7e76a7 100644 --- a/src/CoreEx.Cosmos/README.md +++ b/src/CoreEx.Cosmos/README.md @@ -1,7 +1,5 @@ # CoreEx.Cosmos -> 🚧 **Preview**: newly added in this release. The API surface may still change in a future release without following strict semver until it stabilizes. - > Provides the core [Azure Cosmos DB](https://learn.microsoft.com/en-us/azure/cosmos-db/) access layer: `ICosmosDb`/`CosmosDb` as the CoreEx-Cosmos bridge, `CosmosDbContainer` and `CosmosDbMappedContainer` for typed CRUD + query operations, `CosmosDbQuery` as the composable, invoker-wrapped query/materialization type, `CosmosDbInvoker` for structured operation logging and exception mapping, a `CosmosDbUnitOfWork` transactional outbox (`TransactionalBatch`-based), and a Change Feed Processor-based outbox relay. ## Overview @@ -24,18 +22,22 @@ This package provides the core CRUD + query access layer, a `TransactionalBatch` - 🔍 **Composable query**: `CosmosDbContainer.Query(query?, args?)` returns a `CosmosDbQuery` — a dedicated wrapper (not a bare `IQueryable`) constructed over `Container.GetItemLinqQueryable()`; additional filtering/ordering is composed via the `query` delegate (standard LINQ `Where`/`OrderBy`), and any `WithTenantFilter`/`WithLogicalDeleteFilter`/`WithTypeDiscriminator` predicates from `CosmosDbModelOptions` are applied automatically (see `CosmosDbQuery.AsQueryable(CosmosDbArgs?)`, with an optional `CosmosDbArgs.BypassFilters` override). - 📄 **Invoker-wrapped materializers**: `CosmosDbQuery` provides instance-method materializers — `ToListAsync`, `ToCollectionAsync`, `ToItemsResultAsync`, `SingleAsync`/`SingleOrDefaultAsync`/`FirstAsync`/`FirstOrDefaultAsync`, `ToMappedItemsAsync`, `ToMappedItemsResultAsync` (plus a `WithResultAsync` ROP counterpart for each) — using `Skip`/`Take` (translated by the Cosmos DB LINQ provider to `OFFSET…LIMIT`) via `WithPaging(PagingArgs?)`. Being instance methods on `CosmosDbQuery` rather than `IQueryable` extensions, they structurally cannot collide with `CoreEx.EntityFrameworkCore`'s identically-named `EfDbExtensions` (see [AGENTS.md](./AGENTS.md#do-not)), and every materializer routes through `CosmosDbInvoker` for structured logging and `CosmosException` mapping. - 🏷️ **ETag / concurrency**: for an `UpdateAsync`, the model's `IETag.ETag` is mapped into `ItemRequestOptions.IfMatchEtag` (where `CosmosDbArgs.AutoMapETag` is `true`, the default); Cosmos DB enforces the optimistic-concurrency check server-side and returns a `412 Precondition Failed`, which `CosmosDbInvoker` converts to a `ConcurrencyException`/`Result.ConcurrencyError`. +- 🧠 **Change tracker**: a per-scope snapshot cache (`ICosmosDb.ChangeTracker`) makes repeat point `GetAsync` calls free of RU cost; `UpdateAsync` pre-reads to copy back server-managed values (created audit, unmapped additional properties) and skips the write when nothing changed (`DataResult.WasMutated == false`). Opt out per call with `CosmosDbArgs.ClearChangeTrackerAfterGet`; cleared when the root unit-of-work transaction ends. Per-document version checks prevent in-flight reads from repopulating snapshots after eviction or clearing; immediate writes also evict on completion. Needs `UseSystemTextJsonSerializerWithOptions`. - 🔒 **Multi-tenancy**: non-query operations automatically reject a mismatched `IReadOnlyTenantId.TenantId` as not-found; `Query()` only applies the equivalent `TenantId == executionContext.TenantId` predicate when `CosmosDbModelOptions.WithTenantFilter()` has been configured. - 🗑️ **Logical delete**: entities implementing `ILogicallyDeleted` are soft-deleted (`IsDeleted = true` via a read-modify-`ReplaceItemAsync`) on `DeleteAsync` rather than physically removed; a physical `DeleteAsync` is idempotent (a `404` is not an error). - 🏷️ **Type discriminator (multi-type containers)**: `CosmosDbModelOptions.WithTypeDiscriminator()` reuses the existing `ITypeDiscriminator`/`IReadOnlyTypeDiscriminator` hook (auto-populated by `Model.PrepareCreate`/`PrepareUpdate`) to let several business model types safely share one container/partition — no envelope/wrapper type required. - ⏳ **Time-to-live**: `CosmosDbModelOptions.WithTimeToLive(Func)` computes and applies a document's `ttl` on `CreateAsync`/`UpdateAsync` (requires `TModel` to implement the *mutable* `ITimeToLive` — Cosmos DB's `ttl` is a document-body field, not a separate SDK request option, so a computed value can only take effect by being written back onto the model). Where not configured, a model's own `ITimeToLive.TimeToLive` value (if any) simply serializes through as-is; `ITimeToLive`/`IReadOnlyTimeToLive` live in core `CoreEx.Data` (alongside `IPartitionKey`/`ITypeDiscriminator`) for reuse by a future non-Cosmos NoSQL package. +- 📚 **Reference data**: `CosmosDbReferenceData` provides `CreateAsync`/`UpdateAsync`/`ActivateAsync`/`DeactivateAsync`/`DeleteAsync` for reference data (Cosmos counterpart of `EfDbReferenceData`); wired up by CoreEx.CodeGen when a `repository: Cosmos` entity sets `mutability`. - 🔑 **Fixed partition key**: `CosmosDbModelOptions.WithFixedPartitionKey(string?)` configures one constant partition key value for the whole container — suitable for small, bounded containers where partitioning isn't meaningful (Cosmos DB's own guidance: a container well under the 20 GB/10,000 RU/s per-logical-partition limits typically needs only one or two physical partitions regardless of partition key cardinality). It also defaults `GetAsync`/`DeleteAsync`'s `partitionKey` parameter (now optional) when the caller omits it — `WithPartitionKey(Func)`'s per-model function cannot do this, since Get/Delete have no model instance to invoke it against. The two are mutually exclusive (configuring both throws `InvalidOperationException`), and either always wins over — but must not silently disagree with — a non-null value the model already carries via `IReadOnlyPartitionKey` (a genuine mismatch throws rather than being overridden, since Cosmos DB itself requires the document body's partition-key-path value to agree with the value supplied for the operation). - 📝 **Structured logging**: `CosmosDbInvoker` wraps every CRUD `CosmosDb` operation with structured log entries (tracing/`Activity` spans are intentionally disabled via `IsTracingDisabled` - CRUD is high-frequency) and converts `CosmosException` into the corresponding CoreEx exception (`NotFoundException`/`DuplicateException`/`ConcurrencyException`). The outbox relay's `CosmosDbOutboxRelayInvoker` (`Outbox` sub-namespace) is a separate, tracing-*enabled* invoker - relay batch processing is comparatively low-frequency and specifically where distributed-tracing visibility matters most. - 🔄 **Transactional outbox**: `CosmosDbUnitOfWork` implements `IUnitOfWork`, enlisting `CosmosDbContainer` Create/Update/Delete calls made within its `TransactionAsync` scope into one ambient `TransactionalBatch` (client-side fail-fast if two enlisted operations target different containers/partition keys); `CosmosDbEventPublisher` (an `IEventPublisher`) enlists outbox event documents into the *same* batch, so the business mutation and its event are atomic without a separate outbox table. Outbox documents are auto-excluded from ordinary business queries via a reserved `$outbox` id-prefix (no opt-in required). `IUnitOfWork.SynchronizeETag` resolves a mapped contract's true, server-assigned `ETag` after the batch commits (deferred execution means it isn't known upfront) by correlating on `CompositeKey`, not object reference. - 📤 **Outbox relay**: `CosmosDbOutboxRelay`/`CosmosDbOutboxRelayProcessor` (`Outbox` sub-namespace) consume outbox event documents via a Cosmos DB [Change Feed Processor](https://learn.microsoft.com/en-us/azure/cosmos-db/nosql/change-feed-processor) (push-based, SDK-managed - not a polling loop like the SQL Server/Postgres relay), decode/publish/cleanup-delete each batch, and self-pause/self-resume via a `CircuitBreakerResiliency`-based circuit breaker on a sustained publish-failure ratio. Register via `builder.AddCosmosDbOutboxRelayHostedService(containerId, servicesCount?)` - one call per outbox-hosting container, each with its own concurrency count. This registers the relay only; a genuine *destination* `IEventPublisher` (e.g. Azure Service Bus, matching the SQL Server/Postgres samples) must also be registered - see [AGENTS.md](./AGENTS.md#outbox-relay). `CosmosDbEventPublisher` (the write-side publisher, above) must never be registered for this role; `CosmosDbOutboxRelayProcessor.ProcessBatchAsync` detects that misconfiguration and throws immediately with an actionable message rather than failing deeper inside the publish call. - 📊 **Outbox metrics**: `CosmosMetrics` exposes .NET `Meter` instruments harmonized with `SqlServerMetrics`/`PostgresMetrics`: `cosmos.outbox.enqueue` (counter), `cosmos.outbox.relay.publish` and `cosmos.outbox.relay.publish.failed` (counters), `cosmos.outbox.relay.oldest_lag` and `cosmos.outbox.relay.newest_lag` (histograms in ms), plus Cosmos-specific `cosmos.outbox.relay.cleanup.deleted`/`cosmos.outbox.relay.cleanup.failed` (the relay's own post-publish document cleanup has no SQL Server/Postgres equivalent). -- 📥 **Batch import & container provisioning**: `CosmosDbBatch.ImportBatchAsync(Async)`/`ImportDiscriminatedBatchAsync` load raw JSON directly into a `Container`/`Database` (no `CoreEx.Cosmos` model type involved); `CosmosDbContainerExtensions.ReplaceOrCreateContainerAsync`/`DeleteContainerIfExistsAsync` provision or reset a container from code. Neither depends on the rest of this package - useful for data seeding, bulk/one-off loads, and migrations alike. +- 📥 **Batch import & container provisioning**: `CosmosDbBatch.ImportBatchAsync(Async)`/`ImportDiscriminatedBatchAsync` load raw JSON directly into a `Container`/`Database` (no `CoreEx.Cosmos` model type involved); `CosmosDbContainerExtensions.ReplaceOrCreateContainerAsync`/`DeleteContainerIfExistsAsync` provision or reset a container from code. Neither depends on the rest of this package - useful for data seeding, bulk/one-off loads, and migrations alike. On top of these, `CoreEx.Cosmos.Provisioning` (`CosmosDbProvisioner`, `CosmosDbConsole`, `CosmosDbProvisionArgs`) provides a reusable, DbEx-style Drop/Create/Reset/Data migration console for dev/test - see `Contoso.Customers.Database`. - 🧩 **Multi-set queries**: `ICosmosDb.SelectMultiSetAsync`/`SelectMultiSetWithResultAsync(containerId, MultiSetOptions, cancellationToken?)` (`Extended` namespace) read multiple, type-discriminator-keyed sets of documents from the same container/partition in one round-trip - the Cosmos DB equivalent of `CoreEx.Database.Extended`'s positional/ordered multi-set queries, adapted for a discriminator-keyed (not positional) demux. `MultiSetSingleArgs`/`MultiSetCollArgs` accumulate matching documents (each per-item tenant/logical-delete/additive-filter-checked via `CosmosDbContainer.CheckModel`); `TModel` must implement `IReadOnlyTypeDiscriminator`. The type-discriminator's JSON property name is resolved once per call from the ambient `CosmosClientOptions.UseSystemTextJsonSerializerWithOptions` naming policy (e.g. `camelCase`). Co-located outbox event documents are always excluded server-side; where a model's `CosmosDbModelOptions.WithTenantFilter`/`WithLogicalDeleteFilter` is configured, an additional defensive, `IS_DEFINED`-guarded SQL predicate is layered in as a server-side (RU/bandwidth) optimization - never excluding a document purely for predating the property. `MultiSetOptions.Args.QueryRequestOptions`, where supplied, takes precedence over one built from `MultiSetOptions.PartitionKey` (a genuine mismatch between the two throws `ArgumentException`). `SelectMultiSetWithResultAsync` returns a `Result` (Railway-Oriented Programming) only for a genuine business/domain-level outcome (a mapped `CosmosException`, or a `WithFilter` authorization-style denial) - `MinimumRows`/`MaximumRows`/malformed-response/argument-validation conditions remain plain exceptions even from this method, consistent with `Result` being reserved for expected errors, not exceptions; `SelectMultiSetAsync` is a thin `ThrowOnError()` wrapper over it. See [AGENTS.md](./AGENTS.md#multi-set-queries) for the full mechanism. +Outbox tracing defaults mirror the relational relays: background change-feed polling, lease maintenance, and cleanup SDK/HTTP spans are suppressed, while publishing spans and originating-trace relay markers remain enabled. Metrics and warning/error logs remain available. Set `CosmosDbOutboxRelayOptions.IsInstrumentationEnabledForPolling` to `true` before start/resume to diagnose background processing; `CosmosDbOutboxRelayProcessor.IsInstrumentationEnabledForPublishing` independently controls publishing instrumentation (default `true`). + ## Key types | Type | Description | diff --git a/src/CoreEx.Data/DataResult.cs b/src/CoreEx.Data/DataResult.cs index 8d19eb825..b2ddf1000 100644 --- a/src/CoreEx.Data/DataResult.cs +++ b/src/CoreEx.Data/DataResult.cs @@ -41,4 +41,40 @@ public readonly void WhereMutated(Action action) if (WasMutated) action?.Invoke(); } -} \ No newline at end of file + + /// + /// Invokes the specified where the result is and returns the current . + /// + /// The action to execute. + /// The current . + public readonly DataResult WhereMutatedAnd(Action action) + { + WhereMutated(action); + return this; + } + + /// + /// Invokes the specified where the result is . + /// + /// The function to execute. + /// The . + public readonly Task WhereMutatedAsync(Func func, CancellationToken cancellationToken = default) + { + if (WasMutated) + return func?.Invoke(cancellationToken) ?? Task.CompletedTask; + + return Task.CompletedTask; + } + + /// + /// Invokes the specified where the result is and returns the current . + /// + /// The function to execute. + /// The . + /// The current . + public async readonly Task WhereMutatedAndAsync(Func func, CancellationToken cancellationToken = default) + { + await WhereMutatedAsync(func, cancellationToken); + return this; + } +} diff --git a/src/CoreEx.Data/DataResultT.cs b/src/CoreEx.Data/DataResultT.cs index f9df45720..5d00baf6d 100644 --- a/src/CoreEx.Data/DataResultT.cs +++ b/src/CoreEx.Data/DataResultT.cs @@ -32,7 +32,7 @@ public DataResult(T value, bool wasMutated = true) public bool WasMutated { get; } /// - /// Invokes the specified where the result is . + /// Invokes the specified where the result is returning the . /// /// The action to execute. /// The . @@ -45,10 +45,48 @@ public readonly T WhereMutated(Action action) return Value; } + /// + /// Invokes the specified where the result is returning the . + /// + /// The action to execute. + /// The . + /// This is a convenience method to allow fluent-style method-chaining. + public readonly DataResult WhereMutatedAnd(Action action) + { + WhereMutated(action); + return this; + } + + /// + /// Invokes the specified where the result is returning the . + /// + /// The function to execute. + /// The . + /// The . + public async readonly Task WhereMutatedAsync(Func func, CancellationToken cancellationToken = default) + { + if (WasMutated && func is not null) + await func.Invoke(Value, cancellationToken).ConfigureAwait(false); + + return Value; + } + + /// + /// Invokes the specified where the result is returning the . + /// + /// The function to execute. + /// The . + /// The . + public async readonly Task> WhereMutatedAndAsync(Func func, CancellationToken cancellationToken = default) + { + await WhereMutatedAsync(func, cancellationToken).ConfigureAwait(false); + return this; + } + /// /// Implicitly converts a to a . /// /// The . /// The . public static implicit operator T(DataResult result) => result.Value; -} \ No newline at end of file +} diff --git a/src/CoreEx.Database.Postgres/CoreExPostgresExtensions.DependencyInjection.cs b/src/CoreEx.Database.Postgres/CoreExPostgresExtensions.DependencyInjection.cs index e017d3d6c..ad9d2f439 100644 --- a/src/CoreEx.Database.Postgres/CoreExPostgresExtensions.DependencyInjection.cs +++ b/src/CoreEx.Database.Postgres/CoreExPostgresExtensions.DependencyInjection.cs @@ -91,6 +91,7 @@ public static IServiceCollection AddPostgresOutboxPublisher(this IServi { var outbox = ActivatorUtilities.CreateInstance(sp); configure?.Invoke(sp, outbox); + outbox.PartitionSize = DatabaseOutboxConfiguration.GetPartitionSize(sp.GetService()); return outbox; }, addAsDefaultIEventPublisher); @@ -121,4 +122,4 @@ public static IServiceCollection AddPostgresOutboxRelay return relay; }); } -} \ No newline at end of file +} diff --git a/src/CoreEx.Database.Postgres/GlobalUsing.cs b/src/CoreEx.Database.Postgres/GlobalUsing.cs index dd8726068..308aa5be4 100644 --- a/src/CoreEx.Database.Postgres/GlobalUsing.cs +++ b/src/CoreEx.Database.Postgres/GlobalUsing.cs @@ -17,6 +17,7 @@ global using CoreEx.Mapping.Converters.Abstractions; global using CoreEx.Results.Abstractions; global using CoreEx.Text; +global using Microsoft.Extensions.Configuration; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Logging; global using Npgsql; @@ -26,4 +27,4 @@ global using System.Data.Common; global using System.Diagnostics.Metrics; global using System.Text; -global using System.Text.Json; \ No newline at end of file +global using System.Text.Json; diff --git a/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxPublisher.cs b/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxPublisher.cs index 9ac290d45..628ed7b44 100644 --- a/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxPublisher.cs +++ b/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxPublisher.cs @@ -55,7 +55,7 @@ protected async override Task OnPublishAsync(DestinationEvent[] events, Cancella { // Add each event to the batch statement and parameters collection. var pk = chunk[i].Event.GetPartitionKey(); - var partitionId = PartitionKey.GetPartitionId(string.IsNullOrEmpty(pk) ? Guid.NewGuid().ToString() : pk, PartitionSize); + var partitionId = PartitionKey.GetPartitionId(string.IsNullOrEmpty(pk) ? PartitionKey.DefaultNoPartitionKey : pk, PartitionSize); sb.Append($"{Statement.CommandText}("); diff --git a/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxRelay.cs b/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxRelay.cs index 217a18eb0..96df88995 100644 --- a/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxRelay.cs +++ b/src/CoreEx.Database.Postgres/Outbox/PostgresOutboxRelay.cs @@ -29,24 +29,6 @@ public override void SetStatementsByConvention(string? schema = null) } } - /// - protected override bool IsTransientException(Exception exception) - { - if (exception is NpgsqlException nex) - { - if (nex.IsTransient) - return true; - - switch (nex.SqlState) - { - case "40P01": return true; // deadlock_detected: https://www.postgresql.org/docs/current/errcodes-appendix.html - case "55P03": return true; // lock_not_available: https://www.postgresql.org/docs/current/errcodes-appendix.html - } - } - - return base.IsTransientException(exception); - } - /// protected async override Task CompleteBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) { diff --git a/src/CoreEx.Database.Postgres/PostgresDatabase.cs b/src/CoreEx.Database.Postgres/PostgresDatabase.cs index 4d127b874..18d53ab0b 100644 --- a/src/CoreEx.Database.Postgres/PostgresDatabase.cs +++ b/src/CoreEx.Database.Postgres/PostgresDatabase.cs @@ -15,6 +15,7 @@ namespace CoreEx.Database.Postgres; /// 56010 -> . /// /// This is in addition to the with the corresponding that will also result in a . +/// The override additionally classifies deadlock, lock timeout, and Azure Database for PostgreSQL maintenance/failover/connection-limit error codes as transient (retryable). /// This class also implements the including a transactional outbox. The /// functionality is enabled by the ; note, this is not thread-safe. /// @@ -85,6 +86,37 @@ public partial class PostgresDatabase(NpgsqlDataSource dataSource, JsonSerialize return base.OnDbException(dbex); } + /// + /// only flags network-layer failures (dropped socket/timeout) - it does not consider a server-returned / at all, so every code relevant to a genuine server error (deadlock, lock timeout, failover, connection limits) must be listed explicitly here. Includes + /// well-known concurrency errors (deadlock, lock timeout), plus the codes commonly returned by Azure Database for PostgreSQL during planned maintenance/failover and connection-limit throttling + /// - relevant because Azure Database for PostgreSQL is a supported production target. + protected override bool OnIsTransientException(Exception exception) + { + if (exception is NpgsqlException nex) + { + if (nex.IsTransient) + return true; + + switch (nex.SqlState) + { + case "40P01": return true; // deadlock_detected: https://www.postgresql.org/docs/current/errcodes-appendix.html + case "55P03": return true; // lock_not_available: raised by an explicit 'SET LOCAL lock_timeout'. + case "57014": return true; // query_canceled: statement_timeout (or explicit cancellation) mid-operation. + case "40001": return true; // serialization_failure: defensive - relevant under higher isolation levels or advisory-lock contention. + + // Azure Database for PostgreSQL planned maintenance/failover and connection-limit throttling. + case "57P01": return true; // admin_shutdown: connection terminated by planned maintenance/failover. + case "57P02": return true; // crash_shutdown. + case "57P03": return true; // cannot_connect_now: server starting up/in recovery (e.g. just after a failover). + case "53300": return true; // too_many_connections. + case "53400": return true; // configuration_limit_exceeded. + } + } + + return base.OnIsTransientException(exception); + } + /// protected override void Dispose(bool disposing) { diff --git a/src/CoreEx.Database.SqlServer/CoreExSqlServerExtensions.DependencyInjection.cs b/src/CoreEx.Database.SqlServer/CoreExSqlServerExtensions.DependencyInjection.cs index a863b47bd..85b5b62b4 100644 --- a/src/CoreEx.Database.SqlServer/CoreExSqlServerExtensions.DependencyInjection.cs +++ b/src/CoreEx.Database.SqlServer/CoreExSqlServerExtensions.DependencyInjection.cs @@ -91,6 +91,7 @@ public static IServiceCollection AddSqlServerOutboxPublisher(this IServ { var outbox = ActivatorUtilities.CreateInstance(sp); configure?.Invoke(sp, outbox); + outbox.PartitionSize = DatabaseOutboxConfiguration.GetPartitionSize(sp.GetService()); return outbox; }, addAsDefaultIEventPublisher); @@ -121,4 +122,4 @@ public static IServiceCollection AddSqlServerOutboxRelay - protected override bool IsTransientException(Exception exception) - { - if (exception is SqlException sex && sex.Errors.Count > 0) - { - switch (sex.Errors[0].Number) - { - case 1205: return true; // Deadlock: https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/mssqlserver-1205-database-engine-error - } - } - - return base.IsTransientException(exception); - } - /// protected async override Task CompleteBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) { diff --git a/src/CoreEx.Database.SqlServer/SqlServerDatabase.cs b/src/CoreEx.Database.SqlServer/SqlServerDatabase.cs index e03e93002..52fbe71b2 100644 --- a/src/CoreEx.Database.SqlServer/SqlServerDatabase.cs +++ b/src/CoreEx.Database.SqlServer/SqlServerDatabase.cs @@ -15,6 +15,7 @@ namespace CoreEx.Database.SqlServer; /// 56010 -> . /// /// This is in addition to the with the corresponding that will also result in a . +/// The override additionally classifies deadlock, lock timeout, schema-recompile, and Azure SQL Database throttling/failover/connectivity error codes as transient (retryable). /// This class also implements the including a transactional outbox. The /// functionality is enabled by the ; note, this is not thread-safe. /// @@ -80,4 +81,44 @@ public partial class SqlServerDatabase(SqlConnection connection, JsonSerializerO return base.OnDbException(dbex); } + + /// + /// Includes well-known concurrency errors (deadlock, lock timeout, a stored procedure recompiled mid-flight by a migration), plus the well-known + /// Azure SQL Database throttling/failover/connectivity error codes - relevant + /// because Azure SQL Database is a supported production target. + protected override bool OnIsTransientException(Exception exception) + { + if (exception is SqlException sex && sex.Errors.Count > 0) + { + switch (sex.Errors[0].Number) + { + case 1205: return true; // Deadlock: https://learn.microsoft.com/en-us/sql/relational-databases/errors-events/mssqlserver-1205-database-engine-error + case 1222: return true; // Lock request time out period exceeded. + case 2801: return true; // The definition of object '...' has changed since it was compiled: a migration recompiled the object while this connection had a stale plan. + + // Azure SQL Database throttling/failover (resource governance, DTU/vCore limits, elastic pool, failover group). + case 4060: return true; // Cannot open database requested by the login (e.g. transient during failover). + case 4221: return true; // Login to read-secondary failed due to long wait on 'HADR_DATABASE_WAIT_FOR_TRANSITION_TO_VERSIONING'. + case 10928: return true; // Resource ID: Not enough resources to process request. + case 10929: return true; // Resource ID: The database is currently too busy. + case 40197: return true; // The service has encountered an error processing your request. + case 40501: return true; // The service is currently busy. Retry the request after 10 seconds. + case 40613: return true; // Database on server is not currently available. + case 49918: return true; // Not enough resources to process request. + case 49919: return true; // Too many create/update operations in progress for subscription. + case 49920: return true; // Too many operations in progress for subscription. + + // Network/connectivity transients. + case 233: return true; // The client was unable to establish a connection. + case 64: return true; // A connection was successfully established but then an error occurred during login. + case 20: return true; // The instance of SQL Server does not support encryption (transient during connection negotiation). + case -2: return true; // Client-side command/connection timeout. + case 10053: return true; // A transport-level error occurred: connection aborted. + case 10054: return true; // A transport-level error occurred: connection forcibly closed. + case 10060: return true; // A network-related or instance-specific error: connection attempt timed out. + } + } + + return base.OnIsTransientException(exception); + } } \ No newline at end of file diff --git a/src/CoreEx.Database/AGENTS.md b/src/CoreEx.Database/AGENTS.md index 90d8475a9..833aa633c 100644 --- a/src/CoreEx.Database/AGENTS.md +++ b/src/CoreEx.Database/AGENTS.md @@ -60,6 +60,12 @@ Stored procedures raise user error numbers 56001–56007/56010 to signal domain | 56006 | `ConflictException` | | 56007 | `DuplicateException` | +## Outbox partition rollout + +Writers and relays share `CoreEx:Host:Outbox:PartitionSize`. Preserve the effective size when migrating from the removed relay-only key. Changing the size does not remap persisted outbox rows; shrinking the relay range can leave older partition IDs unprocessed. Pause affected producers and drain the outbox with the old configuration before changing the mapping, then resume with consistent writer/relay settings. Keyless events now map deterministically using `$none`, so old/new placement must not be assumed to preserve ordering. + +Service Bus session mapping is a separate boundary. Coordinate destination policies across all producers, including direct broker publishers; drain affected broker work and in-flight handlers before switching mappings where ordering matters. + ## Do Not - Do not write raw `DbCommand` or `SqlCommand` code — use `DatabaseCommand`. diff --git a/src/CoreEx.Database/Abstractions/DatabaseArgsBase.cs b/src/CoreEx.Database/Abstractions/DatabaseArgsBase.cs index 683bd8f2e..6167a036e 100644 --- a/src/CoreEx.Database/Abstractions/DatabaseArgsBase.cs +++ b/src/CoreEx.Database/Abstractions/DatabaseArgsBase.cs @@ -19,4 +19,21 @@ public abstract record class DatabaseArgsBase : IDataArgs /// Defaults to . /// The will be skipped where set to . public bool TransformException { get; init; } = true; + + /// + /// Indicates whether to automatically retry the operation where the resulting exception is classified as transient (see ). + /// + /// Defaults to - retrying is opt-in per invocation as blind retry is only safe where the caller knows the underlying operation is safe to repeat (e.g. read-only, + /// or otherwise idempotent). Where and is not specified, a default retry pipeline (3 attempts, exponential backoff, starting at 2 seconds) + /// is used; see for the invocation detail. + /// Only an exception already classified as transient is ever retried; any other exception is always rethrown immediately, unretried, before being passed to the existing + /// handling. + public bool RetryOnTransient { get; init; } = false; + + /// + /// Gets or sets an explicit (of ) to use in place of the default retry pipeline, where is . + /// + /// Only consulted where is ; otherwise ignored. This governs only the retry timing/attempts/backoff - the classification of what is + /// considered transient (and therefore ever retried) always remains . + public ResiliencePipeline? RetryResiliencePipeline { get; init; } } \ No newline at end of file diff --git a/src/CoreEx.Database/Abstractions/DatabaseInvoker.cs b/src/CoreEx.Database/Abstractions/DatabaseInvoker.cs index b6f92063d..967e135b6 100644 --- a/src/CoreEx.Database/Abstractions/DatabaseInvoker.cs +++ b/src/CoreEx.Database/Abstractions/DatabaseInvoker.cs @@ -4,7 +4,13 @@ namespace CoreEx.Database.Abstractions; /// Provides the standard invoker functionality. /// /// Catches any unhandled and invokes to handle (where ) is -/// before bubbling up. +/// before bubbling up. +/// Where is , also retries the invocation - via where supplied, otherwise a +/// default pipeline - for any exception classifies as transient. Where retries are exhausted and the +/// invocation's result type is itself an (ROP), the conversion is applied directly and returned as a failure with no exception thrown at all - avoiding both the cost of an +/// unnecessary throw/catch round-trip and any need to synthesize a wrapping exception purely to carry the converted (e.g. /) error back out; +/// only a non-ROP result type, or one where no conversion applies, results in a single unwrapped throw (never an ) of the final (possibly retried) +/// exception. public abstract class DatabaseInvoker : InvokerBase { /// @@ -12,7 +18,9 @@ protected override async Task OnInvokeAsync(InvokerTracer trac { try { - return await base.OnInvokeAsync(tracer, database, dbArgs, func, cancellationToken).ConfigureAwait(false); + return dbArgs.RetryOnTransient + ? await InvokeWithRetryAsync(tracer, database, dbArgs, func, cancellationToken).ConfigureAwait(false) + : await base.OnInvokeAsync(tracer, database, dbArgs, func, cancellationToken).ConfigureAwait(false); } catch (DbException dbex) when (dbArgs.TransformException) { @@ -33,6 +41,114 @@ protected override async Task OnInvokeAsync(InvokerTracer trac } } + /// + /// Executes under the retry pipeline - the caller-supplied where + /// specified; otherwise a cached-per- default. + /// + /// Only a that classifies as transient is ever retried; any other exception is rethrown immediately, + /// unretried, and handled by the existing logic in as before. + private static async Task InvokeWithRetryAsync(InvokerTracer tracer, IDatabase database, DatabaseArgs dbArgs, Func> func, CancellationToken cancellationToken) + { + var context = ResilienceContextPool.Shared.Get(cancellationToken); + + try + { + context.Properties.Set(ResilienceOwner.PropertyKey, database); + + // Where an explicit (non-generic, payload-less) pipeline has been supplied, carry the successful value out via a captured local as Result itself carries no payload. + if (dbArgs.RetryResiliencePipeline is not null) + { + TResult? value = default; + + var result = await dbArgs.RetryResiliencePipeline.ExecuteAsync(async rc => + { + try + { + value = await func(tracer, dbArgs, rc.CancellationToken).ConfigureAwait(false); + return Result.Success; + } + catch (DbException dbex) when (database.IsTransientException(dbex)) + { + return Result.Fail(dbex); + } + }, context).ConfigureAwait(false); + + if (result.IsFailure) + { + if (TryConvertExhaustedRetryToResult(result.Error, database, dbArgs, tracer, out TResult? res)) + return res; + + ExceptionDispatchInfo.Capture(result.Error).Throw(); + } + + return value!; + } + + // Otherwise, use the default, cached-per-TResult pipeline. + var typedResult = await DefaultRetryPipeline.Instance.ExecuteAsync(async rc => + { + try + { + return new Result(await func(tracer, dbArgs, rc.CancellationToken).ConfigureAwait(false)); + } + catch (DbException dbex) when (database.IsTransientException(dbex)) + { + return new Result(dbex); + } + }, context).ConfigureAwait(false); + + if (typedResult.IsFailure) + { + if (TryConvertExhaustedRetryToResult(typedResult.Error, database, dbArgs, tracer, out TResult? res)) + return res; + + ExceptionDispatchInfo.Capture(typedResult.Error).Throw(); + } + + return typedResult.Value; + } + finally + { + ResilienceContextPool.Shared.Return(context); + } + } + + /// + /// Attempts to convert an exhausted-retry directly into a failure - without throwing/catching - where + /// applies, converts it, and is itself an (ROP) capable of carrying that conversion as a failure. + /// + /// This exists specifically to avoid an otherwise unnecessary throw/catch round-trip back through purely to reapply a conversion that can be performed + /// here directly - Result's entire reason for being is to avoid the cost (and, for types, the loss of a natural returned-vs-thrown distinction) of exceptions for + /// anticipated/expected failure paths, so an exhausted-retry failure destined for a -shaped should never need to be thrown at all. + /// Returns - leaving the caller to throw unwrapped (see ) - for every other case: a non- + /// error, disabled, no conversion available, or a non-ROP ; that single throw is then handled identically to a + /// non-retried failure by the existing handling in . + private static bool TryConvertExhaustedRetryToResult(Exception error, IDatabase database, DatabaseArgs dbArgs, InvokerTracer tracer, [NotNullWhen(true)] out TResult? result) + { + result = default; + + if (!dbArgs.TransformException || error is not DbException dbex) + return false; + + var hex = database.HandleDbException(dbex); + if (hex is null) + return false; + + if (tracer.Logger is not null && tracer.Logger.IsEnabled(LogLevel.Debug)) + tracer.Logger.LogDebug(dbex, "Database exception converted to '{ExceptionType}': {Message} [DatabaseId: {DatabaseId}]", hex.GetType().Name, hex.Message, database.DatabaseId); + + return ExtendedException.TryConvertExceptionToResult(hex, out result); + } + + /// + /// Caches the pipeline per closed , avoiding rebuilding it on + /// every retry-enabled invocation. + /// + private static class DefaultRetryPipeline + { + public static readonly ResiliencePipeline> Instance = DatabaseInvokerResiliency.CreateDefaultRetry(); + } + /// /// Provides standardized database transaction handling for a unit-of-work, including support for nested transactions via save-points, and outbox/event publishing where supported. /// diff --git a/src/CoreEx.Database/Abstractions/DatabaseInvokerResiliency.cs b/src/CoreEx.Database/Abstractions/DatabaseInvokerResiliency.cs new file mode 100644 index 000000000..d300e9c33 --- /dev/null +++ b/src/CoreEx.Database/Abstractions/DatabaseInvokerResiliency.cs @@ -0,0 +1,22 @@ +namespace CoreEx.Database.Abstractions; + +/// +/// Provides the default wiring used by for its opt-in support. +/// +public static class DatabaseInvokerResiliency +{ + /// + /// Creates the default used by where is and no + /// override has been supplied. + /// + /// The invocation result . + /// The delay between retry attempts. + /// The maximum number of retry attempts. + /// The strategy. + /// A configured instance. + /// Classification of what is transient is deliberately not performed here via ShouldHandle - see , which only ever surfaces a + /// failure to this pipeline once has already confirmed the causing exception is transient; anything reaching this pipeline as a failure is + /// therefore, by construction, always worth retrying. + public static ResiliencePipeline> CreateDefaultRetry(TimeSpan? delay = null, int maxRetryAttempts = 3, DelayBackoffType backoffType = DelayBackoffType.Exponential) + => RetryResiliency.Create>(result => result.IsFailure, database => database.Logger ?? NullLogger.Instance, delay, maxRetryAttempts, backoffType); +} diff --git a/src/CoreEx.Database/Database.cs b/src/CoreEx.Database/Database.cs index 5661daa3e..8075a1c23 100644 --- a/src/CoreEx.Database/Database.cs +++ b/src/CoreEx.Database/Database.cs @@ -117,7 +117,7 @@ public async Task GetConnectionAsync(CancellationToken cancellation } catch (Exception ex) { - if (Logger is not null && Logger.IsEnabled(LogLevel.Error)) + if (!(ex.IsCanceled() && cancellationToken.IsCancellationRequested) && Logger is not null && Logger.IsEnabled(LogLevel.Error)) Logger.LogError(ex, "Error occurred whilst opening the database connection. [DatabaseId: {DatabaseId}]", DatabaseId); throw; @@ -153,6 +153,18 @@ public async Task GetConnectionAsync(CancellationToken cancellation /// Where overriding and the is not specifically handled then invoke the base to ensure any standard handling is executed. protected virtual Exception? OnDbException(DbException dbex) => null; + /// + public bool IsTransientException(Exception exception) => OnIsTransientException(exception); + + /// + /// Provides the transient classification as a result of . + /// + /// The . + /// where the is considered transient (retryable); otherwise, . + /// Provides an opportunity to classify provider-specific error codes as transient. + /// Where overriding and the is not specifically classified then invoke the base to ensure any standard handling is executed. + protected virtual bool OnIsTransientException(Exception exception) => false; + /// /// Gets the next (monotonic counter) save-point name. /// diff --git a/src/CoreEx.Database/GlobalUsing.cs b/src/CoreEx.Database/GlobalUsing.cs index 3906a45f4..c97e48273 100644 --- a/src/CoreEx.Database/GlobalUsing.cs +++ b/src/CoreEx.Database/GlobalUsing.cs @@ -20,14 +20,17 @@ global using Microsoft.Extensions.Configuration; global using Microsoft.Extensions.Diagnostics.HealthChecks; global using Microsoft.Extensions.Logging; +global using Microsoft.Extensions.Logging.Abstractions; global using OpenTelemetry; global using Polly; global using System.Collections; global using System.Data; global using System.Data.Common; global using System.Diagnostics; +global using System.Diagnostics.CodeAnalysis; global using System.Globalization; global using System.Linq.Dynamic.Core; global using System.Reflection; +global using System.Runtime.ExceptionServices; global using System.Text; global using System.Text.Json; \ No newline at end of file diff --git a/src/CoreEx.Database/IDatabase.cs b/src/CoreEx.Database/IDatabase.cs index c35979be4..052eaaccb 100644 --- a/src/CoreEx.Database/IDatabase.cs +++ b/src/CoreEx.Database/IDatabase.cs @@ -107,6 +107,15 @@ public interface IDatabase /// Provides an opportunity to inspect and convert the exception before it continues to bubble. Exception? HandleDbException(DbException dbex); + /// + /// Determines whether the specified represents a transient (retryable) database failure. + /// + /// The . + /// where the is considered transient, that is, a retry of the same operation could reasonably be expected to succeed + /// (e.g. a deadlock, lock timeout, or a provider-specific throttling/failover condition); otherwise, . + /// Defaults to so that only providers (or callers) that have explicitly classified their own transient error codes will opt in. + bool IsTransientException(Exception exception) => false; + /// /// Creates a new database parameter. /// diff --git a/src/CoreEx.Database/Outbox/DatabaseOutboxConfiguration.cs b/src/CoreEx.Database/Outbox/DatabaseOutboxConfiguration.cs new file mode 100644 index 000000000..17d870656 --- /dev/null +++ b/src/CoreEx.Database/Outbox/DatabaseOutboxConfiguration.cs @@ -0,0 +1,20 @@ +namespace CoreEx.Database.Outbox; + +/// +/// Provides configuration shared by relational outbox publishers and relays. +/// +public static class DatabaseOutboxConfiguration +{ + /// + /// Gets the configuration key for the partition size shared by relational outbox publishers and relays. + /// + public const string PartitionSizeConfigurationKey = "CoreEx:Host:Outbox:PartitionSize"; + + /// + /// Gets the configured relational outbox partition size, or the default when not configured. + /// + /// The optional configuration. + /// The validated partition size. + public static int GetPartitionSize(IConfiguration? configuration = null) + => PartitionKey.ValidatePartitionSize(Internal.GetConfigurationValue(PartitionSizeConfigurationKey, PartitionKey.DefaultPartitionSize, configuration)); +} diff --git a/src/CoreEx.Database/Outbox/DatabaseOutboxPublisherBase.cs b/src/CoreEx.Database/Outbox/DatabaseOutboxPublisherBase.cs index 43adde0d5..81a50b4a8 100644 --- a/src/CoreEx.Database/Outbox/DatabaseOutboxPublisherBase.cs +++ b/src/CoreEx.Database/Outbox/DatabaseOutboxPublisherBase.cs @@ -32,6 +32,6 @@ public abstract class DatabaseOutboxPublisherBase(TDatabase database, /// /// Gets or sets the partition size to use when calculating the partition id for each event. /// - /// This is used to ensure that events with the same partition key are stored in the same partition, which guarantees that events are processed in order within a partition. + /// DI-registered publishers resolve this from ; the same key is used by relational relay hosts. Explicit per-instance settings outside DI must match that shared value. public int PartitionSize { get; set; } = PartitionKey.DefaultPartitionSize; -} \ No newline at end of file +} diff --git a/src/CoreEx.Database/Outbox/DatabaseOutboxRelayBase.cs b/src/CoreEx.Database/Outbox/DatabaseOutboxRelayBase.cs index f43361d99..9ca5334c2 100644 --- a/src/CoreEx.Database/Outbox/DatabaseOutboxRelayBase.cs +++ b/src/CoreEx.Database/Outbox/DatabaseOutboxRelayBase.cs @@ -46,6 +46,12 @@ public DatabaseOutboxRelayBase(TDatabase database, IEventPublisher eventPublishe /// protected DatabaseOutboxRelayInvoker Invoker => _invoker.Value; + /// + /// Gets or sets the maximum duration allowed to cancel (release) a claimed batch following a relay failure. + /// + /// Defaults to 30 seconds. This is independent of the relay cancellation token as that is likely already cancelled where the failure was due to exceeding the lease duration. + public TimeSpan CancelBatchTimeout { get; set; } = TimeSpan.FromSeconds(30); + /// /// Gets or sets the used to claim the next batch of events from the . /// @@ -129,13 +135,17 @@ public async Task RelayAsync(DatabaseOutboxRelayArgs args, CancellationTok result.ThrowOnError(); } + catch (Exception ex) when (ex.IsCanceled() && cancellationToken.IsCancellationRequested) + { + // The caller (e.g. host shutdown) requested cancellation; stop processing and let the cancellation flow. + throw; + } catch (Exception ex) when (ex.IsCanceled()) { + // The partition exceeded the lease duration; this is a partition-level failure (already observed by any resiliency pipeline), not a shutdown - rethrowing a cancellation here would be + // interpreted by the hosting service as a stop request and silently swallowed. Continue on to the next partition; the claim has been released (or will expire with the lease). if (Logger?.IsEnabled(LogLevel.Warning) is true) - Logger.LogWarning("The relay operation for partition '{PartitionId}' was cancelled due to exceeding lease-duration timeout; cancellation exception will continue to throw.", partitionId); - - // Keep throwing as the cancellation is likely to be due to exceeding the lease duration which is a serious failure that should be surfaced and not treated as a transient exception. - throw; + Logger.LogWarning("The relay operation for partition '{PartitionId}' was cancelled due to exceeding the lease duration of {LeaseDuration}; consider increasing the lease duration and/or reducing the batch size.", partitionId, args.LeaseDuration); } catch (Exception ex) { @@ -208,10 +218,12 @@ await Invoker.InvokeAsync(this, async (tracer, cancellationToken) => } catch (Exception ex) { - // Cancel the batch; guard against a secondary failure masking the original exception. + // Cancel the batch; guard against a secondary failure masking the original exception. A separate bounded token is used as the original is likely already cancelled (e.g. the lease-duration + // timeout elapsed) and would otherwise prevent the claim from being released, leaving the events locked until the database lease expires. try { - await CancelBatchAsync(args, leaseId, cancellationToken).ConfigureAwait(false); + using var cancelBatchCts = new CancellationTokenSource(CancelBatchTimeout); + await CancelBatchAsync(args, leaseId, cancelBatchCts.Token).ConfigureAwait(false); if (Logger?.IsEnabled(LogLevel.Debug) is true) Logger.LogDebug("Outbox batch was cancelled due to error: {Error}", ex.Message); @@ -308,6 +320,8 @@ protected virtual Task CancelBatchAsync(DatabaseOutboxRelayArgs args, Guid lease /// /// The . /// where the exception is considered transient; otherwise, . - /// For example, a timeout or deadlock exception that may occur during the claim of the batch and is expected to be transient in nature. - protected virtual bool IsTransientException(Exception exception) => false; + /// For example, a timeout or deadlock exception that may occur during the claim of the batch and is expected to be transient in nature. + /// Defaults to so the classification is reused wherever the is used, not just here; override to add relay-specific + /// classification in addition. + protected virtual bool IsTransientException(Exception exception) => Database.IsTransientException(exception); } \ No newline at end of file diff --git a/src/CoreEx.Database/Outbox/DatabaseOutboxRelayHostedServiceBase.cs b/src/CoreEx.Database/Outbox/DatabaseOutboxRelayHostedServiceBase.cs index a334f3ae4..b6646c3f2 100644 --- a/src/CoreEx.Database/Outbox/DatabaseOutboxRelayHostedServiceBase.cs +++ b/src/CoreEx.Database/Outbox/DatabaseOutboxRelayHostedServiceBase.cs @@ -48,7 +48,7 @@ public DatabaseOutboxRelayHostedServiceBase(IServiceProvider serviceProvider, IL /// /// Gets or sets the partition size. /// - /// Defaults to . + /// Defaults to and resolves from , which is shared with relational outbox publishers. public int PartitionSize { get; set => field = SetValueWhenStatusIsInitializedOnly(value); } /// @@ -72,7 +72,11 @@ protected async override Task OnInitializeAsync(CancellationToken cancellationTo BatchSize = Internal.GetConfigurationValueWithFallback($"CoreEx:Host:Services:{ServiceConfigurationSectionName}:OutboxRelay:BatchSize", "CoreEx:Host:Services:OutboxRelay:BatchSize", 25, Configuration); LeaseDuration = Internal.GetConfigurationValueWithFallback($"CoreEx:Host:Services:{ServiceConfigurationSectionName}:OutboxRelay:LeaseDuration", "CoreEx:Host:Services:OutboxRelay:LeaseDuration", TimeSpan.FromMinutes(5), Configuration); BackOffDuration = Internal.GetConfigurationValueWithFallback($"CoreEx:Host:Services:{ServiceConfigurationSectionName}:OutboxRelay:BackOffDuration", "CoreEx:Host:Services:OutboxRelay:BackOffDuration", TimeSpan.FromSeconds(5), Configuration); - PartitionSize = Internal.GetConfigurationValueWithFallback($"CoreEx:Host:Services:{ServiceConfigurationSectionName}:OutboxRelay:PartitionSize", "CoreEx:Host:Services:OutboxRelay:PartitionSize", PartitionKey.DefaultPartitionSize, Configuration); + var legacyPartitionSizeKey = $"CoreEx:Host:Services:{ServiceConfigurationSectionName}:OutboxRelay:PartitionSize"; + if (Configuration[legacyPartitionSizeKey] is not null || Configuration["CoreEx:Host:Services:OutboxRelay:PartitionSize"] is not null) + throw new InvalidOperationException($"The relay-specific PartitionSize configuration has been removed. Configure '{DatabaseOutboxConfiguration.PartitionSizeConfigurationKey}' instead so the outbox publisher and relay use the same value."); + + PartitionSize = DatabaseOutboxConfiguration.GetPartitionSize(Configuration); // Default capped at PartitionSize (whatever it resolved to above, default or configured) - PartitionPicker requires perWorkerPartitionCount <= partitionSize; an unconditional literal default // here would silently throw at startup whenever it exceeds the resolved PartitionSize (e.g. the out-of-the-box defaults: PartitionSize=4 but a literal 6 here). @@ -101,4 +105,4 @@ protected override HealthCheckResult OnReportHealthStatus(Dictionary` contains the relay logic and is parameterized by both the `IDatabase` type and the self-referencing relay type, enabling per-deployment SQL statement customization via `SetStatementsByConvention(schemaName?)`. +Relational outbox publishers and relay hosts use the same `CoreEx:Host:Outbox:PartitionSize` configuration value (default `4`, valid range `1`–`256`). Configure the same value in every host that writes or relays the outbox. The relay reports the effective size in its startup log and detailed health data. The former relay-only `CoreEx:Host:Services:OutboxRelay:PartitionSize` setting is rejected with a migration error; it is not a fallback. Keep this value stable while outbox rows are pending. + +Events without a partition key are assigned a stable sentinel before the relational outbox partition hash, so all keyless events share one outbox partition and preserve their relative relay order. The event itself remains keyless. This deliberately serializes keyless events and does not apply to Cosmos DB's native logical partitioning. + ## Key capabilities - ⚛️ **Atomic write**: `DatabaseOutboxPublisherBase.PublishAsync` inserts serialized events into the outbox table within the calling `IUnitOfWork` transaction — no two-phase commit required. @@ -38,4 +42,4 @@ The transactional outbox pattern ensures that domain events are reliably publish - **[`CoreEx.Database`](../README.md)** - `IDatabase` and `DatabaseCommand` are used by relay and publisher implementations to execute outbox SQL. - **[`CoreEx.Events`](../../CoreEx.Events/README.md)** - `IEventPublisher` is the relay's publication target; `EventData` is the serialized event payload stored in the outbox table. - **[`CoreEx.Hosting`](../../CoreEx/Hosting/README.md)** - `TimerHostedServiceBase` is the base class for `DatabaseOutboxRelayHostedServiceBase`, providing timer, pause/resume, and health-check integration. -- **[`CoreEx.Invokers`](../../CoreEx/Invokers/README.md)** - `DatabaseOutboxRelayInvoker` extends `InvokerBase` for relay-batch OpenTelemetry tracing. \ No newline at end of file +- **[`CoreEx.Invokers`](../../CoreEx/Invokers/README.md)** - `DatabaseOutboxRelayInvoker` extends `InvokerBase` for relay-batch OpenTelemetry tracing. diff --git a/src/CoreEx.Database/Templates/EfModelBuilder_cs.hbs b/src/CoreEx.Database/Templates/EfModelBuilder_cs.hbs index d2afb3164..ca6b16c8e 100644 --- a/src/CoreEx.Database/Templates/EfModelBuilder_cs.hbs +++ b/src/CoreEx.Database/Templates/EfModelBuilder_cs.hbs @@ -37,16 +37,16 @@ public partial class {{Domain}}DbContext e.Property(p => p.{{Property}}).HasColumnName("{{Name}}").HasColumnType("{{DbColumn.SqlType2}}"){{#if DbColumn.IsComputed}}.ValueGeneratedOnAddOrUpdate(){{/if}}{{#if DbColumn.IsRowVersionColumn}}.IsRowVersion(){{/if}}{{#if DbColumn.IsCreatedAudit}}.ValueGeneratedOnUpdate(){{/if}}{{#if DbColumn.IsUpdatedAudit}}.ValueGeneratedOnAdd(){{/if}}{{#if ValueConverter}}.HasConversion({{ValueConverter}}){{else}}{{#if DbColumn.IsJsonContent}}{{#ifne Type 'string' 'string?'}}.HasConversion(TypeToJsonStringEfConverter<{{Type}}>.Default, TypeToJsonStringEfComparer<{{Type}}>.Default){{/ifne}}{{/if}}{{/if}}; {{/each}} {{#if HasColumnCreatedBy}} - e.Property(p => p.CreatedBy).HasColumnName("{{ColumnCreatedBy.Name}}").HasColumnType("{{ColumnCreatedBy.DbColumn.SqlType2}}"); + e.Property(p => p.CreatedBy).HasColumnName("{{ColumnCreatedBy.Name}}").HasColumnType("{{ColumnCreatedBy.DbColumn.SqlType2}}").ValueGeneratedOnUpdate(); {{/if}} {{#if HasColumnCreatedOn}} - e.Property(p => p.CreatedOn).HasColumnName("{{ColumnCreatedOn.Name}}").HasColumnType("{{ColumnCreatedOn.DbColumn.SqlType2}}"); + e.Property(p => p.CreatedOn).HasColumnName("{{ColumnCreatedOn.Name}}").HasColumnType("{{ColumnCreatedOn.DbColumn.SqlType2}}").ValueGeneratedOnUpdate(); {{/if}} {{#if HasColumnUpdatedBy}} - e.Property(p => p.UpdatedBy).HasColumnName("{{ColumnUpdatedBy.Name}}").HasColumnType("{{ColumnUpdatedBy.DbColumn.SqlType2}}"); + e.Property(p => p.UpdatedBy).HasColumnName("{{ColumnUpdatedBy.Name}}").HasColumnType("{{ColumnUpdatedBy.DbColumn.SqlType2}}").ValueGeneratedOnAdd(); {{/if}} {{#if HasColumnUpdatedOn}} - e.Property(p => p.UpdatedOn).HasColumnName("{{ColumnUpdatedOn.Name}}").HasColumnType("{{ColumnUpdatedOn.DbColumn.SqlType2}}"); + e.Property(p => p.UpdatedOn).HasColumnName("{{ColumnUpdatedOn.Name}}").HasColumnType("{{ColumnUpdatedOn.DbColumn.SqlType2}}").ValueGeneratedOnAdd(); {{/if}} {{#if HasColumnRowVersion}} e.Property(p => p.ETag).HasColumnName("{{ColumnRowVersion.Name}}").HasColumnType("{{ColumnRowVersion.DbColumn.SqlType2}}").IsRowVersion().HasConversion(ValueConverterBridge.Create(BaseDatabase.RowVersionConverter)); diff --git a/src/CoreEx.Database/Templates/EfModel_cs.hbs b/src/CoreEx.Database/Templates/EfModel_cs.hbs index 92059adb8..aaa959f76 100644 --- a/src/CoreEx.Database/Templates/EfModel_cs.hbs +++ b/src/CoreEx.Database/Templates/EfModel_cs.hbs @@ -18,7 +18,7 @@ public partial class {{EfModelName}} : ReferenceDataModelBase<{{PrimaryKeyIdenti {{/unless}} /// Gets or sets the value of the '{{DbColumn.Name}}' column (type '{{DbColumn.SqlType}}'). - public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#ifeq DbColumn.DotNetType "string"}} = default!;{{/ifeq}}{{/unless}}{{/ifval}} + public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#if DbColumn.IsDotNetTypeAClass}} = default!;{{/if}}{{/unless}}{{/ifval}} {{/each}} } {{/ifne}} @@ -38,7 +38,7 @@ public partial class {{EfModelName}}{{#if HasPrimaryKeyIdentifier}} : ModelBase< {{#if DbColumn.IsPrimaryKey}} /// This is {{#ifeq Parent.DbTable.PrimaryKeyColumns.Count 1}}the primary key{{else}}part of the primary key{{/ifeq}}. {{/if}} - public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#ifeq DbColumn.DotNetType "string"}} = default!;{{/ifeq}}{{/unless}}{{/ifval}} + public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#if DbColumn.IsDotNetTypeAClass}} = default!;{{/if}}{{/unless}}{{/ifval}} {{/each}} {{#if HasColumnIsDeleted}} @@ -59,10 +59,10 @@ public partial class {{EfModelName}}{{#if HasPrimaryKeyIdentifier}} : ModelBase< {{#if DbColumn.IsPrimaryKey}} /// This is {{#ifeq Parent.DbTable.PrimaryKeyColumns.Count 1}}the primary key{{else}}part of the primary key{{/ifeq}}. {{/if}} - public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#ifeq DbColumn.DotNetType "string"}} = default!;{{/ifeq}}{{/unless}}{{/ifval}} + public {{Type}} {{Property}} { get; set; }{{#ifval Default}} = {{Default}};{{else}}{{#unless DbColumn.IsNullable}}{{#if DbColumn.IsDotNetTypeAClass}} = default!;{{/if}}{{/unless}}{{/ifval}} {{/each}} {{/if}} } {{/if}} -#nullable restore \ No newline at end of file +#nullable restore diff --git a/src/CoreEx.DomainDriven/AGENTS.md b/src/CoreEx.DomainDriven/AGENTS.md index 4dcd0646c..b4508dd7e 100644 --- a/src/CoreEx.DomainDriven/AGENTS.md +++ b/src/CoreEx.DomainDriven/AGENTS.md @@ -97,4 +97,3 @@ CoreEx does not provide a native in-process domain-event bus. Use `IUnitOfWork.E - [CoreEx.EntityFrameworkCore](../CoreEx.EntityFrameworkCore/README.md) — persists `Entity`/`Aggregate` types using `PersistenceState`. - [Domain layer](../../samples/docs/domain-layer.md) — real-world aggregate design, mutation guards, integration-event accumulation, and `Result` pipeline usage in the Shopping sample. - [Patterns](../../samples/docs/patterns.md) — aggregate-oriented service patterns, domain event flow, and mutation-state tracking. - diff --git a/src/CoreEx.EntityFrameworkCore/EfDbModelOptions.cs b/src/CoreEx.EntityFrameworkCore/EfDbModelOptions.cs index 6050ac61f..0427cef20 100644 --- a/src/CoreEx.EntityFrameworkCore/EfDbModelOptions.cs +++ b/src/CoreEx.EntityFrameworkCore/EfDbModelOptions.cs @@ -11,6 +11,11 @@ public class EfDbModelOptions where TModel : class private Func? _updateModelMapper; private bool _tenantFilterEnabled; + /// + /// The server-managed properties that are never copied from the updated model into the existing model by default; i.e. these are set (or retained) by the framework, not the caller. + /// + private static readonly IReadOnlyCollection _updateExcludedProperties = [.. GetUpdateExcludedProperties()]; + /// /// Indicates whether and/or is supported for the . /// @@ -239,7 +244,7 @@ public EfDbModelOptions WithOnBeforeCreateOrUpdate(FuncThe to support fluent-style method-chaining. /// /// The function takes two parameters: the updated model (first parameter) and the existing model (second parameter) and returns a indicating whether any changes were made. - /// This enables custom mapping logic to be specified for update operations; otherwise, by default, is used internally. + /// This enables custom mapping logic to be specified for update operations; otherwise, by default, is used internally, excluding the server-managed (created and updated) and properties (these are set by the framework and are not caller-updatable; a custom mapper must honour the same). public EfDbModelOptions WithUpdateModelMapper(Func updateModelMapper) { _updateModelMapper = updateModelMapper.ThrowIfNull(); @@ -255,8 +260,25 @@ public EfDbModelOptions WithUpdateModelMapper(Func internal bool MapModelForUpdate(TModel update, TModel existing) { if (_updateModelMapper is null) - return Metadata.RuntimeMetadata.TryCopyInto(update, existing); + return Metadata.RuntimeMetadata.TryCopyInto(update, existing, _updateExcludedProperties); else return _updateModelMapper(update, existing); } + + /// + /// Gets the server-managed property names applicable to the . + /// + private static IEnumerable GetUpdateExcludedProperties() + { + if (typeof(IReadOnlyChangeLogEx).IsAssignableFrom(typeof(TModel))) + { + yield return nameof(IReadOnlyChangeLogEx.CreatedBy); + yield return nameof(IReadOnlyChangeLogEx.CreatedOn); + yield return nameof(IReadOnlyChangeLogEx.UpdatedBy); + yield return nameof(IReadOnlyChangeLogEx.UpdatedOn); + } + + if (typeof(IReadOnlyTenantId).IsAssignableFrom(typeof(TModel))) + yield return nameof(IReadOnlyTenantId.TenantId); + } } \ No newline at end of file diff --git a/src/CoreEx.EntityFrameworkCore/EfDbReferenceData.cs b/src/CoreEx.EntityFrameworkCore/EfDbReferenceData.cs new file mode 100644 index 000000000..0de37c4e0 --- /dev/null +++ b/src/CoreEx.EntityFrameworkCore/EfDbReferenceData.cs @@ -0,0 +1,180 @@ +namespace CoreEx.EntityFrameworkCore; + +/// +/// Provides standardized operations for Entity Framework Core. +/// +/// This encapsulates common CRUD operations enabling consistent handling of reference data. +public static class EfDbReferenceData +{ + /// + /// Creates the specified in the database. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The EF database model. + /// The value to create. + /// The mapper. + /// The cancellation token. + /// The containing the created value. + public static async Task>> CreateAsync(EfDbModel ef, TRef value, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : ReferenceDataModelBase + where TMapper : class, IBiDirectionMapper + { + var model = mapper.To.Map(value); + var pr = CheckPropertySupport(ef, model); + if (pr.IsFailure) + return pr; + + model.Id = await Runtime.GenerateIdentifierAsync().ConfigureAwait(false); + model.IsActive = false; + + var result = await ef.CreateWithResultAsync(model, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Updates the specified in the database. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The EF database model. + /// The identifier of the value to update. + /// The value to update. + /// The mapper. + /// The cancellation token. + /// The containing the updated value. + public static async Task>> UpdateAsync(EfDbModel ef, TId id, TRef value, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : ReferenceDataModelBase + where TMapper : class, IBiDirectionMapper + { + var existing = await ef.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.AsResult(); + + var model = mapper.To.Map(value); + var pr = CheckPropertySupport(ef, model); + if (pr.IsFailure) + return pr; + + model.Id = existing.Value.Id; + model.Code = existing.Value.Code; // Code is immutable and cannot be updated. + model.IsActive = existing.Value.IsActive; // IsActive can only be updated via specific methods. + + var result = await ef.UpdateWithResultAsync(model, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Checks whether the specified has values where the underlying property is not supported. + /// + private static Result CheckPropertySupport(EfDbModel ef, TModel model) where TModel : ReferenceDataModelBase + => CheckWhetherPropertyIsSupported(ef, nameof(ReferenceDataModelBase<>.Description), model.Description) + .Then(() => CheckWhetherPropertyIsSupported(ef, nameof(ReferenceDataModelBase<>.StartsOn), model.StartsOn)) + .Then(() => CheckWhetherPropertyIsSupported(ef, nameof(ReferenceDataModelBase<>.EndsOn), model.EndsOn)); + + /// + /// Checks whether the specified is supported by the . + /// + private static Result CheckWhetherPropertyIsSupported(EfDbModel ef, string propertyName, T value) where TModel : class + { + if (value is null) + return Result.Success; + + var entityType = ef.EfDb.DbContext.Model.FindEntityType(typeof(TModel)); + var property = entityType?.FindProperty(propertyName); + + if (property is null) + return Result.Fail($"{propertyName.ToSentenceCase()} is not currently supported and as such cannot be set.", c => c.WithErrorCode("not-supported")); + + return Result.Success; + } + + /// + /// Activates the specified in the database. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The EF database model. + /// The identifier of the value to activate. + /// The mapper. + /// The cancellation token. + /// The containing the activated value. + public static async Task>> ActivateAsync(EfDbModel ef, TId id, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : ReferenceDataModelBase + where TMapper : class, IBiDirectionMapper + { + var existing = await ef.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.AsResult(); + + if (existing.Value.IsActive) + return new DataResult(mapper.From.Map(existing.Value)!, false); + + existing.Value.IsActive = true; + var result = await ef.UpdateWithResultAsync(existing.Value, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Deactivates the specified in the database. + /// + /// The identifier . + /// The . + /// The model . + /// The . + /// The EF database model. + /// The identifier of the value to deactivate. + /// The mapper. + /// The cancellation token. + /// The containing the deactivated value. + public static async Task>> DeactivateAsync(EfDbModel ef, TId id, TMapper mapper, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : ReferenceDataModelBase + where TMapper : class, IBiDirectionMapper + { + var existing = await ef.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.AsResult(); + + if (!existing.Value.IsActive) + return new DataResult(mapper.From.Map(existing.Value)!, false); + + existing.Value.IsActive = false; + var result = await ef.UpdateWithResultAsync(existing.Value, cancellationToken).ConfigureAwait(false); + return result.ThenAs(dr => new DataResult(mapper.From.Map(dr.Value)!, dr.WasMutated)); + } + + /// + /// Deletes the specified from the database. + /// + /// The identifier . + /// The . + /// The model . + /// The EF database model. + /// The identifier of the value to delete. + /// The cancellation token. + /// A + /// A delete is considered idempotent and as such no will be thrown. The returning is informational only. + public static async Task> DeleteAsync(EfDbModel ef, TId id, CancellationToken cancellationToken = default) + where TRef : class, IReferenceData + where TModel : ReferenceDataModelBase + { + var existing = await ef.GetWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + if (existing.IsFailure) + return existing.IsNotFoundError ? DataResult.False : existing.AsResult(); + + if (existing.Value.IsActive) + return Result.Fail("An active reference data value cannot be deleted.", c => c.WithErrorCode("cannot-delete-active")); + + return await ef.DeleteWithResultAsync(CompositeKey.Create(id), cancellationToken).ConfigureAwait(false); + } +} diff --git a/src/CoreEx.EntityFrameworkCore/GlobalUsing.cs b/src/CoreEx.EntityFrameworkCore/GlobalUsing.cs index 3340ead44..fe3fb49c2 100644 --- a/src/CoreEx.EntityFrameworkCore/GlobalUsing.cs +++ b/src/CoreEx.EntityFrameworkCore/GlobalUsing.cs @@ -1,6 +1,7 @@ global using CoreEx; global using CoreEx.Abstractions; global using CoreEx.Data; +global using CoreEx.Data.Models; global using CoreEx.Database; global using CoreEx.Database.Abstractions; global using CoreEx.Entities; @@ -8,6 +9,7 @@ global using CoreEx.Invokers; global using CoreEx.Json; global using CoreEx.Mapping; +global using CoreEx.RefData; global using CoreEx.Results; global using Microsoft.EntityFrameworkCore; global using Microsoft.EntityFrameworkCore.ChangeTracking; @@ -18,4 +20,4 @@ global using System.Diagnostics.CodeAnalysis; global using System.Linq; global using System.Reflection; -global using System.Text.Json; \ No newline at end of file +global using System.Text.Json; diff --git a/src/CoreEx.EntityFrameworkCore/README.md b/src/CoreEx.EntityFrameworkCore/README.md index fe55ef805..753c63e27 100644 --- a/src/CoreEx.EntityFrameworkCore/README.md +++ b/src/CoreEx.EntityFrameworkCore/README.md @@ -33,7 +33,7 @@ The central type is `EfDb`, which holds the `DbContext`, bridges its | **[`EfDbExtensions`](./EfDbExtensions.cs)** | `IQueryable` extensions: `ToMappedItemsResultAsync`, `ToItemsResultAsync`, `ToMappedCollectionAsync`, `BuildQuery(QueryArgs, QueryArgsConfig)`. | | **[`EfDbArgs`](./EfDbArgs.cs)** | Per-operation options: `QueryTracking`, `ClearChangeTrackerAfterGet`, `SaveChanges`, `BypassFilters` (plus inherited `Refresh`/`TransformException`); defaults sourced from `EfDbModelOptions.Args` then `EfDb.Options.Args`. | | **[`EfDbOptions`](./EfDbOptions.cs)** | Instance-level options for `EfDb`: default `EfDbArgs`, per-model options registry via `GetOrAddModelOptions()`. | -| **[`EfDbModelOptions`](./EfDbModelOptions.cs)** | Per-model configuration: `WithArgs`, `WithGetKey`, `WithFilter`/`WithTenantFilter`/`WithLogicalDeleteFilter`, `WithOnBeforeCreateOrUpdate`, `WithUpdateModelMapper`. | +| **[`EfDbModelOptions`](./EfDbModelOptions.cs)** | Per-model configuration: `WithArgs`, `WithGetKey`, `WithFilter`/`WithTenantFilter`/`WithLogicalDeleteFilter`, `WithOnBeforeCreateOrUpdate`, `WithUpdateModelMapper` (the default update mapping copies the candidate over the existing model but excludes the server-managed change-log `Created*` and tenant properties, so they cannot be overwritten). | | **[`EfDbInvoker`](./EfDbInvoker.cs)** | `InvokerBase` emitting structured log entries for every EfDb operation (tracing intentionally disabled); `Default` singleton used by `EfDb`. | | **[`ValueConverterBridge`](./Converters/ValueConverterBridgeT2.cs)** | EF Core `ValueConverter` that delegates to a CoreEx `IConverter`, bridging CoreEx converter types into EF model configuration. | | **[`JsonElementStringEfConverter`](./Converters/JsonElementStringEfConverter.cs)** | EF Core `ValueConverter` serializing `JsonElement` values to/from `string` for storing JSON fragments in a text column. | diff --git a/src/CoreEx.Events/AGENTS.md b/src/CoreEx.Events/AGENTS.md index 97cc6564e..4af33ff0d 100644 --- a/src/CoreEx.Events/AGENTS.md +++ b/src/CoreEx.Events/AGENTS.md @@ -61,16 +61,15 @@ See [CoreEx.Azure.Messaging.ServiceBus](../CoreEx.Azure.Messaging.ServiceBus/REA ## DestinationProvider -Implement `IDestinationProvider` to derive topic/queue names from `EventData`. Return `null` to fall back to the next registered provider. +An `IDestinationProvider` derives the topic/queue name for each published message. Register exactly one; the built-ins cover the common cases: + +- `AddNamedDestinationProvider()` (**default for new solutions**) — events go to the single shared destination (topic) read from `CoreEx:Events:Destination`; commands go to a per-target-domain queue named `{Destination}-{domain}` (e.g. `contoso-products`), so only the named domain consumes them. A command with no destination domain is self-addressed to the host domain. Both explicit and defaulted command domains use invariant lowercase, keeping destination-specific session policy lookups consistent; the base destination and explicitly supplied destination names are unchanged. +- `AddFixedDestinationProvider("name")` — every message (events and commands) goes to one fixed destination. + +For anything else, implement `IDestinationProvider` (`CreateFrom(EventData, bool)`, `CreateFrom(string, bool)`, `CreateNew(MessageType, string?, bool)`) and register it in place of the built-ins. ```csharp -public class MyDestinationProvider : IDestinationProvider -{ - public string? GetDestination(EventData @event) - => @event.Subject?.StartsWith("contoso.orders") == true - ? "orders-topic" - : null; -} +builder.Services.AddNamedDestinationProvider(); // Destination defaults from CoreEx:Events:Destination. ``` ## Do Not diff --git a/src/CoreEx.Events/CoreExEventsExtensions.DependencyInjection.cs b/src/CoreEx.Events/CoreExEventsExtensions.DependencyInjection.cs index f3461cc1d..f864eb37d 100644 --- a/src/CoreEx.Events/CoreExEventsExtensions.DependencyInjection.cs +++ b/src/CoreEx.Events/CoreExEventsExtensions.DependencyInjection.cs @@ -32,6 +32,16 @@ public static IServiceCollection AddEventFormatter(this IServiceCollection servi public static IServiceCollection AddFixedDestinationProvider(this IServiceCollection services, string destination) => services.ThrowIfNull().AddSingleton(new FixedDestinationProvider { Destination = destination.ThrowIfNullOrEmpty() }); + /// + /// Adds a singleton service for the as the . + /// + /// The . + /// The optional base destination name (i.e. topic); where the 'CoreEx:Events:Destination' configuration setting is used. + /// The for fluent-style method-chaining. + /// Events are sent to the (shared topic) whilst commands are sent to a per-domain destination (queue). + public static IServiceCollection AddNamedDestinationProvider(this IServiceCollection services, string? destination = null) + => services.ThrowIfNull().AddSingleton(destination is null ? new NamedDestinationProvider() : new NamedDestinationProvider { Destination = destination }); + /// /// Adds a singleton service. /// diff --git a/src/CoreEx.Events/EventData.cs b/src/CoreEx.Events/EventData.cs index 143aebd21..ec8859037 100644 --- a/src/CoreEx.Events/EventData.cs +++ b/src/CoreEx.Events/EventData.cs @@ -37,6 +37,12 @@ public EventData() /// /// Gets or sets the domain (DDD bounded context) name. /// + /// Represents the domain that owns the message contract (the and vocabulary), and is the first segment of the (see ). This is not the source of the message; that is conveyed by , which is derived from the host settings. + /// + /// : the publishing domain; optional and defaults to the current host domain () and is not used for destination resolution. + /// : the target domain that is to action the command (see ), which is also used to resolve the destination (see ). Where not specified the command is self-addressed; i.e. defaults to the current host domain. + /// + /// Where the is parsed (see ) this reflects the first segment of the received . public string? DomainName { get; set; } /// diff --git a/src/CoreEx.Events/EventFormatter.cs b/src/CoreEx.Events/EventFormatter.cs index a1238da85..8bf68ac62 100644 --- a/src/CoreEx.Events/EventFormatter.cs +++ b/src/CoreEx.Events/EventFormatter.cs @@ -110,7 +110,7 @@ public EventFormatter(IHostSettings? hostSettings = null) /// /// Gets or sets the default Domain (DDD) name to be used where not specified on the (see ). /// - /// This defaults to the . + /// This defaults to the ; therefore an event defaults to being owned by the publishing (current) domain, and a without a domain is self-addressed. public string? DomainName { get; set; } /// @@ -137,7 +137,11 @@ public virtual EventData Format(EventData @event) if (@event.Title is null) { - @event.Title = ApplyInvariantCasing(Cleaner.Clean(string.Join('.', [.. tpa, @event.DomainName ?? DomainName ?? _defaultSegment, @event.Entity ?? _defaultSegment, @event.Action ?? _defaultSegment]), casing: StringCase.None), TitleCase); + var domainName = string.IsNullOrEmpty(@event.DomainName) ? DomainName : @event.DomainName; + if (string.IsNullOrEmpty(domainName) && @event.MessageType == MessageType.Command) + throw new InvalidOperationException($"A {nameof(EventData)}.{nameof(EventData.DomainName)} is required for a '{nameof(MessageType.Command)}' where the {nameof(EventFormatter)}.{nameof(DomainName)} is not available to default the (self-addressed) target domain."); + + @event.Title = ApplyInvariantCasing(Cleaner.Clean(string.Join('.', [.. tpa, domainName ?? _defaultSegment, @event.Entity ?? _defaultSegment, @event.Action ?? _defaultSegment]), casing: StringCase.None), TitleCase); if (@event.DataSchemaVersion is not null) @event.Title += $".v{@event.DataSchemaVersion.Major}"; } @@ -214,9 +218,9 @@ public virtual EventData Parse(EventData @event) return @event; // At this point we believe we have at least 3 segments: DomainName, Entity, Action - which is enough to carry on. - @event.DomainName = segments[0]; - @event.Entity = segments[1]; - @event.Action = segments[2]; + @event.DomainName ??= segments[0]; + @event.Entity ??= segments[1]; + @event.Action ??= segments[2]; // The fourth segment is optional and represents the DataSchemaVersion. if (segments.Length >= 4) diff --git a/src/CoreEx.Events/GlobalUsing.cs b/src/CoreEx.Events/GlobalUsing.cs index 202bfd87d..b124bc958 100644 --- a/src/CoreEx.Events/GlobalUsing.cs +++ b/src/CoreEx.Events/GlobalUsing.cs @@ -15,6 +15,7 @@ global using CoreEx.Results; global using CoreEx.Schemas; global using CoreEx.Security; +global using CoreEx.Text; global using CoreEx.Validation; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Logging; @@ -33,4 +34,4 @@ global using System.Text.Json; global using System.Text.Json.Nodes; global using System.Text.Json.Serialization; -global using System.Text.RegularExpressions; \ No newline at end of file +global using System.Text.RegularExpressions; diff --git a/src/CoreEx.Events/Publishing/FixedDestinationProvider.cs b/src/CoreEx.Events/Publishing/FixedDestinationProvider.cs index 99462f1f9..ef2810066 100644 --- a/src/CoreEx.Events/Publishing/FixedDestinationProvider.cs +++ b/src/CoreEx.Events/Publishing/FixedDestinationProvider.cs @@ -3,7 +3,7 @@ namespace CoreEx.Events.Publishing; /// /// Provides a where the same (fixed) is used regardless of contents; i.e. all messages are published to a single centralized destination. /// -/// The will default to the value of the 'CoreEx.Events:Destination' configuration setting or 'default' as a fallback. +/// The will default to the value of the 'CoreEx:Events:Destination' configuration setting or 'default' as a fallback. public class FixedDestinationProvider : IDestinationProvider { private string? _destination = null; diff --git a/src/CoreEx.Events/Publishing/IDestinationProvider.cs b/src/CoreEx.Events/Publishing/IDestinationProvider.cs index 6f276dc74..ff6f65b34 100644 --- a/src/CoreEx.Events/Publishing/IDestinationProvider.cs +++ b/src/CoreEx.Events/Publishing/IDestinationProvider.cs @@ -25,7 +25,7 @@ public interface IDestinationProvider /// /// Creates the destination name using the specified parameters. /// - /// The recipient domain (DDD) name. + /// The target (recipient) domain (DDD) name; applicable to messages. /// The . /// Indicates whether to provide a dead-letter specific destination or not. /// The resulting destination name. diff --git a/src/CoreEx.Events/Publishing/NamedDestinationProvider.cs b/src/CoreEx.Events/Publishing/NamedDestinationProvider.cs new file mode 100644 index 000000000..ec719e551 --- /dev/null +++ b/src/CoreEx.Events/Publishing/NamedDestinationProvider.cs @@ -0,0 +1,78 @@ +namespace CoreEx.Events.Publishing; + +/// +/// Provides an that publishes messages to a single shared (i.e. topic), and messages to a destination (i.e. queue) specific to the target . +/// +/// +/// Events are fire-and-forget and intended for any interested subscriber, so share a single destination. A command is intended for a single consumer (the target domain), so is given its own destination. +/// The destination name is created as follows: +/// +/// : the ; the is ignored. +/// : the followed by the and the (the target domain). Where the domain name is or empty the command is +/// self-addressed; that is, the of the current host is used (matching the title defaulting). An is thrown where there is no available to resolve it. +/// Any other (i.e. ) is not supported and will result in a . +/// +/// The command domain segment is normalized to invariant lowercase; the base destination and explicitly supplied destination names are unchanged. +/// Where a dead-letter destination is requested, the and 'dead-letter' are appended to the resulting name. +/// Where not explicitly specified, the will default to the value of the 'CoreEx:Events:Destination' configuration setting, or 'default' as a fallback. +/// +public class NamedDestinationProvider : IDestinationProvider +{ + private string? _destination = null; + + /// + /// Gets the default . + /// + /// The is resolved from configuration (see class remarks) on first use and then retained for the lifetime of the process. + public static NamedDestinationProvider Default { get; } = new NamedDestinationProvider(); + + /// + /// Gets the character used to separate the from the ( messages only) and from the 'dead-letter' suffix. Defaults to '-'. + /// + public char AppendSeparatorCharacter { get; init; } = '-'; + + /// + /// Gets or sets the used to format the for use in the destination name. Defaults to . + /// + public Func DomainNameFormatter { get; init => field = value.ThrowIfNull(); } = dn => dn.ToLowerInvariant(); + + /// + /// Gets the base destination name (i.e. topic). + /// + /// Where not explicitly set (initialized), the 'CoreEx:Events:Destination' configuration setting is used (or 'default' where not configured). The configured value is resolved on first access and then retained. + /// Setting a or empty value will result in an . + public string Destination + { + get => _destination ??= Internal.GetConfigurationValue("CoreEx:Events:Destination", "default")!; + init => _destination = value.ThrowIfNullOrEmpty(); + } + + /// + public string CreateFrom(EventData @event, bool isDeadLetter = false) => CreateNew(@event.ThrowIfNull().MessageType, @event.DomainName, isDeadLetter); + + /// + public string CreateFrom(string destination, bool isDeadLetter = false) => isDeadLetter ? $"{destination}{AppendSeparatorCharacter}dead-letter" : destination; + + /// + public string CreateNew(MessageType messageType = MessageType.Event, string? domainName = null, bool isDeadLetter = false) + { + return messageType switch + { + MessageType.Event => CreateFrom(Destination, isDeadLetter), + MessageType.Command => CreateFrom($"{Destination}{AppendSeparatorCharacter}{GetCommandDomainName(domainName)}", isDeadLetter), + _ => throw new NotSupportedException($"The message type '{messageType}' is not supported.") + }; + } + + /// + /// Gets the command target domain name; defaults to the (i.e. self-addressed) where not specified. + /// + private string GetCommandDomainName(string? domainName) + { + if (!string.IsNullOrEmpty(domainName)) + return DomainNameFormatter(domainName); + + return DomainNameFormatter(ExecutionContext.GetService()?.DomainName + ?? throw new InvalidOperationException($"A domain name is required for '{nameof(MessageType.Command)}' messages where the {nameof(IHostSettings)}.{nameof(IHostSettings.DomainName)} is not available to default the (self-addressed) target domain.")); + } +} diff --git a/src/CoreEx.Events/Publishing/README.md b/src/CoreEx.Events/Publishing/README.md index fbb134368..6815c5387 100644 --- a/src/CoreEx.Events/Publishing/README.md +++ b/src/CoreEx.Events/Publishing/README.md @@ -27,7 +27,8 @@ Transport-specific publisher implementations (e.g. Azure Service Bus, RabbitMQ) | [`IEventPublisher`](./IEventPublisher.cs) | Extends `IEventQueue` with `PublishAsync()`, `HasBeenPublished`, `Reset()`, `Rollback(count)`, and `GetEvents()`. | | _[`EventPublisherBase`](./EventPublisherBase.cs)_ | Thread-safe abstract base; wires formatting, destination resolution, single-publish guard, and OpenTelemetry; implementors override `OnPublishAsync`. | | [`IDestinationProvider`](./IDestinationProvider.cs) | Resolves destination (topic/queue) names from `EventData`, a string, or `MessageType` + domain name. | -| **[`FixedDestinationProvider`](./FixedDestinationProvider.cs)** | `IDestinationProvider` that routes all events to a single destination; defaults to the `CoreEx.Events:Destination` configuration key or `"default"`. | +| **[`FixedDestinationProvider`](./FixedDestinationProvider.cs)** | `IDestinationProvider` that routes all events to a single destination; defaults to the `CoreEx:Events:Destination` configuration key or `"default"`. | +| **[`NamedDestinationProvider`](./NamedDestinationProvider.cs)** | `IDestinationProvider` that routes events to a single shared destination (topic) and commands to a per-target-domain destination (queue), e.g. `{Destination}-{domain}`; a command without a domain is self-addressed (host domain). Defaults `Destination` from `CoreEx:Events:Destination` or `"default"`. | | **[`DestinationEvent`](./DestinationEvent.cs)** | Immutable record pairing a resolved destination name with a formatted `CloudEvent`; passed to `OnPublishAsync`. | | **[`EventPublisherInvoker`](./EventPublisherInvoker.cs)** | `InvokerBase` subclass that wraps publish operations with OpenTelemetry activity spans. | | **[`NoOpEventPublisher`](./NoOpEventPublisher.cs)** | Silent `IEventPublisher` implementation that discards all events without error. | diff --git a/src/CoreEx.Events/Subscribing/EventSubscriberBase.cs b/src/CoreEx.Events/Subscribing/EventSubscriberBase.cs index 70fb6f143..2bf263781 100644 --- a/src/CoreEx.Events/Subscribing/EventSubscriberBase.cs +++ b/src/CoreEx.Events/Subscribing/EventSubscriberBase.cs @@ -153,22 +153,12 @@ private async Task ReceiveWrapperAsync(EventSubscriberArgs args, Func - /// Determines whether the is an (including -wrapped) attributable specifically to the given . - /// - /// Unlike the general-purpose (which matches any cancellation regardless of source), this distinguishes the receive's own cancellation - /// (e.g. a host/message-pump shutdown, which should bubble up unclassified) from an unrelated cancellation elsewhere in the call stack (e.g. an inner timeout using its - /// own ), which should be classified normally via the like any other exception. - private static bool IsCanceledByToken(Exception ex, CancellationToken cancellationToken) - => (ex is OperationCanceledException oce && oce.CancellationToken == cancellationToken) - || (ex is AggregateException aex && aex.InnerException is OperationCanceledException ioce && ioce.CancellationToken == cancellationToken); - /// /// Deserializes the value to the specified type. /// diff --git a/src/CoreEx.Events/Subscribing/SubscribedManager.cs b/src/CoreEx.Events/Subscribing/SubscribedManager.cs index eee361e8d..339b6304e 100644 --- a/src/CoreEx.Events/Subscribing/SubscribedManager.cs +++ b/src/CoreEx.Events/Subscribing/SubscribedManager.cs @@ -252,7 +252,7 @@ public async Task ReceiveAsync(ExecutionContext executionContext, Subscr return Result.Go(await subscribed.ReceiveAsync(@event, args, cancellationToken).ConfigureAwait(false)) .OnFailure(result => subscribed.ErrorHandler is not null ? subscribed.ErrorHandler.Handle(new ErrorHandlerArgs { SubscriberArgs = args, SourceType = subscribed.GetType(), Exception = result.Error }, null) : result); } - catch (Exception ex) when (subscribed.ErrorHandler is not null && ex is not IEventSubscriberException && !IsCanceledByToken(ex, cancellationToken)) // Ignore IEventSubscriberException's; and only ignore a cancellation attributable to *this* receive's own cancellationToken (e.g. host shutdown) as that is intended to bubble up - any other (unrelated) cancellation is classified as normal. + catch (Exception ex) when (subscribed.ErrorHandler is not null && ex is not IEventSubscriberException && !ex.IsCanceledBy(cancellationToken)) // Ignore IEventSubscriberException's; and only ignore a cancellation attributable to *this* receive's own cancellationToken (e.g. host shutdown) as that is intended to bubble up - any other (unrelated) cancellation is classified as normal. { return subscribed.ErrorHandler.Handle(new ErrorHandlerArgs { SubscriberArgs = args, SourceType = subscribed.GetType(), Exception = ex }, null); } @@ -262,14 +262,4 @@ public async Task ReceiveAsync(ExecutionContext executionContext, Subscr } }, cancellationToken).ConfigureAwait(false); } - - /// - /// Determines whether the is an (including -wrapped) attributable specifically to the given . - /// - /// Unlike the general-purpose (which matches any cancellation regardless of source), this distinguishes the receive's own cancellation - /// (e.g. a host/message-pump shutdown, which should bubble up unclassified) from an unrelated cancellation elsewhere in the call stack (e.g. an inner timeout using its - /// own ), which should be classified normally via the like any other exception. - private static bool IsCanceledByToken(Exception ex, CancellationToken cancellationToken) - => (ex is OperationCanceledException oce && oce.CancellationToken == cancellationToken) - || (ex is AggregateException aex && aex.InnerException is OperationCanceledException ioce && ioce.CancellationToken == cancellationToken); } \ No newline at end of file diff --git a/src/CoreEx.RefData/AGENTS.md b/src/CoreEx.RefData/AGENTS.md index d832939af..605aac00a 100644 --- a/src/CoreEx.RefData/AGENTS.md +++ b/src/CoreEx.RefData/AGENTS.md @@ -12,7 +12,7 @@ The canonical way to introduce reference data is through the **`*.CodeGen` proje | `*.g.cs` — `ReferenceDataCollection` class | Thread-safe, cache-friendly collection | | `*.g.cs` — `IReferenceDataRepository` | Repository interface for loading from the database | | `*.g.cs` — `ReferenceDataRepository` | EF Core implementation of the repository | -| `*.g.cs` — `IReferenceDataProvider` / `ReferenceDataService` | Orchestrator provider wiring all types together | +| `*.g.cs` — `IReferenceDataProvider` / `ReferenceDataProvider` | Orchestrator provider wiring all types together | | `*.g.cs` — `ReferenceDataController` | API controller exposing all reference data types | ### `ref-data.yaml` — the single source of truth @@ -55,16 +55,16 @@ if (movement.Status == MovementStatus.Pending) { ... } ### Registration -Register the generated `IReferenceDataProvider` with the orchestrator, then dynamically register all generated services and repositories: +Add the non-generic orchestrator (it binds the generated `IReferenceDataProvider` from DI at runtime, so there is no compile-time dependency on the generated type), then dynamically register all generated services and repositories: ```csharp // Program.cs builder.Services - .AddReferenceDataOrchestrator() // generated IReferenceDataProvider + .AddReferenceDataOrchestrator() // binds the generated IReferenceDataProvider from DI ... -// Dynamic registration discovers ReferenceDataService and ReferenceDataRepository via [ScopedService] -builder.Services.AddDynamicServicesUsing(); +// Dynamic registration discovers ReferenceDataProvider and ReferenceDataRepository via [ScopedService] +builder.Services.AddDynamicServicesUsing(); ``` The orchestrator resolves `IHybridCache` from DI to cache loaded collections. Register FusionCache separately — see [CoreEx.Caching.FusionCache](../CoreEx.Caching.FusionCache/README.md). diff --git a/src/CoreEx.RefData/Abstractions/ReferenceDataCore.cs b/src/CoreEx.RefData/Abstractions/ReferenceDataCore.cs index 40e503be0..e2e1445d6 100644 --- a/src/CoreEx.RefData/Abstractions/ReferenceDataCore.cs +++ b/src/CoreEx.RefData/Abstractions/ReferenceDataCore.cs @@ -8,7 +8,7 @@ namespace CoreEx.RefData.Abstractions; /// superfluous from an equality perspective. The and properties should be both unique within their owning collection; the /// ensures this. [DebuggerDisplay("Id = {Id}, Code = {Code}, IsActive = {IsActive}, IsValid = {IsValid}")] -public abstract class ReferenceDataCore : IReferenceData +public abstract class ReferenceDataCore : IReferenceData, IETag { private string? _text; private string? _description; @@ -26,10 +26,12 @@ public abstract class ReferenceDataCore : IReferenceData /// [JsonPropertyOrder(-999)] + [ReadOnly(true)] public TId Id { get; init; } /// [JsonPropertyOrder(-998)] + [ReadOnly(true)] public string? Code { get; init; } /// @@ -51,6 +53,7 @@ public abstract class ReferenceDataCore : IReferenceData /// [JsonPropertyOrder(-994)] + [ReadOnly(true)] public virtual bool IsInactive { get @@ -89,7 +92,8 @@ public virtual bool IsInactive public DateTimeOffset? EndsOn { get; init; } /// - public string? ETag { get; init; } + [ReadOnly(true)] + public string? ETag { get; set; } /// [JsonIgnore] @@ -142,4 +146,4 @@ public override bool Equals(object? obj) return Equals(Id, rd.Id) && Equals(Code, rd.Code); } -} \ No newline at end of file +} diff --git a/src/CoreEx.RefData/GlobalUsing.cs b/src/CoreEx.RefData/GlobalUsing.cs index 5dd4814ac..c654b88ab 100644 --- a/src/CoreEx.RefData/GlobalUsing.cs +++ b/src/CoreEx.RefData/GlobalUsing.cs @@ -10,12 +10,14 @@ global using CoreEx.RefData.Abstractions; global using CoreEx.RefData.HealthChecks; global using CoreEx.Results; +global using CoreEx.Validation; global using Microsoft.Extensions.DependencyInjection.Extensions; global using Microsoft.Extensions.Diagnostics.HealthChecks; global using Microsoft.Extensions.Logging; global using System.Collections; global using System.Collections.Concurrent; global using System.Collections.ObjectModel; +global using System.ComponentModel; global using System.Diagnostics; global using System.Diagnostics.CodeAnalysis; global using System.Linq.Dynamic.Core; diff --git a/src/CoreEx.RefData/ReferenceDataHybridCache.TypedInvoker.cs b/src/CoreEx.RefData/ReferenceDataHybridCache.TypedInvoker.cs index 26bd0d32a..3ca839485 100644 --- a/src/CoreEx.RefData/ReferenceDataHybridCache.TypedInvoker.cs +++ b/src/CoreEx.RefData/ReferenceDataHybridCache.TypedInvoker.cs @@ -7,13 +7,19 @@ public partial class ReferenceDataHybridCache { /* * This functionality is required as the underlying cache *may* leverage serialization, and as such, we have to get it in a typed manner as IReferenceDataCollection (interface) is not valid. + * This applies equally to both reading (TryGetByKeyAsync) and getting-or-creating (GetOrCreateByKeyAsync) - relying on C# generic type inference for the latter would infer the factory + * delegate's declared IReferenceDataCollection return type rather than the concrete collection type, which a serializing (e.g. distributed) cache cannot deserialize back. */ private static readonly MethodInfo _tryGetByKeyAsync_OpenGeneric = typeof(IHybridCache).GetMethod(nameof(IHybridCache.TryGetByKeyAsync)) ?? throw new InvalidOperationException($"{nameof(IHybridCache)}.{nameof(IHybridCache.TryGetByKeyAsync)} public instance method not found."); + private static readonly MethodInfo _getOrCreateTypedAsync_OpenGeneric = typeof(ReferenceDataHybridCache).GetMethod(nameof(GetOrCreateTypedAsync), BindingFlags.NonPublic | BindingFlags.Static) ?? throw new InvalidOperationException($"{nameof(ReferenceDataHybridCache)}.{nameof(GetOrCreateTypedAsync)} private static method not found."); private static readonly ConcurrentDictionary _invokers = new(); + private static readonly ConcurrentDictionary _getOrCreateInvokers = new(); private delegate Task<(bool Exists, object? Value)> TryGetByKeyInvoker(IHybridCache cache, string key, HybridCacheEntryOptions options, CancellationToken cancellationToken); + private delegate Task GetOrCreateByKeyInvoker(IHybridCache cache, string key, Func> factory, Type type, HybridCacheEntryOptions options, CancellationToken cancellationToken); + /// /// Gets (or adds) the for the specified type. /// @@ -42,4 +48,26 @@ private static TryGetByKeyInvoker GetInvokerForType(Type type) => _invokers.GetO /// Underlying method to invoke the typed . /// private static async Task<(bool Exists, object? Value)> ToTupleAsync(Task<(bool Exists, T? Value)> task) => await task.ConfigureAwait(false); + + /// + /// Gets (or adds) the for the specified type. + /// + private static GetOrCreateByKeyInvoker GetOrCreateInvokerForType(Type type) => _getOrCreateInvokers.GetOrAdd(type, type => + { + // Close the generic: GetOrCreateTypedAsync. + var closed = _getOrCreateTypedAsync_OpenGeneric.MakeGenericMethod(type); + return (GetOrCreateByKeyInvoker)Delegate.CreateDelegate(typeof(GetOrCreateByKeyInvoker), closed); + }); + + /// + /// Underlying method to invoke the typed ensuring the concrete (as opposed to the + /// interface) is used as the generic type argument; this is essential where the underlying cache leverages serialization (e.g. a distributed cache), as an interface/abstract collection type cannot + /// be deserialized. + /// + private static async Task GetOrCreateTypedAsync(IHybridCache cache, string key, Func> factory, Type type, HybridCacheEntryOptions options, CancellationToken cancellationToken) where T : IReferenceDataCollection + => await cache.GetOrCreateByKeyAsync(key, async ct => + { + var value = await factory(type, ct).ConfigureAwait(false) ?? throw new InvalidOperationException($"The '{type.Name}' (reference data) collection returned from the factory must not be null."); + return (T)value; + }, options, cancellationToken).ConfigureAwait(false); } diff --git a/src/CoreEx.RefData/ReferenceDataHybridCache.cs b/src/CoreEx.RefData/ReferenceDataHybridCache.cs index 5c5e2dcd0..92bd9b794 100644 --- a/src/CoreEx.RefData/ReferenceDataHybridCache.cs +++ b/src/CoreEx.RefData/ReferenceDataHybridCache.cs @@ -6,13 +6,9 @@ namespace CoreEx.RefData; /// The underlying . public partial class ReferenceDataHybridCache(IHybridCache cache) : IReferenceDataCache { - private readonly ConcurrentDictionary _semaphores = new(); + // IReferenceDataCache is scoped, so the gate must be shared across instances to cover concurrent requests. + private static readonly ConcurrentDictionary _semaphores = new(); private readonly ConcurrentDictionary _entryOptions = new(); -#if NET8_0 - private readonly object _lock = new(); -#else - private readonly Lock _lock = new(); -#endif /// /// Gets the underlying . @@ -29,6 +25,14 @@ private HybridCacheEntryOptions GetOrCreateEntryOptions(Type type) => _entryOpti return options; }); + + /// + /// Provides an opportunity to further configure the . + /// + /// The . + /// The . + protected virtual void OnCreateCacheEntry(Type type, HybridCacheEntryOptions entry) { } + /// public async Task GetOrCreateAsync(Type type, Func> factory, CancellationToken cancellationToken = default) { @@ -43,23 +47,34 @@ public async Task GetOrCreateAsync(Type type, Func new SemaphoreSlim(1, 1)); + semaphore.Release(); } + } + + /// + /// The removal is performed under the same process-wide per-key semaphore as cache loads, including loads in other request scopes. Any in-flight load completes (and is then evicted) rather than repopulating the cache with stale data after the removal. + public async Task RemoveAsync(Type type, CancellationToken cancellationToken = default) + { + var key = $"RefData:{(Internal.GetNamespaceFormattedName(type))}"; + var semaphore = GetSemaphore(key); - // Use the semaphore to manage a single thread to perform the "expensive" get operation. await semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); try { - // Does a get or create as it may have been added as we went to lock. - return (await Cache.GetOrCreateByKeyAsync(key, async cancellationToken => - { - return await factory(type, cancellationToken).ConfigureAwait(false) ?? throw new InvalidOperationException($"The '{type.Name}' (reference data) collection returned from the factory must not be null."); - }, options, cancellationToken).ConfigureAwait(false))!; + await Cache.RemoveByKeyAsync(key, GetOrCreateEntryOptions(type), cancellationToken).ConfigureAwait(false); } finally { @@ -68,11 +83,9 @@ public async Task GetOrCreateAsync(Type type, Func - /// Provides an opportunity to further configure the . + /// Gets or adds the used to manage single concurrency for the specified cache . /// - /// The . - /// The . - protected virtual void OnCreateCacheEntry(Type type, HybridCacheEntryOptions entry) { } + private static SemaphoreSlim GetSemaphore(string key) => _semaphores.GetOrAdd(key, _ => new SemaphoreSlim(1, 1)); /// /// Registers the for the specified . @@ -97,4 +110,4 @@ public ReferenceDataHybridCache RegisterCacheEntryOptions(Type type, HybridCache _entryOptions[type] = options; return this; } -} \ No newline at end of file +} diff --git a/src/CoreEx.Template/CoreEx.Template.csproj b/src/CoreEx.Template/CoreEx.Template.csproj index 41dbce154..52594a280 100644 --- a/src/CoreEx.Template/CoreEx.Template.csproj +++ b/src/CoreEx.Template/CoreEx.Template.csproj @@ -137,6 +137,7 @@ + @@ -211,6 +212,15 @@ + + + diff --git a/src/CoreEx.Template/README.md b/src/CoreEx.Template/README.md index 76d66a818..131b5dea9 100644 --- a/src/CoreEx.Template/README.md +++ b/src/CoreEx.Template/README.md @@ -14,16 +14,18 @@ | `coreex-api` | CoreEx API host | `src/[name].Api/` host project + `tests/[solution].Test.Api/` integration test project | | `coreex-relay` | CoreEx Outbox Relay host | `src/[name].Relay/` host project + `tests/[solution].Test.Relay/` integration test project | | `coreex-subscribe` | CoreEx Subscriber host | `src/[name].Subscribe/` host project + `tests/[solution].Test.Subscribe/` integration test project | -| `coreex-aspire` | CoreEx Aspire AppHost | `src/[name].Aspire/` AppHost project orchestrating this solution's own hosts — add-on, run after the hosts it references already exist | +| `coreex-aspire` | CoreEx Aspire AppHost | `aspire/[name].Aspire/` AppHost project orchestrating this solution's own hosts — add-on, run after the hosts it references already exist | Parameters are consistent across templates -- the same `--data-provider`, `--messaging-provider`, and feature flags appear in every template that needs them, ensuring the generated code is coherent regardless of which templates you use. ## Installation ```sh -dotnet new install CoreEx.Template +dotnet new install CoreEx.Template:: ``` +Pin `` to the project's referenced CoreEx NuGet version. For first-time adoption without a CoreEx reference, resolve the latest stable release explicitly and record that version. + To verify: ```sh @@ -33,7 +35,7 @@ dotnet new list --tag CoreEx To update after a new release: ```sh -dotnet new update +dotnet new install CoreEx.Template:: ``` To uninstall: @@ -48,10 +50,14 @@ dotnet new uninstall CoreEx.Template - `.github/instructions/coreex.instructions.md` -- CoreEx coding context (replaces `copilot-instructions.md`) - `.github/instructions/*.instructions.md` -- scoped coding conventions per layer/file type -- `.github/prompts/coreex-scaffold.prompt.md` -- `/coreex-scaffold`: interviews for project needs and recommends the smallest safe `dotnet new coreex*` command set +- `.github/skills/` -- 23 CoreEx skills: bootstrap, scaffold, docs-sync, 16 per-capability (L1) skills, and four end-to-end (L2) workflows +- `.github/prompts/` -- 22 wrappers for bootstrap, scaffold, and all L1/L2 skills; docs-sync is invoked directly as a skill - `.github/agents/coreex-expert.agent.md` -- `/coreex-expert`: architecture guidance - `.claude/commands/coreex-expert.md` -- `/coreex-expert` for Claude Code -- `.claude/commands/coreex-docs-sync.md` -- `/coreex-docs-sync`: refresh the local CoreEx doc cache after a version bump +- `.claude/commands/` -- 24 commands: one per CoreEx skill plus expert guidance; `/coreex-docs-sync` refreshes the entire version-pinned bundle after a package version bump +- `.github/docs/coreex/` -- 12 architecture/setup docs, 18 package guides, and `manifest.txt` recording the installed `coreex-version` + +The generic `acquire-codebase-knowledge` and `aspire` skills are framework-repository-only and are not emitted by `coreex-ai`. See the [full workflow inventory](../../.github/coreex-ai-workflows.md). **Recommended day-1 workflow:** @@ -130,6 +136,7 @@ This template emits nothing but `.github/` and `.claude/` artefacts: no `AGENTS. coreex.instructions.md # CoreEx coding context (applyTo: "**" or "/**") coreex-api-controllers.instructions.md coreex-application-services.instructions.md + coreex-aspire.instructions.md coreex-contracts.instructions.md coreex-conventions.instructions.md coreex-domain.instructions.md @@ -140,15 +147,23 @@ This template emits nothing but `.github/` and `.claude/` artefacts: no `AGENTS. coreex-tooling.instructions.md coreex-validators.instructions.md prompts/ - coreex-scaffold.prompt.md + coreex-*.prompt.md # 22 wrappers: bootstrap, scaffold, and L1/L2 skills + skills/ + coreex-*/SKILL.md # 23 CoreEx skills, with references/assets where applicable agents/ coreex-expert.agent.md + coreex-ai-workflows.md # Command catalog and asset guidance + docs/coreex/ + manifest.txt # Installed coreex-version + *.md # 12 architecture/setup docs + agents/*.md # 18 package guides .claude/ commands/ coreex-bootstrap.md # /coreex-bootstrap blank-repo initialisation command coreex-scaffold.md # /coreex-scaffold guided solution scaffolding command coreex-expert.md # /coreex-expert architecture guidance command - coreex-docs-sync.md # /coreex-docs-sync doc cache refresh command + coreex-docs-sync.md # /coreex-docs-sync version-pinned AI bundle refresh + coreex-*.md # L1/L2 skill wrappers (24 commands total) ``` ### `--app-folder` behaviour @@ -181,9 +196,9 @@ Run this **once per domain** from the solution root directory. | Parameter | Type | Default | Description | |---|---|---|---| | `-n` / `--name` | string | _(required)_ | Solution base name, e.g. `Avanade.Erp.Sales`. Drives all file names and namespaces. Format: `[Company].[Product].[Domain]`. | -| `--refdata-enabled` | bool | `true` | Includes the reference-data pattern: `IReferenceDataRepository` (Application), `ReferenceDataService` (Application), `ReferenceDataRepository` (Infrastructure), and the `CodeGen` tool project. | +| `--refdata-enabled` | bool | `true` | Includes the reference-data pattern: `IReferenceDataRepository` (Application), `ReferenceDataProvider` (Application), `ReferenceDataRepository` (Infrastructure), and the `CodeGen` tool project. | | `--rop-enabled` | bool | `false` | Enables Railway-Oriented Programming -- `Result`/`Result` return types throughout the solution. | -| `--data-provider` | `SqlServer` \| `Postgres` \| `None` | `SqlServer` | The data persistence technology. `None` is for facade scenarios (e.g. over Dynamics 365) where there is no local database -- the `Database` tool project, EF Core packages, DbContext, and EfDb are all omitted. | +| `--data-provider` | `SqlServer` \| `Postgres` \| `Cosmos` \| `None` | `SqlServer` | The data persistence technology. `SqlServer`/`Postgres` use EF Core (`DbContext`/`EfDb`) with a DbEx `Database` tool project; `Cosmos` (Azure Cosmos DB) uses `CoreEx.Cosmos` directly (a `[Domain]CosmosDb` class, no EF Core, no SQL migrations) and provisions containers code-first from the `Database` tool project (`ConfigureProvisionArgs` plus seed data) -- a fresh solution ships a single `ref-data` container (when `--refdata-enabled`), with entity containers added later. `None` is for facade scenarios (e.g. over Dynamics 365) where there is no local database -- the `Database` tool project, data-access packages and persistence classes are all omitted. | | `--outbox-enabled` | bool | `true` | Includes transactional-outbox wiring in the Infrastructure project. Has no effect when `--data-provider None`. | | `--messaging-provider` | `ServiceBus` \| `None` | `ServiceBus` | The messaging technology. `None` omits all messaging configuration. | @@ -203,18 +218,19 @@ src/ [name].Application/ [name].Application.csproj GlobalUsing.cs - ReferenceDataService.cs (refdata-enabled only) + ReferenceDataProvider.g.cs (refdata-enabled only) Repositories/ IReferenceDataRepository.cs (refdata-enabled only) [name].Infrastructure/ [name].Infrastructure.csproj GlobalUsing.cs Repositories/ - [Domain]DbContext.cs (data-provider != None) - [Domain]EfDb.cs (data-provider != None) + [Domain]DbContext.cs (data-provider == SqlServer | Postgres) + [Domain]EfDb.cs (data-provider == SqlServer | Postgres) + [Domain]CosmosDb.cs (data-provider == Cosmos -- the CosmosDb with the `ref-data` container and outbox event wiring) ReferenceDataRepository.cs (refdata-enabled && data-provider != None) tools/ - [name].Database/ (data-provider != None) + [name].Database/ (data-provider != None; Cosmos: no dbex.yaml/Migrations -- containers are provisioned code-first) [name].Database.csproj Program.cs [name].CodeGen/ (refdata-enabled && data-provider != None) @@ -225,7 +241,9 @@ tests/ [name].Test.Common/ [name].Test.Common.csproj TestData.cs - Data/ (data-provider != None -- embedded seed data for DbEx) + ServiceBus.cs (messaging-provider == ServiceBus -- code-based topic/queue topology for tests; servicebus/Config.json is empty) + GlobalUsing.cs (messaging-provider == ServiceBus) + Data/ (data-provider != None -- embedded seed data for DbEx / the Cosmos provisioner) [name].Test.Unit/ [name].Test.Unit.csproj ``` @@ -306,8 +324,8 @@ Run this from the **solution root** (the directory created by `coreex`). The tem |---|---|---|---| | `-n` / `--name` | string | _(required)_ | Full project name, e.g. `Avanade.Erp.Sales.Api`. Must match the solution name with `.Api` appended. | | `--refdata-enabled` | bool | `true` | Wires up `ReferenceDataOrchestrator` and dynamic service registration for reference-data caching. Match the value used with `coreex`. | -| `--data-provider` | `SqlServer` \| `Postgres` \| `None` | `SqlServer` | Database technology. Selects the Aspire connection, CoreEx database, unit-of-work, and outbox publisher registration. | -| `--outbox-enabled` | bool | `true` | Registers the outbox publisher (`SqlServerOutboxPublisher` or `PostgresOutboxPublisher`) as the `IEventPublisher`. Has no effect when `--data-provider None`. | +| `--data-provider` | `SqlServer` \| `Postgres` \| `Cosmos` \| `None` | `SqlServer` | Database technology. Selects the Aspire connection, CoreEx database (or `AddCosmosDb` for Cosmos), unit-of-work, health check and outbox publisher registration. | +| `--outbox-enabled` | bool | `true` | Registers the outbox publisher (`SqlServerOutboxPublisher`, `PostgresOutboxPublisher` or `CosmosDbEventPublisher`) as the `IEventPublisher`. Has no effect when `--data-provider None`. | ### Output @@ -351,6 +369,7 @@ tests/ "Microsoft": { "Data": { "SqlClient": { "ConnectionString": "Data Source=127.0.0.1,1433;Initial Catalog=Sales;..." } } }, // OR PostgreSQL (implement-postgres): "Npgsql": { "ConnectionString": "Server=127.0.0.1;Database=sales;..." }, + // (Cosmos (implement-cosmos) uses a top-level "ConnectionStrings": { "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=..." } entry instead.) "StackExchange": { "Redis": { "ConnectionString": "localhost:6379" } } } } @@ -381,14 +400,15 @@ Scaffolds an ASP.NET Core background-service host that reads committed events fr Run this from the **solution root** (the directory created by `coreex`). The template emits into both `src/` and `tests/` so it must be run at the root level. -> **Note:** When `--data-provider None` is selected the outbox relay has nothing to relay. Scaffold this host only when your solution uses the outbox pattern (i.e. `--data-provider` is `SqlServer` or `Postgres` and `--outbox-enabled true`). +> **Note:** When `--data-provider None` is selected the outbox relay has nothing to relay. Scaffold this host only when your solution uses the outbox pattern (i.e. `--data-provider` is `SqlServer`, `Postgres` or `Cosmos` and `--outbox-enabled true`). ### Parameters | Parameter | Type | Default | Description | |---|---|---|---| | `-n` / `--name` | string | _(required)_ | Full project name, e.g. `Avanade.Erp.Sales.Relay`. | -| `--data-provider` | `SqlServer` \| `Postgres` \| `None` | `SqlServer` | Database technology used for reading the outbox. | +| `--data-provider` | `SqlServer` \| `Postgres` \| `Cosmos` \| `None` | `SqlServer` | Database technology used for reading the outbox. For `Cosmos` the outbox is relayed from the Change Feed, one `AddCosmosDbOutboxRelayHostedService("")` call per outbox-hosting container (a fresh solution ships `ref-data`). | +| `--refdata-enabled` | bool | `true` | Cosmos only: a fresh Cosmos solution's only container is `ref-data`, so the relay hosted service (and its health checks/tests) are only emitted when this is `true`. Match the value used with `coreex`. | | `--messaging-provider` | `ServiceBus` \| `None` | `ServiceBus` | Messaging provider to publish forwarded events to. | ### Output @@ -419,7 +439,7 @@ tests/ "OutboxRelay": { "BatchSize": 10, "PerWorkerPartitionCount": 2, - "LeaseDuration": "00:00:05", + "LeaseDuration": "00:00:30", "BackoffDuration": "00:00:05", "ServicesCount": 4 } @@ -429,12 +449,14 @@ tests/ } ``` +For relational outbox hosts, configure `CoreEx:Host:Outbox:PartitionSize` identically in every API/Subscribe writer and Relay host. The default is `4` and the supported range is `1`–`256`. The former relay-only `CoreEx:Host:Services:OutboxRelay:PartitionSize` key is removed; move its value to the shared `Outbox:PartitionSize` setting. When Service Bus sessions are used, `CoreEx:Host:ServiceBus:SessionIdPartitionSize` is an independent publisher-wide default (default `4`), not coupled to the outbox partition count. Override a destination's `SessionIdStrategy` or bucket count under `CoreEx:Host:ServiceBus:Destinations:{exact-destination-name}` when its ordering/concurrency profile warrants it. This bucket count maps producer keys to sessions; the receiver's `MaxConcurrentSessions` independently controls consumer concurrency. See [Service Bus session guidance](../CoreEx.Azure.Messaging.ServiceBus/README.md#publishing). + **`appsettings.Development.json`** (connection strings vary by provider): ```jsonc { "Aspire": { - // SQL Server or PostgreSQL connection (same pattern as coreex-api) + // SQL Server, PostgreSQL or Cosmos connection (same pattern as coreex-api) "Azure": { // implement-servicebus only: "Messaging": { "ServiceBus": { "ConnectionString": "Endpoint=sb://localhost;..." } } @@ -455,6 +477,11 @@ PostgreSQL outbox -> Azure Service Bus: dotnet new coreex-relay -n Avanade.Erp.Sales.Relay --data-provider Postgres ``` +Cosmos DB outbox (Change Feed) -> Azure Service Bus: +```sh +dotnet new coreex-relay -n Avanade.Erp.Sales.Relay --data-provider Cosmos +``` + --- ## Template 6 -- `coreex-subscribe` (Subscriber host) @@ -469,7 +496,8 @@ Run this from the **solution root** (the directory created by `coreex`). The tem |---|---|---|---| | `-n` / `--name` | string | _(required)_ | Full project name, e.g. `Avanade.Erp.Sales.Subscribe`. | | `--refdata-enabled` | bool | `true` | Wires up `ReferenceDataOrchestrator` and dynamic service registration for reference-data caching. Match the value used with `coreex`. | -| `--data-provider` | `SqlServer` \| `Postgres` \| `None` | `SqlServer` | Database technology. When not `None`, EF Core and outbox publisher are wired so subscriber logic can persist state and emit its own events. | +| `--data-provider` | `SqlServer` \| `Postgres` \| `Cosmos` \| `None` | `SqlServer` | Database technology. When not `None`, the data-access services (EF Core, or `CoreEx.Cosmos`) and, when `--outbox-enabled`, the outbox publisher are wired so subscriber logic can persist state and emit its own events. | +| `--outbox-enabled` | bool | `true` | Registers the outbox publisher as the `IEventPublisher` (Service Bus is then a named, non-default publisher). When `false`, Service Bus is the default `IEventPublisher`. Match the value used with `coreex`. | | `--messaging-provider` | `ServiceBus` \| `None` | `ServiceBus` | Messaging provider to receive events from. When `None`, no receiver is configured. | ### Output @@ -510,7 +538,7 @@ tests/ ```jsonc { "Aspire": { - // SQL Server or PostgreSQL connection (same pattern as coreex-api) + // SQL Server, PostgreSQL or Cosmos connection (same pattern as coreex-api) "Azure": { // implement-servicebus only: "Messaging": { @@ -547,11 +575,13 @@ dotnet new coreex-subscribe -n Avanade.Erp.Sales.Subscribe --data-provider None ## Template 7 -- `coreex-aspire` (Aspire AppHost) -Scaffolds a .NET Aspire AppHost project that orchestrates this solution's own `Api`/`Relay`/`Subscribe` hosts for local development, plus an `Extensions.cs` providing dashboard sugar (health-check deep links, and "Pause all services"/"Resume all services" buttons for hosts running CoreEx hosted services). +Scaffolds a .NET Aspire AppHost project that orchestrates this solution's own `Api`/`Relay`/`Subscribe` hosts for local development, plus two sibling projects: `[name].Aspire.MockHost` (a WireMock.Net-based console host for stubbing external dependencies) and `[solution-name].Test.Aspire` (an NUnit project that spins up the whole AppHost via `WithAspireTester<...>` for smoke/integration testing). Dashboard sugar (health-check deep links, and "Pause all services"/"Resume all services" buttons for hosts running CoreEx hosted services) and Aspire test helpers come from the `CoreEx.UnitTesting` package (`` in `[name].Aspire.csproj`) -- there is no local `Extensions.cs`. -Run this from the **solution root**, after the host projects it will reference already exist. Unlike `coreex-api`/`coreex-relay`/`coreex-subscribe`, host inclusion is not derived from a shared `coreex` parameter -- pass `--has-api`/`--has-relay`/`--has-subscribe` explicitly to match whichever hosts this solution actually has. +The generated `HostTests` require HTTP `200 OK` from health endpoints. Startup `503 ServiceUnavailable` responses are retried for up to 60 attempts with one-second delays; persistent unhealthiness or any other unexpected status fails the test rather than passing as endpoint reachability. -> **Note:** If a new host is added to the solution *after* `coreex-aspire` has already been run, don't re-run it with `--force` -- that would overwrite any customisation already made to `AppHost.cs`/`Extensions.cs`. Add the missing `` and `builder.AddProject<...>(...)` line by hand instead. +Run this from the **solution root**, after the host projects it will reference already exist. Unlike `coreex-api`/`coreex-relay`/`coreex-subscribe`, host inclusion is not derived from a shared `coreex` parameter -- pass `--has-api`/`--has-relay`/`--has-subscribe` explicitly to match whichever hosts this solution actually has. `--data-provider`/`--messaging-provider` should match the values used when scaffolding those hosts, so `AppHost.cs` declares the right connection-string resources. + +> **Note:** If a new host is added to the solution *after* `coreex-aspire` has already been run, don't re-run it with `--force` -- that would overwrite any customisation already made to `AppHost.cs`. Add the missing `` and `builder.AddProject<...>(...)` line by hand instead. ### Parameters @@ -561,34 +591,62 @@ Run this from the **solution root**, after the host projects it will reference a | `--has-api` | bool | `true` | Include a project reference and `AddProject<...>` call for this solution's `Api` host. | | `--has-relay` | bool | `false` | Include a project reference and `AddProject<...>` call for this solution's `Relay` host. | | `--has-subscribe` | bool | `false` | Include a project reference and `AddProject<...>` call for this solution's `Subscribe` host. | +| `--data-provider` | `SqlServer` \| `Postgres` \| `Cosmos` \| `None` | `Postgres` | Database technology; determines which connection-string resource (`SqlServer`/`Postgres`/`Cosmos`) `AppHost.cs` declares and each host references. | +| `--messaging-provider` | `ServiceBus` \| `None` | `ServiceBus` | Messaging technology; determines whether `AppHost.cs` declares a `ServiceBus` connection-string resource for the Relay/Subscribe hosts. | ### Output ``` -src/ +aspire/ [name].Aspire/ [name].Aspire.csproj AppHost.cs - Extensions.cs appsettings.json appsettings.Development.json Properties/launchSettings.json AGENTS.md + [name].Aspire.MockHost/ + [name].Aspire.MockHost.csproj + Program.cs + [solution-name].Test.Aspire/ + [solution-name].Test.Aspire.csproj + GlobalUsing.cs + HostTests.cs ``` -**`AppHost.cs`** (illustrative, all three hosts included): +**`AppHost.cs`** (illustrative, all three hosts included, Postgres + Service Bus): ```csharp var builder = DistributedApplication.CreateBuilder(args); +builder.DisableHttpCertificateValidation(); + +var db = builder.AddExternalConnectionString("Postgres").WithIconName("DatabaseMultiple"); +var redis = builder.AddExternalConnectionString("redis").WithIconName("Database"); +var serviceBus = builder.AddExternalConnectionString("ServiceBus").WithIconName("MailMultiple"); + // Sales domain. -builder.AddProject("sales-api").AddEndpoints("/health/ready/detailed"); -builder.AddProject("sales-relay").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); -builder.AddProject("sales-subscribe").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); +builder.AddProject("sales-api").WithReference(db).WithReference(redis).AddEndpoints("/health/ready/detailed"); +builder.AddProject("sales-relay").WithReference(db).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); +builder.AddProject("sales-subscribe").WithReference(db).WithReference(redis).WithReference(serviceBus).AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); builder.Build().Run(); ``` +`AddExternalConnectionString` is supplied by **CoreEx.UnitTesting** in the `UnitTestEx` namespace, +alongside `AddEndpoints` and the other Aspire helpers; there is no local implementation. It wraps secret parameters from +`ConnectionStrings:{name}` in visible `ConnectionStringResource` nodes. Aspire 13.5.4 excludes parameter-only +`AddConnectionString(name)` resources from Graph/Table views. Existing connection names and host references are +preserved; Compose still owns the actual infrastructure lifecycle. A node's running status means configuration +resolved, not that its service is healthy; host readiness checks perform those probes. + +For Cosmos the scaffold passes `endpointKey: "AccountEndpoint"` and derives a credential-free resource URL from +that connection string, correcting dashboard peer matching. Relational connection strings should include explicit +ports. Shared localhost addresses and missing span ports can still cause misattribution; inspect +`server.address`, `server.port`, and `peer.service` rather than treating a dashboard label as proof of provider wiring. +For an existing AppHost, update its pinned CoreEx package version and resource declarations manually; do not overwrite custom wiring +with `dotnet new coreex-aspire --force`. + ### Examples API only: @@ -598,7 +656,7 @@ dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true Full event-driven (API + Relay + Subscribe): ```sh -dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true +dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true --data-provider Postgres --messaging-provider ServiceBus ``` --- @@ -645,8 +703,10 @@ dotnet sln Avanade.Erp.Sales.slnx add tests/Avanade.Erp.Sales.Test.Subscribe Once the hosts above exist, optionally scaffold an AppHost for local orchestration: ```sh -dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true -dotnet sln Avanade.Erp.Sales.slnx add src/Avanade.Erp.Sales.Aspire +dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true --data-provider SqlServer --messaging-provider ServiceBus +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Aspire +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Aspire.MockHost +dotnet sln Avanade.Erp.Sales.slnx add aspire/Avanade.Erp.Sales.Test.Aspire ``` ### Resulting directory structure @@ -662,7 +722,6 @@ Avanade.Erp.Sales/ Avanade.Erp.Sales.Api/ Avanade.Erp.Sales.Relay/ Avanade.Erp.Sales.Subscribe/ - Avanade.Erp.Sales.Aspire/ tools/ Avanade.Erp.Sales.Database/ Avanade.Erp.Sales.CodeGen/ @@ -672,6 +731,10 @@ Avanade.Erp.Sales/ Avanade.Erp.Sales.Test.Api/ Avanade.Erp.Sales.Test.Relay/ Avanade.Erp.Sales.Test.Subscribe/ + aspire/ + Avanade.Erp.Sales.Aspire/ + Avanade.Erp.Sales.Aspire.MockHost/ + Avanade.Erp.Sales.Test.Aspire/ ``` --- @@ -687,7 +750,19 @@ dotnet new coreex -n Avanade.Erp.Sales dotnet new coreex-api -n Avanade.Erp.Sales.Api dotnet new coreex-relay -n Avanade.Erp.Sales.Relay dotnet new coreex-subscribe -n Avanade.Erp.Sales.Subscribe -dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true +dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true --data-provider SqlServer --messaging-provider ServiceBus +``` + +### Cosmos DB variant + +Cosmos needs no SQL migrations; the `Database` tool provisions the `ref-data` container code-first (and the test projects do the same against the emulator). + +```sh +dotnet new coreex -n Avanade.Erp.Sales --data-provider Cosmos +dotnet new coreex-api -n Avanade.Erp.Sales.Api --data-provider Cosmos +dotnet new coreex-relay -n Avanade.Erp.Sales.Relay --data-provider Cosmos +dotnet new coreex-subscribe -n Avanade.Erp.Sales.Subscribe --data-provider Cosmos +dotnet new coreex-aspire -n Avanade.Erp.Sales.Aspire --has-api true --has-relay true --has-subscribe true --data-provider Cosmos --messaging-provider ServiceBus ``` ### PostgreSQL variant diff --git a/src/CoreEx.Template/content/CoreEx.Api/.template.config/template.json b/src/CoreEx.Template/content/CoreEx.Api/.template.config/template.json index ec1a3c137..ba4799f75 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/.template.config/template.json +++ b/src/CoreEx.Template/content/CoreEx.Api/.template.config/template.json @@ -74,6 +74,11 @@ "displayName": "PostgreSQL", "description": "Indicates to use PostgreSQL as the data provider." }, + { + "choice": "Cosmos", + "displayName": "Azure Cosmos DB", + "description": "Indicates to use Azure Cosmos DB as the data provider." + }, { "choice": "None", "displayName": "None", @@ -92,6 +97,14 @@ "type": "computed", "value": "data-provider == 'Postgres'" }, + "implement-cosmos": { + "type": "computed", + "value": "data-provider == 'Cosmos'" + }, + "implement-relational": { + "type": "computed", + "value": "(data-provider == 'SqlServer' || data-provider == 'Postgres')" + }, "implement-none-data": { "type": "computed", "value": "data-provider == 'None'" diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/AGENTS.md index c9a10f887..334d3084d 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/AGENTS.md +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/AGENTS.md @@ -53,18 +53,22 @@ Consult `.github/docs/coreex/agents/CoreEx.Caching.FusionCache.md` for caching p - **Data provider:** SQL Server -- `builder.AddSqlClientConnection("SqlServer")` (Aspire connection name) + +- **Data provider:** Azure Cosmos DB -- `builder.AddAzureCosmosClient("Cosmos")` (Aspire connection name) plus `AddCosmosDb("domain-name-lower")` - **Data provider:** PostgreSQL -- `builder.AddNpgsqlDataSource("Postgres")` (Aspire connection name) - **Data provider:** None -- this host uses no database; services call external systems directly - + +- **Transactional outbox:** Enabled -- events are written to the Cosmos DB outbox by `CosmosDbEventPublisher` (inside `CosmosDbUnitOfWork.TransactionAsync`); the Relay host's Change Feed Processor reads and forwards them + - **Transactional outbox:** Enabled -- events are written to the DB outbox by `domain-nameOutboxPublisher`; the Relay host reads and forwards them - **Transactional outbox:** Disabled -- events are published directly to the message broker -- **Reference data:** Enabled -- `ReferenceDataOrchestrator` is registered; reference data is hydrated via `ReferenceDataRepository` +- **Reference data:** Enabled -- the non-generic `AddReferenceDataOrchestrator()` is registered and binds the CodeGen-generated `ReferenceDataProvider` (`IReferenceDataProvider`) from DI at runtime; reference data is hydrated via `ReferenceDataRepository` - **Reference data:** Disabled @@ -84,7 +88,10 @@ Consult `.github/docs/coreex/agents/CoreEx.Caching.FusionCache.md` for caching p | `CoreEx.Database.Postgres` | PostgreSQL database access | - + +| `CoreEx.Cosmos` | Azure Cosmos DB access (`CosmosDb`, typed containers, outbox, unit of work) | + + | `CoreEx.EntityFrameworkCore` | EF Core integration (`EfDb`, `IEfDbContext`) | @@ -101,9 +108,12 @@ Consult `.github/docs/coreex/agents/CoreEx.Caching.FusionCache.md` for caching p - `.github/docs/coreex/local-dev.md` -- running locally with .NET Aspire - `.github/docs/coreex/agents/CoreEx.AspNetCore.md` -- Web API patterns - `.github/docs/coreex/agents/CoreEx.Caching.FusionCache.md` -- caching - + - `.github/docs/coreex/agents/CoreEx.EntityFrameworkCore.md` -- EF Core patterns + +- `.github/docs/coreex/agents/CoreEx.Cosmos.md` -- Cosmos DB patterns + - `.github/docs/coreex/agents/CoreEx.RefData.md` -- reference data patterns diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/GlobalUsing.cs index 15dbfe594..3e8cc7bd3 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/GlobalUsing.cs +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/GlobalUsing.cs @@ -1,7 +1,10 @@ global using CoreEx; global using CoreEx.AspNetCore.Mvc; global using CoreEx.Caching; -// #if has-data-provider +// #if implement-cosmos +global using CoreEx.Cosmos; +// #endif +// #if implement-relational global using CoreEx.Database; // #endif // #if implement-sqlserver @@ -16,7 +19,11 @@ global using CoreEx.RefData; // #endif global using CoreEx.Validation; +global using Microsoft.AspNetCore.Authorization; global using Microsoft.AspNetCore.Mvc; +// #if implement-cosmos +global using Microsoft.Azure.Cosmos; +// #endif global using NSwag.Annotations; global using System.Net; global using System.Text.Json; diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/Program.cs b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/Program.cs index 0c731adcf..f8c0dee75 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/Program.cs +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/Program.cs @@ -24,6 +24,7 @@ private static void Main(string[] args) builder.Services .AddPrecisionTimeProvider() .AddExecutionContext() + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider (default); events go to the shared topic, commands to a per-domain queue. // #if refdata-enabled .AddReferenceDataOrchestrator() // Resolves the (CodeGen-generated) IReferenceDataProvider from DI at runtime — no compile-time dependency on the generated type. // #endif @@ -62,6 +63,27 @@ private static void Main(string[] args) // #endif .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. +// #elif implement-cosmos + // Add the Cosmos DB client (Aspire); in Development, accept the local emulator's self-signed certificate and use Gateway mode. + builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => + { + o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; + + if (builder.Environment.IsDevelopment()) + { + o.ConnectionMode = ConnectionMode.Gateway; + o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); + } + }); + + builder.Services.AddCosmosDb("domain-name-lower"); // Adds the CoreEx extended Cosmos DB service for the database. + builder.Services + .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. +// #if outbox-enabled + .AddCosmosDbEventPublisher() // Adds the CosmosDbEventPublisher as the IEventPublisher. +// #endif + .AddCosmosDbUnitOfWork() // Adds the CosmosDbUnitOfWork as the IUnitOfWork. + .AddCosmosDbHealthCheck(); // Adds the CosmosDbHealthCheck; Aspire's AddAzureCosmosClient does not register one. // #elif implement-postgres builder.AddNpgsqlDataSource("Postgres"); // Adds the NpgsqlDataSource (using Aspire library). builder.Services @@ -92,6 +114,8 @@ private static void Main(string[] args) builder.WithCoreExTelemetry() // #if implement-sqlserver .WithCoreExSqlServerTelemetry() +// #elif implement-cosmos + .WithCoreExCosmosDbTelemetry() // #elif implement-postgres .WithCoreExPostgresTelemetry() // #endif diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/app-name.Api.csproj b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/app-name.Api.csproj index 61a61690a..a6111214f 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/app-name.Api.csproj +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/app-name.Api.csproj @@ -13,6 +13,9 @@ + + + diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.Development.json b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.Development.json index e99d5fb4b..19a6531ba 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.Development.json +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.Development.json @@ -11,6 +11,11 @@ "StackExchange": "Warning" } }, +//#if implement-cosmos + "ConnectionStrings": { + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==" + }, +//#endif "Aspire": { //#if implement-sqlserver "Microsoft": { diff --git a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.json b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.json index c68c250bd..49f9fc8de 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.json +++ b/src/CoreEx.Template/content/CoreEx.Api/src/app-name.Api/appsettings.json @@ -3,6 +3,16 @@ "Host": { "SolutionName": "solution-parent-name", "DomainName": "domain-name" +// #if (implement-sqlserver || implement-postgres) && outbox-enabled + ,"Outbox": { + "PartitionSize": 4 + } +// #endif +// #if implement-servicebus + ,"ServiceBus": { + "SessionIdPartitionSize": 4 + } +// #endif }, "Events": { "Destination": "domain-parent-lower" // Topic/queue name. diff --git a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/GlobalUsing.cs index 1ee0d6d54..dc3718a3a 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/GlobalUsing.cs +++ b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/GlobalUsing.cs @@ -1,4 +1,7 @@ global using CoreEx; +// #if (implement-cosmos && outbox-enabled) +global using CoreEx.Cosmos.Outbox; +// #endif // #if (implement-postgres && outbox-enabled) global using CoreEx.Database.Postgres.Outbox; // #endif @@ -11,7 +14,7 @@ global using UnitTestEx; global using UnitTestEx.NUnit; global using UnitTestEx.Expectations; -// #if (implement-sqlserver || implement-postgres) +// #if has-data-provider global using DbMigration = solution-name.Database.Program; global using TestData = solution-name.Test.Common.TestData; // #endif diff --git a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/HostTests.cs b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/HostTests.cs index 8b1e7f23c..e8231040b 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/HostTests.cs +++ b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/HostTests.cs @@ -7,6 +7,8 @@ public async Task OneTimeSetUpAsync() { // #if implement-sqlserver await Test.MigrateSqlServerDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); +// #elif implement-cosmos + await Test.MigrateCosmosDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureProvisionArgs).ConfigureAwait(false); // #elif implement-postgres await Test.MigratePostgresDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // #endif @@ -64,6 +66,8 @@ public void Health_Detailed(string path, bool minimal) "$.entries['stackExchange.Redis']", // #if implement-sqlserver "$.entries.sqlServer" +// #elif implement-cosmos + "$.entries.cosmos-database" // #elif implement-postgres "$.entries.postgreSql" // #endif diff --git a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/solution-name.Test.Api.csproj b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/solution-name.Test.Api.csproj index 619335bc3..4dfc1d6bf 100644 --- a/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/solution-name.Test.Api.csproj +++ b/src/CoreEx.Template/content/CoreEx.Api/tests/solution-name.Test.Api/solution-name.Test.Api.csproj @@ -12,7 +12,7 @@ - + diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/.template.config/template.json b/src/CoreEx.Template/content/CoreEx.Aspire/.template.config/template.json index 02d700e7b..4a7b26acb 100644 --- a/src/CoreEx.Template/content/CoreEx.Aspire/.template.config/template.json +++ b/src/CoreEx.Template/content/CoreEx.Aspire/.template.config/template.json @@ -72,6 +72,86 @@ "displayName": "Has Subscribe host", "description": "Indicates this solution already has (or will have) a Subscribe host to orchestrate." }, + "data-provider": { + "type": "parameter", + "datatype": "choice", + "choices": [ + { + "choice": "SqlServer", + "displayName": "SQL Server", + "description": "Indicates to use SQL Server as the data provider." + }, + { + "choice": "Postgres", + "displayName": "PostgreSQL", + "description": "Indicates to use PostgreSQL as the data provider." + }, + { + "choice": "Cosmos", + "displayName": "Azure Cosmos DB", + "description": "Indicates to use Azure Cosmos DB as the data provider." + }, + { + "choice": "None", + "displayName": "None", + "description": "Indicates no data provider is used (e.g. a facade over an external system such as Dynamics 365)." + } + ], + "defaultValue": "Postgres", + "displayName": "Data provider", + "description": "Determines the data provider technology to use." + }, + "implement-sqlserver": { + "type": "computed", + "value": "data-provider == 'SqlServer'" + }, + "implement-postgres": { + "type": "computed", + "value": "data-provider == 'Postgres'" + }, + "implement-cosmos": { + "type": "computed", + "value": "data-provider == 'Cosmos'" + }, + "implement-relational": { + "type": "computed", + "value": "(data-provider == 'SqlServer' || data-provider == 'Postgres')" + }, + "implement-none-data": { + "type": "computed", + "value": "data-provider == 'None'" + }, + "has-data-provider": { + "type": "computed", + "value": "data-provider != 'None'" + }, + "messaging-provider": { + "type": "parameter", + "datatype": "choice", + "choices": [ + { + "choice": "ServiceBus", + "displayName": "Azure Service Bus", + "description": "Indicates to use Azure Service Bus as the messaging provider." + }, + { + "choice": "None", + "displayName": "None", + "description": "Indicates no messaging provider is used." + } + ], + "defaultValue": "ServiceBus", + "displayName": "Messaging provider", + "description": "Determines the messaging provider technology to use." + }, + "implement-servicebus": { + "type": "computed", + "value": "messaging-provider == 'ServiceBus'" + }, + "implement-none-messaging": { + "type": "computed", + "value": "messaging-provider == 'None'" + }, "appHttpPort": { "type": "parameter", "datatype": "integer", @@ -225,6 +305,13 @@ "replaces": "20089" } }, + "sources": [ + { + "rename": { + "aspire/_Directory.Build.props": "aspire/Directory.Build.props" + } + } + ], "specialCustomOperations": { "*.md": { "operations": [ diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/_Directory.Build.props b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/_Directory.Build.props new file mode 100644 index 000000000..77633f74c --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/_Directory.Build.props @@ -0,0 +1,8 @@ + + + + + + IDE1006 + + diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/Program.cs b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/Program.cs new file mode 100644 index 000000000..40bce4621 --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/Program.cs @@ -0,0 +1,6 @@ +await WireMockConsole.RunAsync(settings => +{ + // Stubbed request/response logging is verbose under load; raise to LogLevel.Information when diagnosing mock matching. + settings.RequestResponseLogLevel = LogLevel.None; + return WireMockServer.Start(settings); +}); diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/app-name.Aspire.MockHost.csproj b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/app-name.Aspire.MockHost.csproj new file mode 100644 index 000000000..8748f23c7 --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire.MockHost/app-name.Aspire.MockHost.csproj @@ -0,0 +1,33 @@ + + + + + + Exe + enable + enable + false + + + + + + + + + + + + + + diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AGENTS.md new file mode 100644 index 000000000..ca91f06b2 --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AGENTS.md @@ -0,0 +1,77 @@ +# domain-name Aspire AppHost -- AI Agent Guide + +This is the **.NET Aspire AppHost** for the `domain-name` domain, part of the `solution-name` microservice. It +orchestrates this solution's own runtime hosts for local development and exposes the Aspire dashboard. + +> **Before answering any CoreEx question:** check whether `.github/docs/coreex/` is populated at the solution root. +> If empty, run `/coreex-docs-sync` first. `.github/docs/coreex/local-dev.md` and `.github/docs/coreex/aspire.md` +> are especially relevant to this project. + +--- + +## What This Project Does + +`AppHost.cs` first declares visible connection-string resources (SQL Server/Postgres/Cosmos DB, Redis, Service Bus -- whichever +apply per `data-provider`/`messaging-provider`), matching the connection name each host passes to its own Aspire +client-integration package (e.g. `AddAzureNpgsqlDataSource("Postgres")`). It then calls +`builder.AddProject(...)` once per runtime host this solution has, chaining `.WithReference(...)` for +only the resources that host actually needs (Api: data provider + Redis; Relay: data provider + Service Bus; +Subscribe: data provider + Redis + Service Bus), then runs the distributed application. It does **not** contain +business logic or DI registrations for the hosts themselves -- those live in each host's own +`Program.cs`/`appsettings*.json`. This project only wires hosts and their dependencies together and adds dashboard +sugar (`AddEndpoints`/`AddCommand`/`AddHostedServiceSupport`/`DisableHttpCertificateValidation`, provided by the +`CoreEx.UnitTesting` package referenced by this project -- see `` in +`app-name.Aspire.csproj`). + +`AddExternalConnectionString` is supplied by `CoreEx.UnitTesting` in the `UnitTestEx` namespace, alongside +the other Aspire helpers; do not duplicate it locally. It wraps secret parameters from `ConnectionStrings:{name}` in visible `ConnectionStringResource` +nodes; plain `AddConnectionString(name)` creates parameters excluded from Aspire's Graph/Table views. +Keep the existing connection names and `.WithReference(...)` chains. Compose owns the infrastructure lifecycle; +the node's running status indicates resolved configuration, not a successful connectivity probe. Hosts own readiness +checks. Cosmos exposes its configured `AccountEndpoint` as a resource URL for peer matching; never include credentials +in dashboard URLs. Use explicit PostgreSQL/SQL Server ports and inspect span endpoint attributes before diagnosing +mislabelled dependencies; shared localhost addresses do not identify the database provider. + +This solution was generated with: + + +- **Api host** included. + + +- **Relay host** included. + + +- **Subscribe host** included. + + +## Sibling Projects + +- `app-name.Aspire.MockHost` -- a WireMock.Net-based console host for stubbing out external dependencies during + local runs/tests via `AddMockHostProject(...)` (`UnitTestEx.Aspire`). + Currently referenced but not wired into `AppHost.cs` -- uncomment the `mockhost` line once it has actual + request/response mappings configured. +- `solution-name.Test.Aspire` -- an NUnit project that spins up this AppHost via `WithAspireTester` + for smoke/integration testing across all orchestrated hosts together. + +## Guidance + +For AppHost wiring, service-bus topology, third-party HTTP (MockHost) versus in-AppHost domains (`WithReference`), and the `Test.Aspire` `OnBeforeStartAsync` / `OnAfterStartAsync` lifecycle, see the `coreex-aspire` skill (`.github/skills/coreex-aspire/SKILL.md`) and `.github/instructions/coreex-aspire.instructions.md` when installed via `dotnet new coreex-ai`. + +## Adding a Host Later + +If a new `Api`, `Relay`, or `Subscribe` host is added to this solution *after* this AppHost was generated, do not +re-run `dotnet new coreex-aspire --force` -- it will overwrite any customisation already made here. Instead, add +the missing pieces by hand: + +1. A `` to the new host's `.csproj` in `app-name.Aspire.csproj`. +2. A `builder.AddProject("...")` call in `AppHost.cs`, chaining `.WithReference(...)` for whichever + resources it needs, following the pattern already used for the other hosts. + +## Running + +```sh +dotnet run --project aspire/solution-name.Aspire +``` + +Consult `.github/docs/coreex/local-dev.md` and `.github/docs/coreex/aspire.md` for the full local-development +workflow, including the Aspire CLI (`aspire run`, `aspire logs`, etc.). diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AppHost.cs b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AppHost.cs new file mode 100644 index 000000000..964ad2c1c --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/AppHost.cs @@ -0,0 +1,61 @@ +var builder = DistributedApplication.CreateBuilder(args); + +// dotnet dev-certs https --trust is not fully supported on Linux, so the ASP.NET Core dev cert used by each +// project resource's https endpoint is not OS-trusted on Linux CI runners. Both the health check probes (via +// AddEndpoints) and AspireTesterBase's CreateHttpClient() resolve their HttpClient via this same DI container's +// IHttpClientFactory, so disabling certificate validation here (dev/test-only AppHost, never shipped) covers both. +builder.DisableHttpCertificateValidation(); + +// IsRunMode: never surface this test-only resource in a published manifest (per the self-hosted WireMock.Net housekeeping note - see AppHost.cs's header comment). +//var mockhost = builder.AddMockHostProject("mock-host"); + +// Compose owns the infrastructure lifecycle; visible connection-string resources expose graph dependencies. +// Secret parameters retain the existing ConnectionStrings configuration and are not graph nodes. +// #if implement-postgres +var db = builder.AddExternalConnectionString("Postgres").WithIconName("DatabaseMultiple"); +// #elif implement-cosmos +var db = builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint").WithIconName("DatabaseMultiple"); +// #elif implement-sqlserver +var db = builder.AddExternalConnectionString("SqlServer").WithIconName("DatabaseMultiple"); +// #endif +var redis = builder.AddExternalConnectionString("redis").WithIconName("Database"); +// #if implement-servicebus +var serviceBus = builder.AddExternalConnectionString("ServiceBus").WithIconName("MailMultiple"); +// #endif +// domain-name domain. +// #if has-api + +builder.AddProject("domain-name-lower-api") +// #if has-data-provider + .WithReference(db) +// #endif + .WithReference(redis) + .AddEndpoints("/health/ready/detailed"); +// #endif +// #if has-relay + +builder.AddProject("domain-name-lower-relay") +// #if has-data-provider + .WithReference(db) +// #endif +// #if implement-servicebus + .WithReference(serviceBus) +// #endif + .AddEndpoints("/health/ready/detailed") + .AddHostedServiceSupport(); +// #endif +// #if has-subscribe + +builder.AddProject("domain-name-lower-subscribe") +// #if has-data-provider + .WithReference(db) +// #endif + .WithReference(redis) +// #if implement-servicebus + .WithReference(serviceBus) +// #endif + .AddEndpoints("/health/ready/detailed") + .AddHostedServiceSupport(); +// #endif + +builder.Build().Run(); diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/Properties/launchSettings.json b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/Properties/launchSettings.json similarity index 100% rename from src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/Properties/launchSettings.json rename to src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/Properties/launchSettings.json diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/app-name.Aspire.csproj b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/app-name.Aspire.csproj similarity index 51% rename from src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/app-name.Aspire.csproj rename to src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/app-name.Aspire.csproj index 68b5f8a66..d27dc23bb 100644 --- a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/app-name.Aspire.csproj +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/app-name.Aspire.csproj @@ -1,4 +1,4 @@ - + Exe enable @@ -12,14 +12,23 @@ + + + + + + + + + - + - + - + diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/appsettings.Development.json b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/appsettings.Development.json new file mode 100644 index 000000000..d321710de --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/appsettings.Development.json @@ -0,0 +1,21 @@ +{ + "Logging": { + "LogLevel": { + "Default": "Information", + "Microsoft.AspNetCore": "Warning" + } + }, + "ConnectionStrings": { +//#if implement-postgres + "Postgres": "Server=127.0.0.1;Database=domain-name-lower;Username=postgres;******", +//#elif implement-cosmos + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==", +//#elif implement-sqlserver + "SqlServer": "Data Source=127.0.0.1,1433;Initial Catalog=domain-name;User id=sa;******;TrustServerCertificate=true", +//#endif +//#if implement-servicebus + "ServiceBus": "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;", +//#endif + "redis": "localhost:6379" + } +} diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/appsettings.json b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/appsettings.json similarity index 100% rename from src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/appsettings.json rename to src/CoreEx.Template/content/CoreEx.Aspire/aspire/app-name.Aspire/appsettings.json diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/GlobalUsing.cs new file mode 100644 index 000000000..2e19ab58f --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/GlobalUsing.cs @@ -0,0 +1,13 @@ +global using Aspire.Hosting; +global using AwesomeAssertions; +global using CoreEx; +// #if has-data-provider +global using DbMigration = solution-name.Database.Program; +// #endif +global using NUnit.Framework; +global using System.Net; +// #if has-data-provider +global using TestData = solution-name.Test.Common.TestData; +// #endif +global using UnitTestEx; +global using UnitTestEx.Expectations; diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/HostTests.cs b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/HostTests.cs new file mode 100644 index 000000000..85afc6715 --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/HostTests.cs @@ -0,0 +1,70 @@ +namespace solution-name.Test.Aspire; + +public class HostTests : WithAspireTester +{ + protected override async Task OnBeforeStartAsync(DistributedApplication app) + { +// #if implement-sqlserver + await app.MigrateSqlServerDataAsync("SqlServer", ["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs); +// #elif implement-cosmos + await app.MigrateCosmosDataAsync("Cosmos", "domain-name-lower", ["no-data.seed.yaml"], DbMigration.ConfigureProvisionArgs); +// #elif implement-postgres + await app.MigratePostgresDataAsync("Postgres", ["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs); +// #endif + + // Clear the Redis cache. + await app.ClearRedisCacheAsync("redis"); + +// #if implement-servicebus + // Reset the Azure Service Bus queues and topics/subscriptions to an initial state (see the Test.Common ServiceBus). + await app.ResetAzureServiceBusAsync("ServiceBus", Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()); +// #endif + } + + protected override async Task OnAfterStartAsync(DistributedApplication app) + { + // Hang about until the host(s) are available. + await app.WaitForResourceAsync([ +// #if has-api + "domain-name-lower-api", +// #endif +// #if has-relay + "domain-name-lower-relay", +// #endif +// #if has-subscribe + "domain-name-lower-subscribe", +// #endif + ]); + } + +// #if has-api + [TestCase("/health/live")] + [TestCase("/health/ready")] + public Task Api_Health(string path) => AssertHealthyAsync("domain-name-lower-api", path); +// #endif + +// #if has-relay + [Test] + public Task Relay_Health() => AssertHealthyAsync("domain-name-lower-relay", "/health/ready"); +// #endif + +// #if has-subscribe + [Test] + public Task Subscribe_Health() => AssertHealthyAsync("domain-name-lower-subscribe", "/health/ready"); +// #endif + + private async Task AssertHealthyAsync(string resourceName, string path) + { + for (var attempt = 0; attempt < 60; attempt++) + { + var response = Test.Http(resourceName).Run(HttpMethod.Get, path); + if (response.Response.StatusCode != HttpStatusCode.ServiceUnavailable || attempt == 59) + { + response.Response.StatusCode.Should().Be(HttpStatusCode.OK, $"{resourceName} {path} must report healthy"); + return; + } + + await Task.Delay(TimeSpan.FromSeconds(1)); + } + } +} diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/solution-name.Test.Aspire.csproj b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/solution-name.Test.Aspire.csproj new file mode 100644 index 000000000..ef191537d --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Aspire/aspire/solution-name.Test.Aspire/solution-name.Test.Aspire.csproj @@ -0,0 +1,26 @@ + + + false + + + + + + + + + + + + + + + + + + + + + + + diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AGENTS.md deleted file mode 100644 index ac4d78231..000000000 --- a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AGENTS.md +++ /dev/null @@ -1,59 +0,0 @@ -# domain-name Aspire AppHost -- AI Agent Guide - -This is the **.NET Aspire AppHost** for the `domain-name` domain, part of the `solution-name` microservice. It -orchestrates this solution's own runtime hosts for local development and exposes the Aspire dashboard. - -> **Before answering any CoreEx question:** check whether `.github/docs/coreex/` is populated at the solution root. -> If empty, run `/coreex-docs-sync` first. `.github/docs/coreex/local-dev.md` and `.github/docs/coreex/aspire.md` -> are especially relevant to this project. - ---- - -## What This Project Does - -`AppHost.cs` calls `builder.AddProject(...)` once per runtime host this solution has, then runs the -distributed application. It does **not** contain business logic, DI registrations for the hosts themselves, or -connection strings -- those live in each host's own `Program.cs`/`appsettings*.json`. This project only wires -hosts together and adds dashboard sugar (see `Extensions.cs`). - -This solution was generated with: - - -- **Api host** included. - - -- **Relay host** included. - - -- **Subscribe host** included. - - -## Adding a Host Later - -If a new `Api`, `Relay`, or `Subscribe` host is added to this solution *after* this AppHost was generated, do not -re-run `dotnet new coreex-aspire --force` -- it will overwrite any customisation already made here. Instead, add -the missing pieces by hand: - -1. A `` to the new host's `.csproj` in `app-name.Aspire.csproj`. -2. A `builder.AddProject("...")` call in `AppHost.cs`, following the pattern already used for the - other hosts. - -## `Extensions.cs` - -Provides fluent dashboard sugar used from `AppHost.cs`: - -- `AddEndpoints(...)` -- annotates dashboard-visible URLs (e.g. health-check deep links) on a resource. -- `AddCommand(...)` -- adds a dashboard button that invokes an HTTP verb against an endpoint. -- `AddHostedServiceSupport()` -- composes the above to add "Pause all services"/"Resume all services" dashboard - buttons for hosts that run `CoreEx` hosted services (Relay and Subscribe hosts expose - `/hosted-services/all/{status,pause,resume}` via `MapHostedServices()`; the Api host does not, so it is never - called there). - -## Running - -```sh -dotnet run --project src/solution-name.Aspire -``` - -Consult `.github/docs/coreex/local-dev.md` and `.github/docs/coreex/aspire.md` for the full local-development -workflow, including the Aspire CLI (`aspire run`, `aspire logs`, etc.). diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AppHost.cs b/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AppHost.cs deleted file mode 100644 index a3442f902..000000000 --- a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/AppHost.cs +++ /dev/null @@ -1,14 +0,0 @@ -var builder = DistributedApplication.CreateBuilder(args); - -// domain-name domain. -// #if has-api -builder.AddProject("domain-name-lower-api").AddEndpoints("/health/ready/detailed"); -// #endif -// #if has-relay -builder.AddProject("domain-name-lower-relay").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); -// #endif -// #if has-subscribe -builder.AddProject("domain-name-lower-subscribe").AddEndpoints("/health/ready/detailed").AddHostedServiceSupport(); -// #endif - -builder.Build().Run(); diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/Extensions.cs b/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/Extensions.cs deleted file mode 100644 index bbf1c3455..000000000 --- a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/Extensions.cs +++ /dev/null @@ -1,49 +0,0 @@ -/// -/// Provides extensions that add Aspire dashboard sugar (deep-linked endpoints -/// and command buttons) for the s this AppHost orchestrates. -/// -internal static class Extensions -{ - /// - /// Annotates the resource's http endpoint with one or more relative so they appear as clickable links in the Aspire dashboard. - /// - /// The of . - /// The relative URLs to add, e.g. "/health/ready/detailed". - /// The for fluent-style method-chaining. - public static IResourceBuilder AddEndpoints(this IResourceBuilder builder, params string[] urls) - { - var httpEndpoint = builder.GetEndpoint("http"); - foreach (var url in urls) - { - builder.WithAnnotation(new ResourceUrlAnnotation { Endpoint = httpEndpoint, Url = url }); - } - - return builder; - } - - /// - /// Adds a dashboard command button that invokes an HTTP against a relative on the resource. - /// - /// The of . - /// The to invoke. - /// The relative path to invoke, e.g. "/hosted-services/all/pause". - /// The button's display name shown in the dashboard. - /// The optional Fluent UI icon name for the button; see the icon catalog. - /// The for fluent-style method-chaining. - public static IResourceBuilder AddCommand(this IResourceBuilder builder, HttpMethod method, string path, string displayName, string? iconName) - => builder.WithHttpCommand( - path: path, - displayName: displayName, - commandOptions: new HttpCommandOptions() { Method = method, IconName = iconName }); - - /// - /// Adds dashboard support for the standard CoreEx hosted-service management endpoints (/hosted-services/all/{status,pause,resume} via MapHostedServices()) -- a status link plus "Pause all services"/"Resume all services" command buttons. - /// - /// The of . - /// The for fluent-style method-chaining. - /// The stock Api host template doesn't register AddHostedServiceManager()/MapHostedServices() -- Relay and Subscribe do. This split is a workload-isolation convention, not a technical requirement; for a small, low-traffic solution, consolidating hosted-service processing into the Api host is a reasonable simplification. Call this method wherever those endpoints are actually mapped. - public static IResourceBuilder AddHostedServiceSupport(this IResourceBuilder builder) - => builder.AddEndpoints("/hosted-services/all/status") - .AddCommand(HttpMethod.Post, "/hosted-services/all/pause", "Pause all services", "Pause") - .AddCommand(HttpMethod.Post, "/hosted-services/all/resume", "Resume all services", "PauseOff"); -} diff --git a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/appsettings.Development.json b/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/appsettings.Development.json deleted file mode 100644 index 0967ef424..000000000 --- a/src/CoreEx.Template/content/CoreEx.Aspire/src/app-name.Aspire/appsettings.Development.json +++ /dev/null @@ -1 +0,0 @@ -{} diff --git a/src/CoreEx.Template/content/CoreEx.Core/.template.config/template.json b/src/CoreEx.Template/content/CoreEx.Core/.template.config/template.json index eb7651bf8..0e7f0462d 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/.template.config/template.json +++ b/src/CoreEx.Template/content/CoreEx.Core/.template.config/template.json @@ -103,6 +103,11 @@ "displayName": "PostgreSQL", "description": "Indicates to use PostgreSQL as the data provider." }, + { + "choice": "Cosmos", + "displayName": "Azure Cosmos DB", + "description": "Indicates to use Azure Cosmos DB as the data provider." + }, { "choice": "None", "displayName": "None", @@ -121,6 +126,14 @@ "type": "computed", "value": "data-provider == 'Postgres'" }, + "implement-cosmos": { + "type": "computed", + "value": "data-provider == 'Cosmos'" + }, + "implement-relational": { + "type": "computed", + "value": "(data-provider == 'SqlServer' || data-provider == 'Postgres')" + }, "implement-none-data": { "type": "computed", "value": "data-provider == 'None'" @@ -169,6 +182,7 @@ "rename": { "_Directory.Build.props": "Directory.Build.props", "_Directory.Packages.props": "Directory.Packages.props", + "tests/_Directory.Build.props": "tests/Directory.Build.props", "_editorconfig": ".editorconfig", "_gitattributes": ".gitattributes", "_gitignore": ".gitignore", @@ -178,7 +192,7 @@ { "condition": "(!refdata-enabled)", "exclude": [ - "src/app-name.Application/ReferenceDataService.g.cs", + "src/app-name.Application/ReferenceDataProvider.g.cs", "src/app-name.Application/Repositories/IReferenceDataRepository.g.cs", "src/app-name.Infrastructure/Repositories/ReferenceDataRepository.cs", "tools/app-name.CodeGen/**", @@ -192,9 +206,23 @@ "tools/app-name.CodeGen/**", "src/app-name.Infrastructure/Repositories/domain-nameDbContext.cs", "src/app-name.Infrastructure/Repositories/domain-nameEfDb.cs", + "src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs", "tests/app-name.Test.Common/Data/**" ] }, + { + "condition": "(implement-cosmos)", + "exclude": [ + "tools/app-name.Database/dbex.yaml", + "tools/app-name.Database/Migrations/**", + "src/app-name.Infrastructure/Repositories/domain-nameDbContext.cs", + "src/app-name.Infrastructure/Repositories/domain-nameEfDb.cs" + ] + }, + { + "condition": "(!implement-cosmos)", + "exclude": [ "src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs" ] + }, { "condition": "(implement-sqlserver)", "exclude": [ "tools/app-name.Database/Migrations/*.pgsql" ] @@ -209,7 +237,7 @@ }, { "condition": "(!implement-servicebus)", - "exclude": [ "servicebus/**" ] + "exclude": [ "servicebus/**", "tests/app-name.Test.Common/ServiceBus.cs", "tests/app-name.Test.Common/GlobalUsing.cs" ] } ] } @@ -361,4 +389,4 @@ "steps": [ "ValueSecondToLastDotDelimited", "LowerCase" ] } } -} \ No newline at end of file +} diff --git a/src/CoreEx.Template/content/CoreEx.Core/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Core/AGENTS.md index 9f996add0..6cdf9fb44 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/AGENTS.md +++ b/src/CoreEx.Template/content/CoreEx.Core/AGENTS.md @@ -28,12 +28,12 @@ app-name/ | +-- app-name.Contracts/ # Public contracts: entities, DTOs, event schemas | +-- app-name.Application/ # Business logic: services, validators, repository interfaces | +-- app-name.Domain/ # (coreex-domain addon, optional) Aggregates, value objects, domain events -| +-- app-name.Infrastructure/ # EF Core repositories, outbox, external integrations +| +-- app-name.Infrastructure/ # EF Core / Cosmos DB repositories, outbox, external integrations +-- tools/ -| +-- app-name.Database/ # (data-provider != None) Database migrations (DbEx) +| +-- app-name.Database/ # (data-provider != None) Database migrations (DbEx) or, for Cosmos, container provisioning + seed data | +-- app-name.CodeGen/ # (refdata-enabled && data-provider != None) Ref-data code gen +-- tests/ -| +-- app-name.Test.Common/ # Shared test infrastructure: TestData marker, embedded seed data +| +-- app-name.Test.Common/ # Shared test infrastructure: TestData marker, embedded seed data, ServiceBus (code-based topic/queue configuration) | +-- app-name.Test.Unit/ # Fast isolated unit tests (validators, services, no I/O) +-- Directory.Packages.props # Central NuGet version management (no versions in .csproj) ``` @@ -46,11 +46,13 @@ app-name/ - **Data provider:** SQL Server (`CoreEx.Database.SqlServer`, `CoreEx.EntityFrameworkCore`) - **Data provider:** PostgreSQL (`CoreEx.Database.Postgres`, `CoreEx.EntityFrameworkCore`) + +- **Data provider:** Azure Cosmos DB (`CoreEx.Cosmos`) -- no EF Core; containers declared in `domain-nameCosmosDb` and provisioned by `tools/app-name.Database` (`CosmosDbConsole`) - **Data provider:** None -- facade solution (e.g. over Dynamics 365 via HttpClient) -- **Reference data:** Enabled -- `src/app-name.Application/ReferenceDataService.cs` and `tools/app-name.CodeGen/` +- **Reference data:** Enabled -- `src/app-name.Application/ReferenceDataProvider.g.cs` and `tools/app-name.CodeGen/` - **Reference data:** Disabled @@ -82,8 +84,7 @@ app-name/ - `.github/docs/coreex/application-scaffolding-guide.md` -- choosing the smallest safe CoreEx solution shape before adding code - `.github/docs/coreex/contracts-layer.md` -- entities, DTOs, event schemas - `.github/docs/coreex/application-layer.md` -- services, validators, repository interfaces -- `.github/docs/coreex/infrastructure-layer.md` -- EF Core, outbox, external integrations +- `.github/docs/coreex/infrastructure-layer.md` -- EF Core / Cosmos DB, outbox, external integrations - `.github/docs/coreex/testing.md` -- test project setup, `WithGenericTester`, `WithApiTester` - `.github/docs/coreex/local-dev.md` -- running locally with .NET Aspire - `.github/docs/coreex/tooling.md` -- Database and CodeGen tool projects - diff --git a/src/CoreEx.Template/content/CoreEx.Core/README.md b/src/CoreEx.Template/content/CoreEx.Core/README.md index 5d203d896..f98075895 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/README.md +++ b/src/CoreEx.Template/content/CoreEx.Core/README.md @@ -10,11 +10,14 @@ A CoreEx microservice for the `domain-name` domain. - **Data provider:** PostgreSQL (`CoreEx.Database.Postgres`, `CoreEx.EntityFrameworkCore`) + +- **Data provider:** Azure Cosmos DB (`CoreEx.Cosmos`) + - **Data provider:** None — facade over an external system (no local database) -- **Reference data:** Enabled — `tools/app-name.CodeGen/` + `src/app-name.Application/ReferenceDataService.cs` +- **Reference data:** Enabled — `tools/app-name.CodeGen/` + `src/app-name.Application/ReferenceDataProvider.g.cs` - **Railway-oriented programming:** Enabled — services return `Result`/`Result` @@ -52,16 +55,20 @@ app-name/ ├── src/ │ ├── app-name.Contracts/ # Public contracts, DTOs, event schemas │ ├── app-name.Application/ # Services, validators, repository interfaces -│ └── app-name.Infrastructure/ # EF Core repositories, outbox, external adapters +│ └── app-name.Infrastructure/ # EF Core / Cosmos DB repositories, outbox, external adapters ├── tools/ │ ├── app-name.CodeGen/ # Reference data C# generation (reads ref-data.yaml) + +│ └── app-name.Database/ # Cosmos DB database/container provisioning and seeding + │ └── app-name.Database/ # Database migrations and seeding (DbEx) + └── tests/ - ├── app-name.Test.Common/ # Shared test infrastructure and seed data + ├── app-name.Test.Common/ # Shared test infrastructure, seed data and (Service Bus) topology └── app-name.Test.Unit/ # Fast isolated unit tests (no I/O) ``` @@ -87,6 +94,9 @@ docker compose up -d # Docker | `db-postgres` | 5432 | PostgreSQL — domain schema and data | + +| `cosmos-emulator` | 8081 (HTTPS) | Azure Cosmos DB (vNext) emulator — domain database and containers | + | `redis-cache` | 6379 | Redis — FusionCache distributed backplane | | `servicebus-emulator` | 5672 (AMQP), 5300 (mgmt) | Azure Service Bus emulator | @@ -106,6 +116,18 @@ This solution is a facade (no local database). Start Redis for caching and the d podman compose up -d ``` + + +### Connection strings + +The connection string is in each host's `appsettings.Development.json` under `ConnectionStrings:Cosmos` (the well-known local emulator account key). The database identifier is `domain-name-lower`: + +```json +"ConnectionStrings": { + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==" +} +``` + ### Connection strings @@ -141,7 +163,25 @@ Connection strings are in each host's `appsettings.Development.json` under the ` --- - + +## Database + +The Cosmos DB database and containers are provisioned (not migrated) from code, then seeded — required once on first run and after any container change: + +```bash +dotnet run --project tools/app-name.Database -- All +``` + +| Command | Effect | +|---|---| +| `-- All` | Create the database and containers, then import the seed data | +| `-- DropAndAll` | Drop the database, then create, provision and seed it | +| `-- ResetAndData` | Empty the containers and re-import the seed data (the database must exist) | +| `--help` | List all commands and options | + +Containers are declared in `tools/app-name.Database/Program.cs` (`ConfigureProvisionArgs`) and must match `src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs`; seed data is in `tools/app-name.Database/Data/` (top-level key = container id). + + ## Database Migrate and seed — required once on first run and after any schema change: @@ -174,7 +214,7 @@ dotnet run --project tools/app-name.CodeGen Commit the generated `*.g.cs` files alongside the `ref-data.yaml` changes. **Never edit generated files by hand** — they are overwritten on the next run. > **Run this before your first `dotnet run`, not just after editing `ref-data.yaml`.** The scaffold's -> `ReferenceDataService` requires `IReferenceDataRepository`, which only gets a DI registration once CodeGen +> `ReferenceDataProvider` requires `IReferenceDataRepository`, which only gets a DI registration once CodeGen > generates it. `dotnet build` succeeds either way, but starting a host with `ASPNETCORE_ENVIRONMENT=Development` > (the default for IDE launch profiles) before CodeGen has ever run throws a DI-validation exception at startup. > diff --git a/src/CoreEx.Template/content/CoreEx.Core/_Directory.Build.props b/src/CoreEx.Template/content/CoreEx.Core/_Directory.Build.props index 1faab5f18..ea804a9df 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/_Directory.Build.props +++ b/src/CoreEx.Template/content/CoreEx.Core/_Directory.Build.props @@ -4,5 +4,9 @@ enable enable latest + + + true + \ No newline at end of file diff --git a/src/CoreEx.Template/content/CoreEx.Core/_Directory.Packages.props b/src/CoreEx.Template/content/CoreEx.Core/_Directory.Packages.props index ce8f0dfef..8beb6b94c 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/_Directory.Packages.props +++ b/src/CoreEx.Template/content/CoreEx.Core/_Directory.Packages.props @@ -16,6 +16,9 @@ + + + @@ -40,6 +43,9 @@ + + + @@ -70,10 +76,10 @@ - + - + @@ -85,6 +91,8 @@ - + + + diff --git a/src/CoreEx.Template/content/CoreEx.Core/docker-compose.yml b/src/CoreEx.Template/content/CoreEx.Core/docker-compose.yml index 9cd5c1d0b..3a6ea5e3b 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/docker-compose.yml +++ b/src/CoreEx.Template/content/CoreEx.Core/docker-compose.yml @@ -17,6 +17,14 @@ services: POSTGRES_PASSWORD: "yourStrong#!Password" ports: - "5432:5432" +#endif +#if implement-cosmos + # Azure Cosmos DB "vNext" Linux emulator; the .NET SDK only speaks HTTPS (vNext defaults to HTTP), hence `--protocol https` (self-signed certificate; clients must accept it). + # Host networking is used because rootless Podman on Windows (WSL) does not forward the TLS gateway port correctly via published ports; it also works as-is with Docker on Linux. + cosmos-emulator: + image: mcr.microsoft.com/cosmosdb/linux/azure-cosmos-emulator:vnext-latest + command: ["--protocol", "https", "--health-port", "8083"] + network_mode: host #endif redis-cache: image: docker.io/redis:latest diff --git a/src/CoreEx.Template/content/CoreEx.Core/servicebus/Config.json b/src/CoreEx.Template/content/CoreEx.Core/servicebus/Config.json index 7599a6af3..c5df36229 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/servicebus/Config.json +++ b/src/CoreEx.Template/content/CoreEx.Core/servicebus/Config.json @@ -3,27 +3,8 @@ "Namespaces": [ { "Name": "sbemulatorns", - "Topics": [ - { - "Name": "domain-parent-lower", - "Properties": { - "DefaultMessageTimeToLive": "PT1H", - "RequiresDuplicateDetection": false - }, - "Subscriptions": [ - { - "Name": "domain-name-lower", - "Properties": { - "DefaultMessageTimeToLive": "PT1H", - "LockDuration": "PT1M", - "MaxDeliveryCount": 10, - "DeadLetteringOnMessageExpiration": false, - "RequiresSession": true - } - } - ] - } - ] + "Queues": [], + "Topics": [] } ], "Logging": { diff --git a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataService.g.cs b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataProvider.g.cs similarity index 86% rename from src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataService.g.cs rename to src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataProvider.g.cs index 9ffbae8c6..64689d341 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataService.g.cs +++ b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Application/ReferenceDataProvider.g.cs @@ -6,7 +6,7 @@ namespace app-name.Application; /// Provides the implementation. [ScopedService] -public partial class ReferenceDataService(IReferenceDataRepository repository) : IReferenceDataProvider +public partial class ReferenceDataProvider(IReferenceDataRepository repository) : IReferenceDataProvider { private readonly IReferenceDataRepository _repository = repository.ThrowIfNull(); diff --git a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/GlobalUsing.cs index 39c2df1b7..82f935f6f 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/GlobalUsing.cs +++ b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/GlobalUsing.cs @@ -1,4 +1,7 @@ global using CoreEx; +// #if implement-cosmos +global using CoreEx.Cosmos; +// #endif global using CoreEx.Data; global using CoreEx.Data.Models; // #if (implement-sqlserver || implement-postgres) @@ -26,8 +29,16 @@ // #if refdata-enabled global using CoreEx.RefData; // #endif +global using CoreEx.Results; +// #if implement-cosmos +global using Microsoft.Azure.Cosmos; +// #endif // #if (implement-sqlserver || implement-postgres) global using Microsoft.EntityFrameworkCore; // #endif global using Microsoft.Extensions.DependencyInjection; -global using System.Text.Json.Serialization; \ No newline at end of file +global using System.Text.Json.Serialization; +global using ExecutionContext = CoreEx.ExecutionContext; +// #if implement-cosmos +global using PartitionKey = Microsoft.Azure.Cosmos.PartitionKey; +// #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/ReferenceDataRepository.cs b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/ReferenceDataRepository.cs index 8b8cef96b..f638a237c 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/ReferenceDataRepository.cs +++ b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/ReferenceDataRepository.cs @@ -1,7 +1,14 @@ namespace app-name.Infrastructure.Repositories; /// Provides the implementation (see generated ReferenceDataRepository.g.cs for entity-specific providers). +// #if implement-cosmos +public partial class ReferenceDataRepository(domain-nameCosmosDb cosmos) +{ + private readonly domain-nameCosmosDb _cosmos = cosmos.ThrowIfNull(); +} +// #else public partial class ReferenceDataRepository(domain-nameEfDb ef) { private readonly domain-nameEfDb _ef = ef.ThrowIfNull(); -} \ No newline at end of file +} +// #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs new file mode 100644 index 000000000..455d42a33 --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/Repositories/domain-nameCosmosDb.cs @@ -0,0 +1,18 @@ +namespace app-name.Infrastructure.Repositories; + +/// Provides the domain-name wrapper; declares the containers and their accessors. +/// The container identifiers declared here must match those provisioned by app-name.Database (see Program.ConfigureProvisionArgs). +public class domain-nameCosmosDb(CosmosClient client, string databaseId) : CosmosDb(client, databaseId, _options) +{ +// #if refdata-enabled + private const string RefDataContainerId = "ref-data"; + + // The reference-data container hosts the transactional outbox events for reference data; add further containers (each with their own outbox, as required) alongside. + private static readonly CosmosDbOptions _options = new CosmosDbOptions().Container(RefDataContainerId, c => c.WithReferenceDataOutboxEvent()); + + // Add one accessor per reference data type; e.g. CodeGen emits the persistence model and mapper per type, then expose using: + // public CosmosDbContainer Genders => Container(RefDataContainerId, o => o.WithTypeDiscriminator()); +// #else + private static readonly CosmosDbOptions _options = new(); +// #endif +} diff --git a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/app-name.Infrastructure.csproj b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/app-name.Infrastructure.csproj index ac90b112e..7251735c6 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/app-name.Infrastructure.csproj +++ b/src/CoreEx.Template/content/CoreEx.Core/src/app-name.Infrastructure/app-name.Infrastructure.csproj @@ -15,6 +15,9 @@ + + + diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/_Directory.Build.props b/src/CoreEx.Template/content/CoreEx.Core/tests/_Directory.Build.props new file mode 100644 index 000000000..77633f74c --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/_Directory.Build.props @@ -0,0 +1,8 @@ + + + + + + IDE1006 + + diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/mutate-data.seed.yaml b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/mutate-data.seed.yaml index 1e3f23477..c8a652fbd 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/mutate-data.seed.yaml +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/mutate-data.seed.yaml @@ -1,3 +1,25 @@ +# #if implement-cosmos +# Mutate test dataset for the 'domain-name' domain — loaded by the XxxMutateTests classes via: +# Test.MigrateCosmosDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureProvisionArgs) +# +# Contains the BASELINE documents that POST/PUT/PATCH/DELETE tests act on — e.g. a document to update, a document +# to delete, an existing key for a duplicate-conflict test. Kept separate from 'read-data.seed.yaml' so +# mutations never disturb read expectations. +# +# Cosmos DB YAML format: container id (top-level) -> array of documents. +# * The top-level key MUST match a container declared in Database/Program.cs 'ConfigureProvisionArgs' (otherwise the seed fails). +# * Identifiers use '^N' (deterministic: ^1 == 1.ToGuid()). +# * Property names are camelCase (the CosmosClient serializer naming policy), e.g. 'firstName'. +# * Reference data is linked by CODE (e.g. genderCode: M), never by id. +# * Set scenario flags explicitly where a test needs them (e.g. isDeleted: true). +# * Give each DESTRUCTIVE test its own '^N' document (Update/Delete/Patch target different ids; NUnit randomises order). +# +# Example (replace with your domain's containers/documents): +# +# employees: +# - { id: ^1, firstName: Bob, lastName: Smith, genderCode: M, salary: 50000, dateOfBirth: 1990-01-01 } +# - { id: ^2, firstName: Jane, lastName: Doe, genderCode: F, salary: 60000, dateOfBirth: 1985-06-15 } +# #else # Mutate test dataset for the 'domain-name' domain — loaded by the XxxMutateTests classes via: # MigrateXxxDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs) # @@ -21,3 +43,4 @@ # - employee: # - { employee_id: ^1, first_name: Bob, last_name: Smith, gender_code: M, salary: 50000, date_of_birth: 1990-01-01 } # - { employee_id: ^2, first_name: Jane, last_name: Doe, gender_code: F, salary: 60000, date_of_birth: 1985-06-15 } +# #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/read-data.seed.yaml b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/read-data.seed.yaml index 345f5a785..7faf2fffd 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/read-data.seed.yaml +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/Data/read-data.seed.yaml @@ -1,3 +1,23 @@ +# #if implement-cosmos +# Read test dataset for the 'domain-name' domain — loaded by the XxxReadTests classes via: +# Test.MigrateCosmosDataAsync(["read-data.seed.yaml"], DbMigration.ConfigureProvisionArgs) +# +# Contains the STABLE documents that GET/query tests assert against. Mutate tests load the separate +# 'mutate-data.seed.yaml' so mutations never disturb these read expectations. +# +# Cosmos DB YAML format: container id (top-level) -> array of documents. +# * The top-level key MUST match a container declared in Database/Program.cs 'ConfigureProvisionArgs' (otherwise the seed fails). +# * Identifiers use '^N' (deterministic: ^1 == 1.ToGuid()). +# * Property names are camelCase (the CosmosClient serializer naming policy), e.g. 'firstName'. +# * Reference data is linked by CODE (e.g. genderCode: M), never by id. +# * A reference data container (e.g. 'ref-data') is instead grouped by type: '- Gender: [ { code: M, text: Male } ]'. +# +# Example (replace with your domain's containers/documents): +# +# employees: +# - { id: ^1, firstName: Bob, lastName: Smith, genderCode: M, salary: 50000, dateOfBirth: 1990-01-01 } +# - { id: ^2, firstName: Jane, lastName: Doe, genderCode: F, salary: 60000, dateOfBirth: 1985-06-15 } +# #else # Read test dataset for the 'domain-name' domain — loaded by the XxxReadTests classes via: # MigrateXxxDataAsync(["read-data.seed.yaml"], DbMigration.ConfigureMigrationArgs) # @@ -18,3 +38,4 @@ # - employee: # - { employee_id: ^1, first_name: Bob, last_name: Smith, gender_code: M, salary: 50000, date_of_birth: 1990-01-01 } # - { employee_id: ^2, first_name: Jane, last_name: Doe, gender_code: F, salary: 60000, date_of_birth: 1985-06-15 } +# #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/GlobalUsing.cs new file mode 100644 index 000000000..beba9bd5a --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/GlobalUsing.cs @@ -0,0 +1 @@ +global using Azure.Messaging.ServiceBus.Administration; diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/ServiceBus.cs b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/ServiceBus.cs new file mode 100644 index 000000000..8dff37cdb --- /dev/null +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/ServiceBus.cs @@ -0,0 +1,24 @@ +namespace app-name.Test.Common; + +/// +/// Defines the Azure Service Bus entities (queues, topics and subscriptions) for the 'domain-name' domain that the integration tests reset to a known state via Test.ResetAzureServiceBusAsync; this is the +/// code-based configuration and is therefore the place to add entities (the emulator servicebus/Config.json is intentionally empty). +/// +public static class ServiceBus +{ + /// + /// Gets the queues; none by default. Add a session-enabled queue named 'domain-parent-lower-{domain}' for each domain that is to receive commands (see NamedDestinationProvider). + /// + public static CreateQueueOptions[]? GetQueues() => []; + + /// + /// Gets the topics and their subscriptions; a single shared topic with a session-enabled subscription for this domain. + /// + public static (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? GetTopicsAndSubscriptions() => + [ + (new CreateTopicOptions("domain-parent-lower"), + [ + new CreateSubscriptionOptions("domain-parent-lower", "domain-name-lower") { RequiresSession = true } + ]) + ]; +} \ No newline at end of file diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/app-name.Test.Common.csproj b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/app-name.Test.Common.csproj index 10ab2a71f..1b751afae 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/app-name.Test.Common.csproj +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Common/app-name.Test.Common.csproj @@ -3,6 +3,11 @@ + + + + + diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/EntryPoint.cs b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/EntryPoint.cs index f9f3d778a..0b7f7219f 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/EntryPoint.cs +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/EntryPoint.cs @@ -10,25 +10,34 @@ public static void ConfigureApplication(IHostApplicationBuilder builder) builder.Services.AddExecutionContext(); builder.Services.AddMemoryCache(); // #if refdata-enabled - builder.Services.AddReferenceDataOrchestrator(); + builder.Services.AddReferenceDataOrchestrator(); // Reuse the "real" database configured reference data. +// #if implement-cosmos + var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.CamelCase)); +// #else var jdr = JsonDataReader.ParseYaml("ref-data.seed.yaml", JsonDataReaderOptions.CreateForReferenceData(JsonPropertyNamingConvention.SnakeCase)); - builder.Services.AddSingleton(new ReferenceDataServiceDecorator(jdr)); +// #endif + builder.Services.AddSingleton(new ReferenceDataProviderDecorator(jdr)); // #endif } // #if refdata-enabled - /// Provides a decorator for the to use JSON data for unit tests. - public class ReferenceDataServiceDecorator(JsonDataReader jdr) : ReferenceDataService(Mock.Of()) + /// Provides a decorator for the to use JSON data for unit tests. + public class ReferenceDataProviderDecorator(JsonDataReader jdr) : ReferenceDataProvider(Mock.Of()) { public override Task GetAsync(Type type, CancellationToken cancellationToken = default) => type switch { // Add a case per reference data type a validator under test needs. '{RefData}' is the contract type and +// #if implement-cosmos + // '{RefData}Collection' its collection; 'ref-data.$^Type' is the key into the seed data (container id, then the type discriminator group). + //_ when type == typeof(ref-data-name) => Task.FromResult((IReferenceDataCollection)jdr.Deserialize("ref-data.$^Type")!), +// #else // '{RefData}Collection' its collection; 'schema.$^table' is the appropriately-cased key into the seed data. //_ when type == typeof(ref-data-name) => Task.FromResult((IReferenceDataCollection)jdr.Deserialize("schema.$^table")!), +// #endif _ => throw new InvalidOperationException($"Type {type.FullName} is not a known {nameof(IReferenceData)}.") }; } // #endif -} \ No newline at end of file +} diff --git a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/app-name.Test.Unit.csproj b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/app-name.Test.Unit.csproj index fc1154d5e..7318a17bb 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/app-name.Test.Unit.csproj +++ b/src/CoreEx.Template/content/CoreEx.Core/tests/app-name.Test.Unit/app-name.Test.Unit.csproj @@ -13,7 +13,7 @@ - + diff --git a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.CodeGen/ref-data.yaml b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.CodeGen/ref-data.yaml index e78cbcfff..30df23eb9 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.CodeGen/ref-data.yaml +++ b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.CodeGen/ref-data.yaml @@ -5,5 +5,11 @@ applicationProjectPath: ../../src/app-name.Application contractsProjectPath: ../../src/app-name.Contracts dataProjectPath: ../../src/app-name.Infrastructure collectionSortOrder: Code +route: /ref +routeConvention: KebabCase +# #if implement-cosmos +repository: Cosmos +# #else repository: EntityFramework +# #endif entities: diff --git a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Data/ref-data.seed.yaml b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Data/ref-data.seed.yaml index 444baa819..78f9af5ce 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Data/ref-data.seed.yaml +++ b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Data/ref-data.seed.yaml @@ -2,4 +2,6 @@ domain-name: [] # #elif implement-postgres pg-schema: [] -# #endif \ No newline at end of file +# #elif implement-cosmos +ref-data: [] +# #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Program.cs b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Program.cs index 6b1f34739..7630eac8a 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Program.cs +++ b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/Program.cs @@ -1,5 +1,5 @@ -using CoreEx.Database; // #if implement-sqlserver +using CoreEx.Database; using DbEx.Migration; using DbEx.SqlServer.Console; @@ -23,6 +23,7 @@ public static MigrationArgs ConfigureMigrationArgs(MigrationArgs args) } } // #elif implement-postgres +using CoreEx.Database; using DbEx.Migration; using DbEx.Postgres.Console; @@ -45,4 +46,38 @@ public static MigrationArgs ConfigureMigrationArgs(MigrationArgs args) return args; } } -// #endif \ No newline at end of file +// #elif implement-cosmos +using CoreEx.Cosmos.Provisioning; + +namespace app-name.Database; + +/// Represents the database utilities program; provisions the Cosmos DB database and containers, and imports the seed data (the Cosmos DB equivalent of a relational DbEx migration). +/// Run using dotnet run -- All (or DropAndAll; ResetAndData requires the database to already exist); see --help for all commands. The container identifiers declared here must match those used by domain-nameCosmosDb. +public class Program +{ + /// The default (local emulator) connection string; override using -cs. + public const string DefaultConnectionString = "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="; + + /// The default database identifier; override using -d. + public const string DefaultDatabaseId = "domain-name-lower"; + + /// Main startup. + public static Task Main(string[] args) => CosmosDbConsole + .Create(DefaultConnectionString, DefaultDatabaseId) + .Configure(c => ConfigureProvisionArgs(c.Args)) + .RunAsync(args); + + /// Declares the containers (and embedded Data seed resources); shared by the console and the tests so both provision identically. + /// The . + /// The . + public static CosmosDbProvisionArgs ConfigureProvisionArgs(CosmosDbProvisionArgs args) => args + .AddAssembly() // Program = this project's embedded data. REQUIRED — the tests call ConfigureProvisionArgs directly (not via Main), so the Database assembly must be added here. Do not remove. +// #if refdata-enabled + .ReferenceDataContainer("ref-data") +// #endif +// #if outbox-enabled + .OutboxLeaseContainer() // The Change Feed Processor lease container shared by the Relay host(s); the relay never creates it (see AddCosmosDbOutboxRelayHostedService). +// #endif + ; // Add further containers here (using Container(...)), including any indexing policy required. +} +// #endif diff --git a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/app-name.Database.csproj b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/app-name.Database.csproj index 58e719ee0..35fbef28b 100644 --- a/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/app-name.Database.csproj +++ b/src/CoreEx.Template/content/CoreEx.Core/tools/app-name.Database/app-name.Database.csproj @@ -3,7 +3,11 @@ Exe + + + + @@ -12,8 +16,12 @@ + + + + diff --git a/src/CoreEx.Template/content/CoreEx.Relay/.template.config/template.json b/src/CoreEx.Template/content/CoreEx.Relay/.template.config/template.json index 32f3efa7f..da7eb8744 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/.template.config/template.json +++ b/src/CoreEx.Template/content/CoreEx.Relay/.template.config/template.json @@ -53,6 +53,13 @@ "valueTransform": "DomainParentLower", "replaces": "domain-parent-lower" }, + "refdata-enabled": { + "type": "parameter", + "datatype": "bool", + "displayName": "Reference data enabled", + "description": "Indicates whether reference data pattern/capabilities is enabled; determines whether the Cosmos DB 'ref-data' outbox container is relayed.", + "defaultValue": "true" + }, "data-provider": { "type": "parameter", "datatype": "choice", @@ -67,6 +74,11 @@ "displayName": "PostgreSQL", "description": "Indicates to use PostgreSQL as the data provider." }, + { + "choice": "Cosmos", + "displayName": "Azure Cosmos DB", + "description": "Indicates to use Azure Cosmos DB as the data provider." + }, { "choice": "None", "displayName": "None", @@ -85,6 +97,14 @@ "type": "computed", "value": "data-provider == 'Postgres'" }, + "implement-cosmos": { + "type": "computed", + "value": "data-provider == 'Cosmos'" + }, + "implement-relational": { + "type": "computed", + "value": "(data-provider == 'SqlServer' || data-provider == 'Postgres')" + }, "implement-none-data": { "type": "computed", "value": "data-provider == 'None'" @@ -121,6 +141,19 @@ "value": "messaging-provider == 'None'" } }, + "sources": [ + { + "modifiers": [ + { + "condition": "(implement-cosmos)", + "exclude": [ + "tests/solution-name.Test.Relay/RelayTests.cs", + "tests/solution-name.Test.Relay/Resources/**" + ] + } + ] + } + ], "specialCustomOperations": { "*.md": { "operations": [ diff --git a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/AGENTS.md index 479835f7c..e0827f399 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/AGENTS.md +++ b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/AGENTS.md @@ -36,6 +36,8 @@ The relay connects to the same database as the API host -- use the **same Aspire - `builder.AddSqlServerClient("SqlServer")` -- matches the Aspire SQL Server resource - `builder.AddAzureNpgsqlDataSource("Postgres")` -- matches the Aspire PostgreSQL resource + +- `builder.AddAzureCosmosClient("Cosmos")` -- matches the Aspire Cosmos DB connection string (`ConnectionStrings:Cosmos`); the database id is `domain-name-lower` - No database configured -- this relay has no data provider @@ -53,11 +55,14 @@ Relay timing and retry settings are configured via `appsettings.json` -- read `. - **Database:** SQL Server outbox (`AddSqlServerOutboxRelay()` + `AddSqlServerOutboxRelayHostedService()`) - **Database:** PostgreSQL outbox (`AddPostgresOutboxRelay()` + `AddPostgresOutboxRelayHostedService()`) + +- **Database:** Azure Cosmos DB outbox -- a Change Feed Processor per outbox-hosting container (`AddCosmosDbOutboxRelayHostedService("")`; push-based, not a polling loop). Outbox event documents are co-located in the same container/partition as the business data, so **call it once per container that hosts outbox documents** (e.g. add `builder.AddCosmosDbOutboxRelayHostedService("customer")` when a new entity container is added); concurrency is set per container via `CoreEx:Host:Services:CosmosOutboxRelay:{containerId}:ServicesCount`. The shared lease container (`$outbox-leases`) is **never** created by the relay (production identities cannot create containers) -- it is declared in the Database project via `.OutboxLeaseContainer()`, and the relay fails fast at startup if it is missing. Poison-message/dead-letter handling is not yet implemented for Cosmos. - **Database:** None -- this relay has no database outbox configured - **Message broker:** Azure Service Bus (`AddAzureServiceBusPublisher()`) + - Session routing is producer-side: use `CoreEx:Host:ServiceBus:Destinations:{exact-destination-name}` for per-destination overrides; a Relay must carry the same profile as any other host publishing to that destination. - **Message broker:** None configured @@ -74,6 +79,9 @@ Relay timing and retry settings are configured via `appsettings.json` -- read `. | `CoreEx.Database.Postgres` | PostgreSQL outbox relay implementation | + +| `CoreEx.Cosmos` | Cosmos DB Change Feed Processor outbox relay implementation | + | `CoreEx.Azure.Messaging.ServiceBus` | Azure Service Bus publisher | @@ -92,6 +100,9 @@ Relay timing and retry settings are configured via `appsettings.json` -- read `. - `.github/docs/coreex/agents/CoreEx.Database.Postgres.md` -- PostgreSQL outbox details + +- `.github/docs/coreex/agents/CoreEx.Cosmos.md` -- Cosmos DB outbox relay details + - `.github/docs/coreex/agents/CoreEx.Azure.Messaging.ServiceBus.md` -- Service Bus publisher diff --git a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/Program.cs b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/Program.cs index 2cb4c97c8..376fbbb7d 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/Program.cs +++ b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/Program.cs @@ -1,8 +1,14 @@ // #if implement-servicebus global using CoreEx.Azure.Messaging.ServiceBus; // #endif +// #if implement-cosmos +global using Microsoft.Azure.Cosmos; +// #endif global using OpenTelemetry; global using OpenTelemetry.Trace; +// #if implement-cosmos +global using System.Text.Json; +// #endif namespace app-name.Relay; @@ -37,6 +43,26 @@ private static void Main(string[] args) .AddSqlServerOutboxRelay(); // Adds the SqlServerOutboxRelay. builder.AddSqlServerOutboxRelayHostedService(); // Adds the SqlServerOutboxRelayHostedService. +// #elif implement-cosmos + // Add the Cosmos DB client (Aspire); in Development, accept the local emulator's self-signed certificate and use Gateway mode. + builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => + { + o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; + + if (builder.Environment.IsDevelopment()) + { + o.ConnectionMode = ConnectionMode.Gateway; + o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); + } + }); + + builder.Services + .AddCosmosDb("domain-name-lower") // Adds the CosmosDb for the database. + .AddCosmosDbHealthCheck(); // Adds the CosmosDbHealthCheck; Aspire's AddAzureCosmosClient does not register one. + +// #if refdata-enabled + builder.AddCosmosDbOutboxRelayHostedService("ref-data"); // Adds the CosmosDbOutboxRelayHostedService(s) (Change Feed Processor) for the 'ref-data' container; add one call per outbox-hosting container. +// #endif // #elif implement-postgres builder.AddAzureNpgsqlDataSource("Postgres"); // Adds the NpgsqlDataSource (using Aspire library). builder.Services @@ -63,6 +89,8 @@ private static void Main(string[] args) builder.WithCoreExTelemetry() // #if implement-sqlserver .WithCoreExSqlServerTelemetry() +// #elif implement-cosmos + .WithCoreExCosmosDbTelemetry() // #elif implement-postgres .WithCoreExPostgresTelemetry() // #endif diff --git a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/app-name.Relay.csproj b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/app-name.Relay.csproj index 83c5a7d06..5905ebfee 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/app-name.Relay.csproj +++ b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/app-name.Relay.csproj @@ -5,6 +5,10 @@ + + + + diff --git a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.Development.json b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.Development.json index 29ccd7fb1..88b01310f 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.Development.json +++ b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.Development.json @@ -9,6 +9,11 @@ "Microsoft": "Warning" } }, +//#if implement-cosmos + "ConnectionStrings": { + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==" + }, +//#endif "Aspire": { //#if implement-sqlserver "Microsoft": { diff --git a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.json b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.json index 90ff665c5..ffebf5a89 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.json +++ b/src/CoreEx.Template/content/CoreEx.Relay/src/app-name.Relay/appsettings.json @@ -5,14 +5,32 @@ "DomainName": "domain-name", "Services": { "Interval": "00:00:00.500", +//#if implement-cosmos + "CosmosOutboxRelay": { + "ref-data": { + "ServicesCount": 4 + } + } +//#else "OutboxRelay": { "BatchSize": 10, "PerWorkerPartitionCount": 2, - "LeaseDuration": "00:00:05", + "LeaseDuration": "00:00:30", "BackoffDuration": "00:00:05", "ServicesCount": 4 } +//#endif + } +// #if !implement-cosmos + ,"Outbox": { + "PartitionSize": 4 + } +// #endif +// #if implement-servicebus + ,"ServiceBus": { + "SessionIdPartitionSize": 4 } +// #endif } }, "Logging": { diff --git a/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/HostTests.cs b/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/HostTests.cs index c9b992229..6a4455971 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/HostTests.cs +++ b/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/HostTests.cs @@ -7,6 +7,8 @@ public async Task OneTimeSetUpAsync() { // #if implement-sqlserver await Test.MigrateSqlServerDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); +// #elif implement-cosmos + await Test.MigrateCosmosDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureProvisionArgs).ConfigureAwait(false); // #elif implement-postgres await Test.MigratePostgresDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // #endif @@ -35,6 +37,14 @@ public void Health_Detailed(string path, bool minimal) "$.entries.sqlserver-outbox-relay-01", "$.entries.sqlserver-outbox-relay-02", "$.entries.sqlserver-outbox-relay-03" +// #elif implement-cosmos + "$.entries.cosmos-database", +// #if refdata-enabled + "$.entries.cosmos-outbox-relay-ref-data-00", + "$.entries.cosmos-outbox-relay-ref-data-01", + "$.entries.cosmos-outbox-relay-ref-data-02", + "$.entries.cosmos-outbox-relay-ref-data-03", +// #endif // #elif implement-postgres "$.entries.postgreSql", "$.entries.postgres-outbox-relay-00", @@ -57,11 +67,14 @@ public void Health_Detailed(string path, bool minimal) json.ContainAll(_paths); } +// #if (!implement-cosmos || refdata-enabled) [Test] public void HostedService_Pause_And_Resume() { // #if implement-sqlserver const string Service = "sqlserver-outbox-relay-03"; +// #elif implement-cosmos + const string Service = "cosmos-outbox-relay-ref-data-03"; // #elif implement-postgres const string Service = "postgres-outbox-relay-03"; // #endif @@ -88,4 +101,5 @@ public void HostedService_Pause_And_Resume() .Run(HttpMethod.Get, $"/hosted-services/{Service}/status") .Value.Should().BeOneOf("Running", "Sleeping"); } +// #endif } \ No newline at end of file diff --git a/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/RelayTests.cs b/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/RelayTests.cs index 54ac1b69d..362a1c5e0 100644 --- a/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/RelayTests.cs +++ b/src/CoreEx.Template/content/CoreEx.Relay/tests/solution-name.Test.Relay/RelayTests.cs @@ -18,7 +18,7 @@ public async Task OneTimeSetUpAsync() await Test.MigratePostgresDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // #endif // #if implement-servicebus - await Test.GetAndClearAzureServiceBusAsync(ServiceBusSessionReceiverOptions.CreateForTopicSubscription("domain-parent-lower", "domain-name-lower")); + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); // #endif } diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/.template.config/template.json b/src/CoreEx.Template/content/CoreEx.Subscribe/.template.config/template.json index b9dc7dd67..7429e7c62 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/.template.config/template.json +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/.template.config/template.json @@ -60,6 +60,13 @@ "description": "Indicates whether reference data pattern/capabilities is enabled.", "defaultValue": "true" }, + "outbox-enabled": { + "type": "parameter", + "datatype": "bool", + "displayName": "Outbox enabled", + "description": "Indicates whether a transactional-outbox is enabled.", + "defaultValue": "true" + }, "data-provider": { "type": "parameter", "datatype": "choice", @@ -74,6 +81,11 @@ "displayName": "PostgreSQL", "description": "Indicates to use PostgreSQL as the data provider." }, + { + "choice": "Cosmos", + "displayName": "Azure Cosmos DB", + "description": "Indicates to use Azure Cosmos DB as the data provider." + }, { "choice": "None", "displayName": "None", @@ -92,6 +104,14 @@ "type": "computed", "value": "data-provider == 'Postgres'" }, + "implement-cosmos": { + "type": "computed", + "value": "data-provider == 'Cosmos'" + }, + "implement-relational": { + "type": "computed", + "value": "(data-provider == 'SqlServer' || data-provider == 'Postgres')" + }, "implement-none-data": { "type": "computed", "value": "data-provider == 'None'" diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/AGENTS.md b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/AGENTS.md index 40db9eb34..1b2af2f06 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/AGENTS.md +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/AGENTS.md @@ -59,6 +59,10 @@ The subscriber uses Azure Service Bus **session-enabled topics** for ordered, pa `MaxConcurrentSessions` (default: 4) controls parallelism -- each session processes messages sequentially, preventing ordering violations. +The receiver, subscriber and hosted service are registered with service keys (`receiver-events`, `subscriber-events`, `hosted-subscriber-events`) so further receivers -- e.g. a command queue named `domain-parent-lower-domain-name-lower` (as routed by the default `NamedDestinationProvider`: events to the shared topic, commands to `{topic}-{domain}` queues) -- can be added alongside with their own keys. Tests resolve the subscriber via `GetRequiredKeyedService("subscriber-events")`. The Service Bus topology used by the tests (and `Test.ResetAzureServiceBusAsync`) is defined in code in the `Test.Common` project `ServiceBus` class; the emulator `servicebus/Config.json` is intentionally empty. + +When this host publishes to multiple session-enabled destinations, set any destination-specific producer profile under `CoreEx:Host:ServiceBus:Destinations:{exact-destination-name}` in each publishing host's configuration. The bucket count groups producer keys into sessions; it is separate from this receiver's `MaxConcurrentSessions` and does not represent Service Bus physical partitions. + --- @@ -78,13 +82,15 @@ The subscriber wires FusionCache with both in-memory (L1) and Redis distributed - **Database:** SQL Server -- used for outbox publishing when subscribers need to emit their own events + +- **Database:** Azure Cosmos DB -- `builder.AddAzureCosmosClient("Cosmos")` plus `AddCosmosDb("domain-name-lower")`; used for outbox publishing when subscribers need to emit their own events - **Database:** PostgreSQL -- used for outbox publishing when subscribers need to emit their own events - **Database:** None -- no database configured; subscribers do not persist data directly -- **Reference data:** Enabled -- `ReferenceDataOrchestrator` is registered; reference data is available in subscriber logic +- **Reference data:** Enabled -- the non-generic `AddReferenceDataOrchestrator()` is registered and binds the CodeGen-generated `ReferenceDataProvider` (`IReferenceDataProvider`) from DI at runtime; reference data is available in subscriber logic - **Reference data:** Disabled @@ -106,7 +112,10 @@ The subscriber wires FusionCache with both in-memory (L1) and Redis distributed | `CoreEx.Database.Postgres` | PostgreSQL outbox for outbound events | - + +| `CoreEx.Cosmos` | Cosmos DB access and transactional outbox (`CosmosDbEventPublisher`) for outbound events | + + | `CoreEx.EntityFrameworkCore` | EF Core integration (`EfDb`, `IEfDbContext`) | diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/GlobalUsing.cs index e4673a795..79207f9e5 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/GlobalUsing.cs +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/GlobalUsing.cs @@ -4,7 +4,10 @@ global using CoreEx.Azure.Messaging.ServiceBus; // #endif global using CoreEx.Caching; -// #if has-data-provider +// #if implement-cosmos +global using CoreEx.Cosmos; +// #endif +// #if implement-relational global using CoreEx.Database; // #endif // #if implement-sqlserver @@ -24,6 +27,9 @@ // #endif global using CoreEx.Validation; global using Microsoft.AspNetCore.Mvc; +// #if implement-cosmos +global using Microsoft.Azure.Cosmos; +// #endif global using Microsoft.Extensions.Options; global using OpenTelemetry; global using OpenTelemetry.Trace; diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/Program.cs b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/Program.cs index 76b92529d..e9e6995da 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/Program.cs +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/Program.cs @@ -27,6 +27,7 @@ private static void Main(string[] args) builder.Services .AddPrecisionTimeProvider() .AddExecutionContext() + .AddNamedDestinationProvider() // Adds the NamedDestinationProvider (default); events go to the shared topic, commands to a per-domain queue. // #if refdata-enabled .AddReferenceDataOrchestrator() // Resolves the (CodeGen-generated) IReferenceDataProvider from DI at runtime — no compile-time dependency on the generated type. // #endif @@ -61,16 +62,41 @@ private static void Main(string[] args) .AddSqlServerDatabase() // Adds the SqlServerDatabase. .AddSqlServerUnitOfWork() // Adds the SqlServerUnitOfWork for the SqlServerDatabase. .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. +// #if outbox-enabled .AddSqlServerOutboxPublisher() // Adds the SqlServerOutboxPublisher as the IEventPublisher. +// #endif .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. +// #elif implement-cosmos + // Add the Cosmos DB client (Aspire); in Development, accept the local emulator's self-signed certificate and use Gateway mode. + builder.AddAzureCosmosClient("Cosmos", configureClientOptions: o => + { + o.UseSystemTextJsonSerializerWithOptions = new JsonSerializerOptions { PropertyNamingPolicy = JsonNamingPolicy.CamelCase }; + + if (builder.Environment.IsDevelopment()) + { + o.ConnectionMode = ConnectionMode.Gateway; + o.HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }); + } + }); + + builder.Services.AddCosmosDb("domain-name-lower"); // Adds the CoreEx extended Cosmos DB service for the database. + builder.Services + .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. +// #if outbox-enabled + .AddCosmosDbEventPublisher() // Adds the CosmosDbEventPublisher as the IEventPublisher. +// #endif + .AddCosmosDbUnitOfWork() // Adds the CosmosDbUnitOfWork as the IUnitOfWork. + .AddCosmosDbHealthCheck(); // Adds the CosmosDbHealthCheck; Aspire's AddAzureCosmosClient does not register one. // #elif implement-postgres builder.AddAzureNpgsqlDataSource("Postgres"); // Adds the NpgsqlDataSource (using Aspire library). builder.Services .AddPostgresDatabase() // Adds the PostgresDatabase. .AddPostgresUnitOfWork() // Adds the PostgresUnitOfWork for the PostgresDatabase. .AddEventFormatter() // Adds the EventFormatter to enable message formatting for publishing. +// #if outbox-enabled .AddPostgresOutboxPublisher() // Adds the PostgresOutboxPublisher as the IEventPublisher. +// #endif .AddDbContext() // Adds the standard EF DbContext. .AddEfDb(); // Adds the CoreEx extended EF service. // #endif @@ -81,19 +107,23 @@ private static void Main(string[] args) builder.Services.AddAzureServiceBusPublisher((_, c) => // Adds the service bus as the IEventPublisher. { c.SessionIdStrategy = ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId; // Use a partition-id as the session-id. +// #if (outbox-enabled && has-data-provider) }, addAsDefaultIEventPublisher: false); // Add as a named IEventPublisher, not the default (as the default is the OutboxPublisher). +// #else + }); +// #endif // Add event formatter and subscribed-manager. builder.Services .AddEventFormatter() // Adds the EventFormatter to enable message parsing/formatting. .AddSubscribedManager((_, c) => c.AddSubscribersUsing()); // Add all subscribers from this assembly. - // Build and create the Azure Service Bus receiving services. + // Build and create the Azure Service Bus receiving services; the receiver, subscriber and hosted service are keyed so that additional receivers (e.g. a command queue) can be added alongside. builder.Services.AzureServiceBusReceiving() - .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForTopicSubscription()) // Set the topic and subscription, etc. from configuration. - .WithSubscribedSubscriber() // Adds the service bus subscriber using the SubscribedManager. - .WithHostedService() // Adds the service bus receiver as a hosted service. - .Build(); // Builds all the services and adds to the service collection. + .WithSessionReceiver(_ => ServiceBusSessionReceiverOptions.CreateForTopicSubscription(), "receiver-events") // Set the topic and subscription, etc. from configuration. + .WithKeyedSubscribedSubscriber("subscriber-events") // Adds the service bus subscriber using the SubscribedManager. + .WithHostedService("hosted-subscriber-events") // Adds the service bus receiver as a hosted service (this key is also the health-check and hosted-service management name). + .Build(); // Builds all the services and adds to the service collection. // #endif // Post-configure all health-checks; adds the standard tags. @@ -113,6 +143,8 @@ private static void Main(string[] args) builder.WithCoreExTelemetry() // #if implement-sqlserver .WithCoreExSqlServerTelemetry() +// #elif implement-cosmos + .WithCoreExCosmosDbTelemetry() // #elif implement-postgres .WithCoreExPostgresTelemetry() // #endif diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/app-name.Subscribe.csproj b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/app-name.Subscribe.csproj index 391479a6c..543c15d6b 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/app-name.Subscribe.csproj +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/app-name.Subscribe.csproj @@ -16,6 +16,9 @@ + + + diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.Development.json b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.Development.json index 5a67ef29a..5660782fa 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.Development.json +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.Development.json @@ -15,6 +15,11 @@ "StackExchange": "Warning" } }, +//#if implement-cosmos + "ConnectionStrings": { + "Cosmos": "AccountEndpoint=https://localhost:8081/;AccountKey=C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw==" + }, +//#endif "Aspire": { //#if implement-sqlserver "Microsoft": { diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.json b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.json index f98ff39dc..c046d3566 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.json +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/src/app-name.Subscribe/appsettings.json @@ -6,6 +6,16 @@ "Services": { "Interval": "00:00:00.500" } +// #if (implement-sqlserver || implement-postgres) && outbox-enabled + ,"Outbox": { + "PartitionSize": 4 + } +// #endif +// #if implement-servicebus + ,"ServiceBus": { + "SessionIdPartitionSize": 4 + } +// #endif }, "Events": { "Destination": "domain-parent-lower" // Topic/queue name. @@ -20,4 +30,4 @@ } }, "AllowedHosts": "*" -} \ No newline at end of file +} diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/GlobalUsing.cs b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/GlobalUsing.cs index b31c09cc4..46a6cf2e1 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/GlobalUsing.cs +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/GlobalUsing.cs @@ -11,8 +11,7 @@ global using UnitTestEx; global using UnitTestEx.NUnit; global using UnitTestEx.Expectations; -global using solution-name.Contracts; -// #if (implement-sqlserver || implement-postgres) +// #if has-data-provider global using DbMigration = solution-name.Database.Program; global using TestData = solution-name.Test.Common.TestData; -// #endif \ No newline at end of file +// #endif diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/HostTests.cs b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/HostTests.cs index 2e717548b..f0f880e95 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/HostTests.cs +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/HostTests.cs @@ -7,10 +7,15 @@ public async Task OneTimeSetUpAsync() { // #if implement-sqlserver await Test.MigrateSqlServerDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); +// #elif implement-cosmos + await Test.MigrateCosmosDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureProvisionArgs).ConfigureAwait(false); // #elif implement-postgres await Test.MigratePostgresDataAsync(["no-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // #endif await Test.ClearFusionCacheAsync().ConfigureAwait(false); +// #if implement-servicebus + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); +// #endif } [TestCase("/health/live")] @@ -36,11 +41,13 @@ public void Health_Detailed(string path, bool minimal) "$.entries['stackExchange.Redis']", // #if implement-sqlserver "$.entries.sqlServer", +// #elif implement-cosmos + "$.entries.cosmos-database", // #elif implement-postgres "$.entries.postgreSql", // #endif // #if implement-servicebus - "$.entries.azure-service-bus-session-receiver" + "$.entries.hosted-subscriber-events" // #endif ]; @@ -61,7 +68,7 @@ public void Health_Detailed(string path, bool minimal) public void HostedService_Pause_And_Resume() { // #if implement-servicebus - const string Service = "azure-service-bus-session-receiver"; + const string Service = "hosted-subscriber-events"; // #endif Test.Http() diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/SubscriberTests.cs b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/SubscriberTests.cs index 902f04e57..a6345211b 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/SubscriberTests.cs +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/SubscriberTests.cs @@ -7,14 +7,21 @@ public async Task OneTimeSetUpAsync() { // #if implement-sqlserver await Test.MigrateSqlServerDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); + // #elif implement-cosmos + await Test.MigrateCosmosDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureProvisionArgs).ConfigureAwait(false); // #elif implement-postgres await Test.MigratePostgresDataAsync(["mutate-data.seed.yaml"], DbMigration.ConfigureMigrationArgs).ConfigureAwait(false); // #endif await Test.ClearFusionCacheAsync().ConfigureAwait(false); + // #if implement-servicebus + await Test.ResetAzureServiceBusAsync(Common.ServiceBus.GetQueues(), Common.ServiceBus.GetTopicsAndSubscriptions()).ConfigureAwait(false); + // #endif - // #if implement-sqlserver + // #if (outbox-enabled && implement-sqlserver) Test.UseExpectedSqlServerOutboxPublisher(); - // #elif implement-postgres + // #elif (outbox-enabled && implement-cosmos) + Test.UseExpectedCosmosDbOutboxPublisher(); + // #elif (outbox-enabled && implement-postgres) Test.UseExpectedPostgresOutboxPublisher(); // #endif } @@ -28,7 +35,7 @@ public void NotSubscribed_CompleteAsSilent() => Test.Scoped(test => test.Run(async _ => { - var sbs = test.Services.GetRequiredService(); + var sbs = test.Services.GetRequiredKeyedService("subscriber-events"); var r = await sbs.ReceiveAsync(sbm); r.IsFailure.Should().BeTrue(); diff --git a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/solution-name.Test.Subscribe.csproj b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/solution-name.Test.Subscribe.csproj index 11e21bfe1..6c6e66ea5 100644 --- a/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/solution-name.Test.Subscribe.csproj +++ b/src/CoreEx.Template/content/CoreEx.Subscribe/tests/solution-name.Test.Subscribe/solution-name.Test.Subscribe.csproj @@ -12,7 +12,7 @@ - + diff --git a/src/CoreEx.UnitTesting/AGENTS.md b/src/CoreEx.UnitTesting/AGENTS.md index 2298f08f6..d93eccf40 100644 --- a/src/CoreEx.UnitTesting/AGENTS.md +++ b/src/CoreEx.UnitTesting/AGENTS.md @@ -10,6 +10,17 @@ The single test-support package for the entire CoreEx ecosystem. One ` + diff --git a/src/CoreEx.UnitTesting/Events/EventExpectations.cs b/src/CoreEx.UnitTesting/Events/EventExpectations.cs index 90976cf16..ce130dae4 100644 --- a/src/CoreEx.UnitTesting/Events/EventExpectations.cs +++ b/src/CoreEx.UnitTesting/Events/EventExpectations.cs @@ -11,12 +11,15 @@ public class EventExpectations : ExpectationsBase /// /// Initializes a new instance of the class. /// - /// The owning . + /// The owning . /// The initiating tester. /// The request identifier. /// The assembly to use for resource resolution. - internal EventExpectations(TesterBase owner, TTester tester, string? requestId, Assembly assembly) : base(owner, tester) + internal EventExpectations(TesterBaseCore owner, TTester tester, string? requestId, Assembly assembly) : base(owner, tester) { + if (owner is not TesterBase) + throw new ArgumentException($"The {nameof(owner)} must be of type {nameof(TesterBase)} for event {nameof(EventExpectations<>)}.", nameof(owner)); + RequestId = requestId; ResourceAssembly = assembly; } @@ -68,7 +71,7 @@ private EventExpectationsConfig GetOrAddConfig(string serviceKey) { if (!_expectations.TryGetValue(serviceKey.ThrowIfNullOrEmpty(), out var config)) { - config = new EventExpectationsConfig(Owner, serviceKey, RequestId, ResourceAssembly); + config = new EventExpectationsConfig((TesterBase)Owner, serviceKey, RequestId, ResourceAssembly); _expectations[serviceKey] = config; } @@ -85,4 +88,4 @@ protected override Task OnAssertAsync(AssertArgs args) return Task.CompletedTask; } -} \ No newline at end of file +} diff --git a/src/CoreEx.UnitTesting/GlobalUsing.cs b/src/CoreEx.UnitTesting/GlobalUsing.cs index cad63d634..bb507f6cc 100644 --- a/src/CoreEx.UnitTesting/GlobalUsing.cs +++ b/src/CoreEx.UnitTesting/GlobalUsing.cs @@ -1,11 +1,17 @@ +global using Aha = Aspire.Hosting.ApplicationModel; /* "take on me..." https://www.youtube.com/watch?v=djV11Xbc914&list=RDdjV11Xbc914 */ +global using Asb = Azure.Messaging.ServiceBus; +global using Aspire.Hosting; +global using Aspire.Hosting.Testing; global using AwesomeAssertions; global using AwesomeAssertions.Execution; global using AwesomeAssertions.Primitives; +global using Azure.Messaging.ServiceBus.Administration; global using CloudNative.CloudEvents; global using CoreEx; global using CoreEx.Azure.Messaging.ServiceBus; global using CoreEx.Cosmos; global using CoreEx.Cosmos.Outbox; +global using CoreEx.Cosmos.Provisioning; global using CoreEx.Data; global using CoreEx.Data.Json; global using CoreEx.Database.Postgres.Outbox; @@ -28,10 +34,14 @@ global using DbEx.Migration.Data; global using DbEx.Postgres.Migration; global using DbEx.SqlServer.Migration; +global using ExecutionContext = CoreEx.ExecutionContext; global using Microsoft.Azure.Cosmos; +global using Microsoft.Extensions.Configuration; global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Logging; +global using StackExchange.Redis; global using System.Collections.Concurrent; +global using System.Data.Common; global using System.Diagnostics.CodeAnalysis; global using System.Reflection; global using System.Text; @@ -41,8 +51,7 @@ global using UnitTestEx; global using UnitTestEx.Abstractions; global using UnitTestEx.AspNetCore; +global using UnitTestEx.Aspire; global using UnitTestEx.Assertors; global using UnitTestEx.Expectations; global using UnitTestEx.Hosting; -global using Asb = Azure.Messaging.ServiceBus; -global using ExecutionContext = CoreEx.ExecutionContext; diff --git a/src/CoreEx.UnitTesting/README.md b/src/CoreEx.UnitTesting/README.md index e5a214b05..f2a05ef08 100644 --- a/src/CoreEx.UnitTesting/README.md +++ b/src/CoreEx.UnitTesting/README.md @@ -29,13 +29,31 @@ The package extends [UnitTestEx](https://github.com/Avanade/UnitTestEx) — the - 🔄 **FusionCache test reset**: `ClearFusionCacheAsync` clears the registered `IFusionCache` between test runs to prevent state bleed in cached reference-data or other cache-backed scenarios. ## Key types - | Type | Description | |------|-------------| | **[`UnitTestExOneOffTestSetUp`](./UnitTestExOneOffTestSetUp.cs)** | Internal one-off UnitTestEx initializer; configures CoreEx JSON serialization defaults, the environment user name, and `ValidationException` error-assertion integration automatically on assembly load. | | **[`UnitTestExExpectations`](./UnitTestExExpectations.cs)** | Static partial class; aggregates all CoreEx-specific value and tester expectation extension methods: `ExpectIdentifier`, `ExpectETag`, `ExpectChangeLogCreated`, `ExpectChangeLogUpdated`, `IgnorePaths`, and the full event and outbox expectation surface. | | **[`UnitTestExExtensions`](./UnitTestExExtensions.cs)** | Static partial class; aggregates all CoreEx-specific `TesterBase` and `IValidator` extension methods: `Scoped`, `CreateCloudEventFrom`, `AssertProblemDetails`, `ClearFusionCacheAsync`, `UseExpectedEventPublisher`, `AssertSuccess`, and `AssertErrors`. | + +### Aspire external infrastructure + +`AddExternalConnectionString` (in the `UnitTestEx` namespace, alongside the other Aspire extensions) wraps a secret +parameter sourced from `ConnectionStrings:{name}` in a visible connection-string resource: + +```csharp +var database = builder.AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint").WithIconName("DatabaseMultiple"); +var redis = builder.AddExternalConnectionString("redis").WithIconName("Database"); +builder.AddProject("customers-api").WithReference(database).WithReference(redis); +``` + +Use `"Postgres"` or `"SqlServer"` without `endpointKey` for relational providers. Plain `AddConnectionString(name)` +creates parameter-only resources omitted from Aspire 13.5.4's Graph/Table views. Compose retains infrastructure +lifecycle ownership; the resource's running status indicates resolved configuration, not connectivity. Hosts still +perform readiness checks. Cosmos's configured endpoint is exposed as a credential-free HTTP/HTTPS URL for dashboard +peer matching; missing, invalid, non-HTTP, or credential-bearing endpoint URLs fail explicitly. Dashboard labels +depend on span endpoint attributes, not database-provider names; explicit ports help avoid ambiguous local matches. + ## Namespaces | Namespace | Description | Documentation | @@ -61,4 +79,4 @@ The package extends [UnitTestEx](https://github.com/Avanade/UnitTestEx) — the ## AI Usage Guide -An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. \ No newline at end of file +An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. diff --git a/src/CoreEx.UnitTesting/UnitTestExExpectations.ChangeLog.cs b/src/CoreEx.UnitTesting/UnitTestExExpectations.ChangeLog.cs index bfac6edb1..ba110ef7e 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExpectations.ChangeLog.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExpectations.ChangeLog.cs @@ -10,7 +10,7 @@ public static partial class UnitTestExExpectations /// The expectations . /// The value . /// The tester. - /// The specific value where specified (can include wildcards); otherwise, indicates to check for user running the test (see ). + /// The specific value where specified (can include wildcards); otherwise, indicates to check for user running the test (see ). /// The in which the should be greater than or equal to; where null it will default to . /// The instance to support fluent-style method-chaining. public static TSelf ExpectChangeLogCreated(this IValueExpectations tester, string? createdBy = null, DateTimeOffset? createdOn = null) where TSelf : IValueExpectations @@ -65,7 +65,7 @@ Task extension(AssertArgs args) /// The expectations . /// The value . /// The tester. - /// The specific value where specified (can include wildcards); otherwise, indicates to check for user running the test (see ). + /// The specific value where specified (can include wildcards); otherwise, indicates to check for user running the test (see ). /// The in which the should be greater than or equal to; where null it will default to . /// The instance to support fluent-style method-chaining. public static TSelf ExpectChangeLogUpdated(this IValueExpectations tester, string? updatedBy = null, DateTimeOffset? updatedOn = null) where TSelf : IValueExpectations @@ -123,4 +123,4 @@ Task extension(AssertArgs args) /// The instance to support fluent-style method-chaining. public static TSelf IgnoreChangeLog(this IValueExpectations tester) where TSelf : IValueExpectations => IgnorePaths(tester, nameof(IChangeLog.ChangeLog), nameof(IChangeLogEx.CreatedBy), nameof(IChangeLogEx.CreatedOn), nameof(IChangeLogEx.UpdatedBy), nameof(IChangeLogEx.UpdatedOn)); -} \ No newline at end of file +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.Aspire.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.Aspire.cs new file mode 100644 index 000000000..87050ccf4 --- /dev/null +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.Aspire.cs @@ -0,0 +1,100 @@ +#pragma warning disable IDE0130 // Namespace does not match folder structure; by design. +namespace UnitTestEx; +#pragma warning restore IDE0130 // Namespace does not match folder structure + +public static partial class UnitTestExExtensions +{ + /// + /// Models externally managed infrastructure as a visible connection-string resource backed by secret configuration. + /// + /// The . + /// The existing connection-string name. + /// An optional connection-string key containing an absolute HTTP or HTTPS endpoint URL for dashboard peer matching. + /// The visible resource, without taking ownership of the infrastructure lifecycle. + /// Reads ConnectionStrings:{name} through a secret parameter. Unlike parameter-only AddConnectionString(name), + /// this resource appears in Aspire's Graph/Table views. Its running status indicates resolved configuration, not service health; + /// hosts remain responsible for readiness probes. For Cosmos DB, specify endpointKey: "AccountEndpoint" because Aspire's + /// connection-string parser does not recognise that key. Credentials are not permitted in the endpoint URL. + public static Aha.IResourceBuilder AddExternalConnectionString(this IDistributedApplicationBuilder builder, string name, string? endpointKey = null) + { + builder.ThrowIfNull(); + name.ThrowIfNullOrEmpty(); + var parameter = builder.AddParameterFromConfiguration($"{name}-connection", $"ConnectionStrings:{name}", secret: true); + var resource = builder.AddConnectionString(name, Aha.ReferenceExpression.Create($"{parameter}")); + + if (endpointKey is not null) + { + endpointKey.ThrowIfNullOrEmpty(); + var connectionString = builder.Configuration.GetConnectionString(name) ?? throw new InvalidOperationException($"Connection string '{name}' is required."); + var properties = new DbConnectionStringBuilder { ConnectionString = connectionString }; + + if (!properties.TryGetValue(endpointKey, out var endpoint) || endpoint is not string url || !Uri.TryCreate(url, UriKind.Absolute, out var uri) + || (uri.Scheme != Uri.UriSchemeHttp && uri.Scheme != Uri.UriSchemeHttps) || !string.IsNullOrEmpty(uri.UserInfo)) + throw new InvalidOperationException($"Connection string '{name}' must contain an absolute HTTP or HTTPS '{endpointKey}' URL without credentials."); + + resource.WithUrl(uri.GetLeftPart(UriPartial.Authority)); + } + + return resource; + } + + /// + /// Annotates the resource's http endpoint with one or more relative so they appear as clickable links in the Aspire dashboard. + /// + /// The of . + /// The relative URLs to add, e.g. "/health/ready/detailed". + /// The for fluent-style method-chaining. + public static Aha.IResourceBuilder AddEndpoints(this Aha.IResourceBuilder builder, params string[] urls) + { + var httpEndpoint = builder.GetEndpoint("http"); + foreach (var url in urls) + { + builder.WithAnnotation(new Aha.ResourceUrlAnnotation { Endpoint = httpEndpoint, Url = url }); + } + + return builder; + } + + /// + /// Adds a dashboard command button that invokes an HTTP against a relative on the resource. + /// + /// The of . + /// The to invoke. + /// The relative path to invoke, e.g. "/hosted-services/all/pause". + /// The button's display name shown in the dashboard. + /// The optional Fluent UI icon name for the button; see the icon catalog. + /// The for fluent-style method-chaining. + public static Aha.IResourceBuilder AddCommand(this Aha.IResourceBuilder builder, HttpMethod method, string path, string displayName, string? iconName) + => builder.WithHttpCommand( + path: path, + displayName: displayName, + commandOptions: new Aha.HttpCommandOptions() { Method = method, IconName = iconName }); + + /// + /// Adds dashboard support for the standard CoreEx hosted-service management endpoints (/hosted-services/all/{status,pause,resume} via MapHostedServices()) -- a status link plus "Pause all services"/"Resume all services" command buttons. + /// + /// The of . + /// The for fluent-style method-chaining. + /// The stock Api host template doesn't register AddHostedServiceManager()/MapHostedServices() -- Relay and Subscribe do. This split is a workload-isolation convention, not a technical requirement; for a small, low-traffic solution, consolidating hosted-service processing into the Api host is a reasonable simplification. Call this method wherever those endpoints are actually mapped. + public static Aha.IResourceBuilder AddHostedServiceSupport(this Aha.IResourceBuilder builder) + => builder.AddEndpoints("/hosted-services/all/status") + .AddCommand(HttpMethod.Post, "/hosted-services/all/pause", "Pause all services", "Pause") + .AddCommand(HttpMethod.Post, "/hosted-services/all/resume", "Resume all services", "PauseOff"); + + /// + /// Disables TLS/SSL certificate validation for all outbound instances created via ConfigureHttpClientDefaults. + /// + /// The . + /// The for fluent-style method-chaining. + /// Intended for local development and testing only, e.g. trusting self-signed certificates presented by emulators (Service Bus, Cosmos DB, etc.). It's also useful because dotnet dev-certs https --trust isn't fully supported on Linux, so an ASP.NET Core project resource's HTTPS endpoint may present a dev certificate that isn't OS-trusted (e.g. on Linux CI runners); anything resolving an via this same DI container's IHttpClientFactory -- health check probes registered against a project resource's endpoint, or test-harness clients such as AspireTesterBase.CreateHttpClient() -- is covered by this call. This bypasses all certificate validation for every in the app, so it must never be used in production. + public static IDistributedApplicationBuilder DisableHttpCertificateValidation(this IDistributedApplicationBuilder builder) + { + builder.Services.ConfigureHttpClientDefaults(http => + http.ConfigurePrimaryHttpMessageHandler(() => new HttpClientHandler + { + ServerCertificateCustomValidationCallback = (_, _, _, _) => true + })); + + return builder; + } +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.Caching.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.Caching.cs index eb3616aac..f80f7f475 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExtensions.Caching.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.Caching.cs @@ -10,4 +10,23 @@ public static partial class UnitTestExExtensions /// The . /// The service must be registered within the underlying test host. public static async Task ClearFusionCacheAsync(this TesterBase tester) => await tester.ThrowIfNull().Services.GetRequiredService().ClearAsync(false); -} \ No newline at end of file + + /// + /// Clears the underlying Redis cache by executing the operation. + /// + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// A task that represents the asynchronous operation. + public static async Task ClearRedisCacheAsync(this DistributedApplication app, string aspireResourceName) + { + var cs = (await AspireTesterBase.GetConnectionStringAsync(app.ThrowIfNull(), aspireResourceName.ThrowIfNullOrEmpty()).ConfigureAwait(false)) ?? throw new InvalidOperationException($"The '{aspireResourceName}' connection string not found."); + + var options = ConfigurationOptions.Parse(cs); + options.AllowAdmin = true; + + await using var mux = await ConnectionMultiplexer.ConnectAsync(options); + + foreach (var ep in mux.GetEndPoints()) + await mux.GetServer(ep).FlushDatabaseAsync(); + } +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.Cosmos.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.Cosmos.cs index a411875dd..95a8c3490 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExtensions.Cosmos.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.Cosmos.cs @@ -4,6 +4,9 @@ namespace UnitTestEx; public static partial class UnitTestExExtensions { + private static readonly ConcurrentDictionary _cosmosDatabases = new(); + private static readonly ConcurrentDictionary _cosmosDatabaseLocks = new(); + /// /// Replaces the registered with a decorator () that also captures the published events for expectation assertions. /// @@ -71,4 +74,78 @@ public static async Task GetCosmosDatabaseAsync(this TesterBase tester /// private static bool IsTransientCosmosConnectionFailure(Exception exception) => exception is System.Net.Http.HttpRequestException or IOException or System.Net.Sockets.SocketException || exception.InnerException is not null && IsTransientCosmosConnectionFailure(exception.InnerException); + + /// + /// Provisions (resets) the host's Cosmos DB database containers and imports the seed data using the ; the Cosmos DB equivalent of MigratePostgresDataAsync. + /// + /// The to infer the containing the named ; only these named resources are loaded from it (add the assembly holding the default Data folder resources using within ). + /// The . + /// The named resource files to include in the data loading; imported after the default Data folder resources. + /// An optional function to configure the (e.g. to declare the containers). + /// The . + /// The host's own and database identifier are used (see ). As per MigratePostgresDataAsync, the first call per account endpoint and database + /// is | (the database is only created where explicitly requested; a reset on its own fails where it does not exist), with all subsequent calls being . + public static async Task MigrateCosmosDataAsync(this TesterBase tester, string[]? resourceFileNames = null, Func? configure = null, CancellationToken cancellationToken = default) + { + // The ICosmosDb is scoped so a short-lived scope is used purely to resolve it (see GetCosmosDatabaseAsync); the underlying CosmosClient is independently registered and remains usable. + using var scope = tester.ThrowIfNull().Services.CreateScope(); + var cosmosDb = scope.ServiceProvider.GetRequiredService(); + var client = cosmosDb.Client; + var databaseId = cosmosDb.Database.Id; + + var key = $"{client.Endpoint}|{databaseId}"; + var semaphore = _cosmosDatabaseLocks.GetOrAdd(key, static _ => new SemaphoreSlim(1, 1)); + await semaphore.WaitAsync(cancellationToken).ConfigureAwait(false); + try + { + var first = !_cosmosDatabases.ContainsKey(key); + await ProvisionCosmosAsync(client, databaseId, resourceFileNames, configure, first ? CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData : CosmosDbProvisionCommand.ResetAndData, cancellationToken).ConfigureAwait(false); + _cosmosDatabases.TryAdd(key, 0); + } + finally + { + semaphore.Release(); + } + } + + /// + /// Provisions (creates and resets) the Cosmos DB database containers and imports the seed data using the , using the specified to retrieve the connection string from the distributed application; always | . + /// + /// The to infer the containing the named ; only these named resources are loaded from it (add the assembly holding the default Data folder resources using within ). + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// The . + /// The named resource files to include in the data loading; imported after the default Data folder resources. + /// An optional function to configure the (e.g. to declare the containers). + /// The . + public static async Task MigrateCosmosDataAsync(this DistributedApplication app, string aspireResourceName, string databaseId, string[]? resourceFileNames = null, Func? configure = null, CancellationToken cancellationToken = default) + { + var cs = (await AspireTesterBase.GetConnectionStringAsync(app.ThrowIfNull(), aspireResourceName.ThrowIfNullOrEmpty(), cancellationToken).ConfigureAwait(false)) ?? throw new InvalidOperationException($"The '{aspireResourceName}' connection string not found."); + using var client = CosmosDbClientFactory.Create(cs); + await ProvisionCosmosAsync(client, databaseId, resourceFileNames, configure, CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData, cancellationToken).ConfigureAwait(false); + } + + /// + /// Runs the and throws including the captured output on failure. + /// + private static async Task ProvisionCosmosAsync(CosmosClient client, string databaseId, string[]? resourceFileNames, Func? configure, CosmosDbProvisionCommand command, CancellationToken cancellationToken) + { + var output = new StringWriter(); + var args = new CosmosDbProvisionArgs { DatabaseId = databaseId, Output = output }; + + if (configure is not null) + args = configure(args); + + if (resourceFileNames is { Length: > 0 }) + args.AddDataResource(resourceFileNames); + + try + { + await new CosmosDbProvisioner(client, args).RunAsync(command, cancellationToken).ConfigureAwait(false); + } + catch (Exception ex) + { + throw new InvalidOperationException("CosmosDbProvisioner failed:" + Environment.NewLine + output + ex.Message, ex); + } + } } diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.Postgres.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.Postgres.cs index 89daab466..9d51bacce 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExtensions.Postgres.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.Postgres.cs @@ -101,4 +101,66 @@ public static async Task MigratePostgresDataAsync(this TesterBase tester, Func + /// Execute the using the specified to retrieve the connection string from the distributed application. + /// + /// The to infer the underlying . + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// The resource file names to include in the data loading; see . + /// An optional function to configure the . + /// A task that represents the asynchronous operation. + public static Task MigratePostgresDataAsync(this DistributedApplication app, string aspireResourceName, string[]? resourceFileNames = null, Func? configureMigrationArgs = null) + { + resourceFileNames ??= []; + + if (configureMigrationArgs is null) + { + return MigratePostgresDataAsync(app, aspireResourceName, ma => + { + ma.DataParserArgs.AddNamed(resourceFileNames); + return ma; + }, typeof(TAssembly).Assembly); + } + else + { + return MigratePostgresDataAsync(app, aspireResourceName, ma => + { + var cma = configureMigrationArgs(ma); + cma.DataParserArgs.AddNamed(resourceFileNames); + return cma; + }, typeof(TAssembly).Assembly); + } + } + + /// + /// Execute the using the specified to retrieve the connection string from the distributed application. + /// + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// An optional function to configure the . + /// An optional array of assemblies to include in the migration. + public static async Task MigratePostgresDataAsync(this DistributedApplication app, string aspireResourceName, Func? configureMigrationArgs = null, params Assembly[] assemblies) + { + var cs = (await AspireTesterBase.GetConnectionStringAsync(app.ThrowIfNull(), aspireResourceName.ThrowIfNullOrEmpty()).ConfigureAwait(false)) ?? throw new InvalidOperationException($"The '{aspireResourceName}' connection string not found."); + var ma = new MigrationArgs(MigrationCommand.All | MigrationCommand.ResetAndData, cs); + + if (configureMigrationArgs is not null) + ma = configureMigrationArgs(ma); + + if (assemblies.Length > 0) + ma.AddAssembly(assemblies); + + // Execute the postgres migration. + using var m = new PostgresMigration(ma); + var (Success, Output) = await m.MigrateAndLogAsync().ConfigureAwait(false); + + if (!Success) + throw new InvalidOperationException("PostgresMigration failed:" + Environment.NewLine + Output); + } + + #endregion +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.ServiceBus.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.ServiceBus.cs index bec185255..5be86f791 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExtensions.ServiceBus.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.ServiceBus.cs @@ -4,6 +4,27 @@ namespace UnitTestEx; public static partial class UnitTestExExtensions { + /// + /// Resets the Azure Service Bus queues and topics/subscriptions to an initial state by deleting and recreating them. + /// + /// The . + /// The queues to reset. + /// The topics and their subscriptions to reset. + /// The named connection string/configuration section as used by AddAzureServiceBusClient/AddKeyedAzureServiceBusClient to configure the underlying ; defaults to "ServiceBus". + /// The port to use for the administration client. + /// The registered by Aspire does not retain the connection string it was created from, so this reads it directly from configuration instead - first + /// the standard Aspire-orchestrated ConnectionStrings:{connectionName} key, then falling back to the Aspire:Azure:Messaging:ServiceBus:ConnectionString section used when the + /// connection string is configured directly (e.g. via appsettings.Development.json). + public static async Task ResetAzureServiceBusAsync(this TesterBase tester, CreateQueueOptions[]? queues = null, (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? topicsAndSubscriptions = null, string connectionName = "ServiceBus", int adminPort = 5300) + { + var config = tester.ThrowIfNull().Configuration; + var cs = config.GetConnectionString(connectionName.ThrowIfNullOrEmpty()) + ?? config[$"Aspire:Azure:Messaging:ServiceBus:ConnectionString"] + ?? throw new InvalidOperationException($"The '{connectionName}' Azure Service Bus connection string was not found in configuration."); + + await ResetAzureServiceBusAsync(CreateAzureServiceBusAdminConnectionString(cs, adminPort), queues, topicsAndSubscriptions).ConfigureAwait(false); + } + /// /// Gets all messages for the Azure Service Bus queue or topic subscription completing each resulting in all messages also being cleared. /// @@ -161,4 +182,84 @@ public static Asb.ServiceBusReceivedMessage ToServiceBusReceivedMessage(this Asb enqueuedTime: DateTimeOffset.UtcNow ); } -} \ No newline at end of file + + #region Aspire + + /// + /// Resets the Azure Service Bus queues and topics/subscriptions to an initial state by deleting and recreating them. + /// + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// The queues to reset. + /// The topics and their subscriptions to reset. + /// The port to use for the administration client. + public static async Task ResetAzureServiceBusAsync(this DistributedApplication app, string aspireResourceName, CreateQueueOptions[]? queues = null, (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? topicsAndSubscriptions = null, int adminPort = 5300) + { + var cs = (await AspireTesterBase.GetConnectionStringAsync(app.ThrowIfNull(), aspireResourceName.ThrowIfNullOrEmpty()).ConfigureAwait(false)) ?? throw new InvalidOperationException($"The '{aspireResourceName}' connection string not found."); + await ResetAzureServiceBusAsync(CreateAzureServiceBusAdminConnectionString(cs, adminPort), queues, topicsAndSubscriptions).ConfigureAwait(false); + } + + #endregion + + #region Admin + + /// + /// Creates an Azure Service Bus connection string for the administration client by replacing the port in the Endpoint with the specified . + /// + /// The original Azure Service Bus connection string. + /// The port to use for the administration client. + /// The modified Azure Service Bus connection string for the administration client. + public static string CreateAzureServiceBusAdminConnectionString(string connectionString, int adminPort = 5300) + { + var parts = connectionString.ThrowIfNullOrEmpty().Split(';', StringSplitOptions.RemoveEmptyEntries); + for (var i = 0; i < parts.Length; i++) + { + var eq = parts[i].IndexOf('='); + if (eq < 0 || !parts[i][..eq].Equals("Endpoint", StringComparison.OrdinalIgnoreCase)) + continue; + var uri = new Uri(parts[i][(eq + 1)..], UriKind.Absolute); + parts[i] = $"Endpoint={new UriBuilder(uri) { Port = adminPort }.Uri}"; + } + + return string.Join(";", parts) + ";"; + } + + /// + /// Resets the Azure Service Bus queues and topics/subscriptions to an initial state by deleting and recreating them. + /// + /// The Azure Service Bus administration connection string. + /// The queues to reset. + /// The topics and their subscriptions to reset. + public static async Task ResetAzureServiceBusAsync(string adminConnectionString, CreateQueueOptions[]? queues = null, (CreateTopicOptions Topic, CreateSubscriptionOptions[] Subscriptions)[]? topicsAndSubscriptions = null) + { + var admin = new ServiceBusAdministrationClient(adminConnectionString); + + // Recreate each queue fresh - deleting first (if present) so the outcome is identical regardless of prior state. + foreach (var queue in queues ?? []) + { + if (await admin.QueueExistsAsync(queue.Name).ConfigureAwait(false)) + await admin.DeleteQueueAsync(queue.Name).ConfigureAwait(false); + + await admin.CreateQueueAsync(queue).ConfigureAwait(false); + } + + // Recreate each topic (and its subscriptions) fresh - deleting the topic first (if present), which also removes any stale subscriptions. + foreach (var (topic, subscriptions) in topicsAndSubscriptions ?? []) + { + if (await admin.TopicExistsAsync(topic.Name).ConfigureAwait(false)) + await admin.DeleteTopicAsync(topic.Name).ConfigureAwait(false); + + await admin.CreateTopicAsync(topic).ConfigureAwait(false); + + foreach (var subscription in subscriptions) + { + if (subscription.TopicName != topic.Name) + throw new InvalidOperationException($"The subscription '{subscription.SubscriptionName}' TopicName '{subscription.TopicName}' does not match the corresponding topic '{topic.Name}'."); + + await admin.CreateSubscriptionAsync(subscription).ConfigureAwait(false); + } + } + } + + #endregion +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.SqlServer.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.SqlServer.cs index 6eb417e41..805d6aecd 100644 --- a/src/CoreEx.UnitTesting/UnitTestExExtensions.SqlServer.cs +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.SqlServer.cs @@ -101,4 +101,66 @@ public static async Task MigrateSqlServerDataAsync(this TesterBase tester, Func< if (!Success) tester.Implementor.AssertFail("SqlServerMigration failed:" + Environment.NewLine + Output); } -} \ No newline at end of file + + #region Aspire + + /// + /// Execute the using the specified to retrieve the connection string from the distributed application. + /// + /// The to infer the underlying . + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// The resource file names to include in the data loading; see . + /// An optional function to configure the . + /// A task that represents the asynchronous operation. + public static Task MigrateSqlServerDataAsync(this DistributedApplication app, string aspireResourceName, string[]? resourceFileNames = null, Func? configureMigrationArgs = null) + { + resourceFileNames ??= []; + + if (configureMigrationArgs is null) + { + return MigrateSqlServerDataAsync(app, aspireResourceName, ma => + { + ma.DataParserArgs.AddNamed(resourceFileNames); + return ma; + }, typeof(TAssembly).Assembly); + } + else + { + return MigrateSqlServerDataAsync(app, aspireResourceName, ma => + { + var cma = configureMigrationArgs(ma); + cma.DataParserArgs.AddNamed(resourceFileNames); + return cma; + }, typeof(TAssembly).Assembly); + } + } + + /// + /// Execute the using the specified to retrieve the connection string from the distributed application. + /// + /// The . + /// The name of the Aspire resource to retrieve the connection string for. + /// An optional function to configure the . + /// An optional array of assemblies to include in the migration. + public static async Task MigrateSqlServerDataAsync(this DistributedApplication app, string aspireResourceName, Func? configureMigrationArgs = null, params Assembly[] assemblies) + { + var cs = (await AspireTesterBase.GetConnectionStringAsync(app.ThrowIfNull(), aspireResourceName.ThrowIfNullOrEmpty()).ConfigureAwait(false)) ?? throw new InvalidOperationException($"The '{aspireResourceName}' connection string not found."); + var ma = new MigrationArgs(MigrationCommand.All | MigrationCommand.ResetAndData, cs); + + if (configureMigrationArgs is not null) + ma = configureMigrationArgs(ma); + + if (assemblies.Length > 0) + ma.AddAssembly(assemblies); + + // Execute the sql-server migration. + using var m = new SqlServerMigration(ma); + var (Success, Output) = await m.MigrateAndLogAsync().ConfigureAwait(false); + + if (!Success) + throw new InvalidOperationException("SqlServerMigration failed:" + Environment.NewLine + Output); + } + + #endregion +} diff --git a/src/CoreEx.UnitTesting/UnitTestExExtensions.WithAspireTester.cs b/src/CoreEx.UnitTesting/UnitTestExExtensions.WithAspireTester.cs new file mode 100644 index 000000000..2574c5cff --- /dev/null +++ b/src/CoreEx.UnitTesting/UnitTestExExtensions.WithAspireTester.cs @@ -0,0 +1,55 @@ +#pragma warning disable IDE0130 // Namespace does not match folder structure; by design. +namespace UnitTestEx; +#pragma warning restore IDE0130 // Namespace does not match folder structure + +/// +/// Provides a shared to enable usage of the same underlying instance across multiple tests. +/// +/// The AppHost (a Projects.* type generated for the AppHost's ProjectReference). +public abstract class WithAspireTester : IAsyncDisposable where TAppHost : class +{ + private AspireTester? _aspireTester; + + /// + /// Initializes a new instance of the class. + /// + /// The optional function to create the instance. + public WithAspireTester(Func? createFactory = null) + => _aspireTester ??= AspireTester.Create(createFactory).BeforeStart(app => OnBeforeStartAsync(app)).AfterStart(app => OnAfterStartAsync(app)); + + /// + /// Gets the underlying for testing. + /// + public AspireTester Test => _aspireTester ?? throw new ObjectDisposedException(nameof(Test)); + + /// + /// Provides an opportunity to perform any pre-start logic before the is started. + /// + /// The instance that is being started. + protected virtual Task OnBeforeStartAsync(DistributedApplication app) => Task.CompletedTask; + + /// + /// Provides an opportunity to perform any post-start logic after the has been started. + /// + /// The instance that has been started. + protected virtual Task OnAfterStartAsync(DistributedApplication app) => Task.CompletedTask; + + /// + public async ValueTask DisposeAsync() + { + await DisposeCoreAsync().ConfigureAwait(false); + GC.SuppressFinalize(this); + } + + /// + /// Performs the core asynchronous disposal logic; override to extend disposal behavior in derived classes. + /// + protected virtual async ValueTask DisposeCoreAsync() + { + if (_aspireTester is null) + return; + + await _aspireTester.DisposeAsync().ConfigureAwait(false); + _aspireTester = null; + } +} diff --git a/src/CoreEx.Validation/AGENTS.md b/src/CoreEx.Validation/AGENTS.md index d97faf35f..6a6ad8a70 100644 --- a/src/CoreEx.Validation/AGENTS.md +++ b/src/CoreEx.Validation/AGENTS.md @@ -23,7 +23,7 @@ public class ProductValidator : Validator Property(p => p.Price) .Mandatory() - .CompareValue(CompareOperator.GreaterThan, 0m); + .GreaterThan(0m); } } ``` @@ -65,10 +65,19 @@ Property(p => p.DiscountCode) .WhenEntity(product => product.HasDiscount); Property(p => p.ExpiresOn) - .CompareValue(CompareOperator.GreaterThan, () => Runtime.UtcNow) + .GreaterThan(_ => Runtime.UtcNow) .WhenHasValue(); ``` +### Comparing two properties + +Use `CompareProperty` to compare a property against another property on the **same entity** — do not hand-write it in `OnValidateAsync`. The comparison is skipped when either value is `null` or the compare-to property already has an error, so it only runs when both are present and valid. Chain `Mandatory()` first where the property must also be supplied. + +```csharp +Property(p => p.EndsOn).CompareProperty(CompareOperator.GreaterThanOrEqualTo, p => p.StartsOn); +Property(p => p.EndsOn).Mandatory().CompareProperty(CompareOperator.GreaterThanOrEqualTo, p => p.StartsOn); +``` + ## Reusable Validators Use `CommonValidator` for logic reused across multiple entity validators. @@ -83,8 +92,10 @@ HasRuleFor(p => p.Sku).Common(SkuValidator); ## Do Not -- Do not call `ValidateAsync` and ignore errors — always call `ValidateAndThrowAsync` or check `HasErrors` and throw `ValidationException` explicitly. -- Do not add try/catch around `ValidateAndThrowAsync` to rethrow as a different exception type — `ValidationException` maps to HTTP 400 automatically. +- Do not compare two properties of the same entity imperatively in `OnValidateAsync` — use `CompareProperty(op, p => p.Other)`; it already handles nulls and cascading errors. +- Do not use `CompareValue(...)` — no such extension exists; use `Compare(op, value)` or the dedicated `GreaterThan`/`LessThan`/etc. rules. +- Do not call `ValidateAsync` and ignore errors — use `ValidateAndThrowAsync` or inspect and handle the validation result. +- Do not add try/catch around application `ValidateAndThrowAsync` to rethrow as a different exception type — `ValidationException` maps to HTTP 400 automatically. - Do not validate inside `TransactionAsync` — validate first (before the transaction) so failed validation never opens a database transaction. - Do not use `DataAnnotations` attributes alongside CoreEx validators — pick one approach per entity and stay consistent. - Do not use FluentValidation alongside CoreEx validators unless bridging with `InteropRule` is explicitly needed. diff --git a/src/CoreEx.Validation/README.md b/src/CoreEx.Validation/README.md index 9f6858f18..f268a8c53 100644 --- a/src/CoreEx.Validation/README.md +++ b/src/CoreEx.Validation/README.md @@ -64,8 +64,8 @@ Rules apply the actual validation logic to a property value. Each rule class is | **[`BetweenRule`](./Rules/BetweenRule.cs)** | Validates that a comparable value lies between two bounds (inclusive or exclusive). | `Between(min, max)`, `InclusiveBetween(min, max)`, `ExclusiveBetween(min, max)` | | **[`CollectionRule`](./Rules/CollectionRule.cs)** | Validates an `IEnumerable` collection: min/max count, duplicate detection (by Id, key, property, or equality), and optional per-item entity validation. | `Collection(with)`, `Collection(maxCount)`, `Collection(minCount, maxCount)`; `With` helpers: `WithDuplicateIdCheck()`, `WithDuplicateKeyCheck()`, `WithDuplicatePropertyCheck()`, `WithDuplicateCheck()` | | **[`CommonRule`](./Rules/CommonRule.cs)** | Applies a `CommonValidator` (or `InlineValidator`) to a property, enabling shared rule logic to be reused across multiple properties or validators. | `Common(commonValidator)` | -| **[`ComparePropertyRule`](./Rules/ComparePropertyRule.cs)** | Compares the property value against another property on the same entity using a `CompareOperator`; skips if the target property already has an error. | `CompareProperty(op, expression)` | -| **[`CompareValueRule`](./Rules/CompareValueRule.cs)** | Compares the property value against a supplied constant or function value using a `CompareOperator`. | `CompareValue(op, value)`, `Equal(value)`, `NotEqual(value)`, `LessThan(value)`, `LessThanOrEqualTo(value)`, `GreaterThan(value)`, `GreaterThanOrEqualTo(value)` | +| **[`ComparePropertyRule`](./Rules/ComparePropertyRule.cs)** | Compares the property value against another property on the same entity using a `CompareOperator`; skips if either value is `null` or the target property already has an error (so it compares only when both are present and valid; chain `Mandatory()` to also require the value). | `CompareProperty(op, expression)` | +| **[`CompareValueRule`](./Rules/CompareValueRule.cs)** | Compares the property value against a supplied constant or function value using a `CompareOperator`. | `Compare(op, value)`, `Equal(value)`, `NotEqual(value)`, `LessThan(value)`, `LessThanOrEqualTo(value)`, `GreaterThan(value)`, `GreaterThanOrEqualTo(value)` | | **[`CompareValuesRule`](./Rules/CompareValuesRule.cs)** | Validates that the property value is equal to one of a supplied set of values; optionally replaces the value with the matched entry. | `CompareValues(values)`, `CompareValues(Func)` | | **[`DecimalRule`](./Rules/DecimalRule.cs)** | Validates `decimal` precision and scale, and optional sign constraint. Also covers `IFloatingPoint` via `PrecisionScale`. | `Decimal(precision, scale)`, `PrecisionScale(precision, scale)` | | **[`DictionaryRule`](./Rules/DictionaryRule.cs)** | Validates an `IDictionary`: min/max count and optional per-entry value validation. | `Dictionary(with)`, `Dictionary(maxCount)`, `Dictionary(minCount, maxCount)` | @@ -101,4 +101,4 @@ Rules apply the actual validation logic to a property value. Each rule class is ## AI Usage Guide -An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. \ No newline at end of file +An [`AGENTS.md`](./AGENTS.md) file is included with this package. AI coding assistants (GitHub Copilot, Claude, Cursor, etc.) that support workspace-injected package documentation will automatically surface concise usage guidance, code examples, and `Do Not` rules for this package without requiring a local CoreEx checkout. diff --git a/src/CoreEx.Validation/ReferenceDataValidator.cs b/src/CoreEx.Validation/ReferenceDataValidator.cs new file mode 100644 index 000000000..1f145816a --- /dev/null +++ b/src/CoreEx.Validation/ReferenceDataValidator.cs @@ -0,0 +1,34 @@ +namespace CoreEx.Validation; + +/// +/// Provides the base validator. +/// +/// The . +public partial class ReferenceDataValidator : Validator where TRef : class, IReferenceData +{ + /// + /// Gets or sets the maximum length of the property. + /// + public static int CodeMaximumLength { get; set; } = 50; + + /// + /// Gets or sets the maximum length of the property. + /// + public static int TextMaximumLength { get; set; } = 250; + + /// + /// Gets or sets the maximum length of the property. + /// + public static int DescriptionMaximumLength { get; set; } = 1000; + + /// + /// Initializes a new instance of the class. + /// + public ReferenceDataValidator() + { + Property(p => p.Code).Mandatory().MaximumLength(CodeMaximumLength); + Property(p => p.Text).Mandatory().MaximumLength(TextMaximumLength); + Property(p => p.Description).MaximumLength(DescriptionMaximumLength); + Property(p => p.EndsOn).CompareProperty(CompareOperator.GreaterThanOrEqualTo, p => p.StartsOn); + } +} diff --git a/src/CoreEx.Validation/Rules/ComparePropertyRule.cs b/src/CoreEx.Validation/Rules/ComparePropertyRule.cs index 84f3ba501..2be139659 100644 --- a/src/CoreEx.Validation/Rules/ComparePropertyRule.cs +++ b/src/CoreEx.Validation/Rules/ComparePropertyRule.cs @@ -10,6 +10,7 @@ namespace CoreEx.Validation.Rules; /// >The to reference the compare-to entity property. /// The value text formatter (used in the error message); otherwise, uses the resulting compare-to value. /// The optional . +/// Where the is null then then is used. public sealed class ComparePropertyRule(CompareOperator compareOperator, Expression> compareToPropertyExpression, Func? compareToText = null, IComparer? comparer = null) : CompareRuleBase(compareOperator, compareToText, comparer) where TEntity : class where TProperty : IComparable { @@ -35,7 +36,7 @@ protected override Task OnValidateAsync(PropertyContext cont if (compareTo is TProperty casted) { if (!Compare(context.Value, casted)) - CreateErrorMessage(context, casted); + CreateErrorMessage(context, casted, _compareToProperty.Text); return Task.CompletedTask; } @@ -45,7 +46,7 @@ protected override Task OnValidateAsync(PropertyContext cont { var changed = (TProperty)Convert.ChangeType(compareTo, typeof(TProperty)); if (!Compare(context.Value, changed)) - CreateErrorMessage(context, changed); + CreateErrorMessage(context, changed, _compareToProperty.Text); } catch (Exception ex) when (ex is InvalidCastException || ex is FormatException || ex is OverflowException) { @@ -54,4 +55,4 @@ protected override Task OnValidateAsync(PropertyContext cont return Task.CompletedTask; } -} \ No newline at end of file +} diff --git a/src/CoreEx.Validation/Rules/CompareRuleBase.cs b/src/CoreEx.Validation/Rules/CompareRuleBase.cs index 0e5046a3d..0ea48edea 100644 --- a/src/CoreEx.Validation/Rules/CompareRuleBase.cs +++ b/src/CoreEx.Validation/Rules/CompareRuleBase.cs @@ -47,10 +47,11 @@ public abstract class CompareRuleBase(CompareOperator compar /// /// The . /// The compare-to value. - protected void CreateErrorMessage(PropertyContext context, TProperty compareToValue) + /// The optional compare-to text backup. + protected void CreateErrorMessage(PropertyContext context, TProperty compareToValue, LText? compareToTextBackup = null) { context.ThrowIfNull(); - var compareToText = CompareToText?.Invoke(compareToValue) ?? context.FormatValue(compareToValue); + var compareToText = CompareToText?.Invoke(compareToValue) ?? compareToTextBackup ?? context.FormatValue(compareToValue); switch (Operator) { @@ -62,4 +63,4 @@ protected void CreateErrorMessage(PropertyContext context, T case CompareOperator.GreaterThanOrEqualTo: context.AddError(ErrorText ?? ValidatorStrings.CompareGreaterThanEqualFormat, compareToText); break; } } -} \ No newline at end of file +} diff --git a/src/CoreEx.Validation/Rules/README.md b/src/CoreEx.Validation/Rules/README.md index c87d409a9..67eacd0ea 100644 --- a/src/CoreEx.Validation/Rules/README.md +++ b/src/CoreEx.Validation/Rules/README.md @@ -35,8 +35,8 @@ | **[`DecimalRule`](./DecimalRule.cs)** | Validates `decimal` precision and scale; `PrecisionScale` covers generic `IFloatingPoint`; wired via `Decimal()`, `PrecisionScale()`. | | **[`BetweenRule`](./BetweenRule.cs)** | Validates a comparable value lies between two bounds; wired via `Between()`, `InclusiveBetween()`, `ExclusiveBetween()`. | | _[`CompareRuleBase`](./CompareRuleBase.cs)_ | Abstract base for `CompareValueRule` and `ComparePropertyRule`; holds the `CompareOperator` and optional `IComparer`. | -| **[`CompareValueRule`](./CompareValueRule.cs)** | Compares property value against a constant or function value using a `CompareOperator`; wired via `CompareValue()`, `Equal()`, `NotEqual()`, `LessThan()`, `LessThanOrEqualTo()`, `GreaterThan()`, `GreaterThanOrEqualTo()`. | -| **[`ComparePropertyRule`](./ComparePropertyRule.cs)** | Compares property value against another property on the same entity; skips if the target property has an error; wired via `CompareProperty(op, expression)`. | +| **[`CompareValueRule`](./CompareValueRule.cs)** | Compares property value against a constant or function value using a `CompareOperator`; wired via `Compare()`, `Equal()`, `NotEqual()`, `LessThan()`, `LessThanOrEqualTo()`, `GreaterThan()`, `GreaterThanOrEqualTo()`. | +| **[`ComparePropertyRule`](./ComparePropertyRule.cs)** | Compares property value against another property on the same entity; skips if either value is `null` or the target property has an error (so it compares only when both are present and valid; chain `Mandatory()` to also require the value); wired via `CompareProperty(op, expression)`. | | **[`CompareValuesRule`](./CompareValuesRule.cs)** | Validates equality against a set of allowed values, optionally replacing the property value with the matched entry; wired via `CompareValues(values)`. | | **[`EnumRule`](./EnumRule.cs)** | Validates a `struct Enum` is defined or within an allowed subset; wired via `Enum()`, `Enum(allowed[])`. | | **[`EnumStringRule`](./EnumStringRule.cs)** | Validates a `string` can be parsed to an `Enum` type, configured via the `EnumWith` nested builder; wired via `Enum(Func)`. | @@ -58,4 +58,4 @@ - **[`CoreEx.Validation`](../README.md)** - Root namespace; `ValidationExtensions` partial classes are the sole public entry point for creating and attaching all rule types defined here. - **[`CoreEx.Validation.Abstractions`](../Abstractions/README.md)** - Defines `IPropertyRule`, `PropertyRuleBase`, `IPropertyContext`, and `IValidatorEx` consumed by every rule implementation. - **[`CoreEx.Validation.Clauses`](../Clauses/README.md)** - Clause implementations that guard rule execution; clauses are attached to `IPropertyRule` instances defined here. -- **[`CoreEx.RefData`](../../CoreEx.RefData/README.md)** - Provides `IReferenceData` and `IReferenceDataCodeCollection` validated by `ReferenceDataRule`, `ReferenceDataCodeRule`, and `ReferenceDataCodeCollectionRule`. \ No newline at end of file +- **[`CoreEx.RefData`](../../CoreEx.RefData/README.md)** - Provides `IReferenceData` and `IReferenceDataCodeCollection` validated by `ReferenceDataRule`, `ReferenceDataCodeRule`, and `ReferenceDataCodeCollectionRule`. diff --git a/src/CoreEx.Validation/ValidationExtensions.ComparePropertyRule.cs b/src/CoreEx.Validation/ValidationExtensions.ComparePropertyRule.cs index 7ca903985..8bccb9417 100644 --- a/src/CoreEx.Validation/ValidationExtensions.ComparePropertyRule.cs +++ b/src/CoreEx.Validation/ValidationExtensions.ComparePropertyRule.cs @@ -13,6 +13,7 @@ public static partial class ValidationExtensions /// >The to reference the compare-to entity property. /// The compare-to value text formatter (used in the error message); otherwise, uses the resulting compare-to value. /// The optional . + /// Where the is null then then is used. public static IPropertyRule CompareProperty(this IPropertyRule rule, CompareOperator compareOperator, Expression> compareToPropertyExpression, Func? compareToText = null, IComparer? comparer = null) where TEntity : class where TProperty : IComparable => Chain(rule, new ComparePropertyRule(compareOperator, compareToPropertyExpression, compareToText, comparer)); @@ -27,6 +28,7 @@ public static IPropertyRule CompareProperty>The to reference the compare-to entity property. /// The compare-to value text formatter (used in the error message); otherwise, uses the resulting compare-to value. /// The optional . + /// Where the is null then then is used. public static IPropertyRule CompareProperty(this IPropertyRule rule, CompareOperator compareOperator, Expression> compareToPropertyExpression, Func? compareToText = null, IComparer? comparer = null) where TEntity : class where TProperty : struct, IComparable => Chain(rule, new ComparePropertyRule(compareOperator, compareToPropertyExpression, compareToText, comparer)); -} \ No newline at end of file +} diff --git a/src/CoreEx/AGENTS.md b/src/CoreEx/AGENTS.md index 9728fca70..65e0e9762 100644 --- a/src/CoreEx/AGENTS.md +++ b/src/CoreEx/AGENTS.md @@ -53,6 +53,35 @@ return await GetOrderAsync(id) .ConfigureAwait(false); ``` +## Validating HTTP Response Values + +Use `response.WithValidator(validator).ToResultAsync(ct)` to validate a successful, +non-null deserialized dependency response. The typed wrapper returns the validation result's +`Value`; reported validation errors become failed results containing `HttpRequestException` +with structured inner validation diagnostics. `GetValueAsync(ct)` is the throwing equivalent. +Standard API handling treats these as internal 500 errors, not caller 400 validation errors. + +Required methods reject missing content and null validated values; `OrDefault` methods allow +null and skip validation for absent response values. Existing HTTP/ProblemDetails handling, +deserialization exceptions, unexpected validator exceptions, and cancellation are unchanged. +Dispose the response at the call site; the wrapper does not own it. + +For small client-only rules, configure a private static readonly `Validator` once with +`Validator.Create().HasProperty(...)` from the `CoreEx.Validation` package. Keep these rules +inside the Infrastructure HTTP client, not Application validators; validate consumed fields, +not create/update requirements or identifier matching by default. Test through the client. + +```csharp +private static readonly Validator _validator = Validator.Create() + .HasProperty(x => x.Email, c => c.Mandatory().Email()); + +// Inside the asynchronous HTTP client method. +using var response = await _httpClient.GetAsync($"api/customers/{Uri.EscapeDataString(id)}", ct).ConfigureAwait(false); +return await response.WithValidator(_validator).ToResultAsync(ct).ConfigureAwait(false); +``` + +See [usage, inline examples, and semantics](./README.md#validating-http-response-values). + ## Entity Contracts Use the standard interfaces on your contracts so the framework's ETag, paging, and change-log handling works automatically. @@ -88,6 +117,8 @@ public class Product : IIdentifier, IETag, IChangeLog Server-managed fields (e.g. `ETag`, `ChangeLog`) should be marked `[ReadOnly(true)]` on generated contracts so they are excluded from inbound deserialization. +Properties that are mandatory/required but declared nullable (`string?`, `int?`) for DTO flexibility should be marked `[NonNullable]` (`CoreEx.Entities`) so OpenAPI (via `CoreEx.AspNetCore.NSwag`) reports them as `nullable: false`. Do not confuse it with `System.Diagnostics.CodeAnalysis.NotNull`. + ## Dependency Injection Attributes Mark implementation classes with `[ScopedService]`, `[SingletonService]`, or `[TransientService]` and register them all at once via `AddDynamicServicesUsing()`. @@ -110,7 +141,7 @@ builder.Services.AddPrecisionTimeProvider(); ## Do Not -- Do not catch `IExtendedException` types to re-wrap them — let the framework middleware translate them. +- Do not catch application `IExtendedException` types to re-wrap them — let the framework middleware translate them. For reported external-response validation failures, use `WithValidator` to classify them as dependency failures instead. - Do not use `AutoMapper` — use explicit `Mapper` or `BiDirectionMapper`. - Do not use `DateTime.UtcNow` directly — use `Runtime.UtcNow`, which returns `ExecutionContext.Timestamp` when a context is active and falls back to `TimeProvider.System.GetUtcNow()` otherwise. diff --git a/src/CoreEx/Caching/IHybridCache.cs b/src/CoreEx/Caching/IHybridCache.cs index bf3d3e639..c3eff4ca5 100644 --- a/src/CoreEx/Caching/IHybridCache.cs +++ b/src/CoreEx/Caching/IHybridCache.cs @@ -53,6 +53,33 @@ public interface IHybridCache /// The cached value. Task GetOrCreateByKeyAsync(string key, Func> factory, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default); + /// + /// Gets the cached value for the specified key using the to create and set only where successful, where not found. + /// + /// The cache value . + /// The cache key. + /// The function used to create the (e.g. a remote call) where not cached. + /// The optional . + /// The . + /// The cached (or created) value as a successful ; otherwise, the failure from the (which is never cached). + /// Failures (e.g. a not-found) are returned as-is and never cached; no exception is thrown for them. + /// The default implementation performs a get followed by a set, which is not atomic; i.e. concurrent cache misses may each invoke the (no stampede protection). + /// Implementations that support it (e.g. FusionCache) override this to invoke the at most once per key concurrently (per node by default). + async Task> GetOrCreateByKeyWithResultAsync(string key, Func>> factory, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) + { + factory.ThrowIfNull(); + + var (exists, value) = await TryGetByKeyAsync(key, options, cancellationToken).ConfigureAwait(false); + if (exists) + return value!; + + var result = await factory(cancellationToken).ConfigureAwait(false); + if (result.IsSuccess) + await SetByKeyAsync(key, result.Value, options, cancellationToken).ConfigureAwait(false); + + return result; + } + /// /// Removes the cached value for the specified key. /// diff --git a/src/CoreEx/Caching/README.md b/src/CoreEx/Caching/README.md index 94288aadb..2e806e198 100644 --- a/src/CoreEx/Caching/README.md +++ b/src/CoreEx/Caching/README.md @@ -26,7 +26,8 @@ | **[`HybridCacheEntryOptions`](./HybridCacheEntryOptions.cs)** | Per-entry options record specifying `CacheStrategy`, `LocalExpiration`, `DistributedExpiration`, and `Tags`; reads global defaults from `IConfiguration`. | | **[`MemoryOnlyHybridCache`](./MemoryOnlyHybridCache.cs)** | `IHybridCache` implementation backed by `IMemoryCache`; ignores the `Strategy` setting and always stores locally. | | **[`CacheStrategy`](./CacheStrategy.cs)** | Flags enum: `Local` (in-process), `Distributed`, or `Hybrid` (both). | -| [`IHybridCache`](./IHybridCache.cs) | Core cache interface: `TryGetByKeyAsync`, `GetOrCreateByKeyAsync`, `GetOrDefaultByKeyAsync`, `SetByKeyAsync`, `RemoveByKeyAsync`, and tag-based `RemoveByTagAsync`. | +| [`IHybridCache`](./IHybridCache.cs) | Core cache interface: `TryGetByKeyAsync`, `GetOrCreateByKeyAsync`, `GetOrCreateByKeyWithResultAsync` (default interface method; caches successful `Result` only), `GetOrDefaultByKeyAsync`, `SetByKeyAsync`, `RemoveByKeyAsync`, and tag-based `RemoveByTagAsync`. | +| [`Extensions.IHybridCache`](../Extensions.IHybridCache.cs) | Result-aware helpers: `GetOrCreateByKeyWithResultAsync` is an `IHybridCache` member (default: non-atomic get-then-set; `FusionHybridCache` overrides it with FusionCache's single-flight so concurrent misses invoke the factory once per key per node, or once cluster-wide where a FusionCache distributed locker is configured). `GetOrCreateWithResultAsync` is the `CompositeKey` extension. Both take a `Func>>` factory and cache **only on success** (failures pass through uncached). Ideal for one-line cached adapter lookups. | | [`ICacheKeyProvider`](./ICacheKeyProvider.cs) | Interface for producing fully-qualified string cache keys from raw keys or `IEntityKey` / `CompositeKey` pairs. | ## Related Namespaces diff --git a/src/CoreEx/CoreEx.csproj b/src/CoreEx/CoreEx.csproj index f7e9b5b41..fbde69965 100644 --- a/src/CoreEx/CoreEx.csproj +++ b/src/CoreEx/CoreEx.csproj @@ -41,6 +41,7 @@ + diff --git a/src/CoreEx/CoreExExtensions.ApplicationBuilder.cs b/src/CoreEx/CoreExExtensions.ApplicationBuilder.cs index 998931c72..471de27bf 100644 --- a/src/CoreEx/CoreExExtensions.ApplicationBuilder.cs +++ b/src/CoreEx/CoreExExtensions.ApplicationBuilder.cs @@ -31,7 +31,7 @@ public static IHostApplicationBuilder AddHostSettings(this IHostApplicationBuild } /// - /// Adds an opinionated typed with idempotency key handler and standard resilience handlers. + /// Adds an opinionated typed with idempotency key handler, service discovery, and standard resilience handlers. /// /// The typed client. /// The . @@ -40,13 +40,14 @@ public static IHostApplicationBuilder AddHostSettings(this IHostApplicationBuild /// An optional action to configure the . /// The for fluent-style method-chaining. /// The also represents the configuration section name for the ; as a minimum define the . - /// The two handlers added are added, in order specified, as follows: + /// The handlers are added, in order specified, as follows: /// /// - Adds an idempotency key to outgoing HTTP requests. + /// Service discovery - Resolves the host as a logical service name (e.g. https://products-api) against a Services:{name}:{scheme} configuration section (or, on supporting platforms, a native resolver) before falling back unchanged to a literal address if no matching entry is found - see Microsoft.Extensions.ServiceDiscovery. This makes an Aspire-orchestrated BaseAddress (populated via the AppHost's WithReference) and a literal production URL both work unchanged through the same code path - no Aspire AppHost is required at runtime for the latter. /// Standard resilience handlers - Adds standard resilience policies as enabled by Microsoft.Extensions.Http.Resilience. /// /// - /// Example configuration section for a named 'ProductsApi': + /// Example configuration section for a named 'ProductsApi' pointing to a literal production address: /// /// { /// "ProductsApi": { @@ -57,13 +58,24 @@ public static IHostApplicationBuilder AddHostSettings(this IHostApplicationBuild /// } /// } /// + /// Or, when consumed from within an Aspire AppHost (local development/testing), pointing to the logical Aspire resource name (e.g. products-api) instead of a hardcoded host/port - the AppHost's WithReference(productsApi) then supplies the actual resolved endpoint(s) via the Services:products-api:* configuration section regardless of which port the referenced project ultimately binds to: + /// + /// { + /// "ProductsApi": { + /// "BaseAddress": "https+http://products-api" + /// } + /// } + /// /// public static IHttpClientBuilder AddTypedHttpClient(this IHostApplicationBuilder builder, string name, Action? configureClient = null, Action? configureIdempotency = null) where TClient : class { - var config = builder.Configuration.GetSection(name) ?? throw new ArgumentException($"Unable to find configuration section for '{name}'."); + var config = builder.ThrowIfNull().Configuration.GetSection(name) ?? throw new ArgumentException($"Unable to find configuration section for '{name}'."); + + // Registers the underlying service endpoint resolver providers (configuration-based + pass-through); safe/idempotent to call more than once per host. + builder.Services.AddServiceDiscovery(); - var cb = builder.ThrowIfNull().Services.AddHttpClient(name, client => + var cb = builder.Services.AddHttpClient(name, client => { // Set the standard configured setting. client.BaseAddress = config.GetValue("BaseAddress") ?? throw new ArgumentException($"Unable to find '{nameof(HttpClient.BaseAddress)}' configuration for '{name}'."); @@ -71,6 +83,7 @@ public static IHttpClientBuilder AddTypedHttpClient(this IHostApplicati }); cb.AddIdempotencyKeyHandler((sp, handler) => configureIdempotency?.Invoke(sp, handler)); + cb.AddServiceDiscovery(); if (config.GetSection("Resilience").Exists()) cb.AddStandardResilienceHandler(config); @@ -93,13 +106,14 @@ public static IHttpClientBuilder AddTypedHttpClient(this IHostApplicati /// An optional action to configure the . /// The for fluent-style method-chaining. /// The also represents the configuration section name for the ; as a minimum define the . - /// The two handlers added are added, in order specified, as follows: + /// The handlers are added, in order specified, as follows: /// /// - Adds an idempotency key to outgoing HTTP requests. + /// Service discovery - Resolves the host as a logical service name (e.g. https://products-api) against a Services:{name}:{scheme} configuration section (or, on supporting platforms, a native resolver) before falling back unchanged to a literal address if no matching entry is found - see Microsoft.Extensions.ServiceDiscovery. This makes an Aspire-orchestrated BaseAddress (populated via the AppHost's WithReference) and a literal production URL both work unchanged through the same code path - no Aspire AppHost is required at runtime for the latter. /// Standard resilience handlers - Adds standard resilience policies as enabled by Microsoft.Extensions.Http.Resilience. /// /// - /// Example configuration section for a named 'ProductsApi': + /// Example configuration section for a named 'ProductsApi' pointing to a literal production address: /// /// { /// "ProductsApi": { @@ -110,13 +124,24 @@ public static IHttpClientBuilder AddTypedHttpClient(this IHostApplicati /// } /// } /// + /// Or, when consumed from within an Aspire AppHost (local development/testing), pointing to the logical Aspire resource name (e.g. products-api) instead of a hardcoded host/port - the AppHost's WithReference(productsApi) then supplies the actual resolved endpoint(s) via the Services:products-api:* configuration section regardless of which port the referenced project ultimately binds to: + /// + /// { + /// "ProductsApi": { + /// "BaseAddress": "https+http://products-api" + /// } + /// } + /// /// public static IHttpClientBuilder AddTypedHttpClient(this IHostApplicationBuilder builder, string name, Action? configureClient = null, Action? configureIdempotency = null) where TClient : class where TImplementation : class, TClient { - var config = builder.Configuration.GetSection(name) ?? throw new ArgumentException($"Unable to find configuration section for '{name}'."); + var config = builder.ThrowIfNull().Configuration.GetSection(name) ?? throw new ArgumentException($"Unable to find configuration section for '{name}'."); + + // Registers the underlying service endpoint resolver providers (configuration-based + pass-through); safe/idempotent to call more than once per host. + builder.Services.AddServiceDiscovery(); - var cb = builder.ThrowIfNull().Services.AddHttpClient(name, client => + var cb = builder.Services.AddHttpClient(name, client => { // Set the standard configured setting. client.BaseAddress = config.GetValue("BaseAddress") ?? throw new ArgumentException($"Unable to find '{nameof(HttpClient.BaseAddress)}' configuration for '{name}'."); @@ -124,6 +149,7 @@ public static IHttpClientBuilder AddTypedHttpClient(th }); cb.AddIdempotencyKeyHandler((sp, handler) => configureIdempotency?.Invoke(sp, handler)); + cb.AddServiceDiscovery(); if (config.GetSection("Resilience").Exists()) cb.AddStandardResilienceHandler(config); diff --git a/src/CoreEx/Data/DataExtensions.ITotalCount.cs b/src/CoreEx/Data/DataExtensions.ITotalCount.cs index 9616d1c2b..16ecd5e39 100644 --- a/src/CoreEx/Data/DataExtensions.ITotalCount.cs +++ b/src/CoreEx/Data/DataExtensions.ITotalCount.cs @@ -70,7 +70,7 @@ public async static Task WithTotalCountAsync(this TSource sour { source.WithTotalCount(await totalCount(cancellationToken).ConfigureAwait(false)); } - catch (Exception ex) + catch (Exception ex) when (!ex.IsCanceledBy(cancellationToken)) // A requested cancellation must flow, not be swallowed as an unknown count. { WithTotalCountException(ex); } diff --git a/src/CoreEx/Data/PartitionKey.cs b/src/CoreEx/Data/PartitionKey.cs index 1cf24f1ff..e2e409650 100644 --- a/src/CoreEx/Data/PartitionKey.cs +++ b/src/CoreEx/Data/PartitionKey.cs @@ -10,6 +10,18 @@ public class PartitionKey /// public const int DefaultPartitionSize = 4; + /// + /// Gets the fixed partition-key value used when an event has no partition key. + /// + public const string DefaultNoPartitionKey = "$none"; + + /// + /// Validates and returns the specified partition size. + /// + /// The partition size (i.e. the number of possible partitions). + /// The validated partition size. + public static int ValidatePartitionSize(int partitionSize) => partitionSize.ThrowWhen(ps => ps <= 0 || ps > 256, nameof(partitionSize)); + /// /// Gets the maximum byte count for stack allocation when converting strings to UTF-8. /// @@ -26,7 +38,7 @@ public class PartitionKey public static int GetPartitionId(string partitionKey, int partitionSize = DefaultPartitionSize, bool ignoreCase = true) { partitionKey = partitionKey.ThrowIfNull().Trim().ThrowIfNullOrEmpty().ThrowWhen(pk => pk.Length > 256, nameof(partitionKey)); - partitionSize.ThrowWhen(ps => ps <= 0 || ps > 256, nameof(partitionSize)); + partitionSize = ValidatePartitionSize(partitionSize); static int GenerateIdFromHash(ReadOnlySpan bytes, int partitionSize) { @@ -76,4 +88,4 @@ public static string GetPartitionIdAsString(string partitionKey, int partitionSi return id.ToString($"D{digits}", CultureInfo.InvariantCulture); } -} \ No newline at end of file +} diff --git a/src/CoreEx/Entities/Extended/IIdentifierGenerator.cs b/src/CoreEx/Entities/Extended/IIdentifierGenerator.cs index 8cc9d3ef1..69cc8ac8e 100644 --- a/src/CoreEx/Entities/Extended/IIdentifierGenerator.cs +++ b/src/CoreEx/Entities/Extended/IIdentifierGenerator.cs @@ -32,5 +32,5 @@ public interface IIdentifierGenerator /// /// The to generate for. /// The value to assign an identifier for. - Task AssignIdentifierAsync(TFor value) where TFor : class; -} \ No newline at end of file + Task AssignIdentifierAsync(TFor value) where TFor : class, IIdentifier; +} diff --git a/src/CoreEx/Entities/IdentifierGenerator.cs b/src/CoreEx/Entities/IdentifierGenerator.cs index f8165bc4f..1c76fe935 100644 --- a/src/CoreEx/Entities/IdentifierGenerator.cs +++ b/src/CoreEx/Entities/IdentifierGenerator.cs @@ -33,11 +33,8 @@ Type _ when typeof(TId) == typeof(Guid) => Internal.Cast(GenerateGuid public async Task GenerateIdentifierAsync() where TFor : class => await GenerateIdentifierAsync().ConfigureAwait(false); /// - public async Task AssignIdentifierAsync(TFor value) where TFor : class + public async Task AssignIdentifierAsync(TFor value) where TFor : class, IIdentifier { - if (value is not IReadOnlyIdentifier ii) - return; - if (value is IIdentifier iis) iis.Id ??= await GenerateIdentifierAsync().ConfigureAwait(false); else if (value is IIdentifier iig) @@ -46,6 +43,6 @@ public async Task AssignIdentifierAsync(TFor value) where TFor : class iig.Id = await GenerateIdentifierAsync().ConfigureAwait(false); } else - throw new NotSupportedException($"Identifier Type '{ii.IdType.Name}' is not supported; only String or Guid."); + throw new NotSupportedException($"Identifier Type '{value.IdType.Name}' is not supported; only String or Guid."); } -} \ No newline at end of file +} diff --git a/src/CoreEx/Entities/NonNullableAttribute.cs b/src/CoreEx/Entities/NonNullableAttribute.cs new file mode 100644 index 000000000..e24f5e032 --- /dev/null +++ b/src/CoreEx/Entities/NonNullableAttribute.cs @@ -0,0 +1,9 @@ +namespace CoreEx.Entities; + +/// +/// Specifies that the property is not intended to be nullable, even though it may be nullable in the code. +/// +/// This attribute is typically used to indicate that a property for the likes of OpenAPI is not-nullable. This is intended for a contract (DTO) that is typically used to transfer data between +/// layers or systems, and this attribute helps to signal non-nullability in such scenarios. +[AttributeUsage(AttributeTargets.Property, AllowMultiple = false, Inherited = true)] +public sealed class NonNullableAttribute : Attribute { } diff --git a/src/CoreEx/Entities/README.md b/src/CoreEx/Entities/README.md index d5c862fa8..4a5641371 100644 --- a/src/CoreEx/Entities/README.md +++ b/src/CoreEx/Entities/README.md @@ -20,6 +20,7 @@ The namespace also includes supporting utilities: `Cleaner` for normalizing stri - 🧹 **Cleaning and normalization**: `Cleaner` applies configurable `StringTrim`, `StringTransform` (null/empty), `StringCase` (upper/lower/title), and `DateTimeTransform` (UTC/local) policies across entity properties before save. - 📐 **Feature detection**: `FeatureSupport` and extension methods allow infrastructure code to detect which interfaces an entity implements and act accordingly (e.g. auto-populate ETags or change logs). - ✍️ **Write-intent descriptors**: `Writable` enum and `[Writable]` attribute annotate entity properties with create-only / update-only / never-writable semantics for OpenAPI documentation and tooling. +- ✅ **Non-nullable intent**: `[NonNullable]` marks a property as mandatory/non-null for OpenAPI tooling even though it is declared nullable (`string?`, `int?`) in code for DTO flexibility. - 🔧 **Identifier generation**: `IdentifierGenerator` and `IIdentifierGenerator` provide DI-replaceable new `Guid`/`string` identifier creation (using `Guid.CreateVersion7()` on .NET 9+). ## Key types @@ -38,6 +39,7 @@ The namespace also includes supporting utilities: `Cleaner` for normalizing stri | **[`MessageItem`](./MessageItem.cs)** | Record carrying a single structured message — type (Error/Warning/Info), text (`LText`), and optional property name — returned from validation and service operations. | | **[`MessageCollection`](./MessageCollection.cs)** | Collection of `MessageItem` instances; attached to `ValidationException` and surfaced via `ExecutionContext.Messages`. | | **[`ValueResult`](./ValueResult.cs)** | Wraps a value alongside an `IResult` to carry both a typed result and an associated result state. | +| **[`NonNullableAttribute`](./NonNullableAttribute.cs)** | Property attribute declaring a nullable-in-code property as non-nullable for OpenAPI (read by `CoreEx.AspNetCore.NSwag`). | | **[`Writable`](./Writable.cs)** | Enum describing property write intent: `Always`, `Never`, `CreateOnly`, or `UpdateOnly`; used with `[WritableAttribute]` for OpenAPI tooling. | | [`IContract`](./IContract.cs) | Marker interface combining `IRuntimeMetadata`, `ICopyFrom`, and `IDefault` — the aggregate contract implemented by generated domain types. | | [`IContract`](./IContractT.cs) | Generic version of `IContract` providing strongly-typed `CopyFrom(T)` and `IsDefault()` implementations via `RuntimeMetadata`. | diff --git a/src/CoreEx/Extensions.HttpResponseMessage.cs b/src/CoreEx/Extensions.HttpResponseMessage.cs index 33a86a546..6241e52d1 100644 --- a/src/CoreEx/Extensions.HttpResponseMessage.cs +++ b/src/CoreEx/Extensions.HttpResponseMessage.cs @@ -2,6 +2,15 @@ namespace CoreEx; public static partial class Extensions { + /// + /// Configures validation of a successful, non-null deserialized HTTP response value. + /// + /// The response value type. + /// The response, whose lifetime remains the caller's responsibility. + /// The response value validator. + /// A typed wrapper that validates when a value is consumed. + public static ValidatedHttpResponse WithValidator(this HttpResponseMessage response, Validation.IValidator validator) => new(response, validator); + /// /// Converts the into a where not and the content media type is . /// @@ -105,29 +114,63 @@ public static async Task ToResultAsync(this HttpResponseMessage response /// Where the response is not successful () and the content media type is , the content is /// converted into a and returned as the error; otherwise, an is returned as the error. /// Additionally, where the is considered a , it is returned as such. + /// Finally, where the response is successful, a is returned that includes the deserialized response value. Where the response content is empty or deserializes to , an is returned as the error; + /// use where a value is valid. + public static async Task> ToResultAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + { + var result = await ToResultOrDefaultAsync(response, jsonSerializerOptions, cancellationToken).ConfigureAwait(false); + if (result.IsFailure) + return result.Error; + + return result.Value is null + ? Result.Fail(new HttpRequestException($"The response content was empty or null; a value of type '{typeof(T).Name}' was expected.", null, response.StatusCode)) + : result.Value; + } + + /// + /// Converts the into a including the deserialized JSON response value where successful. + /// + /// The . + /// The . + /// The corresponding . + /// Where the response is not successful () and the content media type is , the content is + /// converted into a and returned as the error; otherwise, an is returned as the error. + /// Additionally, where the is considered a , it is returned as such. /// Finally, where the response is successful, a is returned that includes the deserialized response value. - public static async Task> ToResultAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions = null, CancellationToken cancellationToken = default) + public static Task> ToResultAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => ToResultAsync(response, null, cancellationToken); + + /// + /// Converts the into a including the deserialized JSON response value where successful; a successful response with empty or content results in a value. + /// + /// The response value . + /// The . + /// The optional . + /// The . + /// The corresponding . + /// Where the response is not successful the error handling is the same as . Use where a value is always expected. + public static async Task> ToResultOrDefaultAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) { // Where not successful, reuse the non-generic version above to get the error details. if (!response.ThrowIfNull().IsSuccessStatusCode) return await ToResultAsync(response, cancellationToken).ConfigureAwait(false); - // Where successful, attempt to read the content as JSON and return as the value. + // Where successful, attempt to read the content as JSON and return as the value; empty content (e.g. 204) is treated as no value as it is not valid JSON. + if (response.StatusCode == HttpStatusCode.NoContent || response.Content.Headers.ContentLength == 0) + return default; + var value = await response.Content.ReadFromJsonAsync(jsonSerializerOptions ?? JsonDefaults.SerializerOptions, cancellationToken).ConfigureAwait(false); return value; } /// - /// Converts the into a including the deserialized JSON response value where successful. + /// Converts the into a including the deserialized JSON response value where successful; a successful response with empty or content results in a value. /// + /// The response value . /// The . /// The . /// The corresponding . - /// Where the response is not successful () and the content media type is , the content is - /// converted into a and returned as the error; otherwise, an is returned as the error. - /// Additionally, where the is considered a , it is returned as such. - /// Finally, where the response is successful, a is returned that includes the deserialized response value. - public static Task> ToResultAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => ToResultAsync(response, null, cancellationToken); + /// See . + public static Task> ToResultOrDefaultAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => ToResultOrDefaultAsync(response, null, cancellationToken); /// /// Gets the deserialized JSON response value from the where successful; otherwise, will throw a corresponding exception. @@ -140,8 +183,8 @@ public static async Task ToResultAsync(this HttpResponseMessage response /// Where the response is not successful () and the content media type is , the content is /// converted into a and returned as the error; otherwise, an is returned as the error. /// Additionally, where the is considered a , it is thrown as such. - /// Finally, where the response is successful, the deserialized response value is returned. - public async static Task GetValueAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + /// Finally, where the response is successful, the deserialized response value is returned; an empty or response content is treated as an error. Use where a value is valid. + public async static Task GetValueAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) { var result = await response.ThrowIfNull().ToResultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false); return result.Value; @@ -157,6 +200,31 @@ public static async Task ToResultAsync(this HttpResponseMessage response /// Where the response is not successful () and the content media type is , the content is /// converted into a and returned as the error; otherwise, an is returned as the error. /// Additionally, where the is considered a , it is thrown as such. - /// Finally, where the response is successful, the deserialized response value is returned. - public static Task GetValueAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => GetValueAsync(response, null, cancellationToken); -} \ No newline at end of file + /// Finally, where the response is successful, the deserialized response value is returned; an empty or response content is treated as an error. Use where a value is valid. + public static Task GetValueAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => GetValueAsync(response, null, cancellationToken); + + /// + /// Gets the deserialized JSON response value from the where successful, or where the content is empty or ; otherwise, will throw a corresponding exception. + /// + /// The response value . + /// The . + /// The optional . + /// The . + /// The value where successful. + /// Where not successful the error handling is the same as . + public async static Task GetValueOrDefaultAsync(this HttpResponseMessage response, JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + { + var result = await response.ThrowIfNull().ToResultOrDefaultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false); + return result.Value; + } + + /// + /// Gets the deserialized JSON response value from the where successful, or where the content is empty or ; otherwise, will throw a corresponding exception. + /// + /// The response value . + /// The . + /// The . + /// The value where successful. + /// See . + public static Task GetValueOrDefaultAsync(this HttpResponseMessage response, CancellationToken cancellationToken = default) => GetValueOrDefaultAsync(response, null, cancellationToken); +} diff --git a/src/CoreEx/Extensions.IHybridCache.cs b/src/CoreEx/Extensions.IHybridCache.cs index 3c55e8681..1df17728b 100644 --- a/src/CoreEx/Extensions.IHybridCache.cs +++ b/src/CoreEx/Extensions.IHybridCache.cs @@ -94,6 +94,20 @@ public static async Task GetOrCreateAsync(this IHybridCache cache, Composi public static async Task GetOrCreateAsync(this IHybridCache cache, CompositeKey key, Func> factory, HybridCacheEntryOptions? options, CancellationToken cancellationToken = default) where T : IEntityKey => await cache.GetOrCreateByKeyAsync(cache.KeyProvider.GetEntityCacheKey(key), factory, options ?? HybridCacheEntryOptions.CreateFor(), cancellationToken).ConfigureAwait(false); + /// + /// Gets the cached value for the specified key using the to create and set only where successful, where not found. + /// + /// The cache value . + /// The . + /// The . + /// The function used to create the (e.g. a remote call) where not cached. + /// The optional . + /// The . + /// The cached (or created) value as a successful ; otherwise, the failure from the (which is never cached). + /// Uses the to determine the cache key. See for further details (including atomicity). + public static async Task> GetOrCreateWithResultAsync(this IHybridCache cache, CompositeKey key, Func>> factory, HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) where T : IEntityKey + => await cache.GetOrCreateByKeyWithResultAsync(cache.KeyProvider.GetEntityCacheKey(key), factory, options ?? HybridCacheEntryOptions.CreateFor(), cancellationToken).ConfigureAwait(false); + /// /// Removes the cached value for the specified key. /// diff --git a/src/CoreEx/Extensions.cs b/src/CoreEx/Extensions.cs index 3507ed5e3..0e39af008 100644 --- a/src/CoreEx/Extensions.cs +++ b/src/CoreEx/Extensions.cs @@ -178,10 +178,25 @@ public static T ThrowIfEqualToZero(this T value, string? message = null, [Cal public static string? ToSentenceCase(this string? text) => SentenceCase.ToSentenceCase(text); /// - /// Indicates whether the exception is (including ). + /// Indicates whether the exception is (including where wrapped by one or more nested levels). /// /// The . /// indicates canceled; otherwise, . + /// Nested wrapping occurs where a cancellation passes through multiple Result.ThrowOnError() boundaries, each of which wraps the error in a new . [DebuggerStepThrough] - public static bool IsCanceled(this Exception ex) => ex is OperationCanceledException || (ex is AggregateException aex && aex.InnerException is OperationCanceledException); -} \ No newline at end of file + public static bool IsCanceled(this Exception ex) => ex is OperationCanceledException || (ex is AggregateException aex && aex.InnerException is not null && aex.InnerException.IsCanceled()); + + /// + /// Indicates whether the exception is (including where wrapped by one or more nested levels) that is + /// attributable specifically to the . + /// + /// The . + /// The that the cancellation must be attributable to. + /// indicates canceled by the ; otherwise, . + /// A non-cancelable token (such as ) cannot identify the cancellation source and always returns . Otherwise, unlike + /// (which matches any cancellation regardless of source), this distinguishes an operation's own cancellation (e.g. a host shutdown or client disconnect, which + /// should typically bubble up unclassified) from an unrelated cancellation elsewhere in the call stack (e.g. an inner timeout using its own ). + [DebuggerStepThrough] + public static bool IsCanceledBy(this Exception ex, CancellationToken cancellationToken) + => cancellationToken.CanBeCanceled && ((ex is OperationCanceledException oce && oce.CancellationToken == cancellationToken) || (ex is AggregateException aex && aex.InnerException is not null && aex.InnerException.IsCanceledBy(cancellationToken))); +} diff --git a/src/CoreEx/Hosting/RetryResiliency.cs b/src/CoreEx/Hosting/RetryResiliency.cs index 17c90e84b..390ec3862 100644 --- a/src/CoreEx/Hosting/RetryResiliency.cs +++ b/src/CoreEx/Hosting/RetryResiliency.cs @@ -21,11 +21,32 @@ public static class RetryResiliency /// The strategy. /// A configured instance. /// The caller is responsible for flowing the owning instance into the via before - /// executing the pipeline. + /// executing the pipeline. + /// This is a convenience overload of for the common case where the underlying operation + /// has no value to carry through a retry (see ); where a value must be carried through (e.g. the outcome of a query), use with that overload instead. public static ResiliencePipeline Create(Func shouldHandle, Func logger, TimeSpan? delay = null, int maxRetryAttempts = 3, DelayBackoffType backoffType = DelayBackoffType.Exponential) + => Create(shouldHandle, logger, delay, maxRetryAttempts, backoffType); + + /// + /// Creates a standardized with retry capabilities for a caller-classified subset of failures, for any (e.g. + /// or ). + /// + /// The (typically or a closed ). + /// The predicate a failing must satisfy to be retried (e.g. a specific, known-transient exception type); a failure that does not satisfy this is never + /// retried and is allowed straight through. Unlike , there is no "retry everything" default - blindly retrying an unclassified failure risks retrying one + /// that retrying can never fix, so the caller must always specify what is worth retrying. + /// Accessor for the owning 's , used to log each retry attempt. + /// The delay between retry attempts. + /// The maximum number of retry attempts. + /// The strategy. + /// A configured instance. + /// The caller is responsible for flowing the owning instance into the via before + /// executing the pipeline. + public static ResiliencePipeline Create(Func shouldHandle, Func logger, TimeSpan? delay = null, int maxRetryAttempts = 3, DelayBackoffType backoffType = DelayBackoffType.Exponential) + where TResult : struct, IResult { - return new ResiliencePipelineBuilder() - .AddRetry(new RetryStrategyOptions() + return new ResiliencePipelineBuilder() + .AddRetry(new RetryStrategyOptions() { ShouldHandle = args => ValueTask.FromResult(args.Outcome.Result.IsFailure && shouldHandle(args.Outcome.Result)), Delay = delay ?? TimeSpan.FromSeconds(2), diff --git a/src/CoreEx/Hosting/TimerHostedServiceBase.cs b/src/CoreEx/Hosting/TimerHostedServiceBase.cs index e61b8bb60..ef30aab47 100644 --- a/src/CoreEx/Hosting/TimerHostedServiceBase.cs +++ b/src/CoreEx/Hosting/TimerHostedServiceBase.cs @@ -180,16 +180,28 @@ private async Task RunPeriodicAsync(CancellationToken cancellationToken) await _signal.WaitAsync(cancellationToken).ConfigureAwait(false); else { - _delayCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); - var delayTask = Task.Delay(nextInterval, _delayCts.Token); - var signalTask = _signal.WaitAsync(cancellationToken); - await Task.WhenAny(delayTask, signalTask).ConfigureAwait(false); - _delayCts?.Dispose(); - _delayCts = null; + // The signal wait shares the per-iteration token so that the losing wait is cancelled (and dequeued from the semaphore); otherwise, abandoned waiters accumulate and swallow subsequent signals. + var delayCts = CancellationTokenSource.CreateLinkedTokenSource(cancellationToken); + Volatile.Write(ref _delayCts, delayCts); + try + { + var delayTask = Task.Delay(nextInterval, delayCts.Token); + var signalTask = _signal.WaitAsync(delayCts.Token); + await Task.WhenAny(delayTask, signalTask).ConfigureAwait(false); + } + finally + { + Interlocked.CompareExchange(ref _delayCts, null, delayCts); + TryCancel(delayCts); + delayCts.Dispose(); + } } } catch (Exception ex) when (ex.IsCanceled()) { break; } + if (cancellationToken.IsCancellationRequested) + break; + lock (SyncLock) { // Where not currently sleeping, then we need to wait for a status change. @@ -456,8 +468,8 @@ protected sealed override async Task OnStopAsync(CancellationToken cancellationT { try { - _delayCts?.Cancel(); - _backgroundCts?.Cancel(); + TryCancel(Volatile.Read(ref _delayCts)); + TryCancel(_backgroundCts); } finally { @@ -543,9 +555,26 @@ private void OneOffIntervalAdjustInternal(TimeSpan oneOffInterval) /// private void SignalWakeUp() { - _delayCts?.Cancel(); - if (_signal.CurrentCount == 0) - _signal.Release(); + TryCancel(Volatile.Read(ref _delayCts)); + + try + { + if (_signal.CurrentCount == 0) + _signal.Release(); + } + catch (ObjectDisposedException) { } // Disposed; there is nothing left to wake. + } + + /// + /// Cancels the where not null, ignoring where already disposed (the background loop may concurrently complete and dispose it). + /// + private static void TryCancel(CancellationTokenSource? cts) + { + try + { + cts?.Cancel(); + } + catch (ObjectDisposedException) { } } /// @@ -553,19 +582,17 @@ protected override void Dispose(bool disposing) { if (disposing) { - try { _backgroundCts?.Cancel(); } catch { } - try { _delayCts?.Cancel(); } catch { } + TryCancel(_backgroundCts); + TryCancel(Interlocked.Exchange(ref _delayCts, null)); try { _signal?.Dispose(); - _delayCts?.Dispose(); _backgroundCts?.Dispose(); } catch { } finally { - _delayCts = null; _backgroundTask = null; _backgroundCts = null; } diff --git a/src/CoreEx/Http/README.md b/src/CoreEx/Http/README.md index a0602144c..a4b568658 100644 --- a/src/CoreEx/Http/README.md +++ b/src/CoreEx/Http/README.md @@ -25,10 +25,20 @@ The `Abstractions` sub-namespace defines additional lightweight types and interf | **[`ProblemDetails`](./Abstractions/ProblemDetails.cs)** | RFC 7807 problem-details record with `Type`, `Title`, `Status`, `Detail`, `Instance`, and an `Extensions` dictionary; includes `GetValidationErrors(string? key)`. | | **[`ProblemDetailsException`](./ProblemDetailsException.cs)** | Exception wrapping a `ProblemDetails`; provides `ToException()` to convert it back to a typed CoreEx semantic exception. | | **[`HttpNames`](./HttpNames.cs)** | Static class of configurable string properties for all standard paging query-string names, response-header names, and the idempotency-key header name. | +| **[`ValidatedHttpResponse`](./ValidatedHttpResponseT.cs)** | Thin wrapper returned by `HttpResponseMessage.WithValidator(validator)`; validates successful non-null response values when consumed via `ToResult*` or `GetValue*`. Does not own the response. | | **[`HttpResult`](./HttpResult.cs)** | Readonly struct pairing an `HttpResponseMessage` with either a deserialized value or a `ProblemDetails` from the response; exposes `IsSuccess`, `StatusCode`, and `ThrowOnError()`. | | **[`HttpResult`](./HttpResultT.cs)** | Typed variant of `HttpResult` adding a strongly-typed `Value` property for the deserialized response body. | | [`IExtendedHttpClient`](./Abstractions/IExtendedHttpClient.cs) | Interface implemented by `TypedHttpClientBase` for type-safe HTTP client wrappers; exposes `SendAsync`, `GetAsync`, `PostAsync`, `PutAsync`, `PatchAsync`, `DeleteAsync` returning `HttpResult`. | +## Validating response values + +Use `response.WithValidator(validator).ToResultAsync(ct)` for opt-in response-contract validation. +The wrapper returns the validation result's `Value`; reported failures contain an +`HttpRequestException` with inner validation diagnostics. `GetValueAsync(ct)` throws on failure. +Required methods reject null, while `OrDefault` methods allow null and skip validation for absent +values. HTTP status handling remains unchanged, and the caller retains response ownership. +See [the full usage guide and inline client example](../README.md#validating-http-response-values). + ## Related Namespaces - **[`CoreEx`](../README.md)** - Semantic exception types (`ValidationException`, `NotFoundException`, etc.) are the target for `ProblemDetailsException.ToException()` mapping. @@ -37,4 +47,4 @@ The `Abstractions` sub-namespace defines additional lightweight types and interf ## Additional Resources -- [RFC 7807 — Problem Details for HTTP APIs](https://tools.ietf.org/html/rfc7807) - The specification that `ProblemDetails` implements. \ No newline at end of file +- [RFC 7807 — Problem Details for HTTP APIs](https://tools.ietf.org/html/rfc7807) - The specification that `ProblemDetails` implements. diff --git a/src/CoreEx/Http/ValidatedHttpResponseT.cs b/src/CoreEx/Http/ValidatedHttpResponseT.cs new file mode 100644 index 000000000..6c0f8b399 --- /dev/null +++ b/src/CoreEx/Http/ValidatedHttpResponseT.cs @@ -0,0 +1,107 @@ +namespace CoreEx.Http; + +/// +/// Enables opt-in validation when consuming a deserialized HTTP response value. +/// +/// The response value type. +/// This wrapper does not own the response. Validation failures are dependency failures represented by +/// , not caller validation errors. Successful validation returns the validation result's value. +/// The response. +/// The response value validator. +public sealed class ValidatedHttpResponse(HttpResponseMessage response, Validation.IValidator validator) +{ + private readonly HttpResponseMessage _response = response.ThrowIfNull(); + private readonly Validation.IValidator _validator = validator.ThrowIfNull(); + + /// + /// Gets a result containing the validated value; missing content or a null validated value is a failure. + /// + /// The cancellation token. + /// The validated result. + public Task> ToResultAsync(CancellationToken cancellationToken = default) => ToResultAsync(null, cancellationToken); + + /// + /// Gets a result containing the validated value; missing content or a null validated value is a failure. + /// + /// The optional JSON serializer options. + /// The cancellation token. + /// The validated result. + public async Task> ToResultAsync(JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + { + var result = await _response.ToResultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false); + var validated = await ValidateAsync(result.AsResult(), true, cancellationToken).ConfigureAwait(false); + return validated.AsResult(); + } + + /// + /// Gets a result containing the validated value, allowing null; absent response values are not validated. + /// + /// The cancellation token. + /// The validated result. + public Task> ToResultOrDefaultAsync(CancellationToken cancellationToken = default) => ToResultOrDefaultAsync(null, cancellationToken); + + /// + /// Gets a result containing the validated value, allowing null; absent response values are not validated. + /// + /// The optional JSON serializer options. + /// The cancellation token. + /// The validated result. + public async Task> ToResultOrDefaultAsync(JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + { + var result = await _response.ToResultOrDefaultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false); + return await ValidateAsync(result, false, cancellationToken).ConfigureAwait(false); + } + + /// + /// Gets the validated value, throwing on failure, including missing content or a null validated value. + /// + /// The cancellation token. + /// The validated value. + public Task GetValueAsync(CancellationToken cancellationToken = default) => GetValueAsync(null, cancellationToken); + + /// + /// Gets the validated value, throwing on failure, including missing content or a null validated value. + /// + /// The optional JSON serializer options. + /// The cancellation token. + /// The validated value. + public async Task GetValueAsync(JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + => (await ToResultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false)).Value; + + /// + /// Gets the validated value, allowing null and throwing on failure; absent response values are not validated. + /// + /// The cancellation token. + /// The validated value or null. + public Task GetValueOrDefaultAsync(CancellationToken cancellationToken = default) => GetValueOrDefaultAsync(null, cancellationToken); + + /// + /// Gets the validated value, allowing null and throwing on failure; absent response values are not validated. + /// + /// The optional JSON serializer options. + /// The cancellation token. + /// The validated value or null. + public async Task GetValueOrDefaultAsync(JsonSerializerOptions? jsonSerializerOptions, CancellationToken cancellationToken = default) + => (await ToResultOrDefaultAsync(jsonSerializerOptions, cancellationToken).ConfigureAwait(false)).Value; + + /// + /// Validates a successfully deserialized non-null value and classifies reported errors as dependency failures. + /// + private async Task> ValidateAsync(Result result, bool required, CancellationToken cancellationToken) + { + if (result.IsFailure || result.Value is null) + return result; + + var validation = await _validator.ValidateAsync(result.Value, cancellationToken).ConfigureAwait(false); + if (validation.HasErrors) + { + var error = validation.ToException() ?? new ValidationException(validation.Messages ?? []); + return Result.Fail(new HttpRequestException($"The response value of type '{typeof(T).Name}' failed validation.", error, _response.StatusCode)); + } + + if (required && validation.Value is null) + return Result.Fail(new HttpRequestException($"Validation returned null; a response value of type '{typeof(T).Name}' was expected.", null, _response.StatusCode)); + + return Result.Ok(validation.Value); + } +} diff --git a/src/CoreEx/Json/IExtensionData.cs b/src/CoreEx/Json/IExtensionData.cs new file mode 100644 index 000000000..36483dd88 --- /dev/null +++ b/src/CoreEx/Json/IExtensionData.cs @@ -0,0 +1,14 @@ +namespace CoreEx.Json; + +/// +/// Enables an entity/model to carry (round-trip) any JSON properties it does not explicitly declare, typically via . +/// +/// Intended for replace-style persistence (e.g. a document store replace overwrites the whole document) so that data is never lost simply because the model is unaware of it. Values +/// deserialized into the bag are typically . +public interface IExtensionData +{ + /// + /// Gets or sets the extension data (the properties not explicitly declared by the implementing type). + /// + Dictionary? ExtensionData { get; set; } +} diff --git a/src/CoreEx/Json/JsonMergePatch.cs b/src/CoreEx/Json/JsonMergePatch.cs index d7bbb595a..82d8a3e26 100644 --- a/src/CoreEx/Json/JsonMergePatch.cs +++ b/src/CoreEx/Json/JsonMergePatch.cs @@ -52,9 +52,13 @@ public Result> Merge(BinaryData patch, T target) /// The . /// The . /// Provides the opportunity to validate the JSON before getting the value where this execution order is important; i.e. get operation is expensive (latency). A that returns - /// will immediately exit without performing any merge. + /// will return a not found result. public Task>> MergeAsync(BinaryData patch, Func> getTarget, CancellationToken cancellationToken = default) - => MergeWithResultAsync(patch, async ct => Result.Ok(await getTarget.ThrowIfNull()(ct).ConfigureAwait(false)), cancellationToken); + => MergeWithResultAsync(patch, async ct => + { + var v = await getTarget.ThrowIfNull()(ct).ConfigureAwait(false); + return v is null ? Result.NotFoundError() : Result.Ok(v); + }, cancellationToken); /// /// Merges the content into the value returned by the function. @@ -66,7 +70,7 @@ public Task>> MergeAsync(BinaryData patch, Fun /// The . /// Provides the opportunity to validate the JSON before getting the value where this execution order is important; i.e. get operation is expensive (latency). A that returns /// will immediately exit without performing any merge. - public async Task>> MergeWithResultAsync(BinaryData patch, Func>> getTarget, CancellationToken cancellationToken = default) + public async Task>> MergeWithResultAsync(BinaryData patch, Func>> getTarget, CancellationToken cancellationToken = default) { // Parse ensuring the JSON is valid for the type and can be navigated. if (!TryParseJson(patch.ThrowIfNull(), out var r)) @@ -75,7 +79,7 @@ public async Task>> MergeWithResultAsync(Binar // Get the value and exit where nothing to merge into. var target = await getTarget.ThrowIfNull().Invoke(cancellationToken).ConfigureAwait(false); if (target.IsFailure) - return target.Error; + return target.AsResult(); if (target.Value is null) return new JsonMergePatchResult(); @@ -230,11 +234,7 @@ private bool TryMerge(JsonElement patch, JsonElement target, Utf8JsonWriter writ case JsonValueKind.Array: // An array is always a replacement. patch.WriteTo(writer); -#if NET8_0 - if (patch.GetArrayLength() != target.GetArrayLength() || !DeepEquals(patch, target)) -#else - if (patch.GetArrayLength() != target.GetArrayLength() || !JsonElement.DeepEquals(patch, target)) -#endif + if (!DeepEquals(patch, target)) changed = true; break; @@ -242,11 +242,7 @@ private bool TryMerge(JsonElement patch, JsonElement target, Utf8JsonWriter writ default: // Accept merge as-is. patch.WriteTo(writer); -#if NET8_0 if (!DeepEquals(patch, target)) -#else - if (!JsonElement.DeepEquals(patch, target)) -#endif changed = true; break; @@ -323,29 +319,36 @@ private static bool TryGetPropertyWithComparer(JsonElement json, string property return false; } -#if NET8_0 /// - /// Provides a deep equals for two instances. + /// Provides a deep equals for two instances; property order is not significant, array order is, numbers are compared by value, and strings are compared after unescaping. /// /// The left . /// The right . + /// This is deliberately a single implementation for all target frameworks, as JsonElement.DeepEquals is not available prior to .NET 9 and JsonNode.DeepEquals compares numbers by their raw text on .NET 8 but by value from .NET 9. internal static bool DeepEquals(JsonElement left, JsonElement right) { if (left.ValueKind != right.ValueKind) - { return false; - } switch (left.ValueKind) { + case JsonValueKind.Undefined: case JsonValueKind.Null: case JsonValueKind.False: case JsonValueKind.True: // These are the same by kind, so carry on! return true; - case JsonValueKind.Number: case JsonValueKind.String: + return left.GetString() == right.GetString(); + + case JsonValueKind.Number: + if (left.TryGetDecimal(out var ld) && right.TryGetDecimal(out var rd)) + return ld == rd; + + if (left.TryGetDouble(out var ldbl) && right.TryGetDouble(out var rdbl)) + return ldbl == rdbl; + return left.GetRawText() == right.GetRawText(); case JsonValueKind.Array: @@ -365,21 +368,17 @@ internal static bool DeepEquals(JsonElement left, JsonElement right) default: foreach (var l in left.EnumerateObject()) { - if (!right.TryGetProperty(l.Name, out var r)) - { - if (!DeepEquals(l.Value, r)) - return false; - } + if (!right.TryGetProperty(l.Name, out var r) || !DeepEquals(l.Value, r)) + return false; } foreach (var r in right.EnumerateObject()) { - if (!left.TryGetProperty(r.Name, out var _)) + if (!left.TryGetProperty(r.Name, out _)) return false; } return true; } } -#endif } \ No newline at end of file diff --git a/src/CoreEx/Mapping/BiDirectionMapperT2.cs b/src/CoreEx/Mapping/BiDirectionMapperT2.cs index 4b642af99..729613e62 100644 --- a/src/CoreEx/Mapping/BiDirectionMapperT2.cs +++ b/src/CoreEx/Mapping/BiDirectionMapperT2.cs @@ -35,12 +35,12 @@ public BiDirectionMapper() protected abstract TDestination OnMap(TSource source); /// - /// Maps the () value to a new destination () value. + /// Maps the () value to a new destination () value. /// - /// The source value. + /// The source value. /// The destination value. /// This represents the right-to-left mapping direction. - protected abstract TSource OnMap(TDestination source); + protected abstract TSource OnMap(TDestination destination); /// /// Provides the underlying to mapping. @@ -61,6 +61,6 @@ public sealed class DestinationToSourceMapper(Func map) : private readonly Func _map = map; /// - protected override TSource OnMap(TDestination source) => _map(source); + protected override TSource OnMap(TDestination destination) => _map(destination); } } diff --git a/src/CoreEx/Mapping/BiDirectionMapperT3.cs b/src/CoreEx/Mapping/BiDirectionMapperT3.cs index 363be01e9..e05c7f65f 100644 --- a/src/CoreEx/Mapping/BiDirectionMapperT3.cs +++ b/src/CoreEx/Mapping/BiDirectionMapperT3.cs @@ -49,12 +49,12 @@ static BiDirectionMapper() protected abstract TDestination OnMap(TSource source); /// - /// Maps the () value to a new destination () value. + /// Maps the () value to a new destination () value. /// - /// The source value. - /// The destination value. + /// The destination value. + /// The source value. /// This represents the right-to-left mapping direction. - protected abstract TSource OnMap(TDestination source); + protected abstract TSource OnMap(TDestination destination); /// /// Provides the underlying to mapping. @@ -75,6 +75,6 @@ public sealed class DestinationToSourceMapper(Func map) : private readonly Func _map = map; /// - protected override TSource OnMap(TDestination source) => _map(source); + protected override TSource OnMap(TDestination destination) => _map(destination); } -} \ No newline at end of file +} diff --git a/src/CoreEx/Metadata/RuntimeMetadata.AreEqual.cs b/src/CoreEx/Metadata/RuntimeMetadata.AreEqual.cs index c9069814a..1e05e9303 100644 --- a/src/CoreEx/Metadata/RuntimeMetadata.AreEqual.cs +++ b/src/CoreEx/Metadata/RuntimeMetadata.AreEqual.cs @@ -19,8 +19,25 @@ public static partial class RuntimeMetadata /// are performed: comparison, comparison, per item and comparisons, /// item comparisons, and nested and comparisons. This is to achieve a best attempt deep-equals where a contract-style /// class (such as a ) constrains itself to simple and known types such as those described above, and/or overrides accordingly. - public static bool AreEqual(T? left, T? right) + public static bool AreEqual(T? left, T? right) => AreEqual(left, right, null); + + /// + /// Compare two values for equality, ignoring the specified top-level (root) properties. + /// + /// The value . + /// The left-side value. + /// The right-side value. + /// The names of the root-level properties of / to exclude from the comparison; indicates that none are excluded. + /// indicates they are equal; otherwise, . + /// Exclusion applies to the top-level properties only (no paths); nested values are always compared in full. Where properties are excluded the fast-path is skipped as + /// it cannot honor the exclusions. See for the comparison behavior. + public static bool AreEqual(T? left, T? right, IEnumerable? excludedProperties) { + // Materialized once (unless already a collection) so a lazy sequence is not re-enumerated per property; empty is treated as none. + var excluded = excludedProperties as ICollection ?? excludedProperties?.ToArray(); + if (excluded is { Count: 0 }) + excluded = null; + if (ReferenceEquals(left, right)) return true; if (left is null || right is null) return false; @@ -43,7 +60,16 @@ public static bool AreEqual(T? left, T? right) var epr = ((IRuntimeMetadataCore)right).GetPropertyRuntimeMetadata().GetEnumerator(); while (epl.MoveNext()) { - if (!epr.MoveNext() || !AreEqual(epl.Current.GetValue(left), epr.Current.GetValue(right))) + if (!epr.MoveNext()) + { + set.Remove(pair); + return false; + } + + if (excluded is not null && excluded.Contains(epl.Current.Name)) + continue; + + if (!AreEqual(epl.Current.GetValue(left), epr.Current.GetValue(right))) { set.Remove(pair); return false; @@ -61,7 +87,7 @@ public static bool AreEqual(T? left, T? right) } // Fast-path explicit equality implementation. - if (left is IEquatable leq) + if (excluded is null && left is IEquatable leq) return leq.Equals(right); // Short circuit arrays, collections, lists, and dictionaries based on count difference. @@ -80,11 +106,7 @@ public static bool AreEqual(T? left, T? right) // Special handling for JsonElement comparison. if (left is JsonElement lje && right is JsonElement rje) -#if NET8_0 return CoreEx.Json.JsonMergePatch.DeepEquals(lje, rje); -#else - return JsonElement.DeepEquals(lje, rje); -#endif // Default value type comparison. var type = left.GetType(); @@ -103,6 +125,9 @@ public static bool AreEqual(T? left, T? right) foreach (var p in GetCachedProperties(type).Values) { + if (excluded is not null && excluded.Contains(p.Name)) + continue; + if (!AreEqual(p.GetValue(left), p.GetValue(right))) { set.Remove(pair); diff --git a/src/CoreEx/Metadata/RuntimeMetadata.CopyInto.cs b/src/CoreEx/Metadata/RuntimeMetadata.CopyInto.cs index ed7148f66..297623515 100644 --- a/src/CoreEx/Metadata/RuntimeMetadata.CopyInto.cs +++ b/src/CoreEx/Metadata/RuntimeMetadata.CopyInto.cs @@ -64,7 +64,19 @@ public static void CopyInto(TFrom from, TInto into) where TFrom : /// This method ignores and values, and the value must be the same type or is same assignable/subclass as the value. /// In these instances no copying can or will be performed; i.e. is a no-op. /// This will leverage either the underlying implementation or reflection () depending on the types. - public static bool TryCopyInto(TFrom from, TInto into) where TFrom : class where TInto : class + public static bool TryCopyInto(TFrom from, TInto into) where TFrom : class where TInto : class => TryCopyInto(from, into, null); + + /// + /// Copies (shallow) a value another value where they share mutable properties, excluding the specified properties, and returns a value indicating whether changes where made. + /// + /// The from . + /// The into . + /// The from value. + /// The into value. + /// The names of the properties to exclude from the copy (and therefore from the change detection); indicates that none are excluded. + /// true where changes were made; otherwise, false. + /// See for the copy semantics. + public static bool TryCopyInto(TFrom from, TInto into, IReadOnlyCollection? excludedProperties) where TFrom : class where TInto : class { from.ThrowIfNull(); into.ThrowIfNull(); @@ -84,6 +96,9 @@ public static bool TryCopyInto(TFrom from, TInto into) where TFrom { foreach (var fp in frm.GetPropertyRuntimeMetadata().Where(p => !p.IsReadOnly)) { + if (excludedProperties is not null && excludedProperties.Contains(fp.Name)) + continue; + if (dict.TryGetValue(fp.Name, out var im) && !im.IsReadOnly && fp.Type == im.Type) { var gv = fp.GetValue(from); @@ -102,6 +117,9 @@ public static bool TryCopyInto(TFrom from, TInto into) where TFrom { foreach (var fp in GetCachedProperties().Where(p => !p.Value.IsReadOnly)) { + if (excludedProperties is not null && excludedProperties.Contains(fp.Value.Name)) + continue; + if (dict.TryGetValue(fp.Value.Name, out var im) && !im.IsReadOnly && fp.Value.Type == im.Type) { var gv = fp.Value.GetValue(from); diff --git a/src/CoreEx/README.md b/src/CoreEx/README.md index 0872c4b44..d9328da39 100644 --- a/src/CoreEx/README.md +++ b/src/CoreEx/README.md @@ -49,6 +49,57 @@ All other CoreEx packages (`CoreEx.Events`, `CoreEx.Database`, `CoreEx.AspNetCor See the [Extended Exceptions](#extended-exceptions) section below for the full list of semantic exception types. +## Validating HTTP response values + +HTTP success and JSON deserialization do not guarantee a valid dependency response. Use +`WithValidator` with a `CoreEx.Validation.IValidator` to opt into response-contract validation: + +```csharp +using var response = await _httpClient.GetAsync($"api/customers/{Uri.EscapeDataString(id)}", ct).ConfigureAwait(false); +return await response.WithValidator(customerValidator).ToResultAsync(ct).ConfigureAwait(false); +``` + +The [typed wrapper](./Http/ValidatedHttpResponseT.cs) reuses the existing response converters and +validates only successful, non-null deserialized values. It returns the **validation result's +Value**, including replacements or normalization. `ToResultAsync` requires a non-null value both +before and after validation; `ToResultOrDefaultAsync` allows null and skips validation when the +response has no value. `GetValueAsync` and `GetValueOrDefaultAsync` provide the throwing equivalents. +All four methods accept optional JSON serializer options and a cancellation token. + +Reported validation errors become `HttpRequestException` failures with the validation exception +as the inner exception, preserving structured messages. `ToResult*` returns these failures; +`GetValue*` throws them. Under standard CoreEx API exception handling, these are internal **500** +errors, not caller **400** validation errors. The exception's upstream status is diagnostic context, +not the status returned to the caller. Outer messages do not contain the response body. +Existing HTTP/ProblemDetails handling is unchanged; cancellation, deserialization exceptions, +and unexpected exceptions thrown by a validator still propagate. + +Use a response-specific validator rather than assuming create/update rules apply to read responses. +For small client-only rules, prefer a private static readonly `Validator` configured once with +`Validator.Create().HasProperty(...)` from the `CoreEx.Validation` package inside the Infrastructure client: + +```csharp +private static readonly Validator _validator = Validator.Create() + .HasProperty(x => x.Email, c => c.Mandatory().Email()); + +// Inside the asynchronous HTTP client method. +using var response = await _httpClient.GetAsync($"api/customers/{Uri.EscapeDataString(id)}", ct).ConfigureAwait(false); +return await response.WithValidator(_validator).ToResultAsync(ct).ConfigureAwait(false); +``` + +Extract a colocated validator only for substantial rules or genuine reuse. Validate consumed +fields rather than requiring every DTO property or matching identifiers by default. +Configure shared rules once without captured request state or scoped dependencies, and test inline +rules through client unit tests rather than exposing them solely for testing. Invalid responses +must not be cached. See the +[Customers client sample](../../samples/src/Contoso.Shopping.Infrastructure/Clients/Customers/CustomersHttpClient.cs) +for a complete implementation. +Validators can normalize values or perform I/O, so consider their effects explicitly. The wrapper +does not own or dispose the response, cache validation results, or support concurrent response +consumption. Keep responsibility for disposing the response at the call site. +Like the underlying converters, a terminal operation consumes the content stream; creating a +wrapper does not buffer the body for repeated reads. + ## Errors vs Exceptions `CoreEx` distinguishes between expected and unexpected errors: diff --git a/src/CoreEx/RefData/IReferenceDataCache.cs b/src/CoreEx/RefData/IReferenceDataCache.cs index 7efec8a70..a068d1c65 100644 --- a/src/CoreEx/RefData/IReferenceDataCache.cs +++ b/src/CoreEx/RefData/IReferenceDataCache.cs @@ -13,4 +13,11 @@ public interface IReferenceDataCache /// The . /// The from the cache. Task GetOrCreateAsync(Type type, Func> factory, CancellationToken cancellationToken = default); -} \ No newline at end of file + + /// + /// Removes the for the specified from the cache. + /// + /// The . + /// The . + Task RemoveAsync(Type type, CancellationToken cancellationToken = default); +} diff --git a/src/CoreEx/RefData/ReferenceDataOrchestrator.cs b/src/CoreEx/RefData/ReferenceDataOrchestrator.cs index 6a629e564..267615f21 100644 --- a/src/CoreEx/RefData/ReferenceDataOrchestrator.cs +++ b/src/CoreEx/RefData/ReferenceDataOrchestrator.cs @@ -290,7 +290,7 @@ public ReferenceDataOrchestrator Register() where TProvider : IRefere { return await GetByTypeInNewScopeAsync(rdo, ec, scope, type, providerType, cancellationToken).ConfigureAwait(false); } - catch (Exception ex) when (Logger?.IsEnabled(LogLevel.Error) == true) + catch (Exception ex) when (!(ex.IsCanceled() && cancellationToken.IsCancellationRequested) && Logger?.IsEnabled(LogLevel.Error) == true) { Logger.LogError(ex, "Reference data type {RefDataType} cache load failed in worker task: {ex.Message}", type.FullName, ex.Message); throw; // Re-throw to propagate @@ -391,6 +391,68 @@ public IReferenceDataCollection GetByNameRequired(string name) public Task GetByNameRequiredAsync(string name, CancellationToken cancellationToken = default) => _nameToType.TryGetValue(name.ThrowIfNull(), out var type) ? GetByTypeRequiredAsync(type, cancellationToken) : throw new InvalidOperationException($"Reference data collection for name '{name}' does not exist."); + /// + /// Invalidates the cached for the specified . + /// + /// The . + /// The . + /// A representing the asynchronous operation. + /// This will result in the cached being removed, and then re-cached on next access. + public Task InvalidateAsync(CancellationToken cancellationToken = default) where TRef : IReferenceData => InvalidateAsync(typeof(TRef), cancellationToken); + + /// + /// Invalidates the cached for the specified . + /// + /// The . + /// The . + /// This will result in the cached being removed, and then re-cached on next access. + public async Task InvalidateAsync(Type type, CancellationToken cancellationToken = default) + { + if (!_typeToProvider.TryGetValue(type.ThrowIfNull(), out var providerType)) + return; + + if (!ExecutionContext.HasCurrent) + throw new InvalidOperationException($"The {nameof(ReferenceDataOrchestrator)} requires an active {nameof(ExecutionContext)} to support underlying scoped service resolution."); + + // Get the underlying scoped cache. + var cache = ExecutionContext.GetRequiredService(); + + // The cache is keyed by the reference data collection type (see GetByTypeAsync), so the corresponding collection type is required for the removal. + await cache.RemoveAsync(_typeToCollType[type], cancellationToken).ConfigureAwait(false); + } + + /// + /// Attempts a best-effort invalidation of the cached for the specified . + /// + /// The . + /// The . + /// where the invalidation succeeded (or there was nothing to invalidate); otherwise, where it failed. + /// See . + public Task TryInvalidateAsync(CancellationToken cancellationToken = default) where TRef : IReferenceData => TryInvalidateAsync(typeof(TRef), cancellationToken); + + /// + /// Attempts a best-effort invalidation of the cached for the specified . + /// + /// The . + /// The . + /// where the invalidation succeeded (or there was nothing to invalidate); otherwise, where it failed. + /// Unlike , any failure (including cancellation) is logged as a warning and swallowed; the invalidation is still awaited so that read-after-write is preserved. + /// This is intended to be invoked after the underlying data change has been committed, where the change is permanent and a failure to evict must not be surfaced as a failure of the operation; the cached + /// collection will then remain stale until it expires. Do not invoke within the transaction as a concurrent reader could re-cache the pre-commit data. + public async Task TryInvalidateAsync(Type type, CancellationToken cancellationToken = default) + { + try + { + await InvalidateAsync(type, cancellationToken).ConfigureAwait(false); + return true; + } + catch (Exception ex) + { + Logger.LogWarning(ex, "Reference data type {RefDataType} cache invalidation failed; the cached collection will remain stale until it expires: {Message}", type?.FullName, ex.Message); + return false; + } + } + /// /// Gets the dictionary of the single definitive external name for every registered . /// diff --git a/src/CoreEx/Runtime.cs b/src/CoreEx/Runtime.cs index 8230052e4..f8e87548a 100644 --- a/src/CoreEx/Runtime.cs +++ b/src/CoreEx/Runtime.cs @@ -23,4 +23,19 @@ public static class Runtime /// /// Note: It is recommended that the method is used for assigning identifiers to entities. public static string NewId() => IdentifierGenerator.Current.GenerateGuid().ToString(); -} \ No newline at end of file + + /// + /// Gets a new identifier value of the specified type using the . + /// + /// The identifier . + /// The newly generated identifier. + public static Task GenerateIdentifierAsync() => IdentifierGenerator.Current.GenerateIdentifierAsync(); + + /// + /// Gets a new identifier value of the specified type for the specified entity type using the . + /// + /// The identifier . + /// The entity . + /// The newly generated identifier. + public static Task GenerateIdentifierAsync() where TFor : class => IdentifierGenerator.Current.GenerateIdentifierAsync(); +} diff --git a/tests/CoreEx.AspNetCore.Test.Api/Controllers/OtherController.cs b/tests/CoreEx.AspNetCore.Test.Api/Controllers/OtherController.cs index 6bdf237ef..873f1c4eb 100644 --- a/tests/CoreEx.AspNetCore.Test.Api/Controllers/OtherController.cs +++ b/tests/CoreEx.AspNetCore.Test.Api/Controllers/OtherController.cs @@ -12,6 +12,11 @@ public class OtherController(WebApi webApi) : ControllerBase [ProducesResponseType(204)] public IActionResult OptionalBody() => NoContent(); + [HttpPost("non-nullable-map")] + [Accepts>] + [ProducesResponseType(204)] + public IActionResult NonNullableMap() => NoContent(); + [HttpGet("messages")] [ProducesResponseType(204)] public IActionResult Messages() diff --git a/tests/CoreEx.AspNetCore.Test.Api/Entities/NonNullableMapValue.cs b/tests/CoreEx.AspNetCore.Test.Api/Entities/NonNullableMapValue.cs new file mode 100644 index 000000000..03912d2a6 --- /dev/null +++ b/tests/CoreEx.AspNetCore.Test.Api/Entities/NonNullableMapValue.cs @@ -0,0 +1,23 @@ +namespace CoreEx.AspNetCore.Test.Api.Entities; + +/// +/// Provides a recursive dictionary value for OpenAPI non-nullability regression tests. +/// +public class NonNullableMapValue +{ + /// + /// Gets or sets the code advertised as non-nullable. + /// + [CoreEx.Entities.NonNullable] + public string? RequiredCode { get; set; } + + /// + /// Gets or sets the code advertised as nullable. + /// + public string? OptionalCode { get; set; } + + /// + /// Gets or sets nested values of the same type. + /// + public CoreEx.Entities.DataMap? Children { get; set; } +} diff --git a/tests/CoreEx.AspNetCore.Test.Api/Entities/Person.cs b/tests/CoreEx.AspNetCore.Test.Api/Entities/Person.cs index f96efdc4a..4f6b84ac6 100644 --- a/tests/CoreEx.AspNetCore.Test.Api/Entities/Person.cs +++ b/tests/CoreEx.AspNetCore.Test.Api/Entities/Person.cs @@ -8,10 +8,12 @@ public partial class Person : IIdentifier, IETag, IChangeLog { public string? Id { get; set; } + [NonNullable] public string? FirstName { get; set; } public string? LastName { get; set; } + [NonNullable] public DateOnly? Birthday { get; set; } [ReferenceData] diff --git a/tests/CoreEx.AspNetCore.Test.Api/Services/PersonService2.cs b/tests/CoreEx.AspNetCore.Test.Api/Services/PersonService2.cs index f0d4e5514..830623e31 100644 --- a/tests/CoreEx.AspNetCore.Test.Api/Services/PersonService2.cs +++ b/tests/CoreEx.AspNetCore.Test.Api/Services/PersonService2.cs @@ -53,7 +53,7 @@ public class PersonService2 } }; - public Task> GetAsync(string id) => Task.FromResult(Result.Ok(_people.TryGetValue(id, out var person) ? person : null)); + public Task> GetAsync(string id) => Task.FromResult(_people.TryGetValue(id, out var person) ? Result.Ok(person) : Result.NotFoundError()); public Task> CreateAsync(Person person) { diff --git a/tests/CoreEx.AspNetCore.Test.Unit/OtherApiTests.cs b/tests/CoreEx.AspNetCore.Test.Unit/OtherApiTests.cs index 57a6bdd36..7d47c5f9c 100644 --- a/tests/CoreEx.AspNetCore.Test.Unit/OtherApiTests.cs +++ b/tests/CoreEx.AspNetCore.Test.Unit/OtherApiTests.cs @@ -128,6 +128,38 @@ public void Swagger_JSON_AcceptsAttribute_RequestBodyRequired() optionalBody.TryGetProperty("required", out _).Should().BeFalse(); } + [Test] + public void Swagger_JSON_NonNullableAttribute_NotNullable() + { + // Properties marked [NonNullable] are non-nullable in the spec even though declared nullable (string? / DateOnly?); unmarked ones remain nullable. + var json = Test.Http() + .Run(HttpMethod.Get, "/swagger/v1/swagger.json") + .Assert(HttpStatusCode.OK) + .GetContent(); + + using var doc = JsonDocument.Parse(json!); + var props = doc.RootElement.GetProperty("components").GetProperty("schemas").GetProperty("Person").GetProperty("properties"); + + props.GetProperty("firstName").GetProperty("nullable").GetBoolean().Should().BeFalse(); + props.GetProperty("birthday").GetProperty("nullable").GetBoolean().Should().BeFalse(); + props.GetProperty("lastName").GetProperty("nullable").GetBoolean().Should().BeTrue(); + } + + [Test] + public void Swagger_JSON_NonNullableAttribute_DataMapValue_NotNullable() + { + var json = Test.Http() + .Run(HttpMethod.Get, "/swagger/v1/swagger.json") + .Assert(HttpStatusCode.OK) + .GetContent(); + + using var doc = JsonDocument.Parse(json!); + var props = doc.RootElement.GetProperty("components").GetProperty("schemas").GetProperty("NonNullableMapValue").GetProperty("properties"); + + props.GetProperty("requiredCode").GetProperty("nullable").GetBoolean().Should().BeFalse(); + props.GetProperty("optionalCode").GetProperty("nullable").GetBoolean().Should().BeTrue(); + } + [TestCase("/health/live")] [TestCase("/health/startup")] [TestCase("/health/ready")] diff --git a/tests/CoreEx.AspNetCore.Test.Unit/WebApiTestsBase.MergePatchWithResult.cs b/tests/CoreEx.AspNetCore.Test.Unit/WebApiTestsBase.MergePatchWithResult.cs index e71417078..93ffafb72 100644 --- a/tests/CoreEx.AspNetCore.Test.Unit/WebApiTestsBase.MergePatchWithResult.cs +++ b/tests/CoreEx.AspNetCore.Test.Unit/WebApiTestsBase.MergePatchWithResult.cs @@ -41,7 +41,7 @@ public void MergePatchWithResult_Invalid_Json() public void MergePatchWithResult_Not_Found() { Test.Type() - .Run(async w => await w.PatchWithResultAsync(Test.CreateHttpRequest(HttpMethod.Patch, "test", "{}", r => r.ContentType = HttpNames.MergePatchJsonMediaTypeName), (ro, ct) => Task.FromResult(Result.Ok(null)), (ro, ct) => throw new InvalidOperationException())) + .Run(async w => await w.PatchWithResultAsync(Test.CreateHttpRequest(HttpMethod.Patch, "test", "{}", r => r.ContentType = HttpNames.MergePatchJsonMediaTypeName), (ro, ct) => Task.FromResult(Result.Ok(null!)), (ro, ct) => throw new InvalidOperationException())) .ToHttpResponseMessageAssertor() .AssertNotFound(); } @@ -53,7 +53,7 @@ public void MergePatchWithResult_Concurrency() hr.Headers.IfMatch = new EntityTagHeaderValue("\"123456\"", true).ToString(); Test.Type() - .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdefg"))), (ro, ct) => throw new InvalidOperationException())) + .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdefg"))), (ro, ct) => throw new InvalidOperationException())) .ToHttpResponseMessageAssertor() .AssertPreconditionFailed(); } @@ -66,7 +66,7 @@ public void MergePatchWithResult_No_Changes() // As there are no changes as a result of merging then the corresponding put operation should not be invoked; should just return as-if it had done something :-) Test.Type() - .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdef"))), (ro, ct) => throw new InvalidOperationException())) + .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdef"))), (ro, ct) => throw new InvalidOperationException())) .ToHttpResponseMessageAssertor() .AssertOK() .AssertValue(Person.GetPerson("abcdef")); @@ -82,7 +82,7 @@ public void MergePatchWithResult_With_Changes() ep.Age = 40; Test.Type() - .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdef"))), (ro, ct) => Task.FromResult(Result.Ok(ro.Value.Adjust(p => p.ETag = "qrstuv"))))) + .Run(async w => await w.PatchWithResultAsync(hr, (ro, ct) => Task.FromResult(Result.Ok(Person.GetPerson("abcdef"))), (ro, ct) => Task.FromResult(Result.Ok(ro.Value.Adjust(p => p.ETag = "qrstuv"))))) .ToHttpResponseMessageAssertor() .AssertOK() .AssertValue(ep, "etag"); diff --git a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusPublisherTests.cs b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusPublisherTests.cs index 13d73d51a..84a7648a9 100644 --- a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusPublisherTests.cs +++ b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusPublisherTests.cs @@ -1,5 +1,7 @@ using Azure.Messaging.ServiceBus; +using CoreEx.Azure.Messaging.ServiceBus; using CoreEx.Events.Publishing; +using Microsoft.Extensions.Configuration; using Microsoft.Extensions.DependencyInjection; using Microsoft.Extensions.Logging; using UnitTestEx.Expectations; @@ -8,6 +10,38 @@ namespace CoreEx.Azure.Messaging.ServiceBus.Test.Unit; public class ServiceBusPublisherTests : WithGenericTester { + [Test] + public void AddAzureServiceBusPublisher_UsesIndependentSessionPartitionSize() + { + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection(new Dictionary + { + ["CoreEx:Host:Outbox:PartitionSize"] = "2", + [ServiceBusPublisher.SessionIdPartitionSizeConfigurationKey] = "8", + ["CoreEx:Host:ServiceBus:Destinations:contoso:SessionIdStrategy"] = "UsePartitionKeyAsIs", + ["CoreEx:Host:ServiceBus:Destinations:contoso-products:SessionIdStrategy"] = "UsePartitionKeyConvertedToAnId", + ["CoreEx:Host:ServiceBus:Destinations:contoso-products:SessionIdPartitionSize"] = "2" + }) + .Build(); + var services = new ServiceCollection(); + services.AddSingleton(configuration); + services.AddLogging(); + services.AddSingleton(new ServiceBusClient("Endpoint=sb://localhost/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")); + services.AddAzureServiceBusPublisher((_, publisher) => + { + publisher.SessionIdStrategy = ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId; + publisher.DestinationSettings["contoso"] = new() { SessionIdPartitionSize = 4 }; + }); + using var provider = services.BuildServiceProvider(); + + var publisher = (ServiceBusPublisher)provider.GetRequiredKeyedService(ServiceBusPublisher.DefaultServiceKey); + publisher.SessionIdPartitionSize.Should().Be(8); + publisher.DestinationSettings["contoso"].SessionIdStrategy.Should().Be(ServiceBusSessionStrategy.UsePartitionKeyAsIs); + publisher.DestinationSettings["contoso"].SessionIdPartitionSize.Should().Be(4); + publisher.DestinationSettings["contoso-products"].SessionIdStrategy.Should().Be(ServiceBusSessionStrategy.UsePartitionKeyConvertedToAnId); + publisher.DestinationSettings["contoso-products"].SessionIdPartitionSize.Should().Be(2); + } + [Test] public void PublishAsync_SingleBatchOfOne() => Test.ScopedType(test => { @@ -101,4 +135,4 @@ public void PublishAsync_UseSessions_UsePartitionKeyConvertedToAnId_NoPartitionK await sbp.PublishAsync(); }).AssertSuccess(); }); -} \ No newline at end of file +} diff --git a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusReceiverTests.cs b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusReceiverTests.cs index 2ab3eddae..37127b790 100644 --- a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusReceiverTests.cs +++ b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/ServiceBusReceiverTests.cs @@ -162,40 +162,87 @@ public void WithKeyedSubscriber_WithHostedService_ResolvesReceiver() => Test.Sco }); [Test] - public void ReceiveAsync_OwnTokenCancellation_DoesNotLogAsUnhandled() => Test.ScopedType(async test => + public void WithReceiver_MultipleKeyedReceivers_ResolveIndependently() => Test.ScopedType(async test => { - // Regression: a cancellation attributable to the receiver's own cancellationToken (simulating a host/processor - // shutdown while a message is in flight) must not be logged as "An unhandled error has occurred" and must not - // throw attempting to abandon the message with the already-cancelled token. - var sp = (ServiceBusPublisher)test.Services.GetRequiredKeyedService(ServiceBusPublisher.DefaultServiceKey); - sp.Add(EventData.CreateEventWith(new Subscribers.Product { Id = 200, Sku = "SKU-200" }, "Created")); - await sp.PublishAsync(); + // Two receivers of the same receiver type (i.e. a topic subscription and a queue) must not collide; each is registered under its own receiver key. + var services = new ServiceCollection(); + services.AddSingleton(test.Services.GetRequiredService()); + services.AddSingleton(test.Services.GetRequiredService()); + services.AddLogging(); - var o = ServiceBusReceiverOptions.CreateForTopicSubscription("unit-test", "default"); - var sbr = ActivatorUtilities.CreateInstance>(Test.Services, o); + services.AzureServiceBusReceiving() + .WithReceiver(_ => ServiceBusReceiverOptions.CreateForTopicSubscription("unit-test", "default"), "receiver-topic") + .WithKeyedSubscribedSubscriber("subscriber-topic") + .WithHostedService("hosted-topic") + .Build(); - var cts = new CancellationTokenSource(); - cts.CancelAfter(10000); // Ensure test doesn't run indefinitely. - sbr.MessageProcessed += (sender, e) => { }; // No-op: Id 200 blocks on the token itself and is only "processed" once cancellation propagates. + services.AzureServiceBusReceiving() + .WithReceiver(_ => ServiceBusReceiverOptions.CreateForQueue("unit-test-queue"), "receiver-queue") + .WithKeyedSubscribedSubscriber("subscriber-queue") + .WithHostedService("hosted-queue") + .Build(); - var assertor = Test.ExpectLogContains("Received product with Id: 200 and Sku: SKU-200.") - .Run(async () => - { - try - { - await sbr.StartAsync(cts.Token).ConfigureAwait(false); - await Task.Delay(200, cts.Token).ConfigureAwait(false); // Allow the message to be received and start processing (blocking on cts.Token). - cts.Cancel(); // Simulate host/processor shutdown while the message is in flight. - await Task.Delay(Timeout.Infinite, cts.Token).ConfigureAwait(false); - } - finally - { - await sbr.StopAsync().ConfigureAwait(false); - await sbr.DisposeAsync().ConfigureAwait(false); - } - }).AssertException(); + await using var sp = services.BuildServiceProvider(); + + var topic = sp.GetRequiredKeyedService>("receiver-topic"); + var queue = sp.GetRequiredKeyedService>("receiver-queue"); + + topic.Should().NotBeSameAs(queue); + topic.Options.QueueOrTopicName.Should().Be("unit-test"); + topic.Options.SubscriptionName.Should().Be("default"); + topic.Options.SubscriberServiceKey.Should().Be("subscriber-topic"); + queue.Options.QueueOrTopicName.Should().Be("unit-test-queue"); + queue.Options.IsSubscription.Should().BeFalse(); + queue.Options.SubscriberServiceKey.Should().Be("subscriber-queue"); + + sp.GetServices().OfType>>().Should().HaveCount(2); + }); + + [Test] + public void ReceiveAsync_OwnTokenCancellation_DoesNotLogAsUnhandled() => Test.ScopedType(async test => + { + // An isolated subscription excludes poison messages abandoned by the circuit-breaker test. + var configuration = test.Services.GetRequiredService(); + var connectionString = configuration.GetConnectionString("ServiceBus") ?? configuration["Aspire:Azure:Messaging:ServiceBus:ConnectionString"] + ?? throw new InvalidOperationException("The Service Bus connection string is missing."); + var admin = new global::Azure.Messaging.ServiceBus.Administration.ServiceBusAdministrationClient(UnitTestExExtensions.CreateAzureServiceBusAdminConnectionString(connectionString)); + var subscription = $"cancellation-{Guid.NewGuid():N}"; + await admin.CreateSubscriptionAsync("unit-test", subscription).ConfigureAwait(false); + try + { + var processing = Subscribers.ProductSubscriber.Id200Processing = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var sp = (ServiceBusPublisher)test.Services.GetRequiredKeyedService(ServiceBusPublisher.DefaultServiceKey); + sp.Add(EventData.CreateEventWith(new Subscribers.Product { Id = 200, Sku = "SKU-200" }, "Created")); + await sp.PublishAsync().ConfigureAwait(false); - assertor.LogMessages.Any(x => x?.Contains("An unhandled error has occurred") == true).Should().BeFalse(); + var o = ServiceBusReceiverOptions.CreateForTopicSubscription("unit-test", subscription); + var sbr = ActivatorUtilities.CreateInstance>(Test.Services, o); + using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(30)); + + var assertor = Test.ExpectLogContains("Received product with Id: 200 and Sku: SKU-200.") + .Run(async () => + { + try + { + await sbr.StartAsync(cts.Token).ConfigureAwait(false); + await processing.Task.WaitAsync(cts.Token).ConfigureAwait(false); + cts.Cancel(); + await Task.Delay(Timeout.Infinite, cts.Token).ConfigureAwait(false); + } + finally + { + await sbr.StopAsync().ConfigureAwait(false); + await sbr.DisposeAsync().ConfigureAwait(false); + } + }).AssertException(); + + assertor.LogMessages.Any(x => x?.Contains("An unhandled error has occurred") == true).Should().BeFalse(); + } + finally + { + Subscribers.ProductSubscriber.Id200Processing = null; + await admin.DeleteSubscriptionAsync("unit-test", subscription).ConfigureAwait(false); + } }); [Test] @@ -359,4 +406,4 @@ private bool ReceiveAsync_CircuitBreaker_Internal() return circuitBreakerTripped; } -} \ No newline at end of file +} diff --git a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/Subscribers/ProductSubscriber.cs b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/Subscribers/ProductSubscriber.cs index 038cb46ec..d482be8c2 100644 --- a/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/Subscribers/ProductSubscriber.cs +++ b/tests/CoreEx.Azure.Messaging.ServiceBus.Test.Unit/Subscribers/ProductSubscriber.cs @@ -9,6 +9,11 @@ namespace CoreEx.Azure.Messaging.ServiceBus.Test.Unit.Subscribers; [Subscribe("**.product.**")] public class ProductSubscriber(ILogger logger) : SubscribedBase { + /// + /// Gets or sets the signal completed once a message with an Id of 200 has started processing (allows tests to deterministically wait rather than delay). + /// + public static volatile TaskCompletionSource? Id200Processing; + protected override async Task OnReceiveAsync(Product value, EventData @event, EventSubscriberArgs args, CancellationToken cancellationToken = default) { logger.LogInformation("Received product with Id: {Id} and Sku: {Sku}.", value.Id, value.Sku); @@ -20,7 +25,10 @@ protected override async Task OnReceiveAsync(Product value, EventData @e else if (value.Id == 109) return Result.Fail(new DivideByZeroException("Might be poison?!")); else if (value.Id == 200) + { + Id200Processing?.TrySetResult(); await Task.Delay(Timeout.Infinite, cancellationToken); // Simulates a long-running operation observing the receiver's own cancellation token (e.g. host/processor shutdown). + } return Result.Success; } @@ -31,4 +39,4 @@ public record class Product : IReadOnlyIdentifier public int Id { get; init; } public required string Sku { get; init; } -} \ No newline at end of file +} diff --git a/tests/CoreEx.Caching.Redis.Test.Unit/HybridCacheTests.cs b/tests/CoreEx.Caching.Redis.Test.Unit/HybridCacheTests.cs index f769d3a53..a71599d81 100644 --- a/tests/CoreEx.Caching.Redis.Test.Unit/HybridCacheTests.cs +++ b/tests/CoreEx.Caching.Redis.Test.Unit/HybridCacheTests.cs @@ -1,5 +1,6 @@ using CoreEx.Caching.FusionCache; using CoreEx.Entities; +using CoreEx.Results; using Microsoft.Extensions.Caching.Distributed; using Microsoft.Extensions.Caching.Memory; using Microsoft.Extensions.DependencyInjection; @@ -225,6 +226,77 @@ public async Task Cache_ByCompositeKey() }); } + [Test] + public void GetOrCreateWithResult_Success_IsCached() => Test.ScopedType(async test => + { + var key = "withresult-success-key"; + await ClearKeyAsync(test.Services, key); + + var cache = test.Services.GetRequiredService(); + var options = new HybridCacheEntryOptions { Strategy = CacheStrategy.Hybrid }; + + var r = await cache.GetOrCreateByKeyWithResultAsync(key, _ => Task.FromResult(Result.Ok("value")), options); + r.IsSuccess.Should().BeTrue(); + r.Value.Should().Be("value"); + + IsInDistributedCache(test.Services, key).Should().BeTrue(); + IsInMemoryCache(test.Services, key).Should().BeTrue(); + + r = await cache.GetOrCreateByKeyWithResultAsync(key, _ => throw new InvalidOperationException("Should be cached?!"), options); + r.IsSuccess.Should().BeTrue(); + r.Value.Should().Be("value"); + }); + + [Test] + [TestCase(CacheStrategy.Hybrid)] + [TestCase(CacheStrategy.Local)] + [TestCase(CacheStrategy.Distributed)] + public void GetOrCreateWithResult_Failure_IsNeverCached(CacheStrategy strategy) => Test.ScopedType(async test => + { + var key = $"withresult-failure-key-{strategy}"; + await ClearKeyAsync(test.Services, key); + + var cache = test.Services.GetRequiredService(); + var options = new HybridCacheEntryOptions { Strategy = strategy }; + + var r = await cache.GetOrCreateByKeyWithResultAsync(key, _ => Task.FromResult>(Result.NotFoundError()), options); + r.IsFailure.Should().BeTrue(); + r.Error.Should().BeOfType(); + + IsInDistributedCache(test.Services, key).Should().BeFalse(); + IsInMemoryCache(test.Services, key).Should().BeFalse(); + + // A subsequent success must invoke the factory (i.e. the failure was not cached) and then be cached. + var invoked = 0; + r = await cache.GetOrCreateByKeyWithResultAsync(key, _ => { invoked++; return Task.FromResult(Result.Ok("value")); }, options); + r.IsSuccess.Should().BeTrue(); + r.Value.Should().Be("value"); + invoked.Should().Be(1); + + r = await cache.GetOrCreateByKeyWithResultAsync(key, _ => throw new InvalidOperationException("Should be cached?!"), options); + r.Value.Should().Be("value"); + }); + + [Test] + public void GetOrCreateWithResult_ConcurrentMisses_InvokeFactoryOnce() => Test.ScopedType(async test => + { + var key = "withresult-concurrent-key"; + await ClearKeyAsync(test.Services, key); + + var cache = test.Services.GetRequiredService(); + var invoked = 0; + + var results = await Task.WhenAll(Enumerable.Range(0, 10).Select(_ => cache.GetOrCreateByKeyWithResultAsync(key, async ct => + { + Interlocked.Increment(ref invoked); + await Task.Delay(250, ct).ConfigureAwait(false); + return Result.Ok("value"); + }))); + + invoked.Should().Be(1); + results.Should().OnlyContain(r => r.IsSuccess && r.Value == "value"); + }); + [Test] public void RemoveByTagAsync_QualifiesTag_DoesNotCrossTenantInvalidate() => Test.ScopedType(async test => { diff --git a/tests/CoreEx.Caching.Redis.Test.Unit/ReferenceDataCacheTests.cs b/tests/CoreEx.Caching.Redis.Test.Unit/ReferenceDataCacheTests.cs index 0b56b5e96..612ca9931 100644 --- a/tests/CoreEx.Caching.Redis.Test.Unit/ReferenceDataCacheTests.cs +++ b/tests/CoreEx.Caching.Redis.Test.Unit/ReferenceDataCacheTests.cs @@ -122,6 +122,17 @@ public void InMemory_Expired_GetFromDistributed() colors.Should().NotBeNull().And.BeOfType().Which.Should().HaveCount(3); }).AssertSuccess(); }); + + // Remove the cache. + Test.ScopedType(test => + { + test.ExpectLogContains("[DC] removing distributed entry") + .Run(async _ => + { + var rdo = test.Services.GetRequiredService(); + await rdo.InvalidateAsync(); + }).AssertSuccess(); + }); } public class Color : ReferenceData { } @@ -135,4 +146,4 @@ public class ReferenceDataProvider : IReferenceDataProvider public Task GetAsync(Type type, CancellationToken cancellationToken = default) => Task.FromResult(new ColorCollection { new Color { Id = 1, Code = "R", Text = "Red" }, new Color { Id = 2, Code = "G", Text = "Green" }, new Color { Id = 3, Code = "B", Text = "Blue" } }); } -} \ No newline at end of file +} diff --git a/tests/CoreEx.Cosmos.Test.Unit/CoreEx.Cosmos.Test.Unit.csproj b/tests/CoreEx.Cosmos.Test.Unit/CoreEx.Cosmos.Test.Unit.csproj index c1c0fe890..73768d3ae 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/CoreEx.Cosmos.Test.Unit.csproj +++ b/tests/CoreEx.Cosmos.Test.Unit/CoreEx.Cosmos.Test.Unit.csproj @@ -8,6 +8,8 @@ false true false + true + ../../strong-name-key.snk @@ -24,10 +26,12 @@ + + diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerConcurrencyTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerConcurrencyTests.cs new file mode 100644 index 000000000..0cbaf4dcb --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerConcurrencyTests.cs @@ -0,0 +1,70 @@ +namespace CoreEx.Cosmos.Test.Unit; + +[TestFixture] +public class CosmosDbChangeTrackerConcurrencyTests +{ + [TestCase(false)] + [TestCase(true)] + public async Task DelayedRead_AfterInvalidation_CannotReplaceNewSnapshot(bool clear) + { + var tracker = new CosmosDbChangeTracker(); + var options = new System.Text.Json.JsonSerializerOptions(); + var partitionKey = new PartitionKey("partition"); + var oldVersion = tracker.CaptureVersion("items", partitionKey, "id"); + var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var oldRead = Task.Run(async () => + { + await release.Task.ConfigureAwait(false); + tracker.Set(options, "items", partitionKey, "id", new TestItem { Name = "Old" }, oldVersion); + }); + + if (clear) + tracker.Clear(); + else + tracker.Remove("items", partitionKey, "id"); + + var newVersion = tracker.CaptureVersion("items", partitionKey, "id"); + tracker.Set(options, "items", partitionKey, "id", new TestItem { Name = "New" }, newVersion); + release.SetResult(); + await oldRead.ConfigureAwait(false); + + tracker.Count.Should().Be(1); + tracker.TryGet(options, "items", partitionKey, "id", out var item).Should().BeTrue(); + item!.Name.Should().Be("New"); + } + + [Test] + public void ReadStartedDuringWrite_IsInvalidatedAtWriteCompletion() + { + var tracker = new CosmosDbChangeTracker(); + var options = new System.Text.Json.JsonSerializerOptions(); + var partitionKey = new PartitionKey("partition"); + tracker.Remove("items", partitionKey, "id"); + var version = tracker.CaptureVersion("items", partitionKey, "id"); + + tracker.Remove("items", partitionKey, "id"); + tracker.Set(options, "items", partitionKey, "id", new TestItem { Name = "Old" }, version); + + tracker.Count.Should().Be(0); + } + + [Test] + public void Invalidation_CoversAllModelTypes_ButNotOtherDocuments() + { + var tracker = new CosmosDbChangeTracker(); + var options = new System.Text.Json.JsonSerializerOptions(); + var partitionKey = new PartitionKey("partition"); + var version = tracker.CaptureVersion("items", partitionKey, "id"); + var otherVersion = tracker.CaptureVersion("items", partitionKey, "other"); + tracker.Set(options, "items", partitionKey, "id", new TestItem(), version); + tracker.Set(options, "items", partitionKey, "id", new object(), version); + + tracker.Remove("items", partitionKey, "id"); + tracker.Set(options, "items", partitionKey, "id", new TestItem(), version); + tracker.Set(options, "items", partitionKey, "id", new object(), version); + tracker.Set(options, "items", partitionKey, "other", new TestItem(), otherVersion); + + tracker.Count.Should().Be(1); + tracker.TryGet(options, "items", partitionKey, "other", out _).Should().BeTrue(); + } +} diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerTests.cs new file mode 100644 index 000000000..5cc8bdf9f --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbChangeTrackerTests.cs @@ -0,0 +1,222 @@ +namespace CoreEx.Cosmos.Test.Unit; + +[TestFixture] +public class CosmosDbChangeTrackerTests : CosmosTestBase +{ + private const string ContainerId = "tracker-items"; + + private static async Task<(CosmosDb Db, CosmosDbContainer Container)> GetAsync() + { + await GetOrCreateContainerAsync(ContainerId).ConfigureAwait(false); + var db = CreateCosmosDb(); + return (db, db.Container(ContainerId, o => o.WithPartitionKey(m => m.PartitionKey))); + } + + private static async Task<(string Id, TestItem Created)> SeedAsync(string name = "Original") + { + var (_, c) = await GetAsync(); + var id = NewId(); + var created = (await c.CreateAsync(new TestItem { Id = id, PartitionKey = id, Name = name })).Value; + return (id, created); + } + + [Test] + public async Task Get_SecondRead_IsServedFromSnapshot() + { + var (id, _) = await SeedAsync(); + var (db, scoped) = await GetAsync(); + + var first = await scoped.GetAsync(CompositeKey.Create(id), id); + db.ChangeTracker.Count.Should().Be(1); + + // Change the document from a different scope; this scope must still see the snapshot. + var (_, other) = await GetAsync(); + var fresh = (await other.GetAsync(CompositeKey.Create(id), id))!; + fresh.Name = "Changed"; + await other.UpdateAsync(fresh); + + var second = await scoped.GetAsync(CompositeKey.Create(id), id); + second!.Name.Should().Be("Original"); + second.Should().NotBeSameAs(first); + } + + [Test] + public async Task Get_MutatingReturnedInstance_DoesNotCorruptSnapshot() + { + var (id, _) = await SeedAsync(); + var (_, c) = await GetAsync(); + + var first = (await c.GetAsync(CompositeKey.Create(id), id))!; + first.Name = "Tampered"; + + var second = await c.GetAsync(CompositeKey.Create(id), id); + second!.Name.Should().Be("Original"); + } + + [Test] + public async Task Get_WithClearChangeTrackerAfterGet_DoesNotTrack_AndBypassesSnapshot() + { + var (id, _) = await SeedAsync(); + var (db, c) = await GetAsync(); + + await c.GetAsync(c.Args with { ClearChangeTrackerAfterGet = true }, CompositeKey.Create(id), id); + db.ChangeTracker.Count.Should().Be(0); + + await c.GetAsync(CompositeKey.Create(id), id); + db.ChangeTracker.Count.Should().Be(1); + + var (_, other) = await GetAsync(); + var fresh = (await other.GetAsync(CompositeKey.Create(id), id))!; + fresh.Name = "Changed"; + await other.UpdateAsync(fresh); + + var bypass = await c.GetAsync(c.Args with { ClearChangeTrackerAfterGet = true }, CompositeKey.Create(id), id); + bypass!.Name.Should().Be("Changed"); + db.ChangeTracker.Count.Should().Be(0); + } + + [Test] + public async Task Get_NotFound_IsNotTracked() + { + var (db, c) = await GetAsync(); + var id = NewId(); + (await c.GetAsync(CompositeKey.Create(id), id)).Should().BeNull(); + db.ChangeTracker.Count.Should().Be(0); + } + + [Test] + public async Task Update_Evicts_AndSubsequentGetSeesNewValue() + { + var (id, _) = await SeedAsync(); + var (db, c) = await GetAsync(); + + var item = (await c.GetAsync(CompositeKey.Create(id), id))!; + item.Name = "Updated"; + await c.UpdateAsync(item); + + db.ChangeTracker.Count.Should().Be(0); + (await c.GetAsync(CompositeKey.Create(id), id))!.Name.Should().Be("Updated"); + } + + [Test] + public async Task Delete_BypassesSnapshot_AndEvicts() + { + var (id, _) = await SeedAsync(); + var (db, c) = await GetAsync(); + + await c.GetAsync(CompositeKey.Create(id), id); + db.ChangeTracker.Count.Should().Be(1); + + // Delete from elsewhere; the snapshot here is now stale. + var (_, other) = await GetAsync(); + await other.DeleteAsync(CompositeKey.Create(id), id); + + await c.DeleteAsync(CompositeKey.Create(id), id); + db.ChangeTracker.Count.Should().Be(0); + (await c.GetAsync(CompositeKey.Create(id), id)).Should().BeNull(); + } + + [Test] + public async Task Update_WithoutChangeLogOnCandidate_RetainsCreatedAudit() + { + var (id, created) = await SeedAsync(); + var (_, c) = await GetAsync(); + + created.ChangeLog.Should().NotBeNull(); + + var candidate = new TestItem { Id = id, PartitionKey = id, Name = "Replaced" }; + var updated = (await c.UpdateAsync(candidate)).Value; + + updated.ChangeLog!.CreatedOn.Should().Be(created.ChangeLog!.CreatedOn); + updated.ChangeLog.CreatedBy.Should().Be(created.ChangeLog.CreatedBy); + updated.ChangeLog.UpdatedOn.Should().NotBeNull(); + } + + [Test] + public async Task Update_ForwardsExtensionData_CandidateKeysWin() + { + var (id, _) = await SeedAsync(); + var (db, c) = await GetAsync(); + + // Write unmapped properties directly via the SDK, as another (newer) version of the model might have. + var raw = Client.GetContainer(TestDatabase.Id, ContainerId); + var doc = (await raw.ReadItemAsync(id, new Microsoft.Azure.Cosmos.PartitionKey(id))).Resource; + doc["keep"] = "kept"; + doc["both"] = "server"; + await raw.ReplaceItemAsync(doc, id, new Microsoft.Azure.Cosmos.PartitionKey(id)); + + var candidate = new TestItem { Id = id, PartitionKey = id, Name = "Replaced", ExtensionData = new() { ["both"] = "candidate" } }; + await c.UpdateAsync(candidate); + + var after = (await raw.ReadItemAsync(id, new Microsoft.Azure.Cosmos.PartitionKey(id))).Resource; + after["keep"]!.GetValue().Should().Be("kept"); + after["both"]!.GetValue().Should().Be("candidate"); + after["name"]!.GetValue().Should().Be("Replaced"); + db.ChangeTracker.Count.Should().Be(0); + } + + [Test] + public async Task Update_NoChanges_IsNoOp_AndDoesNotMutate() + { + var (id, _) = await SeedAsync(); + var (_, c) = await GetAsync(); + + var item = (await c.GetAsync(CompositeKey.Create(id), id))!; + var etag = item.ETag; + + var result = await c.UpdateAsync(item); + result.WasMutated.Should().BeFalse(); + result.Value.ETag.Should().Be(etag); + + var raw = Client.GetContainer(TestDatabase.Id, ContainerId); + (await raw.ReadItemAsync(id, new Microsoft.Azure.Cosmos.PartitionKey(id))).Resource["_etag"]!.GetValue().Should().Be(etag); + } + + [Test] + public async Task Update_NoChanges_WithStaleETag_StillHitsServer_AndThrowsConcurrency() + { + var (id, created) = await SeedAsync(); + + // Move the server on so the created ETag is stale. + var (_, other) = await GetAsync(); + var fresh = (await other.GetAsync(CompositeKey.Create(id), id))!; + fresh.Name = "Winner"; + await other.UpdateAsync(fresh); + + // The candidate is value-identical to the persisted document but carries the stale ETag, so the no-op shortcut must not apply. + var (_, c) = await GetAsync(); + var stale = new TestItem { Id = id, PartitionKey = id, Name = "Winner", ETag = created.ETag }; + var act = () => c.UpdateAsync(stale); + await act.Should().ThrowAsync(); + } + + [Test] + public async Task Update_Changed_IsMutated() + { + var (id, _) = await SeedAsync(); + var (_, c) = await GetAsync(); + + var item = (await c.GetAsync(CompositeKey.Create(id), id))!; + item.Name = "Different"; + (await c.UpdateAsync(item)).WasMutated.Should().BeTrue(); + } + + [Test] + public async Task UnitOfWork_RootTransaction_ClearsTrackerOnCompletion() + { + var (id, _) = await SeedAsync(); + var (db, c) = await GetAsync(); + var uow = new CosmosDbUnitOfWork(db); + + await uow.TransactionAsync(async ct => + { + var item = (await c.GetAsync(CompositeKey.Create(id), id, ct).ConfigureAwait(false))!; + db.ChangeTracker.Count.Should().Be(1); + item.Name = "InTxn"; + await c.UpdateAsync(item, ct).ConfigureAwait(false); + }); + + db.ChangeTracker.Count.Should().Be(0); + (await c.GetAsync(CompositeKey.Create(id), id))!.Name.Should().Be("InTxn"); + } +} \ No newline at end of file diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbContainerTimeToLiveTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbContainerTimeToLiveTests.cs index 65accb926..4804d7a56 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbContainerTimeToLiveTests.cs +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbContainerTimeToLiveTests.cs @@ -43,6 +43,33 @@ public async Task UpdateAsync_WithTimeToLive_RecomputesTtl() updated.Value.TimeToLive.Should().Be(120); } + [Test] + public async Task UpdateAsync_NoTimeToLiveConfigured_CandidateWithoutTtl_RetainsPersistedTtl() + { + await GetOrCreateContainerAsync(ContainerId).ConfigureAwait(false); + var container = CreateCosmosDb().Container(ContainerId, o => o.WithPartitionKey(m => m.PartitionKey)); + + var id = NewId(); + await container.CreateAsync(new TestItem { Id = id, PartitionKey = id, Name = "Original", TimeToLive = 600 }); + + var updated = await container.UpdateAsync(new TestItem { Id = id, PartitionKey = id, Name = "Replaced" }); + updated.Value.TimeToLive.Should().Be(600); + + var other = CreateCosmosDb().Container(ContainerId, o => o.WithPartitionKey(m => m.PartitionKey)); + (await other.GetAsync(CompositeKey.Create(id), id))!.TimeToLive.Should().Be(600); + } + + [Test] + public async Task UpdateAsync_NoTimeToLiveConfigured_ExplicitCandidateTtl_Wins() + { + await GetOrCreateContainerAsync(ContainerId).ConfigureAwait(false); + var container = CreateCosmosDb().Container(ContainerId, o => o.WithPartitionKey(m => m.PartitionKey)); + + var id = NewId(); + await container.CreateAsync(new TestItem { Id = id, PartitionKey = id, Name = "Original", TimeToLive = 600 }); + + (await container.UpdateAsync(new TestItem { Id = id, PartitionKey = id, Name = "Original", TimeToLive = -1 })).Value.TimeToLive.Should().Be(-1); + } [Test] public void WithTimeToLive_ModelWithoutITimeToLive_Throws() { diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbOutboxRelayTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbOutboxRelayTests.cs index ff445c211..7385d5d54 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbOutboxRelayTests.cs +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbOutboxRelayTests.cs @@ -265,8 +265,7 @@ public async Task ProcessBatchAsync_NonOutboxDocument_IsIgnored() testPublisher.Published.Should().BeEmpty(); } - // Fixed (not per-run GUID-suffixed) container names, matching every other fixture in this project - the local emulator caps the TOTAL number of containers across the whole account - // (AZURE_COSMOS_EMULATOR_PARTITION_COUNT, see docker-compose.yml), and a new container pair per test run/rerun burns through that budget fast for no benefit (confirmed the hard way this session). + // Fixed (not per-run GUID-suffixed) container names, matching every other fixture in this project - a new container pair per test run/rerun just accumulates emulator state for no benefit. private const string CircuitBreakerContainerId = "relay-cb-items"; private const string CircuitBreakerLeaseContainerId = "relay-cb-items-leases"; @@ -333,6 +332,115 @@ await unitOfWork.TransactionAsync(async ct => private const string DisposeLeaseContainerId = "relay-items-leases"; + [TestCase(false, true)] + [TestCase(true, true)] + [TestCase(false, false)] + public async Task Relay_Instrumentation_SuppressesPollingButPreservesPublishing(bool pollingEnabled, bool publishingEnabled) + { + const string containerId = "relay-telemetry-items"; + const string leaseContainerId = "relay-telemetry-leases"; + await GetOrCreateContainerAsync(containerId).ConfigureAwait(false); + await TestDatabase.CreateContainerIfNotExistsAsync(leaseContainerId, "/id").ConfigureAwait(false); + + var activities = new ConcurrentQueue(); + using var telemetry = Sdk.CreateTracerProviderBuilder() + .AddSource("Azure.Cosmos.Operation", "test.outbox.*", CloudEventTracingExtensions.RelayMarkerActivitySourceName) + .AddInvokerAsSource().AddHttpClientInstrumentation() + .SetSampler(new AlwaysOnSampler()) + .AddProcessor(new CaptureActivityProcessor(activities)).Build(); + CosmosClient client; + // The client starts its own account-discovery work during construction, outside the relay's lifecycle. + using (SuppressInstrumentationScope.Begin(!pollingEnabled)) + client = CreateClient(enableTracing: true); + using var ownedClient = client; + var cosmosDb = new CosmosDb(client, TestDatabase.Id); + var publisher = new TestEventPublisher(); + using var sp = new ServiceCollection() + .AddScoped(_ => new CosmosDb(client, TestDatabase.Id)) + .AddSingleton(publisher).BuildServiceProvider(); + var processor = new CosmosDbOutboxRelayProcessor(sp, containerId, NullLogger.Instance) + { + IsInstrumentationEnabledForPublishing = publishingEnabled + }; + var options = new CosmosDbOutboxRelayOptions + { + ContainerId = containerId, + LeaseContainerId = leaseContainerId, + InstanceName = $"instance-{NewId()}", + PollInterval = TimeSpan.FromMilliseconds(200), + IsInstrumentationEnabledForPolling = pollingEnabled + }; + await using var relay = new CosmosDbOutboxRelay(cosmosDb.Database, options, processor, NullLogger.Instance); + + await relay.StartAsync().ConfigureAwait(false); + await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false); + AssertPollingActivities(); + + await relay.PauseAsync("Verify instrumentation after resume.").ConfigureAwait(false); + activities.Clear(); + await relay.ResumeAsync().ConfigureAwait(false); + await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false); + AssertPollingActivities(); + + var id = NewId(); + using var producerSource = new ActivitySource("test.outbox.producer"); + ActivityContext origin; + using (var producer = producerSource.StartActivity("original-request")) + { + producer.Should().NotBeNull(); + origin = producer!.Context; + var outbox = new CosmosDbEventPublisher(cosmosDb); + var unitOfWork = new CosmosDbUnitOfWork(cosmosDb, outbox); + await unitOfWork.TransactionAsync(async ct => + { + var created = await cosmosDb.Container(containerId, o => o.WithPartitionKey(m => m.PartitionKey)) + .CreateAsync(new TestItem { Id = id, PartitionKey = id, Name = "Telemetry" }, ct).ConfigureAwait(false); + unitOfWork.Events.Add(EventData.CreateEventWith(created.Value, EventAction.Created).WithSource(new Uri("https://unittest/coreex-cosmos")).WithPartitionKey(id)); + }).ConfigureAwait(false); + } + + await WaitUntilAsync(() => + { + lock (publisher.Published) + return publisher.Published.Any(e => e.Event.Subject == id); + }, TimeSpan.FromSeconds(15)).ConfigureAwait(false); + await relay.StopAsync().ConfigureAwait(false); + publisher.Published.Should().Contain(e => e.Event.Subject == id); + + var cleanup = activities.Where(a => a.Source.Name == "Azure.Cosmos.Operation" && a.DisplayName.StartsWith("delete_item", StringComparison.Ordinal)).ToArray(); + if (pollingEnabled) + cleanup.Should().NotBeEmpty(); + else + cleanup.Should().BeEmpty("cleanup uses the suppressed polling context, not the publishing context"); + + var delivery = activities.Where(a => a.OperationName == "test-publish").ToArray(); + var markers = activities.Where(a => a.Source.Name == CloudEventTracingExtensions.RelayMarkerActivitySourceName && a.TraceId == origin.TraceId).ToArray(); + if (publishingEnabled) + { + delivery.Should().NotBeEmpty(); + activities.Should().Contain(a => a.GetTagItem("outbox.container") as string == containerId); + markers.Should().NotBeEmpty(); + markers.Should().OnlyContain(a => a.ParentSpanId == origin.SpanId); + } + else + { + delivery.Should().BeEmpty(); + markers.Should().BeEmpty(); + activities.Should().NotContain(a => a.GetTagItem("outbox.container") as string == containerId); + } + + void AssertPollingActivities() + { + if (pollingEnabled) + { + activities.Select(a => $"{a.Source.Name}: {a.DisplayName}").Should().Contain(s => s.StartsWith("Azure.Cosmos", StringComparison.Ordinal)); + activities.Select(a => a.Source.Name).Should().Contain("System.Net.Http"); + } + else + activities.Should().BeEmpty("idle change-feed and lease maintenance must not emit SDK or HTTP spans"); + } + } + [Test] public async Task DisposeAsync_WithoutPriorStop_StopsProcessorAndIsIdempotent() { @@ -364,21 +472,136 @@ public async Task DisposeAsync_WithoutPriorStop_StopsProcessorAndIsIdempotent() await act.Should().NotThrowAsync(); } - private const string AutoProvisionContainerId = "relay-autoprovision-items"; - private const string AutoProvisionLeaseContainerId = "relay-autoprovision-items-leases"; + private const string ResetContainerId = "relay-reset-items"; + private const string ResetLeaseContainerId = "relay-reset-items-leases"; [Test] - public async Task StartAsync_LeaseContainerDoesNotAlreadyExist_IsAutoProvisioned() + public async Task Relay_FirstStart_PublishesExistingOutboxBacklog() { - // Regression: StartAsync previously only ever resolved the lease container via Database.GetContainer (a proxy reference, not a create) - against a fresh Cosmos DB database where the lease - // container had never been created, the underlying ChangeFeedProcessor.StartAsync would fail trying to acquire leases against a container that does not exist. Deliberately does NOT pre-create - // the lease container here (unlike every other test in this fixture), to prove StartAsync itself now provisions it. - await GetOrCreateContainerAsync(AutoProvisionContainerId).ConfigureAwait(false); + const string containerId = "relay-backlog-items"; + const string leaseContainerId = "relay-backlog-items-leases"; + await TestDatabase.ReplaceOrCreateContainerAsync(new ContainerProperties(containerId, "/partitionKey")).ConfigureAwait(false); + await TestDatabase.ReplaceOrCreateContainerAsync(new ContainerProperties(leaseContainerId, "/id")).ConfigureAwait(false); + + var cosmosDb = CreateCosmosDb(); + var container = cosmosDb.Container(containerId, o => o.WithPartitionKey(m => m.PartitionKey)); + var unitOfWork = new CosmosDbUnitOfWork(cosmosDb, new CosmosDbEventPublisher(cosmosDb)); + var pk = NewId(); + await unitOfWork.TransactionAsync(async ct => + { + var created = await container.CreateAsync(new TestItem { Id = NewId(), PartitionKey = pk, Name = "ExistingBacklog" }, ct).ConfigureAwait(false); + unitOfWork.Events.Add(EventData.CreateEventWith(created.Value, EventAction.Created).WithSource(new Uri("https://unittest/coreex-cosmos", UriKind.Absolute)).WithPartitionKey(pk)); + }).ConfigureAwait(false); + + // Separate the write from startup beyond Cosmos's timestamp granularity. + await Task.Delay(TimeSpan.FromSeconds(2)).ConfigureAwait(false); + + var testPublisher = new TestEventPublisher(); + using var sp = CreateServiceProvider(testPublisher); + var processor = new CosmosDbOutboxRelayProcessor(sp, containerId, NullLogger.Instance); + var options = new CosmosDbOutboxRelayOptions + { + ContainerId = containerId, + LeaseContainerId = leaseContainerId, + InstanceName = $"instance-{NewId()}", + PollInterval = TimeSpan.FromMilliseconds(200) + }; + + await using var relay = new CosmosDbOutboxRelay(cosmosDb.Database, options, processor, NullLogger.Instance); + await relay.StartAsync().ConfigureAwait(false); + await WaitUntilAsync(HasPublished, TimeSpan.FromSeconds(30)).ConfigureAwait(false); + HasPublished().Should().BeTrue("a fresh lease must not skip outbox events written before relay startup"); + + bool HasPublished() + { + lock (testPublisher.Published) + return testPublisher.Published.Any(e => e.Event.Data?.ToString()?.Contains("ExistingBacklog") ?? false); + } + } + + [Test] + public async Task Relay_ContainersReplacedWhileRunning_SelfRecoversAndRelaysSubsequentEvents() + { + // Regression: a provisioning reset (delete + recreate; e.g. CosmosDbProvisionCommand.Reset) of the monitored and lease containers while the relay was running left the Change Feed Processor bound to + // the deleted containers (its lease prefix embeds the container resource ids) - endlessly reporting lost leases and never relaying another event until the host was restarted. + await GetOrCreateContainerAsync(ResetContainerId).ConfigureAwait(false); + await TestDatabase.CreateContainerIfNotExistsAsync(ResetLeaseContainerId, "/id").ConfigureAwait(false); + + var cosmosDb = CreateCosmosDb(); + var testPublisher = new TestEventPublisher(); + using var sp = CreateServiceProvider(testPublisher); + var processor = new CosmosDbOutboxRelayProcessor(sp, ResetContainerId, NullLogger.Instance); + + var options = new CosmosDbOutboxRelayOptions + { + ContainerId = ResetContainerId, + LeaseContainerId = ResetLeaseContainerId, + InstanceName = $"instance-{NewId()}", + PollInterval = TimeSpan.FromMilliseconds(200), + RecoveryDelay = TimeSpan.FromMilliseconds(500) + }; + + async Task CreateWithEventAsync(string name) + { + var db = CreateCosmosDb(); + var container = db.Container(ResetContainerId, o => o.WithPartitionKey(m => m.PartitionKey)); + var unitOfWork = new CosmosDbUnitOfWork(db, new CosmosDbEventPublisher(db)); + var pk = NewId(); + await unitOfWork.TransactionAsync(async ct => + { + var created = await container.CreateAsync(new TestItem { Id = NewId(), PartitionKey = pk, Name = name }, ct).ConfigureAwait(false); + unitOfWork.Events.Add(EventData.CreateEventWith(created.Value, EventAction.Created).WithSource(new Uri("https://unittest/coreex-cosmos", UriKind.Absolute)).WithPartitionKey(pk)); + }); + } + + bool HasPublished(string name) + { + lock (testPublisher.Published) + return testPublisher.Published.Any(e => e.Event.Data?.ToString()?.Contains(name) ?? false); + } + + await using var relay = new CosmosDbOutboxRelay(cosmosDb.Database, options, processor, NullLogger.Instance); + await relay.StartAsync(); + try + { + await CreateWithEventAsync("BeforeReset"); + await WaitUntilAsync(() => HasPublished("BeforeReset"), TimeSpan.FromSeconds(30)); + HasPublished("BeforeReset").Should().BeTrue(); + + // Simulate a provisioning reset of both containers underneath the running relay. + await TestDatabase.ReplaceOrCreateContainerAsync(new ContainerProperties(ResetContainerId, "/partitionKey")); + await TestDatabase.ReplaceOrCreateContainerAsync(new ContainerProperties(ResetLeaseContainerId, "/id")); + + await CreateWithEventAsync("AfterReset"); + await WaitUntilAsync(() => HasPublished("AfterReset"), TimeSpan.FromSeconds(60)); + HasPublished("AfterReset").Should().BeTrue(); + + // Keep observing for a while; a further event must still relay (i.e. not a one-off delivery from a stale processor). + await Task.Delay(TimeSpan.FromSeconds(20)); + await CreateWithEventAsync("Later"); + await WaitUntilAsync(() => HasPublished("Later"), TimeSpan.FromSeconds(60)); + HasPublished("Later").Should().BeTrue(); + + relay.Status.Should().Be(ServiceStatus.Running); + } + finally + { + await relay.StopAsync(); + } + } + + private const string MissingLeaseContainerId = "relay-missing-lease-items"; + + [Test] + public async Task StartAsync_LeaseContainerDoesNotExist_FailsFastAndDoesNotCreateIt() + { + // The relay must never create its lease container (a control-plane operation a production identity is not permitted); it fails fast with actionable guidance instead. + await GetOrCreateContainerAsync(MissingLeaseContainerId).ConfigureAwait(false); + const string leaseContainerId = "relay-missing-lease-items-leases"; - // Confirm the lease container genuinely does not exist yet - a stale container from a prior interrupted run would invalidate this test's premise. try { - await TestDatabase.GetContainer(AutoProvisionLeaseContainerId).DeleteContainerAsync().ConfigureAwait(false); + await TestDatabase.GetContainer(leaseContainerId).DeleteContainerAsync().ConfigureAwait(false); } catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) { @@ -387,36 +610,35 @@ public async Task StartAsync_LeaseContainerDoesNotAlreadyExist_IsAutoProvisioned var cosmosDb = CreateCosmosDb(); using var sp = CreateServiceProvider(new TestEventPublisher()); - var processor = new CosmosDbOutboxRelayProcessor(sp, AutoProvisionContainerId, NullLogger.Instance); + var processor = new CosmosDbOutboxRelayProcessor(sp, MissingLeaseContainerId, NullLogger.Instance); var options = new CosmosDbOutboxRelayOptions { - ContainerId = AutoProvisionContainerId, - LeaseContainerId = AutoProvisionLeaseContainerId, + ContainerId = MissingLeaseContainerId, + LeaseContainerId = leaseContainerId, InstanceName = $"instance-{NewId()}" }; await using var relay = new CosmosDbOutboxRelay(cosmosDb.Database, options, processor, NullLogger.Instance); Func act = async () => await relay.StartAsync(); - await act.Should().NotThrowAsync(); - relay.Status.Should().Be(ServiceStatus.Running); - - // The lease container must now actually exist (StartAsync provisioned it) - ReadContainerAsync throws CosmosException(NotFound) otherwise. - var readResponse = await TestDatabase.GetContainer(AutoProvisionLeaseContainerId).ReadContainerAsync().ConfigureAwait(false); - readResponse.Resource.Id.Should().Be(AutoProvisionLeaseContainerId); - - await relay.StopAsync(); + (await act.Should().ThrowAsync()).WithMessage($"*'{leaseContainerId}'*OutboxLeaseContainer*"); + relay.Status.Should().Be(ServiceStatus.Initializing); // Unchanged; remains startable once the lease container is provisioned. + Func read = async () => await TestDatabase.GetContainer(leaseContainerId).ReadContainerAsync(); + (await read.Should().ThrowAsync()).Which.StatusCode.Should().Be(HttpStatusCode.NotFound); } private sealed class TestEventPublisher : EventPublisherBase { + private static readonly ActivitySource _source = new("test.outbox.publisher"); + public List Published { get; } = []; public bool ThrowOnPublish { get; set; } protected override Task OnPublishAsync(DestinationEvent[] events, CancellationToken cancellationToken = default) { + using var activity = _source.StartActivity("test-publish"); if (ThrowOnPublish) throw new InvalidOperationException("Simulated publish failure."); @@ -426,4 +648,9 @@ protected override Task OnPublishAsync(DestinationEvent[] events, CancellationTo return Task.CompletedTask; } } + + private sealed class CaptureActivityProcessor(ConcurrentQueue activities) : BaseProcessor + { + public override void OnEnd(Activity data) => activities.Enqueue(data); + } } diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbProvisionerTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbProvisionerTests.cs new file mode 100644 index 000000000..5299fd85d --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbProvisionerTests.cs @@ -0,0 +1,444 @@ +namespace CoreEx.Cosmos.Test.Unit; + +/// +/// Proves the (and its console/client factory) end-to-end against the local emulator; each test uses its own client and uniquely named database (dropped afterwards) so replacing +/// and dropping resources cannot leave stale metadata caches behind for other tests. +/// +[TestFixture] +public class CosmosDbProvisionerTests : CosmosTestBase +{ + private CosmosClient _client = null!; + private string _databaseId = null!; + private const string ItemsId = "prov-items"; + private const string RefId = "prov-ref"; + + private new CosmosClient Client => _client; + + private string DatabaseId => _databaseId; + + private CosmosDbProvisionArgs CreateArgs(StringWriter? output = null) => new CosmosDbProvisionArgs { DatabaseId = DatabaseId, Output = output ?? new StringWriter() } + .Container(ItemsId) + .ReferenceDataContainer(RefId); + + private CosmosDbProvisioner CreateProvisioner(CosmosDbProvisionArgs args) => new(Client, args); + + private static PartitionKey PartitionOf(string containerId) => containerId == ItemsId ? new PartitionKey("pk") : PartitionKey.None; + + private async Task CountAsync(string containerId) + { + using var iterator = Client.GetContainer(DatabaseId, containerId).GetItemQueryIterator("SELECT VALUE COUNT(1) FROM c", requestOptions: new QueryRequestOptions { PartitionKey = PartitionOf(containerId) }); + var total = 0; + while (iterator.HasMoreResults) + { + foreach (var count in await iterator.ReadNextAsync().ConfigureAwait(false)) + { + total += count; + } + } + + return total; + } + + [SetUp] + public void SetUpClient() + { + _client = CreateClient(); + _databaseId = $"CoreEx.Cosmos.Test.Unit.Provisioning.{NewId()}"; + } + + [TearDown] + public async Task TearDownAsync() + { + if (_client is null) + return; + + try { await _client.GetDatabase(_databaseId).DeleteAsync().ConfigureAwait(false); } + catch (CosmosException cex) when (cex.StatusCode == HttpStatusCode.NotFound) { } + finally { _client.Dispose(); _client = null!; } + } + + [Test] + public async Task Create_IsIdempotent_AndPreservesExistingData() + { + var p = CreateProvisioner(CreateArgs()); + await p.RunAsync(CosmosDbProvisionCommand.Create); + await Client.GetContainer(DatabaseId, ItemsId).CreateItemAsync(new { id = "keep", partitionKey = "pk" }, new PartitionKey("pk")); + + await p.RunAsync(CosmosDbProvisionCommand.Create); + + (await CountAsync(ItemsId)).Should().Be(1); + } + + [Test] + public async Task OutboxLeaseContainer_Create_IsProvisionedAndLogged() + { + var output = new StringWriter(); + var args = CreateArgs(output).OutboxLeaseContainer(); + args.Containers.Single(x => x.IsOutboxLease).Id.Should().Be(CosmosDbOutboxRelayOptions.DefaultLeaseContainerId).And.Be("$outbox-leases"); + + var p = CreateProvisioner(args); + await p.RunAsync(CosmosDbProvisionCommand.Create); + + var lease = await Client.GetContainer(DatabaseId, "$outbox-leases").ReadContainerAsync(); + lease.Resource.PartitionKeyPath.Should().Be("/id"); + output.ToString().Should().Contain($"Outbox lease container '$outbox-leases' created."); + + output.GetStringBuilder().Clear(); + await p.RunAsync(CosmosDbProvisionCommand.Create); + output.ToString().Should().Contain("already exists and therefore not created."); + } + + [Test] + public async Task OutboxLeaseContainer_Reset_IsReplacedAndLogged() + { + var output = new StringWriter(); + var p = CreateProvisioner(CreateArgs(output).OutboxLeaseContainer()); + await p.RunAsync(CosmosDbProvisionCommand.Create); + var lease = Client.GetContainer(DatabaseId, "$outbox-leases"); + await lease.CreateItemAsync(new { id = "lease1" }, new PartitionKey("lease1")); + + await p.RunAsync(CosmosDbProvisionCommand.Reset); + + using var iterator = lease.GetItemQueryIterator("SELECT VALUE COUNT(1) FROM c"); + (await iterator.ReadNextAsync()).Single().Should().Be(0); + output.ToString().Should().Contain($"Outbox lease container '$outbox-leases' replaced (empty)."); + } + + [Test] + public void OutboxLeaseContainer_CustomId_AndDuplicate() + { + new CosmosDbProvisionArgs().OutboxLeaseContainer("custom").Containers.Single().Id.Should().Be("custom"); + FluentActions.Invoking(() => new CosmosDbProvisionArgs().OutboxLeaseContainer().Container("$outbox-leases")).Should().Throw(); + } + + [Test] + public async Task RunAndLogAsync_Success_ReturnsCapturedOutput_AndRestoresOutput() + { + var original = new StringWriter(); + var args = CreateArgs(original); + + var (success, output) = await CreateProvisioner(args).RunAndLogAsync(CosmosDbProvisionCommand.Create); + + success.Should().BeTrue(); + output.Should().Contain("DATABASE CREATE"); + args.Output.Should().BeSameAs(original); + original.ToString().Should().Be(output); + (await Client.GetContainer(DatabaseId, ItemsId).ReadContainerAsync()).StatusCode.Should().Be(HttpStatusCode.OK); + } + + [Test] + public async Task RunAndLogAsync_Failure_ReturnsFalse_WithMessage_AndDoesNotThrow() + { + var args = CreateArgs(); + var original = args.Output; + + var (success, output) = await CreateProvisioner(args).RunAndLogAsync(CosmosDbProvisionCommand.ResetAndData); + + success.Should().BeFalse(); + output.Should().Contain("Failed:").And.Contain("does not exist"); + args.Output.Should().BeSameAs(original); + } + + [Test] + public async Task Reset_EmptiesContainers() + { + var p = CreateProvisioner(CreateArgs()); + await p.RunAsync(CosmosDbProvisionCommand.Create); + await Client.GetContainer(DatabaseId, ItemsId).CreateItemAsync(new { id = "gone", partitionKey = "pk" }, new PartitionKey("pk")); + (await CountAsync(ItemsId)).Should().Be(1); + + await p.RunAsync(CosmosDbProvisionCommand.Reset); + + (await CountAsync(ItemsId)).Should().Be(0); + } + + [Test] + public async Task Reset_DatabaseNotExisting_FailsAndDoesNotCreate() + { + await FluentActions.Awaiting(() => CreateProvisioner(CreateArgs()).RunAsync(CosmosDbProvisionCommand.ResetAndData)).Should().ThrowAsync().WithMessage("*does not exist*"); + + await FluentActions.Awaiting(() => Client.GetDatabase(DatabaseId).ReadAsync()).Should().ThrowAsync().Where(x => x.StatusCode == HttpStatusCode.NotFound); + } + + [Test] + public async Task ResetAndData_ImportsPlainAndDiscriminatedData() + { + var args = CreateArgs().AddDataResource("provisioning.seed.yaml"); + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + (await CountAsync(ItemsId)).Should().Be(2); + (await CountAsync(RefId)).Should().Be(3); + + using var iterator = Client.GetContainer(DatabaseId, RefId).GetItemQueryIterator("SELECT VALUE c.typeDiscriminator FROM c ORDER BY c.typeDiscriminator", requestOptions: new QueryRequestOptions { PartitionKey = PartitionKey.None }); + var types = new List(); + while (iterator.HasMoreResults) + { + types.AddRange(await iterator.ReadNextAsync()); + } + + types.Should().Equal("Colour", "Colour", "Shape"); + } + + [Test] + public async Task ReferenceDataContainer_RejectsDuplicateTypeDiscriminatorAndCode() + { + await CreateProvisioner(CreateArgs()).RunAsync(CosmosDbProvisionCommand.Create); + var container = Client.GetContainer(DatabaseId, RefId); + + await container.CreateItemAsync(new { id = "a", partitionKey = "pk", typeDiscriminator = "Colour", code = "R" }, new PartitionKey("pk")); + var ex = await FluentActions.Awaiting(() => container.CreateItemAsync(new { id = "b", partitionKey = "pk", typeDiscriminator = "Colour", code = "R" }, new PartitionKey("pk"))).Should().ThrowAsync(); + + ex.Which.StatusCode.Should().Be(HttpStatusCode.Conflict); + } + + [Test] + public async Task Data_UnknownContainerKey_Throws() + { + var args = CreateArgs().AddDataResource("provisioning-unknown.seed.yaml"); + var p = CreateProvisioner(args); + await p.RunAsync(CosmosDbProvisionCommand.Create); + + await FluentActions.Awaiting(() => p.RunAsync(CosmosDbProvisionCommand.Data)).Should().ThrowAsync().WithMessage("*unknown-container*"); + } + + [Test] + public async Task Drop_RemovesDatabase_AndIsSafeWhenMissing() + { + var p = CreateProvisioner(CreateArgs()); + await p.RunAsync(CosmosDbProvisionCommand.Create); + + await p.RunAsync(CosmosDbProvisionCommand.Drop); + await p.RunAsync(CosmosDbProvisionCommand.Drop); + + await FluentActions.Awaiting(() => Client.GetDatabase(DatabaseId).ReadAsync()).Should().ThrowAsync().Where(x => x.StatusCode == HttpStatusCode.NotFound); + } + + [Test] + public void DuplicateContainerDeclaration_Throws() + => FluentActions.Invoking(() => new CosmosDbProvisionArgs().Container("a").Container("a")).Should().Throw(); + + [Test] + public void ReferenceDataContainer_DefinesUniqueKey_AndPlainContainerDoesNot() + { + var args = CreateArgs(); + args.Containers.Single(x => x.Id == RefId).CreateProperties().UniqueKeyPolicy.UniqueKeys.Single().Paths.Should().Equal("/typeDiscriminator", "/code"); + args.Containers.Single(x => x.Id == ItemsId).CreateProperties().UniqueKeyPolicy.UniqueKeys.Should().BeEmpty(); + } + + [Test] + public void ClientFactory_LocalEndpoint_UsesGateway_AndRemoteDoesNot() + { + const string key = "C2y6yDjf5/R+ob0N8A7Cgv30VRDJIWEHLM+4QDU5DE2nQ9nDuVTqobD4b8mGGyPMbIZnqyMsEcaGQy67XIw/Jw=="; + + using var local = CosmosDbClientFactory.Create($"AccountEndpoint=https://localhost:8081/;AccountKey={key}"); + local.ClientOptions.ConnectionMode.Should().Be(ConnectionMode.Gateway); + + using var remote = CosmosDbClientFactory.Create($"AccountEndpoint=https://example.documents.azure.com:443/;AccountKey={key}"); + remote.ClientOptions.ConnectionMode.Should().Be(ConnectionMode.Direct); + } + + [Test] + public async Task Console_ParsesArguments() + { + var console = CosmosDbConsole.Create("AccountEndpoint=https://localhost:8081/;AccountKey=abc", DatabaseId); + + (await console.RunAsync(["--help"])).Should().Be(0); + (await console.RunAsync([])).Should().Be(1); + (await console.RunAsync(["NotACommand"])).Should().Be(1); + (await console.RunAsync(["Create", "-d"])).Should().Be(1); + (await console.RunAsync(["Create", "-p", "NoEquals"])).Should().Be(1); + (await console.RunAsync(["Create", "-cv", "COREEX_COSMOS_TEST_UNSET_VARNAME"])).Should().Be(1); + (await console.RunAsync(["Create", "--not-an-option"])).Should().Be(1); + } + + [Test] + public async Task Console_DestructiveCommands_RequireConfirmation() + { + var originalIn = Console.In; + var originalError = Console.Error; + try + { + // Not confirmed: stops before any connection is made, so nothing is executed. + var error = new StringWriter(); + Console.SetError(error); + var console = CosmosDbConsole.Create("AccountEndpoint=https://localhost:1/;AccountKey=abc", DatabaseId); + + Console.SetIn(new StringReader("n")); + (await console.RunAsync(["Drop"])).Should().Be(1); + error.ToString().Should().Contain("Database drop was not confirmed"); + + error.GetStringBuilder().Clear(); + Console.SetIn(new StringReader(string.Empty)); + (await console.RunAsync(["ResetAndData"])).Should().Be(1); + error.ToString().Should().Contain("Container reset was not confirmed"); + } + finally + { + Console.SetIn(originalIn); + Console.SetError(originalError); + } + } + + [Test] + public async Task Console_ConnectionVarName_AndParams_AreApplied() + { + var varName = $"COREEX_COSMOS_TEST_{Guid.NewGuid():N}"; + Environment.SetEnvironmentVariable(varName, "AccountEndpoint=https://localhost:1/;AccountKey=abc"); + try + { + var console = CosmosDbConsole.Create("AccountEndpoint=https://localhost:8081/;AccountKey=abc", DatabaseId); + await console.RunAsync(["Create", "-cv", varName, "-p", "A=1", "--param", "B=x=y", "--accept-prompts"]); + + console.ConnectionString.Should().Contain("localhost:1"); + console.Args.AcceptPrompts.Should().BeTrue(); + console.Args.Parameters.Should().BeEquivalentTo(new Dictionary { ["A"] = "1", ["B"] = "x=y" }); + } + finally + { + Environment.SetEnvironmentVariable(varName, null); + } + } + + [Test] + public async Task Data_Parameters_AreSubstituted() + { + var args = CreateArgs().AddDataResource("provisioning-param.seed.yaml"); + args.Parameters["Who"] = "Eric"; + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + (await QueryAsync(ItemsId, "SELECT VALUE c.name FROM c")).Should().Equal("Eric"); + } + + private async Task> QueryAsync(string containerId, string sql) + { + using var iterator = Client.GetContainer(DatabaseId, containerId).GetItemQueryIterator(sql, requestOptions: new QueryRequestOptions { PartitionKey = PartitionOf(containerId) }); + var items = new List(); + while (iterator.HasMoreResults) + { + items.AddRange(await iterator.ReadNextAsync()); + } + + return items; + } + + [Test] + public async Task Data_ContainerDataOptions_AreApplied() + { + var args = new CosmosDbProvisionArgs { DatabaseId = DatabaseId, Output = new StringWriter() } + .Container(ItemsId, dataOptions: ctx => { var o = new JsonDataReaderOptions(ctx.NamingConvention); o.Properties.Add("tag", "container"); return o; }) + .ReferenceDataContainer(RefId) + .AddDataResource("provisioning.seed.yaml"); + + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + (await QueryAsync(ItemsId, "SELECT VALUE c.tag FROM c")).Should().Equal("container", "container"); + (await QueryAsync(RefId, "SELECT VALUE c.tag FROM c")).Should().BeEmpty(); + } + + [Test] + public async Task Data_ResourceDataOptions_TakePrecedence_AndNullDefersToContainer() + { + var args = new CosmosDbProvisionArgs { DatabaseId = DatabaseId, Output = new StringWriter() } + .Container(ItemsId, dataOptions: ctx => { var o = new JsonDataReaderOptions(ctx.NamingConvention); o.Properties.Add("tag", "container"); return o; }) + .ReferenceDataContainer(RefId, dataOptions: ctx => { var o = JsonDataReaderOptions.CreateForReferenceData(ctx.NamingConvention); o.Properties.Add("tag", "refcontainer"); return o; }) + .AddDataResource("provisioning.seed.yaml", ctx => + { + if (ctx.Container.Id != ItemsId) + return null; + + var o = new JsonDataReaderOptions(ctx.NamingConvention); + o.Properties.Add("tag", "resource"); + return o; + }); + + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + (await QueryAsync(ItemsId, "SELECT VALUE c.tag FROM c")).Should().Equal("resource", "resource"); + (await QueryAsync(RefId, "SELECT VALUE c.tag FROM c")).Should().Equal("refcontainer", "refcontainer", "refcontainer"); + } + + [Test] + public async Task Data_ReferenceDataContainer_CustomIdGenerator_ViaDataOptions() + { + var n = 0; + var args = new CosmosDbProvisionArgs { DatabaseId = DatabaseId, Output = new StringWriter() } + .Container(ItemsId) + .ReferenceDataContainer(RefId, dataOptions: ctx => JsonDataReaderOptions.CreateForReferenceData(ctx.NamingConvention, () => $"gen-{Interlocked.Increment(ref n)}")) + .AddDataResource("provisioning.seed.yaml"); + + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + (await QueryAsync(RefId, "SELECT VALUE c.id FROM c ORDER BY c.id")).Should().Equal("gen-1", "gen-2", "gen-3"); + } + + [Test] + public async Task Data_DollarPrefix_Merges_AndIsRerunnableWithStableIds() + { + var args = CreateArgs().AddDataResource("provisioning-merge.seed.yaml"); + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + var ids = await RefIdsAsync(); + ids.Should().HaveCount(2); + + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.All); + + (await CountAsync(ItemsId)).Should().Be(1); + (await CountAsync(RefId)).Should().Be(2); + (await RefIdsAsync()).Should().Equal(ids); + } + + [Test] + public async Task Data_NoDollarPrefix_Inserts_SoRerunConflicts() + { + var args = CreateArgs().AddDataResource("provisioning-insert.seed.yaml"); + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + (await CountAsync(RefId)).Should().Be(1); + + var act = () => CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Data); + (await act.Should().ThrowAsync()).Which.StatusCode.Should().Be(System.Net.HttpStatusCode.Conflict); + + var plain = CreateArgs().AddDataResource("provisioning-insert-plain.seed.yaml"); + await CreateProvisioner(plain).RunAsync(CosmosDbProvisionCommand.Data); + var act2 = () => CreateProvisioner(plain).RunAsync(CosmosDbProvisionCommand.Data); + (await act2.Should().ThrowAsync()).Which.StatusCode.Should().Be(System.Net.HttpStatusCode.Conflict); + } + + private async Task> RefIdsAsync() + { + using var iterator = Client.GetContainer(DatabaseId, RefId).GetItemQueryIterator("SELECT VALUE c.id FROM c ORDER BY c.code", requestOptions: new QueryRequestOptions { PartitionKey = PartitionKey.None }); + var ids = new List(); + while (iterator.HasMoreResults) + { + ids.AddRange(await iterator.ReadNextAsync()); + } + + return ids; + } + + [Test] + public async Task Output_WritesSections_AndMergeCounts() + { + var first = new StringWriter(); + var args = CreateArgs(first).AddDataResource("provisioning-merge.seed.yaml"); + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.Create | CosmosDbProvisionCommand.ResetAndData); + + var text = first.ToString(); + text.Should().Contain("DATABASE CREATE:").And.Contain("CONTAINER RESET:").And.Contain("DATABASE DATA:"); + text.Should().Contain("** Parsing and executing: ").And.Contain("provisioning-merge.seed.yaml"); + text.Should().Contain("Result: 2 item(s) (2 created, 0 replaced).").And.Contain("Complete. ["); + text.Should().Contain(new string('-', 80)); + + var second = new StringWriter(); + args.Output = second; + await CreateProvisioner(args).RunAsync(CosmosDbProvisionCommand.All); + + text = second.ToString(); + text.Should().Contain("DATABASE CREATE:").And.Contain("already exists and therefore not created."); + text.Should().Contain("Result: 2 item(s) (0 created, 2 replaced)."); + } + + [Test] + public void AddDataResource_Duplicate_Throws() + => FluentActions.Invoking(() => new CosmosDbProvisionArgs().AddDataResource("provisioning.seed.yaml").AddDataResource("provisioning.seed.yaml", _ => null)) + .Should().Throw(); +} diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosDbReferenceDataTests.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbReferenceDataTests.cs new file mode 100644 index 000000000..8fce61eca --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosDbReferenceDataTests.cs @@ -0,0 +1,259 @@ +namespace CoreEx.Cosmos.Test.Unit; + +[TestFixture] +public class CosmosDbReferenceDataTests : CosmosTestBase +{ + private const string ContainerId = "ref-data-items"; + private const string OutboxContainerId = "ref-data-outbox-items"; + private const string NoCodeOutboxContainerId = "ref-data-outbox-nocode-items"; + + private static readonly TestRefMapper _mapper = new(); + + private static async Task EnsureContainerAsync() + { + await EnsureUniqueContainerAsync(ContainerId).ConfigureAwait(false); + } + + // The unique key mirrors the one the reference data containers use (type discriminator + code) so that a duplicate code is rejected by Cosmos DB. + private static async Task EnsureUniqueContainerAsync(string containerId) + { + var properties = new ContainerProperties(containerId, "/partitionKey"); + properties.UniqueKeyPolicy.UniqueKeys.Add(new UniqueKey { Paths = { "/typeDiscriminator", "/code" } }); + await TestDatabase.CreateContainerIfNotExistsAsync(properties).ConfigureAwait(false); + } + + // Creates within a unit-of-work (so that an outbox event is written to the same container/partition as the business document). + private static async Task>> CreateWithOutboxAsync(CosmosDb cosmosDb, string code, string containerId = OutboxContainerId) + { + var container = cosmosDb.Container(containerId, o => o.WithTypeDiscriminator()); + var unitOfWork = new CosmosDbUnitOfWork(cosmosDb, new CosmosDbEventPublisher(cosmosDb)); + + return await unitOfWork.TransactionAsync(async ct => + { + var result = await CosmosDbReferenceData.CreateAsync(container, new TestRef { Code = code, Text = code }, _mapper, ct).ConfigureAwait(false); + if (result.IsSuccess) + unitOfWork.Events.Add(EventData.CreateEventWith(result.Value.Value, EventAction.Created).WithSource(new Uri("https://unittest/coreex-cosmos", UriKind.Absolute))); + + return result; + }).ConfigureAwait(false); + } + + private async Task> GetContainerAsync() + { + await EnsureContainerAsync().ConfigureAwait(false); + return CreateCosmosDb().Container(ContainerId, o => o.WithTypeDiscriminator()); + } + + private static Task>> CreateAsync(CosmosDbContainer container, string code, string? text = null) + => CosmosDbReferenceData.CreateAsync(container, new TestRef { Code = code, Text = text ?? code }, _mapper); + + private static Task>> UpdateAsync(CosmosDbContainer container, TestRef value) + => CosmosDbReferenceData.UpdateAsync(container, value.Id, value, _mapper); + + private static Task>> ActivateAsync(CosmosDbContainer container, string id) + => CosmosDbReferenceData.ActivateAsync(container, id, _mapper); + + private static Task>> DeactivateAsync(CosmosDbContainer container, string id) + => CosmosDbReferenceData.DeactivateAsync(container, id, _mapper); + + private static Task> DeleteAsync(CosmosDbContainer container, string id) + => CosmosDbReferenceData.DeleteAsync(container, id); + + [Test] + public async Task CreateAsync_GeneratesIdentifier_AndIsInactive() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var code = NewId(); + + var result = await CreateAsync(container, code, "Text").ConfigureAwait(false); + + result.IsSuccess.Should().BeTrue(); + result.Value.WasMutated.Should().BeTrue(); + var rd = result.Value.Value; + rd.Id.Should().NotBeNullOrEmpty(); + rd.Code.Should().Be(code); + rd.IsInactive.Should().BeTrue(); + rd.ETag.Should().NotBeNullOrEmpty(); + + var model = await container.GetAsync(CompositeKey.Create(rd.Id)).ConfigureAwait(false); + model.Should().NotBeNull(); + model!.Code.Should().Be(code); + model.IsActive.Should().BeFalse(); + model.TypeDiscriminator.Should().Be(nameof(TestRefItem)); + model.ChangeLog.Should().NotBeNull(); + model.ChangeLog!.CreatedOn.Should().NotBeNull(); + } + + [Test] + public async Task CreateAsync_WithDuplicateCode_Fails() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var code = NewId(); + + (await CreateAsync(container, code).ConfigureAwait(false)).IsSuccess.Should().BeTrue(); + var result = await CreateAsync(container, code).ConfigureAwait(false); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + } + + [Test] + public async Task UpdateAsync_ChangesText_ButNotCodeOrActive_AndPreservesChangeLog() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var created = (await CreateAsync(container, NewId(), "Before").ConfigureAwait(false)).Value.Value; + var activated = (await ActivateAsync(container, created.Id).ConfigureAwait(false)).Value.Value; + + var update = new TestRef { Id = activated.Id, Code = "IGNORED", Text = "After", SortOrder = 7, ETag = activated.ETag }; + var result = await UpdateAsync(container, update).ConfigureAwait(false); + + result.IsSuccess.Should().BeTrue(); + result.Value.WasMutated.Should().BeTrue(); + var rd = result.Value.Value; + rd.Code.Should().Be(created.Code); + rd.GetText().Should().Be("After"); + rd.SortOrder.Should().Be(7); + rd.IsInactive.Should().BeFalse(); + rd.ETag.Should().NotBe(activated.ETag); + + var model = await container.GetAsync(CompositeKey.Create(created.Id)).ConfigureAwait(false); + model!.ChangeLog!.CreatedOn.Should().NotBeNull(); + model.ChangeLog.UpdatedOn.Should().NotBeNull(); + } + + [Test] + public async Task UpdateAsync_WithStaleETag_Fails_WithConcurrencyError() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var created = (await CreateAsync(container, NewId()).ConfigureAwait(false)).Value.Value; + + var first = await UpdateAsync(container, new TestRef { Id = created.Id, Text = "One", ETag = created.ETag }).ConfigureAwait(false); + first.IsSuccess.Should().BeTrue(); + + var stale = await UpdateAsync(container, new TestRef { Id = created.Id, Text = "Two", ETag = created.ETag }).ConfigureAwait(false); + stale.IsFailure.Should().BeTrue(); + stale.Error.Should().BeOfType(); + } + + [Test] + public async Task UpdateAsync_NotFound_Fails() + { + var container = await GetContainerAsync().ConfigureAwait(false); + + var result = await UpdateAsync(container, new TestRef { Id = NewId(), Code = "X", Text = "X" }).ConfigureAwait(false); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + } + + [Test] + public async Task ActivateAsync_And_DeactivateAsync_ToggleState_AndAreIdempotent() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var created = (await CreateAsync(container, NewId()).ConfigureAwait(false)).Value.Value; + + var deactivatedAlready = await DeactivateAsync(container, created.Id).ConfigureAwait(false); + deactivatedAlready.IsSuccess.Should().BeTrue(); + deactivatedAlready.Value.WasMutated.Should().BeFalse(); + + var activated = await ActivateAsync(container, created.Id).ConfigureAwait(false); + activated.IsSuccess.Should().BeTrue(); + activated.Value.WasMutated.Should().BeTrue(); + activated.Value.Value.IsInactive.Should().BeFalse(); + activated.Value.Value.ETag.Should().NotBe(created.ETag); + + var activatedAgain = await ActivateAsync(container, created.Id).ConfigureAwait(false); + activatedAgain.Value.WasMutated.Should().BeFalse(); + + var deactivated = await DeactivateAsync(container, created.Id).ConfigureAwait(false); + deactivated.Value.WasMutated.Should().BeTrue(); + deactivated.Value.Value.IsInactive.Should().BeTrue(); + } + + [Test] + public async Task ActivateAsync_NotFound_Fails() + { + var container = await GetContainerAsync().ConfigureAwait(false); + + var result = await ActivateAsync(container, NewId()).ConfigureAwait(false); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + } + + [Test] + public async Task DeleteAsync_Active_Fails() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var created = (await CreateAsync(container, NewId()).ConfigureAwait(false)).Value.Value; + await ActivateAsync(container, created.Id).ConfigureAwait(false); + + var result = await DeleteAsync(container, created.Id).ConfigureAwait(false); + + result.IsFailure.Should().BeTrue(); + (await container.GetAsync(CompositeKey.Create(created.Id)).ConfigureAwait(false)).Should().NotBeNull(); + } + + [Test] + public async Task DeleteAsync_Inactive_Succeeds_AndMissingIsNotMutated() + { + var container = await GetContainerAsync().ConfigureAwait(false); + var created = (await CreateAsync(container, NewId()).ConfigureAwait(false)).Value.Value; + + var result = await DeleteAsync(container, created.Id).ConfigureAwait(false); + result.IsSuccess.Should().BeTrue(); + result.Value.WasMutated.Should().BeTrue(); + (await container.GetAsync(CompositeKey.Create(created.Id)).ConfigureAwait(false)).Should().BeNull(); + + var again = await DeleteAsync(container, created.Id).ConfigureAwait(false); + again.IsSuccess.Should().BeTrue(); + again.Value.WasMutated.Should().BeFalse(); + } + + [Test] + public async Task CreateAsync_WithinUnitOfWork_PersistsOnCommit() + { + await EnsureContainerAsync().ConfigureAwait(false); + var cosmosDb = CreateCosmosDb(); + var container = cosmosDb.Container(ContainerId, o => o.WithTypeDiscriminator()); + var unitOfWork = new CosmosDbUnitOfWork(cosmosDb); + + var result = await unitOfWork.TransactionAsync(_ => CreateAsync(container, NewId())).ConfigureAwait(false); + + result.IsSuccess.Should().BeTrue(); + var model = await container.GetAsync(CompositeKey.Create(result.Value.Value.Id)).ConfigureAwait(false); + model.Should().NotBeNull(); + model!.IsActive.Should().BeFalse(); + } + + [Test] + public async Task CreateAsync_WithOutbox_UniqueKeyAndReferenceDataOutboxEvent_AllowsMultipleCreatesAndRejectsDuplicate() + { + await EnsureUniqueContainerAsync(OutboxContainerId).ConfigureAwait(false); + var cosmosDb = new CosmosDb(Client, TestDatabase.Id, new CosmosDbOptions().Container(OutboxContainerId, c => c.WithReferenceDataOutboxEvent()), executionContext: new ExecutionContext { TenantId = "tenant-a" }); + + // Each create writes an outbox event into the same container/partition; none may collide with each other (or with the business documents) on the unique key. + var code = NewId(); + (await CreateWithOutboxAsync(cosmosDb, code).ConfigureAwait(false)).IsSuccess.Should().BeTrue(); + (await CreateWithOutboxAsync(cosmosDb, NewId()).ConfigureAwait(false)).IsSuccess.Should().BeTrue(); + (await CreateWithOutboxAsync(cosmosDb, NewId()).ConfigureAwait(false)).IsSuccess.Should().BeTrue(); + + // The duplicate code is atomically rejected by the unique key. + var duplicate = await CreateWithOutboxAsync(cosmosDb, code).ConfigureAwait(false); + duplicate.IsFailure.Should().BeTrue(); + duplicate.Error.Should().BeOfType(); + } + + [Test] + public async Task CreateAsync_WithOutbox_UniqueKeyWithoutReferenceDataOutboxEvent_FailsFastWithClearError() + { + // Outbox events lack /typeDiscriminator and /code so would all share the same (null, null) unique key (and the second would fail with a misleading duplicate); the publisher detects this up front. + await EnsureUniqueContainerAsync(NoCodeOutboxContainerId).ConfigureAwait(false); + var cosmosDb = new CosmosDb(Client, TestDatabase.Id, new CosmosDbOptions(), executionContext: new ExecutionContext { TenantId = "tenant-a" }); + + var act = () => CreateWithOutboxAsync(cosmosDb, NewId(), NoCodeOutboxContainerId); + + var thrown = await act.Should().ThrowAsync(); + thrown.Which.Message.Should().Contain("unique key").And.Contain(nameof(CosmosDbContainerOptions.WithReferenceDataOutboxEvent)); + } +} diff --git a/tests/CoreEx.Cosmos.Test.Unit/CosmosTestBase.cs b/tests/CoreEx.Cosmos.Test.Unit/CosmosTestBase.cs index 36e0e5954..a7ec522e6 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/CosmosTestBase.cs +++ b/tests/CoreEx.Cosmos.Test.Unit/CosmosTestBase.cs @@ -16,9 +16,17 @@ public abstract class CosmosTestBase /// /// Gets the shared (Gateway mode, pointed at the local emulator, accepting its self-signed certificate). /// - protected static CosmosClient Client => _client ??= new CosmosClient(Endpoint, Key, new CosmosClientOptions + protected static CosmosClient Client => _client ??= CreateClient(); + + /// + /// Creates a new, independent (own metadata caches) configured as per ; the caller owns (and must dispose) it. + /// + /// Whether to enable the SDK's distributed tracing for telemetry assertions. + /// Useful where a test drops/replaces databases or containers, as a client that has cached the replaced resource can otherwise surface stale-cache failures. + protected static CosmosClient CreateClient(bool enableTracing = false) => new(Endpoint, Key, new CosmosClientOptions { ConnectionMode = ConnectionMode.Gateway, + CosmosClientTelemetryOptions = new CosmosClientTelemetryOptions { DisableDistributedTracing = !enableTracing }, HttpClientFactory = () => new HttpClient(new HttpClientHandler { ServerCertificateCustomValidationCallback = HttpClientHandler.DangerousAcceptAnyServerCertificateValidator }), // CosmosDbModelBase uses System.Text.Json's [JsonPropertyName] to map the id/_etag/ttl reserved properties; the SDK's default serializer is Newtonsoft.Json-based and would not honour those // attributes, so opt into the SDK's System.Text.Json serializer explicitly (camelCase for everything else, matching typical Cosmos DB document conventions). @@ -76,10 +84,8 @@ public async Task EnsureAvailableOrIgnoreAsync() /// /// Creates (if not already existing) a test container with the specified and (defaults to /partitionKey). /// - /// The local emulator occasionally responds with a transient 503 ServiceUnavailable ("high demand") when several containers are created in quick succession; a short retry-with-backoff - /// smooths over this. Note: this exact response is also what the emulator returns when its AZURE_COSMOS_EMULATOR_PARTITION_COUNT (the cap on the total number of containers it can host, not - /// "partitions per container") has been exhausted - that failure mode is deterministic, not transient, and no amount of retrying fixes it (confirmed the hard way); see the setting's own comment in - /// docker-compose.yml. If this retry starts failing consistently for a new container, check whether the count needs raising before assuming it is another transient blip. + /// The local emulator can respond with a transient 503 ServiceUnavailable when containers are created in quick succession (the legacy emulator image also returned the same response, deterministically, once its ~35 container cap was exhausted); a short retry-with-backoff + /// smooths over this. protected static async Task GetOrCreateContainerAsync(string id, string partitionKeyPath = "/partitionKey") { for (var attempt = 1; ; attempt++) diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert-plain.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert-plain.seed.yaml new file mode 100644 index 000000000..9ffe58ce4 --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert-plain.seed.yaml @@ -0,0 +1,3 @@ +# Insert: no '$' prefix. +prov-items: + - { id: prov-1, partitionKey: pk, name: One } diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert.seed.yaml new file mode 100644 index 000000000..d61abb57d --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-insert.seed.yaml @@ -0,0 +1,4 @@ +# Insert: no '$' prefix (the '^' generates the identifier). +prov-ref: + - ^Colour: + - R: Red diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-merge.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-merge.seed.yaml new file mode 100644 index 000000000..9f97932cf --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-merge.seed.yaml @@ -0,0 +1,8 @@ +# Merge (upsert): '$' prefixed container key and type group. +$prov-items: + - { id: prov-1, partitionKey: pk, name: One } + +prov-ref: + - $^Colour: + - R: Red + - G: Green diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-param.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-param.seed.yaml new file mode 100644 index 000000000..ede57dbce --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-param.seed.yaml @@ -0,0 +1,3 @@ +# Parameter supplied via CosmosDbProvisionArgs.Parameters. +$prov-items: + - { id: param-1, partitionKey: pk, name: ^Who } diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-unknown.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-unknown.seed.yaml new file mode 100644 index 000000000..4a2c6bca2 --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning-unknown.seed.yaml @@ -0,0 +1,2 @@ +unknown-container: + - { id: x } diff --git a/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning.seed.yaml b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning.seed.yaml new file mode 100644 index 000000000..6b27248df --- /dev/null +++ b/tests/CoreEx.Cosmos.Test.Unit/Data/provisioning.seed.yaml @@ -0,0 +1,11 @@ +# Plain container (partitionKey-bearing documents) and a type-discriminated reference data container ($^TypeName shorthand), keyed by container id. +prov-items: + - { id: prov-1, partitionKey: pk, name: One } + - { id: prov-2, partitionKey: pk, name: Two } + +prov-ref: + - $^Colour: + - R: Red + - G: Green + - $^Shape: + - SQ: Square diff --git a/tests/CoreEx.Cosmos.Test.Unit/GlobalUsing.cs b/tests/CoreEx.Cosmos.Test.Unit/GlobalUsing.cs index afa479bf5..d4a35356c 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/GlobalUsing.cs +++ b/tests/CoreEx.Cosmos.Test.Unit/GlobalUsing.cs @@ -2,6 +2,7 @@ global using CoreEx.Cosmos; global using CoreEx.Cosmos.Extended; global using CoreEx.Cosmos.Outbox; +global using CoreEx.Cosmos.Provisioning; global using CoreEx.Data; global using CoreEx.Data.Json; global using CoreEx.Entities; @@ -9,6 +10,7 @@ global using CoreEx.Events.Publishing; global using CoreEx.Hosting; global using CoreEx.Mapping; +global using CoreEx.RefData; global using CoreEx.Results; global using Microsoft.Azure.Cosmos; global using Microsoft.Azure.Cosmos.Linq; @@ -16,5 +18,9 @@ global using Microsoft.Extensions.DependencyInjection; global using Microsoft.Extensions.Logging; global using Microsoft.Extensions.Logging.Abstractions; -global using System.Net; +global using OpenTelemetry; +global using OpenTelemetry.Trace; global using PartitionKey = Microsoft.Azure.Cosmos.PartitionKey; +global using System.Collections.Concurrent; +global using System.Diagnostics; +global using System.Net; diff --git a/tests/CoreEx.Cosmos.Test.Unit/TestModels.cs b/tests/CoreEx.Cosmos.Test.Unit/TestModels.cs index ecf84fd08..9e52f0f76 100644 --- a/tests/CoreEx.Cosmos.Test.Unit/TestModels.cs +++ b/tests/CoreEx.Cosmos.Test.Unit/TestModels.cs @@ -210,3 +210,42 @@ private sealed class FromMapper : IMapper : new TestValue { Id = source.Id, Name = source.Name, ETag = source.ETag }; } } + +/// +/// A reference data model (stored using the shared type discriminator) used to exercise . +/// +[Schemas.Schema(Name = nameof(TestRefItem))] +public class TestRefItem : CosmosDbReferenceDataModelBase, ITypeDiscriminator +{ + public string? TypeDiscriminator { get; set; } +} + +/// +/// A reference data "contract" mapped to/from . +/// +public class TestRef : ReferenceData { } + +/// +/// A hand-written between and . +/// +public class TestRefMapper : IBiDirectionMapper +{ + public IMapper To { get; } = new ToMapper(); + + public IMapper From { get; } = new FromMapper(); + + private sealed class ToMapper : IMapper + { + // The identifier, active state and (on update) the code are intentionally not mapped back as they are managed by CosmosDbReferenceData. + public TestRefItem? Map(TestRef? source) => source is null + ? null + : new TestRefItem { Code = source.Code!, Text = source.GetText(), Description = source.GetDescription(), SortOrder = source.SortOrder, StartsOn = source.StartsOn, EndsOn = source.EndsOn, ETag = source.ETag }; + } + + private sealed class FromMapper : IMapper + { + public TestRef? Map(TestRefItem? source) => source is null + ? null + : new TestRef { Id = source.Id, Code = source.Code, Text = source.Text, Description = source.Description, SortOrder = source.SortOrder, IsInactive = !source.IsActive, StartsOn = source.StartsOn, EndsOn = source.EndsOn, ETag = source.ETag }; + } +} diff --git a/tests/CoreEx.Database.Postgres.Test.Unit/EntityFrameworkCrudTests.Update.cs b/tests/CoreEx.Database.Postgres.Test.Unit/EntityFrameworkCrudTests.Update.cs index c7fd2d11b..2104459aa 100644 --- a/tests/CoreEx.Database.Postgres.Test.Unit/EntityFrameworkCrudTests.Update.cs +++ b/tests/CoreEx.Database.Postgres.Test.Unit/EntityFrameworkCrudTests.Update.cs @@ -125,6 +125,51 @@ public void Update_Success_Detached() => Test.ScopedType(test ObjectComparer.Assert(u.Value, r); }).AssertSuccess()); + [Test] + public void Update_Success_FreshModel_PreservesCreated() => Test.ScopedType(test => test.Run(async _ => + { + var id = 8.ToGuid(); + var ef = ExecutionContext.GetRequiredService(); + var dc = ExecutionContext.GetRequiredService(); + + var e = await ef.Table.GetAsync(id).ConfigureAwait(false); + e.Should().NotBeNull(); + e.CreatedBy.Should().NotBeNull(); + e.CreatedOn.Should().NotBeNull(); + var (createdBy, createdOn) = (e.CreatedBy, e.CreatedOn); + dc.ChangeTracker.Clear(); + + // A fresh (detached) model that carries no audit information, with a modified value. + var m = new TestTable { Id = e.Id, Text = e.Text + "FFF", Number = e.Number, Amount = e.Amount, Flag = e.Flag, Date = e.Date, Time = e.Time, Json = e.Json, ETag = e.ETag, TenantId = e.TenantId }; + var u = await ef.Table.UpdateAsync(m).ConfigureAwait(false); + u.WasMutated.Should().BeTrue(); + + dc.ChangeTracker.Clear(); + var r = await ef.Table.GetAsync(id).ConfigureAwait(false); + r.Should().NotBeNull(); + r.Text.Should().Be(m.Text); + r.CreatedBy.Should().Be(createdBy); + r.CreatedOn.Should().Be(createdOn); + r.UpdatedBy.Should().NotBeNull(); + r.UpdatedOn.Should().NotBeNull(); + }).AssertSuccess()); + + [Test] + public void Update_Success_FreshModel_NoChange() => Test.ScopedType(test => test.Run(async _ => + { + var id = 8.ToGuid(); + var ef = ExecutionContext.GetRequiredService(); + var dc = ExecutionContext.GetRequiredService(); + + var e = await ef.Table.GetAsync(id).ConfigureAwait(false); + e.Should().NotBeNull(); + dc.ChangeTracker.Clear(); + + // A fresh (detached) model that carries no audit information, with no modified values; the audit information must not be seen as a change. + var m = new TestTable { Id = e.Id, Text = e.Text, Number = e.Number, Amount = e.Amount, Flag = e.Flag, Date = e.Date, Time = e.Time, Json = e.Json, ETag = e.ETag, TenantId = e.TenantId }; + var u = await ef.Table.UpdateAsync(m).ConfigureAwait(false); + u.WasMutated.Should().BeFalse(); + }).AssertSuccess()); [Test] public void Update_Mapped_Success() => Test.ScopedType(test => test.Run(async _ => { diff --git a/tests/CoreEx.Database.SqlServer.Test.Unit/EntityFrameworkCrudTests.Update.cs b/tests/CoreEx.Database.SqlServer.Test.Unit/EntityFrameworkCrudTests.Update.cs index cedac5600..e8c02271f 100644 --- a/tests/CoreEx.Database.SqlServer.Test.Unit/EntityFrameworkCrudTests.Update.cs +++ b/tests/CoreEx.Database.SqlServer.Test.Unit/EntityFrameworkCrudTests.Update.cs @@ -125,6 +125,51 @@ public void Update_Success_Detached() => Test.ScopedType(test ObjectComparer.Assert(u.Value, r); }).AssertSuccess()); + [Test] + public void Update_Success_FreshModel_PreservesCreated() => Test.ScopedType(test => test.Run(async _ => + { + var id = 8.ToGuid(); + var ef = ExecutionContext.GetRequiredService(); + var dc = ExecutionContext.GetRequiredService(); + + var e = await ef.Table.GetAsync(id).ConfigureAwait(false); + e.Should().NotBeNull(); + e.CreatedBy.Should().NotBeNull(); + e.CreatedOn.Should().NotBeNull(); + var (createdBy, createdOn) = (e.CreatedBy, e.CreatedOn); + dc.ChangeTracker.Clear(); + + // A fresh (detached) model that carries no audit information, with a modified value. + var m = new TestTable { Id = e.Id, Text = e.Text + "FFF", Number = e.Number, Amount = e.Amount, Flag = e.Flag, Date = e.Date, Time = e.Time, KvpJson = e.KvpJson, ETag = e.ETag, TenantId = e.TenantId }; + var u = await ef.Table.UpdateAsync(m).ConfigureAwait(false); + u.WasMutated.Should().BeTrue(); + + dc.ChangeTracker.Clear(); + var r = await ef.Table.GetAsync(id).ConfigureAwait(false); + r.Should().NotBeNull(); + r.Text.Should().Be(m.Text); + r.CreatedBy.Should().Be(createdBy); + r.CreatedOn.Should().Be(createdOn); + r.UpdatedBy.Should().NotBeNull(); + r.UpdatedOn.Should().NotBeNull(); + }).AssertSuccess()); + + [Test] + public void Update_Success_FreshModel_NoChange() => Test.ScopedType(test => test.Run(async _ => + { + var id = 8.ToGuid(); + var ef = ExecutionContext.GetRequiredService(); + var dc = ExecutionContext.GetRequiredService(); + + var e = await ef.Table.GetAsync(id).ConfigureAwait(false); + e.Should().NotBeNull(); + dc.ChangeTracker.Clear(); + + // A fresh (detached) model that carries no audit information, with no modified values; the audit information must not be seen as a change. + var m = new TestTable { Id = e.Id, Text = e.Text, Number = e.Number, Amount = e.Amount, Flag = e.Flag, Date = e.Date, Time = e.Time, KvpJson = e.KvpJson, ETag = e.ETag, TenantId = e.TenantId }; + var u = await ef.Table.UpdateAsync(m).ConfigureAwait(false); + u.WasMutated.Should().BeFalse(); + }).AssertSuccess()); [Test] public void Update_Mapped_Success() => Test.ScopedType(test => test.Run(async _ => { diff --git a/tests/CoreEx.Database.Test.Unit/DatabaseInvokerRetryTests.cs b/tests/CoreEx.Database.Test.Unit/DatabaseInvokerRetryTests.cs new file mode 100644 index 000000000..1d483d508 --- /dev/null +++ b/tests/CoreEx.Database.Test.Unit/DatabaseInvokerRetryTests.cs @@ -0,0 +1,196 @@ +using CoreEx.Database.Abstractions; +using CoreEx.Database.Extended; +using CoreEx.Database.SqlServer; +using CoreEx.Entities; +using CoreEx.Hosting; +using CoreEx.Mapping.Converters.Abstractions; +using CoreEx.Results; +using Microsoft.Extensions.Logging; +using Microsoft.Extensions.Logging.Abstractions; +using Polly; +using System.Data.Common; +using System.Text.Json; + +namespace CoreEx.Database.Test.Unit; + +[TestFixture] +public class DatabaseInvokerRetryTests +{ + private static DatabaseInvoker CreateInvoker() => new TestInvoker(); + + private static ResiliencePipeline CreateFastCustomPipeline(int maxRetryAttempts = 3) + => RetryResiliency.Create(r => r.IsFailure, _ => NullLogger.Instance, delay: TimeSpan.FromMilliseconds(1), maxRetryAttempts: maxRetryAttempts); + + [Test] + public async Task RetryOnTransient_DefaultsToFalse_TransientException_IsNotRetried() + { + // Even though the exception is classified as transient, RetryOnTransient is false (default) so it must propagate on the very first attempt. + var database = new FakeDatabase(isTransient: true); + var attempts = 0; + + Func act = () => CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = false }, (_, _, _) => + { + attempts++; + throw new TestDbException("boom"); + }, CancellationToken.None); + + await act.Should().ThrowAsync().ConfigureAwait(false); + attempts.Should().Be(1); + } + + [Test] + public async Task RetryOnTransient_True_NonTransientException_IsNeverRetried() + { + // Classification says "not transient" so, even opted in, it must never be retried - it bypasses the pipeline entirely and propagates unwrapped. + var database = new FakeDatabase(isTransient: false); + var attempts = 0; + + Func act = () => CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = false, RetryOnTransient = true, RetryResiliencePipeline = CreateFastCustomPipeline() }, (_, _, _) => + { + attempts++; + throw new TestDbException("boom"); + }, CancellationToken.None); + + await act.Should().ThrowAsync().ConfigureAwait(false); + attempts.Should().Be(1); + } + + [Test] + public async Task RetryOnTransient_True_CustomPipeline_RetriesTransientException_UntilSuccess() + { + var database = new FakeDatabase(isTransient: true); + var attempts = 0; + + var result = await CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = false, RetryOnTransient = true, RetryResiliencePipeline = CreateFastCustomPipeline() }, (_, _, _) => + { + attempts++; + if (attempts < 3) + throw new TestDbException("boom"); + + return Task.FromResult(42); + }, CancellationToken.None).ConfigureAwait(false); + + result.Should().Be(42); + attempts.Should().Be(3); + } + + [Test] + public async Task RetryOnTransient_True_CustomPipeline_ExhaustsRetries_ThenThrows() + { + var database = new FakeDatabase(isTransient: true); + var attempts = 0; + + Func act = () => CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = false, RetryOnTransient = true, RetryResiliencePipeline = CreateFastCustomPipeline(maxRetryAttempts: 2) }, (_, _, _) => + { + attempts++; + throw new TestDbException("boom"); + }, CancellationToken.None); + + // ExceptionDispatchInfo preserves both the original exception type and its stack trace (no AggregateException wrapping) so the exhausted-retry outcome is indistinguishable, from the + // caller's perspective, from a single failed attempt - and so it can still be pattern-matched by the outer TransformException catch (see the dedicated test below). + var ex = await act.Should().ThrowAsync().ConfigureAwait(false); + ex.Which.Message.Should().Be("boom"); + attempts.Should().Be(3); // The initial attempt plus 2 retries. + } + + [Test] + public async Task RetryOnTransient_True_NoCustomPipeline_UsesDefault_RetriesTransientException_UntilSuccess() + { + // No RetryResiliencePipeline supplied - exercises the real DatabaseInvokerResiliency default (cached per-TResult) pipeline; a single retry keeps this within a reasonable runtime + // despite the default's non-trivial (2s+) backoff. + var database = new FakeDatabase(isTransient: true); + var attempts = 0; + + var result = await CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = false, RetryOnTransient = true }, (_, _, _) => + { + attempts++; + if (attempts < 2) + throw new TestDbException("boom"); + + return Task.FromResult("ok"); + }, CancellationToken.None).ConfigureAwait(false); + + result.Should().Be("ok"); + attempts.Should().Be(2); + } + + [Test] + public async Task RetryOnTransient_True_ExhaustedRetry_StillAppliesOuterTransformException() + { + // Confirms the pre-existing TransformException conversion layer remains the outermost handler, applied to whatever the (possibly-retried) exception ends up being. + var database = new FakeDatabase(isTransient: true, handleDbException: _ => new BusinessException("converted")); + + Func act = () => CreateInvoker().InvokeAsync(database, new SqlServerDatabaseArgs { TransformException = true, RetryOnTransient = true, RetryResiliencePipeline = CreateFastCustomPipeline(maxRetryAttempts: 1) }, (_, _, _) => + throw new TestDbException("boom"), CancellationToken.None); + + await act.Should().ThrowAsync().WithMessage("converted").ConfigureAwait(false); + } + + [Test] + public async Task RetryOnTransient_True_ExhaustedRetry_ResultShapedTResult_ConvertsDirectly_WithoutThrowing() + { + // Where TResult is itself an IResult (ROP) and TransformException converts the exhausted-retry exception, the failure must come back as a Result directly - with NO exception ever + // thrown out of InvokeAsync at all - rather than round-tripping through a throw/catch purely to reapply a conversion already known here. + var database = new FakeDatabase(isTransient: true, handleDbException: _ => new NotFoundException()); + var attempts = 0; + + Result result = await CreateInvoker().InvokeAsync>(database, new SqlServerDatabaseArgs { TransformException = true, RetryOnTransient = true, RetryResiliencePipeline = CreateFastCustomPipeline(maxRetryAttempts: 1) }, (_, _, _) => + { + attempts++; + throw new TestDbException("boom"); + }, CancellationToken.None).ConfigureAwait(false); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeOfType(); + attempts.Should().Be(2); // The initial attempt plus 1 retry. + } + + private sealed class TestInvoker : DatabaseInvoker; + + private sealed class TestDbException(string message) : DbException(message); + + private sealed class FakeDatabase(bool isTransient, Func? handleDbException = null) : IDatabase + { + public ILogger? Logger => null; + + public DatabaseInvoker Invoker => throw new NotSupportedException(); + + public DatabaseArgs DbArgs => throw new NotSupportedException(); + + public string DatabaseId => "fake"; + + public DateTimeTransform DateTimeTransform { get; set; } + + public bool DateTimeOffsetTransform { get; set; } + + public DatabaseColumns NamedColumns => throw new NotSupportedException(); + + public DatabaseWildcard Wildcard { get; set; } = new(); + + public ISourceConverter RowVersionConverter => throw new NotSupportedException(); + + public JsonSerializerOptions JsonSerializerOptions => throw new NotSupportedException(); + + public DbTransaction? CurrentTransaction => null; + + public bool IsInTransaction => false; + + public void UseTransaction(DbTransaction? transaction) => throw new NotSupportedException(); + + public event EventHandler? UseTransactionChanged { add { } remove { } } + + public DbConnection Connection => throw new NotSupportedException(); + + public Task GetConnectionAsync(CancellationToken cancellationToken = default) => throw new NotSupportedException(); + + public DatabaseCommand Statement(SqlStatement statement) => throw new NotSupportedException(); + + public Exception? HandleDbException(DbException dbex) => handleDbException?.Invoke(dbex); + + public bool IsTransientException(Exception exception) => isTransient; + + public DbParameter CreateParameter() => throw new NotSupportedException(); + + public string GetNextSavePointName() => throw new NotSupportedException(); + } +} diff --git a/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayBaseTests.cs b/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayBaseTests.cs index d3a7b0a29..a1512d074 100644 --- a/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayBaseTests.cs +++ b/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayBaseTests.cs @@ -13,12 +13,12 @@ public class DatabaseOutboxRelayBaseTests { private static SqlServerDatabase CreateDatabase() => new((SqlConnection)SqlClientFactory.Instance.CreateConnection()); - private static DatabaseOutboxRelayArgs CreateArgs(DatabaseOutboxRelayResiliencyExecutor? resiliencyExecutor = null) => new() + private static DatabaseOutboxRelayArgs CreateArgs(DatabaseOutboxRelayResiliencyExecutor? resiliencyExecutor = null, TimeSpan? leaseDuration = null) => new() { // partitionSize == perWorkerPartitionCount triggers PartitionPicker's "probe all partitions" path - deterministic, covers every partition every call. PartitionPicker = new PartitionPicker(partitionSize: 2, perWorkerPartitionCount: 2), BatchSize = 10, - LeaseDuration = TimeSpan.FromSeconds(5), + LeaseDuration = leaseDuration ?? TimeSpan.FromSeconds(5), BackOffDuration = TimeSpan.FromSeconds(1), ResiliencyExecutor = resiliencyExecutor ?? ((work, ct) => work(ct)) }; @@ -96,14 +96,52 @@ public async Task RelayAsync_EmitsPerEventRelayMarker_ParentedToOriginatingTrace markers[0].GetTagItem("outbox.destination").Should().Be("test-destination"); } + [Test] + public async Task RelayAsync_LeaseDurationExceeded_ReleasesClaimAndContinues() + { + // Regression: a publish exceeding the lease duration (surfacing as a nested AggregateException-wrapped cancellation) must release the claim using a live token, and must not abort the tick. + var relay = new TestOutboxRelay(CreateDatabase(), new NoOpEventPublisher()) + { + EventsForPartition = _ => [new DestinationEvent("test-destination", new CloudNative.CloudEvents.CloudEvent { Id = Guid.NewGuid().ToString(), Type = "test.event", Source = new Uri("urn:test") })], + WaitForLeaseExpiry = true, + PublishException = ct => new AggregateException(new AggregateException(new TaskCanceledException(null, null, ct))) + }; + + await relay.RelayAsync(CreateArgs(leaseDuration: TimeSpan.FromMilliseconds(50)), CancellationToken.None); + + relay.AttemptedPartitions.Should().BeEquivalentTo([0, 1]); + relay.CancelBatchTokenWasCanceled.Should().BeEquivalentTo([false, false]); + } + + [Test] + public async Task RelayAsync_CallerCancellation_Rethrows() + { + using var cts = new CancellationTokenSource(); + var relay = new TestOutboxRelay(CreateDatabase(), new NoOpEventPublisher()) + { + EventsForPartition = _ => [new DestinationEvent("test-destination", new CloudNative.CloudEvents.CloudEvent { Id = Guid.NewGuid().ToString(), Type = "test.event", Source = new Uri("urn:test") })], + PublishException = _ => { cts.Cancel(); return new AggregateException(new OperationCanceledException(cts.Token)); } + }; + + var act = () => relay.RelayAsync(CreateArgs(), cts.Token); + await act.Should().ThrowAsync(); + relay.AttemptedPartitions.Should().ContainSingle(); + } + private sealed class TestOutboxRelay(SqlServerDatabase database, IEventPublisher eventPublisher) : DatabaseOutboxRelayBase(database, eventPublisher) { public List AttemptedPartitions { get; } = []; + public List CancelBatchTokenWasCanceled { get; } = []; + public int? FailingPartitionId { get; set; } public Func>? EventsForPartition { get; set; } + public Func? PublishException { get; set; } + + public bool WaitForLeaseExpiry { get; set; } + public override void SetStatementsByConvention(string? schema = null) { } protected override Task> ClaimNextBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, int partitionId, CancellationToken cancellationToken) @@ -118,10 +156,25 @@ protected override Task> ClaimNextBatchAsync(DatabaseOutb return Task.FromResult(EventsForPartition?.Invoke(partitionId) ?? []); } - // No-op - avoids requiring a real database connection for tests that DO claim events (CreateDatabase() has no live connection); the claim/publish path is exercised via - // ClaimNextBatchAsync/EventPublisher instead, which is all these tests care about. - protected override Task CompleteBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) => Task.CompletedTask; + // Simulates a publish failure (CompleteBatchAsync runs inside the same try as the publish); otherwise a no-op - avoids requiring a real database connection for tests that DO claim events + // (CreateDatabase() has no live connection); the claim/publish path is exercised via ClaimNextBatchAsync/EventPublisher instead, which is all these tests care about. + protected override async Task CompleteBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) + { + if (PublishException is null) + return; + + if (WaitForLeaseExpiry) + await Task.Delay(Timeout.Infinite, cancellationToken).ContinueWith(_ => { }, TaskScheduler.Default); + + throw PublishException(cancellationToken); + } - protected override Task CancelBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) => Task.CompletedTask; + protected override Task CancelBatchAsync(DatabaseOutboxRelayArgs args, Guid leaseId, CancellationToken cancellationToken) + { + lock (CancelBatchTokenWasCanceled) + CancelBatchTokenWasCanceled.Add(cancellationToken.IsCancellationRequested); + + return Task.CompletedTask; + } } } diff --git a/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayHostedServiceBaseTests.cs b/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayHostedServiceBaseTests.cs index 13e9c4e76..aa82a6c57 100644 --- a/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayHostedServiceBaseTests.cs +++ b/tests/CoreEx.Database.Test.Unit/Outbox/DatabaseOutboxRelayHostedServiceBaseTests.cs @@ -1,5 +1,6 @@ using CoreEx.Database.Outbox; using CoreEx.Database.SqlServer; +using CoreEx.Database.SqlServer.Outbox; using CoreEx.Events.Publishing; using CoreEx.Hosting; using Microsoft.Data.SqlClient; @@ -48,6 +49,77 @@ public async Task DefaultSettings_DoNotThrowOnStart() await svc.StopAsync(CancellationToken.None); } + [Test] + public async Task SharedPartitionSize_IsUsedAndReported() + { + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection( + [ + new(DatabaseOutboxConfiguration.PartitionSizeConfigurationKey, "2"), + new("CoreEx:Host:Services:OutboxRelay:PerWorkerPartitionCount", "1") + ]) + .Build(); + var sc = new ServiceCollection(); + sc.AddSingleton(configuration); + sc.AddExecutionContext(); + using var sp = sc.BuildServiceProvider(); + + var svc = new TestOutboxRelayHostedService(sp, NullLogger.Instance) + { + RelayFactory = _ => new TestOutboxRelay(CreateDatabase(), new NoOpEventPublisher(), new TestRelayState { ThrowAlways = false }) + }; + + await svc.StartAsync(CancellationToken.None); + try + { + svc.PartitionSize.Should().Be(2); + var data = new Dictionary(); + svc.GetHealthStatus(data); + ((Dictionary)data["OutboxRelay"])["PartitionSize"].Should().Be(2); + } + finally + { + await svc.StopAsync(CancellationToken.None); + } + } + + [Test] + public void LegacyPartitionSize_FailsAtStartup() + { + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection([new("CoreEx:Host:Services:OutboxRelay:PartitionSize", "2")]) + .Build(); + var sc = new ServiceCollection(); + sc.AddSingleton(configuration); + sc.AddExecutionContext(); + using var sp = sc.BuildServiceProvider(); + + var svc = new TestOutboxRelayHostedService(sp, NullLogger.Instance) + { + RelayFactory = _ => new TestOutboxRelay(CreateDatabase(), new NoOpEventPublisher(), new TestRelayState { ThrowAlways = false }) + }; + + var ex = Assert.ThrowsAsync(async () => await svc.StartAsync(CancellationToken.None)); + ex!.Message.Should().Contain(DatabaseOutboxConfiguration.PartitionSizeConfigurationKey); + } + + [Test] + public void SqlServerOutboxPublisher_UsesSharedPartitionSize() + { + var configuration = new ConfigurationBuilder() + .AddInMemoryCollection([new(DatabaseOutboxConfiguration.PartitionSizeConfigurationKey, "3")]) + .Build(); + var services = new ServiceCollection(); + services.AddSingleton(configuration); + services.AddLogging(); + services.AddScoped(_ => CreateDatabase()); + services.AddSqlServerOutboxPublisher((_, publisher) => publisher.PartitionSize = 8); + using var sp = services.BuildServiceProvider(); + + var publisher = (SqlServerOutboxPublisher)sp.GetRequiredKeyedService(SqlServerOutboxPublisher.DefaultServiceKey); + publisher.PartitionSize.Should().Be(3); + } + [Test] public async Task Relay_CircuitBreaker_TripsOnSustainedFailure_ThenSelfRecovers() { @@ -58,7 +130,7 @@ public async Task Relay_CircuitBreaker_TripsOnSustainedFailure_ThenSelfRecovers( var configuration = new ConfigurationBuilder() .AddInMemoryCollection( [ - new("CoreEx:Host:Services:OutboxRelay:PartitionSize", "1"), + new(DatabaseOutboxConfiguration.PartitionSizeConfigurationKey, "1"), new("CoreEx:Host:Services:OutboxRelay:PerWorkerPartitionCount", "1") ]) .Build(); @@ -114,5 +186,7 @@ private sealed class TestOutboxRelayHostedService : DatabaseOutboxRelayHostedSer { public TestOutboxRelayHostedService(IServiceProvider serviceProvider, ILogger logger) : base(serviceProvider, logger) => Resiliency = CreateDefaultResiliency(minimumThroughput: 2, samplingDuration: TimeSpan.FromSeconds(30), breakDuration: TimeSpan.FromMilliseconds(200)); + + public void GetHealthStatus(Dictionary data) => _ = OnReportHealthStatus(data); } } diff --git a/tests/CoreEx.Events.Test.Unit/EventFormatterTests.cs b/tests/CoreEx.Events.Test.Unit/EventFormatterTests.cs index 7abef833c..682a339fc 100644 --- a/tests/CoreEx.Events.Test.Unit/EventFormatterTests.cs +++ b/tests/CoreEx.Events.Test.Unit/EventFormatterTests.cs @@ -110,6 +110,53 @@ public void Format_Title_From_HostSettings() result.Source.Should().Be(new Uri("urn:coreex:test:dom")); } + [Test] + public void Format_Event_NoDomainName_DefaultsToFormatterDomain() + { + var ef = new EventFormatter { TitlePrefix = "pre", DomainName = "dom", PartitionKeyIsRequired = false }; + ef.Format(new EventData { Entity = "ent", Action = "act" }).Title.Should().Be("pre.dom.ent.act"); + } + + [Test] + public void Format_Command_ExplicitDomainName_UsesTargetDomain() + { + var ef = new EventFormatter { TitlePrefix = "pre", DomainName = "self", PartitionKeyIsRequired = false }; + ef.Format(EventData.CreateCommand("target", "ent", "act")).Title.Should().Be("pre.target.ent.act"); + } + + [Test] + public void Format_Command_NoDomainName_SelfAddressesUsingFormatterDomain() + { + var ef = new EventFormatter { TitlePrefix = "pre", DomainName = "self", PartitionKeyIsRequired = false }; + ef.Format(new EventData { MessageType = MessageType.Command, Entity = "ent", Action = "act" }).Title.Should().Be("pre.self.ent.act"); + } + + [Test] + public void Format_Command_NoDomainNameAndNoFormatterDomain_ShouldThrowInvalidOperationException() + { + var ef = new EventFormatter { PartitionKeyIsRequired = false }; + Action act = () => ef.Format(new EventData { MessageType = MessageType.Command, Entity = "ent", Action = "act" }); + act.Should().Throw(); + } + + [Test] + public void Format_Command_NoDomainNameWithExplicitTitle_ShouldNotThrow() + { + var ef = new EventFormatter { PartitionKeyIsRequired = false }; + ef.Format(new EventData { MessageType = MessageType.Command, Title = "explicit.title.here" }).Title.Should().Be("explicit.title.here"); + } + + [Test] + public void Parse_Title_ExistingValues_ShouldNotBeOverwritten() + { + var ef = new EventFormatter(); + var ed = new EventData { Title = "dom.ent.act", DomainName = "existing" }; + var result = ef.Parse(ed); + result.DomainName.Should().Be("existing"); + result.Entity.Should().Be("ent"); + result.Action.Should().Be("act"); + } + [Test] public void Parse_Title_Full() { diff --git a/tests/CoreEx.Events.Test.Unit/Publishing/NamedDestinationProviderTests.cs b/tests/CoreEx.Events.Test.Unit/Publishing/NamedDestinationProviderTests.cs new file mode 100644 index 000000000..a00fdc38e --- /dev/null +++ b/tests/CoreEx.Events.Test.Unit/Publishing/NamedDestinationProviderTests.cs @@ -0,0 +1,304 @@ +using CoreEx.Events.Publishing; +using Microsoft.Extensions.Configuration; +using Microsoft.Extensions.DependencyInjection; + +namespace CoreEx.Events.Test.Unit.Publishing; + +[TestFixture] +public class NamedDestinationProviderTests +{ + [SetUp] + public void SetUp() => ExecutionContext.Reset(); + + [TearDown] + public void TearDown() => ExecutionContext.Reset(); + + private static void SetConfiguration(params (string Key, string Value)[] settings) + { + var config = new ConfigurationBuilder().AddInMemoryCollection(settings.Select(s => new KeyValuePair(s.Key, s.Value))).Build(); + var services = new ServiceCollection().AddSingleton(config).BuildServiceProvider(); + ExecutionContext.SetCurrent(new ExecutionContext { ServiceProvider = services }); + } + + private static void SetHostSettings(string domainName) + { + var services = new ServiceCollection().AddSingleton(new CoreEx.Hosting.HostSettings { SolutionName = "Contoso", DomainName = domainName, EnvironmentName = "Test" }).BuildServiceProvider(); + ExecutionContext.SetCurrent(new ExecutionContext { ServiceProvider = services }); + } + + #region DI + + [Test] + public void AddNamedDestinationProvider_WithDestination_ShouldRegister() + { + using var sp = new ServiceCollection().AddNamedDestinationProvider("my-topic").BuildServiceProvider(); + var provider = sp.GetRequiredService(); + provider.Should().BeOfType(); + provider.CreateNew(MessageType.Command, "products").Should().Be("my-topic-products"); + provider.CreateNew(MessageType.Event).Should().Be("my-topic"); + } + + [Test] + public void AddNamedDestinationProvider_NoDestination_ShouldUseConfiguration() + { + SetConfiguration(("CoreEx:Events:Destination", "configured-dest")); + using var sp = new ServiceCollection().AddNamedDestinationProvider().BuildServiceProvider(); + sp.GetRequiredService().CreateNew(MessageType.Event).Should().Be("configured-dest"); + } + + #endregion + + #region Properties + + [Test] + public void Defaults_ShouldHaveExpectedValues() + { + var provider = new NamedDestinationProvider(); + provider.AppendSeparatorCharacter.Should().Be('-'); + } + + [Test] + public void Default_ShouldBeSingleton() => NamedDestinationProvider.Default.Should().BeSameAs(NamedDestinationProvider.Default); + + [Test] + public void Destination_SetAndGet_ShouldReturnValue() + { + var provider = new NamedDestinationProvider { Destination = "my-destination" }; + provider.Destination.Should().Be("my-destination"); + } + + [Test] + public void Destination_SetNull_ShouldThrowArgumentNullException() + { + Action act = static () => _ = new NamedDestinationProvider { Destination = null! }; + act.Should().Throw().WithParameterName("value"); + } + + [Test] + public void Destination_SetEmpty_ShouldThrowArgumentException() + { + Action act = static () => _ = new NamedDestinationProvider { Destination = "" }; + act.Should().Throw().WithParameterName("value"); + } + + [Test] + public void Destination_NotSetNoConfiguration_ShouldFallbackToDefault() + => new NamedDestinationProvider().Destination.Should().Be("default"); + + [Test] + public void Destination_NotSetWithConfiguration_ShouldUseConfiguredValue() + { + SetConfiguration(("CoreEx:Events:Destination", "configured-dest")); + new NamedDestinationProvider().Destination.Should().Be("configured-dest"); + } + + [Test] + public void Destination_NotSetConfigurationWithoutSetting_ShouldFallbackToDefault() + { + SetConfiguration(("Other:Setting", "value")); + new NamedDestinationProvider().Destination.Should().Be("default"); + } + + [Test] + public void Destination_ExplicitlySetWithConfiguration_ShouldPreferExplicitValue() + { + SetConfiguration(("CoreEx:Events:Destination", "configured-dest")); + new NamedDestinationProvider { Destination = "explicit" }.Destination.Should().Be("explicit"); + } + + [Test] + public void Destination_ConfigurationResolvedOnce_ShouldBeCached() + { + SetConfiguration(("CoreEx:Events:Destination", "first")); + var provider = new NamedDestinationProvider(); + provider.Destination.Should().Be("first"); + + ExecutionContext.Reset(); + SetConfiguration(("CoreEx:Events:Destination", "second")); + provider.Destination.Should().Be("first"); + } + + #endregion + + #region CreateFrom(string) + + [Test] + public void CreateFrom_String_ShouldReturnSuppliedDestination() + => new NamedDestinationProvider { Destination = "fixed" }.CreateFrom("any-dest").Should().Be("any-dest"); + + [Test] + public void CreateFrom_String_DeadLetter_ShouldAppendDeadLetter() + => new NamedDestinationProvider { Destination = "fixed" }.CreateFrom("any-dest", true).Should().Be("any-dest-dead-letter"); + + [Test] + public void CreateFrom_String_DeadLetterCustomSeparator_ShouldUseSeparator() + => new NamedDestinationProvider { Destination = "fixed", AppendSeparatorCharacter = '.' }.CreateFrom("any-dest", true).Should().Be("any-dest.dead-letter"); + + #endregion + + #region CreateFrom(EventData) + + [Test] + public void CreateFrom_EventData_Null_ShouldThrowArgumentNullException() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + Action act = () => provider.CreateFrom((EventData)null!); + act.Should().Throw(); + } + + [Test] + public void CreateFrom_EventData_Event_ShouldIgnoreDomainName() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + var ed = new EventData { MessageType = MessageType.Event, DomainName = "shopping" }; + provider.CreateFrom(ed).Should().Be("fixed"); + } + + [Test] + public void CreateFrom_EventData_Event_DeadLetter_ShouldAppendDeadLetter() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + var ed = new EventData { MessageType = MessageType.Event, DomainName = "shopping" }; + provider.CreateFrom(ed, true).Should().Be("fixed-dead-letter"); + } + + [TestCase("shopping")] + [TestCase("Shopping")] + [TestCase("SHOPPING")] + public void CreateFrom_EventData_Command_ShouldAppendDomainName(string domainName) + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + var ed = new EventData { MessageType = MessageType.Command, DomainName = domainName }; + provider.CreateFrom(ed).Should().Be("fixed-shopping"); + } + + [Test] + public void CreateFrom_EventData_Command_DeadLetter_ShouldAppendDomainNameThenDeadLetter() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + var ed = new EventData { MessageType = MessageType.Command, DomainName = "shopping" }; + provider.CreateFrom(ed, true).Should().Be("fixed-shopping-dead-letter"); + } + + [Test] + public void CreateFrom_EventData_Command_NoDomainName_ShouldSelfAddressUsingHostDomain() + { + SetHostSettings("Products"); + var provider = new NamedDestinationProvider { Destination = "fixed" }; + provider.CreateFrom(new EventData { MessageType = MessageType.Command }).Should().Be("fixed-products"); + provider.CreateFrom(new EventData { MessageType = MessageType.Command }, true).Should().Be("fixed-products-dead-letter"); + } + + [Test] + public void CreateFrom_EventData_Command_NoDomainNameNoHostSettings_ShouldThrowInvalidOperationException() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + Action act = () => provider.CreateFrom(new EventData { MessageType = MessageType.Command }); + act.Should().Throw(); + } + + [Test] + public void CreateFrom_EventData_UnsupportedMessageType_ShouldThrowNotSupportedException() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + var ed = new EventData { MessageType = (MessageType)999 }; + Action act = () => provider.CreateFrom(ed); + act.Should().Throw(); + } + + #endregion + + #region CreateNew + + [Test] + public void CreateNew_DefaultParams_ShouldReturnDestination() + => new NamedDestinationProvider { Destination = "fixed" }.CreateNew().Should().Be("fixed"); + + [Test] + public void CreateNew_DefaultParams_NoDestinationSet_ShouldReturnConfiguredDestination() + { + SetConfiguration(("CoreEx:Events:Destination", "configured-dest")); + new NamedDestinationProvider().CreateNew().Should().Be("configured-dest"); + } + + [Test] + public void CreateNew_Event_ShouldIgnoreDomainName() + => new NamedDestinationProvider { Destination = "fixed" }.CreateNew(MessageType.Event, "shopping").Should().Be("fixed"); + + [Test] + public void CreateNew_Event_DeadLetter_ShouldAppendDeadLetter() + => new NamedDestinationProvider { Destination = "fixed" }.CreateNew(MessageType.Event, "shopping", true).Should().Be("fixed-dead-letter"); + + [TestCase("shopping")] + [TestCase("Shopping")] + [TestCase("SHOPPING")] + public void CreateNew_Command_WithDomainName_ShouldAppendDomainName(string domainName) + { + var provider = new NamedDestinationProvider { Destination = "Fixed" }; + provider.CreateNew(MessageType.Command, domainName).Should().Be("Fixed-shopping"); + provider.CreateNew(MessageType.Command, domainName, true).Should().Be("Fixed-shopping-dead-letter"); + } + + [Test] + public void CreateNew_Command_WithDomainName_DeadLetter_ShouldAppendDomainNameThenDeadLetter() + => new NamedDestinationProvider { Destination = "fixed" }.CreateNew(MessageType.Command, "shopping", true).Should().Be("fixed-shopping-dead-letter"); + + [TestCase(null)] + [TestCase("")] + public void CreateNew_Command_NullOrEmptyDomainName_ShouldSelfAddressUsingHostDomain(string? domainName) + { + SetHostSettings("Products"); + var provider = new NamedDestinationProvider { Destination = "fixed" }; + provider.CreateNew(MessageType.Command, domainName).Should().Be("fixed-products"); + provider.CreateNew(MessageType.Command, domainName, true).Should().Be("fixed-products-dead-letter"); + } + + [Test] + public void CreateNew_Command_ExplicitDomainName_ShouldPreferExplicitOverHostDomain() + { + SetHostSettings("Products"); + new NamedDestinationProvider { Destination = "fixed" }.CreateNew(MessageType.Command, "shopping").Should().Be("fixed-shopping"); + } + + [TestCase(null)] + [TestCase("")] + public void CreateNew_Command_NullOrEmptyDomainNameNoHostSettings_ShouldThrowInvalidOperationException(string? domainName) + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + Action act = () => provider.CreateNew(MessageType.Command, domainName); + act.Should().Throw(); + } + + [Test] + public void CreateFrom_EventData_CreateCommand_ShouldUseTargetDomain() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + provider.CreateFrom(EventData.CreateCommand("shopping", "order", "place")).Should().Be("fixed-shopping"); + } + + [Test] + public void CreateNew_Command_CustomSeparator_ShouldUseSeparatorForDomainAndDeadLetter() + { + var provider = new NamedDestinationProvider { Destination = "fixed", AppendSeparatorCharacter = '_' }; + provider.CreateNew(MessageType.Command, "shopping").Should().Be("fixed_shopping"); + provider.CreateNew(MessageType.Command, "shopping", true).Should().Be("fixed_shopping_dead-letter"); + } + + [Test] + public void CreateNew_ReplyTo_ShouldThrowNotSupportedException() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + Action act = () => provider.CreateNew(MessageType.ReplyTo); + act.Should().Throw().WithMessage("*ReplyTo*"); + } + + [Test] + public void CreateNew_UnsupportedMessageType_ShouldThrowNotSupportedException() + { + var provider = new NamedDestinationProvider { Destination = "fixed" }; + Action act = () => provider.CreateNew((MessageType)999); + act.Should().Throw().WithMessage("*999*"); + } + + #endregion +} diff --git a/tests/CoreEx.Events.Test.Unit/Subscribing/EventSubscriberBaseTests.cs b/tests/CoreEx.Events.Test.Unit/Subscribing/EventSubscriberBaseTests.cs index b1bebad09..a72176039 100644 --- a/tests/CoreEx.Events.Test.Unit/Subscribing/EventSubscriberBaseTests.cs +++ b/tests/CoreEx.Events.Test.Unit/Subscribing/EventSubscriberBaseTests.cs @@ -27,6 +27,25 @@ public async Task ReceiveAsync_CancellationMatchingOwnToken_BubblesUnclassified( result.Error.Should().BeSameAs(thrown); } + [Test] + public async Task ReceiveAsync_NestedAggregateCancellationMatchingOwnToken_BubblesUnclassified() + { + // Regression: multiple Result.ThrowOnError() boundaries each wrap the error in a new AggregateException; an own-token cancellation must still bubble up unclassified however deeply wrapped. + using var cts = new CancellationTokenSource(); + Exception? thrown = null; + + var subscriber = new TestEventSubscriber(ct => + { + thrown = new AggregateException(new AggregateException(new OperationCanceledException("shutdown", ct))); + throw thrown; + }); + + var result = await subscriber.InvokeReceiveAsync(new EventData(), null, cts.Token); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeSameAs(thrown); + } + [Test] public async Task ReceiveAsync_UnrelatedCancellation_IsClassifiedViaConfiguredRule() { diff --git a/tests/CoreEx.Events.Test.Unit/Subscribing/SubscribedManagerTests.cs b/tests/CoreEx.Events.Test.Unit/Subscribing/SubscribedManagerTests.cs index dff0728b5..fe4b0ef19 100644 --- a/tests/CoreEx.Events.Test.Unit/Subscribing/SubscribedManagerTests.cs +++ b/tests/CoreEx.Events.Test.Unit/Subscribing/SubscribedManagerTests.cs @@ -30,6 +30,26 @@ public async Task ReceiveAsync_CancellationMatchingOwnToken_BubblesUnclassified( result.Error.Should().BeSameAs(thrown); } + [Test] + public async Task ReceiveAsync_NestedAggregateCancellationMatchingOwnToken_BubblesUnclassified() + { + // Regression: multiple Result.ThrowOnError() boundaries each wrap the error in a new AggregateException; an own-token cancellation must still bubble up unclassified however deeply wrapped. + using var cts = new CancellationTokenSource(); + Exception? thrown = null; + + var subscribed = new TestSubscribed(ct => + { + thrown = new AggregateException(new AggregateException(new OperationCanceledException("shutdown", ct))); + throw thrown; + }) { ErrorHandler = new ErrorHandler().Add(ErrorHandling.DeadLetter) }; + + var (manager, executionContext, args) = CreateManager(); + var result = await manager.ReceiveAsync(executionContext, subscribed, new EventData(), args, cts.Token); + + result.IsFailure.Should().BeTrue(); + result.Error.Should().BeSameAs(thrown); + } + [Test] public async Task ReceiveAsync_UnrelatedCancellation_IsClassifiedViaConfiguredRule() { diff --git a/tests/CoreEx.RefData.Test.Unit/ReferenceDataHybridCacheTests.cs b/tests/CoreEx.RefData.Test.Unit/ReferenceDataHybridCacheTests.cs index fa696e0a1..6e6a8333e 100644 --- a/tests/CoreEx.RefData.Test.Unit/ReferenceDataHybridCacheTests.cs +++ b/tests/CoreEx.RefData.Test.Unit/ReferenceDataHybridCacheTests.cs @@ -123,6 +123,57 @@ private class TrackingReferenceDataHybridCache(IHybridCache cache) : ReferenceDa protected override void OnCreateCacheEntry(Type type, Caching.HybridCacheEntryOptions entry) => CreatedFor.Add(type); } + /// + /// An stand-in that records the generic type argument used for each call, simulating a serializing (e.g. distributed) cache implementation where the + /// generic argument determines the type the underlying serializer must construct. + /// + private class TypeRecordingHybridCache : IHybridCache + { + private readonly Dictionary _store = []; + + public List ObservedTypes { get; } = []; + + public ICacheKeyProvider KeyProvider { get; } = new DefaultCacheKeyProvider(); + + public Task<(bool Exists, T? Value)> TryGetByKeyAsync(string key, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) + { + ObservedTypes.Add(typeof(T)); + return Task.FromResult(_store.TryGetValue(key, out var value) ? (true, (T?)value) : (false, default)); + } + + public async Task GetOrCreateByKeyAsync(string key, Func> factory, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) + { + ObservedTypes.Add(typeof(T)); + if (_store.TryGetValue(key, out var existing)) + return (T)existing!; + + var value = await factory(cancellationToken).ConfigureAwait(false); + _store[key] = value; + return value; + } + + public Task GetOrDefaultByKeyAsync(string key, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => throw new NotSupportedException(); + public Task SetByKeyAsync(string key, T value, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => throw new NotSupportedException(); + public Task RemoveByKeyAsync(string key, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => throw new NotSupportedException(); + public Task RemoveByTagAsync(string tag, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => throw new NotSupportedException(); + public Task RemoveByTagAsync(IEnumerable tags, Caching.HybridCacheEntryOptions? options = null, CancellationToken cancellationToken = default) => throw new NotSupportedException(); + } + + [Test] + public async Task GetOrCreateAsync_CacheMiss_NeverUsesInterfaceAsUnderlyingCacheGenericType() + { + // This proves the fix for the FUSION [DC] deserialization bug: the slow (semaphore-protected) path must invoke the underlying IHybridCache using the concrete collection type + // (e.g. DummyRefDataCollection), never the IReferenceDataCollection interface - a serializing cache (e.g. FusionCache's distributed tier) cannot construct an interface/abstract type. + var backing = new TypeRecordingHybridCache(); + var cache = new ReferenceDataHybridCache(backing); + + var coll = await cache.GetOrCreateAsync(typeof(DummyRefDataCollection), (t, ct) => Task.FromResult(new DummyRefDataCollection { new DummyRefData { Id = 1, Code = "A" } })); + + coll.Should().BeOfType(); + backing.ObservedTypes.Should().NotContain(typeof(IReferenceDataCollection)); + backing.ObservedTypes.Should().Contain(typeof(DummyRefDataCollection)); + } + [Test] public async Task OnCreateCacheEntry_InvokedOnce_ForNewType() { @@ -142,4 +193,80 @@ public void OnCreateCacheEntry_NotInvoked_WhenOptionsPreRegistered() cache.CreatedFor.Should().BeEmpty(); } + + [Test] + public async Task RemoveAsync_WaitsForInFlightLoad_ThenEvictsStaleValue() + { + // A removal must be serialized with an in-flight load, otherwise the load completes after the removal and repopulates the cache with stale data. + var cache = new ReferenceDataHybridCache(new Caching.MemoryOnlyHybridCache()); + var started = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var callCount = 0; + + async Task Factory(Type t, CancellationToken ct) + { + callCount++; + started.TrySetResult(); + await release.Task.ConfigureAwait(false); + return new DummyRefDataCollection { new DummyRefData { Id = 1, Code = "A" } }; + } + + var load = cache.GetOrCreateAsync(typeof(DummyRefDataCollection), Factory); + await started.Task; + + var remove = cache.RemoveAsync(typeof(DummyRefDataCollection)); + await Task.Delay(100); + remove.IsCompleted.Should().BeFalse("the removal must wait for the in-flight load"); + + release.SetResult(); + await load; + await remove; + + await cache.GetOrCreateAsync(typeof(DummyRefDataCollection), (t, ct) => + { + callCount++; + return Task.FromResult(new DummyRefDataCollection { new DummyRefData { Id = 1, Code = "A" } }); + }); + + callCount.Should().Be(2, "the stale value loaded before the removal must have been evicted"); + } + + [Test] + public async Task RemoveAsync_WaitsForLoadFromAnotherScopedCache_ThenEvictsStaleValue() + { + // The default cache registration is scoped, so independent cache instances must still coordinate their load and removal. + var backing = new Caching.MemoryOnlyHybridCache(); + var loadingCache = new ReferenceDataHybridCache(backing); + var removingCache = new ReferenceDataHybridCache(backing); + var started = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously); + var callCount = 0; + + async Task Factory(Type t, CancellationToken ct) + { + Interlocked.Increment(ref callCount); + started.TrySetResult(); + await release.Task.ConfigureAwait(false); + return new DummyRefDataCollection { new DummyRefData { Id = 1, Code = "A" } }; + } + + var load = loadingCache.GetOrCreateAsync(typeof(DummyRefDataCollection), Factory); + await started.Task; + + var remove = removingCache.RemoveAsync(typeof(DummyRefDataCollection)); + var removalWaitedForLoad = !remove.IsCompleted; + + release.SetResult(); + await load; + await remove; + + await removingCache.GetOrCreateAsync(typeof(DummyRefDataCollection), (t, ct) => + { + Interlocked.Increment(ref callCount); + return Task.FromResult(new DummyRefDataCollection { new DummyRefData { Id = 1, Code = "A" } }); + }); + + removalWaitedForLoad.Should().BeTrue("removal through another scoped cache must wait for the in-flight load"); + callCount.Should().Be(2, "the stale value loaded before removal must have been evicted"); + } } diff --git a/tests/CoreEx.RefData.Test.Unit/ReferenceDataOrchestratorTests.cs b/tests/CoreEx.RefData.Test.Unit/ReferenceDataOrchestratorTests.cs index e00f54b83..0640ee613 100644 --- a/tests/CoreEx.RefData.Test.Unit/ReferenceDataOrchestratorTests.cs +++ b/tests/CoreEx.RefData.Test.Unit/ReferenceDataOrchestratorTests.cs @@ -393,6 +393,116 @@ public async Task QueryAsync_PerTypeConfig_FallsBackToDefault_ForOtherTypes() ir.Items!.Count().Should().Be(2); } + // ── InvalidateAsync ────────────────────────────────────────────────────── + + private class MutableData + { + public List Items { get; } = [new DummyRefData { Id = 1, Code = "A", Text = "Alpha" }]; + } + + private class DummyMutableProvider(MutableData data) : IReferenceDataProvider + { + public IEnumerable<(Type, Type)> Types => [(typeof(DummyRefData), typeof(DummyRefDataCollection))]; + + public Task GetAsync(Type type, CancellationToken cancellationToken = default) + { + var coll = new DummyRefDataCollection(); + foreach (var item in data.Items) + coll.Add(new DummyRefData { Id = item.Id, Code = item.Code, Text = item.Text }); + + return Task.FromResult(coll); + } + } + + [Test] + public async Task InvalidateAsync_RemovesCachedCollection_ReloadedOnNextAccess() + { + var data = new MutableData(); + var sc = new ServiceCollection(); + sc.AddExecutionContext(sp => new ExecutionContext { ServiceProvider = sp }); + sc.AddSingleton(new ReferenceDataHybridCache(new Caching.MemoryOnlyHybridCache())); + sc.AddSingleton(data); + sc.AddScoped(); + var sp = sc.BuildServiceProvider(); + + var orch = new ReferenceDataOrchestrator(sp, Mock.Of>()); + orch.Register(); + ReferenceDataOrchestrator.SetCurrent(orch); + _ = sp.GetRequiredService(); + + // Load and cache the collection. + var coll = await orch.GetByTypeAsync(); + coll!.AllItems.Should().HaveCount(1); + + // The underlying data changes; the cached collection remains until invalidated. + data.Items.Add(new DummyRefData { Id = 2, Code = "B", Text = "Beta" }); + coll = await orch.GetByTypeAsync(); + coll!.AllItems.Should().HaveCount(1); + + // Invalidate by the reference data type (not the collection type); the next access must reload. + await orch.InvalidateAsync(); + coll = await orch.GetByTypeAsync(); + coll!.AllItems.Should().HaveCount(2); + } + + [Test] + public async Task InvalidateAsync_UnregisteredType_IsNoOp() + => await ReferenceDataOrchestrator.Current.InvalidateAsync(); + + [Test] + public async Task TryInvalidateAsync_RemovesCachedCollection_ReloadedOnNextAccess() + { + var data = new MutableData(); + var sc = new ServiceCollection(); + sc.AddExecutionContext(sp => new ExecutionContext { ServiceProvider = sp }); + sc.AddSingleton(new ReferenceDataHybridCache(new Caching.MemoryOnlyHybridCache())); + sc.AddSingleton(data); + sc.AddScoped(); + var sp = sc.BuildServiceProvider(); + + var orch = new ReferenceDataOrchestrator(sp, Mock.Of>()); + orch.Register(); + ReferenceDataOrchestrator.SetCurrent(orch); + _ = sp.GetRequiredService(); + + (await orch.GetByTypeAsync())!.AllItems.Should().HaveCount(1); + data.Items.Add(new DummyRefData { Id = 2, Code = "B", Text = "Beta" }); + + (await orch.TryInvalidateAsync()).Should().BeTrue(); + (await orch.GetByTypeAsync())!.AllItems.Should().HaveCount(2); + } + + [Test] + public async Task TryInvalidateAsync_CacheFails_LogsWarningAndReturnsFalse() + { + var cache = new Mock(); + cache.Setup(c => c.RemoveAsync(It.IsAny(), It.IsAny())).ThrowsAsync(new InvalidOperationException("Redis is down.")); + + var logger = new Mock>(); + var sc = new ServiceCollection(); + sc.AddExecutionContext(sp => new ExecutionContext { ServiceProvider = sp }); + sc.AddSingleton(cache.Object); + sc.AddSingleton(new MutableData()); + sc.AddScoped(); + var sp = sc.BuildServiceProvider(); + + var orch = new ReferenceDataOrchestrator(sp, logger.Object); + orch.Register(); + ReferenceDataOrchestrator.SetCurrent(orch); + _ = sp.GetRequiredService(); + + // The strict variant surfaces the failure; the try variant swallows (and logs) it. + Func strict = () => orch.InvalidateAsync(); + await strict.Should().ThrowAsync(); + + (await orch.TryInvalidateAsync()).Should().BeFalse(); + + logger.Verify(l => l.Log(LogLevel.Warning, It.IsAny(), It.IsAny(), It.IsAny(), It.IsAny>()), Times.Once); + } + + [Test] + public async Task TryInvalidateAsync_UnregisteredType_ReturnsTrue() + => (await ReferenceDataOrchestrator.Current.TryInvalidateAsync()).Should().BeTrue(); // ── AlternateNames duplicate type enforcement (item 12) ────────────────── private class DummyProviderWithDuplicateAlternateNames : IReferenceDataProvider diff --git a/tests/CoreEx.Test.Unit/Caching/HybridCacheResultExtensionsTests.cs b/tests/CoreEx.Test.Unit/Caching/HybridCacheResultExtensionsTests.cs new file mode 100644 index 000000000..c41281a1c --- /dev/null +++ b/tests/CoreEx.Test.Unit/Caching/HybridCacheResultExtensionsTests.cs @@ -0,0 +1,64 @@ +using CoreEx.Caching; +using CoreEx.Entities; +using CoreEx.Results; + +namespace CoreEx.Test.Unit.Caching; + +[TestFixture] +public class HybridCacheResultExtensionsTests +{ + private class Widget : IIdentifier + { + public string? Id { get; set; } + } + + private static readonly HybridCacheEntryOptions _options = new() { Strategy = CacheStrategy.Local }; + + [Test] + public async Task GetOrCreateWithResultAsync_Success_IsCachedAndFactoryNotReinvoked() + { + var cache = new MemoryOnlyHybridCache(); + var count = 0; + Task> Factory(CancellationToken ct) { count++; return Task.FromResult>(new Widget { Id = "w1" }); } + + var r1 = await cache.GetOrCreateWithResultAsync("w1", Factory, _options); + var r2 = await cache.GetOrCreateWithResultAsync("w1", Factory, _options); + + r1.IsSuccess.Should().BeTrue(); + r2.IsSuccess.Should().BeTrue(); + r2.Value.Id.Should().Be("w1"); + count.Should().Be(1); + } + + [Test] + public async Task GetOrCreateWithResultAsync_Failure_IsReturnedAndNeverCached() + { + var cache = new MemoryOnlyHybridCache(); + var count = 0; + Task> Factory(CancellationToken ct) { count++; return Task.FromResult>(count == 1 ? Result.NotFoundError() : new Widget { Id = "w2" }); } + + var r1 = await cache.GetOrCreateWithResultAsync("w2", Factory, _options); + r1.IsNotFoundError.Should().BeTrue(); + + var r2 = await cache.GetOrCreateWithResultAsync("w2", Factory, _options); + r2.IsSuccess.Should().BeTrue(); + count.Should().Be(2); + + // Now cached; the factory is not invoked again. + var r3 = await cache.GetOrCreateWithResultAsync("w2", Factory, _options); + r3.IsSuccess.Should().BeTrue(); + count.Should().Be(2); + } + + [Test] + public async Task GetOrCreateByKeyWithResultAsync_Success_IsCached() + { + IHybridCache cache = new MemoryOnlyHybridCache(); + var count = 0; + Task> Factory(CancellationToken ct) { count++; return Task.FromResult>("value"); } + + (await cache.GetOrCreateByKeyWithResultAsync("k1", Factory, _options)).Value.Should().Be("value"); + (await cache.GetOrCreateByKeyWithResultAsync("k1", Factory, _options)).Value.Should().Be("value"); + count.Should().Be(1); + } +} diff --git a/tests/CoreEx.Test.Unit/Data/DataExtensionsTests.cs b/tests/CoreEx.Test.Unit/Data/DataExtensionsTests.cs index e547cbf13..5aaff02b6 100644 --- a/tests/CoreEx.Test.Unit/Data/DataExtensionsTests.cs +++ b/tests/CoreEx.Test.Unit/Data/DataExtensionsTests.cs @@ -155,6 +155,25 @@ public async Task WithTotalCountAsync_Requested_ExceptionSwallowed_LeavesTotalCo result.TotalCount.Should().BeNull(); } + [Test] + public async Task WithTotalCountAsync_CanceledByToken_Propagates() + { + // Regression: a requested cancellation must flow rather than be swallowed as an unknown total count. + using var cts = new CancellationTokenSource(); + await cts.CancelAsync(); + + var act = async () => await new PagingResult(PagingArgs.CreateWithCount()).WithTotalCountAsync(ct => Task.FromException(new OperationCanceledException(ct)), cts.Token); + await act.Should().ThrowAsync(); + } + + [Test] + public async Task WithTotalCountAsync_TokenlessCancellationWithDefaultToken_IsSwallowed() + { + var result = await new PagingResult(PagingArgs.CreateWithCount()).WithTotalCountAsync(_ => Task.FromException(new OperationCanceledException()), CancellationToken.None); + + result.TotalCount.Should().BeNull(); + } + [Test] public async Task WithTotalCountAsync_WithCancellationToken_PassesTokenThrough() { diff --git a/tests/CoreEx.Test.Unit/Data/PartitionKeyTests.cs b/tests/CoreEx.Test.Unit/Data/PartitionKeyTests.cs index f3d06751a..3193b412f 100644 --- a/tests/CoreEx.Test.Unit/Data/PartitionKeyTests.cs +++ b/tests/CoreEx.Test.Unit/Data/PartitionKeyTests.cs @@ -24,4 +24,20 @@ public void GetPartitionId_CaseSensitive() PartitionKey.GetPartitionId("abc", 4, false).Should().Be(2); PartitionKey.GetPartitionId("ABC", 4, false).Should().Be(1); } -} \ No newline at end of file + + [Test] + public void DefaultNoPartitionKey_IsStable() + { + PartitionKey.GetPartitionId(PartitionKey.DefaultNoPartitionKey, 4) + .Should().Be(PartitionKey.GetPartitionId("$none", 4)); + } + + [Test] + public void ValidatePartitionSize() + { + PartitionKey.ValidatePartitionSize(1).Should().Be(1); + PartitionKey.ValidatePartitionSize(256).Should().Be(256); + Assert.Throws(() => PartitionKey.ValidatePartitionSize(0)); + Assert.Throws(() => PartitionKey.ValidatePartitionSize(257)); + } +} diff --git a/tests/CoreEx.Test.Unit/Data/PartitionPickerTests.cs b/tests/CoreEx.Test.Unit/Data/PartitionPickerTests.cs index 7df8d976e..80ac873c9 100644 --- a/tests/CoreEx.Test.Unit/Data/PartitionPickerTests.cs +++ b/tests/CoreEx.Test.Unit/Data/PartitionPickerTests.cs @@ -72,17 +72,22 @@ public void GetNextPartitions_ShouldChangeAcrossEpochs() } [Test] - public void TwoPickers_ShouldHaveLowIntersection() + public void TwoPickers_ShouldHaveLowAverageIntersection() { - // Collision reduction heuristic - var a = new PartitionPicker(32, 6, 5); - var b = new PartitionPicker(32, 6, 5); + // Collision reduction heuristic; each picker has a random worker-id so any single pair can legitimately overlap heavily (about 2% of pairs share 4+ of 6), hence assert on the average + // over many pairs (statistically expected to be ~1.1 for random 6-of-32 selections; the threshold leaves a very wide margin so this is not flaky). var now = DateTimeOffset.UtcNow; + const int pairs = 1000; + var total = 0; - var pa = a.GetNextPartitions(now); - var pb = b.GetNextPartitions(now); + for (int i = 0; i < pairs; i++) + { + var pa = new PartitionPicker(32, 6, 5).GetNextPartitions(now); + var pb = new PartitionPicker(32, 6, 5).GetNextPartitions(now); + total += pa.Intersect(pb).Count(); + } - pa.Intersect(pb).Count().Should().BeLessThanOrEqualTo(3); + ((double)total / pairs).Should().BeLessThanOrEqualTo(1.5); } [TestCase(32, 10)] diff --git a/tests/CoreEx.Test.Unit/Entities/IdentifierGeneratorTests.cs b/tests/CoreEx.Test.Unit/Entities/IdentifierGeneratorTests.cs index 186b64481..37cd8ac7b 100644 --- a/tests/CoreEx.Test.Unit/Entities/IdentifierGeneratorTests.cs +++ b/tests/CoreEx.Test.Unit/Entities/IdentifierGeneratorTests.cs @@ -113,15 +113,6 @@ public async Task AssignIdentifierAsync_UnsupportedIdType_ThrowsNotSupportedExce await act.Should().ThrowAsync().WithMessage("*Int32*"); } - [Test] - public async Task AssignIdentifierAsync_NotAnIdentifier_NoOp() - { - var entity = new PlainEntity(); - Func act = () => new IdentifierGenerator().AssignIdentifierAsync(entity); - - await act.Should().NotThrowAsync(); - } - [Test] public void Current_NoExecutionContextService_ReturnsDefaultInstance() { diff --git a/tests/CoreEx.Test.Unit/Hosting/RetryResiliencyTests.cs b/tests/CoreEx.Test.Unit/Hosting/RetryResiliencyTests.cs index 806845cdc..9fb7cd1e5 100644 --- a/tests/CoreEx.Test.Unit/Hosting/RetryResiliencyTests.cs +++ b/tests/CoreEx.Test.Unit/Hosting/RetryResiliencyTests.cs @@ -91,6 +91,59 @@ public async Task Create_MatchingFailure_ExhaustsRetries_ThenFails() } } + [Test] + public async Task CreateOfTResult_TypedResult_RetriesMatchingFailure_UntilSuccessWithinBudget() + { + // Exercises the generalized Create(...) overload directly (not just via the Create(...) forwarder) using Result as TResult - confirms it carries a genuine payload + // through a successful retry, unlike the payload-less Result overload above. + var owner = new TestOwner(); + var pipeline = RetryResiliency.Create>(r => r.Error is TransientException, o => o.Logger, delay: TimeSpan.FromMilliseconds(1), maxRetryAttempts: 3); + + var ctx = CreateContext(owner); + try + { + var attempts = 0; + var result = await pipeline.ExecuteAsync(async _ => + { + attempts++; + return attempts < 3 ? new Result(new TransientException()) : new Result(99); + }, ctx); + + result.IsSuccess.Should().BeTrue(); + result.Value.Should().Be(99); + attempts.Should().Be(3); + } + finally + { + ResilienceContextPool.Shared.Return(ctx); + } + } + + [Test] + public async Task CreateOfTResult_TypedResult_NonMatchingFailure_IsNeverRetried() + { + var owner = new TestOwner(); + var pipeline = RetryResiliency.Create>(r => r.Error is TransientException, o => o.Logger, delay: TimeSpan.FromMilliseconds(1), maxRetryAttempts: 3); + + var ctx = CreateContext(owner); + try + { + var attempts = 0; + var result = await pipeline.ExecuteAsync(async _ => + { + attempts++; + return new Result(new InvalidOperationException("not retry-worthy")); + }, ctx); + + result.IsFailure.Should().BeTrue(); + attempts.Should().Be(1); + } + finally + { + ResilienceContextPool.Shared.Return(ctx); + } + } + private sealed class TransientException : Exception; private sealed class TestOwner diff --git a/tests/CoreEx.Test.Unit/Hosting/SynchronizedTimerHostedServiceBaseTests.cs b/tests/CoreEx.Test.Unit/Hosting/SynchronizedTimerHostedServiceBaseTests.cs index 14050d0a1..0c96b9a1c 100644 --- a/tests/CoreEx.Test.Unit/Hosting/SynchronizedTimerHostedServiceBaseTests.cs +++ b/tests/CoreEx.Test.Unit/Hosting/SynchronizedTimerHostedServiceBaseTests.cs @@ -183,17 +183,18 @@ private class TestSynchronizer : ISynchronizer public Task EnterAsync(string? name = null, CancellationToken cancellationToken = default) { - Interlocked.Increment(ref _enterCount); LastEnterName = name; LastEnterType = typeof(T); + Interlocked.Increment(ref _enterCount); return Task.FromResult(ShouldEnterSucceed); } public Task ExitAsync(string? name = null, CancellationToken cancellationToken = default) { - Interlocked.Increment(ref _exitCount); + // Set the values before incrementing the count, as tests poll the count and then read the values. LastExitName = name; LastExitType = typeof(T); + Interlocked.Increment(ref _exitCount); return Task.CompletedTask; } diff --git a/tests/CoreEx.Test.Unit/Hosting/TimerHostedServiceBaseTests.cs b/tests/CoreEx.Test.Unit/Hosting/TimerHostedServiceBaseTests.cs index 6dd8513ca..b8bdd5bac 100644 --- a/tests/CoreEx.Test.Unit/Hosting/TimerHostedServiceBaseTests.cs +++ b/tests/CoreEx.Test.Unit/Hosting/TimerHostedServiceBaseTests.cs @@ -137,6 +137,97 @@ public async Task UnhandledException_WithoutPauseOnUnhandledException_ContinuesE } } + [Test] + public async Task Resume_AfterManyTimedIterations_WakesService() + { + using var sp = CreateServiceProvider(); + var svc = new TestTimerService(sp, NullLogger.Instance) { Interval = TimeSpan.FromMilliseconds(5), FirstInterval = TimeSpan.FromMilliseconds(5) }; + + await svc.StartAsync(CancellationToken.None); + try + { + // Many timed iterations (delay wins) previously left abandoned semaphore waiters that would swallow subsequent wake-up signals. + await WaitUntilAsync(() => svc.ExecuteCount >= 20, TimeSpan.FromSeconds(5)); + + await svc.PauseAsync(CancellationToken.None); + await WaitUntilAsync(() => svc.Status == ServiceStatus.Paused, TimeSpan.FromSeconds(5)); + await Task.Delay(50); + + var count = svc.ExecuteCount; + await svc.ResumeAsync(CancellationToken.None); + await WaitUntilAsync(() => svc.ExecuteCount > count, TimeSpan.FromSeconds(5)); + } + finally + { + await svc.StopAsync(CancellationToken.None); + } + } + + [Test] + public async Task OneOffTrigger_WhilePaused_WakesService() + { + using var sp = CreateServiceProvider(); + var svc = new TestTimerService(sp, NullLogger.Instance) { Interval = TimeSpan.FromMilliseconds(5), FirstInterval = TimeSpan.FromMilliseconds(5), PauseOnUnhandledException = true, ThrowOnExecute = true }; + + await svc.StartAsync(CancellationToken.None); + try + { + await WaitUntilAsync(() => svc.Status == ServiceStatus.Paused, TimeSpan.FromSeconds(5)); + svc.ThrowOnExecute = false; + + // Run a number of timed iterations, then pause again so the loop waits on the signal only. + await svc.ResumeAsync(CancellationToken.None); + var count = svc.ExecuteCount; + await WaitUntilAsync(() => svc.ExecuteCount >= count + 20, TimeSpan.FromSeconds(5)); + await svc.PauseAsync(CancellationToken.None); + await Task.Delay(50); + + count = svc.ExecuteCount; + await svc.ResumeAsync(CancellationToken.None); + await WaitUntilAsync(() => svc.ExecuteCount > count, TimeSpan.FromSeconds(5)); + } + finally + { + await svc.StopAsync(CancellationToken.None); + } + } + + [Test] + public async Task StartTriggerStop_Stress_DoesNotThrow() + { + using var sp = CreateServiceProvider(); + + for (var i = 0; i < 200; i++) + { + using var svc = new TestTimerService(sp, NullLogger.Instance) { Interval = TimeSpan.FromMilliseconds(1), FirstInterval = TimeSpan.FromMilliseconds(1) }; + await svc.StartAsync(CancellationToken.None); + + using var cts = new CancellationTokenSource(); + var trigger = Task.Run(() => + { + while (!cts.IsCancellationRequested) + svc.OneOffTrigger(); + }); + + await Task.Delay(i % 5); + await svc.StopAsync(CancellationToken.None); + cts.Cancel(); + await trigger; + } + } + + [Test] + public async Task OneOffTrigger_AfterDispose_DoesNotThrow() + { + using var sp = CreateServiceProvider(); + var svc = new TestTimerService(sp, NullLogger.Instance) { Interval = TimeSpan.FromMilliseconds(5), FirstInterval = TimeSpan.FromMilliseconds(5) }; + + await svc.StartAsync(CancellationToken.None); + svc.Dispose(); + + svc.Invoking(s => s.OneOffTrigger()).Should().NotThrow(); + } + private class TestTimerService(IServiceProvider serviceProvider, ILogger logger) : TimerHostedServiceBase(serviceProvider, logger) { private int _executeCount; diff --git a/tests/CoreEx.Test.Unit/Http/HttpResponseMessageExtensionsTests.cs b/tests/CoreEx.Test.Unit/Http/HttpResponseMessageExtensionsTests.cs index da0563f5c..72999e74a 100644 --- a/tests/CoreEx.Test.Unit/Http/HttpResponseMessageExtensionsTests.cs +++ b/tests/CoreEx.Test.Unit/Http/HttpResponseMessageExtensionsTests.cs @@ -30,6 +30,103 @@ public async Task ToProblemDetailsAsync_OtherException_StillSwallowedAsNotProble result.Should().BeNull(); } + private class Widget + { + public string? Name { get; set; } + } + + private static HttpResponseMessage Json(string json, HttpStatusCode status = HttpStatusCode.OK) + => new(status) { Content = new StringContent(json, new MediaTypeHeaderValue("application/json")) }; + + private static HttpResponseMessage Empty(HttpStatusCode status = HttpStatusCode.OK) => new(status); + + [Test] + public async Task ToResultAsync_Value_ReturnsValue() + { + var r = await Json("{\"name\":\"abc\"}").ToResultAsync(); + r.IsSuccess.Should().BeTrue(); + r.Value.Name.Should().Be("abc"); + } + + [Test] + public async Task ToResultAsync_NullBody_IsFailure() + { + var r = await Json("null").ToResultAsync(); + r.IsFailure.Should().BeTrue(); + r.Error.Should().BeOfType(); + } + + [Test] + public async Task ToResultAsync_EmptyBody_IsFailure() + { + var r = await Empty(HttpStatusCode.NoContent).ToResultAsync(); + r.IsFailure.Should().BeTrue(); + r.Error.Should().BeOfType(); + r.Error.Message.Should().Contain("The response content was empty or null"); + r.Error.Message.Should().NotContain("Response status code does not indicate success"); + } + + [Test] + public async Task ToResultAsync_NotSuccess_IsFailure() + { + var r = await Empty(HttpStatusCode.InternalServerError).ToResultAsync(); + r.IsFailure.Should().BeTrue(); + r.Error.Should().BeOfType(); + } + + [Test] + public async Task ToResultOrDefaultAsync_Value_ReturnsValue() + { + var r = await Json("{\"name\":\"abc\"}").ToResultOrDefaultAsync(); + r.IsSuccess.Should().BeTrue(); + r.Value!.Name.Should().Be("abc"); + } + + [Test] + public async Task ToResultOrDefaultAsync_NullBody_IsSuccessWithNull() + { + var r = await Json("null").ToResultOrDefaultAsync(); + r.IsSuccess.Should().BeTrue(); + r.Value.Should().BeNull(); + } + + [Test] + public async Task ToResultOrDefaultAsync_EmptyBody_IsSuccessWithNull() + { + var r = await Empty(HttpStatusCode.NoContent).ToResultOrDefaultAsync(); + r.IsSuccess.Should().BeTrue(); + r.Value.Should().BeNull(); + } + + [Test] + public async Task ToResultOrDefaultAsync_NotSuccess_IsFailure() + { + var r = await Empty(HttpStatusCode.InternalServerError).ToResultOrDefaultAsync(); + r.IsFailure.Should().BeTrue(); + } + + [Test] + public async Task GetValueAsync_Value_ReturnsValue() + => (await Json("{\"name\":\"abc\"}").GetValueAsync()).Name.Should().Be("abc"); + + [Test] + public async Task GetValueAsync_NullBody_Throws() + { + var act = async () => await Json("null").GetValueAsync(); + await act.Should().ThrowAsync(); + } + + [Test] + public async Task GetValueOrDefaultAsync_NullBody_ReturnsNull() + => (await Json("null").GetValueOrDefaultAsync()).Should().BeNull(); + + [Test] + public async Task GetValueOrDefaultAsync_NotSuccess_Throws() + { + var act = async () => await Empty(HttpStatusCode.InternalServerError).GetValueOrDefaultAsync(); + await act.Should().ThrowAsync(); + } + private sealed class ThrowingContent : HttpContent { private readonly Exception _exception; diff --git a/tests/CoreEx.Test.Unit/Http/ValidatedHttpResponseTests.cs b/tests/CoreEx.Test.Unit/Http/ValidatedHttpResponseTests.cs new file mode 100644 index 000000000..2d996b762 --- /dev/null +++ b/tests/CoreEx.Test.Unit/Http/ValidatedHttpResponseTests.cs @@ -0,0 +1,283 @@ +namespace CoreEx.Test.Unit.Http; + +[TestFixture] +public class ValidatedHttpResponseTests +{ + private sealed class Widget + { + public string? Name { get; set; } + } + + private sealed class Validator(Func>> validate) : CoreEx.Validation.IValidator + { + public int Calls { get; private set; } + + public Task> ValidateAsync(Widget value, CancellationToken cancellationToken = default) + { + Calls++; + return validate(value, cancellationToken); + } + } + + private sealed class ValidationResult(Widget? value, bool hasErrors = false, Exception? error = null, CoreEx.Entities.MessageItemCollection? messages = null) : CoreEx.Validation.IValidationResult + { + public Widget? Value => value; + public bool HasErrors => hasErrors; + public CoreEx.Entities.MessageItemCollection? Messages => messages; + public Exception? ToException() => error; + public CoreEx.Validation.IValidationResult ThrowOnError() + { + if (error is not null) + throw error; + return this; + } + public CoreEx.Results.Result ToResult() => error is null ? CoreEx.Results.Result.Success : CoreEx.Results.Result.Fail(error); + } + + private static Validator CreateValidator(Widget? replacement, Exception? error = null) + => new((_, _) => Task.FromResult>(new ValidationResult(replacement, error is not null, error))); + + private static HttpResponseMessage Json(string json, System.Net.HttpStatusCode status = System.Net.HttpStatusCode.OK, string mediaType = "application/json") + => new(status) { Content = new StringContent(json, System.Text.Encoding.UTF8, mediaType) }; + + private static Task> ConsumeAsync(CoreEx.Http.ValidatedHttpResponse response, string method, System.Text.Json.JsonSerializerOptions? options = null, CancellationToken cancellationToken = default) + => method switch + { + "Result" => ConsumeRequiredAsync(response, options, cancellationToken), + "OptionalResult" => response.ToResultOrDefaultAsync(options, cancellationToken), + "Value" => ConsumeValueAsync(response, false, options, cancellationToken), + "OptionalValue" => ConsumeValueAsync(response, true, options, cancellationToken), + _ => throw new InvalidOperationException(method) + }; + + private static async Task> ConsumeRequiredAsync(CoreEx.Http.ValidatedHttpResponse response, System.Text.Json.JsonSerializerOptions? options, CancellationToken cancellationToken) + { + var result = await response.ToResultAsync(options, cancellationToken); + return result.IsFailure ? CoreEx.Results.Result.Fail(result.Error) : CoreEx.Results.Result.Ok(result.Value); + } + + private static async Task> ConsumeValueAsync(CoreEx.Http.ValidatedHttpResponse response, bool optional, System.Text.Json.JsonSerializerOptions? options, CancellationToken cancellationToken) + => CoreEx.Results.Result.Ok(optional ? await response.GetValueOrDefaultAsync(options, cancellationToken) : await response.GetValueAsync(options, cancellationToken)); + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task Success_ReturnsValidationResultValue(string method) + { + using var response = Json("{\"name\":\"original\"}"); + var replacement = new Widget { Name = "replacement" }; + var validator = CreateValidator(replacement); + var result = await ConsumeAsync(response.WithValidator(validator), method); + result.IsSuccess.Should().BeTrue(); + result.Value.Should().BeSameAs(replacement); + validator.Calls.Should().Be(1); + response.Content.Headers.ContentType.Should().NotBeNull(); + } + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task InvalidValue_PreservesStructuredInnerException(string method) + { + using var response = Json("{\"name\":\"sensitive-value\"}"); + var error = CoreEx.ValidationException.Create("name", "Name is invalid."); + var validator = CreateValidator(null, error); + HttpRequestException exception; + if (method.EndsWith("Result", StringComparison.Ordinal)) + { + var result = await ConsumeAsync(response.WithValidator(validator), method); + result.IsFailure.Should().BeTrue(); + exception = result.Error.Should().BeOfType().Subject; + } + else + { + Func act = async () => await ConsumeAsync(response.WithValidator(validator), method); + exception = (await act.Should().ThrowAsync()).Which; + } + + exception.InnerException.Should().BeSameAs(error); + exception.StatusCode.Should().Be(System.Net.HttpStatusCode.OK); + exception.Message.Should().Contain(nameof(Widget)).And.NotContain("sensitive-value"); + error.Messages.Should().ContainSingle().Which.Property.Should().Be("name"); + validator.Calls.Should().Be(1); + } + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task SuccessfulValidation_NullValue_PreservesRequiredSemantics(string method) + { + using var response = Json("{}"); + var validator = CreateValidator(null); + if (method == "Value") + { + Func act = async () => await ConsumeAsync(response.WithValidator(validator), method); + await act.Should().ThrowAsync(); + } + else + { + var result = await ConsumeAsync(response.WithValidator(validator), method); + if (method == "Result") + result.Error.Should().BeOfType(); + else + result.Value.Should().BeNull(); + } + validator.Calls.Should().Be(1); + } + + [TestCase("null", false)] + [TestCase("", false)] + [TestCase("", true)] + public async Task MissingValue_SkipsValidation(string body, bool noContent) + { + var validator = CreateValidator(new Widget()); + foreach (var method in new[] { "Result", "OptionalResult", "Value", "OptionalValue" }) + { + using var response = Json(body, noContent ? System.Net.HttpStatusCode.NoContent : System.Net.HttpStatusCode.OK); + if (method == "Value") + { + Func act = async () => await ConsumeAsync(response.WithValidator(validator), method); + await act.Should().ThrowAsync(); + } + else + { + var result = await ConsumeAsync(response.WithValidator(validator), method); + if (method == "Result") + result.Error.Should().BeOfType(); + else + result.Value.Should().BeNull(); + } + } + validator.Calls.Should().Be(0); + } + + [TestCase("application/json", "{}", false)] + [TestCase("application/problem+json", "{\"status\":500,\"title\":\"Dependency failed\"}", false)] + [TestCase("application/problem+json", "{\"status\":400,\"title\":\"Business failed\",\"errorType\":\"business\"}", true)] + public async Task HttpError_PreservesClassificationAndSkipsValidation(string mediaType, string body, bool business) + { + using var response = Json(body, business ? System.Net.HttpStatusCode.BadRequest : System.Net.HttpStatusCode.InternalServerError, mediaType); + var validator = CreateValidator(new Widget()); + var expected = await response.ToResultAsync(); + var actual = await response.WithValidator(validator).ToResultAsync(); + actual.Error.GetType().Should().Be(expected.Error.GetType()); + actual.Error.Message.Should().Be(expected.Error.Message); + (await response.WithValidator(validator).ToResultOrDefaultAsync()).IsFailure.Should().BeTrue(); + Func act = async () => await response.WithValidator(validator).GetValueAsync(); + (await act.Should().ThrowAsync()).Which.GetType().Should().Be(expected.Error.GetType()); + validator.Calls.Should().Be(0); + } + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task SerializerOptionsAndToken_AreForwarded(string method) + { + using var response = Json("{\"Name\":\"case-sensitive\"}"); + using var cts = new CancellationTokenSource(); + var validator = new Validator((value, token) => + { + value.Name.Should().Be("case-sensitive"); + token.Should().Be(cts.Token); + return Task.FromResult>(new ValidationResult(value)); + }); + var options = new System.Text.Json.JsonSerializerOptions { PropertyNameCaseInsensitive = false }; + (await ConsumeAsync(response.WithValidator(validator), method, options, cts.Token)).IsSuccess.Should().BeTrue(); + } + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task UnexpectedValidatorException_PropagatesUnchanged(string method) + { + using var response = Json("{}"); + var error = new InvalidOperationException("Validator bug."); + var validator = new Validator((_, _) => throw error); + Func act = async () => await ConsumeAsync(response.WithValidator(validator), method); + (await act.Should().ThrowAsync()).Which.Should().BeSameAs(error); + } + + [TestCase("Result")] + [TestCase("OptionalResult")] + [TestCase("Value")] + [TestCase("OptionalValue")] + public async Task CancellationDuringValidation_Propagates(string method) + { + using var response = Json("{}"); + using var cts = new CancellationTokenSource(); + var validator = new Validator((_, token) => + { + cts.Cancel(); + token.ThrowIfCancellationRequested(); + throw new InvalidOperationException("Cancellation was not observed."); + }); + Func act = async () => await ConsumeAsync(response.WithValidator(validator), method, cancellationToken: cts.Token); + (await act.Should().ThrowAsync()).Which.CancellationToken.Should().Be(cts.Token); + } + + [Test] + public async Task CancellationBeforeDeserialization_SkipsValidation() + { + using var response = Json("{}"); + using var cts = new CancellationTokenSource(); + cts.Cancel(); + var validator = CreateValidator(new Widget()); + Func act = async () => await response.WithValidator(validator).ToResultAsync(cts.Token); + await act.Should().ThrowAsync(); + validator.Calls.Should().Be(0); + } + + [Test] + public async Task MalformedJson_PropagatesWithoutValidation() + { + using var response = Json("{"); + var validator = CreateValidator(new Widget()); + Func act = async () => await response.WithValidator(validator).ToResultAsync(); + await act.Should().ThrowAsync(); + validator.Calls.Should().Be(0); + } + + [Test] + public async Task MissingValidationException_PreservesMessages() + { + using var response = Json("{}"); + var messages = new CoreEx.Entities.MessageItemCollection { CoreEx.Entities.MessageItem.CreateErrorMessage("name", "Invalid name.") }; + var validator = new Validator((value, _) => Task.FromResult>(new ValidationResult(value, true, messages: messages))); + var result = await response.WithValidator(validator).ToResultAsync(); + result.Error.InnerException.Should().BeOfType().Which.Messages.Should().BeEquivalentTo(messages); + } + + [Test] + public async Task IndependentWrappers_DoNotChangeUnderlyingResponse() + { + using var response = Json("{\"name\":\"original\"}"); + var first = new Widget { Name = "first" }; + var second = new Widget { Name = "second" }; + var firstWrapper = response.WithValidator(CreateValidator(first)); + var secondWrapper = response.WithValidator(CreateValidator(second)); + (await response.GetValueAsync()).Name.Should().Be("original"); + // Existing converters consume the content stream; refresh content before each subsequent consumption. + response.Content.Dispose(); + response.Content = new StringContent("{\"name\":\"original\"}", System.Text.Encoding.UTF8, "application/json"); + (await firstWrapper.GetValueAsync()).Should().BeSameAs(first); + response.Content.Dispose(); + response.Content = new StringContent("{\"name\":\"original\"}", System.Text.Encoding.UTF8, "application/json"); + (await secondWrapper.GetValueOrDefaultAsync()).Should().BeSameAs(second); + } + + [Test] + public void NullArguments_AreRejectedImmediately() + { + using var response = Json("{}"); + // Null arguments deliberately exercise runtime guards. + Action noResponse = () => Extensions.WithValidator(null!, CreateValidator(new Widget())); + Action noValidator = () => response.WithValidator(null!); + noResponse.Should().Throw(); + noValidator.Should().Throw(); + } +} diff --git a/tests/CoreEx.Test.Unit/IsCanceledTests.cs b/tests/CoreEx.Test.Unit/IsCanceledTests.cs new file mode 100644 index 000000000..937a4d257 --- /dev/null +++ b/tests/CoreEx.Test.Unit/IsCanceledTests.cs @@ -0,0 +1,57 @@ +namespace CoreEx.Test.Unit; + +[TestFixture] +public class IsCanceledTests +{ + [Test] + public void IsCanceled_OperationCanceled() => new OperationCanceledException().IsCanceled().Should().BeTrue(); + + [Test] + public void IsCanceled_TaskCanceled() => new TaskCanceledException().IsCanceled().Should().BeTrue(); + + [Test] + public void IsCanceled_SingleAggregate() => new AggregateException(new TaskCanceledException()).IsCanceled().Should().BeTrue(); + + [Test] + public void IsCanceled_NestedAggregate() + { + // Regression: multiple Result.ThrowOnError() boundaries each wrap the error in a new AggregateException. + new AggregateException(new AggregateException(new TaskCanceledException())).IsCanceled().Should().BeTrue(); + } + + [Test] + public void IsCanceled_NotCanceled() + { + new InvalidOperationException().IsCanceled().Should().BeFalse(); + new AggregateException(new AggregateException(new InvalidOperationException())).IsCanceled().Should().BeFalse(); + new AggregateException().IsCanceled().Should().BeFalse(); + } + + [Test] + public void IsCanceledBy_MatchingToken() + { + using var cts = new CancellationTokenSource(); + new OperationCanceledException(cts.Token).IsCanceledBy(cts.Token).Should().BeTrue(); + new AggregateException(new TaskCanceledException(null, null, cts.Token)).IsCanceledBy(cts.Token).Should().BeTrue(); + new AggregateException(new AggregateException(new OperationCanceledException(cts.Token))).IsCanceledBy(cts.Token).Should().BeTrue(); + } + + [Test] + public void IsCanceledBy_DifferentToken() + { + using var cts = new CancellationTokenSource(); + using var other = new CancellationTokenSource(); + new OperationCanceledException(other.Token).IsCanceledBy(cts.Token).Should().BeFalse(); + new AggregateException(new AggregateException(new OperationCanceledException(other.Token))).IsCanceledBy(cts.Token).Should().BeFalse(); + new InvalidOperationException().IsCanceledBy(cts.Token).Should().BeFalse(); + new AggregateException().IsCanceledBy(cts.Token).Should().BeFalse(); + } + + [Test] + public void IsCanceledBy_DefaultToken_DoesNotMatchTokenlessCancellation() + { + new OperationCanceledException().IsCanceledBy(CancellationToken.None).Should().BeFalse(); + new OperationCanceledException(CancellationToken.None).IsCanceledBy(CancellationToken.None).Should().BeFalse(); + new AggregateException(new OperationCanceledException()).IsCanceledBy(CancellationToken.None).Should().BeFalse(); + } +} diff --git a/tests/CoreEx.Test.Unit/Json/JsonMergeTests.cs b/tests/CoreEx.Test.Unit/Json/JsonMergeTests.cs index b7e3605c5..a42354813 100644 --- a/tests/CoreEx.Test.Unit/Json/JsonMergeTests.cs +++ b/tests/CoreEx.Test.Unit/Json/JsonMergeTests.cs @@ -1,4 +1,5 @@ using CoreEx.Json; +using CoreEx.Results; namespace CoreEx.Test.Unit.Json; @@ -305,9 +306,8 @@ public async Task MergeAsync_Get_Null() var p = new JsonMergePatch(); var r = await p.MergeAsync(new BinaryData("""{"name":"bob","age":30}"""), _ => Task.FromResult(null)); r.Should().NotBeNull(); - r.IsSuccess.Should().BeTrue(); - r.Value.HasChanges.Should().BeFalse(); - r.Value.Merged.Should().BeNull(); + r.IsSuccess.Should().BeFalse(); + r.IsNotFoundError.Should().BeTrue(); } public class Person @@ -324,4 +324,4 @@ public class Address public string? Street { get; set; } public string? City { get; set; } } -} \ No newline at end of file +} diff --git a/tests/CoreEx.Test.Unit/Results/ExtensionsTests.cs b/tests/CoreEx.Test.Unit/Results/ExtensionsTests.cs index 884fc67f9..5fb2a90b6 100644 --- a/tests/CoreEx.Test.Unit/Results/ExtensionsTests.cs +++ b/tests/CoreEx.Test.Unit/Results/ExtensionsTests.cs @@ -356,4 +356,4 @@ public async Task AsResult_ValueTask_T_U_Failure() ret.IsFailure.Should().BeTrue(); ret.Error.Should().Be(ex); } -} \ No newline at end of file +} diff --git a/tests/CoreEx.Test.Unit/Runtime/RuntimeMetadataTests.cs b/tests/CoreEx.Test.Unit/Runtime/RuntimeMetadataTests.cs index 652324c33..cf151c523 100644 --- a/tests/CoreEx.Test.Unit/Runtime/RuntimeMetadataTests.cs +++ b/tests/CoreEx.Test.Unit/Runtime/RuntimeMetadataTests.cs @@ -112,6 +112,46 @@ public void AreEqual_Strings() RuntimeMetadata.AreEqual("abc", "def").Should().BeFalse(); } + public class PlainPoco + { + public int Id { get; set; } + public string? Name { get; set; } + public PlainPoco? Child { get; set; } + } + + [Test] + public void AreEqual_ExcludedProperties_Contract() + { + var a = new EntityA { Id = 1, Name = "Bob" }; + var b = new EntityA { Id = 2, Name = "Bob" }; + + RuntimeMetadata.AreEqual(a, b).Should().BeFalse(); + RuntimeMetadata.AreEqual(a, b, [nameof(EntityA.Id)]).Should().BeTrue(); + RuntimeMetadata.AreEqual(a, b, [nameof(EntityA.Name)]).Should().BeFalse(); + RuntimeMetadata.AreEqual(a, b, []).Should().BeFalse(); + RuntimeMetadata.AreEqual(a, b, null).Should().BeFalse(); + } + + [Test] + public void AreEqual_ExcludedProperties_PlainClass_RootOnly() + { + var a = new PlainPoco { Id = 1, Name = "Bob", Child = new PlainPoco { Id = 5 } }; + var b = new PlainPoco { Id = 2, Name = "Bob", Child = new PlainPoco { Id = 5 } }; + RuntimeMetadata.AreEqual(a, b, [nameof(PlainPoco.Id)]).Should().BeTrue(); + + // Exclusion is top-level only; the same name nested still counts. + var c = new PlainPoco { Id = 1, Name = "Bob", Child = new PlainPoco { Id = 6 } }; + RuntimeMetadata.AreEqual(a, c, [nameof(PlainPoco.Id)]).Should().BeFalse(); + } + + [Test] + public void AreEqual_ExcludedProperties_SameReference_AndNulls() + { + var a = new PlainPoco { Id = 1 }; + RuntimeMetadata.AreEqual(a, a, [nameof(PlainPoco.Id)]).Should().BeTrue(); + RuntimeMetadata.AreEqual(null, null, [nameof(PlainPoco.Id)]).Should().BeTrue(); + RuntimeMetadata.AreEqual(a, null, [nameof(PlainPoco.Id)]).Should().BeFalse(); + } [Test] public void AreEqual_IEnumerable_ValueType() { @@ -136,6 +176,28 @@ public void AreEqual_IEnumerable_ReferenceType() RuntimeMetadata.AreEqual(arr1, arr4).Should().BeFalse(); } + [TestCase("{\"a\":1,\"b\":2}", "{\"b\":2,\"a\":1}", true)] + [TestCase("{\"a\":{\"b\":[1,{\"c\":1,\"d\":2}]}}", "{\"a\":{\"b\":[1,{\"d\":2,\"c\":1}]}}", true)] + [TestCase("{\"a\":1}", "{\"a\":2}", false)] + [TestCase("{\"a\":\"x\"}", "{\"a\":\"y\"}", false)] + [TestCase("{\"a\":1}", "{\"a\":1,\"b\":2}", false)] + [TestCase("{\"a\":1,\"b\":2}", "{\"a\":1}", false)] + [TestCase("{\"a\":null}", "{}", false)] + [TestCase("{\"a\":{\"b\":[1,{\"c\":1}]}}", "{\"a\":{\"b\":[1,{\"c\":2}]}}", false)] + [TestCase("[1,2]", "[2,1]", false)] + [TestCase("[1,2]", "[1,2,3]", false)] + [TestCase("1", "1.0", true)] + [TestCase("1e2", "100", true)] + [TestCase("10.50", "10.5", true)] + [TestCase("12345678901234567890.123", "12345678901234567890.124", false)] + [TestCase("1e999", "1e999", true)] + [TestCase("\"\\u0041\"", "\"A\"", true)] + [TestCase("\"1\"", "1", false)] + [TestCase("true", "false", false)] + [TestCase("null", "null", true)] + public void AreEqual_JsonElement(string left, string right, bool expected) + => RuntimeMetadata.AreEqual(System.Text.Json.JsonDocument.Parse(left).RootElement, System.Text.Json.JsonDocument.Parse(right).RootElement).Should().Be(expected); + // Local iterator methods (not backed by ICollection) so the AreEqual ICollection.Count short-circuit is bypassed // and TypedEnumerateAreEqual/EnumerateObjectAreEqual are exercised directly. private static IEnumerable LazyInts(params int[] values) diff --git a/tests/CoreEx.Validation.Test.Unit/ReferenceDataValidatorTests.cs b/tests/CoreEx.Validation.Test.Unit/ReferenceDataValidatorTests.cs new file mode 100644 index 000000000..c92665796 --- /dev/null +++ b/tests/CoreEx.Validation.Test.Unit/ReferenceDataValidatorTests.cs @@ -0,0 +1,134 @@ +using CoreEx.Entities; +using CoreEx.Entities.Abstractions; +using CoreEx.RefData.Abstractions; +using System.Diagnostics.CodeAnalysis; + +namespace CoreEx.Validation.Test.Unit; + +public class ReferenceDataValidatorTests +{ + [Test] + public void Validate_Success() + { + var rd = new TestRefData { Code = "ABC", Text = "Some text" }; + var v = new ReferenceDataValidator(); + + v.ValidateAsSuccess(rd); + } + + [Test] + public void Validate_Code_Mandatory() + { + var rd = new TestRefData { Code = null, Text = "Some text" }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "code", "Code is required."); + } + + [Test] + public void Validate_Code_MaximumLength() + { + var rd = new TestRefData { Code = new string('x', ReferenceDataValidator.CodeMaximumLength + 1), Text = "Some text" }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "code", "Code must not exceed"); + } + + [Test] + public void Validate_Text_Mandatory() + { + var rd = new TestRefData { Code = "ABC", Text = null }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "text", "Text is required."); + } + + [Test] + public void Validate_Text_MaximumLength() + { + var rd = new TestRefData { Code = "ABC", Text = new string('x', ReferenceDataValidator.TextMaximumLength + 1) }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "text", "Text must not exceed"); + } + + [Test] + public void Validate_Description_MaximumLength() + { + var rd = new TestRefData { Code = "ABC", Text = "Some text", Description = new string('x', ReferenceDataValidator.DescriptionMaximumLength + 1) }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "description", "Description must not exceed"); + } + + [Test] + public void Validate_EndsOn_MustBeGreaterThanOrEqualToStartsOn() + { + var rd = new TestRefData { Code = "ABC", Text = "Some text", StartsOn = new DateTimeOffset(2023, 1, 1, 0, 0, 0, TimeSpan.Zero), EndsOn = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero) }; + var v = new ReferenceDataValidator(); + + v.ValidateAsError(rd, "endsOn", "Ends on must be greater than or equal to Starts on."); + } + + [Test] + public void Validate_EndsOn_WithStartsOn_Success() + { + var rd = new TestRefData { Code = "ABC", Text = "Some text", StartsOn = new DateTimeOffset(2022, 1, 1, 0, 0, 0, TimeSpan.Zero), EndsOn = new DateTimeOffset(2023, 1, 1, 0, 0, 0, TimeSpan.Zero) }; + var v = new ReferenceDataValidator(); + + v.ValidateAsSuccess(rd); + } + + private sealed class TestRefData : IReferenceData + { + object? IReferenceData.Id { get; init; } + + object? IIdentifierCore.Id => ((IReferenceData)this).Id; + + public string? Code { get; init; } + + public string? Text { get; init; } + + public string? Description { get; init; } + + public int SortOrder { get; init; } + + public bool IsInactive { get; init; } + + public bool IsActive => !IsInactive; + + public DateTimeOffset? StartsOn { get; init; } + + public DateTimeOffset? EndsOn { get; init; } + + public bool IsValid => true; + + public bool HasMappings => false; + + public IReadOnlyDictionary? Mappings => null; + + public string? ETag => null; + + public Type IdType => typeof(object); + + public bool IsIdReadOnly => true; + + public CompositeKey EntityKey => CompositeKey.Create(Code); + + public void SetIdentifier(object? id) => throw new NotSupportedException(); + + public void SetInvalid() => throw new NotSupportedException(); + + public string? GetText() => Text; + + public string? GetDescription() => Description; + + public void SetMapping(string name, T? value) where T : IComparable, IEquatable => throw new NotSupportedException(); + + public bool TryGetMapping(string name, [NotNullWhen(true)] out T? value) where T : IComparable, IEquatable + { + value = default; + return false; + } + } +} diff --git a/tests/CoreEx.Validation.Test.Unit/Rules/ComparePropertyRuleTests.cs b/tests/CoreEx.Validation.Test.Unit/Rules/ComparePropertyRuleTests.cs index 5629802e8..d18efe0f8 100644 --- a/tests/CoreEx.Validation.Test.Unit/Rules/ComparePropertyRuleTests.cs +++ b/tests/CoreEx.Validation.Test.Unit/Rules/ComparePropertyRuleTests.cs @@ -14,13 +14,13 @@ public void CompareProperty() v.ValidateAsSuccess(new Ranges { FromNumber = null, ToNumber = 2 }); v.ValidateAsSuccess(new Ranges { FromNumber = 1, ToNumber = null }); v.ValidateAsSuccess(new Ranges { FromNumber = 1, ToNumber = 2 }); - v.ValidateAsError(new Ranges { FromNumber = 2, ToNumber = 1 }, "toNumber", "To number must be greater than or equal to '2'."); + v.ValidateAsError(new Ranges { FromNumber = 2, ToNumber = 1 }, "toNumber", "To number must be greater than or equal to From number."); v.ValidateAsSuccess(new Ranges()); v.ValidateAsSuccess(new Ranges { FromText = null, ToText = "b" }); v.ValidateAsSuccess(new Ranges { FromText = "a", ToText = null }); v.ValidateAsSuccess(new Ranges { FromText = "a", ToText = "b" }); - v.ValidateAsError(new Ranges { FromText = "b", ToText = "a" }, "toText", "To text must be greater than or equal to 'b'."); + v.ValidateAsError(new Ranges { FromText = "b", ToText = "a" }, "toText", "To text must be greater than or equal to From text."); } [Test] @@ -63,4 +63,4 @@ private class Ranges public string? ToText { get; set; } public long? FromLong { get; set; } } -} \ No newline at end of file +} diff --git a/tools/validate-template-pack.ps1 b/tools/validate-template-pack.ps1 index bc5f27bd1..c5541f98c 100644 --- a/tools/validate-template-pack.ps1 +++ b/tools/validate-template-pack.ps1 @@ -73,6 +73,10 @@ $testScenarios = @( ".github/skills/coreex-bootstrap/SKILL.md" ".github/skills/coreex-docs-sync/SKILL.md" ".github/skills/coreex-scaffold/SKILL.md" + ".github/skills/coreex-aspire/SKILL.md" + ".github/skills/coreex-command-publish-e2e/SKILL.md" + ".github/skills/coreex-command-subscribe-e2e/SKILL.md" + ".github/instructions/coreex-aspire.instructions.md" ".github/docs/coreex/manifest.txt" ".github/coreex-ai-workflows.md" ".claude/commands/coreex-bootstrap.md" @@ -80,6 +84,9 @@ $testScenarios = @( ".claude/commands/coreex-docs-sync.md" ".claude/commands/coreex-scaffold.md" ".claude/commands/coreex-adapter.md" + ".claude/commands/coreex-aspire.md" + ".claude/commands/coreex-command-publish-e2e.md" + ".claude/commands/coreex-command-subscribe-e2e.md" ".claude/commands/coreex-aggregate.md" ".claude/commands/coreex-api.md" ".claude/commands/coreex-api-e2e.md" @@ -121,6 +128,10 @@ $testScenarios = @( ".github/instructions/coreex-validators.instructions.md" ".github/skills/coreex-docs-sync/SKILL.md" ".github/skills/coreex-scaffold/SKILL.md" + ".github/skills/coreex-aspire/SKILL.md" + ".github/skills/coreex-command-publish-e2e/SKILL.md" + ".github/skills/coreex-command-subscribe-e2e/SKILL.md" + ".github/instructions/coreex-aspire.instructions.md" ".github/docs/coreex/manifest.txt" ".github/coreex-ai-workflows.md" ".claude/commands/coreex-docs-sync.md" @@ -158,13 +169,16 @@ $testScenarios = @( "tools/App.CodeGen/App.CodeGen.csproj" "tests/App.Test.Common/App.Test.Common.csproj" "tests/App.Test.Unit/App.Test.Unit.csproj" + "tests/Directory.Build.props" ) FilesAbsent = @( ".github" "src/App.Domain" + "tests/_Directory.Build.props" ) FileContains = @{ "src/App.Infrastructure/App.Infrastructure.csproj" = "CoreEx.Database.Postgres" + "tests/Directory.Build.props" = "IDE1006" } } Build = $true @@ -280,6 +294,8 @@ $testScenarios = @( "tools/App.Database" "tools/App.CodeGen" "src/App.Domain" + "tests/App.Test.Common/ServiceBus.cs" + "tests/App.Test.Common/GlobalUsing.cs" ) } Build = $true @@ -303,7 +319,7 @@ $testScenarios = @( FilesAbsent = @( ".github" "tools/App.CodeGen" - "src/App.Application/ReferenceDataService.cs" + "src/App.Application/ReferenceDataProvider.g.cs" "src/App.Domain" ) } @@ -468,41 +484,269 @@ $testScenarios = @( TestPath = "test-aspire-full-stack" Verify = @{ FilesPresent = @( - "src/App.Aspire/App.Aspire.csproj" - "src/App.Aspire/AppHost.cs" - "src/App.Aspire/Extensions.cs" + "aspire/App.Aspire/App.Aspire.csproj" + "aspire/App.Aspire/AppHost.cs" + "aspire/App.Aspire.MockHost/App.Aspire.MockHost.csproj" + "aspire/App.Aspire.MockHost/Program.cs" + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" + "aspire/App.Test.Aspire/GlobalUsing.cs" + "aspire/App.Test.Aspire/HostTests.cs" + "aspire/Directory.Build.props" + "tests/App.Test.Common/ServiceBus.cs" ) FileContains = @{ - "src/App.Aspire/AppHost.cs" = "Projects.App_Api" + "aspire/Directory.Build.props" = "IDE1006" + "aspire/App.Aspire/AppHost.cs" = "Projects.App_Api" + "aspire/App.Aspire/App.Aspire.csproj" = "CoreEx.UnitTesting" + "src/App.Api/Program.cs" = "AddNamedDestinationProvider" + "src/App.Subscribe/Program.cs" = "WithKeyedSubscribedSubscriber" + "tests/App.Test.Common/ServiceBus.cs" = "CreateTopicOptions" + "aspire/App.Test.Aspire/HostTests.cs" = "must report healthy" } + FileNotContains = @{ + "aspire/App.Test.Aspire/HostTests.cs" = "BeOneOf" + } + FilesAbsent = @( + # Extensions.cs was superseded by CoreEx.UnitTesting's Aspire extension methods (UnitTestExExtensions.Aspire.cs). + "aspire/App.Aspire/Extensions.cs" + "aspire/App.Aspire/InfrastructureResourceExtensions.cs" + "aspire/App.Test.Aspire/InfrastructureResourceTests.cs" + "aspire/_Directory.Build.props" + ) } - Build = $true - BuildTarget = "src/App.Aspire/App.Aspire.csproj" # building the AppHost transitively builds every host it references + Build = $true + BuildTargets = @( + "aspire/App.Aspire/App.Aspire.csproj" # transitively builds every host it references + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" # AppHost doesn't reference this, so build it explicitly + "tests/App.Test.Subscribe/App.Test.Subscribe.csproj" # Compiles the keyed-subscriber tests and the Test.Common ServiceBus reset. + "tests/App.Test.Relay/App.Test.Relay.csproj" + ) }, @{ Name = "coreex-aspire-api-only" Steps = @( @{ Template = "coreex"; Name = "App"; Parameters = @{ "data-provider" = "SqlServer"; "messaging-provider" = "None"; "refdata-enabled" = "false"; "outbox-enabled" = "false"; "rop-enabled" = "false" } } @{ Template = "coreex-api"; Name = "App.Api"; Parameters = @{ "data-provider" = "SqlServer"; "refdata-enabled" = "false"; "outbox-enabled" = "false" } } - @{ Template = "coreex-aspire"; Name = "App.Aspire"; Parameters = @{ "has-api" = "true"; "has-relay" = "false"; "has-subscribe" = "false" } } + @{ Template = "coreex-aspire"; Name = "App.Aspire"; Parameters = @{ "has-api" = "true"; "has-relay" = "false"; "has-subscribe" = "false"; "data-provider" = "SqlServer"; "messaging-provider" = "None" } } ) TestPath = "test-aspire-api-only" Verify = @{ FilesPresent = @( - "src/App.Aspire/App.Aspire.csproj" + "aspire/App.Aspire/App.Aspire.csproj" + "aspire/App.Aspire.MockHost/App.Aspire.MockHost.csproj" + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" + ) + FilesAbsent = @( + # Extensions.cs was superseded by CoreEx.UnitTesting's Aspire extension methods (UnitTestExExtensions.Aspire.cs). + "aspire/App.Aspire/Extensions.cs" ) FileContains = @{ - "src/App.Aspire/AppHost.cs" = "Projects.App_Api" + "aspire/App.Aspire/AppHost.cs" = "Projects.App_Api" } FileNotContains = @{ # has-relay/has-subscribe are false — confirms the #if stripping actually drops # the other hosts' AddProject calls and ProjectReferences, not just that has-api's survive. - "src/App.Aspire/AppHost.cs" = "Projects.App_Relay" - "src/App.Aspire/App.Aspire.csproj" = "App.Relay" + "aspire/App.Aspire/AppHost.cs" = "Projects.App_Relay" + "aspire/App.Aspire/App.Aspire.csproj" = "App.Relay" + } + } + Build = $true + BuildTargets = @( + "aspire/App.Aspire/App.Aspire.csproj" + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" + ) + }, + @{ + Name = "coreex-aspire-no-data" + Steps = @( + @{ Template = "coreex"; Name = "App"; Parameters = @{ "data-provider" = "None"; "messaging-provider" = "None"; "refdata-enabled" = "false"; "outbox-enabled" = "false"; "rop-enabled" = "false" } } + @{ Template = "coreex-api"; Name = "App.Api"; Parameters = @{ "data-provider" = "None"; "refdata-enabled" = "false"; "outbox-enabled" = "false" } } + @{ Template = "coreex-aspire"; Name = "App.Aspire"; Parameters = @{ "has-api" = "true"; "has-relay" = "false"; "has-subscribe" = "false"; "data-provider" = "None"; "messaging-provider" = "None" } } + ) + TestPath = "test-aspire-no-data" + Verify = @{ + FilesAbsent = @( + "aspire/App.Test.Aspire/InfrastructureResourceTests.cs" + "aspire/App.Aspire/InfrastructureResourceExtensions.cs" + ) + FileContains = @{ + "aspire/App.Aspire/AppHost.cs" = 'AddExternalConnectionString("redis")' + } + FileNotContains = @{ + "aspire/App.Aspire/AppHost.cs" = ".WithReference(db)" + } + } + Build = $true + BuildTargets = @( + "aspire/App.Aspire/App.Aspire.csproj" + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" + ) + }, + @{ + Name = "coreex-cosmos" + Template = "coreex" + Parameters = @{ + "data-provider" = "Cosmos" + "messaging-provider" = "ServiceBus" + "refdata-enabled" = "true" + "outbox-enabled" = "true" + "rop-enabled" = "false" + } + TestPath = "test-coreex-cosmos" + Verify = @{ + FilesPresent = @( + "src/App.Infrastructure/App.Infrastructure.csproj" + "src/App.Infrastructure/Repositories/AppCosmosDb.cs" + "tools/App.Database/App.Database.csproj" + "tools/App.Database/Data/ref-data.seed.yaml" + "tools/App.CodeGen/App.CodeGen.csproj" + ) + FilesAbsent = @( + ".github" + "src/App.Domain" + "tools/App.Database/dbex.yaml" + "tools/App.Database/Migrations" + ) + FileContains = @{ + "src/App.Infrastructure/App.Infrastructure.csproj" = "CoreEx.Cosmos" + "docker-compose.yml" = "cosmos-emulator" + "tools/App.CodeGen/ref-data.yaml" = "repository: Cosmos" + } + FileNotContains = @{ + "src/App.Infrastructure/App.Infrastructure.csproj" = "EntityFrameworkCore" + } + } + Build = $true + }, + @{ + Name = "coreex-cosmos-no-refdata" + Template = "coreex" + Parameters = @{ + "data-provider" = "Cosmos" + "messaging-provider" = "ServiceBus" + "refdata-enabled" = "false" + "outbox-enabled" = "true" + "rop-enabled" = "false" + } + TestPath = "test-coreex-cosmos-no-refdata" + Verify = @{ + FilesPresent = @( + "src/App.Infrastructure/App.Infrastructure.csproj" + "tools/App.Database/App.Database.csproj" + ) + FilesAbsent = @( + ".github" + "tools/App.CodeGen" + "src/App.Domain" + ) + } + Build = $true + }, + @{ + Name = "coreex-cosmos-no-outbox" + Template = "coreex" + Parameters = @{ + "data-provider" = "Cosmos" + "messaging-provider" = "ServiceBus" + "refdata-enabled" = "true" + "outbox-enabled" = "false" + "rop-enabled" = "false" + } + TestPath = "test-coreex-cosmos-no-outbox" + Verify = @{ + FilesPresent = @( + "src/App.Infrastructure/App.Infrastructure.csproj" + "tools/App.Database/App.Database.csproj" + ) + } + Build = $true + }, + @{ + Name = "coreex-cosmos-full-stack" + Steps = @( + @{ Template = "coreex"; Name = "App"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "true"; "outbox-enabled" = "true"; "rop-enabled" = "false" } } + @{ Template = "coreex-api"; Name = "App.Api"; Parameters = @{ "data-provider" = "Cosmos"; "refdata-enabled" = "true"; "outbox-enabled" = "true" } } + @{ Template = "coreex-relay"; Name = "App.Relay"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "true" } } + @{ Template = "coreex-subscribe"; Name = "App.Subscribe"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "true"; "outbox-enabled" = "true" } } + @{ Template = "coreex-aspire"; Name = "App.Aspire"; Parameters = @{ "has-api" = "true"; "has-relay" = "true"; "has-subscribe" = "true"; "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus" } } + ) + TestPath = "test-cosmos-full-stack" + Verify = @{ + FilesPresent = @( + "aspire/App.Aspire/AppHost.cs" + "src/App.Api/Program.cs" + "src/App.Relay/Program.cs" + "tests/App.Test.Relay/HostTests.cs" + ) + FilesAbsent = @( + # The Cosmos relay test is health-only: a fresh solution has no business mutation to enlist in the outbox transaction. + "tests/App.Test.Relay/RelayTests.cs" + ) + FileContains = @{ + "aspire/App.Aspire/AppHost.cs" = 'AddExternalConnectionString("Cosmos", endpointKey: "AccountEndpoint")' + "src/App.Api/Program.cs" = "AddCosmosDbEventPublisher" + "src/App.Relay/Program.cs" = "AddCosmosDbOutboxRelayHostedService" + "src/App.Subscribe/Program.cs" = "AddCosmosDbEventPublisher" + "tools/App.Database/Program.cs" = "OutboxLeaseContainer" + } + FileNotContains = @{ + # The write-side publisher must never be registered on the Relay host. + "src/App.Relay/Program.cs" = "AddCosmosDbEventPublisher" + } + } + Build = $true + BuildTargets = @( + "aspire/App.Aspire/App.Aspire.csproj" + "aspire/App.Test.Aspire/App.Test.Aspire.csproj" + "tests/App.Test.Api/App.Test.Api.csproj" + "tests/App.Test.Subscribe/App.Test.Subscribe.csproj" + "tests/App.Test.Relay/App.Test.Relay.csproj" + ) + }, + @{ + Name = "coreex-cosmos-hosts-no-refdata-no-outbox" + # Regression guard: with refdata disabled a fresh Cosmos solution has no containers, so the relay hosted service + # and the relay tests that depend on it must be gated out; with the outbox disabled the Subscribe host must not + # register a Cosmos event publisher and Service Bus becomes the default IEventPublisher. + Steps = @( + @{ Template = "coreex"; Name = "App"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "false"; "outbox-enabled" = "false"; "rop-enabled" = "false" } } + @{ Template = "coreex-api"; Name = "App.Api"; Parameters = @{ "data-provider" = "Cosmos"; "refdata-enabled" = "false"; "outbox-enabled" = "false" } } + @{ Template = "coreex-relay"; Name = "App.Relay"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "false" } } + @{ Template = "coreex-subscribe"; Name = "App.Subscribe"; Parameters = @{ "data-provider" = "Cosmos"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "false"; "outbox-enabled" = "false" } } + ) + TestPath = "test-cosmos-hosts-minimal" + Verify = @{ + FileNotContains = @{ + "src/App.Subscribe/Program.cs" = "AddCosmosDbEventPublisher" + "src/App.Relay/Program.cs" = "AddCosmosDbOutboxRelayHostedService" + } + } + Build = $true + BuildTargets = @( + "tests/App.Test.Api/App.Test.Api.csproj" + "tests/App.Test.Subscribe/App.Test.Subscribe.csproj" + "tests/App.Test.Relay/App.Test.Relay.csproj" + ) + }, + @{ + Name = "coreex-subscribe-no-outbox-regression" + # Regression guard: the Subscribe host used to register the SQL Server/Postgres outbox publisher unconditionally, + # and Service Bus as a non-default publisher, even when --outbox-enabled false. + Steps = @( + @{ Template = "coreex"; Name = "App"; Parameters = @{ "data-provider" = "SqlServer"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "false"; "outbox-enabled" = "false"; "rop-enabled" = "false" } } + @{ Template = "coreex-subscribe"; Name = "App.Subscribe"; Parameters = @{ "data-provider" = "SqlServer"; "messaging-provider" = "ServiceBus"; "refdata-enabled" = "false"; "outbox-enabled" = "false" } } + ) + TestPath = "test-subscribe-no-outbox-sqlserver" + Verify = @{ + FileNotContains = @{ + "src/App.Subscribe/Program.cs" = "AddSqlServerOutboxPublisher" } } Build = $true - BuildTarget = "src/App.Aspire/App.Aspire.csproj" + BuildTargets = @( + "src/App.Subscribe/App.Subscribe.csproj" + "tests/App.Test.Subscribe/App.Test.Subscribe.csproj" + ) }, @{ Name = "coreex-domain-regression" @@ -620,7 +864,7 @@ try { # Use dotnet build rather than dotnet pack: GeneratePackageOnBuild=true (from # Directory.Build.props) means the build target already produces the nupkg, and # dotnet pack alone skips compilation on a clean runner (no prior bin/ output). - dotnet build -c Release --nologo + dotnet build -c Release --nologo -m:1 if ($LASTEXITCODE -ne 0) { throw "Failed to build CoreEx.Template" } Pop-Location Write-Pass "Template packed successfully" @@ -640,7 +884,7 @@ try { Write-Header "Building local NuGet feed" # Build the solution so binaries and assets exist before packing individually. - dotnet build (Join-Path $repoRoot "CoreEx.sln") -c Release --nologo + dotnet build (Join-Path $repoRoot "CoreEx.sln") -c Release --nologo -m:1 if ($LASTEXITCODE -ne 0) { throw "Failed to build solution" } $srcRoot = Join-Path $repoRoot "src" @@ -660,6 +904,56 @@ try { } Write-Pass "Local feed populated: $localFeedPath" + # NuGet's global-packages cache is keyed purely by package id + version. Every package here is + # packed at whatever version is currently in Directory.Build.props (typically unchanged between + # local runs), so once a version has been restored once, NuGet will happily keep serving that + # STALE cached copy to every future scaffold — silently ignoring the freshly-packed content above + # and making local source changes invisible to validation. Evict exactly these package ids from + # the global cache before anything restores from the local feed, so each run reflects current source. + $globalPackagesFolder = (dotnet nuget locals global-packages --list) -replace '^global-packages:\s*', '' + $globalPackagesFolder = $globalPackagesFolder.Trim() + if ($globalPackagesFolder -and (Test-Path $globalPackagesFolder)) { + foreach ($proj in $libraryProjects) { + $packageId = $proj.BaseName.ToLowerInvariant() + $cachedPackagePath = Join-Path $globalPackagesFolder $packageId + if (Test-Path $cachedPackagePath) { + Write-Verbose "Evicting stale cached package: $cachedPackagePath" + Remove-Item -Path $cachedPackagePath -Recurse -Force + } + } + Write-Pass "Evicted stale CoreEx.* packages from the global NuGet cache" + } + + # Some CoreEx package versions depend on a paired prerelease of a third-party package (e.g. UnitTestEx.Aspire) + # that may not be published on NuGet.org yet, only in a developer's own private local NuGet source. Adding + # that whole source to the generated nuget.config is dangerous: if it also happens to hold an older, stale + # copy of a CoreEx.* package at the SAME version this script just packed fresh (very possible — it's the + # same developer machine, same unchanged version numbers), NuGet may resolve that stale copy instead of the + # one in $localFeedPath, silently re-introducing the exact staleness bug evicted from the global cache above. + # So instead of adding the source, copy only the packages it holds that this script does NOT itself own + # (i.e. not one of $libraryProjects) into $localFeedPath — this keeps a single trusted source of truth for + # every CoreEx.* package while still filling in missing third-party prerelease dependencies. + $libraryPackageIds = [System.Collections.Generic.HashSet[string]]::new([string[]]($libraryProjects | ForEach-Object { $_.BaseName.ToLowerInvariant() })) + $localFileSources = @(dotnet nuget list source --format short) | ForEach-Object { + if ($_ -match '^(?\S+)\s+(?.+)$') { + $flags = $Matches.flags + $value = $Matches.value.Trim() + if ($flags -notmatch 'D' -and $value -notmatch '^https?://' -and (Test-Path $value)) { $value } + } + } + foreach ($source in $localFileSources) { + Get-ChildItem -Path $source -Filter "*.nupkg" -ErrorAction SilentlyContinue | ForEach-Object { + if ($_.Name -match '^(?.+?)\.(?\d+\.\d+\.\d+(\.\d+)?(-[0-9A-Za-z.]+)?)\.nupkg$') { + $id = $Matches.id.ToLowerInvariant() + $destination = Join-Path $localFeedPath $_.Name + if (-not $libraryPackageIds.Contains($id) -and -not (Test-Path $destination)) { + Write-Verbose "Pulling in third-party package from private local source: $($_.Name)" + Copy-Item -Path $_.FullName -Destination $destination + } + } + } + } + # Step 4: Install template pack (uninstall any existing version first to avoid duplicate registrations) Write-Header "Installing template pack" dotnet new uninstall CoreEx.Template 2>&1 | Out-Null @@ -736,7 +1030,9 @@ try { + nuget.org — avoids failures against unreachable corporate/internal feeds in CI. Any + third-party prerelease dependency only available on a developer's private local source + (e.g. UnitTestEx.Aspire) has already been copied into local-coreex above. --> @@ -746,23 +1042,31 @@ try { Set-Content -Path (Join-Path $testDir "nuget.config") -Value $nugetConfigContent -Encoding utf8 Write-Output " Building generated output..." - if ($scenario.BuildTarget) { - # Explicit target — required for composite scenarios: later steps (hosts) are never - # `dotnet sln add`-ed to the first step's .slnx, so auto-detecting the .slnx would build - # only the `coreex` solution's own projects and silently skip the host being tested. - $buildPath = Join-Path $testDir $scenario.BuildTarget + # Explicit target(s) — required for composite scenarios: later steps (hosts) are never + # `dotnet sln add`-ed to the first step's .slnx, so auto-detecting the .slnx would build + # only the `coreex` solution's own projects and silently skip the host being tested. + # BuildTargets (plural) lets a scenario build more than one entry point — e.g. the Aspire + # scenarios build both the AppHost (which transitively builds every host it references) and + # the Test.Aspire project (which the AppHost does NOT reference, so it would otherwise never compile). + $buildTargets = if ($scenario.BuildTargets) { $scenario.BuildTargets } elseif ($scenario.BuildTarget) { @($scenario.BuildTarget) } else { $null } + + if ($buildTargets) { + $buildPaths = $buildTargets | ForEach-Object { Join-Path $testDir $_ } } else { $buildTarget = (Get-ChildItem $testDir -Filter "*.slnx" -Recurse | Select-Object -First 1) if (-not $buildTarget) { $buildTarget = Get-ChildItem $testDir -Filter "*.sln" -Recurse | Select-Object -First 1 } if (-not $buildTarget) { $buildTarget = Get-ChildItem $testDir -Filter "*.csproj" -Recurse | Select-Object -First 1 } - $buildPath = if ($buildTarget) { $buildTarget.FullName } else { $testDir } + $buildPaths = @(if ($buildTarget) { $buildTarget.FullName } else { $testDir }) } - dotnet build $buildPath --nologo --verbosity minimal 2>&1 | Where-Object { $_ -match "error|warning|succeeded|failed" } - if ($LASTEXITCODE -ne 0) { - $scenarioFailures += "dotnet build failed" - Write-Fail "Build FAILED" - } else { - Write-Pass "Build succeeded" + + foreach ($buildPath in $buildPaths) { + dotnet build $buildPath --nologo --verbosity minimal -m:1 2>&1 | Where-Object { $_ -match "error|warning|succeeded|failed" } + if ($LASTEXITCODE -ne 0) { + $scenarioFailures += "dotnet build failed for '$buildPath'" + Write-Fail "Build FAILED ($buildPath)" + } else { + Write-Pass "Build succeeded ($buildPath)" + } } }