diff --git a/CHANGELOG.md b/CHANGELOG.md index a3d3b5045..18c5c5ae5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,6 +81,19 @@ here.** `fixtures/dependency-conformance/` (the 11 check cases pin the Node `meta deps check`); Kotlin loads through the same loader and adds a codegen test. Spring runtime auto-load and `verify` drift for dependencies are later slices. +- **C# consumes a shared model at build time (FR-023 Phase 2).** `dotnet meta` now reads + `dependencies` from `.metaobjects/config.json` at every rung of its source ladder, verifies the + committed snapshot against `deps.lock.json` offline, and loads the snapshots ahead of the + project's own files, so an `extends`, `overlay: true` or plain FQN reference onto a dependency's + node resolves. Errors carry the same codes as TypeScript and Python + (`ERR_DEPENDENCY_SNAPSHOT_STALE`, `_METAMODEL_INCOMPATIBLE`, `_NODE_COLLISION`, + `_PACKAGE_NOT_OWNED`), and a load error in a dependency's file names `dep:/`. + Imported nodes load but are not generated: every C# generator, the `verify --codegen` + regeneration and the requirement gate's coverage denominator leave them out unless + `scope.include` names their package. C# runs the 26 resolution cases (the 11 check cases pin the Node `meta deps check`) of `fixtures/dependency-conformance/`. + The Node `meta deps sync` stays the only writer of the lock and the snapshot. + See [docs/features/metadata-dependencies.md](docs/features/metadata-dependencies.md). + - **Own your generators, and give back the ones you did not change.** A team is expected to write generators of its own, because a generator it owns is what keeps output from drifting. Two things now help it carry fewer copies than it needs. `meta gen` prints one line naming diff --git a/README.md b/README.md index 673b168b9..2a37f4104 100644 --- a/README.md +++ b/README.md @@ -276,8 +276,8 @@ sixth ships two libraries at their own stability labels: `meta eject `. See [`docs/features/libraries.md`](docs/features/libraries.md). **Sharing a model across your own projects** *(the TypeScript toolchain publishes; -TypeScript, Python, Java and Kotlin projects consume, from an installed npm or Python -package or over a `path` dependency such as a sibling checkout; C# arrives in Phase 2)* — one project publishes +TypeScript, Python, Java, Kotlin and C# projects consume, from an installed npm or Python +package or over a `path` dependency such as a sibling checkout)* — one project publishes part of its model (a common `Customer`, an audited base, a set of enums) with `sharedModelFile()`, and another declares it in `dependencies`, syncs it into a committed, hash-locked snapshot and builds on it with `extends` and references. When the diff --git a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index bb2eb0f07..c6d41380b 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -48,7 +48,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/provider-composition-conformance/`](../fixtures/provider-composition-conformance/) | 9 (5 error-shape + 4 compose-load) | ✓ | ✓ | — (JVM registry via Java) | ✓ | ✓ | | [`fixtures/source-resolution-conformance/`](../fixtures/source-resolution-conformance/) | 25 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/scope-conformance/`](../fixtures/scope-conformance/) | 10 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ | -| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 37 cases (26 resolution + 11 check) + 14 transport cases | ✓ (reference implementation; all three arms) | ✓ (resolution arm, all 26; no check, so the 11 check cases are not run) | inherits via Java | — (Phase 2) | ✓ (resolution arm, all 26 identically; no check yet, so the 11 check cases are not run; the transport arm is Node-only) | +| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 37 cases (26 resolution + 11 check) + 14 transport cases | ✓ (reference implementation; all three arms) | ✓ (resolution arm, all 26; no check, so the 11 check cases are not run) | inherits via Java | ✓ (resolution arm, all 26; no check, so the 11 check cases are not run) | ✓ (resolution arm, all 26 identically; no check yet, so the 11 check cases are not run; the transport arm is Node-only) | | [`fixtures/shared-model-conformance/`](../fixtures/shared-model-conformance/) | 11 cases (7 publish + 4 refusal) | ✓ (reference implementation) | — (Phase 2) | — (Phase 2) | — (Phase 2) | ✓ (serializer arm: the 7 publish cases' document bytes; no publisher) | | [`fixtures/agent-context-conformance/`](../fixtures/agent-context-conformance/) | 4 | ✓ (the emitter is TS-owned) | — | — | — | — | | [`fixtures/metamodel-docs/`](../fixtures/metamodel-docs/) | 1 | ✓ (docs emit is TS-owned) | — | — | — | — | @@ -465,7 +465,7 @@ above: one committed `cases.json`, no per-port fixture, no ledger. The corpus ha - **Resolution (26 cases).** Every case's `expectFiles`/`expectImported`/`expectSelected`/ `expectMigrateGoverned`/`expectLoadError` assertions. **TypeScript (the reference - implementation), Python and Java all run all 26, identically**, with no exemption. The two + implementation), Python, Java and C# all run all 26, identically**, with no exemption. The two overlay cases that need a view child as incidental content use `view.currency` (the one concrete `view.*` subtype registered cross-port), not `view.text` — a prior revision used `view.text`, which Python did not register then, and carried a since-discharged allowlist @@ -482,7 +482,11 @@ above: one committed `cases.json`, no per-port fixture, no ledger. The corpus ha the only writer of snapshots and the lock. `cases.json` also carries an `npm` and a `python` dependency loading from their committed snapshot with nothing installed, which both ports run. -C# has no runner yet; it arrives in a later Phase 2 slice. +**C#** (S6 of FR-023 Phase 2) runs the resolution arm on two rungs of the source ladder: the +neutral one (`Resolve(null, dir)`), and the explicit metadata directory a +`dotnet meta gen ` takes, where the case has one. Its runner is +`DependencyConformanceTests`; it skips the 11 check cases (C# has no `deps check` yet) and the +transport cases (C# never syncs). ### `fixtures/shared-model-conformance/` (11 cases) diff --git a/docs/features/cli.md b/docs/features/cli.md index d692ea2ae..545aaf6ae 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -33,7 +33,7 @@ command surface splits in two: | **Template/prompt drift** (`verify --templates`) | **Node `meta`** | `meta verify --templates` | TS reference (ADR-0021 D2) — `{{field}}`↔payload; part of the bare-`verify` default | | **Vocabulary upgrade** (`upgrade`) | **Node `meta`** | `meta upgrade [--to ] [--apply]` | **any backend** — rewrites RETIRED metadata vocabulary (`@violation` → `@counterexample`, `@readOnly` → `@mutability`, dropping `@verifiedBy`) and resolves ATTRIBUTE CONTRADICTIONS (`@fields` beside `@expr` on an index key). Node-only because it edits the metadata documents themselves, which every port shares; a non-TS project runs `npx meta upgrade` against its own `metaobjects/`. **Canonical JSON and YAML alike.** Previews by default. Retirements needing a human decision are refused and the run exits non-zero, so CI cannot record a partial migration as finished | | **Vocabulary search** (`types`) | **Node `meta`** | `meta types [query]` | **any backend** — apropos/`kubectl explain` over the live metamodel registry (names + descriptions + when-to-use); the vocabulary is cross-port identical (registry-conformance) | -| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency (`path`, or an installed `npm` / `python` package) into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript, Python and Java then LOAD that snapshot at every rung of their own source ladder (Kotlin loads through Java's loader and Maven plugin); C# doesn't read `dependencies` yet (Phase 2). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) | +| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency (`path`, or an installed `npm` / `python` package) into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript, Python and Java then LOAD that snapshot at every rung of their own source ladder (Kotlin loads through Java's loader and Maven plugin); C# loads the snapshot through `dotnet meta` (Phase 2 S6). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) | | **Canonical format** (`fmt`) | every port | `meta fmt` / `dotnet meta fmt` / `mvn metaobjects:fmt` / `metaobjects fmt` | (#304) ships in **every** port — the canonical serializer already exists in all five, so formatting is local, not Node-only. `--check` lists non-canonical files and exits non-zero without changing anything. Formats each file STANDALONE — own-mode, declared-here layer only (ADR-0039); an `extends` onto another file's base is preserved as-is, never resolved or erred on; a file declaring `overlay: true` anywhere — whether or not a same-file base exists — is reported as skipped rather than merged (merging would drop the overlay marker, a structural change fmt never makes). A YAML file is always skipped — no canonical YAML emitter exists (ADR-0006). Before writing, every port reloads the whole project with the candidate swapped in and refuses the write unless the reloaded model's canonical form is byte-identical to the untouched baseline — fmt never changes meaning. See [Fixture corpus](#meta-fmt--canonical-formatting-304) below and `fixtures/fmt-conformance/README.md` | | TS codegen | Node `meta` | `meta gen` | TS projects. **No `--template-spec` flag, deliberately** — `metaobjects.config.ts` already takes generator VALUES, so a declarative template generator is declared there (`templateGenerator()`, or `templateSpecToGenerators(parseTemplateSpec(spec))` to reuse a C#/Python spec file). Keeping it in the config is what lets `meta verify --codegen` regenerate WITH it, since that gate re-runs the config's generator list; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) | | C# codegen | `dotnet meta` | `dotnet meta init` / `gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates` plus `--codegen` when an output directory (`csharp.outDir` in `.metaobjects/config.json`, or `--out`) and `codegen/Generators.cs` are present). **The generator list is `codegen/Generators.cs`** (ADR-0034 Amendment 9): `dotnet meta init` writes it with `codegen/Program.cs` (two lines that run it) and `codegen/Codegen.csproj`, and `gen` / `verify --codegen` always run it through `dotnet run --project codegen`. With no `codegen/Generators.cs` they exit 2 and write nothing; `--generators` only narrows the list to declared entries. Every entry may set its own output directory (`ApiDocsGenerator { OutDir = ... }`), and `verify --codegen` compares each one. A declarative Mustache template-spec (the cross-port JSON contract shared with Python) is one more entry, `TemplateSpecGenerator { Spec = "", TemplateRoot = "" }`; `gen` no longer reads `--template-spec` or a project-root `template-spec.json`, and a run that finds either with no such entry fails before writing anything; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes). `dotnet meta docs` was removed in 1.1.0: it prints the line that points at [the migration section](migrations/docs-are-generators.md#c) and exits 2 | diff --git a/docs/features/metadata-dependencies.md b/docs/features/metadata-dependencies.md index 64821f3ba..2e504d276 100644 --- a/docs/features/metadata-dependencies.md +++ b/docs/features/metadata-dependencies.md @@ -569,16 +569,31 @@ Then `meta deps sync` as for any other installed package. Nothing about the regi reaches `.metaobjects/`: the lock records the transport (`npm`/`python` + `dir`), never a URL. -**Only TypeScript publishes in Phase 1a — TypeScript, Python, Java and Kotlin consume.** -TypeScript, Python, Java and Kotlin can *consume* a dependency (load its snapshot, extend -it, overlay it); C# doesn't read `dependencies` yet (Phase 2). Java reads it at every rung -of the Maven plugin's source ladder (``/``, the neutral config, the default -directory), and Kotlin inherits that through the same plugin and loader, so a Spring -or Kotlin build consumes a shared model with no extra configuration. Only the TypeScript -toolchain can *publish*: it alone can generate the artifact a publisher ships. A Python, -Java or Kotlin project that wants to publish a shared model runs a TypeScript -`sharedModelFile()` generator against its own metadata the same way any TypeScript consumer -would; a C# project cannot yet participate as either a publisher or a consumer. +**Only TypeScript publishes in Phase 1a — TypeScript, Python, Java, Kotlin and C# consume.** +TypeScript, Python, Java, Kotlin and C# can *consume* a dependency (load its snapshot, extend +it, overlay it). Java reads it at every rung of the Maven plugin's source ladder +(``/``, the neutral config, the default directory), and Kotlin inherits that +through the same plugin and loader, so a Spring or Kotlin build consumes a shared model with no +extra configuration. Only the TypeScript toolchain can *publish*: it alone can generate the +artifact a publisher ships, and the Node `meta deps sync` is the only writer of the lock and the +snapshot. A Python, Java, Kotlin or C# project that wants to publish a shared model runs a +TypeScript `sharedModelFile()` generator against its own metadata the same way any TypeScript +consumer would. + +**C#.** `dotnet meta` reads `dependencies` from `.metaobjects/config.json` at every rung of its +source ladder — C# has no native config surface, so the explicit `` form reads the +project's file too — verifies the committed snapshot against `deps.lock.json` offline, and loads +the snapshots first under `dep:/` source ids. It never writes either: run +`meta deps sync` (the Node CLI) to refresh them, and a stale snapshot fails every command with +`ERR_DEPENDENCY_SNAPSHOT_STALE`. An imported node is excluded from every C# generator, the +`verify --codegen` regeneration and the requirement gate's entity-coverage denominator unless +`scope.include` names its package; the one thing C# still emits from an imported package is the +POCO of a *value object* (and the shared enums it uses) that a generated type reaches — an entity +or local value object that holds it, or a local template whose `@payloadRef` / `@responseRef` names +it — because that generated type needs it to compile. `inScope` is Python's formula (no `matchesScope` on the +project's own objects), and C# has no entity-name positional on `gen`, so the TypeScript +"refuse an explicitly named imported object" check has no C# surface. Known edge: a local entity +that `extends` an imported concrete entity in a table-per-hierarchy mapping is not specially handled. ## Vendoring vs. depending @@ -613,10 +628,10 @@ cross-port contract every consumer gets today. ## Phase 2: Java, Kotlin and C# Tracked in [#482](https://github.com/metaobjectsdev/metaobjects/issues/482). Today TypeScript, -Python, Java and Kotlin consume a dependency (Java and Kotlin shipped in slice S3); C# ignores the -`dependencies` key, and a model there that references a node a dependency would supply fails -loudly (`ERR_UNRESOLVED_OBJECT_REF`), never silently. Phase 2 adds no new contract. The manifest, the -lock, the committed snapshot and the 24-case resolution corpus are plain files that every port +Python, Java, Kotlin and C# consume a dependency (Java and Kotlin shipped in slice S3, C# in S6). +A port that does not read the lock fails loudly on a model that references a node a dependency +would supply (`ERR_UNRESOLVED_OBJECT_REF`), never silently. Phase 2 adds no new contract. The manifest, the +lock, the committed snapshot and the 26-case resolution corpus are plain files that every port already has the pieces to read; what is missing is each port *loading* the snapshot. Two rulings shape it: @@ -675,9 +690,8 @@ but **not** built yet — do not assume any of the following exists: - **A runtime `ObjectManager` scope predicate.** Nothing at runtime refuses to serve an imported entity — "what's imported" is a load-time/codegen concept today, not a runtime one. -- **C# as a dependency consumer, and Java, Kotlin and C# as publishers** — Java and - Kotlin consume from Phase 2 slice S3; C# arrives in a later slice. Spring apps do not yet - auto-load the snapshot at runtime (S4). +- **Java, Kotlin, Python and C# as publishers** — Java, Kotlin and C# consume (S3 and S6); + publishing is TypeScript only. Spring apps do not yet auto-load the snapshot at runtime (S4). ## Verified by diff --git a/docs/features/metadata-sources.md b/docs/features/metadata-sources.md index f6ea3abdb..4c61d71ab 100644 --- a/docs/features/metadata-sources.md +++ b/docs/features/metadata-sources.md @@ -57,9 +57,9 @@ sources.** It answers a different question than the ladder above — not "where my metadata live" but "what does my metadata build on that lives elsewhere" — so a TypeScript project using a declared `sources` list and a Python project using its own `metadata:` key both get their declared dependencies resolved the same way, on -top of whichever rung supplied their own files. **TypeScript, Python, Java and Kotlin** -(Kotlin through Java's loader and Maven plugin) — C# doesn't read `dependencies` yet -(Phase 2). See +top of whichever rung supplied their own files. **TypeScript, Python, Java, Kotlin and C#** +(Kotlin through Java's loader and Maven plugin) — each reads `dependencies` at every rung +of its own source ladder. See [`metadata-dependencies.md`](metadata-dependencies.md). A config file that exists but is malformed is an error at its own rung; it never diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index 6f4289905..16c7ee584 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -106,6 +106,18 @@ Drop metadata under `metaobjects/`, for example `metaobjects/meta.blog.json`: }} ``` +### Building on a shared model (optional) + +A model published from another repository is declared under `dependencies` in +`.metaobjects/config.json` and synced by the Node CLI (`meta deps sync` writes +`.metaobjects/deps.lock.json` and a committed snapshot under `.metaobjects/deps/`). `dotnet meta` +reads both, at every rung of the source ladder, and never writes them: a snapshot that no longer +matches the lock fails with `ERR_DEPENDENCY_SNAPSHOT_STALE`. The snapshot loads first, so your +own metadata can `extends`, overlay or reference its nodes, but an imported node is not generated +(and is not counted by the requirement gate) unless `scope.include` names its package. What C# +does and does not do is in +[Metadata dependencies](../features/metadata-dependencies.md). + ### Custom providers (optional) If your app needs a metamodel subtype the core doesn't ship, declare an diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index 54cb17138..3d0cdc883 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -44,8 +44,7 @@ as package data), run `meta gen`, then release the package as usual; consumers declare an `npm` or `python` dependency on it. With no `version` option the manifest takes the nearest `package.json` version, else a `pyproject.toml` `[project]` version — pass `version` when neither is the package consumers install. -Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and -Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and -only the TypeScript toolchain can *publish* one; C# does not read -`dependencies` yet (Phase 2). +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java, Kotlin (through +Java's loader and Maven plugin) and C# can *consume* a dependency, and only the +TypeScript toolchain can *publish* one. Full detail: `docs/features/metadata-dependencies.md`. diff --git a/fixtures/dependency-conformance/README.md b/fixtures/dependency-conformance/README.md index 21be4139c..6239a36d3 100644 --- a/fixtures/dependency-conformance/README.md +++ b/fixtures/dependency-conformance/README.md @@ -247,8 +247,13 @@ climbs out of the package, and a Python module that is not a package are like Python's. - **Kotlin** — inherits via Java: it loads through the same loader and Maven plugin. The Kotlin generators are covered by a codegen test that an imported entity emits nothing. -- **C#** — Phase 2 (does not read `dependencies` yet). Runs the resolution arm when it consumes, - and the check arm when it gains a check. +- **C#** — the resolution arm: resolution, load-time failure, lock/snapshot integrity, on the + neutral source ladder and again through the explicit metadata directory (C# has no native + config surface, so `.metaobjects/config.json` is where a dependency is declared at every + rung). `inScope` is Python's formula, not TypeScript's. Not the check arm: C# has no + `deps check` yet (that is `verify --deps`), so its runner skips the `expectCheck` cases. Not the + transport arm: C# never syncs. The runner is + `server/csharp/MetaObjects.Conformance.Tests/DependencyConformanceTests.cs`. ## Reference implementation diff --git a/server/csharp/MetaObjects.Cli.Tests/DependencyCliTests.cs b/server/csharp/MetaObjects.Cli.Tests/DependencyCliTests.cs new file mode 100644 index 000000000..cb98d46af --- /dev/null +++ b/server/csharp/MetaObjects.Cli.Tests/DependencyCliTests.cs @@ -0,0 +1,92 @@ +// FR-023: `dotnet meta verify` and `fmt` consume a project's metadata dependencies. The corpus +// (fixtures/dependency-conformance/) pins the resolution; these pin that the CLI surface reads it. +using System.Text.Json; +using Xunit; + +namespace MetaObjects.Cli.Tests; + +public sealed class DependencyCliTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "meta-dep-cli-" + Guid.NewGuid().ToString("N")); + + public DependencyCliTests() => Directory.CreateDirectory(_root); + public void Dispose() { try { Directory.Delete(_root, recursive: true); } catch { } } + + private static string CorpusDir() + { + var dir = new DirectoryInfo(AppContext.BaseDirectory); + while (dir is not null && !Directory.Exists(Path.Combine(dir.FullName, "fixtures", "dependency-conformance"))) + dir = dir.Parent; + return Path.Combine(dir!.FullName, "fixtures", "dependency-conformance"); + } + + private void Write(string rel, string content) + { + var path = Path.Combine(_root, rel); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, content); + } + + /// The corpus's one-dependency project, with a local entity that EXTENDS an imported one. + private void Project() + { + using var doc = JsonDocument.Parse(File.ReadAllText(Path.Combine(CorpusDir(), "cases.json"))); + var c = doc.RootElement.GetProperty("cases").EnumerateArray() + .Single(x => x.GetProperty("name").GetString() == "a-dependency-adds-its-artifact-to-the-resolved-set"); + foreach (var f in c.GetProperty("tree").EnumerateObject()) Write(f.Name, f.Value.GetString()!); + var snapshot = Path.Combine(_root, ".metaobjects", "deps", "acme-common", "acme-common.metaobjects.json"); + Directory.CreateDirectory(Path.GetDirectoryName(snapshot)!); + File.Copy(Path.Combine(CorpusDir(), "artifacts", "acme-common-v1.json"), snapshot); + Write(".metaobjects/config.json", c.GetProperty("config").GetRawText()); + Write(".metaobjects/deps.lock.json", c.GetProperty("lock").GetRawText()); + // A bare `verify` also runs the template gate; an empty directory is enough for it. + Directory.CreateDirectory(Path.Combine(_root, "templates")); + Write("metaobjects/meta.vip.json", + """{"metadata.root":{"package":"app","children":[{"object.entity":{"name":"Vip","extends":"acme::common::Customer","children":[{"source.rdb":{"@table":"vips"}}]}}]}}"""); + } + + [Fact] + public void Verify_resolves_an_extends_onto_an_imported_node() + { + Project(); + var (exit, stdout, stderr) = CliProcess.Run(_root, "verify", "--no-field-lint"); + Assert.True(exit == 0, stdout + stderr); + } + + [Fact] + public void Verify_codegen_regenerates_what_gen_wrote_with_a_dependency() + { + Project(); + var gen = OwnedRunner.Run(_root, ["entity"], "gen", "--out", "generated", "--namespace", "Acme.Generated"); + Assert.True(gen.ExitCode == 0, gen.Stdout + gen.Stderr); + Assert.False(Directory.EnumerateFiles(Path.Combine(_root, "generated"), "*Customer*", SearchOption.AllDirectories).Any(), + "an imported entity must not be generated"); + + var verify = OwnedRunner.Run(_root, ["entity"], "verify", "--codegen", "--out", "generated", "--namespace", "Acme.Generated"); + Assert.True(verify.ExitCode == 0, verify.Stdout + verify.Stderr); + } + + [Fact] + public void Fmt_resolves_the_dependency_and_never_rewrites_its_snapshot() + { + Project(); + var snapshot = Path.Combine(_root, ".metaobjects", "deps", "acme-common", "acme-common.metaobjects.json"); + var before = File.ReadAllBytes(snapshot); + + var (exit, stdout, stderr) = CliProcess.Run(_root, "fmt"); + Assert.True(exit == 0, stdout + stderr); + Assert.Equal(before, File.ReadAllBytes(snapshot)); + } + + [Fact] + public void A_stale_snapshot_fails_the_command_and_names_the_fix() + { + Project(); + File.AppendAllText(Path.Combine(_root, ".metaobjects", "deps", "acme-common", "acme-common.metaobjects.json"), "\n"); + + var (exit, stdout, stderr) = CliProcess.Run(_root, "verify", "--no-field-lint"); + Assert.NotEqual(0, exit); + Assert.Contains("ERR_DEPENDENCY_SNAPSHOT_STALE", stdout + stderr); + Assert.Contains("meta deps sync", stdout + stderr); + } +} diff --git a/server/csharp/MetaObjects.Cli/FieldLint.cs b/server/csharp/MetaObjects.Cli/FieldLint.cs index 294799e76..a07aa27e8 100644 --- a/server/csharp/MetaObjects.Cli/FieldLint.cs +++ b/server/csharp/MetaObjects.Cli/FieldLint.cs @@ -206,14 +206,15 @@ public static List LintDuplicateFields(IEnumerable files) /// exit code. Loads LENIENT so the findings are the same with and without --lax; /// a load failure is the gate's to report, so it stays silent here. /// - public static void RunAdvisory(string metadataDir, IReadOnlyList? metadataFiles, IReadOnlyList? libraries, TextWriter stderr) + public static void RunAdvisory( + string metadataDir, IReadOnlyList? metadataFiles, IReadOnlyList? libraries, TextWriter stderr, + MetaObjects.Config.ImportScope? imports = null) { List findings; try { - var load = metadataFiles is { } resolved - ? MetaDataLoader.FromUris(resolved.Select(f => new Uri(f)).ToList(), libraries, false) - : MetaDataLoader.FromDirectory(metadataDir, libraries, strict: false); + var load = new MetaObjects.Config.ResolvedMetadata( + metadataDir, metadataFiles, libraries ?? [], MetaObjects.Meta.PhysicalNaming.Default, null, null, imports).Load(); if (load.Errors.Count > 0) return; var files = metadataFiles ?? new DirectorySource(metadataDir).Expand().Select(f => f.FilePath).ToList(); findings = [.. LintReferenceFields(load.Root), .. LintDuplicateFields(files)]; diff --git a/server/csharp/MetaObjects.Cli/FmtCommand.cs b/server/csharp/MetaObjects.Cli/FmtCommand.cs index db2485929..d080c7843 100644 --- a/server/csharp/MetaObjects.Cli/FmtCommand.cs +++ b/server/csharp/MetaObjects.Cli/FmtCommand.cs @@ -183,6 +183,9 @@ public static RunResult Run(ResolvedMetadata resolved, bool check, IReadOnlyList List BuildSources(string? overridePath = null, string? overrideText = null) { var sources = new List(LibrarySources.Resolve(resolved.Libraries)); + // FR-023: a dependency's snapshot is loaded so the project's `extends` onto it resolves, but it is + // never a fmt TARGET: `allFiles` is the project's own metadata and the snapshot is not that. + if (resolved.Imports is { } imports) sources.AddRange(imports.Sources()); foreach (var p in allFiles) { sources.Add(overridePath is not null && p == overridePath diff --git a/server/csharp/MetaObjects.Cli/Program.cs b/server/csharp/MetaObjects.Cli/Program.cs index eac1ca106..541f5cf41 100644 --- a/server/csharp/MetaObjects.Cli/Program.cs +++ b/server/csharp/MetaObjects.Cli/Program.cs @@ -531,6 +531,7 @@ static int RunVerify(string[] rest) // FR-043 — read regardless of ladder vs. explicit (see // ResolveMetadataDirOrExit). Libraries = resolvedMeta.Libraries, + Imports = resolvedMeta.Imports, // ADR-0058 — same source as Libraries; the --codegen gate's regen reads it from // the loaded root rather than a flag (see VerifyCommand.RunCodegenDrift). Naming = resolvedMeta.Naming, @@ -612,7 +613,7 @@ static int RunVerify(string[] rest) // field name declared twice in one children list. Both load clean on every port. Runs // on every `verify`, whichever gates were selected; warnings ONLY, never the exit code. if (!noFieldLint && Environment.GetEnvironmentVariable(FieldLint.EnvOptOut) != "1") - FieldLint.RunAdvisory(resolvedMeta.Directory, resolvedMeta.Files, resolvedMeta.Libraries, Console.Error); + FieldLint.RunAdvisory(resolvedMeta.Directory, resolvedMeta.Files, resolvedMeta.Libraries, Console.Error, resolvedMeta.Imports); // The handed-off codegen gate already printed its own verdict (inherited console); // fold its exit code into the aggregate the same way every other subverb does — max, diff --git a/server/csharp/MetaObjects.Cli/VerifyCommand.cs b/server/csharp/MetaObjects.Cli/VerifyCommand.cs index 34d3bc89f..7a8a578ce 100644 --- a/server/csharp/MetaObjects.Cli/VerifyCommand.cs +++ b/server/csharp/MetaObjects.Cli/VerifyCommand.cs @@ -98,6 +98,13 @@ public sealed record Options /// public IReadOnlyList Libraries { get; init; } = Array.Empty(); /// + /// FR-023 — the project's resolved metadata dependencies, from the same + /// .metaobjects/config.json as . Their committed snapshots load + /// first, so an extends onto an imported node resolves, and the loaded root records what + /// was imported so the --codegen gate regenerates exactly what gen wrote. Null: none. + /// + public MetaObjects.Config.ImportScope? Imports { get; init; } + /// /// ADR-0058 — the project's physical-naming rules, read from the SAME /// .metaobjects/config.json as (Program.cs's /// ResolveMetadataDirOrExit). Set on the loaded root by @@ -255,13 +262,11 @@ public static SubverbResult RunSubverbs(Options opts) /// private static LoadResult LoadMetadata(Options opts) { - var result = opts.MetadataFiles is { } files - ? MetaDataLoader.FromUris(files.Select(f => new Uri(f)).ToList(), opts.Libraries, opts.Strict) - : MetaDataLoader.FromDirectory(opts.MetadataDir, opts.Libraries, strict: opts.Strict); - // ADR-0058 — every resolver reads naming.columns/naming.tables from the root. - result.Root.PhysicalNaming = opts.Naming; - result.Root.WireFormat = opts.Wire; - return result; + // The same load `gen` runs (ResolvedMetadata.Load): libraries, then the dependency snapshots, then the + // project's own files; ADR-0058 naming and the wire format set on the root; the imports recorded on it. + return new MetaObjects.Config.ResolvedMetadata( + opts.MetadataDir, opts.MetadataFiles, opts.Libraries, opts.Naming, opts.Wire, null, opts.Imports) + .Load(opts.Strict); } // ------------------------------------------------------------------------ diff --git a/server/csharp/MetaObjects.Codegen.Tests/DependencyCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/DependencyCodegenTests.cs new file mode 100644 index 000000000..dcaac174c --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/DependencyCodegenTests.cs @@ -0,0 +1,165 @@ +using System.Text.Json; +using System.Text.RegularExpressions; +using MetaObjects.Config; +using MetaObjects.Meta; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +/// +/// FR-023: metadata a project IMPORTS loads so its own model resolves against it, but is not the +/// project's to generate. Every registered generator must leave an imported node out of its output; +/// the one exception is a value object a generated entity holds, whose POCO the entity's property type +/// needs to compile. The corpus (fixtures/dependency-conformance/) pins the predicates; this pins that +/// the generators honour them. +/// +public sealed class DependencyCodegenTests : IDisposable +{ + private readonly string _tmp = Path.Combine(Path.GetTempPath(), "meta-dep-codegen-" + Guid.NewGuid().ToString("N")); + private string OutDir => Path.Combine(_tmp, "generated"); + + // render-helper needs a template root for its drift gate; template is a primitive configured by its caller. + private static readonly string[] Excluded = ["render-helper", "template"]; + + public void Dispose() { try { Directory.Delete(_tmp, recursive: true); } catch { } } + + private static JsonElement Case(string name) + { + var path = Path.Combine(CorpusPaths.RepoRoot(), "fixtures", "dependency-conformance", "cases.json"); + using var doc = JsonDocument.Parse(File.ReadAllText(path)); + return doc.RootElement.GetProperty("cases").EnumerateArray() + .Single(c => c.GetProperty("name").GetString() == name).Clone(); + } + + /// + /// Lays out the corpus's one-dependency project (a local app::Order, an imported + /// acme::common with Address / Audited / Customer), plus an optional extra local file. + /// + private void Project(string? extraLocalFile = null) + { + var c = Case("a-dependency-adds-its-artifact-to-the-resolved-set"); + foreach (var f in c.GetProperty("tree").EnumerateObject()) + Write(f.Name, f.Value.GetString()!); + foreach (var f in c.GetProperty("treeFiles").EnumerateObject()) + { + var dest = Path.Combine(_tmp, f.Name); + Directory.CreateDirectory(Path.GetDirectoryName(dest)!); + File.Copy(Path.Combine(CorpusPaths.RepoRoot(), "fixtures", "dependency-conformance", f.Value.GetString()!), dest); + } + Write(".metaobjects/config.json", c.GetProperty("config").GetRawText()); + Write(".metaobjects/deps.lock.json", c.GetProperty("lock").GetRawText()); + if (extraLocalFile is not null) Write("metaobjects/meta.extra.json", extraLocalFile); + } + + private void Write(string rel, string content) + { + var path = Path.Combine(_tmp, rel); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, content); + } + + private Dictionary GenerateAll() + { + var load = MetadataLocation.Resolve(null, _tmp).Load(); + Assert.Empty(load.Errors); + // The import is on the loaded root: that is what the generators read. + Assert.NotNull(load.Root.ImportScope); + var config = new GenConfig { OutDir = OutDir, Namespace = "Acme.Generated" }; + var names = GeneratorRegistry.Entries.Keys.Where(n => !Excluded.Contains(n)).ToList(); + CodegenRunner.Run(config, load.Root, GeneratorRegistry.Resolve(names)); + return Directory.EnumerateFiles(OutDir, "*", SearchOption.AllDirectories) + .ToDictionary(p => Path.GetRelativePath(OutDir, p).Replace('\\', '/'), File.ReadAllText); + } + + private static bool DeclaresClass(Dictionary output, string name) => + output.Any(kv => kv.Key.EndsWith(".cs", StringComparison.Ordinal) && Regex.IsMatch(kv.Value, $@"\bclass {name}\b")); + + [Fact] + public void An_imported_node_emits_nothing_from_any_registered_generator() + { + Project(); + var output = GenerateAll(); + + // The project's own entity IS generated, so the run is not vacuous. + Assert.Contains(output, kv => kv.Key.Contains("Order")); + Assert.Contains(output, kv => kv.Value.Contains("Order")); + + foreach (var (path, text) in output) + { + Assert.DoesNotContain("Customer", path); + Assert.DoesNotContain("Address", path); + Assert.DoesNotContain("Audited", path); + Assert.False(text.Contains("Customer") || text.Contains("Address") || text.Contains("acme.common", StringComparison.OrdinalIgnoreCase), + $"{path} mentions an imported node"); + } + } + + [Fact] + public void An_imported_value_object_a_local_entity_holds_is_still_emitted() + { + Project(""" + { "metadata.root": { "package": "app", "children": [ + { "object.entity": { "name": "Invoice", "children": [ + { "source.rdb": { "@table": "invoices" } }, + { "field.long": { "name": "id" } }, + { "field.object": { "name": "billTo", "@objectRef": "acme::common::Address", "@storage": "jsonb" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ]}} + ]}} + """); + var output = GenerateAll(); + + // The entity's property type needs the POCO to compile, so it is the one imported shape emitted. + Assert.True(DeclaresClass(output, "Address")); + // The imported ENTITY that nothing local holds is still not emitted. + Assert.DoesNotContain(output, kv => kv.Key.Contains("Customer") || kv.Value.Contains("Customer")); + } + + [Fact] + public void An_imported_value_object_a_local_value_object_holds_is_still_emitted() + { + Project(""" + { "metadata.root": { "package": "app", "children": [ + { "object.value": { "name": "InvoicePrompt", "children": [ + { "field.string": { "name": "note" } }, + { "field.object": { "name": "billTo", "@objectRef": "acme::common::Address" } } + ]}} + ]}} + """); + var output = GenerateAll(); + + // No entity holds Address: the local value object's property type alone needs its POCO. + Assert.True(DeclaresClass(output, "InvoicePrompt")); + Assert.True(DeclaresClass(output, "Address")); + Assert.DoesNotContain(output, kv => kv.Key.Contains("Customer") || kv.Value.Contains("Customer")); + } + + [Fact] + public void An_imported_value_object_a_local_prompt_names_is_still_emitted() + { + Project(""" + { "metadata.root": { "package": "app", "children": [ + { "template.prompt": { "name": "AddressCheck", "@payloadRef": "acme::common::Address", + "@responseRef": "acme::common::Address", "@textRef": "address-check.mustache", "@format": "json" } } + ]}} + """); + var output = GenerateAll(); + + // The template tier types its payload and its reply by the value object's POCO and declares none of its own. + Assert.True(DeclaresClass(output, "Address")); + Assert.Contains(output, kv => kv.Key.Contains("AddressCheck")); + Assert.DoesNotContain(output, kv => kv.Key.Contains("Customer") || kv.Value.Contains("Customer")); + } + + [Fact] + public void Naming_the_package_in_scope_include_generates_it() + { + Project(); + var cfgPath = Path.Combine(_tmp, ".metaobjects", "config.json"); + var cfg = JsonDocument.Parse(File.ReadAllText(cfgPath)).RootElement; + Write(".metaobjects/config.json", cfg.GetRawText().TrimEnd('}') + ",\"scope\":{\"include\":[\"acme::common::**\",\"app::**\"]}}"); + + var output = GenerateAll(); + Assert.Contains(output, kv => kv.Key.Contains("Customer")); + } +} diff --git a/server/csharp/MetaObjects.Codegen/ApiDocs/CSharpApiModelBuilder.cs b/server/csharp/MetaObjects.Codegen/ApiDocs/CSharpApiModelBuilder.cs index 28ff94044..b60f9bafa 100644 --- a/server/csharp/MetaObjects.Codegen/ApiDocs/CSharpApiModelBuilder.cs +++ b/server/csharp/MetaObjects.Codegen/ApiDocs/CSharpApiModelBuilder.cs @@ -61,7 +61,7 @@ public CSharpApiModel Build(MetaRoot root, string project) var units = new List(); // Objects: one unit per concrete object.entity / object.value. - foreach (var obj in root.Objects()) + foreach (var obj in root.Objects().Where(root.InScope)) { // FR-044: a SERVED report (its read source is a view) is documented from its row // model, the same object the generators emit from, so its unit carries exactly diff --git a/server/csharp/MetaObjects.Codegen/CodegenRunner.cs b/server/csharp/MetaObjects.Codegen/CodegenRunner.cs index 77da05327..71ab61ead 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenRunner.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenRunner.cs @@ -58,7 +58,11 @@ internal static RunResult Run( // generators that emit those ask for the report's row model themselves, through // ReportRows, so every other generator stays report-free without having to know // what a report is. - Entities = root.Objects().Where(o => !o.IsReport()).ToList(), + // + // FR-023: an object a metadata dependency owns is dropped here too, for the same reason + // and at the same place (root.InScope; ImportScope.InScope is the rule). It loaded so the + // project's own model resolves against it, but it is not this project's to generate. + Entities = root.Objects().Where(o => !o.IsReport() && root.InScope(o)).ToList(), Root = root, Config = config, Warn = warnings.Add, @@ -69,7 +73,7 @@ internal static RunResult Run( // entity set every DbSet/route/finder-emitting generator below reads from. // A view-backed report gets a DbSet too, so it is part of the collision check. CSharpNaming.AssertNoCollectionNameCollisions( - [.. ctx.Entities, .. root.Objects().Where(ReportRows.IsViewBacked)]); + [.. ctx.Entities, .. root.Objects().Where(o => ReportRows.IsViewBacked(o) && root.InScope(o))]); Directory.CreateDirectory(config.OutDir); var results = new List(); diff --git a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs index bef3a811f..cbf1848f5 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/EntityGenerator.cs @@ -104,7 +104,7 @@ public virtual IEnumerable Generate(GenContext ctx) // namespace; consuming entity fields REFERENCE them (no nested redeclaration). @provided // shared enums emit nothing here (referenced from configured external namespace). With no // shared enums in the model (the common inline case) no file is emitted — byte-identical. - if (EmitSharedEnumsFile(ctx) is { } sharedEnums) + if (EmitSharedEnumsFile(ctx, valueObjects) is { } sharedEnums) files.Add(sharedEnums); foreach (var e in mapped) @@ -943,15 +943,19 @@ private static string EnumPropertyTypeName(MetaObject owner, MetaField field, Ge // Referencing a @provided enum with no ProvidedEnumNamespace configured throws here too // (the collect walk constructs each consuming reference's type), surfacing the config gap // as a codegen-time error before any file is written. - protected virtual EmittedFile? EmitSharedEnumsFile(GenContext ctx) + protected virtual EmittedFile? EmitSharedEnumsFile(GenContext ctx, IReadOnlyList valueShapes) { // Surface a missing ProvidedEnumNamespace for any consumed @provided enum as a // codegen-time error (ADR-0026), even though @provided emits no file content. - foreach (var shared in Fr019SharedEnum.CollectSharedEnums(ctx.Root)) + // FR-023: an imported value object this run emits a POCO for (a generated type needs it), so + // the enums IT uses are part of the output too. + var importedShapes = valueShapes.Where(vo => !ctx.Root.InScope(vo)).ToList(); + + foreach (var shared in Fr019SharedEnum.CollectSharedEnums(ctx.Root, importedShapes)) if (shared.Provided) _ = Fr019SharedEnum.SharedEnumTypeReference(shared, ctx.Config); - var materialized = Fr019SharedEnum.MaterializedSharedEnums(ctx.Root); + var materialized = Fr019SharedEnum.MaterializedSharedEnums(ctx.Root, importedShapes); if (materialized.Count == 0) return null; var sb = new StringBuilder(); @@ -1943,7 +1947,9 @@ protected static bool OriginGuaranteedNonNull(MetaField field) private static bool ReferencesValueObject(MetaField f) => (f.SubType == FIELD_SUBTYPE_OBJECT || f.SubType == FIELD_SUBTYPE_MAP) && f.ObjectRef is not null; - // Transitive closure of plain value objects reachable through entity object-fields. + // Transitive closure of plain value objects reachable from every type this run emits: entity + // object-fields, callable args, in-scope value shapes' object-fields, and template payload/response + // shapes. FR-023: an imported value object reaches the output only through this walk. protected static List ReferencedValueObjects(IReadOnlyList mapped, GenContext ctx) { var seen = new HashSet(StringComparer.Ordinal); @@ -1964,6 +1970,17 @@ protected static List ReferencedValueObjects(IReadOnlyList !o.IsAbstract && ValueObjectNames.IsValueShape(o))) + foreach (var f in shape.Fields().Where(ReferencesValueObject)) + Enqueue(ValueObjectNames.ResolveFieldRef(f, ctx.Root)); + + // ADR-0056: the template tier types a render helper by its @payloadRef shape and a parser by its + // @responseRef shape, and declares neither, so each must have a POCO here. + foreach (var tmpl in ctx.Root.Children().Where(c => c.Type == global::MetaObjects.Shared.BaseTypes.TYPE_TEMPLATE)) + foreach (var attr in new[] { global::MetaObjects.Template.TemplateConstants.TEMPLATE_ATTR_PAYLOAD_REF, global::MetaObjects.Template.TemplateConstants.TEMPLATE_ATTR_RESPONSE_REF }) + if (ValueObjectNames.PayloadShapeOf(tmpl, attr) is MetaObject { IsAbstract: false } shape && seen.Add(shape.ResolutionKey())) + queue.Enqueue(shape); + while (queue.Count > 0) { var vo = queue.Dequeue(); diff --git a/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs b/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs index 7325f6bce..97106df98 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/Fr019SharedEnum.cs @@ -92,11 +92,14 @@ private static string PackageOf(MetaField decl) /// type name. A declaration nobody extends is not materialized (no dangling type). /// Deterministic order: first-consumption order across entities (ordinal by name). /// - public static IReadOnlyList CollectSharedEnums(MetaRoot root) + public static IReadOnlyList CollectSharedEnums(MetaRoot root, IEnumerable? alsoReached = null) { var seen = new HashSet(StringComparer.Ordinal); var outList = new List(); - foreach (var entity in root.Objects()) + // FR-023: an enum reaches the output iff an object this project generates uses it, decided by the + // USING object, never by the enum's own package. `alsoReached` is what that object pulls in that + // is itself imported (a value object whose POCO is emitted because a generated entity holds one). + foreach (var entity in root.Objects().Where(root.InScope).Concat(alsoReached ?? [])) foreach (var field in entity.Fields()) { if (SharedEnumForField(field) is not { } shared) continue; @@ -106,8 +109,8 @@ public static IReadOnlyList CollectSharedEnums(MetaRoot root) } /// The shared enums that metaobjects MATERIALIZES (non-@provided, consumed). - public static IReadOnlyList MaterializedSharedEnums(MetaRoot root) => - CollectSharedEnums(root).Where(e => !e.Provided).ToList(); + public static IReadOnlyList MaterializedSharedEnums(MetaRoot root, IEnumerable? alsoReached = null) => + CollectSharedEnums(root, alsoReached).Where(e => !e.Provided).ToList(); /// /// The C# type reference a consuming field emits for a shared/provided enum: diff --git a/server/csharp/MetaObjects.Codegen/ReportRows.cs b/server/csharp/MetaObjects.Codegen/ReportRows.cs index afe8752bd..71dc44320 100644 --- a/server/csharp/MetaObjects.Codegen/ReportRows.cs +++ b/server/csharp/MetaObjects.Codegen/ReportRows.cs @@ -209,9 +209,13 @@ private static MetaSource CopySource(MetaSource source) return copy; } - /// The row model of every view-backed report in the model, in declaration order. + /// + /// The row model of every view-backed report in the model, in declaration order. A report a metadata + /// dependency owns is not this project's to serve, so it has no row model here (FR-023, + /// ). + /// public static IReadOnlyList For(MetaRoot root) => - root.Objects().Where(IsViewBacked).Select(r => RowModel(r, root)).ToList(); + root.Objects().Where(r => IsViewBacked(r) && root.InScope(r)).Select(r => RowModel(r, root)).ToList(); /// /// The objects a report-aware generator iterates: the run's entity set with every diff --git a/server/csharp/MetaObjects.Codegen/TemplateCodegen/ScopeWalk.cs b/server/csharp/MetaObjects.Codegen/TemplateCodegen/ScopeWalk.cs index 3baae9016..e2a7c270d 100644 --- a/server/csharp/MetaObjects.Codegen/TemplateCodegen/ScopeWalk.cs +++ b/server/csharp/MetaObjects.Codegen/TemplateCodegen/ScopeWalk.cs @@ -26,7 +26,8 @@ public static Func> ForScope(string sc $"unknown template scope '{scope}' (expected {string.Join(" | ", Scopes)})"); return root => { - var objects = root.Objects(); + // FR-023: an object a metadata dependency owns is not this project's to render a template for. + var objects = root.Objects().Where(root.InScope).ToList(); var concrete = objects.Where(TemplateData.IsConcrete).ToList(); switch (scope) { diff --git a/server/csharp/MetaObjects.Conformance.Tests/DependencyConformanceTests.cs b/server/csharp/MetaObjects.Conformance.Tests/DependencyConformanceTests.cs new file mode 100644 index 000000000..6889324f0 --- /dev/null +++ b/server/csharp/MetaObjects.Conformance.Tests/DependencyConformanceTests.cs @@ -0,0 +1,204 @@ +// Runs the shared dependency corpus (FR-023) against this port. Reads the single committed +// fixtures/dependency-conformance/cases.json — no per-port fixture; its README is the case schema. +// The reference runners are server/typescript/packages/sdk/test/dependency-conformance.test.ts and +// server/python/tests/conformance/test_dependency_conformance.py. +// +// C# has no native config surface, so a dependency is declared in `.metaobjects/config.json` and +// resolved at every rung of MetadataLocation.Resolve. Each case runs on the NEUTRAL ladder +// (`Resolve(null, resolveDir)`), and again through the EXPLICIT metadata-directory rung +// (`Resolve(/metaobjects, ...)`) whenever the case has such a directory — the rung a +// `dotnet meta gen ` takes, which must read the same dependencies. +using System.Text.Json; +using MetaObjects; +using MetaObjects.Config; +using MetaObjects.Meta; +using MetaObjects.Source; +using Xunit; +using static MetaObjects.Shared.BaseTypes; + +namespace MetaObjects.Conformance.Tests; + +public class DependencyConformanceTests : IDisposable +{ + private static readonly string CorpusDir = FindCorpusDir(); + private static readonly JsonElement AllCases = LoadCases(); + + private readonly string _tmp = Directory.CreateTempSubdirectory("mo-dep-conformance-").FullName; + + public void Dispose() + { + try { Directory.Delete(_tmp, true); } catch (IOException) { /* temp dir, best effort */ } + } + + private static string FindCorpusDir() + { + var dir = new DirectoryInfo(AppContext.BaseDirectory); + while (dir is not null && !Directory.Exists(Path.Combine(dir.FullName, "fixtures", "dependency-conformance"))) + dir = dir.Parent; + Assert.NotNull(dir); + return Path.Combine(dir!.FullName, "fixtures", "dependency-conformance"); + } + + private static JsonElement LoadCases() + { + using var doc = JsonDocument.Parse(File.ReadAllText(Path.Combine(FindCorpusDir(), "cases.json"))); + // The corpus has two arms. The resolution arm is this runner's; the check arm (`expectCheck`) is + // `deps check`'s, which this port does not have yet (that is `verify --deps`). + var resolution = doc.RootElement.GetProperty("cases").EnumerateArray() + .Where(c => !c.TryGetProperty("expectCheck", out _)).ToList(); + return JsonSerializer.SerializeToElement(resolution); + } + + public static TheoryData CaseNames() + { + var data = new TheoryData(); + foreach (var c in AllCases.EnumerateArray()) data.Add(c.GetProperty("name").GetString()!); + return data; + } + + [Fact] + public void CorpusIsNotEmpty() => + // A parametrized theory over an empty list collects zero tests and passes; this makes a corpus + // that quietly lost its cases a failure. + Assert.NotEmpty(CaseNames()); + + /// + /// Materializes tree and treeFiles (copied byte-for-byte, so a pinned hash can never + /// drift) under the temp root, then writes config / lock under + /// <resolveFrom>/.metaobjects/. Returns the directory resolution is invoked against. + /// + private string Materialize(JsonElement c) + { + foreach (var f in c.GetProperty("tree").EnumerateObject()) + WriteFile(f.Name, f.Value.GetString()!); + + if (c.TryGetProperty("treeFiles", out var treeFiles)) + foreach (var f in treeFiles.EnumerateObject()) + { + var dest = Path.Combine(_tmp, f.Name); + Directory.CreateDirectory(Path.GetDirectoryName(dest)!); + File.Copy(Path.Combine(CorpusDir, f.Value.GetString()!), dest); + } + + var resolveDir = Path.Combine(_tmp, c.TryGetProperty("resolveFrom", out var rf) ? rf.GetString()! : "."); + var metaobjects = Path.Combine(resolveDir, ".metaobjects"); + if (c.GetProperty("config").ValueKind != JsonValueKind.Null) + { + Directory.CreateDirectory(metaobjects); + File.WriteAllText(Path.Combine(metaobjects, "config.json"), c.GetProperty("config").GetRawText()); + } + if (c.TryGetProperty("lock", out var lockJson)) + { + Directory.CreateDirectory(metaobjects); + File.WriteAllText(Path.Combine(metaobjects, DependencyFiles.LockFile), lockJson.GetRawText()); + } + return Path.GetFullPath(resolveDir); + } + + private void WriteFile(string rel, string content) + { + var path = Path.Combine(_tmp, rel); + Directory.CreateDirectory(Path.GetDirectoryName(path)!); + File.WriteAllText(path, content); + } + + private static List Strings(JsonElement arr) => arr.EnumerateArray().Select(e => e.GetString()!).ToList(); + + private static List Sorted(IEnumerable xs) => xs.Order(StringComparer.Ordinal).ToList(); + + /// Every loaded top-level object's resolution key — the set the three exhaustive arms range over. + private static List TopLevelFqns(MetaRoot root) => + // ADR-0039 sanctioned: a ROOT-level scan; `metadata.root` is never extended, so own and effective + // children are the same set, and the arms are defined over "every loaded top-level object". + root.Objects().Select(o => o.ResolutionKey()).ToList(); + + private string Relative(string path) => Path.GetRelativePath(_tmp, path).Replace('\\', '/'); + + [Theory] + [MemberData(nameof(CaseNames))] + public void CorpusCaseMatchesTheReference(string name) + { + var c = AllCases.EnumerateArray().Single(x => x.GetProperty("name").GetString() == name); + var resolveDir = Materialize(c); + // The explicit rung: `dotnet meta gen ` must read the same dependencies. + var explicitDir = Path.Combine(resolveDir, NeutralConfig.DefaultMetadataDir); + var hasExplicitRung = Directory.Exists(explicitDir); + + if (c.TryGetProperty("expectError", out var expectError)) + { + var ex = Assert.Throws(() => MetadataLocation.Resolve(null, resolveDir)); + Assert.Equal(expectError.GetString(), ex.Code.ToString()); + if (hasExplicitRung) + { + var explicitEx = Assert.Throws(() => MetadataLocation.Resolve(explicitDir, _tmp)); + Assert.Equal(expectError.GetString(), explicitEx.Code.ToString()); + } + return; + } + + Assert.True(c.TryGetProperty("expectFiles", out var expectFiles), + $"corpus case \"{name}\" has neither expectFiles nor expectError"); + + var resolved = MetadataLocation.Resolve(null, resolveDir); + AssertResolvedFiles(resolved, expectFiles, name); + AssertArms(c, resolved, name); + + if (hasExplicitRung) + { + var explicitResolved = MetadataLocation.Resolve(explicitDir, _tmp); + var expectedArtifacts = Strings(expectFiles).Where(f => f.Split('/').Contains(".metaobjects")); + var gotArtifacts = (explicitResolved.Imports?.Dependencies.Select(d => d.ArtifactPath) ?? []).Select(Relative); + Assert.Equal(Sorted(expectedArtifacts), Sorted(gotArtifacts)); + AssertArms(c, explicitResolved, name + " (explicit metadata dir)"); + } + } + + /// Dependency artifacts then the project's own files; compared as a set AND by length. + private void AssertResolvedFiles(ResolvedMetadata resolved, JsonElement expectFiles, string name) + { + var got = (resolved.Imports?.Dependencies.Select(d => d.ArtifactPath) ?? []) + .Concat(resolved.Files ?? []).Select(Relative).ToList(); + var expected = Strings(expectFiles); + Assert.True(Sorted(expected).SequenceEqual(Sorted(got.Distinct())), $"{name}: files\n got {string.Join(", ", got)}"); + Assert.Equal(expected.Count, got.Count); + } + + private static void AssertArms(JsonElement c, ResolvedMetadata resolved, string name) + { + var needsLoad = c.TryGetProperty("expectImported", out _) || c.TryGetProperty("expectSelected", out _) || + c.TryGetProperty("expectMigrateGoverned", out _) || c.TryGetProperty("expectLoadError", out _); + if (!needsLoad) return; + + var result = resolved.Load(); + var imports = resolved.Imports ?? ImportScope.None; + + if (c.TryGetProperty("expectLoadError", out var expectLoadError)) + { + Assert.True(result.Errors.Count > 0, $"{name}: expected a load error but the collection loaded cleanly"); + var first = result.Errors[0]; + Assert.Equal(expectLoadError.GetString(), first.Code.ToString()); + if (c.TryGetProperty("expectErrorFiles", out var expectErrorFiles)) + { + IReadOnlyList files = first.Envelope switch + { + JsonSource j => j.Files, + YamlSource y => y.Files, + _ => first.Source is { } s ? [s] : [], + }; + Assert.Equal(Strings(expectErrorFiles), files.ToList()); + } + return; + } + + Assert.True(result.Errors.Count == 0, + $"{name}: {string.Join("; ", result.Errors.Select(e => $"{e.Code}: {e.Message}"))}"); + var topLevel = TopLevelFqns(result.Root); + + if (c.TryGetProperty("expectImported", out var expectImported)) + Assert.Equal(Sorted(Strings(expectImported)), Sorted(topLevel.Where(imports.Imported))); + if (c.TryGetProperty("expectSelected", out var expectSelected)) + Assert.Equal(Sorted(Strings(expectSelected)), Sorted(topLevel.Where(imports.InScope))); + if (c.TryGetProperty("expectMigrateGoverned", out var expectGoverned)) + Assert.Equal(Sorted(Strings(expectGoverned)), Sorted(topLevel.Where(imports.InMigrateScope))); + } +} diff --git a/server/csharp/MetaObjects.Conformance.Tests/DependencyUnitTests.cs b/server/csharp/MetaObjects.Conformance.Tests/DependencyUnitTests.cs new file mode 100644 index 000000000..0240b6697 --- /dev/null +++ b/server/csharp/MetaObjects.Conformance.Tests/DependencyUnitTests.cs @@ -0,0 +1,157 @@ +// FR-023 — the pieces of the consume side the shared corpus does not enumerate: the config's +// `dependencies` / `scope.include` / `migrate.scope` shape, the lock's shape, and the package-naming rule. +using MetaObjects; +using MetaObjects.Config; +using Xunit; + +namespace MetaObjects.Conformance.Tests; + +public sealed class DependencyUnitTests : IDisposable +{ + private readonly string _root = Directory.CreateTempSubdirectory("mo-dep-unit-").FullName; + + public void Dispose() + { + try { Directory.Delete(_root, true); } catch (IOException) { /* temp dir */ } + } + + private void WriteConfig(string json) + { + Directory.CreateDirectory(Path.Combine(_root, ".metaobjects")); + File.WriteAllText(Path.Combine(_root, ".metaobjects", "config.json"), json); + } + + private void WriteLock(string json) + { + Directory.CreateDirectory(Path.Combine(_root, ".metaobjects")); + File.WriteAllText(Path.Combine(_root, ".metaobjects", DependencyFiles.LockFile), json); + } + + // ---- explicitlyIncludes ------------------------------------------------------------------ + + [Theory] + [InlineData("acme::common::**", "acme::common", true)] + [InlineData("acme::common::Address", "acme::common", true)] + [InlineData("acme::**", "acme::common", false)] + [InlineData("**", "acme::common", false)] + [InlineData("acme::*::Address", "acme::common", false)] + [InlineData("acme::common::**", "acme::other", false)] + [InlineData("acme::common::**", "", false)] + public void ExplicitlyIncludes_requires_the_package_to_be_named_literally(string pattern, string pkg, bool expected) => + Assert.Equal(expected, DependencyResolver.ExplicitlyIncludes([pattern], pkg)); + + [Fact] + public void ExplicitlyIncludes_names_nothing_for_an_absent_or_empty_list() + { + Assert.False(DependencyResolver.ExplicitlyIncludes(null, "acme::common")); + Assert.False(DependencyResolver.ExplicitlyIncludes([], "acme::common")); + } + + // ---- the config's dependency shape ------------------------------------------------------- + + private void AssertConfigRefused(string dependenciesJson) + { + WriteConfig($$"""{ "schema_version": 1, "dependencies": {{dependenciesJson}} }"""); + var ex = Assert.Throws(() => NeutralConfig.Read(_root)); + Assert.Equal(ErrorCode.ERR_BAD_ATTR_VALUE, ex.Code); + } + + [Fact] + public void A_valid_dependency_is_read_with_its_transport() + { + WriteConfig(""" + { "schema_version": 1, "dependencies": [ + { "name": "a", "path": "../a/metaobjects" }, + { "name": "b.c-d", "npm": "@acme/common", "dir": "metaobjects" } ] } + """); + var cfg = NeutralConfig.Read(_root)!; + Assert.Equal([new DependencySpec("a", "path", "../a/metaobjects", null), + new DependencySpec("b.c-d", "npm", "@acme/common", "metaobjects")], cfg.Dependencies); + } + + [Theory] + [InlineData("""[{ "name": "Bad", "path": "x" }]""")] // uppercase name + [InlineData("""[{ "name": "a" }]""")] // no transport + [InlineData("""[{ "name": "a", "path": "x", "npm": "y" }]""")] // two transports + [InlineData("""[{ "name": "a", "path": "x", "dir": "d" }]""")] // dir beside path + [InlineData("""[{ "name": "a", "path": "x", "mode": "snapshot" }]""")] // removed key + [InlineData("""[{ "name": "a", "path": "" }]""")] // empty location + [InlineData("""[{ "name": "a", "path": "x" }, { "name": "a", "python": "y" }]""")] // duplicate name + [InlineData("""{ "name": "a", "path": "x" }""")] // not an array + public void A_malformed_dependency_declaration_is_refused(string dependenciesJson) => + AssertConfigRefused(dependenciesJson); + + [Fact] + public void Scope_and_migrate_scope_lists_are_read_and_other_keys_of_those_blocks_are_ignored() + { + WriteConfig(""" + { "schema_version": 1, "scope": { "include": ["app::**"], "exclude": ["app::Old"] }, + "migrate": { "scope": ["app::**"], "somethingTypeScriptOwns": 1 } } + """); + var cfg = NeutralConfig.Read(_root)!; + Assert.Equal(["app::**"], cfg.ScopeInclude); + Assert.Equal(["app::**"], cfg.MigrateScope); + } + + [Fact] + public void Absent_scope_lists_read_as_empty_and_null() + { + WriteConfig("""{ "schema_version": 1, "scope": null }"""); + var cfg = NeutralConfig.Read(_root)!; + Assert.Empty(cfg.ScopeInclude); + Assert.Null(cfg.MigrateScope); + } + + // ---- the lock --------------------------------------------------------------------------- + + private const string Hash = "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d"; + + private static string Lock(string entryExtras = "", string resolvedFrom = """{ "path": "../a" }""", + string packages = """["acme::common"]""") => $$""" + { "schema_version": 1, "dependencies": { "a": { + "version": "1.0.0", "metamodelVersion": "1.0", "artifact": "a.metaobjects.json", + "integrity": "{{Hash}}", "packages": {{packages}}, "nodes": ["acme::common::X"], + "resolvedFrom": {{resolvedFrom}} {{entryExtras}} } } } + """; + + [Fact] + public void A_well_formed_lock_is_read() + { + WriteLock(Lock()); + var entries = DependencyResolver.ReadLock(_root)!; + Assert.Equal(["acme::common"], entries["a"].Packages); + Assert.Equal(Hash, entries["a"].Integrity); + } + + [Fact] + public void No_lock_file_reads_as_null() => Assert.Null(DependencyResolver.ReadLock(_root)); + + [Theory] + [InlineData(""", "mode": "snapshot" """, """{ "path": "../a" }""", """["acme::common"]""")] // removed key + [InlineData("", """{ "path": "../a", "npm": "x" }""", """["acme::common"]""")] // two transports + [InlineData("", """{ "path": "../a", "dir": "d" }""", """["acme::common"]""")] // dir beside path + [InlineData("", """{ "path": "../a" }""", """["b::x", "a::y"]""")] // unsorted packages + public void A_malformed_lock_is_stale_never_trusted(string extras, string resolvedFrom, string packages) + { + WriteLock(Lock(extras, resolvedFrom, packages)); + var ex = Assert.Throws(() => DependencyResolver.ReadLock(_root)); + Assert.Equal(ErrorCode.ERR_DEPENDENCY_SNAPSHOT_STALE, ex.Code); + } + + [Fact] + public void Unparseable_lock_json_is_stale() + { + WriteLock("{ not json"); + var ex = Assert.Throws(() => DependencyResolver.ReadLock(_root)); + Assert.Equal(ErrorCode.ERR_DEPENDENCY_SNAPSHOT_STALE, ex.Code); + } + + [Fact] + public void Every_stale_refusal_names_the_command_that_fixes_it() + { + WriteConfig("""{ "schema_version": 1, "dependencies": [{ "name": "a", "path": "x" }] }"""); + var ex = Assert.Throws(() => MetadataLocation.Resolve(null, _root)); + Assert.Equal(ErrorCode.ERR_DEPENDENCY_SNAPSHOT_STALE, ex.Code); + Assert.EndsWith("; run `meta deps sync`", ex.Message); + } +} diff --git a/server/csharp/MetaObjects/Config/Dependencies.cs b/server/csharp/MetaObjects/Config/Dependencies.cs new file mode 100644 index 000000000..9076ca429 --- /dev/null +++ b/server/csharp/MetaObjects/Config/Dependencies.cs @@ -0,0 +1,522 @@ +// FR-023 — metadata dependencies, the CONSUME side: the `dependencies` key of +// `.metaobjects/config.json`, the lock (`deps.lock.json`) and the committed snapshot beside it. +// +// Mirrors server/typescript/packages/sdk/src/dependencies.ts and +// server/python/src/metaobjects/config/dependencies.py, in the same order: validate the lock, +// check each snapshot against it (offline), collide-check the exported nodes, then load the +// snapshots FIRST under `dep:/` source ids. The shared corpus +// fixtures/dependency-conformance/ pins the behaviour; this file adds no design of its own. +// +// The Node `meta deps sync` is the ONLY writer of the lock and the snapshot. This port reads and +// verifies them; it never writes either. +// +// A dependency's nodes are LOAD-ONLY by default: they load so the project's own model resolves +// against them, but the codegen selection point drops them unless the project's own +// `scope.include` NAMES their package (see ImportScope.InScope). +using System.Security.Cryptography; +using System.Text.Json; +using System.Text.RegularExpressions; +using MetaObjects.Loader; +using MetaObjects.Meta; + +namespace MetaObjects.Config; + +/// The cross-port file names and prefixes of a dependency set (DESIGN §3). +public static class DependencyFiles +{ + /// Directory under .metaobjects/ holding the synced snapshots, one subdirectory per dependency. + public const string DepsDir = "deps"; + + /// meta deps sync's output. + public const string LockFile = "deps.lock.json"; + + /// Suffix of a dependency's canonical-JSON artifact, e.g. acme-common.metaobjects.json. + public const string ArtifactSuffix = ".metaobjects.json"; + + /// Prefix of a dependency artifact's source id: dep:<name>/<artifact>. + public const string SourceIdPrefix = "dep:"; + + /// Prefix of an integrity value: sha256- plus the lowercase hex sha256 of the artifact bytes. + public const string IntegrityPrefix = "sha256-"; +} + +/// One validated entry of the config's dependencies array. +/// The LOCAL alias the config, lock and .metaobjects/deps/<name>/ key on. +/// The one transport key: path, npm or python. +/// The transport key's value. +/// The optional dir, legal only beside npm / python. +public sealed record DependencySpec(string Name, string Transport, string Location, string? Dir); + +/// One validated entry of the lock's dependencies object. +public sealed record DependencyLockEntry( + string Version, string MetamodelVersion, string Artifact, string Integrity, + IReadOnlyList Packages, IReadOnlyList Nodes); + +/// A dependency that passed the lock check, ready for the loader. +public sealed record ResolvedDependency( + string Name, string Version, IReadOnlyList Packages, IReadOnlyList Nodes, + string ArtifactPath, string SourceId); + +public static partial class DependencyResolver +{ + private static readonly Regex NameRe = NameRegex(); + private static readonly Regex MetamodelVersionRe = MetamodelVersionRegex(); + private static readonly Regex IntegrityRe = IntegrityRegex(); + + [GeneratedRegex("^[a-z0-9][a-z0-9._-]*$")] + private static partial Regex NameRegex(); + + [GeneratedRegex(@"^\d+\.\d+$")] + private static partial Regex MetamodelVersionRegex(); + + [GeneratedRegex("^sha256-[0-9a-f]{64}$")] + private static partial Regex IntegrityRegex(); + + private static readonly string[] TransportKeys = ["path", "npm", "python"]; + private static readonly string[] LockKeys = ["schema_version", "dependencies"]; + private static readonly string[] LockEntryKeys = + ["version", "metamodelVersion", "artifact", "integrity", "packages", "nodes", "resolvedFrom"]; + + private const string MetaObjectsDir = ".metaobjects"; + + // ------------------------------------------------------------------ + // The config's `dependencies` array + // ------------------------------------------------------------------ + + /// + /// Validate one entry of dependencies: a name, exactly one transport key, an + /// optional dir legal only beside npm / python, and nothing else (a + /// mode key included — it was removed). Mirrors the TypeScript DependencySpecSchema. + /// + internal static DependencySpec ReadSpec(JsonElement dep, string configPath) + { + MetaModelException Fail(string reason) => new( + $"{configPath}: invalid \"dependencies\" entry ({reason})", ErrorCode.ERR_BAD_ATTR_VALUE); + + if (dep.ValueKind != JsonValueKind.Object) throw Fail("must be an object"); + + if (!dep.TryGetProperty("name", out var nameEl) || nameEl.ValueKind != JsonValueKind.String || + !NameRe.IsMatch(nameEl.GetString()!)) + throw Fail("\"name\" must match ^[a-z0-9][a-z0-9._-]*$"); + var name = nameEl.GetString()!; + + var transports = TransportKeys.Where(k => dep.TryGetProperty(k, out _)).ToList(); + if (transports.Count != 1) throw Fail("exactly one of \"path\" / \"npm\" / \"python\" is required"); + var transport = transports[0]; + + var location = dep.GetProperty(transport); + if (location.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(location.GetString())) + throw Fail($"\"{transport}\" must be a non-empty string"); + + var allowed = new HashSet(StringComparer.Ordinal) { "name", transport }; + if (transport is "npm" or "python") allowed.Add("dir"); + var extra = dep.EnumerateObject().Select(p => p.Name).Where(n => !allowed.Contains(n)) + .Order(StringComparer.Ordinal).ToList(); + if (extra.Count > 0) throw Fail($"unknown key(s): [{string.Join(", ", extra)}]"); + + string? dir = null; + if (dep.TryGetProperty("dir", out var dirEl)) + { + if (dirEl.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(dirEl.GetString())) + throw Fail("\"dir\" must be a non-empty string"); + dir = dirEl.GetString(); + } + + return new DependencySpec(name, transport, location.GetString()!, dir); + } + + // ------------------------------------------------------------------ + // The lock + // ------------------------------------------------------------------ + + private static MetaModelException Stale(string message) => + new(message, ErrorCode.ERR_DEPENDENCY_SNAPSHOT_STALE); + + /// Every stale-snapshot refusal, in one place, so each ends with the command that fixes it. + private static MetaModelException StaleSnapshot(string detail) => Stale($"{detail}; run `meta deps sync`"); + + private static bool IsStrictlyAscending(IReadOnlyList values) + { + for (var i = 0; i + 1 < values.Count; i++) + if (string.CompareOrdinal(values[i], values[i + 1]) >= 0) return false; + return true; + } + + private static IReadOnlyList SortedStrings(JsonElement obj, string field, string context) + { + if (!obj.TryGetProperty(field, out var el) || el.ValueKind != JsonValueKind.Array || + el.EnumerateArray().Any(v => v.ValueKind != JsonValueKind.String || v.GetString()!.Length == 0)) + throw Stale($"{context}: \"{field}\" must be an array of non-empty strings"); + var values = el.EnumerateArray().Select(v => v.GetString()!).ToList(); + if (!IsStrictlyAscending(values)) throw Stale($"{context}: \"{field}\" must be sorted"); + return values; + } + + private static string RequiredString(JsonElement obj, string field, string context, string what) + { + if (!obj.TryGetProperty(field, out var el) || el.ValueKind != JsonValueKind.String || + string.IsNullOrEmpty(el.GetString())) + throw Stale($"{context}: \"{field}\" must be {what}"); + return el.GetString()!; + } + + private static void ValidateResolvedFrom(JsonElement entry, string context) + { + if (!entry.TryGetProperty("resolvedFrom", out var rf) || rf.ValueKind != JsonValueKind.Object) + throw Stale($"{context}: \"resolvedFrom\" must be an object"); + + var transports = TransportKeys.Where(k => rf.TryGetProperty(k, out _)).ToList(); + if (transports.Count != 1) + throw Stale($"{context}: \"resolvedFrom\" must have exactly one of \"path\" / \"npm\" / \"python\""); + var transport = transports[0]; + + var value = rf.GetProperty(transport); + if (value.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(value.GetString())) + throw Stale($"{context}: \"resolvedFrom.{transport}\" must be a non-empty string"); + + var allowed = new HashSet(StringComparer.Ordinal) { transport }; + if (transport is "npm" or "python") allowed.Add("dir"); + var extra = rf.EnumerateObject().Select(p => p.Name).Where(n => !allowed.Contains(n)) + .Order(StringComparer.Ordinal).ToList(); + if (extra.Count > 0) throw Stale($"{context}: \"resolvedFrom\" unknown key(s): [{string.Join(", ", extra)}]"); + + if (rf.TryGetProperty("dir", out var dir) && + (dir.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(dir.GetString()))) + throw Stale($"{context}: \"resolvedFrom.dir\" must be a non-empty string"); + } + + /// + /// Validate a parsed deps.lock.json (DESIGN §3.3). Any shape violation — unsorted keys, an + /// entry with an unknown key (a resurrected mode included), two resolvedFrom + /// transports, a malformed field — is ERR_DEPENDENCY_SNAPSHOT_STALE. Entries come back in + /// dependency-NAME order. + /// + internal static IReadOnlyDictionary ValidateLock(JsonElement root) + { + const string context = DependencyFiles.LockFile; + if (root.ValueKind != JsonValueKind.Object) throw Stale($"{context}: must be an object"); + + var extraTop = root.EnumerateObject().Select(p => p.Name).Where(n => !LockKeys.Contains(n)) + .Order(StringComparer.Ordinal).ToList(); + if (extraTop.Count > 0) throw Stale($"{context}: unknown key(s): [{string.Join(", ", extraTop)}]"); + + if (!root.TryGetProperty("schema_version", out var sv) || sv.ValueKind != JsonValueKind.Number || + !sv.TryGetDouble(out var schemaVersion) || schemaVersion != 1) + throw Stale($"{context}: \"schema_version\" must be 1"); + + if (!root.TryGetProperty("dependencies", out var deps) || deps.ValueKind != JsonValueKind.Object) + throw Stale($"{context}: \"dependencies\" must be an object"); + + var names = deps.EnumerateObject().Select(p => p.Name).ToList(); + if (!IsStrictlyAscending(names)) throw Stale($"{context}: dependency keys must be sorted"); + + var entries = new SortedDictionary(StringComparer.Ordinal); + foreach (var prop in deps.EnumerateObject()) + { + var entry = prop.Value; + var entryContext = $"{context} ({prop.Name})"; + if (entry.ValueKind != JsonValueKind.Object) throw Stale($"{entryContext}: must be an object"); + + var extra = entry.EnumerateObject().Select(p => p.Name).Where(n => !LockEntryKeys.Contains(n)) + .Order(StringComparer.Ordinal).ToList(); + if (extra.Count > 0) throw Stale($"{entryContext}: unknown key(s): [{string.Join(", ", extra)}]"); + + var version = RequiredString(entry, "version", entryContext, "a non-empty string"); + var metamodel = RequiredString(entry, "metamodelVersion", entryContext, "a string matching ^\\d+\\.\\d+$"); + if (!MetamodelVersionRe.IsMatch(metamodel)) + throw Stale($"{entryContext}: \"metamodelVersion\" must match ^\\d+\\.\\d+$"); + var artifact = RequiredString(entry, "artifact", entryContext, $"a string ending with '{DependencyFiles.ArtifactSuffix}'"); + if (!artifact.EndsWith(DependencyFiles.ArtifactSuffix, StringComparison.Ordinal)) + throw Stale($"{entryContext}: \"artifact\" must end with '{DependencyFiles.ArtifactSuffix}'"); + var integrity = RequiredString(entry, "integrity", entryContext, "a string matching ^sha256-[0-9a-f]{64}$"); + if (!IntegrityRe.IsMatch(integrity)) + throw Stale($"{entryContext}: \"integrity\" must match ^sha256-[0-9a-f]{{64}}$"); + var packages = SortedStrings(entry, "packages", entryContext); + var nodes = SortedStrings(entry, "nodes", entryContext); + ValidateResolvedFrom(entry, entryContext); + + entries[prop.Name] = new DependencyLockEntry(version, metamodel, artifact, integrity, packages, nodes); + } + return entries; + } + + /// + /// Read <configDir>/.metaobjects/deps.lock.json. Null when the file does not exist: a + /// project declaring no dependencies has no lock, and that is not an error. A present but + /// malformed file raises. + /// + public static IReadOnlyDictionary? ReadLock(string configDir) + { + var path = Path.Combine(configDir, MetaObjectsDir, DependencyFiles.LockFile); + if (!File.Exists(path)) return null; + JsonDocument doc; + try + { + doc = JsonDocument.Parse(File.ReadAllText(path)); + } + catch (Exception e) when (e is JsonException or IOException) + { + throw Stale($"{path} exists but could not be read as JSON: {e.Message}"); + } + using (doc) return ValidateLock(doc.RootElement); + } + + /// sha256- plus the lowercase hex sha256 of . + public static string Sha256Integrity(byte[] data) => + DependencyFiles.IntegrityPrefix + Convert.ToHexString(SHA256.HashData(data)).ToLowerInvariant(); + + /// dep:<name>/<artifact> — a dependency artifact's source id. + public static string SourceId(string name, string artifact) => + $"{DependencyFiles.SourceIdPrefix}{name}/{artifact}"; + + private static string MetamodelMajor(string version) => version.Split('.')[0]; + + /// + /// Verify a project's committed snapshot against its lock, and resolve the dependencies the load + /// will take FIRST. Offline: nothing here touches a publisher. Two failures are NOT staleness and + /// get their own codes: a dependency published against a different metamodel MAJOR + /// (ERR_DEPENDENCY_METAMODEL_INCOMPATIBLE, ADR-0035 Amendment 2) and two dependencies + /// exporting one fully-qualified node (ERR_DEPENDENCY_NODE_COLLISION). + /// Result order is dependency NAME order, never the config's declaration order: the artifacts + /// lead the load, so declaration order must not decide what the loader sees first. A project + /// with no dependencies and no lock resolves to an empty list without touching the filesystem. + /// + public static IReadOnlyList VerifySnapshot( + string configDir, IReadOnlyList specs, + IReadOnlyDictionary? lockEntries) + { + var declared = specs.Select(s => s.Name).ToList(); + + if (lockEntries is null) + { + if (declared.Count == 0) return []; + throw StaleSnapshot( + $"{declared.Count} dependenc{(declared.Count == 1 ? "y is" : "ies are")} declared " + + $"({string.Join(", ", declared)}) but there is no {MetaObjectsDir}/{DependencyFiles.LockFile}"); + } + + var declaredNames = declared.ToHashSet(StringComparer.Ordinal); + foreach (var name in lockEntries.Keys) + if (!declaredNames.Contains(name)) + throw StaleSnapshot( + $"{MetaObjectsDir}/{DependencyFiles.LockFile} locks dependency \"{name}\", which " + + $"{MetaObjectsDir}/config.json no longer declares"); + + var resolved = new List(); + foreach (var name in declaredNames.Order(StringComparer.Ordinal)) + { + if (!lockEntries.TryGetValue(name, out var entry)) + throw StaleSnapshot( + $"dependency \"{name}\" is declared but {MetaObjectsDir}/{DependencyFiles.LockFile} has no entry for it"); + + var artifactPath = Path.Combine(configDir, MetaObjectsDir, DependencyFiles.DepsDir, name, entry.Artifact); + byte[] data; + try + { + data = File.ReadAllBytes(artifactPath); + } + catch (Exception e) when (e is IOException or UnauthorizedAccessException) + { + throw StaleSnapshot($"the committed snapshot for \"{name}\" is missing (expected {artifactPath})"); + } + + var actual = Sha256Integrity(data); + if (actual != entry.Integrity) + throw StaleSnapshot( + $"the committed snapshot for \"{name}\" does not match the lock — {artifactPath} " + + $"hashes to {actual}, the lock records {entry.Integrity}"); + + if (MetamodelMajor(entry.MetamodelVersion) != MetamodelMajor(RegistryManifest.MetamodelVersion)) + throw new MetaModelException( + $"dependency \"{name}\" was published against metamodel {entry.MetamodelVersion}; " + + $"this toolchain speaks {RegistryManifest.MetamodelVersion}. A different metamodel MAJOR is a " + + $"different metadata contract — upgrade the toolchain, or use a release of \"{name}\" " + + "built against it.", + ErrorCode.ERR_DEPENDENCY_METAMODEL_INCOMPATIBLE); + + resolved.Add(new ResolvedDependency( + name, entry.Version, entry.Packages, entry.Nodes, artifactPath, SourceId(name, entry.Artifact))); + } + + // Collision is checked across the WHOLE resolved set, so the error names the two + // dependencies in name order however the config declared them. + var owner = new Dictionary(StringComparer.Ordinal); + foreach (var dep in resolved) + foreach (var node in dep.Nodes) + { + if (owner.TryGetValue(node, out var prior)) + throw new MetaModelException( + $"dependencies \"{prior}\" and \"{dep.Name}\" both export \"{node}\" — one " + + "fully-qualified node cannot come from two places, and whichever loaded " + + "second would silently win", + ErrorCode.ERR_DEPENDENCY_NODE_COLLISION); + owner[node] = dep.Name; + } + + return resolved; + } + + // ------------------------------------------------------------------ + // Package-keyed predicates + // ------------------------------------------------------------------ + + /// A fully-qualified name minus its trailing ::name; "" for a root-level name. + public static string PackageOf(string key) + { + var i = key.LastIndexOf(PACKAGE_SEPARATOR, StringComparison.Ordinal); + return i == -1 ? "" : key[..i]; + } + + /// + /// Does some pattern in name LITERALLY (DESIGN §11.1 + /// item 2)? Drop the pattern's final segment, which names the node: what remains must be + /// wildcard-free and equal . So acme::common::** and + /// acme::common::Address both name acme::common; acme::** and ** reach + /// its nodes but name nothing, and an absent or empty list names nothing. NOT + /// : a pattern that MATCHES a package's nodes is a weaker statement + /// than one that NAMES the package. + /// + public static bool ExplicitlyIncludes(IReadOnlyList? patterns, string pkg) + { + if (patterns is null || patterns.Count == 0 || pkg.Length == 0) return false; + foreach (var pattern in patterns) + { + var named = PackageOf(pattern); + if (named.Length > 0 && !named.Contains('*') && named == pkg) return true; + } + return false; + } +} + +/// +/// Everything the dependency-aware load resolved for one project: the dependencies whose snapshots +/// lead the load, and the predicates every action surface reads to decide whether an imported object +/// is excluded (DESIGN §11.1 item 2). Attached to the loaded as +/// , beside the project's naming and wire settings, so a generator +/// reads the same answer wherever it sits. +/// +public sealed class ImportScope +{ + /// A project that imports nothing: every predicate answers as if dependencies did not exist. + public static ImportScope None { get; } = new([], [], null); + + /// The resolved dependencies, in dependency-NAME order. + public IReadOnlyList Dependencies { get; } + + /// THE exclusion key (DESIGN §11.5): the union of every dependency's packages. + public IReadOnlySet ImportedPackages { get; } + + /// + /// The union of every dependency's nodes, read only by to tell + /// an overlay of an imported node from a genuinely new local declaration. NOT the exclusion key. + /// + public IReadOnlySet ImportedNodes { get; } + + /// The project's declared scope.include, for the explicit-include rule. + public IReadOnlyList ScopeInclude { get; } + + /// The project's declared migrate.scope, or null when it declares none. + public IReadOnlyList? MigrateScope { get; } + + private readonly CompiledScope? _migrate; + + private ImportScope(IReadOnlyList dependencies, IReadOnlyList scopeInclude, IReadOnlyList? migrateScope) + { + Dependencies = dependencies; + ImportedPackages = dependencies.SelectMany(d => d.Packages).ToHashSet(StringComparer.Ordinal); + ImportedNodes = dependencies.SelectMany(d => d.Nodes).ToHashSet(StringComparer.Ordinal); + ScopeInclude = scopeInclude; + MigrateScope = migrateScope; + // `migrate.scope` is compiled up front so a malformed pattern is a config error here, not a + // surprise at the first predicate call. Undefined (no `migrate.scope`, no dependency) admits all. + _migrate = migrateScope is null && dependencies.Count == 0 ? null : Scope.CompileScope(migrateScope); + } + + /// + /// Resolve the dependency set of the project whose .metaobjects/ directory sits in + /// : read the lock, check every snapshot against it, and gather the + /// predicates' inputs from . Throws with + /// the ERR_DEPENDENCY_* code when the snapshot cannot be trusted. + /// + public static ImportScope Resolve(string configDir, NeutralConfig? config) + { + var specs = config?.Dependencies ?? []; + var scopeInclude = config?.ScopeInclude ?? []; + var migrateScope = config?.MigrateScope; + + var lockEntries = DependencyResolver.ReadLock(configDir); + var resolved = specs.Count == 0 && lockEntries is null + ? [] + : DependencyResolver.VerifySnapshot(configDir, specs, lockEntries); + return new ImportScope(resolved, scopeInclude, migrateScope); + } + + /// Is in a package one of this project's dependencies owns? + public bool Imported(string fqn) => ImportedPackages.Contains(DependencyResolver.PackageOf(fqn)); + + /// + /// The output filter for codegen: an object this project imports is not its own to generate, unless + /// its package is NAMED by scope.include. !Imported(fqn) || ExplicitlyIncludes(scope.include, packageOf(fqn)). + /// Python's shape, not TypeScript's: no matchesScope conjunct on the project's OWN objects, + /// because neither this port nor Python has ever scoped its own generated output. Only the + /// import-exclusion half is new behaviour here; scope.include narrows nothing of the + /// project's own, and only ever widens which imported package is let back in. + /// + public bool InScope(string fqn) + { + var pkg = DependencyResolver.PackageOf(fqn); + return !ImportedPackages.Contains(pkg) || DependencyResolver.ExplicitlyIncludes(ScopeInclude, pkg); + } + + /// + /// The schema-side twin of : matchesScope(fqn, migrate.scope) and the + /// same import rule against migrate.scope. True for everything when nothing is declared and + /// nothing is imported. Nothing in this port consumes it (schema is TypeScript-owned, ADR-0015); it + /// is here so the corpus's expectMigrateGoverned arm passes with no exemption. + /// + public bool InMigrateScope(string fqn) + { + if (_migrate is null) return true; + if (!Scope.MatchesScope(fqn, _migrate)) return false; + var pkg = DependencyResolver.PackageOf(fqn); + return !ImportedPackages.Contains(pkg) || DependencyResolver.ExplicitlyIncludes(MigrateScope, pkg); + } + + /// + /// The snapshot artifacts as loader sources, in dependency-NAME order, each under its + /// dep:<name>/<artifact> id so a load error names the dependency rather than a local file. + /// + public IReadOnlyList Sources() => + Dependencies.Select(d => (IMetaDataSource)new FileSource(d.ArtifactPath, d.SourceId)).ToList(); + + /// + /// FR-023 §11.5 — a consumer may not declare a NEW top-level node into a package one of its + /// dependencies owns. Imported-ness is decided by PACKAGE, so such a node would be excluded from + /// this project's own codegen with no output and no error; this is what keeps that rule from + /// failing silently. An overlay of the dependency's own node is untouched: its resolution key is in + /// . Returns the refusal, or null. Call it AFTER the loader's own errors, + /// never before: an unflagged overlay whose target the upstream removed fails first, with its own code. + /// + public MetaError? FindUnownedPackage(MetaRoot root) + { + if (ImportedPackages.Count == 0) return null; + + // ADR-0039 SANCTIONED own-accessor case: a root-level scan. `MetaRoot` has no super, so own and + // effective children are the same set here, and the question asked is precisely "what did this + // tree declare at the top level", which is the own layer by definition. + foreach (var node in root.Objects()) + { + var key = node.ResolutionKey(); + var pkg = DependencyResolver.PackageOf(key); + if (!ImportedPackages.Contains(pkg) || ImportedNodes.Contains(key)) continue; + return new MetaError( + $"\"{key}\" is declared here, but the package \"{pkg}\" belongs to a metadata dependency this " + + $"project imports, and \"{key}\" is not one of the nodes that dependency exports. Declare it in " + + "a package this project owns and 'extends' the dependency's node if it needs its shape; if it " + + "was meant to AMEND the dependency's node, give it that node's name and 'overlay: true'; if this " + + $"project really does own \"{pkg}\", name it in 'scope.include' and stop importing it.", + ErrorCode.ERR_DEPENDENCY_PACKAGE_NOT_OWNED); + } + return null; + } +} diff --git a/server/csharp/MetaObjects/Config/MetadataLocation.cs b/server/csharp/MetaObjects/Config/MetadataLocation.cs index 9af9afe86..0b28c0d00 100644 --- a/server/csharp/MetaObjects/Config/MetadataLocation.cs +++ b/server/csharp/MetaObjects/Config/MetadataLocation.cs @@ -34,20 +34,36 @@ namespace MetaObjects.Config; /// The generated-code directory the project declared (csharp.outDir in the same /// file), absolute; null when it declared none. An explicit --out takes precedence. /// +/// +/// FR-023 — the project's resolved metadata dependencies, read from the same file at EVERY rung of +/// the ladder (the explicit directory included). Their committed snapshots load FIRST, ahead of +/// 's own files; records what was imported on the loaded root. +/// Null for a caller that builds a location by hand and names no project. +/// public sealed record ResolvedMetadata( string Directory, IReadOnlyList? Files, IReadOnlyList Libraries, PhysicalNaming Naming, - WireFormat? Wire = null, string? OutDir = null) + WireFormat? Wire = null, string? OutDir = null, ImportScope? Imports = null) { /// Load this metadata, with the project's libraries, and set the loaded /// root's from (ADR-0058) and its /// from . public LoadResult Load(bool strict = false) { + var leading = Imports?.Sources(); var result = Files is { } files - ? MetaDataLoader.FromUris(files.Select(f => new Uri(f)).ToList(), Libraries, strict) - : MetaDataLoader.FromDirectory(Directory, Libraries, strict: strict); + ? MetaDataLoader.FromUris( + files.Select(f => new Uri(f)).ToList(), MetaDataLoader.DefaultRegistry(), Libraries, leading, strict) + : MetaDataLoader.FromDirectory( + Directory, MetaDataLoader.DefaultRegistry(), Libraries, leading, strict: strict); result.Root.PhysicalNaming = Naming; result.Root.WireFormat = Wire ?? WireFormat.Default; + if (Imports is null) return result; + + result.Root.ImportScope = Imports; + // The loader's own errors come first: an unflagged overlay whose target the upstream removed + // must fail with ITS code, not be masked by the ownership refusal it would also trip. + if (result.Errors.Count == 0 && Imports.FindUnownedPackage(result.Root) is { } refusal) + return result with { Errors = [refusal], State = "error" }; return result; } } @@ -85,9 +101,9 @@ public static string ProjectRootFor(string metadataDir) => /// /// Resolve where metadata lives. An explicit is used as /// given; otherwise 's .metaobjects/config.json - /// sources (or the default directory) decides. libraries is read from the - /// port-neutral config on BOTH paths — for an explicit directory, from the project that - /// holds it. Throws for a malformed config or an + /// sources (or the default directory) decides. libraries and + /// dependencies are read from the port-neutral config on BOTH paths — for an explicit + /// directory, from the project that holds it. Throws for a malformed config or an /// unresolvable source, and for a source shape /// this port's loader cannot take (several sources, or a file). /// @@ -98,7 +114,10 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd) var explicitCfg = ConfigFor(metadataDir); var explicitLibs = explicitCfg?.Libraries ?? Array.Empty(); var explicitNaming = explicitCfg?.Naming ?? PhysicalNaming.Default; - return new ResolvedMetadata(metadataDir, null, explicitLibs, explicitNaming, explicitCfg?.Wire, explicitCfg?.OutDir); + // No config found means no project to read a lock from: nothing is imported. + var explicitImports = explicitCfg is null ? ImportScope.None : ImportScope.Resolve(explicitCfg.ConfigDir, explicitCfg); + return new ResolvedMetadata( + metadataDir, null, explicitLibs, explicitNaming, explicitCfg?.Wire, explicitCfg?.OutDir, explicitImports); } var cfg = NeutralConfig.Read(cwd); @@ -106,13 +125,14 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd) var libraries = cfg?.Libraries ?? Array.Empty(); var naming = cfg?.Naming ?? PhysicalNaming.Default; var wire = cfg?.Wire; + var imports = ImportScope.Resolve(cwd, cfg); if (specs.Count == 0) { // No declared sources: resolve the DEFAULT directory through the same ladder // the shared conformance corpus gates (ERR_COLLECTION_NOT_FOUND when absent). var defaultFiles = SourceResolver.ResolveCollection(cwd); - return new ResolvedMetadata(Path.Combine(cwd, NeutralConfig.DefaultMetadataDir), defaultFiles, libraries, naming, wire, cfg?.OutDir); + return new ResolvedMetadata(Path.Combine(cwd, NeutralConfig.DefaultMetadataDir), defaultFiles, libraries, naming, wire, cfg?.OutDir, imports); } if (specs.Count > 1) @@ -142,6 +162,6 @@ public static ResolvedMetadata Resolve(string? metadataDir, string cwd) "\"sources\" at the file's containing directory."); } - return new ResolvedMetadata(resolved, files, libraries, naming, wire, cfg?.OutDir); + return new ResolvedMetadata(resolved, files, libraries, naming, wire, cfg?.OutDir, imports); } } diff --git a/server/csharp/MetaObjects/Config/NeutralConfig.cs b/server/csharp/MetaObjects/Config/NeutralConfig.cs index 893b8ba8f..e6e98ea1d 100644 --- a/server/csharp/MetaObjects/Config/NeutralConfig.cs +++ b/server/csharp/MetaObjects/Config/NeutralConfig.cs @@ -1,12 +1,18 @@ // Port-neutral `.metaobjects/config.json` reading. // -// Reads the NEUTRAL SUBSET (`schema_version`, `sources`, `libraries`). The file also -// carries TypeScript-owned keys (`pending_in_git`, `confidence_thresholds`, -// `extract`, `migrate`, `scope`); those are IGNORED rather than modeled, so a -// new TS-only key never becomes a four-port change. `scope` in particular is -// entirely out of scope for this reader — see +// Reads the NEUTRAL SUBSET (`schema_version`, `sources`, `libraries`, `dependencies`, and the two +// pattern lists FR-023 needs: `scope.include` and `migrate.scope`). The file also carries +// TypeScript-owned keys (`pending_in_git`, `confidence_thresholds`, `extract`, and the rest of +// `migrate` and `scope`); those are IGNORED rather than modeled, so a new TS-only key never becomes +// a four-port change. See // docs/superpowers/specs/2026-08-19-cross-port-metadata-sources-design.md §4. // +// `dependencies` (FR-023) is read at EVERY rung of the source ladder, including the explicit +// metadata directory a C# command is handed, because this port has no native config surface of its +// own: `.metaobjects/config.json` is where a dependency is declared. `scope.include` is read only +// for the explicit-include rule (a dependency's package is generated and migrated only when the +// project NAMES it); `scope.exclude` is not read, because this port has never scoped its own output. +// // `libraries` (FR-043) is read here — not just by TypeScript — because this port has // no rung-2 native config surface (unlike Java/Kotlin's pom or Python's // metaobjects.config.yaml): `.metaobjects/config.json` is the ONLY place a C# adopter @@ -53,6 +59,24 @@ public sealed class NeutralConfig /// `resolveCollection` (ERR_UNKNOWN_LIBRARY). public IReadOnlyList Libraries { get; } + /// + /// FR-023 — the declared metadata dependencies, each already validated; empty when the key is absent. + /// Names are unique. Resolving them to a committed snapshot is 's job. + /// + public IReadOnlyList Dependencies { get; } + + /// FR-023 — scope.include, read only for the explicit-include rule. Empty when absent. + public IReadOnlyList ScopeInclude { get; } + + /// FR-023 — migrate.scope, or null when the project declares none. + public IReadOnlyList? MigrateScope { get; } + + /// + /// The directory holding the .metaobjects/ folder this config was read from — where the lock and + /// the committed dependency snapshots live beside it. + /// + public string ConfigDir { get; } + /// ADR-0058 — the project's physical-naming rules (`naming` in config.json), each /// absent rule at its default. Never null: an absent `naming` key is the default, /// same as an empty `naming: {}` block. @@ -69,8 +93,14 @@ public sealed class NeutralConfig private NeutralConfig( IReadOnlyList> sources, IReadOnlyList libraries, - PhysicalNaming naming, WireFormat wire, string? outDir) + PhysicalNaming naming, WireFormat wire, string? outDir, + IReadOnlyList dependencies, IReadOnlyList scopeInclude, + IReadOnlyList? migrateScope, string configDir) { + Dependencies = dependencies; + ScopeInclude = scopeInclude; + MigrateScope = migrateScope; + ConfigDir = configDir; OutDir = outDir; Sources = sources; Libraries = libraries; @@ -193,6 +223,10 @@ private NeutralConfig( } } + var dependencies = ReadDependencies(root, path); + var scopeInclude = ReadPatterns(root, path, "scope", "include") ?? []; + var migrateScope = ReadPatterns(root, path, "migrate", "scope"); + var columns = ColumnNamingStrategy.SnakeCase; var tables = TableNamingRule.SnakeCase; if (root.TryGetProperty("naming", out var naming)) @@ -226,13 +260,45 @@ private NeutralConfig( } } - // Unknown top-level keys (including `scope`/`migrate`) are IGNORED by - // design — see the file header. + // Unknown top-level keys, and every key of `scope` / `migrate` but the two pattern + // lists read above, are IGNORED by design — see the file header. return new NeutralConfig(specs, libraries, new PhysicalNaming(columns, tables), ReadWire(root, path), - ReadOutDir(root, path, configDir)); + ReadOutDir(root, path, configDir), dependencies, scopeInclude, migrateScope, + Path.GetFullPath(configDir)); } } + /// The `dependencies` array (FR-023): present-but-wrong-shaped raises, as `sources` and `libraries` + /// do, and names must be unique because the name is the key of the lock and of the snapshot directory. + private static IReadOnlyList ReadDependencies(JsonElement root, string path) + { + if (!root.TryGetProperty("dependencies", out var deps)) return []; + if (deps.ValueKind != JsonValueKind.Array) + throw new MetaModelException($"{path}: \"dependencies\" must be an array", ErrorCode.ERR_BAD_ATTR_VALUE); + + var specs = deps.EnumerateArray().Select(d => DependencyResolver.ReadSpec(d, path)).ToList(); + if (specs.Select(s => s.Name).Distinct(StringComparer.Ordinal).Count() != specs.Count) + throw new MetaModelException($"{path}: \"dependencies\" names must be unique", ErrorCode.ERR_BAD_ATTR_VALUE); + return specs; + } + + /// One pattern list nested in a TypeScript-owned block: `scope.include` or `migrate.scope`. The + /// block must be an object when present and the list an array of non-empty strings; every OTHER key + /// of the block is TypeScript's and is not read. Null when the block or the list is absent. + private static IReadOnlyList? ReadPatterns(JsonElement root, string path, string block, string key) + { + if (!root.TryGetProperty(block, out var b) || b.ValueKind == JsonValueKind.Null) return null; + if (b.ValueKind != JsonValueKind.Object) + throw new MetaModelException($"{path}: \"{block}\" must be an object", ErrorCode.ERR_BAD_ATTR_VALUE); + if (!b.TryGetProperty(key, out var list) || list.ValueKind == JsonValueKind.Null) return null; + + if (list.ValueKind != JsonValueKind.Array || + list.EnumerateArray().Any(v => v.ValueKind != JsonValueKind.String || string.IsNullOrWhiteSpace(v.GetString()))) + throw new MetaModelException( + $"{path}: \"{block}.{key}\" must be an array of non-empty strings", ErrorCode.ERR_BAD_ATTR_VALUE); + return list.EnumerateArray().Select(v => v.GetString()!).ToList(); + } + /// The `csharp` block: the one C#-scoped setting, `outDir`, where `dotnet meta gen` writes /// and `dotnet meta verify` diffs. The other ports ignore the block, as they ignore any key /// they do not own. An unknown key, a non-string or empty `outDir`, or a non-object block is diff --git a/server/csharp/MetaObjects/Core/Requirement/RequirementCheck.cs b/server/csharp/MetaObjects/Core/Requirement/RequirementCheck.cs index a4850a376..812c81e60 100644 --- a/server/csharp/MetaObjects/Core/Requirement/RequirementCheck.cs +++ b/server/csharp/MetaObjects/Core/Requirement/RequirementCheck.cs @@ -245,6 +245,9 @@ private static List CoverableEntities(MetaData root) => // never inherited, so IsAbstract is read as the node's own flag. root.Children() .Where(n => n.Type == TYPE_OBJECT && n.SubType == OBJECT_SUBTYPE_ENTITY && !n.IsAbstract) + // FR-023: an entity a metadata dependency owns is not this project's to claim; a dependency's + // own coverage is its publisher's gate. MetaRoot.InScope is the rule. + .Where(n => (root as MetaRoot)?.ImportScope?.InScope(n.ResolutionKey()) ?? true) .ToList(); /// diff --git a/server/csharp/MetaObjects/Errors.cs b/server/csharp/MetaObjects/Errors.cs index c2072756f..d2498740b 100644 --- a/server/csharp/MetaObjects/Errors.cs +++ b/server/csharp/MetaObjects/Errors.cs @@ -168,6 +168,26 @@ public enum ErrorCode /// FR-043: a NEW top-level node is declared into a package a shipped library owns while that library is opted in — a later release of the library may ship a node of that name and merge into it. `overlay: true` on one of the library's OWN nodes is the documented amendment door and is untouched. /// ERR_LIBRARY_PACKAGE_NOT_OWNED, + + // FR-023 — metadata dependencies. The consume side (read `dependencies`, load the committed snapshot, + // refuse a stale or colliding one) raises the stale, collision, incompatible and not-owned codes in this + // port; `unresolved`, `manifest invalid` and `upstream drift` belong to the Node `meta deps sync` / `check` + // and are registered here so the shared corpus list stays one set in every port. + + /// FR-023: a declared dependency's transport could not locate a directory holding metaobjects.pkg.json. + ERR_DEPENDENCY_UNRESOLVED, + /// FR-023: a dependency's metaobjects.pkg.json fails its schema, names a different dependency, points at a missing or hash-mismatched artifact, or its artifact does not load standalone / does not declare exactly the listed packages and nodes. + ERR_DEPENDENCY_MANIFEST_INVALID, + /// FR-023: the committed snapshot does not match .metaobjects/deps.lock.json (lock missing, entry missing or extra, artifact missing, or hash mismatch) — run `meta deps sync`. + ERR_DEPENDENCY_SNAPSHOT_STALE, + /// FR-023: two dependencies export the same fully-qualified node. + ERR_DEPENDENCY_NODE_COLLISION, + /// FR-023: a local top-level node is declared into a package owned by a dependency. + ERR_DEPENDENCY_PACKAGE_NOT_OWNED, + /// FR-023: a dependency's metamodelVersion major differs from this toolchain's. + ERR_DEPENDENCY_METAMODEL_INCOMPATIBLE, + /// FR-023: `meta deps check` / `verify --deps` found the installed dependency differs from the committed snapshot. + ERR_DEPENDENCY_UPSTREAM_DRIFT, // FR5c — multi-file overlay merge produced a conflicting attribute value: // two contributors set the same @attr to different non-empty values. ERR_MERGE_CONFLICT, diff --git a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs index e0afab9eb..41ef2e75e 100644 --- a/server/csharp/MetaObjects/Loader/MetaDataLoader.cs +++ b/server/csharp/MetaObjects/Loader/MetaDataLoader.cs @@ -77,7 +77,7 @@ public MetaDataLoader(TypeRegistry registry, bool freeze = true, bool strict = f // Registry.cs::ApplyStrictAttrScoping can consult the id set without a // Registry → Loader back-reference (Loader → Registry/Provider/CoreTypes is the // sole pre-existing dependency direction). - private static TypeRegistry DefaultRegistry() => + internal static TypeRegistry DefaultRegistry() => Provider.ComposeRegistry(CoreTypes.LibraryProviders); // ------------------------------------------------------------------------- @@ -124,6 +124,17 @@ public static LoadResult FromDirectory(string directory, TypeRegistry registry, /// public static LoadResult FromDirectory(string directory, TypeRegistry registry, IEnumerable? libraries, DirectorySource.Options? opts = null, bool strict = false) + => FromDirectory(directory, registry, libraries, null, opts, strict); + + /// + /// + /// plus sources (FR-023: a dependency's committed snapshot artifacts) + /// loaded AFTER the libraries and BEFORE the directory's own files. Internal: a host reaches this + /// through , which also records what was imported. + /// + internal static LoadResult FromDirectory(string directory, TypeRegistry registry, + IEnumerable? libraries, IReadOnlyList? leading, + DirectorySource.Options? opts = null, bool strict = false) { var src = new DirectorySource(directory, opts); List sources; @@ -149,6 +160,7 @@ public static LoadResult FromDirectory(string directory, TypeRegistry registry, }; return new LoadResult(root, Array.Empty(), errors.AsReadOnly(), "error"); } + if (leading is { Count: > 0 }) sources.InsertRange(0, leading); var libSources = Library.LibrarySources.Resolve(libraries); if (libSources.Count > 0) sources.InsertRange(0, libSources); var result = new MetaDataLoader(registry, strict: strict).Load(sources); @@ -202,9 +214,19 @@ public static LoadResult FromUris(IReadOnlyList uris, TypeRegistry registry /// public static LoadResult FromUris( IReadOnlyList uris, TypeRegistry registry, IEnumerable? libraries, bool strict = false) + => FromUris(uris, registry, libraries, null, strict); + + /// + /// plus + /// sources — see the directory twin above. + /// + internal static LoadResult FromUris( + IReadOnlyList uris, TypeRegistry registry, IEnumerable? libraries, + IReadOnlyList? leading, bool strict = false) { var loader = new MetaDataLoader(registry, strict: strict); var sources = uris.Select(u => (IMetaDataSource)new UriSource(u)).ToList(); + if (leading is { Count: > 0 }) sources.InsertRange(0, leading); var libSources = Library.LibrarySources.Resolve(libraries); if (libSources.Count > 0) sources.InsertRange(0, libSources); return loader.Load(sources); diff --git a/server/csharp/MetaObjects/Meta/MetaRoot.cs b/server/csharp/MetaObjects/Meta/MetaRoot.cs index 35fd33728..cf7ada93a 100644 --- a/server/csharp/MetaObjects/Meta/MetaRoot.cs +++ b/server/csharp/MetaObjects/Meta/MetaRoot.cs @@ -27,6 +27,21 @@ public class MetaRoot(TypeId typeId, string name) : MetaData(typeId, name) /// public WireFormat WireFormat { get; set; } = WireFormat.Default; + /// + /// FR-023 — what the project imports from metadata dependencies, set by the same project-aware load + /// as . Null for a tree loaded with no project (nothing is imported). + /// Imported objects load so the project's own model resolves against them, but + /// drops them from every action surface unless the project names their package. + /// + public MetaObjects.Config.ImportScope? ImportScope { get; set; } + + /// + /// Does belong to this project's own action surface (codegen, the requirement + /// ledger's denominator)? False for an object a dependency owns, unless the project's + /// scope.include names its package. Always true when nothing is imported. + /// + public bool InScope(MetaObject obj) => ImportScope?.InScope(obj.ResolutionKey()) ?? true; + /// Object entities defined at this root level. public IReadOnlyList Objects() { diff --git a/spec/roadmap.md b/spec/roadmap.md index ac5e78e0c..65c98d5a3 100644 --- a/spec/roadmap.md +++ b/spec/roadmap.md @@ -142,7 +142,7 @@ under **Shipped**; planned FRs under **Planned** + the **Release plan**. ✅ shi | FR-020 | `@inheritance` joined strategy | 📋 | 1.x | [#6](https://github.com/metaobjectsdev/metaobjects/issues/6) | | FR-021 | api metadata + contract projections | 📋 | 1.3 | [#7](https://github.com/metaobjectsdev/metaobjects/issues/7) | | FR-022 | Contract emitters (JSON Schema/OpenAPI/protobuf) | 📋 | 1.3 | [#8](https://github.com/metaobjectsdev/metaobjects/issues/8) | -| FR-023 | Metadata dependencies (cross-repo sharing: generated shared-model artifact, `meta deps sync`, committed snapshot + lock, default exclusion of imported metadata, consumer extend/overlay across the boundary) | 🟢 Phase 1a (TS + Python) **shipped** per DESIGN §11, Java/Kotlin consume (Phase 2 S3) — the usage-aware breaking-change classifier and the `npm`/`python` transports are deferred (§11.4); C# consumer is Phase 2 | 1.1 | [#9](https://github.com/metaobjectsdev/metaobjects/issues/9) | +| FR-023 | Metadata dependencies (cross-repo sharing: generated shared-model artifact, `meta deps sync`, committed snapshot + lock, default exclusion of imported metadata, consumer extend/overlay across the boundary) | 🟢 Phase 1a (TS + Python) **shipped** per DESIGN §11, Java/Kotlin consume (Phase 2 S3) — the usage-aware breaking-change classifier and the `npm`/`python` transports are deferred (§11.4); C# consumer shipped in Phase 2 S6 | 1.1 | [#9](https://github.com/metaobjectsdev/metaobjects/issues/9) | | FR-024 | Entity surfaces (projection/value + declared API) | 🟢 taxonomy + B3/B4a/B4b/B5/B6 validation parity + projection codegen **shipped 5 ports (1.0)**; declared-API surface (`api.operational`/`operation`/`binding.rest`) **deferred to a later 1.x** (not in 1.1.0; foundation for FR-034) | 1.0 · 1.x (later) | [#10](https://github.com/metaobjectsdev/metaobjects/issues/10) | | FR-025 | Cross-port package-binding codegen config | 📋 | 1.1 | [#11](https://github.com/metaobjectsdev/metaobjects/issues/11) | | FR-026 | Forms completeness (edit forms + view parity) | 🟢 edit forms + view-kind control dispatch **shipped**; renderer parity / validator wiring / `layout.form` **deferred to a later 1.x** (not in 1.1.0) | 1.x (later) | [#12](https://github.com/metaobjectsdev/metaobjects/issues/12) | @@ -415,7 +415,7 @@ _(FR-032 was developed under the working number "FR-026" — see commit history; snapshot + `.metaobjects/deps.lock.json`, and TypeScript's and Python's collection resolvers — the two ports that consume dependencies in Phase 1a — load the snapshot at every rung of their own source ladder (Java and Kotlin followed in - Phase 2 slice S3; C# doesn't read `dependencies` yet). Imported metadata is load-only by default — + Phase 2 slice S3; C# in slice S6). Imported metadata is load-only by default — excluded from codegen, schema, and the requirements ledger unless the consumer's own `scope.include` / `migrate.scope` names the package — with a loud refusal (`ERR_DEPENDENCY_PACKAGE_NOT_OWNED`) rather than a silent drop when a consumer