diff --git a/CHANGELOG.md b/CHANGELOG.md index 2ddc4a16b..d7850af55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -44,6 +44,30 @@ here.** Phase 2 ([#482](https://github.com/metaobjectsdev/metaobjects/issues/482)) are written against. The feature doc records the Maven and NuGet packaged transports as a design for a later slice. +- **`meta deps sync` installs a shared model from an npm or Python package, not only from a + folder path.** A dependency can now be `{ "name": "x", "npm": "@acme/model", "dir": + "metaobjects" }` or `{ "name": "x", "python": "acme_model", "dir": "metaobjects" }`: install + the publisher's package with your own package manager, then `meta deps sync` finds it (the + way Node resolves `@acme/model` from the directory holding `.metaobjects/`, or where + `importlib.util.find_spec` puts `acme_model` in your interpreter), applies `dir`, and + snapshots and locks it exactly as a `path` dependency. `meta deps check` and `verify --deps` + resolve the same way. The Python interpreter is the first of `$METAOBJECTS_PYTHON`, the + active virtualenv, a `.venv` beside `.metaobjects/`, and `python3`, and `sync` prints the + one it used. Before this both transports were accepted in config and refused by `sync` with + "not supported by this toolchain yet". Now a package that is not installed, a missing + interpreter, or a `dir` the package does not ship is `ERR_DEPENDENCY_UNRESOLVED` naming the + `npm install` / `pip install` fix, and a `dir` holding no manifest is + `ERR_DEPENDENCY_MANIFEST_INVALID`. `sync` warns when the installed package's version differs + from its manifest's (a release without regenerating the shared model). The lock format is + unchanged and records no host path or interpreter. Python already loaded such a dependency + from its committed snapshot; two new `dependency-conformance` cases prove both ports do, and + a new `transport-cases.json` arm (14 cases, Node only) pins the transports. On the publisher + side `sharedModelFile()`'s default version now also reads a `pyproject.toml` `[project]` + version, so a Python package stamps the version consumers install. + [docs/features/metadata-dependencies.md](docs/features/metadata-dependencies.md) gains the + two-step publish path for an npm and a Python package and a private-registry recipe (Gitea + and similar). `maven` / `nuget` stay Phase 2. + - **C# reads its generated-code directory from `.metaobjects/config.json`, so a plain `dotnet meta verify` runs the codegen gate.** Set `"csharp": { "outDir": "Generated" }` (relative to the directory holding `.metaobjects/`). `dotnet meta gen` writes there and diff --git a/README.md b/README.md index e6258ac52..031f64467 100644 --- a/README.md +++ b/README.md @@ -171,7 +171,7 @@ first-week wedge plan — and `meta init` picks up from there. | Capability requirements (`requirement.*`) | `meta verify` gate + authoring lint; `requirementTests()` stubs | `mvn metaobjects:verify` gate; JUnit Jupiter `requirement-tests` | Gate via the Java Maven goal; JUnit Jupiter `requirement-tests` | `dotnet meta verify` gate; xUnit `requirement-tests` | `metaobjects verify` gate; pytest `requirement-tests` | | Reporting vocabulary (`dimension` / `measure` / `segment` / `object.report`, FR-044) | Registered; a view-backed report becomes a SQL view in `meta migrate`, is read by `ObjectManager`, and gets generated read-only list routes and a TanStack list hook (no grid or form); the `cube-model` generator writes the vocabulary as Cube data-model files | Registered; OMDB reads a view-backed report; generates a read-only Spring controller, DTO, repository and filter allowlist | Registered (via Java); generates an Exposed table object, a row data class, a filter allowlist and a read-only Spring controller per view-backed report | Registered; generates a keyless EF Core row type, read-only routes and a filter allowlist per view-backed report | Registered; `ObjectManager` reads a view-backed report; generates a Pydantic row model, a read-only FastAPI router and a filter allowlist | | Libraries (`libraries: [...]`) | Yes | Yes | Yes (via Java) | Yes | Yes | -| Metadata dependencies (`dependencies`) | Yes (`meta deps sync`, `path` transport) | Phase 2 | Phase 2 | Phase 2 | Yes (loads the synced snapshot) | +| Metadata dependencies (`dependencies`) | Yes (`meta deps sync`: `path`, `npm` and `python` transports) | Phase 2 | Phase 2 | Phase 2 | Yes (loads the synced snapshot) | | Runtime metadata (ObjectManager-style) | Yes (`runtime-ts`) | Yes (OMDB) | Yes (via Java OMDB + Exposed) | Roadmap | Yes (ObjectManager) | | React / Angular UI client (browser) | React: **published** (`@metaobjectsdev/react` + `@metaobjectsdev/tanstack`), codegen + runtime. Angular 18: **source-only by decision** ([ADR-0048](spec/decisions/ADR-0048-angular-tier-source-only.md)) — `@metaobjectsdev/angular` + `@metaobjectsdev/codegen-ts-angular` build in-repo on their own `0.6.x` line but are deliberately not on npm (`npm i @metaobjectsdev/angular` will 404) until they meet the ADR's promotion bar. Consume them from source. | Consumes TS client via REST | Consumes TS client via REST | Consumes TS client via REST | Consumes TS client via REST | | Own a generator (`eject`) | Yes (`meta eject`) | Yes (`mvn metaobjects:eject`) | Yes (`mvn metaobjects:eject`) | Yes (`dotnet meta eject`) | Yes (`metaobjects eject`) | @@ -261,8 +261,8 @@ sixth ships two libraries at their own stability labels: `meta eject `. See [`docs/features/libraries.md`](docs/features/libraries.md). **Sharing a model across your own projects** *(the TypeScript toolchain publishes; -TypeScript, Python, Java and Kotlin projects consume, over a `path` dependency such as a sibling -checkout or submodule; C# arrives in Phase 2)* — one project publishes +TypeScript, Python, Java and Kotlin projects consume, from an installed npm or Python +package or over a `path` dependency such as a sibling checkout; C# arrives in Phase 2)* — one project publishes part of its model (a common `Customer`, an audited base, a set of enums) with `sharedModelFile()`, and another declares it in `dependencies`, syncs it into a committed, hash-locked snapshot and builds on it with `extends` and references. When the diff --git a/agent-context/skills/metaobjects-authoring/references/metadata-dependencies.md b/agent-context/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/agent-context/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/agent-context/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/agent-context/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 76313a801..60db4be1d 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -46,7 +46,7 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/provider-composition-conformance/`](../fixtures/provider-composition-conformance/) | 9 (5 error-shape + 4 compose-load) | ✓ | ✓ | — (JVM registry via Java) | ✓ | ✓ | | [`fixtures/source-resolution-conformance/`](../fixtures/source-resolution-conformance/) | 25 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/scope-conformance/`](../fixtures/scope-conformance/) | 10 cases | ✓ (reference implementation) | ✓ | inherits via Java | ✓ | ✓ | -| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 35 cases (24 resolution + 11 check) | ✓ (reference implementation; both arms) | ✓ (resolution arm, all 24; no check, so the 11 check cases are not run) | inherits via Java | — (Phase 2) | ✓ (resolution arm, all 24 identically; no check yet, so the 11 check cases are not run) | +| [`fixtures/dependency-conformance/`](../fixtures/dependency-conformance/) | 37 cases (26 resolution + 11 check) + 14 transport cases | ✓ (reference implementation; all three arms) | ✓ (resolution arm, all 26; no check, so the 11 check cases are not run) | inherits via Java | — (Phase 2) | ✓ (resolution arm, all 26 identically; no check yet, so the 11 check cases are not run; the transport arm is Node-only) | | [`fixtures/shared-model-conformance/`](../fixtures/shared-model-conformance/) | 11 cases (7 publish + 4 refusal) | ✓ (reference implementation) | — (Phase 2) | — (Phase 2) | — (Phase 2) | ✓ (serializer arm: the 7 publish cases' document bytes; no publisher) | | [`fixtures/agent-context-conformance/`](../fixtures/agent-context-conformance/) | 4 | ✓ (the emitter is TS-owned) | — | — | — | — | | [`fixtures/metamodel-docs/`](../fixtures/metamodel-docs/) | 1 | ✓ (docs emit is TS-owned) | — | — | — | — | @@ -440,17 +440,17 @@ also matches before a trailing newline. Kotlin has no runner of its own: it inhe the Java loader. Each of those two guards is pinned by a test beside the corpus, since the corpus asserts matches only and carries no invalid-pattern case. -### `fixtures/dependency-conformance/` (35 cases) +### `fixtures/dependency-conformance/` (37 cases + 14 transport cases) -All 35 cases → [features/metadata-dependencies.md](features/metadata-dependencies.md) +All 37 cases → [features/metadata-dependencies.md](features/metadata-dependencies.md) (declaring a dependency, `meta deps sync`, the committed snapshot + lock, default exclusion of imported metadata, overlay/extends across the boundary, every load- and resolution-time failure, and the upstream check). File-shaped like `scope-conformance/` -above: one committed `cases.json`, no per-port fixture, no ledger. The corpus has two arms: +above: one committed `cases.json`, no per-port fixture, no ledger. The corpus has three arms: -- **Resolution (24 cases).** Every case's `expectFiles`/`expectImported`/`expectSelected`/ +- **Resolution (26 cases).** Every case's `expectFiles`/`expectImported`/`expectSelected`/ `expectMigrateGoverned`/`expectLoadError` assertions. **TypeScript (the reference - implementation), Python and Java all run all 24, identically**, with no exemption. The two + implementation), Python and Java all run all 26, identically**, with no exemption. The two overlay cases that need a view child as incidental content use `view.currency` (the one concrete `view.*` subtype registered cross-port), not `view.text` — a prior revision used `view.text`, which Python did not register then, and carried a since-discharged allowlist @@ -460,6 +460,12 @@ above: one committed `cases.json`, no per-port fixture, no ledger. The corpus ha by TypeScript only, in `cli/test/dependency-check-conformance.test.ts`; Python and Java have no `deps check` yet and filter these cases out. The arm exists so that each port's check is written against the same edge cases (Phase 2, #482). +- **Transport (14 cases, `transport-cases.json`).** The `npm` and `python` transports `meta deps sync` + uses to find an installed package: npm packages under a temp `node_modules`, Python packages on a + temp import path, the directory each resolves to, the exact lock written, and each refusal. Run by + Node only, in `cli/test/dependency-transport-conformance.test.ts`, because Node `meta deps sync` is + the only writer of snapshots and the lock. `cases.json` also carries an `npm` and a `python` + dependency loading from their committed snapshot with nothing installed, which both ports run. C# has no runner yet; it arrives in a later Phase 2 slice. @@ -586,7 +592,7 @@ files into a real Cube. The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37 + render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 + -dependency 35) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the +dependency 37) each map to a feature doc, and so do the two requirement corpora, `shared-model-conformance` and the `cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider composition, agent context, docs emit) rather than user-facing metamodel behaviour, diff --git a/docs/features/cli.md b/docs/features/cli.md index aded5baa1..492c05a3e 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -33,7 +33,7 @@ command surface splits in two: | **Template/prompt drift** (`verify --templates`) | **Node `meta`** | `meta verify --templates` | TS reference (ADR-0021 D2) — `{{field}}`↔payload; part of the bare-`verify` default | | **Vocabulary upgrade** (`upgrade`) | **Node `meta`** | `meta upgrade [--to ] [--apply]` | **any backend** — rewrites RETIRED metadata vocabulary (`@violation` → `@counterexample`, `@readOnly` → `@mutability`, dropping `@verifiedBy`) and resolves ATTRIBUTE CONTRADICTIONS (`@fields` beside `@expr` on an index key). Node-only because it edits the metadata documents themselves, which every port shares; a non-TS project runs `npx meta upgrade` against its own `metaobjects/`. **Canonical JSON and YAML alike.** Previews by default. Retirements needing a human decision are refused and the run exits non-zero, so CI cannot record a partial migration as finished | | **Vocabulary search** (`types`) | **Node `meta`** | `meta types [query]` | **any backend** — apropos/`kubectl explain` over the live metamodel registry (names + descriptions + when-to-use); the vocabulary is cross-port identical (registry-conformance) | -| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency's `path` transport into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript, Python and Java then LOAD that snapshot at every rung of their own source ladder (Kotlin loads through Java's loader and Maven plugin); C# doesn't read `dependencies` yet (Phase 2). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) | +| **Dependency sync** (`deps`) | **Node `meta`** | `meta deps sync \| check \| list` | **any backend** — resolves a declared metadata dependency (`path`, or an installed `npm` / `python` package) into a committed snapshot + lock (FR-023), so `sync`/`check`/`list` themselves stay Node-only regardless of your server language. TypeScript, Python and Java then LOAD that snapshot at every rung of their own source ladder (Kotlin loads through Java's loader and Maven plugin); C# doesn't read `dependencies` yet (Phase 2). On a project whose own metadata Node `meta` cannot read (a Maven `src/main/resources/metadata` directory, a C# metadata directory, or another port's vocabulary), `sync` and `check` prove the locked dependencies load together and say that the port's own `verify` is the full check. See [`metadata-dependencies.md`](metadata-dependencies.md) | | **Canonical format** (`fmt`) | every port | `meta fmt` / `dotnet meta fmt` / `mvn metaobjects:fmt` / `metaobjects fmt` | (#304) ships in **every** port — the canonical serializer already exists in all five, so formatting is local, not Node-only. `--check` lists non-canonical files and exits non-zero without changing anything. Formats each file STANDALONE — own-mode, declared-here layer only (ADR-0039); an `extends` onto another file's base is preserved as-is, never resolved or erred on; a file declaring `overlay: true` anywhere — whether or not a same-file base exists — is reported as skipped rather than merged (merging would drop the overlay marker, a structural change fmt never makes). A YAML file is always skipped — no canonical YAML emitter exists (ADR-0006). Before writing, every port reloads the whole project with the candidate swapped in and refuses the write unless the reloaded model's canonical form is byte-identical to the untouched baseline — fmt never changes meaning. See [Fixture corpus](#meta-fmt--canonical-formatting-304) below and `fixtures/fmt-conformance/README.md` | | TS codegen | Node `meta` | `meta gen` | TS projects. **No `--template-spec` flag, deliberately** — `metaobjects.config.ts` already takes generator VALUES, so a declarative template generator is declared there (`templateGenerator()`, or `templateSpecToGenerators(parseTemplateSpec(spec))` to reuse a C#/Python spec file). Keeping it in the config is what lets `meta verify --codegen` regenerate WITH it, since that gate re-runs the config's generator list; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes) | | C# codegen | `dotnet meta` | `dotnet meta init` / `gen` / `verify --templates` / `verify --codegen` | a .NET tool (`ToolCommandName=dotnet-meta`); invoked `dotnet meta` so it never shadows the Node `meta`; ships the ADR-0021 D2 subverbs (`--db` rejected, exit 2; bare `verify` = `--templates` plus `--codegen` when an output directory (`csharp.outDir` in `.metaobjects/config.json`, or `--out`) and `codegen/Generators.cs` are present). **The generator list is `codegen/Generators.cs`** (ADR-0034 Amendment 9): `dotnet meta init` writes it with `codegen/Program.cs` (two lines that run it) and `codegen/Codegen.csproj`, and `gen` / `verify --codegen` always run it through `dotnet run --project codegen`. With no `codegen/Generators.cs` they exit 2 and write nothing; `--generators` only narrows the list to declared entries. Every entry may set its own output directory (`ApiDocsGenerator { OutDir = ... }`), and `verify --codegen` compares each one. A declarative Mustache template-spec (the cross-port JSON contract shared with Python) is one more entry, `TemplateSpecGenerator { Spec = "", TemplateRoot = "" }`; `gen` no longer reads `--template-spec` or a project-root `template-spec.json`, and a run that finds either with no such entry fails before writing anything; see [declarative template scopes](codegen-concepts.md#declarative-template-scopes). `dotnet meta docs` was removed in 1.1.0: it prints the line that points at [the migration section](migrations/docs-are-generators.md#c) and exits 2 | @@ -55,7 +55,7 @@ vocabulary everywhere, each port implementing the modes it supports. | `verify --codegen` | **Codegen drift** — regenerate from metadata into a temp dir and fail if it differs from the committed generated output. Catches "metadata changed but `meta gen` wasn't re-run". A hand-edited generated file is NOT drift — `meta gen` three-way-merges hand edits by design, so the gate compares the *generated contribution* against the committed `.gen-state/.hashes.json` rather than the file byte-for-byte. It still LISTS every generated file carrying a hand edit (count + paths, and how to see the edit) as a notice that does not change the exit code; `--forbid-hand-edits` makes any hand edit fail the build, for teams that want generated code untouchable (Node `meta` only). | no | | `verify --templates` | **Template/prompt drift** — `Renderer.verify` checks each `template.*` node's `{{field}}` references against its payload VO (FR-004). | no | | `verify --docs` | **Removed in 1.1.0** (ADR-0034 Amendment 9). It prints where the check went and exits 2: documentation is written by entries in `generators`, so `verify --codegen` checks it with the rest of the declared output. | — | -| `verify --deps` | **Dependency drift** (FR-023) — re-resolves each declared dependency's `path` and compares its installed artifact's hash against `.metaobjects/deps.lock.json`, same comparison as `meta deps check`. Needs the publisher's `path` reachable, so it is never part of the bare-`verify` default. A `drifted` or `unresolved` dependency fails with `ERR_DEPENDENCY_UPSTREAM_DRIFT`. **Node `meta` only.** See [`metadata-dependencies.md`](metadata-dependencies.md). | no | +| `verify --deps` | **Dependency drift** (FR-023) — re-resolves each declared dependency (its `path`, or its installed `npm` / `python` package) and compares its installed artifact's hash against `.metaobjects/deps.lock.json`, same comparison as `meta deps check`. Needs the publisher reachable, so it is never part of the bare-`verify` default. A `drifted` or `unresolved` dependency fails with `ERR_DEPENDENCY_UPSTREAM_DRIFT`. **Node `meta` only.** See [`metadata-dependencies.md`](metadata-dependencies.md). | no | Rules of the contract: diff --git a/docs/features/metadata-dependencies.md b/docs/features/metadata-dependencies.md index 2589b196b..64821f3ba 100644 --- a/docs/features/metadata-dependencies.md +++ b/docs/features/metadata-dependencies.md @@ -8,7 +8,7 @@ alternative: declare a published metadata package as a dependency, `meta deps sy it into a committed snapshot, and build on it the same way you build on any other metadata in your own tree — `extends`, `overlay: true`, and plain FQN references. -**Phase 1a scope.** This page describes what ships today: the `path` transport, the +**What ships today:** the `path`, `npm` and `python` transports, the generated shared-model artifact, the lock + snapshot, default exclusion from your own codegen/schema/ledger, and the loader errors that fire when upstream moves. See [Deferred](#deferred) for what is designed but not built yet — read that section @@ -45,7 +45,9 @@ going away. "schema_version": 1, "sources": [], "dependencies": [ - { "name": "acme-common", "path": "../acme-common/metaobjects" } + { "name": "acme-common", "path": "../acme-common/metaobjects" }, + { "name": "acme-billing", "npm": "@acme/billing-model", "dir": "metaobjects" }, + { "name": "acme-scheduling", "python": "acme_scheduling", "dir": "metaobjects" } ] } ``` @@ -54,17 +56,77 @@ A dependency spec carries a `name` (`/^[a-z0-9][a-z0-9._-]*$/`, unique across th array — this is the LOCAL alias your config, lock, and `.metaobjects/deps//` snapshot directory key on) and exactly one transport key. -**Only `path` resolves in Phase 1a.** It is resolved relative to the directory -holding `.metaobjects/` and must point at a directory containing a publisher's -`metaobjects.pkg.json` + artifact (below) — a sibling checkout, a git submodule, or -any other directory your own build can reach on disk. `npm` and `python` transport -keys are accepted by the config schema (reserved, the same precedent as `sources`' -`resource`/`package` kinds) but `meta deps sync` refuses them today: +Three transports locate the publisher's `metaobjects.pkg.json` + artifact (below). +Each answers one question, *which directory holds the manifest?*, and only `meta deps +sync` and `meta deps check` ask it. Every other command loads the committed snapshot. + +| Transport | Spec | Where it looks | +|---|---|---| +| `path` | `{ "name": "x", "path": "../lib/metaobjects" }` | that directory, relative to the directory holding `.metaobjects/`: a sibling checkout, a git submodule, or any directory your build can reach | +| `npm` | `{ "name": "x", "npm": "@acme/model", "dir": "metaobjects" }` | the installed npm package, found from the directory holding `.metaobjects/` the way Node resolves it (`node_modules` up the tree, workspaces and hoisting included), then `dir` inside it | +| `python` | `{ "name": "x", "python": "acme_model", "dir": "metaobjects" }` | the installed Python package, as `importlib.util.find_spec("acme_model")` reports it in your project's interpreter, then `dir` inside it | + +`dir` is optional (default: the package root), must stay inside the package, and is +legal only beside `npm` or `python`. `python` takes an importable package name (dotted +names such as `acme.model` work), not the distribution name you `pip install`. The `npm` +transport reads a `node_modules` tree; a Yarn Plug'n'Play install has none, so set +`nodeLinker: node-modules` for that project. + +**Install the package first, with your own package manager.** `meta deps sync` does no +downloading. `npm install @acme/model` (or pnpm, yarn, bun) or `pip install acme-model` +(or uv, poetry) puts the package where the transport finds it, and the ecosystem's own +lock (`package-lock.json`, `uv.lock`, …) pins which version is installed. The +dependency lock records what that version's shared model was when you last synced. +Because the snapshot is committed, only the machine that runs `sync` needs the package +installed. CI, `meta gen`, `meta migrate` and every other port load the snapshot. + +**Which Python interpreter.** `python` runs one interpreter, the first of: +`$METAOBJECTS_PYTHON`; the active virtualenv (`$VIRTUAL_ENV`); a `.venv` beside +`.metaobjects/`; `python3` on `PATH` (`python` on Windows). A set `$METAOBJECTS_PYTHON` or `$VIRTUAL_ENV` +whose interpreter is missing is refused rather than skipped, because falling through +would read a different environment. `sync` prints the interpreter it used: ``` -dependency "acme-common": transport `npm` is not supported by this toolchain yet; use `path` +$ meta deps sync +synced acme-scheduling (new)→0.4.0 ((new)→1c9e44d0) — python: /srv/app/.venv/bin/python (from .venv) ``` +The interpreter path never reaches the lock. `find_spec` imports the parent packages of +a dotted name, so this runs your own environment's code, as `import` would. + +**When a transport cannot find the package**, `sync` and `check` fail with +`ERR_DEPENDENCY_UNRESOLVED`, naming what was tried and the fix: + +``` +dependency "acme-billing": the npm package "@acme/billing-model" is not installed where +/srv/app can resolve it. Install it (`npm install @acme/billing-model`, or your package +manager's equivalent), then re-run. + +dependency "acme-scheduling": the Python package "acme_scheduling" is not installed in +/usr/bin/python3 (from PATH). Install the distribution that provides it into that +environment (for example `pip install acme-scheduling`), or point $METAOBJECTS_PYTHON at +the interpreter that has it. + +dependency "acme-scheduling": no Python interpreter at /opt/py/bin/python (from +$METAOBJECTS_PYTHON). Install Python, activate the project's virtualenv, or point +$METAOBJECTS_PYTHON at the interpreter that has "acme_scheduling" installed. +``` + +If importing a dotted name's parent package raises, `sync` reports that error and asks +you to fix the environment; the package is not missing. + +The package is installed but `dir` names a directory it does not ship: also +`ERR_DEPENDENCY_UNRESOLVED`, telling you to check `dir` or ask the publisher to ship the +output. The directory exists but holds no `metaobjects.pkg.json`: +`ERR_DEPENDENCY_MANIFEST_INVALID`, with the same fix. + +**The installed version should match the manifest.** The publisher's generator stamps +the host package's version into the manifest. When the installed package's own version +(`package.json`, or the Python distribution's metadata) differs, `sync` warns (Python +versions compare in PEP 440 normalized form, so `1.0.0-rc.1` equals `1.0.0rc1`): the +publisher most likely released without regenerating the shared model. The lock records +the manifest's version. + `dependencies` is read at **every rung** of [the source ladder](metadata-sources.md) — wherever your own sources come from (the default `metaobjects/` directory, a declared `sources` list, or a port's native surface), the neutral config's `dependencies` array @@ -72,8 +134,8 @@ is read the same way on top of it. ## `meta deps sync`, `check`, and `list` -**`meta deps sync […] [--dry-run]`** resolves each declared dependency's `path`, -reads its `metaobjects.pkg.json`, verifies the sibling artifact's bytes hash to what +**`meta deps sync […] [--dry-run]`** resolves each declared dependency through its +transport, reads its `metaobjects.pkg.json`, verifies the sibling artifact's bytes hash to what the manifest records, and re-loads the artifact standalone with **core providers only** (a consumer loads it with its own providers, never the publisher's, so a Phase 1a export needing publisher-only vocabulary must fail at *publish* time, not @@ -97,7 +159,7 @@ two steps do, and compares the artifact's *currently installed* hash against wha lock recorded. Once the hashes match, it also loads the committed snapshot and exits 1 with the load error if the model does not load: matching hashes do not prove the model loads. It never writes the lock or the snapshot. It needs the -publisher's `path` reachable right now, which is why it is a separate command from +publisher reachable right now (its `path`, or the package installed), which is why it is a separate command from loading (below): CI running against a committed snapshot does not need the publisher checked out, but a `check` run does. @@ -314,15 +376,17 @@ classifies those as breaking automatically — see [Deferred](#deferred). ## What fails at `meta deps check` / `verify --deps` `verify --deps` runs `meta deps check`'s comparison as one of `verify`'s gated -subverbs (never part of the bare default — it needs the publisher's `path` -reachable, which CI checking out only your own repo may not have). Each declared +subverbs (never part of the bare default — it needs the publisher reachable, its +`path` or its installed package, which CI checking out only your own repo may not +have). Each declared dependency reports one of: - **`current`** — the installed artifact's hash matches the lock. - **`drifted`** — the publisher's artifact has changed since you last synced. Fix: review the artifact diff, then `meta deps sync`. - **`unresolved`** — the dependency could not be resolved at all right now (the - `path` moved, or the manifest/artifact fails validation). + `path` moved, the package is not installed, or the manifest/artifact fails + validation). Any `drifted` or `unresolved` dependency fails the command with `ERR_DEPENDENCY_UPSTREAM_DRIFT` — a check that cannot check must not pass. This is @@ -392,6 +456,119 @@ publishes the same bytes as the packaged generator, and a test has a consumer sy verify what an ejected copy published. See [own-your-codegen](own-your-codegen.md#shared-model-you-own-the-inputs-the-package-owns-the-artifact). +## Shipping the shared model inside your package + +A published model travels inside the package that implements it, the way a JVM +library's jar once carried its `metadata/*.json`. Consumers install your package with +their package manager and declare an `npm` or `python` dependency on it. Publishing is +two steps: generate the shared model into the package, then release the package as you +already do. + +**An npm package.** + +1. Point `sharedModelFile()`'s target inside the package, and list that directory in + `files` so it is packed: + + ```ts + // metaobjects.config.ts + sharedModelFile({ name: "acme-billing", include: ["acme::billing::**"], target: "shared" }), + // … + targets: { shared: { outDir: "metaobjects" } }, + ``` + + ```jsonc + // package.json + { "name": "@acme/billing-model", "version": "2.1.0", "files": ["dist", "metaobjects"] } + ``` + + `meta gen` writes `metaobjects/acme-billing.metaobjects.json` and + `metaobjects/metaobjects.pkg.json`, stamped with `package.json`'s version. +2. Bump the version, run `meta gen`, then `npm publish`. `npm pack --dry-run` lists + the two files if `files` is right. + +Consumers declare `{ "name": "acme-billing", "npm": "@acme/billing-model", "dir": "metaobjects" }`. + +**A Python package.** The generator is TypeScript, so a Python project runs it with the +Node `meta` it already uses for schema work. + +1. Point the target inside the importable package, and make sure the build ships the + JSON files as package data: + + ```ts + targets: { shared: { outDir: "src/acme_scheduling/metaobjects" } }, + ``` + + ```toml + # pyproject.toml + [project] + name = "acme-scheduling" + version = "0.4.0" + + # hatchling ships every file inside the package directory by default. With + # setuptools, declare the data: + [tool.setuptools.package-data] + acme_scheduling = ["metaobjects/*.json"] + ``` + + With no `version` option and no `package.json` version nearer the project root, the + generator stamps `pyproject.toml`'s static `[project]` version. If the Node project + has its own `package.json` version, or the Python package lives elsewhere in the + repository, pass `version` explicitly so the manifest carries the version consumers + install. +2. Bump the version, run `meta gen`, then build and upload as usual + (`python -m build`, then `twine upload` or `uv publish`). `unzip -l dist/*.whl` + shows the two files if the package data is right. + +Consumers declare `{ "name": "acme-scheduling", "python": "acme_scheduling", "dir": "metaobjects" }`. + +Run `meta verify --codegen` in your release pipeline before publishing: it fails when +the committed shared model is stale against your metadata. + +### Recipe: a private registry (Gitea, Forgejo, …) + +A self-hosted Gitea serves an npm and a PyPI registry per owner (user or +organization). The URLs have this shape; substitute your host and owner: + +| Ecosystem | Registry URL | +|---|---| +| npm | `https://git.example.com/api/packages//npm/` | +| PyPI (upload) | `https://git.example.com/api/packages//pypi` | +| PyPI (install index) | `https://git.example.com/api/packages//pypi/simple/` | + +**Publishing needs a token; installing does not** when the packages are public (or the +instance allows anonymous reads). Create a personal access token with package write +scope, and keep it out of the repository: + +```ini +# publisher's .npmrc (the token comes from the environment) +@acme:registry=https://git.example.com/api/packages//npm/ +//git.example.com/api/packages//npm/:_authToken=${GITEA_TOKEN} +``` + +```bash +npm publish # with the .npmrc above +python -m build +twine upload --repository-url https://git.example.com/api/packages//pypi \ + -u -p "$GITEA_TOKEN" dist/* +``` + +Consumers point their package manager at the registry with no credentials: + +```ini +# consumer's .npmrc +@acme:registry=https://git.example.com/api/packages//npm/ +``` + +```bash +pip install --index-url https://git.example.com/api/packages//pypi/simple/ \ + --extra-index-url https://pypi.org/simple/ acme-scheduling +# or in pyproject.toml / uv: an extra index with that URL +``` + +Then `meta deps sync` as for any other installed package. Nothing about the registry +reaches `.metaobjects/`: the lock records the transport (`npm`/`python` + `dir`), never a +URL. + **Only TypeScript publishes in Phase 1a — TypeScript, Python, Java and Kotlin consume.** TypeScript, Python, Java and Kotlin can *consume* a dependency (load its snapshot, extend it, overlay it); C# doesn't read `dependencies` yet (Phase 2). Java reads it at every rung @@ -424,7 +601,7 @@ If you have shipped a JVM library whose jar carried nothing but a `metadata/*.js resources directory, pulled by an ordinary dependency declaration and read off the classpath — this generalizes that pattern rather than replacing it with something unrecognizable. The artifact still travels *inside* the publisher's own ecosystem -package (an npm package's files, in Phase 1a); it is still versioned with the +package (an npm package's files or a Python package's package data); it is still versioned with the package that implements it; a consumer still pulls it through its own package manager. What's different: the artifact is **one flattened, generated file** rather than a directory of hand-placed sources, its manifest is generated rather than @@ -480,13 +657,13 @@ stays a separate, deferred option. ## Deferred Designed (see `docs/superpowers/specs/2026-09-11-fr-023-metadata-dependencies-design.md`) -but **not** built in Phase 1a — do not assume any of the following exists: +but **not** built yet — do not assume any of the following exists: -- **The `npm` and `python` transports** (and `maven`/`nuget`, see - [Phase 2](#phase-2-java-kotlin-and-c)). Only `path` resolves; the others are refused with - `ERR_DEPENDENCY_UNRESOLVED`. +- **The `maven` and `nuget` transports** (Phase 2, with the JVM and .NET consumers; see + [Phase 2](#phase-2-java-kotlin-and-c)). - **A local co-development override** (`deps.local.json`) that would point `sync` at - an unpublished local checkout without editing the committed config. + an unpublished local checkout without editing the committed config. Today, switch the + spec to `path` while co-developing, and back to `npm`/`python` before committing. - **A usage-aware breaking-change classifier** at sync/check time. Nothing automatically distinguishes a widening change (say, `@maxLength` growing) from a breaking one — the committed artifact's diff in your own history is today's @@ -504,8 +681,9 @@ but **not** built in Phase 1a — do not assume any of the following exists: ## Verified by -- [`fixtures/dependency-conformance/`](../../fixtures/dependency-conformance/) — 35 - cases. 24 pin resolution, the exhaustive `imported`/`inScope`/`inMigrateScope` sets, +- [`fixtures/dependency-conformance/`](../../fixtures/dependency-conformance/) — 37 + cases. 26 pin resolution (including `npm` and `python` dependencies loading from their + committed snapshot with nothing installed), the exhaustive `imported`/`inScope`/`inMigrateScope` sets, overlay/extends/reference behavior against a synced snapshot, and every load- and resolution-time error above, run by TypeScript (the reference implementation), Python and Java (Kotlin loads through the Java loader and Maven plugin, so it inherits the @@ -513,12 +691,15 @@ but **not** built in Phase 1a — do not assume any of the following exists: check (`current`, `drifted`, `unresolved`), run by TypeScript; Python and Java have no check yet. Three artifacts under `fixtures/dependency-conformance/artifacts/` are pinned by sha256 in the README so the corpus itself can never silently drift. - Python's runner asserts all 24 resolution cases identically to TypeScript's, with no + Python's runner asserts all 26 resolution cases identically to TypeScript's, with no exemptions — the two overlay cases that need a view child as incidental content use `view.currency` (the one concrete `view.*` subtype registered cross-port, per `fixtures/registry-conformance/expected-registry.json`), not `view.text`, so they exercise the full imported/selected/governed assertions like every - other case. + other case. `transport-cases.json` beside it pins the `npm` and `python` transports + themselves (14 cases: packages laid out under a temp `node_modules` and on a temp + Python import path, the directory each resolves to, the exact lock `sync` writes, and + each refusal), run by Node `meta deps sync`, the only writer. - [`fixtures/shared-model-conformance/`](../../fixtures/shared-model-conformance/) — 11 cases pinning what a publisher writes: the exact artifact and manifest bytes for a source tree and `include` / `exclude` / `files`, and the refusals (an empty selection, a closure failure naming diff --git a/docs/llms/llms-full.txt b/docs/llms/llms-full.txt index 8a3f52e18..b7ea6aef1 100644 --- a/docs/llms/llms-full.txt +++ b/docs/llms/llms-full.txt @@ -115,7 +115,7 @@ A library is **layered, and its core layer is inert**: the core declares no `sou ### Sharing a model across your own projects -Today the TypeScript toolchain publishes and TypeScript and Python projects consume, over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends`, overlays and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. See docs/features/metadata-dependencies.md. +Today the TypeScript toolchain publishes and TypeScript and Python projects consume, from an installed npm or Python package or over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends`, overlays and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. See docs/features/metadata-dependencies.md. --- diff --git a/docs/llms/llms.txt b/docs/llms/llms.txt index cd3ea282b..120d8daf0 100644 --- a/docs/llms/llms.txt +++ b/docs/llms/llms.txt @@ -57,7 +57,7 @@ MetaObjects pillars are capabilities of the same metadata spine, not separate pr - **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. `requirement.functional` / `requirement.architectural` are registered vocabulary in all five ports, with the loader enforcing the closed `@status` enum, and `@implementedBy` is **resolved, not trusted**: it names a real member of the real model, so a claim whose implementation was renamed or deleted fails the build rather than going quietly stale. `meta verify` reports the ledger on every run — unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on — and lints the authoring; the documentation generators render it for humans and for agents (`requirementsFile()`, `agentDocsFile()` and their counterparts in the other ports). A live claim naming nothing is a warning; a green run proves the references resolve, never that an implementation is correct. **Every port ships a `requirement-tests` generator**, which scaffolds a test per claim carrying the statement and counterexample in (a stub you fill in for TypeScript, a test that calls a witness you own in the other four); the vocabulary and the `verify` checks are in every port too, and only the authoring lint is TypeScript-only. A project that declares no `requirement.*` nodes sees no change at all. - **Libraries** — reusable declared design you opt into by name (`"libraries": ["iam"]` in `.metaobjects/config.json`): entities, the requirements they promise and the generators they imply, as one drift-gated artifact. A library is layered and its core layer is inert: it declares no source, so it adds no tables until you also opt into its `/db` layer (`["iam", "iam/db"]`). `meta eject ` copies one into your repo to own. `ai` (the LLM-call trace envelope) is `stable`; `iam` (users, nestable groups, roles, grants) is `preview`. -**Sharing a model across your own projects** — today the TypeScript toolchain publishes and TypeScript and Python projects consume, over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends` and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. +**Sharing a model across your own projects** — today the TypeScript toolchain publishes and TypeScript and Python projects consume, from an installed npm or Python package or over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends` and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. ## Implementations (npm `1.1.4` / Maven Central `8.1.4`) diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md index 4da7390fd..2574655ff 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/metadata-dependencies.md @@ -7,7 +7,16 @@ build on a shared model.** A project may declare `dependencies` in `.metaobjects/config.json` and `meta deps sync` a publisher's metadata into a committed snapshot — that snapshot loads BEFORE your own files, so a foreign abstract resolves via `extends` and a foreign node re-opens via `overlay: true` -exactly like a local one. Two rules are specific to that boundary: +exactly like a local one. A dependency names one transport: `path` (a directory +relative to the directory holding `.metaobjects/`), `npm` (an installed npm +package, e.g. `{ "name": "acme-model", "npm": "@acme/model", "dir": "metaobjects" }`) +or `python` (an installed Python package by its import name, e.g. +`{ "name": "acme-model", "python": "acme_model", "dir": "metaobjects" }`). For +`npm`/`python`, install the package with the project's package manager first; +`meta deps sync` locates it, copies its shared model into the snapshot, and says +what to install when it is missing. Prefer the package transport over a `path` into +someone else's checkout when the publisher releases a package. Two rules are +specific to that boundary: - **Say `overlay: true` on every amendment to a node you don't own.** Within one project the parser merges a same-`(type, package::name)` redeclaration whether diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md index a65e90479..54cb17138 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/dependencies-and-shared-models.md @@ -38,7 +38,13 @@ artifact is a contract whose bytes a cross-port corpus pins and whose hash consumers verify, so everything a consumer checks stays in the package, in the two functions the copy calls (`buildSharedModel`, `renderSharedModelFiles`). Do not replace those calls: an unmodified copy publishes the same bytes as the packaged -generator. Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and +generator. To publish through a package, point its `target` inside the package +(an npm package's directory listed in `files`; a Python package's directory shipped +as package data), run `meta gen`, then release the package as usual; consumers +declare an `npm` or `python` dependency on it. With no `version` option the +manifest takes the nearest `package.json` version, else a `pyproject.toml` +`[project]` version — pass `version` when neither is the package consumers install. +Only TypeScript can run it in Phase 1a. TypeScript, Python, Java and Kotlin (through Java's loader and Maven plugin) can *consume* a dependency, and only the TypeScript toolchain can *publish* one; C# does not read `dependencies` yet (Phase 2). diff --git a/fixtures/dependency-conformance/README.md b/fixtures/dependency-conformance/README.md index ca50ff310..21be4139c 100644 --- a/fixtures/dependency-conformance/README.md +++ b/fixtures/dependency-conformance/README.md @@ -14,6 +14,7 @@ cases.json # { cases: [ { name, tree, treeFiles?, config, lock?, # expectSelected?, expectMigrateGoverned?, # expectLoadError?, expectErrorFiles?, expectError?, # expectCheck? } ] } +transport-cases.json # the npm / python transport arm (Node `meta deps sync` only — see below) README.md artifacts/ # pinned dependency artifacts, referenced by cases via `treeFiles` acme-common-v1.json @@ -192,16 +193,56 @@ source — so `ERR_OVERLAY_NO_TARGET` wins. If this case ever reports `ERR_DEPENDENCY_PACKAGE_NOT_OWNED` instead, that is an ordering regression to fix, not a corpus expectation to update. +## The transport arm — `transport-cases.json` + +`meta deps sync` / `check` find a dependency's manifest directory through its transport +(DESIGN §2.1). Node `meta` is the only writer of snapshots and the lock, so only Node runs +this arm; every port then loads the snapshot, and `cases.json` carries an `npm` and a +`python` dependency (`an-npm-dependency-loads-from-its-committed-snapshot`, +`a-python-dependency-loads-from-its-committed-snapshot`) to prove those load with nothing +installed. + +```jsonc +{ "cases": [ { + "name": "…", + "tree": { … }, "treeFiles": { … }, // as in cases.json: what an install leaves on disk + "config": { … }, // written to /.metaobjects/config.json + "resolveFrom": "app", + "pythonPath": ["site"], // OPTIONAL: project-root-relative directories put on the + // interpreter's import path (PYTHONPATH), standing in for + // an installed site-packages + "expectDirs": { "": "" }, // OPTIONAL: the project-root-relative directory each + // dependency's transport resolves to (EXHAUSTIVE) + "expectLock": { … }, // the lock `sync` writes, exactly, or + "expectError": "ERR_*" // the code `sync` fails with +} ] } +``` + +Exactly one of `expectLock`, `expectError` per case. An `npm` package is laid out under a +`node_modules` directory (beside the config, or hoisted above it); a `python` package under +a `pythonPath` directory. The runner clears `$METAOBJECTS_PYTHON` and `$VIRTUAL_ENV`, so the +interpreter is `python3` on `PATH`; the interpreter ladder itself is a unit test +(`cli/test/deps-sync-packages.test.ts`), because it depends on the machine. No case +touches a network or a package manager. + +The refusals: a package not installed, a `dir` the package does not ship, a `dir` that +climbs out of the package, and a Python module that is not a package are +`ERR_DEPENDENCY_UNRESOLVED`; a `dir` that exists but holds no `metaobjects.pkg.json` is +`ERR_DEPENDENCY_MANIFEST_INVALID`. + ## Which arms each port runs -- **TypeScript** — resolution, load-time failure, lock/snapshot integrity, and the check arm - (`cli/test/dependency-check-conformance.test.ts`; the resolution arm is - `sdk/test/dependency-conformance.test.ts`). +- **TypeScript** — resolution, load-time failure, lock/snapshot integrity + (`sdk/test/dependency-conformance.test.ts`), the check arm + (`cli/test/dependency-check-conformance.test.ts`), and the transport arm + (`cli/test/dependency-transport-conformance.test.ts`). - **Python** — resolution, load-time failure, lock/snapshot integrity. Not the check arm: Python - has no `deps check` yet, so its runner filters the `expectCheck` cases out. + has no `deps check` yet, so its runner filters the `expectCheck` cases out. Not the transport + arm: Python never syncs. - **Java** — resolution, load-time failure, lock/snapshot integrity (`DependencyConformanceTest`). Not the check arm: Java has no `deps check`, so its runner - skips the `expectCheck` cases. `refuseUnownedPackages` runs after the loader's own errors, + skips the `expectCheck` cases. Not the transport arm: Java never syncs. + `refuseUnownedPackages` runs after the loader's own errors, as the ordering note above requires. Java's `inScope` is the import-exclusion conjunct only, like Python's. - **Kotlin** — inherits via Java: it loads through the same loader and Maven plugin. The diff --git a/fixtures/dependency-conformance/cases.json b/fixtures/dependency-conformance/cases.json index 0c952e030..ddabba539 100644 --- a/fixtures/dependency-conformance/cases.json +++ b/fixtures/dependency-conformance/cases.json @@ -1562,6 +1562,122 @@ "acme-common": "drifted", "acme-extra": "unresolved" } + }, + { + "name": "an-npm-dependency-loads-from-its-committed-snapshot", + "tree": { + "metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}" + }, + "treeFiles": { + ".metaobjects/deps/acme-common/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "lock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "npm": "@acme/common-model", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + }, + "expectFiles": [ + ".metaobjects/deps/acme-common/acme-common.metaobjects.json", + "metaobjects/meta.app.json" + ], + "expectImported": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ], + "expectSelected": [ + "app::Order" + ], + "expectMigrateGoverned": [ + "app::Order" + ] + }, + { + "name": "a-python-dependency-loads-from-its-committed-snapshot", + "tree": { + "metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}" + }, + "treeFiles": { + ".metaobjects/deps/acme-common/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common", + "dir": "metaobjects" + } + ] + }, + "lock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "python": "acme_common", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + }, + "expectFiles": [ + ".metaobjects/deps/acme-common/acme-common.metaobjects.json", + "metaobjects/meta.app.json" + ], + "expectImported": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ], + "expectSelected": [ + "app::Order" + ], + "expectMigrateGoverned": [ + "app::Order" + ] } ] } diff --git a/fixtures/dependency-conformance/transport-cases.json b/fixtures/dependency-conformance/transport-cases.json new file mode 100644 index 000000000..31260b9fa --- /dev/null +++ b/fixtures/dependency-conformance/transport-cases.json @@ -0,0 +1,522 @@ +{ + "cases": [ + { + "name": "npm-resolves-the-installed-package-root", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n" + }, + "treeFiles": { + "app/node_modules/@acme/common-model/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model" + } + ] + }, + "resolveFrom": "app", + "expectDirs": { + "acme-common": "app/node_modules/@acme/common-model" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "npm": "@acme/common-model" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "npm-applies-dir-inside-the-package", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/metaobjects/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n" + }, + "treeFiles": { + "app/node_modules/@acme/common-model/metaobjects/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectDirs": { + "acme-common": "app/node_modules/@acme/common-model/metaobjects" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "npm": "@acme/common-model", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "npm-resolves-a-package-hoisted-above-the-config-dir", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "node_modules/@acme/common-model/metaobjects/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n" + }, + "treeFiles": { + "node_modules/@acme/common-model/metaobjects/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectDirs": { + "acme-common": "node_modules/@acme/common-model/metaobjects" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "npm": "@acme/common-model", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "npm-resolves-a-package-whose-exports-hide-package-json", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/metaobjects/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\",\n \"type\": \"module\",\n \"exports\": {\n \".\": \"./index.js\"\n }\n}\n", + "app/node_modules/@acme/common-model/index.js": "export {};\n" + }, + "treeFiles": { + "app/node_modules/@acme/common-model/metaobjects/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectDirs": { + "acme-common": "app/node_modules/@acme/common-model/metaobjects" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "npm": "@acme/common-model", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "npm-a-package-that-is-not-installed-is-unresolved", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}" + }, + "treeFiles": {}, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + }, + { + "name": "npm-a-dir-the-package-does-not-ship-is-unresolved", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/shared/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n" + }, + "treeFiles": { + "app/node_modules/@acme/common-model/shared/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + }, + { + "name": "npm-a-dir-without-a-manifest-is-an-invalid-manifest", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n", + "app/node_modules/@acme/common-model/metaobjects/README.md": "no manifest here\n" + }, + "treeFiles": {}, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "expectDirs": { + "acme-common": "app/node_modules/@acme/common-model/metaobjects" + }, + "expectError": "ERR_DEPENDENCY_MANIFEST_INVALID" + }, + { + "name": "npm-a-dir-that-climbs-out-of-the-package-is-unresolved", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "app/node_modules/@acme/common-model/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "app/node_modules/@acme/common-model/package.json": "{\n \"name\": \"@acme/common-model\",\n \"version\": \"1.0.0\"\n}\n" + }, + "treeFiles": { + "app/node_modules/@acme/common-model/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "npm": "@acme/common-model", + "dir": "../.." + } + ] + }, + "resolveFrom": "app", + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + }, + { + "name": "python-resolves-the-package-find-spec-reports", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/acme_common/metaobjects/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "site/acme_common/__init__.py": "" + }, + "treeFiles": { + "site/acme_common/metaobjects/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectDirs": { + "acme-common": "site/acme_common/metaobjects" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "python": "acme_common", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "python-resolves-a-dotted-subpackage", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/acme/common/metaobjects/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "site/acme/__init__.py": "", + "site/acme/common/__init__.py": "" + }, + "treeFiles": { + "site/acme/common/metaobjects/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme.common", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectDirs": { + "acme-common": "site/acme/common/metaobjects" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "python": "acme.common", + "dir": "metaobjects" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "python-without-dir-resolves-the-package-directory", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/acme_common/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "site/acme_common/__init__.py": "" + }, + "treeFiles": { + "site/acme_common/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectDirs": { + "acme-common": "site/acme_common" + }, + "expectLock": { + "schema_version": 1, + "dependencies": { + "acme-common": { + "version": "1.0.0", + "metamodelVersion": "1.0", + "resolvedFrom": { + "python": "acme_common" + }, + "artifact": "acme-common.metaobjects.json", + "integrity": "sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d", + "packages": [ + "acme::common" + ], + "nodes": [ + "acme::common::Address", + "acme::common::Audited", + "acme::common::Customer" + ] + } + } + } + }, + { + "name": "python-a-package-that-is-not-installed-is-unresolved", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/.keep": "" + }, + "treeFiles": {}, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + }, + { + "name": "python-a-plain-module-is-not-a-package", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/acme_common.py": "" + }, + "treeFiles": {}, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + }, + { + "name": "python-a-dir-the-package-does-not-ship-is-unresolved", + "tree": { + "app/metaobjects/meta.app.json": "{\"metadata.root\":{\"package\":\"app\",\"children\":[{\"object.entity\":{\"name\":\"Order\",\"children\":[{\"source.rdb\":{\"@table\":\"orders\"}},{\"field.long\":{\"name\":\"id\"}},{\"identity.primary\":{\"name\":\"pk\",\"@fields\":[\"id\"]}}]}}]}}", + "site/acme_common/shared/metaobjects.pkg.json": "{\n \"schema_version\": 1,\n \"name\": \"acme-common\",\n \"version\": \"1.0.0\",\n \"metamodelVersion\": \"1.0\",\n \"artifact\": \"acme-common.metaobjects.json\",\n \"integrity\": \"sha256-10fbf886e22faceca32c56e5e647c3ff1c82f503e638cba3bd3aa9390f7c409d\",\n \"packages\": [\n \"acme::common\"\n ],\n \"nodes\": [\n \"acme::common::Address\",\n \"acme::common::Audited\",\n \"acme::common::Customer\"\n ]\n}\n", + "site/acme_common/__init__.py": "" + }, + "treeFiles": { + "site/acme_common/shared/acme-common.metaobjects.json": "artifacts/acme-common-v1.json" + }, + "config": { + "schema_version": 1, + "sources": [], + "dependencies": [ + { + "name": "acme-common", + "python": "acme_common", + "dir": "metaobjects" + } + ] + }, + "resolveFrom": "app", + "pythonPath": [ + "site" + ], + "expectError": "ERR_DEPENDENCY_UNRESOLVED" + } + ] +} diff --git a/server/typescript/packages/cli/src/index.ts b/server/typescript/packages/cli/src/index.ts index fb3327cec..f821f0b30 100644 --- a/server/typescript/packages/cli/src/index.ts +++ b/server/typescript/packages/cli/src/index.ts @@ -36,7 +36,7 @@ COMMANDS: way to get any output MetaObjects does not ship (OpenAPI, JSON Schema, …) eject Copy a reference generator into codegen/generators/ to own it (any time after init) eject --list List every ejectable generator name, grouped by package - deps sync [...] Resolve declared dependencies (path transport): sync the committed + deps sync [...] Resolve declared dependencies (path, npm, python): sync the committed snapshot + sha256 lock (.metaobjects/deps/, .metaobjects/deps.lock.json) deps list One line per locked dependency: name, version, hash, node/package summary types [query] Search the metadata vocabulary (types, subtypes, @attrs) by name or description @@ -258,16 +258,15 @@ emit for your model. deps: `meta deps — sync a declared metadata dependency's committed snapshot USAGE: - meta deps sync [...] Resolve each declared dependency (path transport only in - this release — npm/python refuse by name), validate its - manifest + artifact, copy it into .metaobjects/deps//, + meta deps sync [...] Resolve each declared dependency (path, npm or python + transport), validate its manifest + artifact, copy it into .metaobjects/deps//, and pin it in .metaobjects/deps.lock.json. Naming one or more s syncs only those; bare 'sync' syncs everything declared. A dependency removed from config is PRUNED from the lock and its snapshot dir deleted, every run. meta deps list One line per locked dependency: name, version, hash8, node count, packages. - meta deps check Re-resolve each declared dependency (path transport, same + meta deps check Re-resolve each declared dependency (any transport, same as sync steps 1-2) and compare its INSTALLED artifact's hash against .metaobjects/deps.lock.json. Read-only — never touches the lock or the snapshot. Exits 1 if any dependency @@ -278,9 +277,13 @@ FLAGS: --format Output format (global flag; default toon off-TTY) --help, -h Print this help -Only the "path" transport resolves in this release — a "npm"/"python" dependency spec is -valid config (reserved for a future toolchain) but 'meta deps sync' refuses it by name: -"transport \`npm\` is not supported by this toolchain yet; use \`path\`". +Transports: "path" is a directory relative to the config's directory; "npm" is an installed +npm package, resolved from the config's directory the way Node resolves it; "python" is an +installed Python package, found with importlib.util.find_spec by the project's interpreter +($METAOBJECTS_PYTHON, else $VIRTUAL_ENV, else ./.venv, else python3 — the interpreter used is +printed). "dir" names the shared-model directory inside an npm or python package (default: +the package root). Install the package with your package manager first; sync copies its +shared model into the committed snapshot, so nothing else needs it installed. The lock is the ONLY thing 'meta deps sync' writes to besides the snapshot directory — sync never touches your own metadata files. 'meta deps check' compares the lock against diff --git a/server/typescript/packages/cli/src/lib/dependency-sync.ts b/server/typescript/packages/cli/src/lib/dependency-sync.ts index b9b9e00d9..4624de81d 100644 --- a/server/typescript/packages/cli/src/lib/dependency-sync.ts +++ b/server/typescript/packages/cli/src/lib/dependency-sync.ts @@ -9,10 +9,10 @@ // `planSync` composes them over every declared dependency (transport // resolution, per-dependency validation, the cross-dependency node-collision // check); `applySync` is the only function that touches the filesystem. -import { existsSync, statSync } from "node:fs"; +import { existsSync } from "node:fs"; import { cp, mkdir, mkdtemp, readFile, rm, writeFile } from "node:fs/promises"; import { tmpdir } from "node:os"; -import { join, resolve } from "node:path"; +import { join } from "node:path"; import { codeSource, InMemoryStringSource, @@ -39,6 +39,7 @@ import { type LockEntry, } from "@metaobjectsdev/sdk"; import { describeError } from "./error-text.js"; +import { resolveTransport } from "./dependency-transports.js"; /** `dependency-sync.ts`'s own `ParseError` factory for every * `ERR_DEPENDENCY_MANIFEST_INVALID` case (DESIGN §4.1 step 2 and step 4): @@ -76,30 +77,26 @@ function sameSortedArray(a: readonly string[], b: readonly string[]): boolean { } /** - * DESIGN §4.1 step 1 — resolve a dependency spec's transport to a directory. - * `npm` / `python` refuse by design (Phase 1a ships `path` only — DESIGN - * §11.3's transports table); a `path` that does not resolve to a directory on - * disk is the SAME code (`ERR_DEPENDENCY_UNRESOLVED`) with a different - * detail, because both are "the transport could not locate a directory - * holding metaobjects.pkg.json" — the manifest read is a separate step - * (`readManifestDir`) that only runs once a directory is in hand. + * DESIGN §4.1 step 1 — resolve a dependency spec's transport (`path`, `npm` or + * `python`) to the directory that should hold its `metaobjects.pkg.json`. The + * transports live in `dependency-transports.ts`; a failure is + * `ERR_DEPENDENCY_UNRESOLVED` naming what was tried and the fix. The manifest + * read is a separate step (`readManifestDir`) that only runs once a directory + * is in hand. */ export function resolveDependencyDir(configDir: string, spec: DependencySpec): string { - const name = dependencyName(spec); - if ("path" in spec) { - const dir = resolve(configDir, spec.path); - if (!existsSync(dir) || !statSync(dir).isDirectory()) { - throw new ParseError( - `dependency "${name}": the "path" transport resolved to ${dir}, but no directory exists there`, - { code: "ERR_DEPENDENCY_UNRESOLVED", source: codeSource("resolveDependencyDir") }, - ); - } - return dir; - } - const kind = "npm" in spec ? "npm" : "python"; - throw new ParseError( - `dependency "${name}": transport \`${kind}\` is not supported by this toolchain yet; use \`path\``, - { code: "ERR_DEPENDENCY_UNRESOLVED", source: codeSource("resolveDependencyDir") }, + return resolveTransport(configDir, spec).dir; +} + +/** The fix for an installed package that holds no manifest where `dir` points. */ +function missingManifestFix(spec: DependencySpec | undefined): string { + if (spec === undefined || "path" in spec) return ""; + const shipped = "npm" in spec ? `listed in package.json "files"` : "included as package data"; + const where = spec.dir === undefined ? "the package root" : `"${spec.dir}"`; + return ( + `. The package is installed, but ${where} holds no shared model: set "dir" in ` + + `.metaobjects/config.json to the directory the publisher's sharedModelFile() output ships in, ` + + `or ask the publisher to emit that output into the package (${shipped}) and release again` ); } @@ -117,13 +114,13 @@ export interface ManifestRead { * is caught here, before the (more expensive) standalone load in * {@link standaloneLoadCheck}. */ -export async function readManifestDir(dir: string, name: string): Promise { +export async function readManifestDir(dir: string, name: string, spec?: DependencySpec): Promise { const manifestPath = join(dir, MANIFEST_FILE); let raw: string; try { raw = await readFile(manifestPath, "utf8"); } catch { - manifestInvalid(name, "readManifestDir", `no ${MANIFEST_FILE} found at ${manifestPath}`); + manifestInvalid(name, "readManifestDir", `no ${MANIFEST_FILE} found at ${manifestPath}${missingManifestFix(spec)}`); } let parsed: unknown; @@ -257,6 +254,64 @@ function resolvedFromOf(spec: DependencySpec): LockEntry["resolvedFrom"] { return spec.dir === undefined ? { python: spec.python } : { python: spec.python, dir: spec.dir }; } +const PEP440_PRE_LABEL: Readonly> = { + alpha: "a", + a: "a", + beta: "b", + b: "b", + c: "rc", + pre: "rc", + preview: "rc", + rc: "rc", +}; + +/** PEP 440 Appendix B `VERSION_PATTERN` (as in the `packaging` library), + * anchored and case-insensitive. */ +const PEP440_VERSION = + /^\s*v?(?:(?:(?[0-9]+)!)?(?[0-9]+(?:\.[0-9]+)*)(?:[-_.]?(?alpha|a|beta|b|preview|pre|c|rc)[-_.]?(?[0-9]+)?)?(?:-(?[0-9]+)|[-_.]?(?post|rev|r)[-_.]?(?[0-9]+)?)?(?:[-_.]?(?dev)[-_.]?(?[0-9]+)?)?)(?:\+(?[a-z0-9]+(?:[-_.][a-z0-9]+)*))?\s*$/i; + +const pep440Number = (n: string | undefined): string => (n ?? "0").replace(/^0+(?=\d)/, ""); + +/** The PEP 440 normal form an installed distribution reports (`1.0.0-rc.1` → + * `1.0.0rc1`, `1.0-1` → `1.0.post1`), or undefined when `version` is not + * PEP 440. */ +function pep440Normalized(version: string): string | undefined { + const g = PEP440_VERSION.exec(version)?.groups; + if (g === undefined) return undefined; + const epoch = g.epoch !== undefined && pep440Number(g.epoch) !== "0" ? `${pep440Number(g.epoch)}!` : ""; + const release = g.release!.split(".").map(pep440Number).join("."); + const pre = g.preL !== undefined ? `${PEP440_PRE_LABEL[g.preL.toLowerCase()]}${pep440Number(g.preN)}` : ""; + const post = + g.postN1 !== undefined || g.postL !== undefined ? `.post${pep440Number(g.postN1 ?? g.postN2)}` : ""; + const dev = g.devL !== undefined ? `.dev${pep440Number(g.devN)}` : ""; + const local = g.local !== undefined ? `+${g.local.toLowerCase().replace(/[-_]/g, ".")}` : ""; + return `${epoch}${release}${pre}${post}${dev}${local}`; +} + +/** The installed package's own version should be the manifest's: `sharedModelFile()` + * defaults the manifest version to the host package's. A difference means the + * publisher bumped the package without regenerating the artifact (or set + * `version` by hand). The lock records the manifest's version either way. */ +export function packageVersionWarning( + name: string, + spec: DependencySpec, + packageVersion: string | undefined, + manifestVersion: string, +): string | undefined { + if (packageVersion === undefined) return undefined; + const same = + "python" in spec + ? (pep440Normalized(packageVersion) ?? packageVersion) === (pep440Normalized(manifestVersion) ?? manifestVersion) + : packageVersion === manifestVersion; + if (same) return undefined; + const pkg = "npm" in spec ? `npm package "${spec.npm}"` : "python" in spec ? `Python package "${spec.python}"` : "package"; + return ( + `dependency "${name}": the installed ${pkg} is ${packageVersion}, but its shared-model manifest ` + + `records ${manifestVersion} — the publisher likely released without regenerating the artifact. ` + + `The lock records ${manifestVersion}.` + ); +} + /** One dependency this run will (re)sync — either brand new or a changed/ * unchanged re-sync of one already in the lock. */ export interface SyncedDependencyPlan { @@ -266,6 +321,9 @@ export interface SyncedDependencyPlan { readonly artifactContent: string; readonly resolvedFrom: LockEntry["resolvedFrom"]; readonly oldEntry: LockEntry | undefined; + /** How the transport found it (e.g. the Python interpreter), appended to the + * report line; never written to the lock (DESIGN §3.3: no host paths). */ + readonly via?: string | undefined; } /** A lock entry whose dependency is no longer declared in config. */ @@ -299,6 +357,7 @@ export async function planSync( specs: readonly DependencySpec[], lock: Lock | undefined, filterNames: readonly string[] = [], + env: NodeJS.ProcessEnv = process.env, ): Promise { const allNames = specs.map(dependencyName); const declared = new Set(allNames); @@ -340,10 +399,12 @@ export async function planSync( const plans: SyncedDependencyPlan[] = []; for (const spec of targets) { const name = dependencyName(spec); - const dir = resolveDependencyDir(configDir, spec); - const { manifest, artifactContent } = await readManifestDir(dir, name); + const resolved = resolveTransport(configDir, spec, env); + const { manifest, artifactContent } = await readManifestDir(resolved.dir, name, spec); const warning = validateAgainstSpec(manifest, spec); if (warning !== undefined) warnings.push(warning); + const versionWarning = packageVersionWarning(name, spec, resolved.packageVersion, manifest.version); + if (versionWarning !== undefined) warnings.push(versionWarning); await standaloneLoadCheck(manifest, artifactContent); for (const node of manifest.nodes) { @@ -361,7 +422,15 @@ export async function planSync( const oldEntry = oldEntries[name]; const action: "sync" | "unchanged" = oldEntry !== undefined && oldEntry.integrity === manifest.integrity ? "unchanged" : "sync"; - plans.push({ name, action, manifest, artifactContent, resolvedFrom: resolvedFromOf(spec), oldEntry }); + plans.push({ + name, + action, + manifest, + artifactContent, + resolvedFrom: resolvedFromOf(spec), + oldEntry, + via: resolved.note, + }); } const prunes: PrunedDependencyPlan[] = Object.entries(oldEntries) @@ -419,8 +488,9 @@ export async function applySync( }; dependencies[p.name] = newEntry; + const via = p.via !== undefined ? ` — ${p.via}` : ""; if (p.action === "unchanged") { - report.push({ name: p.name, action: "unchanged", line: `unchanged ${p.name} ${p.manifest.version}` }); + report.push({ name: p.name, action: "unchanged", line: `unchanged ${p.name} ${p.manifest.version}${via}` }); continue; } @@ -431,7 +501,7 @@ export async function applySync( report.push({ name: p.name, action: "sync", - line: `${verb} ${p.name} ${oldVersion}→${p.manifest.version} (${oldHash}→${newHash})`, + line: `${verb} ${p.name} ${oldVersion}→${p.manifest.version} (${oldHash}→${newHash})${via}`, }); if (!opts.dryRun) { @@ -602,7 +672,7 @@ export async function checkDependencies( try { const dir = resolveDependencyDir(configDir, spec); - const { manifest, artifactContent } = await readManifestDir(dir, name); + const { manifest, artifactContent } = await readManifestDir(dir, name, spec); // Never trust the manifest's own `integrity` field for the comparison — // re-hash the bytes actually sitting on disk right now. // diff --git a/server/typescript/packages/cli/src/lib/dependency-transports.ts b/server/typescript/packages/cli/src/lib/dependency-transports.ts new file mode 100644 index 000000000..a6230c311 --- /dev/null +++ b/server/typescript/packages/cli/src/lib/dependency-transports.ts @@ -0,0 +1,289 @@ +// server/typescript/packages/cli/src/lib/dependency-transports.ts +// +// FR-023 DESIGN §2.1 "Transports": a transport answers one question at sync +// time — which directory holds the publisher's `metaobjects.pkg.json`? Node +// `meta deps sync` / `check` are the only callers; every port then loads the +// committed snapshot, never the installed package. +// +// path — the directory, relative to the declaring config's directory. +// npm — the installed package's root (found in `node_modules` up the tree +// from the consumer's config directory, as Node does), then `dir`. +// python — the installed package's directory, as the consumer's own Python +// interpreter's `importlib.util.find_spec` reports it, then `dir`. +// +// A refusal names what was tried and the fix. All are `ERR_DEPENDENCY_UNRESOLVED` +// (the package, the interpreter or the directory could not be found); a +// directory that exists but holds no manifest is the manifest read's +// `ERR_DEPENDENCY_MANIFEST_INVALID`, not this module's. +import { spawnSync } from "node:child_process"; +import { existsSync, readFileSync, realpathSync, statSync } from "node:fs"; +import { dirname, isAbsolute, join, relative, resolve, sep } from "node:path"; +import { codeSource, ParseError } from "@metaobjectsdev/metadata"; +import { dependencyName, type DependencySpec } from "@metaobjectsdev/sdk"; + +/** Where a transport found the publisher's manifest directory. */ +export interface ResolvedTransport { + /** Absolute directory expected to hold `metaobjects.pkg.json`. */ + readonly dir: string; + /** The installed package's own version (npm `package.json` / the Python + * distribution's metadata), when the transport can read one. Compared with + * the manifest's `version` by the caller; never written to the lock. */ + readonly packageVersion?: string | undefined; + /** How it was found, for the report — e.g. the Python interpreter used. */ + readonly note?: string | undefined; +} + +/** The env keys the python transport reads (DESIGN §2.1 interpreter ladder). */ +export const METAOBJECTS_PYTHON_ENV = "METAOBJECTS_PYTHON"; +const VIRTUAL_ENV_ENV = "VIRTUAL_ENV"; + +const NPM_PACKAGE_NAME = /^(@[a-z0-9][a-z0-9._~-]*\/)?[a-z0-9][a-z0-9._~-]*$/; +const PYTHON_MODULE_NAME = /^[A-Za-z_][A-Za-z0-9_]*(\.[A-Za-z_][A-Za-z0-9_]*)*$/; +const PYTHON_TIMEOUT_MS = 60_000; + +function unresolved(name: string, detail: string): never { + throw new ParseError(`dependency "${name}": ${detail}`, { + code: "ERR_DEPENDENCY_UNRESOLVED", + source: codeSource("resolveTransport"), + }); +} + +function isDirectory(p: string): boolean { + return existsSync(p) && statSync(p).isDirectory(); +} + +/** `dir` applied to a package root. Refuses an absolute `dir` or one that climbs + * out of the package: the artifact is part of what the package ships. */ +function applyDir(name: string, kind: string, pkg: string, root: string, dir: string | undefined): string { + if (dir === undefined) return root; + const target = resolve(root, dir); + const rel = relative(root, target); + if (isAbsolute(dir) || rel === ".." || rel.startsWith(`..${sep}`)) { + unresolved(name, `"dir": "${dir}" must be a path inside the ${kind} package "${pkg}" (relative, no "..")`); + } + if (!isDirectory(target)) { + unresolved( + name, + `the ${kind} package "${pkg}" is installed at ${root}, but has no "${dir}" directory. ` + + `Check "dir" in .metaobjects/config.json, or ask the publisher to ship the shared-model ` + + `output in the package (${kind === "npm" ? `package.json "files"` : "package data"}).`, + ); + } + return target; +} + +function readVersion(packageJson: string): string | undefined { + try { + const v = (JSON.parse(readFileSync(packageJson, "utf8")) as { version?: unknown }).version; + return typeof v === "string" ? v : undefined; + } catch { + return undefined; + } +} + +/** + * The installed npm package's `package.json`, found the way Node's resolver + * finds a package from the consumer's config directory: `node_modules/` in + * that directory, then in each parent (DESIGN §2.1). A nested, hoisted + * (workspace) or pnpm-symlinked install all appear there. This deliberately does + * not ask `require.resolve` for the package.json subpath: a package whose `exports` map does + * not expose `./package.json` refuses that subpath, and a model-only package may + * have no entry point to resolve and walk up from. + */ +function findNpmPackageJson(configDir: string, pkg: string): string | undefined { + let dir = configDir; + for (;;) { + const candidate = join(dir, "node_modules", pkg, "package.json"); + if (existsSync(candidate)) return candidate; + const parent = dirname(dir); + if (parent === dir) return undefined; + dir = parent; + } +} + +function resolveNpm(configDir: string, name: string, pkg: string, dir: string | undefined): ResolvedTransport { + if (!NPM_PACKAGE_NAME.test(pkg)) { + unresolved(name, `"npm": "${pkg}" is not an npm package name`); + } + const packageJson = findNpmPackageJson(configDir, pkg); + if (packageJson === undefined) { + unresolved( + name, + `the npm package "${pkg}" is not installed where ${configDir} can resolve it. ` + + `Install it (\`npm install ${pkg}\`, or your package manager's equivalent), then re-run.`, + ); + } + const root = realpathSync(dirname(packageJson)); + return { dir: applyDir(name, "npm", pkg, root, dir), packageVersion: readVersion(packageJson) }; +} + +/** The interpreter ladder (DESIGN §2.1): `$METAOBJECTS_PYTHON` → + * `$VIRTUAL_ENV`'s python → `/.venv`'s python → `python3`. An + * explicitly named interpreter that does not exist is refused, never skipped: + * falling through to another environment would read a different install. */ +function pythonInterpreter( + configDir: string, + name: string, + env: NodeJS.ProcessEnv, +): { readonly exe: string; readonly from: string } { + const explicit = env[METAOBJECTS_PYTHON_ENV]; + if (explicit !== undefined && explicit !== "") return { exe: explicit, from: `$${METAOBJECTS_PYTHON_ENV}` }; + const venvPython = (venv: string): string | undefined => + [join(venv, "bin", "python"), join(venv, "Scripts", "python.exe")].find((p) => existsSync(p)); + const active = env[VIRTUAL_ENV_ENV]; + if (active !== undefined && active !== "") { + const exe = venvPython(active); + if (exe === undefined) { + unresolved( + name, + `$${VIRTUAL_ENV_ENV} is ${active}, but it has no python interpreter. Re-create or ` + + `deactivate that virtualenv, or point $${METAOBJECTS_PYTHON_ENV} at an interpreter.`, + ); + } + return { exe, from: `$${VIRTUAL_ENV_ENV}` }; + } + const local = venvPython(join(configDir, ".venv")); + if (local !== undefined) return { exe: local, from: ".venv" }; + return { exe: process.platform === "win32" ? "python" : "python3", from: "PATH" }; +} + +// Prints one JSON line. `find_spec` imports the parent packages of a dotted +// name, so this runs inside the consumer's own environment (DESIGN §11 risks). +const FIND_SPEC_SCRIPT = ` +import importlib.util, json, sys +name = sys.argv[1] +out = {"executable": sys.executable} +try: + spec = importlib.util.find_spec(name) +except ModuleNotFoundError as e: + # A parent of a dotted name that is itself absent is "not installed"; any + # other missing module was raised by a parent's own import. + if e.name and (name == e.name or name.startswith(e.name + ".")): + out["error"] = "missing" + else: + out["error"] = "import" + out["detail"] = type(e).__name__ + ": " + str(e) + spec = None +except Exception as e: + out["error"] = "import" + out["detail"] = type(e).__name__ + ": " + str(e) + spec = None +if "error" not in out: + if spec is None: + out["error"] = "missing" + elif not spec.submodule_search_locations: + out["error"] = "module" + out["detail"] = str(spec.origin) + else: + out["locations"] = list(spec.submodule_search_locations) + try: + import importlib.metadata as md + dists = md.packages_distributions().get(name.split(".")[0], []) + if len(set(dists)) == 1: + out["version"] = md.version(dists[0]) + except Exception: + pass +print(json.dumps(out)) +`; + +interface FindSpecResult { + readonly executable?: string; + readonly error?: "import" | "missing" | "module"; + readonly detail?: string; + readonly locations?: readonly string[]; + readonly version?: string; +} + +function resolvePython( + configDir: string, + name: string, + module: string, + dir: string | undefined, + env: NodeJS.ProcessEnv, +): ResolvedTransport { + if (!PYTHON_MODULE_NAME.test(module)) { + unresolved(name, `"python": "${module}" is not a Python module name (dotted identifiers, e.g. acme_model)`); + } + const interp = pythonInterpreter(configDir, name, env); + const run = spawnSync(interp.exe, ["-c", FIND_SPEC_SCRIPT, module], { + cwd: configDir, + env: { ...env, PYTHONDONTWRITEBYTECODE: "1" }, + encoding: "utf8", + timeout: PYTHON_TIMEOUT_MS, + }); + const tried = `${interp.exe} (from ${interp.from})`; + if (run.error !== undefined) { + const enoent = (run.error as NodeJS.ErrnoException).code === "ENOENT"; + unresolved( + name, + enoent + ? `no Python interpreter at ${tried}. Install Python, activate the project's virtualenv, ` + + `or point $${METAOBJECTS_PYTHON_ENV} at the interpreter that has "${module}" installed.` + : `running ${tried} failed: ${run.error.message}`, + ); + } + let result: FindSpecResult; + try { + const lastLine = run.stdout.trim().split("\n").pop() ?? ""; + result = JSON.parse(lastLine) as FindSpecResult; + } catch { + unresolved( + name, + `${tried} exited ${run.status ?? "abnormally"} without reporting where "${module}" is` + + (run.stderr.trim() !== "" ? `: ${run.stderr.trim().split("\n").pop()}` : ""), + ); + } + const used = result.executable ?? interp.exe; + if (result.error === "import") { + unresolved( + name, + `importing the parent package of "${module}" in ${used} (from ${interp.from}) raised ` + + `${result.detail ?? "an error"}. Fix that environment so the import succeeds, then re-run.`, + ); + } + if (result.error === "missing") { + unresolved( + name, + `the Python package "${module}" is not installed in ${used} (from ${interp.from}). Install the distribution that provides it into that environment (for example ` + + `\`pip install ${module.split(".")[0]!.replace(/_/g, "-")}\`), or point ` + + `$${METAOBJECTS_PYTHON_ENV} at the interpreter that has it.`, + ); + } + if (result.error === "module" || result.locations === undefined || result.locations.length === 0) { + unresolved( + name, + `"${module}" is a module (${result.detail ?? "no directory"}), not a package directory — ` + + `the shared-model files must ship as package data inside a package.`, + ); + } + // A namespace package can span several directories; take the first that holds `dir`. + const root = + result.locations.find((loc) => dir === undefined || isDirectory(resolve(loc, dir))) ?? result.locations[0]!; + return { + dir: applyDir(name, "python", module, root, dir), + packageVersion: result.version, + note: `python: ${used} (from ${interp.from})`, + }; +} + +/** + * DESIGN §4.1 step 1 — resolve a dependency spec's transport to the directory + * that should hold its `metaobjects.pkg.json`. Throws `ERR_DEPENDENCY_UNRESOLVED` + * naming what was tried and the fix. + */ +export function resolveTransport( + configDir: string, + spec: DependencySpec, + env: NodeJS.ProcessEnv = process.env, +): ResolvedTransport { + const name = dependencyName(spec); + if ("path" in spec) { + const dir = resolve(configDir, spec.path); + if (!isDirectory(dir)) { + unresolved(name, `the "path" transport resolved to ${dir}, but no directory exists there`); + } + return { dir }; + } + if ("npm" in spec) return resolveNpm(configDir, name, spec.npm, spec.dir); + return resolvePython(configDir, name, spec.python, spec.dir, env); +} diff --git a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap index cb6959984..9a85de3d0 100644 --- a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap +++ b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap @@ -16,7 +16,7 @@ COMMANDS: way to get any output MetaObjects does not ship (OpenAPI, JSON Schema, …) eject Copy a reference generator into codegen/generators/ to own it (any time after init) eject --list List every ejectable generator name, grouped by package - deps sync [...] Resolve declared dependencies (path transport): sync the committed + deps sync [...] Resolve declared dependencies (path, npm, python): sync the committed snapshot + sha256 lock (.metaobjects/deps/, .metaobjects/deps.lock.json) deps list One line per locked dependency: name, version, hash, node/package summary types [query] Search the metadata vocabulary (types, subtypes, @attrs) by name or description diff --git a/server/typescript/packages/cli/test/dependency-transport-conformance.test.ts b/server/typescript/packages/cli/test/dependency-transport-conformance.test.ts new file mode 100644 index 000000000..89d5ec924 --- /dev/null +++ b/server/typescript/packages/cli/test/dependency-transport-conformance.test.ts @@ -0,0 +1,108 @@ +// Runs fixtures/dependency-conformance/transport-cases.json (FR-023 DESIGN §2.1) +// against `meta deps sync`'s transports: an `npm` package installed under a temp +// `node_modules`, a `python` package on a temp import path. No network, no +// package manager — each case lays out what an install would leave on disk. +// +// Node `meta deps sync` is the only writer of snapshots and the lock, so only +// this runner executes the transport arm; every port then loads the snapshot +// (the `cases.json` arm, which carries npm and python cases too). +import { afterAll, describe, expect, test } from "bun:test"; +import { spawnSync } from "node:child_process"; +import { copyFile, mkdir, mkdtemp, readFile, realpath, rm, writeFile } from "node:fs/promises"; +import { tmpdir } from "node:os"; +import { delimiter, dirname, join, relative, resolve } from "node:path"; +import { DEFAULT_METAOBJECTS_DIR, LOCK_FILE, dependencyName, loadConfig } from "@metaobjectsdev/sdk"; +import { applySync, planSync } from "../src/lib/dependency-sync.js"; +import { METAOBJECTS_PYTHON_ENV, resolveTransport } from "../src/lib/dependency-transports.js"; + +interface Case { + readonly name: string; + readonly tree: Record; + readonly treeFiles?: Record; + readonly config: unknown; + readonly resolveFrom?: string; + readonly pythonPath?: readonly string[]; + readonly expectDirs?: Record; + readonly expectLock?: unknown; + readonly expectError?: string; +} + +const CORPUS_DIR = resolve(import.meta.dir, "../../../../../fixtures/dependency-conformance"); +const cases: Case[] = JSON.parse(await readFile(join(CORPUS_DIR, "transport-cases.json"), "utf8")).cases; +const PYTHON = process.platform === "win32" ? "python" : "python3"; +const hasPython = spawnSync(PYTHON, ["--version"]).status === 0; + +const roots: string[] = []; +afterAll(async () => { + await Promise.all(roots.map((d) => rm(d, { recursive: true, force: true }))); +}); + +async function materialize(c: Case): Promise<{ root: string; configDir: string }> { + const root = await realpath(await mkdtemp(join(tmpdir(), "mo-dep-transport-"))); + roots.push(root); + for (const [rel, content] of Object.entries(c.tree)) { + const abs = join(root, rel); + await mkdir(dirname(abs), { recursive: true }); + await writeFile(abs, content); + } + for (const [rel, corpusRel] of Object.entries(c.treeFiles ?? {})) { + const abs = join(root, rel); + await mkdir(dirname(abs), { recursive: true }); + await copyFile(join(CORPUS_DIR, corpusRel), abs); + } + const configDir = resolve(root, c.resolveFrom ?? "."); + await mkdir(join(configDir, DEFAULT_METAOBJECTS_DIR), { recursive: true }); + await writeFile(join(configDir, DEFAULT_METAOBJECTS_DIR, "config.json"), JSON.stringify(c.config, null, 2)); + return { root, configDir }; +} + +/** The case's import path stands in for an installed site-packages; the host's + * own virtualenv and interpreter override are cleared so the result does not + * depend on the machine running the corpus. */ +function caseEnv(root: string, c: Case): NodeJS.ProcessEnv { + const env: NodeJS.ProcessEnv = { ...process.env }; + delete env[METAOBJECTS_PYTHON_ENV]; + delete env.VIRTUAL_ENV; + env.PYTHONPATH = (c.pythonPath ?? []).map((p) => join(root, p)).join(delimiter); + return env; +} + +describe("dependency conformance — the transport arm", () => { + test("the corpus carries npm and python transport cases (a silent zero-case run is a failed gate)", () => { + expect(cases.some((c) => JSON.stringify(c.config).includes('"npm"'))).toBe(true); + expect(cases.some((c) => JSON.stringify(c.config).includes('"python"'))).toBe(true); + }); + + for (const c of cases) { + const needsPython = JSON.stringify(c.config).includes('"python"'); + test.skipIf(needsPython && !hasPython)(c.name, async () => { + if ((c.expectLock === undefined) === (c.expectError === undefined)) { + throw new Error(`transport case "${c.name}" must carry exactly one of expectLock, expectError`); + } + const { root, configDir } = await materialize(c); + const env = caseEnv(root, c); + const cfg = await loadConfig(join(configDir, DEFAULT_METAOBJECTS_DIR)); + + if (c.expectDirs !== undefined) { + const dirs = Object.fromEntries( + cfg.dependencies.map((s) => [ + dependencyName(s), + relative(root, resolveTransport(configDir, s, env).dir).split("\\").join("/"), + ]), + ); + expect(dirs).toEqual(c.expectDirs); + } + + if (c.expectError !== undefined) { + await expect(planSync(configDir, cfg.dependencies, undefined, [], env)).rejects.toMatchObject({ + code: c.expectError, + }); + return; + } + const plan = await planSync(configDir, cfg.dependencies, undefined, [], env); + await applySync(configDir, plan, { dryRun: false }); + const written = JSON.parse(await readFile(join(configDir, DEFAULT_METAOBJECTS_DIR, LOCK_FILE), "utf8")); + expect(written).toEqual(c.expectLock); + }); + } +}); diff --git a/server/typescript/packages/cli/test/deps-sync-packages.test.ts b/server/typescript/packages/cli/test/deps-sync-packages.test.ts new file mode 100644 index 000000000..a379f2785 --- /dev/null +++ b/server/typescript/packages/cli/test/deps-sync-packages.test.ts @@ -0,0 +1,368 @@ +// FR-023 DESIGN §2.1 — `meta deps sync` / `check` from an INSTALLED package: +// the `npm` transport (a temp `node_modules`), the `python` transport (a temp +// import path, a `.venv`, `$VIRTUAL_ENV`, `$METAOBJECTS_PYTHON`, and one real +// virtualenv). Nothing touches a network or a package manager: each test lays +// out on disk what an install would leave there. +import { afterAll, afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { spawnSync } from "node:child_process"; +import { mkdirSync, mkdtempSync, readFileSync, realpathSync, rmSync, symlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join, resolve } from "node:path"; +import { METAMODEL_VERSION } from "@metaobjectsdev/metadata"; +import { sha256Integrity } from "@metaobjectsdev/sdk"; +import { run } from "../src/index.js"; +import { packageVersionWarning } from "../src/lib/dependency-sync.js"; +import { METAOBJECTS_PYTHON_ENV, resolveTransport } from "../src/lib/dependency-transports.js"; + +const DEP = "acme-common"; +const ARTIFACT = "acme-common.metaobjects.json"; +const NODES = ["acme::common::Address", "acme::common::Audited", "acme::common::Customer"]; +const CORPUS = resolve(import.meta.dirname, "../../../../../fixtures/dependency-conformance/artifacts"); +const V1 = readFileSync(join(CORPUS, "acme-common-v1.json")); +const WIDENED = readFileSync(join(CORPUS, "acme-common-v1-widened.json")); +const PYTHON = process.platform === "win32" ? "python" : "python3"; +const hasPython = spawnSync(PYTHON, ["--version"]).status === 0; + +const APP = JSON.stringify({ + "metadata.root": { + package: "app", + children: [ + { + "object.entity": { + name: "Order", + children: [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { name: "id" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, + }, + ], + }, +}); + +/** What a publisher's `sharedModelFile()` output looks like inside its package. */ +function publish(dir: string, version: string, artifact: Buffer = V1): void { + mkdirSync(dir, { recursive: true }); + writeFileSync(join(dir, ARTIFACT), artifact); + writeFileSync( + join(dir, "metaobjects.pkg.json"), + JSON.stringify({ + schema_version: 1, + name: DEP, + version, + metamodelVersion: METAMODEL_VERSION, + artifact: ARTIFACT, + integrity: sha256Integrity(artifact), + packages: ["acme::common"], + nodes: NODES, + }), + ); +} + +const roots: string[] = []; +afterAll(() => { + for (const d of roots) rmSync(d, { recursive: true, force: true }); +}); + +function consumer(dependency: Record): string { + const root = realpathSync(mkdtempSync(join(tmpdir(), "deps-pkg-"))); + roots.push(root); + mkdirSync(join(root, "metaobjects"), { recursive: true }); + writeFileSync(join(root, "metaobjects", "meta.app.json"), APP); + mkdirSync(join(root, ".metaobjects"), { recursive: true }); + writeFileSync( + join(root, ".metaobjects", "config.json"), + JSON.stringify({ schema_version: 1, sources: [], dependencies: [{ name: DEP, ...dependency }] }), + ); + return root; +} + +function npmInstall(root: string, version: string, opts: { manifestVersion?: string; artifact?: Buffer } = {}): void { + const pkg = join(root, "node_modules", "@acme", "common-model"); + mkdirSync(pkg, { recursive: true }); + writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@acme/common-model", version, files: ["metaobjects"] })); + publish(join(pkg, "metaobjects"), opts.manifestVersion ?? version, opts.artifact); +} + +function lockOf(root: string): { dependencies: Record> } { + return JSON.parse(readFileSync(join(root, ".metaobjects", "deps.lock.json"), "utf8")); +} + +let out: string[]; +const saved: Record = {}; +const ENV_KEYS = [METAOBJECTS_PYTHON_ENV, "VIRTUAL_ENV", "PYTHONPATH"]; +let origLog: typeof console.log; +let origErr: typeof console.error; +beforeEach(() => { + out = []; + origLog = console.log; + origErr = console.error; + console.log = (...a: unknown[]) => void out.push(a.map(String).join(" ")); + console.error = (...a: unknown[]) => void out.push(a.map(String).join(" ")); + for (const k of ENV_KEYS) { + saved[k] = process.env[k]; + delete process.env[k]; + } +}); +afterEach(() => { + console.log = origLog; + console.error = origErr; + for (const k of ENV_KEYS) { + if (saved[k] === undefined) delete process.env[k]; + else process.env[k] = saved[k]; + } +}); + +const sync = (root: string): Promise => run(["deps", "sync", "--format", "text", "--cwd", root]); +const check = (root: string): Promise => run(["deps", "check", "--format", "text", "--cwd", root]); + +describe("meta deps sync — the npm transport", () => { + test("syncs an installed package's shared model and records the transport verbatim", async () => { + const root = consumer({ npm: "@acme/common-model", dir: "metaobjects" }); + npmInstall(root, "1.0.0"); + + expect(await sync(root)).toBe(0); + expect(readFileSync(join(root, ".metaobjects", "deps", DEP, ARTIFACT))).toEqual(V1); + const entry = lockOf(root).dependencies[DEP]!; + expect(entry.resolvedFrom).toEqual({ npm: "@acme/common-model", dir: "metaobjects" }); + expect(entry.version).toBe("1.0.0"); + expect(JSON.stringify(lockOf(root))).not.toContain(root); + }); + + test("a newer installed release is drift to check, and sync takes it", async () => { + const root = consumer({ npm: "@acme/common-model", dir: "metaobjects" }); + npmInstall(root, "1.0.0"); + expect(await sync(root)).toBe(0); + + npmInstall(root, "1.1.0", { artifact: WIDENED }); + out = []; + expect(await check(root)).toBe(1); + expect(out.join("\n")).toContain("drifted"); + + out = []; + expect(await sync(root)).toBe(0); + expect(out.join("\n")).toContain("synced acme-common 1.0.0→1.1.0"); + expect(lockOf(root).dependencies[DEP]!.version).toBe("1.1.0"); + out = []; + expect(await check(root)).toBe(0); + }); + + test("an installed version that differs from the manifest's is warned about", async () => { + const root = consumer({ npm: "@acme/common-model", dir: "metaobjects" }); + npmInstall(root, "1.2.0", { manifestVersion: "1.1.0" }); + expect(await sync(root)).toBe(0); + const printed = out.join("\n"); + expect(printed).toContain("is 1.2.0, but its shared-model manifest records 1.1.0"); + expect(lockOf(root).dependencies[DEP]!.version).toBe("1.1.0"); + }); + + test("an installed package with no manifest at dir names the publisher fix", async () => { + const root = consumer({ npm: "@acme/common-model", dir: "metaobjects" }); + const pkg = join(root, "node_modules", "@acme", "common-model"); + mkdirSync(join(pkg, "metaobjects"), { recursive: true }); + writeFileSync(join(pkg, "package.json"), JSON.stringify({ name: "@acme/common-model", version: "1.0.0" })); + expect(await sync(root)).toBe(1); + const printed = out.join("\n"); + expect(printed).toContain('package.json "files"'); + }); +}); + +/** A Python package laid out on an import path; optionally with dist-info so + * `importlib.metadata` can report its distribution version. */ +function pyInstall(site: string, version: string, opts: { distVersion?: string } = {}): void { + const pkg = join(site, "acme_common"); + mkdirSync(pkg, { recursive: true }); + writeFileSync(join(pkg, "__init__.py"), ""); + publish(join(pkg, "metaobjects"), version); + if (opts.distVersion !== undefined) { + const info = join(site, `acme_common-${opts.distVersion}.dist-info`); + mkdirSync(info, { recursive: true }); + writeFileSync(join(info, "METADATA"), `Metadata-Version: 2.1\nName: acme-common\nVersion: ${opts.distVersion}\n`); + writeFileSync(join(info, "top_level.txt"), "acme_common\n"); + } +} + +describe.skipIf(!hasPython)("meta deps sync — the python transport", () => { + test("syncs from the interpreter on PATH, prints it, and keeps it out of the lock", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + pyInstall(join(root, "site"), "1.0.0"); + process.env.PYTHONPATH = join(root, "site"); + + expect(await sync(root)).toBe(0); + expect(out.join("\n")).toMatch(/synced acme-common \(new\)→1\.0\.0 .* — python: .* \(from PATH\)/); + const lock = lockOf(root); + expect(lock.dependencies[DEP]!.resolvedFrom).toEqual({ python: "acme_common", dir: "metaobjects" }); + expect(JSON.stringify(lock)).not.toContain("python3"); + expect(JSON.stringify(lock)).not.toContain(root); + out = []; + expect(await check(root)).toBe(0); + }); + + test("reads the installed distribution's version and warns when the manifest disagrees", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + pyInstall(join(root, "site"), "1.0.0", { distVersion: "1.3.0" }); + process.env.PYTHONPATH = join(root, "site"); + const resolved = resolveTransport(root, { name: DEP, python: "acme_common", dir: "metaobjects" }); + expect(resolved.packageVersion).toBe("1.3.0"); + + expect(await sync(root)).toBe(0); + expect(out.join("\n")).toContain('Python package "acme_common" is 1.3.0, but its shared-model manifest records 1.0.0'); + }); + + test("not installed: refuses and names the pip command", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + expect(await sync(root)).toBe(1); + const printed = out.join("\n"); + expect(printed).toContain("pip install acme-common"); + }); + + test("a missing interpreter is refused by name", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + process.env[METAOBJECTS_PYTHON_ENV] = join(root, "no-such-python"); + expect(await sync(root)).toBe(1); + expect(out.join("\n")).toContain(`no Python interpreter at ${join(root, "no-such-python")}`); + }); + + test("the interpreter ladder: $METAOBJECTS_PYTHON, then $VIRTUAL_ENV, then ./.venv, then PATH", () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + const site = join(root, "site"); + pyInstall(site, "1.0.0"); + const real = spawnSync(PYTHON, ["-c", "import sys; print(sys.executable)"], { encoding: "utf8" }).stdout.trim(); + const fakeVenv = (dir: string): string => { + mkdirSync(join(dir, "bin"), { recursive: true }); + symlinkSync(real, join(dir, "bin", "python")); + return dir; + }; + fakeVenv(join(root, ".venv")); + const active = fakeVenv(join(root, "active-venv")); + const spec = { name: DEP, python: "acme_common", dir: "metaobjects" } as const; + const base = { ...process.env, PYTHONPATH: site }; + + expect(resolveTransport(root, spec, base).note).toContain("(from .venv)"); + expect(resolveTransport(root, spec, { ...base, VIRTUAL_ENV: active }).note).toContain("(from $VIRTUAL_ENV)"); + expect( + resolveTransport(root, spec, { ...base, VIRTUAL_ENV: active, [METAOBJECTS_PYTHON_ENV]: real }).note, + ).toContain(`(from $${METAOBJECTS_PYTHON_ENV})`); + rmSync(join(root, ".venv"), { recursive: true }); + expect(resolveTransport(root, spec, base).note).toContain("(from PATH)"); + }); + + test("a $VIRTUAL_ENV with no interpreter is refused, not skipped for a working ./.venv", () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + const site = join(root, "site"); + pyInstall(site, "1.0.0"); + const real = spawnSync(PYTHON, ["-c", "import sys; print(sys.executable)"], { encoding: "utf8" }).stdout.trim(); + mkdirSync(join(root, ".venv", "bin"), { recursive: true }); + symlinkSync(real, join(root, ".venv", "bin", "python")); + const empty = join(root, "empty-venv"); + mkdirSync(empty); + const spec = { name: DEP, python: "acme_common", dir: "metaobjects" } as const; + const base = { ...process.env, PYTHONPATH: site }; + expect(resolveTransport(root, spec, base).note).toContain("(from .venv)"); + + expect(() => resolveTransport(root, spec, { ...base, VIRTUAL_ENV: empty })).toThrow( + expect.objectContaining({ code: "ERR_DEPENDENCY_UNRESOLVED", message: expect.stringContaining("has no python interpreter") }), + ); + }); + + test("a dotted name whose parent package is not installed is 'not installed', with the pip hint", async () => { + const root = consumer({ python: "acme.model", dir: "metaobjects" }); + process.env.PYTHONPATH = join(root, "site"); + + expect(await sync(root)).toBe(1); + const printed = out.join("\n"); + expect(printed).toContain('the Python package "acme.model" is not installed'); + expect(printed).toContain("pip install acme"); + expect(printed).not.toContain("importing the parent package"); + }); + + test("a parent package that fails to import is reported as such, without a pip install hint", async () => { + const root = consumer({ python: "acme.model", dir: "metaobjects" }); + const site = join(root, "site"); + mkdirSync(join(site, "acme", "model"), { recursive: true }); + writeFileSync(join(site, "acme", "__init__.py"), "import acme_missing_optional_dependency\n"); + writeFileSync(join(site, "acme", "model", "__init__.py"), ""); + process.env.PYTHONPATH = site; + + expect(await sync(root)).toBe(1); + const printed = out.join("\n"); + expect(printed).toContain('importing the parent package of "acme.model"'); + expect(printed).toContain("ModuleNotFoundError"); + expect(printed).not.toContain("pip install"); + expect(printed).not.toContain("is not installed"); + }); + + test("a PEP 440-equivalent manifest version does not warn", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + pyInstall(join(root, "site"), "1.0.0-rc.1", { distVersion: "1.0.0rc1" }); + process.env.PYTHONPATH = join(root, "site"); + + expect(await sync(root)).toBe(0); + expect(out.join("\n")).not.toContain("shared-model manifest records"); + }); + + test("a package installed into a real virtualenv resolves through $VIRTUAL_ENV", async () => { + const root = consumer({ python: "acme_common", dir: "metaobjects" }); + const venv = join(root, "venv"); + const made = spawnSync(PYTHON, ["-m", "venv", "--without-pip", venv], { encoding: "utf8" }); + if (made.status !== 0) { + console.warn(`skipping: ${PYTHON} -m venv failed: ${made.stderr}`); + return; + } + const exe = process.platform === "win32" ? join(venv, "Scripts", "python.exe") : join(venv, "bin", "python"); + const purelib = spawnSync(exe, ["-c", "import sysconfig; print(sysconfig.get_paths()['purelib'])"], { + encoding: "utf8", + }).stdout.trim(); + pyInstall(purelib, "1.0.0", { distVersion: "1.0.0" }); + process.env.VIRTUAL_ENV = venv; + + expect(await sync(root)).toBe(0); + expect(out.join("\n")).toContain("(from $VIRTUAL_ENV)"); + expect(readFileSync(join(root, ".metaobjects", "deps", DEP, ARTIFACT))).toEqual(V1); + }); +}); + +// Oracle: PEP 440 "Normalization" (packaging.python.org, version specifiers) — +// each pair spells the same version, as `importlib.metadata` reports it. +describe("the installed-vs-manifest version warning", () => { + const py = { name: DEP, python: "acme_common" } as const; + const npm = { name: DEP, npm: "@acme/common-model" } as const; + + test.each([ + ["1.0.0rc1", "1.0.0-rc.1"], + ["1.0.0rc1", "1.0.0.RC1"], + ["1.0.0rc1.post2", "1.0.0-rc1-post2"], + ["1.0.0+abc", "1.0.0+abc"], + ["1.0.post1", "1.0-1"], + ["1.0.0+ubuntu.1", "1.0.0+Ubuntu-1"], + ["1!2.0", "1!2.0"], + ["2.0", "0!2.0"], + ["1.0a0", "1.0-alpha"], + ["1.0b2", "1.0.beta.2"], + ["1.0rc2", "1.0c2"], + ["1.0rc1", "1.0-preview1"], + ["1.0rc1", "1.0pre1"], + ["1.0.post1", "1.0-post1"], + ["1.0.post0", "1.0post"], + ["1.0.post2", "1.0-rev2"], + ["1.0.dev0", "1.0-dev"], + ["1.0.dev3", "1.0_dev3"], + ["1.0.0", "v1.0.0"], + ])("python: installed %s matches manifest %s", (installed, manifest) => { + expect(packageVersionWarning(DEP, py, installed, manifest)).toBeUndefined(); + }); + + test("python: a genuinely different version still warns", () => { + expect(packageVersionWarning(DEP, py, "1.0.0rc2", "1.0.0-rc.1")).toContain("records 1.0.0-rc.1"); + expect(packageVersionWarning(DEP, py, "1.0.0rc1", "1.0.0rc2")).toContain("records 1.0.0rc2"); + expect(packageVersionWarning(DEP, py, "1.0.0rc1.post2", "1.0.0rc1")).toContain("records 1.0.0rc1"); + expect(packageVersionWarning(DEP, py, "1.0", "1.0.0")).toContain("records 1.0.0"); + expect(packageVersionWarning(DEP, py, "1.1.0", "1.0.0")).toContain("is 1.1.0"); + }); + + test("npm: the comparison stays exact", () => { + expect(packageVersionWarning(DEP, npm, "1.0.0", "1.0.0")).toBeUndefined(); + expect(packageVersionWarning(DEP, npm, "1.0.0-rc.1", "1.0.0rc1")).toContain("is 1.0.0-rc.1"); + }); +}); diff --git a/server/typescript/packages/cli/test/deps-sync.test.ts b/server/typescript/packages/cli/test/deps-sync.test.ts index 412686731..1061a997d 100644 --- a/server/typescript/packages/cli/test/deps-sync.test.ts +++ b/server/typescript/packages/cli/test/deps-sync.test.ts @@ -274,7 +274,7 @@ describe("meta deps sync — path transport (FR-023 Phase 1a Task 14)", () => { expect(existsSync(join(consumerRoot, ".metaobjects", "deps.lock.json"))).toBe(false); }); - test("(g) an npm transport spec refuses — 'not supported by this toolchain yet'", async () => { + test("(g) an npm dependency that is not installed refuses and names the install command", async () => { const root = mkdtempSync(join(tmpdir(), "deps-sync-npm-")); dirs.push(root); mkdirSync(join(root, "metaobjects"), { recursive: true }); @@ -292,7 +292,8 @@ describe("meta deps sync — path transport (FR-023 Phase 1a Task 14)", () => { const exit = await run(["deps", "sync", "--format", "text", "--cwd", root]); expect(exit).toBe(1); - expect([...out, ...err].join("\n")).toContain("not supported by this toolchain yet"); + const printed = [...out, ...err].join("\n"); + expect(printed).toContain("npm install @acme/model"); }); test("(h) list after a sync prints one summary line per lock entry", async () => { diff --git a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts index e06616b76..5381963a8 100644 --- a/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/shared-model-file.ts @@ -32,8 +32,8 @@ export interface SharedModelFileOpts { /** Which of the publisher's own source files may contribute. Default: the run's * own source files (`ctx.sourceFiles` — everything `files?` narrows). */ files?: readonly string[]; - /** The published version. Default: the nearest `package.json` `version` walking - * up from the project (`ctx.inputRoot`). */ + /** The published version. Default: the nearest `package.json` `version` (or a + * `pyproject.toml` `[project]` version) walking up from the project (`ctx.inputRoot`). */ version?: string; /** Named output target, as on every other generator. */ target?: string; @@ -53,6 +53,11 @@ function nearestPackageVersion(startDir: string): string | undefined { // An unparsable package.json is not a version source — keep walking up. } } + const pyprojectPath = join(dir, "pyproject.toml"); + if (existsSync(pyprojectPath)) { + const version = pyprojectVersion(readFileSync(pyprojectPath, "utf-8")); + if (version !== undefined) return version; + } const parent = dirname(dir); if (parent === dir) return undefined; dir = parent; @@ -60,6 +65,22 @@ function nearestPackageVersion(startDir: string): string | undefined { return undefined; } +/** A static `version = "…"` in `pyproject.toml`'s `[project]` table, or undefined + * (no table, or a `dynamic` version a build backend computes). */ +function pyprojectVersion(toml: string): string | undefined { + let inProject = false; + for (const raw of toml.split(/\r?\n/)) { + const line = raw.trim(); + if (line.startsWith("[")) { + inProject = line.replace(/#.*$/, "").trim() === "[project]"; + continue; + } + const m = inProject ? /^version\s*=\s*(?:"([^"]*)"|'([^']*)')/.exec(line) : null; + if (m) return m[1] ?? m[2]; + } + return undefined; +} + export function sharedModelFile(opts: SharedModelFileOpts): Generator { const generator: Generator = { name: "shared-model", @@ -86,7 +107,7 @@ export function sharedModelFile(opts: SharedModelFileOpts): Generator { if (version === undefined) { throw new Error( `sharedModelFile("${opts.name}"): no version available — pass \`version\` explicitly, ` + - `or run inside a project with a package.json version at or above the project root.`, + `or run inside a project with a package.json or pyproject.toml version at or above the project root.`, ); } diff --git a/server/typescript/packages/codegen-ts/src/reference/shared-model.ts b/server/typescript/packages/codegen-ts/src/reference/shared-model.ts index 134edec45..a81388c4f 100644 --- a/server/typescript/packages/codegen-ts/src/reference/shared-model.ts +++ b/server/typescript/packages/codegen-ts/src/reference/shared-model.ts @@ -56,15 +56,16 @@ export interface SharedModelFileOpts { /** Which of the publisher's own source files may contribute. Default: the run's * own source files (`ctx.sourceFiles`). */ files?: readonly string[]; - /** The published version. Default: the nearest `package.json` `version` walking - * up from the project (`ctx.inputRoot`). */ + /** The published version. Default: the nearest `package.json` `version` (or a + * `pyproject.toml` `[project]` version) walking up from the project (`ctx.inputRoot`). */ version?: string; /** Named output target, as on every other generator. */ target?: string; } /** OWNED: where the published version comes from when the `version` option is absent. - * Walks up from `startDir` for the nearest `package.json` carrying a string `version`. */ + * Walks up from `startDir` for the nearest `package.json` carrying a string `version`, + * or a `pyproject.toml` with a static `[project]` version (a Python package). */ function nearestPackageVersion(startDir: string): string | undefined { let dir = startDir; for (let i = 0; i < 64; i++) { @@ -77,6 +78,11 @@ function nearestPackageVersion(startDir: string): string | undefined { // An unparsable package.json is not a version source — keep walking up. } } + const pyprojectPath = join(dir, "pyproject.toml"); + if (existsSync(pyprojectPath)) { + const version = pyprojectVersion(readFileSync(pyprojectPath, "utf-8")); + if (version !== undefined) return version; + } const parent = dirname(dir); if (parent === dir) return undefined; dir = parent; @@ -84,6 +90,22 @@ function nearestPackageVersion(startDir: string): string | undefined { return undefined; } +/** A static `version = "…"` in `pyproject.toml`'s `[project]` table, or undefined + * (no table, or a `dynamic` version a build backend computes). */ +function pyprojectVersion(toml: string): string | undefined { + let inProject = false; + for (const raw of toml.split(/\r?\n/)) { + const line = raw.trim(); + if (line.startsWith("[")) { + inProject = line.replace(/#.*$/, "").trim() === "[project]"; + continue; + } + const m = inProject ? /^version\s*=\s*(?:"([^"]*)"|'([^']*)')/.exec(line) : null; + if (m) return m[1] ?? m[2]; + } + return undefined; +} + export function sharedModelFile(opts: SharedModelFileOpts): Generator { const generator: Generator = { name: "shared-model", @@ -110,7 +132,7 @@ export function sharedModelFile(opts: SharedModelFileOpts): Generator { const version = opts.version ?? nearestPackageVersion(ctx.inputRoot ?? ctx.projectRoot ?? process.cwd()); if (version === undefined) { throw new Error( - `sharedModelFile("${opts.name}"): no version available — pass \`version\` explicitly, or run inside a project with a package.json version at or above the project root.`, + `sharedModelFile("${opts.name}"): no version available — pass \`version\` explicitly, or run inside a project with a package.json or pyproject.toml version at or above the project root.`, ); } diff --git a/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts b/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts index bd5b4e1d2..438403a26 100644 --- a/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts +++ b/server/typescript/packages/codegen-ts/test/shared-model-file.test.ts @@ -216,6 +216,23 @@ describe("sharedModelFile()", () => { }); }); + test("version defaults to a Python package's pyproject.toml [project] version", async () => { + writeFileSync( + join(tmp, "pyproject.toml"), + '[build-system]\nrequires = ["hatchling"]\nversion = "0.0.0"\n\n[project] # the distribution\n' + + 'name = "acme-common"\nversion = "2.3.4"\n\n[tool.hatch]\nversion = "9.9.9"\n', + ); + await gen(sharedModelFile({ name: "acme-common", include: ["acme::common::**"], exclude: ["acme::common::Secret"] }), tmp); + expect(JSON.parse(readFileSync(join(tmp, "metaobjects.pkg.json"), "utf-8")).version).toBe("2.3.4"); + }); + + test("a package.json version in the same directory wins over pyproject.toml", async () => { + writeFileSync(join(tmp, "pyproject.toml"), '[project]\nname = "acme-common"\nversion = "2.3.4"\n'); + writeFileSync(join(tmp, "package.json"), JSON.stringify({ name: "acme-common", version: "1.5.0" })); + await gen(sharedModelFile({ name: "acme-common", include: ["acme::common::**"], exclude: ["acme::common::Secret"] }), tmp); + expect(JSON.parse(readFileSync(join(tmp, "metaobjects.pkg.json"), "utf-8")).version).toBe("1.5.0"); + }); + test("(c) determinism — two runs produce identical bytes", async () => { const opts = () => sharedModelFile({