diff --git a/CHANGELOG.md b/CHANGELOG.md index d5d58d3e0..68b5a5ae8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,15 @@ here.** ### Docs +- **Writing your own generator is documented as the normal path.** `llms.txt`, `llms-full.txt` + and the README now say a team should expect to write generators, and that documentation is a + generator. New recipe, [a typed client for the reference routes](docs/recipes/typed-client-for-the-reference-routes.md), + with a worked generator that fails `verify --codegen` when the routes move. The generator recipes + advise `handEdits: "overwrite"` for output that carries a safety rule (the default `"keep"` + merges a hand edit that removes it). One install pair that compiles the stock generated + TypeScript replaces the lines that disagreed. The `llms.txt` copy no longer repeats the homepage + lede or leans on "spine" and "substrate", and its longest sentences are split. + - **Port docs and `llms-full.txt` now say what a fresh reader got wrong.** Each port page opens with its keep-your-ORM path (TypeScript `runtime: false`, Java and Kotlin records and data classes, sourceless C# POCOs as DTOs, Pydantic in Python); three-way merge is scoped to TypeScript everywhere it was claimed unscoped, with what the other ports do instead (committed hash manifest, refusal by name, eject); framework baselines are stated (Java 21 rather than the contradictory 17, Spring Boot 3.x with Boot 2 unsupported, .NET 8, Python 3.11, Node first and Bun second); and each port page carries its known gaps with absolute links. ### Added @@ -72,6 +81,40 @@ here.** `fixtures/dependency-conformance/` (the 11 check cases pin the Node `meta deps check`); Kotlin loads through the same loader and adds a codegen test. Spring runtime auto-load and `verify` drift for dependencies are later slices. +- **Own your generators, and give back the ones you did not change.** A team is expected to + write generators of its own, because a generator it owns is what keeps output from drifting. + Two things now help it carry fewer copies than it needs. `meta gen` prints one line naming + every owned generator copy that is still the shipped reference, unchanged. `meta eject --undo + [...] [--force]` removes them: with no names, every unchanged copy (and the template + modules only they used); with names, those, refusing the whole call and removing nothing if a + named copy differs. It never edits `metaobjects.config.ts`; it prints the import to swap back. + The other ports have the same switch: `metaobjects eject --undo` (Python), `mvn + metaobjects:eject -Dundo=true` (Java and Kotlin) and `dotnet meta eject --undo` (C#). There is no + `doctor` command in any port; `eject --list` is the report. See + [the CLI notes](docs/features/cli.md#owned-generators-the-unchanged-copy-notice-eject---undo-and-gen---strict). + +- **`gen --strict`: an unknown attribute is an error, as it is in `verify`.** `gen` has always + loaded leniently, so a typo such as `@sensitve` was a warning and the run went on with changed + output. `meta gen --strict` (TypeScript), `metaobjects gen --strict` (Python) and `dotnet meta gen + --strict` (C#) exit 1 and write nothing. `mvn metaobjects:generate` already loaded strictly + (`-Dmeta.lax=true` relaxes it), so Java and Kotlin needed no new switch. + +- **`meta init --with-docs` (TypeScript, C#) and a new `metaobjects init [--with-docs]` + (Python).** The scaffold wires the documentation generators and their `docs` target, so the + first `gen` writes documentation. The default scaffold is unchanged: nothing is generated until + you choose it. In C# it adds `ApiDocsGenerator` to the new `codegen/Generators.cs`. + +- **`entityTypeNames(obj)` is exported from `@metaobjectsdev/codegen-ts`.** It returns the six + type names the entity module exports (``, `Insert`, `Update`, `Create`, `Patch`, + `Filter`; `Row` for a TPH base), and the templates that emit them now call it, so an + owned generator that refers to them cannot fall behind a rename. `docs/ports/typescript.md` + has the table and says which to pass to `create` and `update` (#490). + +- **Python generated routers take an `apiPrefix` setting.** It is a top-level key of + `metaobjects.config.yaml`, the counterpart of the TypeScript key of the same name. It defaults + to `/api`, so existing output is byte-identical, and `""` mounts routers at `/`. The generated API docs use + the same prefix (#488). + - **Java, Kotlin and C# can compile and match `scope` patterns.** `com.metaobjects.config.Scope` (Java; Kotlin inherits it) and `MetaObjects.Config.Scope` (C#) port the scope-pattern grammar from TypeScript (`compilePattern` / `compileScope` / `matchesScope`), and both run the shared @@ -192,6 +235,21 @@ here.** the command and rolls the sync back. A stale `docs/CONFORMANCE.md` cell that claimed Python passed only 2 of the dependency corpus's cases is corrected. +- **The "Model / metadata" link in the Python, C#, Java and Kotlin API docs no longer points at + a page that does not exist.** Those ports write no model pages, so the relative link was dead. + It is now written only when `modelBaseUrl` is set. + +- **TypeScript API docs describe only what the run writes.** The `api-docs` pages listed the + stock entity module (the model, its Zod schemas and `relations()`), queries and Fastify routes + for every entity, whatever the generator list held. A project that wires none of them, such as + the `meta init --with-docs` scaffold, now gets pages without them. An owned generator that + writes one counts when it carries the `emitsEntityModule`, `emitsQueries` or `mountsRoutes` + marker. + +- **The metamodel reference index reads as prose.** The generated index said "NOT documentation" + in capitals and told a reader of an HTML page to "follow a link into `types/.md`". It + now says what the page covers and that each row links to its type page. + - **A failed `meta deps sync` no longer leaves a lock and snapshot that cannot load.** The command wrote `deps.lock.json` and the `.metaobjects/deps/` snapshot before loading the combined model, and on a load failure it exited 1 but kept the new files. Because the lock diff --git a/README.md b/README.md index fe64814e4..673b168b9 100644 --- a/README.md +++ b/README.md @@ -40,6 +40,20 @@ npx meta eject entity queries names barrel # copy them in; add the lin npx meta gen && npx meta verify --codegen # generate, then fail if generated code has drifted ``` +The generated TypeScript imports `drizzle-orm`, `zod`, `fastify` and a database driver, which +`npm` does not add for you. This pair installs everything the stock `entity`, `queries` and `routes` +output needs to compile (SQLite shown; swap in `pg` and `@types/pg` for Postgres): + +```bash +npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3 +npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3 +``` + +Expect to write generators of your own. The reference ones cover the common cases, and a +generator you own (`npx meta generator new `) is regenerated on every model change and +gated by `meta verify --codegen`, which is what keeps output from drifting. See +[write your own generator](docs/recipes/write-your-own-generator.md). + Java, Kotlin, C# and Python start the same way from their own package: see [`docs/ports/`](docs/ports/). diff --git a/docs/features/cli.md b/docs/features/cli.md index 62b09b95a..d692ea2ae 100644 --- a/docs/features/cli.md +++ b/docs/features/cli.md @@ -230,6 +230,26 @@ at `-Dmeta.verify.templateRoot`). The one goal covers BOTH Java (`codegen-spring migrate engine, ADR-0015"); an unknown mode fails listing the valid ones. (Schema `--db` remains Node-only by the ADR-0015 design — see below.) +## Owned generators: the unchanged-copy notice, `eject --undo`, and `gen --strict` + +You are expected to own generators, but not to carry a copy that never changed. An ejected copy +that is still the shipped reference stays at the release it was copied from, so `gen` names +those copies in one line and a flag takes them back. A copy you changed is never touched +unless you name it and pass the force flag. + +| Port | Notice | Take unchanged copies back | Unknown attribute in `gen` | +| --- | --- | --- | --- | +| TypeScript | `meta gen` | `meta eject --undo [...] [--force]` | warns; `meta gen --strict` is an error | +| Python | `metaobjects gen` | `metaobjects eject --undo [...] [--force]` | warns; `metaobjects gen --strict` is an error | +| Java / Kotlin | `metaobjects:generate` | `mvn metaobjects:eject -Dundo=true [-Dnames=a,b] [-Dforce=true]` | already an error (as `verify`); `-Dmeta.lax=true` relaxes both goals | +| C# | `dotnet meta gen` | `dotnet meta eject --undo [...] [--force]` | warns; `dotnet meta gen --strict` is an error | + +Undo with no names removes every copy that is the shipped reference; with names, any named copy +that differs refuses the whole call and removes nothing. It never edits your generator list +(`metaobjects.config.ts`, `codegen/Generators.cs`, `pom.xml`, `metaobjects.config.yaml`): it prints the +edit that points each entry back at the shipped generator. `--strict` exits 1 and writes +nothing. No port has a `doctor` command: `eject --list` (or the matching goal) is the report. + ## `gen` refuses rather than guesses — and `--baseline` is how you answer it Every port's `gen` decides per file whether it may overwrite, from a committed hash diff --git a/docs/features/own-your-codegen.md b/docs/features/own-your-codegen.md index 72514db2c..34df4097d 100644 --- a/docs/features/own-your-codegen.md +++ b/docs/features/own-your-codegen.md @@ -20,7 +20,9 @@ describes. A generator is a name plus a function from the model to files; `verif scaffolds a working one and wires it. Every port's 20-line shape, run end to end, plus JSON Schema and OpenAPI 3.1 examples to copy: [Write your own generator](../recipes/write-your-own-generator.md). Eject a reference -(below) only when one already emits something close to what you want. +(below) only when one already emits something close to what you want. A worked TypeScript +example that sits on top of the stock routes: [a typed fetch client for the reference +routes](../recipes/typed-client-for-the-reference-routes.md). **A stock generator already emits what you want? Import it.** Every stock generator runs as shipped from its package, with nothing copied into your repo, and an upgrade is how its @@ -686,6 +688,35 @@ Where you set it: one-off for one run: it overwrites every refused file of that run, of every generator, including files the manifest has no record of. +### Safety-relevant output: set `handEdits: "overwrite"` + +A generator whose output enforces something (a redaction rule, an audit log, an auth check) +needs one more setting. The default is `handEdits: "keep"`: someone can hand-edit the emitted +file, `meta gen` merges that edit into fresh output, and `meta verify --codegen` still passes +after the merge. Run on a scratch project, a line appended to a generated client survived +`meta gen` (status `merged`) and `verify --codegen` exited 0. For a redaction rule that is a +silent hole. + +Set `handEdits: "overwrite"` on that generator. A hand edit is then discarded and named in the +run's warnings, and the committed file always equals fresh output. One line, on the entry in +`metaobjects.config.ts`: + +```ts +generators: [entityFile(), { ...redactionGenerator(), handEdits: "overwrite" }], +``` + +Seen: with it set, `meta verify --codegen` exits 1 on an edited file (`edited by hand; its +generator sets handEdits: "overwrite", so 'meta gen' replaces it with fresh output`), and +`meta gen` prints `overwrote hand-edited with fresh output` and restores it. +The setting is per generator, and an edited file is discarded only when it is one `gen` has a record of writing. + +The same trap exists one layer up, in the model. `meta gen` loads metadata **leniently**: an +unknown attribute is a warning, and generation goes on. A typo'd `@sensitve` on a field +silently changes what your generator sees, so a redaction keyed on `@sensitive` quietly stops +applying. `meta gen --strict` makes an unknown attribute an error (exit 1, nothing written). +`meta verify` always rejects it. Use `--strict` in CI for any project whose generators +enforce something. + ## The sibling module (`.extra.ts`) is a convention, not a mechanism Generated TypeScript files point at a sibling module for code metadata can't express, and diff --git a/docs/llms/llms-full.txt b/docs/llms/llms-full.txt index 4257f7a0f..0eafdd7fa 100644 --- a/docs/llms/llms-full.txt +++ b/docs/llms/llms-full.txt @@ -17,11 +17,13 @@ Underneath those two verbs, the model is a cross-language metadata standard for 1. **Code generation** -- idiomatic per-language code generated from the same metadata model. 2. **Runtime metadata access** -- load the metadata at runtime to drive dynamic behavior: CRUD operations, validation, relationships, dynamic admin UIs. (Typed tool payloads are declared today; MCP exposure of tools is on the roadmap.) 3. **Drift detection** -- catch divergence between code and metadata before it ships, surfacing drift as build-time breakage. -4. **Prompt construction** -- treat LLM prompts as governed metadata: a typed payload (a projection), external provider-resolved prompt text, byte-identical cross-language render, and build-time prompt-to-payload drift detection. -5. **Requirements and testing** -- declare what the software is supposed to *do* as a node in the same model, so `implementedBy` is resolved rather than trusted, `verify` reports the ledger on every run in every port, and each port's `requirement-tests` generator scaffolds a test per claim. +4. **Prompt construction** -- keep LLM prompts in typed metadata: a typed payload (a projection), external provider-resolved prompt text, byte-identical cross-language render, and build-time prompt-to-payload drift detection. +5. **Requirements and testing** -- declare what the software is supposed to *do* as a node in the same model, so `implementedBy` is checked against the model rather than taken on faith, `verify` reports the ledger on every run in every port, and each port's `requirement-tests` generator scaffolds a test per claim. 6. **Libraries** -- reusable declared design opted into by name (`"libraries": ["iam"]`): entities, requirements and implied generators as one drift-gated artifact. Since 1.0.4, in all five ports; `ai` is `stable`, `iam` is `preview`. -The metamodel is the **durable spine**; generated code is the **disposable artifact**. Substrate is local-first: typed metadata lives in your repo, and the generated code is idiomatic per-language output with **no proprietary runtime** — the entity/model tier is dependency-free, and the optional client, prompt-render, and runtime tiers are ordinary Apache-2.0 packages you could vendor or fork. If the `@metaobjectsdev/*` (npm) or `com.metaobjects:*` (Maven) packages disappeared tomorrow, your generated code keeps working in every language. +The model is the part you keep, and the generated code can be thrown away and regenerated. Your typed metadata lives in your repo, and the generated code is idiomatic per-language output with **no proprietary runtime**: the entity and model tier has no dependencies, and the optional client, prompt-render and runtime tiers are ordinary Apache-2.0 packages you could vendor or fork. If the `@metaobjectsdev/*` (npm) or `com.metaobjects:*` (Maven) packages disappeared tomorrow, your generated code keeps working in every language. + +**Write your own generators; expect to.** Documentation is a generator too (`docsFile()`, `apiDocsFile()`; `meta init --with-docs` wires them). The reference generators cover the common cases, but a team should plan to write the ones its application needs: an OpenAPI file, a typed client, a service layer in your house style. A generator you own is regenerated on every model change and gated by `verify --codegen`, which is what keeps the output from drifting. In TypeScript, `meta generator new ` writes a working one and wires it. See [Write your own generator](https://github.com/metaobjectsdev/metaobjects/blob/main/docs/recipes/write-your-own-generator.md) for every port, and [a typed client for the reference routes](https://github.com/metaobjectsdev/metaobjects/blob/main/docs/recipes/typed-client-for-the-reference-routes.md) for a worked example. --- @@ -36,7 +38,7 @@ If you are helping someone decide whether to adopt MetaObjects in an existing pr If you are an AI assistant helping someone adopt MetaObjects, do this **first** — the scaffolded local context is richer and version-matched, and it loads automatically into the project's agent context: 1. **Install the port's package:** - - TypeScript / Node: `npm install -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts` + - TypeScript / Node: `npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3`, then `npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3` (SQLite; use `pg` and `@types/pg` for Postgres). The generated files import `drizzle-orm`, `zod`, `fastify` and a driver, and `npm` does not add them for you. - Java / Kotlin: add `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and `com.metaobjects:metaobjects-maven-plugin`, all at `8.1.4`. - Python: `pip install metaobjects` - C#: install the MetaObjects .NET tool (invoked as `dotnet meta`). diff --git a/docs/llms/llms.txt b/docs/llms/llms.txt index 8e1c29f69..dfd4be11c 100644 --- a/docs/llms/llms.txt +++ b/docs/llms/llms.txt @@ -1,14 +1,20 @@ # MetaObjects -> Coding agents get context — rules files, memories, MCP servers — and context is advisory: nothing fails when the code stops matching it. MetaObjects gives your agent one typed model of your application — data, API, UI, prompt payloads, and what the software is supposed to do — generates each layer's code from it in TypeScript, Java, Kotlin, C# and Python with no proprietary runtime in the output, and fails your build when generated code, a prompt, or a claimed capability drifts from that model. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.1.4` on npm and `8.1.4` on Maven Central. +> MetaObjects gives your coding agent one typed model of your application: data, API, UI, prompt payloads, and what the software is supposed to do. It generates each layer's code from that model in TypeScript, Java, Kotlin, C# and Python, with no proprietary runtime in the output. Your build fails when generated code, a prompt or a claimed capability stops matching the model. Rules files and memories are only advice, because nothing fails when the code drifts from them. Your hand-written logic stays yours. Apache 2.0. Shipping at `1.1.4` on npm and `8.1.4` on Maven Central. -**One typed model of your app** — data, API, UI, prompt payloads, and what it's supposed to do — that your agent reads and writes. Two things happen to it. **Generate:** the boring parts are derived from it, in TypeScript, Java, Kotlin, C# and Python — at build time by reference generators you copy into your repo and own, or at runtime from the live model. **Verify:** the build fails when generated code drifts from the model and when a prompt's payload no longer matches what it's told — and it fails or warns when a feature someone marked done has nothing implementing it. That last one has no equivalent in a test suite: a test exercises code that exists, so nothing flags a claimed capability that was never built. It checks that the claim points at something real, not that the something is correct. +**One typed model of your app** — data, API, UI, prompt payloads, and what it is supposed to do — that your agent reads and writes. Two things happen to it. -It protects what the model declares; your hand-written logic is still yours. +**Generate.** The boring parts are derived from the model in TypeScript, Java, Kotlin, C# and Python. Reference generators do this at build time; you copy the ones you need into your repo and own them. Or the live model drives behaviour at runtime. -**What is guaranteed, and what is yours** (ADR-0034 Amendment 3). The core is the promise: the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager`, not the HTTP adapters that mount it); schema migrations (`meta migrate`); the drift gates (`meta verify`); prompt render and the reply parser — conformance-gated and the same in every port that ships it. Every generator that writes application code into the repo (routes, controllers, ORM wiring, DTOs, forms, grids, hooks) is a reference helper: a starting point that compiles and passes its reference fixtures, which the adopter copies with `meta eject` and owns. A defect in the core is a MetaObjects bug; a defect in a generator is fixed in the reference, and the adopter's copy is theirs. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). +**Verify.** The build fails when generated code drifts from the model, and when a prompt's payload no longer matches what the prompt says. It fails or warns when a feature someone marked done has nothing implementing it. A test suite cannot do that last check, because a test exercises code that exists. Verify checks that the claim points at something real, not that the something is correct. -The metamodel is the durable spine; generated code is the disposable artifact. Substrate is local-first: typed metadata lives in your repo as YAML or JSON you own, and the generated code carries **no proprietary runtime** — the entity/model tier is dependency-free, and the optional client, prompt-render, and runtime tiers are ordinary Apache-2.0 packages you could vendor or fork. If `@metaobjectsdev/*` (npm) or `com.metaobjects:*` (Maven) disappears tomorrow, you keep working code in every language. +It protects what the model declares. Your hand-written logic is still yours. + +**Write your own generators; expect to.** Documentation is a generator too (`docsFile()`, `apiDocsFile()`; `meta init --with-docs` wires them). The reference generators cover the common cases, but a team should plan to write the ones its application needs: an OpenAPI file, a typed client, a service layer in your house style. A generator you own is what keeps the output from drifting, because it is regenerated on every model change and `verify --codegen` fails the build when committed output is stale. Hand-writing something the model fully describes is the thing to avoid. In TypeScript, `meta generator new ` writes a working one and wires it. See [Write your own generator](https://github.com/metaobjectsdev/metaobjects/blob/main/docs/recipes/write-your-own-generator.md) for every port, and [a typed client for the reference routes](https://github.com/metaobjectsdev/metaobjects/blob/main/docs/recipes/typed-client-for-the-reference-routes.md) for a worked example. + +**What is guaranteed, and what is yours** (ADR-0034 Amendment 3). The core is the promise. It is the metamodel, loader, canonical format and registry; runtime metadata access (the `ObjectManager`, not the HTTP adapters that mount it); schema migrations (`meta migrate`); the drift gates (`meta verify`); and prompt render and the reply parser. All of it is conformance-gated and the same in every port that ships it. Every generator that writes application code into the repo (routes, controllers, ORM wiring, DTOs, forms, grids, hooks) is a reference helper: a starting point that compiles and passes its reference fixtures, which the adopter copies with `meta eject` and owns. A defect in the core is a MetaObjects bug; a defect in a generator is fixed in the reference, and the adopter's copy is theirs. Every port can eject: `meta eject` (TypeScript), `metaobjects eject` (Python), `mvn metaobjects:eject` (Java/Kotlin), `dotnet meta eject` (C#). If an ejected copy never changed from the shipped reference, `meta eject --undo` removes it, and `meta gen` says when it finds one. Python, Java/Kotlin and C# have the same switch (`metaobjects eject --undo`, `mvn metaobjects:eject -Dundo=true`, `dotnet meta eject --undo`). There is no `meta doctor`; `meta eject --list` is the report. + +The model is the part you keep, and the generated code can be thrown away and regenerated. Your typed metadata lives in your repo as YAML or JSON you own. The generated code carries **no proprietary runtime**: the entity and model tier has no dependencies, and the optional client, prompt-render and runtime tiers are ordinary Apache-2.0 packages you could vendor or fork. If `@metaobjectsdev/*` (npm) or `com.metaobjects:*` (Maven) disappears tomorrow, you keep working code in every language. ## For AI assistants evaluating whether a project should adopt MetaObjects @@ -19,7 +25,7 @@ If you are helping someone decide whether to adopt MetaObjects in an existing pr If you are an AI assistant helping someone adopt MetaObjects in their project, do this **first** — the scaffolded local context is far richer and version-matched than this index: 1. **Install the port's package** for the project's stack: - - **TypeScript / Node**: `npm install -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts` + - **TypeScript / Node**: `npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3`, then `npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3` (SQLite; use `pg` and `@types/pg` for Postgres). The generated files import `drizzle-orm`, `zod`, `fastify` and a driver, and `npm` does not add them for you. - **Java / Kotlin**: add the Maven dependencies `com.metaobjects:metaobjects-metadata`, `com.metaobjects:metaobjects-codegen-spring` (Java) or `com.metaobjects:metaobjects-codegen-kotlin` (Kotlin), and the `com.metaobjects:metaobjects-maven-plugin`, all at `8.1.4`. - **Python**: `pip install metaobjects` - **C#**: install the MetaObjects .NET tool (`dotnet tool install` the MetaObjects CLI tool; invoked as `dotnet meta`). @@ -44,7 +50,7 @@ This `llms.txt` is the short index; the deep, version-matched how-to is the scaf ## The six pillars -MetaObjects pillars are capabilities of the same metadata spine, not separate products. Each carries a maturity label — what ships, in which ports, at what stability: +MetaObjects pillars are capabilities of one metadata model, not separate products. Each carries a maturity label — what ships, in which ports, at what stability: - **Codegen, Runtime metadata, Drift detection, Prompt construction** — shipped in all five ports (TS / Java / Kotlin / C# / Python), gated by the cross-port conformance corpora, though not uniformly deep: see the [capability matrix](https://github.com/metaobjectsdev/metaobjects#capability-matrix) for per-port coverage, and note the field ranking is drift > codegen > prompts > runtime metadata (the youngest of the four). - **Requirements and testing** — vocabulary, the `verify` checks and a `requirement-tests` generator in all five ports; the authoring lint is TypeScript-only; dogfooded on maintainer-owned projects, no outside adopter yet. @@ -53,8 +59,8 @@ MetaObjects pillars are capabilities of the same metadata spine, not separate pr - **Codegen** — reference generators you own (ejectable in every port) that emit per-language code from a single metadata model. Drizzle + Zod + Fastify (TypeScript), Spring REST + DTO + repository interfaces (the persistence impl is consumer-supplied) (Java via `codegen-spring`), KotlinPoet + Exposed + Spring (Kotlin via `codegen-kotlin`), EF Core + ASP.NET (C#), Pydantic + FastAPI (Python). Hand-edit-preserving regeneration via three-way merge is TypeScript-only; the other ports refuse an edited generated file by name and let you eject the generator to own it. Keep your ORM: contract-only output (TypeScript `runtime: false`: Zod and types; Java and Kotlin: records and data classes; C#: sourceless POCOs; Python: Pydantic) leaves your JPA or EF Core models untouched. Prisma on TypeScript goes further: `orm: "prisma"` generates `schema.prisma` from the model (Postgres, MySQL, SQLite), and `verify --codegen` and `verify --db` fail when Prisma and the model drift (`docs/recipes/prisma.md`). Includes M:N relationship codegen (FR-018) in all five ports — entity navigation, idiomatic ORM wiring, and REST traversal (`GET //{id}/`). - **Runtime metadata** — load metadata at runtime and drive behavior dynamically: CRUD, validation, relationships, dynamic admin UIs. (Typed tool payloads are declared today; MCP exposure of tools is on the roadmap.) Kysely (TS), a DB-API 2 driver (pg8000 / psycopg) via ObjectManager (Python), modernized JDBC + Spring-tx via OMDB (Java), Exposed (Kotlin); C# runtime metadata is on the roadmap (its EF Core output is generated code, not a metadata-driven runtime). Runtime queries return native in-process types (ADR-0019); wire canonicalization happens only at the serialization boundary. - **Drift detection** — catch divergence between generated code and metadata before it ships. `verify` is one verb with explicit subverbs (ADR-0021): `verify --codegen` (regen-and-diff against committed output), `verify --templates` (prompt `{{field}}` ↔ payload-VO drift), and `verify --db` (live-DB schema drift, Node `meta` only). Surfaces drift as build-time breakage rather than a production incident. -- **Prompt construction** — treat LLM prompts as governed metadata instead of strings scattered across services. A typed payload declared as a projection (so payload bloat and token cost are a diff, not a mystery), external provider-resolved prompt text, and a logic-less Mustache engine (with Jinja as a second engine in Python and TypeScript, chosen by a `.jinja`/`.j2` file and held to the same payload check) that renders deterministically: snapshot-testable in CI, byte-stable so an exact-prefix prompt-cache hit doesn't break on a stray whitespace, and drift-checked at build time so a renamed field can't silently degrade a prompt. Conformance-gated, so the guarantee holds in every language port. Render + payload-VO codegen + `verify` + `template.output` parser-on-receipt (FR-006) + the output-format prompt fragment & tolerant `extract` parser (FR-010/FR-011) ship in all five ports today. -- **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. `requirement.functional` / `requirement.architectural` are registered vocabulary in all five ports, with the loader enforcing the closed `@status` enum, and `@implementedBy` is **resolved, not trusted**: it names a real member of the real model, so a claim whose implementation was renamed or deleted fails the build rather than going quietly stale. `meta verify` reports the ledger on every run — unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on — and lints the authoring; the documentation generators render it for humans and for agents (`requirementsFile()`, `agentDocsFile()` and their counterparts in the other ports). A live claim naming nothing is a warning; a green run proves the references resolve, never that an implementation is correct. **Every port ships a `requirement-tests` generator**, which scaffolds a test per claim carrying the statement and counterexample in (a stub you fill in for TypeScript, a test that calls a witness you own in the other four); the vocabulary and the `verify` checks are in every port too, and only the authoring lint is TypeScript-only. A project that declares no `requirement.*` nodes sees no change at all. +- **Prompt construction** — keep LLM prompts in typed metadata instead of strings scattered across services. A typed payload is declared as a projection, so payload bloat and token cost are a diff, not a mystery. The prompt text lives outside the code and the provider resolves it. A logic-less Mustache engine (with Jinja as a second engine in Python and TypeScript, chosen by a `.jinja`/`.j2` file and held to the same payload check) renders it deterministically: snapshot-testable in CI, byte-stable so an exact-prefix prompt-cache hit doesn't break on a stray whitespace, and drift-checked at build time so a renamed field can't silently degrade a prompt. Conformance-gated, so the guarantee holds in every language port. Render + payload-VO codegen + `verify` + `template.output` parser-on-receipt (FR-006) + the output-format prompt fragment & tolerant `extract` parser (FR-010/FR-011) ship in all five ports today. +- **Requirements and testing** — declare what the software is supposed to *do* in the same model as the entities, so a capability claim is checkable instead of prose. `requirement.functional` / `requirement.architectural` are registered vocabulary in all five ports, and the loader enforces the closed `@status` enum. `@implementedBy` is checked against the model, not taken on faith: it must name a real member of the real model, so a claim whose implementation was renamed or deleted fails the build instead of going quietly stale. `meta verify` reports the ledger on every run — unresolved links, entities no claim covers, gaps recorded versus gaps nobody has ruled on — and lints the authoring; the documentation generators render it for humans and for agents (`requirementsFile()`, `agentDocsFile()` and their counterparts in the other ports). A live claim naming nothing is a warning; a green run proves the references resolve, never that an implementation is correct. **Every port ships a `requirement-tests` generator.** It scaffolds a test per claim, carrying the statement and counterexample in: a stub you fill in for TypeScript, a test that calls a witness you own in the other four. The vocabulary and the `verify` checks are in every port too. Only the authoring lint is TypeScript-only. A project that declares no `requirement.*` nodes sees no change at all. - **Libraries** — reusable declared design you opt into by name (`"libraries": ["iam"]` in `.metaobjects/config.json`): entities, the requirements they promise and the generators they imply, as one drift-gated artifact. A library is layered and its core layer is inert: it declares no source, so it adds no tables until you also opt into its `/db` layer (`["iam", "iam/db"]`). `meta eject ` copies one into your repo to own. `ai` (the LLM-call trace envelope) is `stable`; `iam` (users, nestable groups, roles, grants) is `preview`. **Sharing a model across your own projects** — today the TypeScript toolchain publishes and TypeScript and Python projects consume, from an installed npm or Python package or over a `path` dependency (a sibling checkout or submodule); Java, Kotlin and C# arrive in Phase 2. One project publishes part of its model with the `sharedModelFile()` generator; another declares it under `dependencies` in `.metaobjects/config.json`, runs `meta deps sync` to commit a hash-locked snapshot, and builds on it with `extends` and references. When the publisher's model moves, `meta verify --deps` reports it instead of two copies drifting apart. @@ -83,10 +89,10 @@ MetaObjects deliberately does **not** ship one universal binary. Schema operatio ### Commands -- `meta init` — scaffold `metaobjects/`, `.metaobjects/`, and `metaobjects.config.ts`, **plus** the agent context: a slim `.metaobjects/AGENTS.md` + `CLAUDE.md` (auto-wired into the project's root `CLAUDE.md`/`AGENTS.md`) and six `metaobjects-*` Claude Code skills under `.claude/skills/` scoped to the project's stack. `meta init --refresh-docs` updates only the agent docs. -- `meta gen [...]` — TS codegen from entities defined under `metaobjects/`. Supports `--dry-run` and `--watch`. Generators (`entityFile()`, `queriesFile()`, `routesFile()`, `formFile()`, `tanstackQuery()`, `tanstackGrid()`, `barrel()`) are reference helpers: `meta init` wires none. Import one from its package to run it as shipped (`meta gen --list` prints the import line), and `meta eject ...` copies the ones you need to change (`entity`, `queries`, `routes`, `names`, `barrel`, `form`, `hooks`, `grid`, `grid-hook`, `routes-hono`) into `codegen/generators/` in your own repo (ADR-0034 scaffold-and-own) and prints the local import and entry to add to `metaobjects.config.ts`. The shipped defaults target Fastify/Drizzle/React/TanStack and are retargeted by editing an ejected copy, not by switching tools. Per-port codegen runs in that port's own build tool: `mvn metaobjects:generate` (Java/Kotlin), `dotnet meta gen` (C#), `metaobjects gen` (Python). +- `meta init` — scaffold `metaobjects/`, `.metaobjects/`, and `metaobjects.config.ts` (`--with-docs` also wires `docsFile()` and `apiDocsFile()`), **plus** the agent context: a slim `.metaobjects/AGENTS.md` + `CLAUDE.md` (auto-wired into the project's root `CLAUDE.md`/`AGENTS.md`) and six `metaobjects-*` Claude Code skills under `.claude/skills/` scoped to the project's stack. `meta init --refresh-docs` updates only the agent docs. +- `meta gen [...]` — TS codegen from entities defined under `metaobjects/`. Supports `--dry-run` and `--watch`. Generators (`entityFile()`, `queriesFile()`, `routesFile()`, `formFile()`, `tanstackQuery()`, `tanstackGrid()`, `barrel()`) are reference helpers: `meta init` wires none. Import one from its package to run it as shipped; `meta gen --list` prints the import line. `meta eject ...` copies the ones you need to change (`entity`, `queries`, `routes`, `names`, `barrel`, `form`, `hooks`, `grid`, `grid-hook`, `routes-hono`) into `codegen/generators/` in your own repo (ADR-0034 scaffold-and-own) and prints the local import and entry to add to `metaobjects.config.ts`. The shipped defaults target Fastify/Drizzle/React/TanStack and are retargeted by editing an ejected copy, not by switching tools. `meta gen --strict` turns an unknown attribute into an error, as `meta verify` does. `meta eject --undo` removes ejected copies that never changed. Per-port codegen runs in that port's own build tool: `mvn metaobjects:generate` (Java/Kotlin), `dotnet meta gen` (C#), `metaobjects gen` (Python). - `meta verify` — drift check. `verify --codegen` (regen-and-diff vs committed output), `verify --templates` (prompt `{{field}}` ↔ payload-VO drift), `verify --db` (live-DB schema drift, Node `meta` only). Per-port codegen verify: `mvn metaobjects:verify -Dmeta.verify.mode=codegen|templates` (Java/Kotlin), `dotnet meta verify` (C#), `metaobjects verify` (Python). -- `meta verify` also checks any `requirement.*` nodes the model declares (capability requirements — what the system is supposed to do, as metadata). `@status` is a loader-enforced closed enum: `planned | live | partial | retired`. A dangling `@implementedBy` is an ERROR on `live`/`partial` and CORRECT on `planned` — an intention has nothing to point at yet. A requirement states what SHOULD be true and is never a journal of what happened, and **`retired` satisfies that by stating a prohibition in force**: a capability that was built and then deliberately removed keeps its entry, written as the rule that it must not come back (`@supersededBy` names what replaced it, and is RESOLVED, so it must be a full dotted path; `@implementedBy` is REFUSED on `retired`). Do NOT delete a retired capability's requirement — that entry is the whole point of the feature: given a brief for a capability that had been retired, agents working from the model alone proposed rebuilding it 24 times out of 24. `@verifiedBy` and the `abandoned`/`superseded` statuses were retired in `0.24.0` and now fail the load. Entirely opt-in: no `requirement.*` nodes means no diagnostics. Registered vocabulary in all five ports, and every port's `verify` runs the same gate (`meta verify`, `metaobjects verify`, `mvn metaobjects:verify`, `dotnet meta verify`); the authoring lint is the Node `meta` CLI only. +- `meta verify` also checks any `requirement.*` nodes the model declares (capability requirements — what the system is supposed to do, as metadata). `@status` is a loader-enforced closed enum: `planned | live | partial | retired`. A dangling `@implementedBy` is an ERROR on `live`/`partial` and CORRECT on `planned` — an intention has nothing to point at yet. A requirement states what SHOULD be true and is never a journal of what happened. **`retired` satisfies that by stating a prohibition in force.** A capability that was built and then deliberately removed keeps its entry, written as the rule that it must not come back (`@supersededBy` names what replaced it, and is RESOLVED, so it must be a full dotted path; `@implementedBy` is REFUSED on `retired`). Do NOT delete a retired capability's requirement — that entry is the whole point of the feature: given a brief for a capability that had been retired, agents working from the model alone proposed rebuilding it 24 times out of 24. `@verifiedBy` and the `abandoned`/`superseded` statuses were retired in `0.24.0` and now fail the load. Entirely opt-in: no `requirement.*` nodes means no diagnostics. Registered vocabulary in all five ports, and every port's `verify` runs the same gate (`meta verify`, `metaobjects verify`, `mvn metaobjects:verify`, `dotnet meta verify`); the authoring lint is the Node `meta` CLI only. - `meta migrate --dialect --slug ` — diff metadata vs the committed schema snapshot and emit migration SQL files under `.metaobjects/migrations`; add `--db --apply` to run them. `--dialect` selects the diff pipeline, not just the SQL flavor — required offline and on `baseline`, auto-detected from the URL scheme when `--db` is given. On a brand-new database the first command is `meta migrate --from-db --db --dialect --slug init --apply` (diff against the empty database, emit `CREATE TABLE`, apply, record the snapshot) — **not** `meta migrate baseline`, which is for adopting a database that already has its schema. `meta migrate apply-pending --db --dialect ` replays committed migrations with no diff (fresh DB / CI). Supports SQLite (`file:`, `libsql:`), Postgres (`postgres:`, `postgresql:`), and Cloudflare D1 (TS only). `--dry-run` prints SQL to stdout. **Schema migration is owned by the Node `meta` CLI (ADR-0015) and used by every port regardless of backend language** — a JVM/Python/C# project needs no per-language migrate engine, but DOES need Node (or Bun) available to run it: no pre-built binary is published today. There is no Maven, .NET, or Python migrate command. - `meta export [--out ]` — flatten loaded metadata to one canonical JSON artifact. @@ -99,7 +105,7 @@ MetaObjects deliberately does **not** ship one universal binary. Schema operatio - `server/csharp/` — C# implementation (`MetaObjects`, `MetaObjects.Render`, `MetaObjects.Codegen`) - `server/python/` — Python implementation (`metaobjects`, `ObjectManager`) - `client/web/` — universal web client TypeScript packages consumed by all backends (`@metaobjectsdev/runtime-web`, `@metaobjectsdev/react`, `@metaobjectsdev/tanstack`) -- `examples/advanced-modeling/` — a worked, runnable model exercising projections, value objects, TPH and prompt payloads on one spine (the non-toy reference for authoring) +- `examples/advanced-modeling/` — a worked, runnable model exercising projections, value objects, TPH and prompt payloads together (the non-toy reference for authoring) ## TypeScript packages (npm) diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index bb5fe6342..6f4289905 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -216,6 +216,77 @@ a project whose `codegen/Codegen.csproj` pins an older `MetaObjects.Codegen` tha prints a warning naming the line to change. See [Own your codegen → C#](../features/own-your-codegen.md#c-dotnet-meta-eject). +**Give a copy back.** `dotnet meta eject --undo` drops every owned copy that is identical to the +shipped generator (the comparison ignores blank lines, order and trailing whitespace, and covers +the emitters eject handed over with it); `dotnet meta eject --undo ...` drops only those. +A named copy that differs is refused with exit 2 and nothing is removed, naming how far it +differs, unless you pass `--force` to discard your edits. Every name is checked before anything +is deleted. A companion file goes only when it is identical and no remaining owned generator +uses it, and `codegen/runtime/` goes only when every owned generator that used it is gone. +`--undo` never edits `codegen/Generators.cs` or the project: it prints the line to change for +each entry (`new Codegen.Generators.X()` back to `new MetaObjects.Codegen.Generators.X()`), and +says plainly when nothing is identical. There is no `doctor` command: the one-line notice `gen` +prints when identical copies exist (stderr, advisory, never the exit code) and +`dotnet meta gen --list`, which marks each owned copy `identical` or `DIFFERS`, are where stale +or identical copies show. + +**Strictness.** `dotnet meta gen` loads leniently: an unknown `@attribute` is a warning naming +the attribute, node and file, and gen generates anyway. `dotnet meta gen --strict` loads as +`verify` does (ADR-0023): the same `ERR_UNKNOWN_ATTR` lines with a gen-specific hint (drop `--strict`), exit 1, before any file is +written. `verify` itself stays strict unless you pass `--lax`. + +## Write your own generator + +A generator is a class that implements `IGenerator` (`MetaObjects.Codegen/Generator.cs`): a +kebab-case `Name` and `Generate(GenContext)`, which returns the files (`EmittedFile(path, +content)`, the path relative to the output directory). `ctx.Entities` and `ctx.Root` are the +loaded model. Optional members: `OutDir` (write somewhere other than `--out`), `HandEdits`. + +```csharp +// codegen/generators/EntityList.cs +using MetaObjects.Codegen; + +public sealed class EntityListGenerator : IGenerator +{ + public string Name => "entity-list"; + + public IEnumerable Generate(GenContext ctx) + { + var names = ctx.Entities.Select(e => e.Name).Order(StringComparer.Ordinal); + yield return new EmittedFile("EntityList.g.cs", + "// \npublic static class EntityList\n{\n" + + " public static readonly string[] Names = [" + string.Join(", ", names.Select(n => $"\"{n}\"")) + "];\n}\n"); + } +} +``` + +Wire it by adding one entry to the list in `codegen/Generators.cs`, beside the packaged ones: + +```csharp + [ + new MetaObjects.Codegen.Generators.EntityGenerator(), + new EntityListGenerator(), + ] +``` + +`dotnet meta gen` then runs it with the rest of the list, and `dotnet meta verify --codegen` +regenerates it and compares, so a hand edit to its output or a model change that moves it fails +the gate like any packaged generator's. The example above is compiled and run by +`GeneratorAuthoringExampleTests`. To start from a shipped generator rather than a blank class, +`dotnet meta eject ` copies one in. + +## Documentation + +`dotnet meta init --with-docs` writes the list with the api reference already in it +(`new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = "docs/csharp", ProjectName = "" }`), +so `gen` writes it. It does not write `.metaobjects/config.json`: set `"csharp": { "outDir": "" }` +there and a plain `dotnet meta verify` drift-checks the docs with the rest of the codegen output +(`verify --codegen` compares the whole tree of each entry's `OutDir`, so keep it generated files +only). Against an existing list, init prints the entry to add and edits nothing. Without the flag +init is unchanged. The api pages carry a "Model / metadata" link only when the entry sets +`ModelBaseUrl`: the C# port writes no model pages of its own, so a relative link would resolve to +nothing and is omitted. + Schema migrations are owned by the Node `meta` CLI (ADR-0015) — the C# CLI is `gen` + `verify` only. diff --git a/docs/ports/java.md b/docs/ports/java.md index 414d030a9..ba156e209 100644 --- a/docs/ports/java.md +++ b/docs/ports/java.md @@ -31,6 +31,27 @@ or replaced (`overwrite`). `mvn metaobjects:eject -Dnames generator into your repo to own. Commit the manifest. Details: [Own your codegen](../features/own-your-codegen.md). +## Undo an eject, strict loading, and diagnostics + +**Undo an eject.** `mvn metaobjects:eject -Dundo=true` deletes the owned copies under `codegen/` +that are identical to the generator this plugin ships (the comparison `-Dlist` shows as +`[owned — identical]`: the generator, the classes copied with it and its page templates, the +`package` line and the other lines eject itself rewrites ignored). With `-Dnames=a,b` it removes +those only, and a copy that differs is refused, with nothing removed, unless `-Dforce=true`; every +name is checked before anything is deleted. It never edits `pom.xml`: for each generator removed it +prints the `` to put back, owned to shipped. The shared support classes and helper +runtime that eject also wrote stay (other owned generators may import them) and are named. While an +identical copy exists, `mvn metaobjects:generate` logs one advisory line naming it and this command; +it never changes the outcome of the build. + +**Strict loading.** `metaobjects:generate` already loads strictly, exactly as `metaobjects:verify` +does (ADR-0023): an attribute no registered provider declares fails the goal with +`ERR_UNKNOWN_ATTR` before any generator runs, and no file is written. `-Dmeta.lax=true` is the +opt-out for both goals. There is no separate `-Dstrict` switch. + +**No `doctor` goal.** There is no `metaobjects:doctor`; `metaobjects:verify` (drift and the +requirement gate) and `eject -Dlist` (owned-copy status) are the diagnostics the plugin has. + ## Install Set `${metaobjects.version}` to the current Maven Central release (`8.1.4`) — both @@ -382,7 +403,7 @@ name in every port. | `report-export` | `SpringReportExportGenerator` | `metaobjects-codegen-spring` | Standalone export of a served report (`?format=csv\|json\|xml`, see [Report export](#report-export)): one `Export` per served report beside its controller (columns in derived-field order, wire-form cells, `MAX_ROWS`, `csv(rows)` and `xml(rows)`) and one shared `ReportExportWriter` (CSV quoting, CRLF, the `EXPORT_CSV_BOM` constant, the formula-injection guard, XML over JDK StAX; no new dependency, Jackson is already there). Args: `outputDir` (required), `maxRows` (default `100000`), `xmlNulls` (`empty`, the default, or `nil`), `csvBom` (default `false`) and `writerPackage` (default: the package prefix the served reports share, else `export`). Never implied by a library. Ejectable. | | `trace-helper` | `LlmTraceHelperGenerator` | `metaobjects-codegen-spring` | One `TraceHelper` per concrete entity that transitively `extends` `metaobjects::ai::LlmCallBase` **and** nests a `template.prompt` carrying `@responseRef` — a static `record(...)` that extracts the typed response, builds the `LlmCallBase` trace row, and persists it. Emits nothing for any other entity. | | `template` | `TemplateScopeGenerator` | `metaobjects-codegen-base` | The generic Mustache primitive: walk the model by a named `scope`, render one shared template per walk result, write each file. The declarative alternative to writing a generator class — see [Declarative template-codegen](#declarative-template-codegen-templatescopegenerator) below. It renders through `render.templategen.TemplateGenerator`, the byte-pinned cross-port factory; that factory takes a walk callback and is not itself a `Generator`, so `TemplateScopeGenerator` is the class a pom names. | -| `api-docs` | `JavaApiDocsGenerator` | `metaobjects-codegen-spring` | The Java SDK api reference: one page per entity, value object, projection, served report and template, an index and `AGENT-API.md`, under `/`. A `` entry like any other, run by `metaobjects:generate` and checked by `metaobjects:verify`. Args: `outputDir` (required; give it generated files only, since `verify` compares the whole tree), `apiSubDir` (default `api/java`), `layout` (`flat`, the default, or `package`), `projectName` (the agent page's header label, default `project`) and `modelBaseUrl` (absolute base for the links to the model pages; relative when unset). The `metaobjects:docs` goal was removed in 8.1.0 ([migration](../features/migrations/docs-are-generators.md#java-and-kotlin)). Ejectable: the copy takes the page IR, the builder, the renderer, the page paths and the page templates with it. | +| `api-docs` | `JavaApiDocsGenerator` | `metaobjects-codegen-spring` | The Java SDK api reference: one page per entity, value object, projection, served report and template, an index and `AGENT-API.md`, under `/`. A `` entry like any other, run by `metaobjects:generate` and checked by `metaobjects:verify`. Args: `outputDir` (required; give it generated files only, since `verify` compares the whole tree), `apiSubDir` (default `api/java`), `layout` (`flat`, the default, or `package`), `projectName` (the agent page's header label, default `project`) and `modelBaseUrl` (absolute base for the links to the model pages; with none set the pages carry no `Model / metadata` link, since nothing writes a page for it to point at). The `metaobjects:docs` goal was removed in 8.1.0 ([migration](../features/migrations/docs-are-generators.md#java-and-kotlin)). Ejectable: the copy takes the page IR, the builder, the renderer, the page paths and the page templates with it. | | `plantuml` | `PlantUMLGenerator` | `metaobjects-codegen-plantuml` | One PlantUML class diagram of the object model (`outputDir`, `outputFilename`; `showAttrs`, `showFields`, `showAbstracts`, `drawKeys`, `drawRefs` choose what is drawn). Ejectable: the copy takes `PlantUMLWriter` with it. | | `html-docs` | `MetaDataHtmlDocumentationGenerator` | `metaobjects-codegen-base` | One HTML page describing the registered metamodel types (`outputDir`, `outputFilename`). It reads the type registry, not your model, so it documents the vocabulary your build loads, providers included. Ejectable: the copy takes `MetaDataHtmlDocumentationWriter` with it. | | `ai-docs` | `MetaDataAIDocumentationGenerator` | `metaobjects-codegen-base` | The same type reference as JSON, laid out for an AI reader (`outputDir`, `outputFilename`). Ejectable: the copy takes `MetaDataAIDocumentationWriter` with it. | diff --git a/docs/ports/kotlin.md b/docs/ports/kotlin.md index 217ece0b4..e58a54f77 100644 --- a/docs/ports/kotlin.md +++ b/docs/ports/kotlin.md @@ -28,6 +28,27 @@ manifest (`.metaobjects/.gen-state/.hashes.json`) is overwritten, and an edited name (or replaced with `overwrite`). `mvn metaobjects:eject -Dnames=` copies a generator into your repo to own. +## Undo an eject, strict loading, and diagnostics + +**Undo an eject.** `mvn metaobjects:eject -Dundo=true` deletes the owned copies under `codegen/` +that are identical to the generator this plugin ships (the comparison `-Dlist` shows as +`[owned — identical]`: the generator, the classes copied with it and its page templates, the +`package` line and the other lines eject itself rewrites ignored). With `-Dnames=a,b` it removes +those only, and a copy that differs is refused, with nothing removed, unless `-Dforce=true`; every +name is checked before anything is deleted. It never edits `pom.xml`: for each generator removed it +prints the `` to put back, owned to shipped. The shared support classes and helper +runtime that eject also wrote stay (other owned generators may import them) and are named. While an +identical copy exists, `mvn metaobjects:generate` logs one advisory line naming it and this command; +it never changes the outcome of the build. + +**Strict loading.** `metaobjects:generate` already loads strictly, exactly as `metaobjects:verify` +does (ADR-0023): an attribute no registered provider declares fails the goal with +`ERR_UNKNOWN_ATTR` before any generator runs, and no file is written. `-Dmeta.lax=true` is the +opt-out for both goals. There is no separate `-Dstrict` switch. + +**No `doctor` goal.** There is no `metaobjects:doctor`; `metaobjects:verify` (drift and the +requirement gate) and `eject -Dlist` (owned-copy status) are the diagnostics the plugin has. + ## Install Every version below comes from one `` entry. Set the Java level (Maven defaults to @@ -156,7 +177,7 @@ The 17 generators registered in `codegen-kotlin` (`GeneratorRegistry.kt`): | `KotlinSpringControllerGenerator` | `Controller.kt` — Spring `@RestController` (5 CRUD endpoints; cross-port API contract). **Select `KotlinRelationsGenerator` with it** when the model has a M:N relationship: the traversal routes call the `Query` helpers only that generator emits, and without it the controller does not compile | entities with `source.rdb @kind="table"` | | `KotlinRequirementTestsGenerator` | Per metamodel package that holds a tested requirement, two files in `testPackage`: `Requirements__Witnesses.kt` (an interface whose default members fail with `unimplemented requirement: ...`) and `Requirements__Test.kt` (one JUnit Jupiter `@Test` per requirement, calling a project-owned witness). See "Requirement tests" below | every tested `requirement.*` (functional, level 4 or above by default) | | `KotlinReportExportGenerator` | Standalone export of a served report (`?format=csv\|json\|xml`, see [Report export](#report-export)): one `Export.kt` per served report beside its controller and one shared `ReportExportWriter.kt`. Args: `outputDir` (required), `maxRows` (default `100000`), `xmlNulls` (`empty`, the default, or `nil`), `csvBom` (default `false`) and `writerPackage` (default: the package prefix the served reports share, else `export`). Never implied by a library. Ejectable | every served report | -| `KotlinApiDocsGenerator` | The Kotlin SDK api reference, under `/`: `.md` per entity, value object, projection, served report and template, a `README.md` index and `AGENT-API.md`. A `` entry like any other, run by `metaobjects:generate` and checked by `metaobjects:verify`. Args: `outputDir` (required, generated files only), `apiSubDir` (default `api/kotlin`), `layout` (`flat` or `package`), `projectName` and `modelBaseUrl`, as for the Java `api-docs`. The `metaobjects:docs` goal was removed in 8.1.0 ([migration](../features/migrations/docs-are-generators.md#java-and-kotlin)). Ejectable, with the page model, builder, renderer and path math | the whole model, once | +| `KotlinApiDocsGenerator` | The Kotlin SDK api reference, under `/`: `.md` per entity, value object, projection, served report and template, a `README.md` index and `AGENT-API.md`. A `` entry like any other, run by `metaobjects:generate` and checked by `metaobjects:verify`. Args: `outputDir` (required, generated files only), `apiSubDir` (default `api/kotlin`), `layout` (`flat` or `package`), `projectName` and `modelBaseUrl`, as for the Java `api-docs` (with no `modelBaseUrl` the pages carry no `Model / metadata` link). The `metaobjects:docs` goal was removed in 8.1.0 ([migration](../features/migrations/docs-are-generators.md#java-and-kotlin)). Ejectable, with the page model, builder, renderer and path math | the whole model, once | Maven wiring. The plugin bundles `metaobjects-codegen-kotlin`, so it is not a dependency of your project. Keep `` at plugin level and bind the goal in ``: without the diff --git a/docs/ports/python.md b/docs/ports/python.md index cd392e1d8..96207834f 100644 --- a/docs/ports/python.md +++ b/docs/ports/python.md @@ -157,6 +157,38 @@ config is refused. Column naming is not a Python setting at all: it is `naming` ([physical-naming.md](../features/migrations/physical-naming.md)). See [the migration section](../features/migrations/docs-are-generators.md#python) and the [config reference](../features/cli.md). +**Start a project.** `metaobjects init` writes `metaobjects.config.yaml` (one `app` target +with no generators) and an empty, loadable `metaobjects/meta.app.yaml`, and reports what it +wrote. It never overwrites a file it finds; it says it kept it. `metaobjects init --with-docs` +also adds a `docs` target running `api-docs`; without the flag no documentation generator is +wired. Nothing writes docs until the config lists `api-docs`. The "Model / metadata" link on +each api page is emitted only when `apiDocs.modelBaseUrl` is set (the Python port writes no +model page, so a relative link would be dead). + +**Where routes mount.** `apiPrefix` at the top level of `metaobjects.config.yaml` (the same +key name as the TypeScript config, whose home is `metaobjects.config.ts`) sets the path the +generated FastAPI routers mount under and the api docs name. The default is `/api`, so +existing output is unchanged: `/api/authors`. `apiPrefix: /v1` gives `/v1/authors`; +`apiPrefix: ""` mounts at `/authors`. It must be `""` or start with `/`; a trailing `/` is +dropped. `gen` and `verify --codegen` read the same value. + +**Strict load.** `metaobjects gen` loads leniently and warns about an unknown attribute. +`metaobjects gen --strict` uses the load `verify` uses (ADR-0023): an unknown attribute is an +error with `verify`'s message and a gen-specific hint (drop `--strict`), the exit code is non-zero, and no file is written. + +**Owned copies.** When `gen` runs and an owned generator copy (`codegen/generators/`) is +identical to the shipped one, it prints one `note:` line on stderr naming it and +`metaobjects eject --undo`; the exit code is unchanged and a project with no owned copies pays +nothing. `metaobjects eject --undo` removes every owned copy whose file and companion runtime +modules are identical to the shipped ones (formatting is ignored); `--undo ...` removes +only those names, and a copy that differs is refused with exit 2, nothing removed, and the +`diff` command, unless `--force`. Names are validated before anything is deleted. A runtime +copy in `codegen/runtime/` goes only when it matches the package and no remaining owned +generator reads it. The config is never edited: `--undo` prints the entry to put back +(`replace codegen.generators.entity:entity_model with "entity"`). The Python port has no +`doctor` command; the `gen` notice and `metaobjects gen --list` (which marks each owned copy +`identical` or `DIFFERS`) are where stale or identical copies show. + ### Reports A concrete `object.report` whose read source is a `source.rdb` of `@kind: view` is served diff --git a/docs/ports/typescript.md b/docs/ports/typescript.md index 5b08c688d..c251073a4 100644 --- a/docs/ports/typescript.md +++ b/docs/ports/typescript.md @@ -120,6 +120,19 @@ model walks, the naming and type-mapping tables), which the copies import from `meta eject docs-site` copies instead ([what the copy holds](../features/own-your-codegen.md#what-the-copy-holds)). +A copy that never changed is a cost: it stays at the release it was copied from and misses +every later fix. `meta gen` prints one line naming the owned copies that are still +the shipped reference, unchanged, and `meta eject --undo` takes them back: with no names it +removes every such copy (and the template modules only it used), and with names it removes +those, refusing the whole call, and removing nothing, if a named copy differs (`--force` +removes it anyway). It never edits `metaobjects.config.ts`; it prints the import to swap +back to the package's. There is no `meta doctor`: `meta eject --list` is the report. +A copy you changed stays yours. + +`meta gen` loads leniently, so an attribute the registry does not know (a typo such as +`@sensitve`) is a warning and the run goes on. `meta gen --strict` makes it an error, as +`meta verify` does: it exits 1 and writes nothing. + Documentation is generated the same way, from the same list: there is no docs command (ADR-0034 Amendment 9). `docsFile()`, `apiDocsFile()`, `requirementsFile()`, `agentDocsFile()`, `metamodelDocsFile()` and `docsSiteFile()` write to the target named @@ -473,6 +486,33 @@ operands by those options. `meta migrate` warns for a decimal declared with `@pr on SQLite or D1. `meta verify --codegen` names the `wire` block that reproduces committed output generated under another setting. +### Types exported per entity + +Code that talks to a generated entity imports these names, so they are a contract. Each +entity module (`.ts`) exports six types. `entityTypeNames(obj)`, exported from +`@metaobjectsdev/codegen-ts`, returns the same six spellings for an owned generator, so it +need not rebuild them from the entity name. + +| Type | Derived from | Use it for | +| --- | --- | --- | +| `` | the Drizzle table, `InferSelectModel` | a row read back from the database: what `findById` and `create` return | +| `Insert` | the Drizzle table, `InferInsertModel` | a row as the database accepts it, before validation | +| `Update` | `Partial<Insert>` | the same, every column optional | +| `Create` | `z.input` of `InsertSchema` | **the argument of `create`**, and what a client posts | +| `Patch` | `z.input` of `UpdateSchema` | **the argument of `update`**, and what a client patches | +| `Filter` | the `@filterable` and `@sortable` fields | the `filter`, `sort`, `limit` and `offset` options of `list` and the generated hooks | + +`Create` and `Patch` are the pair to use at a call site. They come from the Zod schemas the +functions validate with, so a renamed or dropped field is a compile error at every call. +`Insert` and `Update` come from the table and describe what the database takes. They are +looser than the validators, so prefer `Create` and `Patch` at the API boundary. + +When the entity is a TPH discriminator base, the row type is `Row`, because +`` is the discriminated union over its subtypes. A read-only projection has no +table to insert into, so do not expect `Create` or `Patch` there; read its generated module +for what it exports. `test/entity-type-names.test.ts` in `codegen-ts` checks the table +against the golden `Subscriber` module. + ### `Names` — the physical names, as constants `meta init` scaffolds a `names` generator, so a new project gets `.names.ts` diff --git a/docs/recipes/typed-client-for-the-reference-routes.md b/docs/recipes/typed-client-for-the-reference-routes.md new file mode 100644 index 000000000..844e84596 --- /dev/null +++ b/docs/recipes/typed-client-for-the-reference-routes.md @@ -0,0 +1,291 @@ +# A typed fetch client for the reference routes + +You run the stock `entity`, `queries` and `routes` generators, which give you a database +schema, queries and a Fastify API. Nothing types the **caller** of that API. This recipe +writes one owned generator that does: per entity, a client for exactly the routes `routes` +mounts, typed with the `X`, `XCreate`, `XPatch` and `XFilter` types the `entity` generator +already wrote. Rename a field and the client's call sites stop compiling. + +Every block below was run: generate, `tsc`, a smoke call against the real Fastify routes, then a +model change that `meta verify --codegen` convicts. Output shown is output seen, on the 1.1.4 +packages. + +## Install + +The stock generated files import `drizzle-orm`, `zod`, `fastify` and a database driver, and the +routes import `@metaobjectsdev/runtime-ts`. They are optional peers of those packages, so +`npm` does not add them for you. This pair compiles the generated TypeScript from a clean +project (SQLite shown): + +```bash +npm init -y && npm pkg set type=module +npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3 +npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3 +``` + +For Postgres swap the driver: `@types/pg` in the first line, `pg` in the second (and +`dialect: "postgres"`). Both were installed into an empty directory and the generated tree +compiled with `tsc` under `"module": "nodenext", "strict": true`. Dropping `fastify` or `zod` +fails with `TS2307: Cannot find module`. Run the smoke script with `npx tsx` (`npm i -D tsx`). + +## The model + +Two entities with a relationship. `@filterable` is what puts a field in the list route's filter +allowlist, and in the `XFilter` type: + +```json +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Customer", "children": [ + { "source.rdb": { "@table": "customers" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@required": true } }, + { "field.string": { "name": "email" } }, + { "relationship.composition": { "name": "orders", "@objectRef": "Order", "@cardinality": "many", "@onDelete": "cascade" } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } } + ] } }, + { "object.entity": { "name": "Order", "children": [ + { "source.rdb": { "@table": "orders" } }, + { "field.long": { "name": "id" } }, + { "field.int": { "name": "total", "@required": true, "@filterable": true } }, + { "field.boolean": { "name": "paid", "@filterable": true } }, + { "field.long": { "name": "customerId", "@required": true, "@filterable": true } }, + { "identity.primary": { "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { "name": "fkCustomer", "@fields": "customerId", "@references": "Customer", "@onDelete": "cascade" } } + ] } } + ] + } +} +``` + +## The stock generators, and what they really mount + +```ts +import { defineConfig } from "@metaobjectsdev/cli"; +import { entityFile, queriesFile, routesFile } from "@metaobjectsdev/codegen-ts/generators"; +import { fetchClientGenerator } from "./codegen/generators/fetch-client.js"; + +export default defineConfig({ + outDir: "src/generated", + dialect: "sqlite", + apiPrefix: "/api", + dbImport: "./db", + generators: [entityFile(), queriesFile(), routesFile(), fetchClientGenerator()], +}); +``` + +`routesFile()` needs `dbImport`: the module that exports `db`. `meta gen` fails and says so if +it is missing. Write `src/generated/db.ts` yourself (it is not generated; an in-memory database keeps the smoke script repeatable): + +```ts +import Database from "better-sqlite3"; +import { drizzle } from "drizzle-orm/better-sqlite3"; +export const sqlite = new Database(":memory:"); +export const db = drizzle(sqlite); +``` + +What `routesFile()` mounts for each entity with a writable source, read from the generated +`Order.routes.ts` and the runtime it calls. The smoke run below exercises list, get, create, +update and delete, plus a 404, a 409 and a 400. `apiPrefix` is the mount +prefix, and `servedPath(obj, apiPrefix)` is the helper that composes it with the object's path +(`/orders`): + +| Call | Route | Success | Failure seen | +|---|---|---|---| +| list | `GET /api/orders` | `200`, a JSON array | `400 {"error":"invalid_filter_value",...}` | +| get | `GET /api/orders/:id` | `200`, the row | `404 {"error":"not_found"}`, `400 {"error":"invalid_id"}` | +| create | `POST /api/orders` | `201`, the row | `409 {"error":"constraint_violation","constraint":"foreign_key"}` | +| update | `PATCH /api/orders/:id` (`PUT` too: both are mounted) | `200`, the row | `404 not_found` | +| delete | `DELETE /api/orders/:id` | `204`, no body | `404 not_found` | + +List query encoding: `filter[][]=` (a bare `filter[total]=5` is not a filter; +the client sends `[eq]`), an `in` list comma-joined (`filter[total][in]=5,50`), `or` and `and` +as `filter[or][0][paid][eq]=true`, `sort=:asc|desc`, and `limit` and `offset` as plain +integers. The operators a field takes are in the generated `FilterAllowlist`, and the +`XFilter` type is derived from it. + +## The generator + +`meta generator new fetch-client` scaffolds `codegen/generators/fetch-client.ts` and wires its +import and entry into `metaobjects.config.ts`. Replace the body with this. It emits one shared +runtime file plus one `X.client.ts` per entity, next to the entity module it imports types from: + +```ts +// fetch-client: a typed fetch client for exactly the routes the stock `routes` generator mounts. +import { + entityOutputPath, isAbstract, servedPath, servesWriteApi, type EmittedFile, type Generator, +} from "@metaobjectsdev/codegen-ts"; + +// One shared runtime file: how a filter becomes a query string, and the error a non-2xx throws. +const RUNTIME = `// @generated by fetch-client +export class ApiError extends Error { + constructor(readonly status: number, readonly body: unknown) { super(\`HTTP \${status}\`); } +} +export type Fetch = (url: string, init?: RequestInit) => Promise; + +/** filter[total][gt]=5, filter[total][in]=1,2, filter[or][0][paid][eq]=true; limit/offset/sort stay top level. */ +export function toQuery(f: Record = {}): string { + const out: string[] = []; + const add = (k: string, v: unknown) => out.push(\`\${k}=\${encodeURIComponent(String(v))}\`); + const walk = (node: Record, prefix: string) => { + for (const [k, v] of Object.entries(node)) { + if (v === undefined) continue; + if ((k === "or" || k === "and") && Array.isArray(v)) { + v.forEach((sub, i) => walk(sub as Record, \`\${prefix}[\${k}][\${i}]\`)); + } else if (v !== null && typeof v === "object") { + for (const [op, x] of Object.entries(v)) if (x !== undefined) add(\`\${prefix}[\${k}][\${op}]\`, Array.isArray(x) ? x.join(",") : x); + } else add(\`\${prefix}[\${k}][eq]\`, v); + } + }; + const { limit, offset, sort, ...filter } = f; + walk(filter, "filter"); + for (const [k, v] of Object.entries({ limit, offset, sort })) if (v !== undefined) add(k, v); + return out.length ? \`?\${out.join("&")}\` : ""; +} + +export async function call(f: Fetch, method: string, url: string, body?: unknown): Promise { + const res = await f(url, { method, headers: body === undefined ? {} : { "content-type": "application/json" }, + body: body === undefined ? undefined : JSON.stringify(body) }); + if (res.status === 404 && method === "GET") return null; + if (!res.ok) throw new ApiError(res.status, await res.json().catch(() => undefined)); + return res.status === 204 ? (undefined as T) : ((await res.json()) as T); +} +`; + +export function fetchClientGenerator(): Generator { + return { + name: "fetch-client", + filter: (obj) => !isAbstract(obj) && servesWriteApi(obj), // the entities `routes` mounts all five verbs for + generate: (ctx): EmittedFile[] => { + const layout = ctx.config.outputLayout ?? "flat"; + const prefix = ctx.renderContext?.apiPrefix ?? ""; // the same apiPrefix routesFile mounts under + const files: EmittedFile[] = [{ path: "fetch-client.runtime.ts", content: RUNTIME }]; + for (const obj of ctx.entities.filter(ctx.matches)) { + const n = obj.name, path = servedPath(obj, prefix); + // Types come from the entity module `entity` wrote next to this file (flat layout). + files.push({ + path: entityOutputPath(layout, undefined, `${n}.client.ts`), + content: `// @generated by fetch-client +import type { ${n}, ${n}Create, ${n}Filter, ${n}Patch } from "./${n}.js"; +import { call, toQuery, type Fetch } from "./fetch-client.runtime.js"; + +export function ${n[0]!.toLowerCase()}${n.slice(1)}Client(baseUrl = "", f: Fetch = fetch) { + const url = (rest = "") => \`\${baseUrl}${path}\${rest}\`; + return { + list: (filter?: ${n}Filter) => call<${n}[]>(f, "GET", url(toQuery(filter))) + .then((r) => r ?? []), + get: (id: ${n}["id"]) => call<${n}>(f, "GET", url(\`/\${id}\`)), // null on 404 + create: (body: ${n}Create) => call<${n}>(f, "POST", url(), body), // 201 + update: (id: ${n}["id"], body: ${n}Patch) => call<${n}>(f, "PATCH", url(\`/\${id}\`), body), + delete: (id: ${n}["id"]) => call(f, "DELETE", url(\`/\${id}\`)), // 204 + }; +} +`, + }); + } + return files; + }, + }; +} +``` + +Three choices worth knowing. It filters with `servesWriteApi`, the question the stock routes +generator asks, so the client covers exactly the entities that have all five routes. It reads +`ctx.renderContext?.apiPrefix`, the config value `routesFile()` mounts under, so the two +cannot disagree. And it writes beside `entity`'s output with `entityOutputPath`, which follows +`outputLayout`. + +`get` returns `null` on a 404 and `list` returns the array; any other non-2xx throws +`ApiError` carrying the status and the parsed body. The client does not use the list route's +count flag. The generated row types are the database row types, so a `date` or `decimal` field +arrives as its JSON form, not as the Drizzle value: type those yourself if you have them. + +## Prove it + +```bash +npx meta gen +npx tsc -p tsconfig.json # exit 0: the entity, queries, routes and client all compile +npx tsc -p tsconfig.codegen.json # exit 0: the generator itself +npx meta verify --codegen # exit 0 +``` + +A smoke script that starts the real routes and calls them through the client: + +```ts +import Fastify from "fastify"; +import { sqlite } from "./src/generated/db.js"; +import { customerRoutes } from "./src/generated/Customer.routes.js"; +import { orderRoutes } from "./src/generated/Order.routes.js"; +import { customerClient } from "./src/generated/Customer.client.js"; +import { orderClient } from "./src/generated/Order.client.js"; +import { ApiError } from "./src/generated/fetch-client.runtime.js"; + +sqlite.exec(`create table customers (id integer primary key autoincrement, name text not null, email text); +create table orders (id integer primary key autoincrement, total integer not null, paid integer, customer_id integer not null references customers(id));`); +const app = Fastify(); +await app.register(customerRoutes); await app.register(orderRoutes); +const base = await app.listen({ port: 0 }); +const customers = customerClient(base), orders = orderClient(base); +const c = await customers.create({ name: "Ada" }); +console.log("created", c); +for (const t of [5, 20, 50]) await orders.create({ total: t, customerId: c.id! , paid: t > 10 }); +console.log("gt 10 sorted desc:", await orders.list({ total: { gt: 10 }, sort: "total:desc" })); +console.log("in [5,50] page:", await orders.list({ total: { in: [5, 50] }, limit: 1, offset: 1 })); +console.log("or:", (await orders.list({ or: [{ total: 5 }, { paid: true }] })).map((o) => o.total)); +console.log("patch:", await orders.update(1, { paid: true })); +console.log("get 999:", await orders.get(999)); +await orders.delete(1); +console.log("after delete:", (await orders.list()).length); +try { await orders.create({ total: 1, customerId: 999 }); } catch (e) { console.log("fk error:", e instanceof ApiError, (e as ApiError).status, JSON.stringify((e as ApiError).body)); } +try { await orders.list({ total: { gt: 1.5 } }); } catch (e) { console.log("bad filter:", (e as ApiError).status, JSON.stringify((e as ApiError).body)); } +await app.close(); +``` + +```text +created { id: 1, name: 'Ada', email: null } +gt 10 sorted desc: [ + { id: 3, total: 50, paid: true, customerId: 1 }, + { id: 2, total: 20, paid: true, customerId: 1 } +] +in [5,50] page: [ { id: 3, total: 50, paid: true, customerId: 1 } ] +or: [ 5, 20, 50 ] +patch: { id: 1, total: 5, paid: true, customerId: 1 } +get 999: null +after delete: 2 +fk error: true 409 {"error":"constraint_violation","constraint":"foreign_key"} +bad filter: 400 {"error":"invalid_filter_value","field":"total","op":"gt","expected":"integer (a whole number)"} +``` + +## The drift gate covers it + +`verify --codegen` re-runs every generator in `generators`, yours included. Rename `total` to +`amount` in the model and do not regenerate: + +```text +meta verify --codegen exit 1 + ~ src/generated/Order.ts (committed content differs from a fresh regen) + Run 'meta gen' to regenerate, then commit the result. +``` + +The type module is stale, and once regenerated `orders.list({ total: ... })` fails `tsc`. A +rename does not change the client files themselves, since they hold no field names. A change +to what they do hold does. Changing `apiPrefix` to `/v1` without regenerating: + +```text +meta verify --codegen exit 1 + ~ src/generated/Customer.client.ts (committed content differs from a fresh regen) + ~ src/generated/Customer.routes.ts (committed content differs from a fresh regen) + ~ src/generated/Order.client.ts (committed content differs from a fresh regen) + ~ src/generated/Order.routes.ts (committed content differs from a fresh regen) +``` + +The client and the routes move together or the build fails. Commit +`.metaobjects/.gen-state/.hashes.json` so a fresh clone can tell a hand edit from stale output. + +## See also + +- [Write your own generator](write-your-own-generator.md), including when to set `handEdits: "overwrite"`. +- [Own your codegen](../features/own-your-codegen.md) +- [API contract](../features/api-contract.md): the routes, filters and errors every port serves. diff --git a/docs/recipes/write-your-own-generator.md b/docs/recipes/write-your-own-generator.md index 57c5c9ba5..ff5591da9 100644 --- a/docs/recipes/write-your-own-generator.md +++ b/docs/recipes/write-your-own-generator.md @@ -44,6 +44,18 @@ its fields, inherited ones included. ### TypeScript — start with the scaffold +Install once. This pair compiles what the stock `entity`, `queries` and `routes` generators +emit (SQLite shown; for Postgres use `pg` and `@types/pg` in place of `better-sqlite3` and +`@types/better-sqlite3`); the generated files import `drizzle-orm`, `zod`, `fastify` and a +driver, and `npm` does not add optional peers for you: + +```bash +npm i -D @metaobjectsdev/cli @metaobjectsdev/codegen-ts @metaobjectsdev/metadata typescript @types/better-sqlite3 +npm i @metaobjectsdev/runtime-ts drizzle-orm zod fastify better-sqlite3 +``` + +Then: + ```bash meta generator new field-list # --scope entity (default) | package | model meta gen # runs it @@ -281,6 +293,35 @@ class KtFieldListGenerator : FileEmittingGenerator() { Wire it with `com.acme.codegen.KtFieldListGenerator` from a module built with `kotlin-maven-plugin`. +## Safety-relevant output: set `handEdits: "overwrite"` + +A generator whose output enforces something (a redaction rule, an audit log, an auth check) +needs one more setting. The default is `handEdits: "keep"`: someone can hand-edit the emitted +file, `meta gen` merges that edit into fresh output, and `meta verify --codegen` still passes +after the merge. Run on a scratch project, a line appended to a generated client survived +`meta gen` (status `merged`) and `verify --codegen` exited 0. For a redaction rule that is a +silent hole. + +Set `handEdits: "overwrite"` on that generator. A hand edit is then discarded and named in the +run's warnings, and the committed file always equals fresh output. One line, on the entry in +`metaobjects.config.ts`: + +```ts +generators: [entityFile(), { ...redactionGenerator(), handEdits: "overwrite" }], +``` + +Seen: with it set, `meta verify --codegen` exits 1 on an edited file (`edited by hand; its +generator sets handEdits: "overwrite", so 'meta gen' replaces it with fresh output`), and +`meta gen` prints `overwrote hand-edited with fresh output` and restores it. +The setting is per generator, and an edited file is discarded only when it is one `gen` has a record of writing. + +The same trap exists one layer up, in the model. `meta gen` loads metadata **leniently**: an +unknown attribute is a warning, and generation goes on. A typo'd `@sensitve` on a field +silently changes what your generator sees, so a redaction keyed on `@sensitive` quietly stops +applying. `meta gen --strict` makes an unknown attribute an error (exit 1, nothing written). +`meta verify` always rejects it. Use `--strict` in CI for any project whose generators +enforce something. + ## Reading the model correctly — the one rule **Read through the resolving accessors.** `extends` is a reference to a parent, not a copy @@ -344,6 +385,8 @@ component schema, and the reference CRUD paths for each object with a source. ## See also +- [A typed fetch client for the reference routes](typed-client-for-the-reference-routes.md): + one owned TypeScript generator, run and drift-gated against the stock routes. - [Own your codegen](../features/own-your-codegen.md) — ejecting a reference generator, and how each port protects hand edits. - [Codegen concepts](../features/codegen-concepts.md) — scopes, programmatic vs Mustache diff --git a/examples/advanced-modeling/codegen/generators/queries.ts b/examples/advanced-modeling/codegen/generators/queries.ts index 7fde437cd..3be4632cb 100644 --- a/examples/advanced-modeling/codegen/generators/queries.ts +++ b/examples/advanced-modeling/codegen/generators/queries.ts @@ -160,6 +160,9 @@ export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generat const generator: Generator = { name: "queries-file", + // Read by the api reference (`apiDocsFile()`): it documents the query helpers only when + // a generator in the run writes them. + emitsQueries: true, filter, generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { diff --git a/examples/advanced-modeling/codegen/templates/entity-type-names.ts b/examples/advanced-modeling/codegen/templates/entity-type-names.ts new file mode 100644 index 000000000..81124bb5d --- /dev/null +++ b/examples/advanced-modeling/codegen/templates/entity-type-names.ts @@ -0,0 +1,46 @@ +// The six type names the entity module exports for an entity. A client written against the +// generated entity (a typed API client, a form, a service layer) imports these by name, so +// the spelling is part of the generator's contract, not an accident of three templates. +// `entityTypeNames` is the one place that spelling lives: the templates that emit the names +// call it, and an owned generator that refers to them calls it too, so a rename cannot +// leave one side behind. `docs/ports/typescript.md` ("Types exported per entity") is the +// reader-facing table; test/entity-type-names.test.ts pins it to the golden output. + +import type { MetaData } from "@metaobjectsdev/metadata"; + +export interface EntityTypeNames { + /** The row type read back from the table: `InferSelectModel` of the Drizzle table. */ + row: string; + /** What an insert accepts: `InferInsertModel` of the Drizzle table. */ + insert: string; + /** `Partial`. */ + update: string; + /** The Zod insert schema's INPUT type, the argument of `create`. */ + create: string; + /** The Zod update schema's INPUT type: every settable field optional, the argument of `update`. */ + patch: string; + /** The client filter object: filterable fields, `sort`, `limit`, `offset`, `or`, `and`. */ + filter: string; +} + +export interface EntityTypeNamesOpts { + /** + * The row type is `Row` instead of `` when the entity is a TPH + * discriminator base, because its discriminated-union type owns the bare name. + */ + tphBase?: boolean; + /** Use this spelling instead of `obj.name` (a value object renamed on a name collision). */ + name?: string; +} + +export function entityTypeNames(obj: MetaData, opts: EntityTypeNamesOpts = {}): EntityTypeNames { + const base = opts.name ?? obj.name; + return { + row: opts.tphBase ? `${base}Row` : base, + insert: `${base}Insert`, + update: `${base}Update`, + create: `${base}Create`, + patch: `${base}Patch`, + filter: `${base}Filter`, + }; +} diff --git a/examples/advanced-modeling/codegen/templates/filter-type.ts b/examples/advanced-modeling/codegen/templates/filter-type.ts index c3e9cc7e7..e2fd030d3 100644 --- a/examples/advanced-modeling/codegen/templates/filter-type.ts +++ b/examples/advanced-modeling/codegen/templates/filter-type.ts @@ -16,6 +16,7 @@ import { opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; +import { entityTypeNames } from "./entity-type-names.js"; import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; // currency is integer minor units (allowlist `integer: true`, row schema `z.number().int()`). @@ -79,6 +80,7 @@ function renderFieldUnion(field: MetaField): string { * the server's per-subtype allowlist. */ export function renderFilterType(entity: MetaObject, exclude?: string): Code { + const filterName = entityTypeNames(entity).filter; // fields() returns effective fields, so inherited fields (from extends:/super:) are included in filter types. const allFields = entity.fields().filter((c) => c.name !== exclude); const filterableFieldsList = allFields.filter((c) => c.attr(FIELD_ATTR_FILTERABLE) === true); @@ -93,12 +95,12 @@ export function renderFilterType(entity: MetaObject, exclude?: string): Code { if (fieldLines.length === 0 && sortFieldNames.length === 0) { return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: string; - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } @@ -112,13 +114,13 @@ export type ${entity.name}Filter = { })(); return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: ${sortType}; ${fieldLines.join("\n")} - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } diff --git a/examples/advanced-modeling/codegen/templates/inferred-types.ts b/examples/advanced-modeling/codegen/templates/inferred-types.ts index 588ec5fde..1eec56f31 100644 --- a/examples/advanced-modeling/codegen/templates/inferred-types.ts +++ b/examples/advanced-modeling/codegen/templates/inferred-types.ts @@ -44,6 +44,7 @@ import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier, providedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { effectivePackage } from "@metaobjectsdev/codegen-ts/engine"; +import { entityTypeNames } from "./entity-type-names.js"; /** * Emit Drizzle's InferSelectModel / InferInsertModel aliases for an entity. @@ -71,7 +72,7 @@ export function renderInferredTypes( const insertSym = imp("t:InferInsertModel@drizzle-orm"); const docs = renderDocsFor(entity); const docsPrefix = docs ? `${docs}\n` : ""; - const rowName = tphBase ? `${entity.name}Row` : entity.name; + const { row: rowName, insert: insertName, update: updateName } = entityTypeNames(entity, { tphBase }); // #214 — a write-through entity read-view routes READS to the replica view, so the // read row type is emitted from the VIEW's read schema (z.infer, carrying // the derived fields) by the entity-file composer, NOT here. `skipRow` suppresses @@ -80,14 +81,14 @@ export function renderInferredTypes( // inferred from the write TABLE (derived-free, #213). if (skipRow) { return code` -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } return code` ${docsPrefix}export type ${rowName} = ${selectSym}; -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } diff --git a/examples/advanced-modeling/codegen/templates/zod-validators.ts b/examples/advanced-modeling/codegen/templates/zod-validators.ts index 5f8d45a2d..ea4945fea 100644 --- a/examples/advanced-modeling/codegen/templates/zod-validators.ts +++ b/examples/advanced-modeling/codegen/templates/zod-validators.ts @@ -9,6 +9,7 @@ // object-array field collapsed to z.array(z.string()) and the JSON Schema sent // downstream (e.g. to LLM tool_use input_schema) lost the nested object shape. +import { entityTypeNames } from "./entity-type-names.js"; import { code, joinCode, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; import { MetaObject, MetaField, type MetaRoot, stripPackage } from "@metaobjectsdev/metadata"; import { @@ -529,6 +530,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const insertSchemaName = `${objName}InsertSchema`; const updateSchemaName = `${objName}UpdateSchema`; const preservingSchemaName = `${objName}InsertPreservingSchema`; + const { create: createName, patch: patchName } = entityTypeNames(obj, { name: objName }); const docs = renderDocsFor(obj); const docsPrefix = docs ? `${docs}\n` : ""; @@ -561,11 +563,11 @@ ${joinCode(updateFieldLines, { on: ",\n" })} /** Typed create shape for ${objName}: the insert schema's INPUT (pre-transform) type. A * renamed/dropped/misspelt field is a compile error at every \`create${objName}\` call site; * the schema still validates at runtime. */ -export type ${objName}Create = ${z}.input; +export type ${createName} = ${z}.input; /** Typed patch shape for ${objName}: every settable field, optional (FR-035 PATCH). A * renamed/dropped field is a compile error at every \`update${objName}\` call site. */ -export type ${objName}Patch = ${z}.input;${preservingBlock} +export type ${patchName} = ${z}.input;${preservingBlock} `; } diff --git a/examples/showcase/codegen/generators/queries.ts b/examples/showcase/codegen/generators/queries.ts index 7fde437cd..3be4632cb 100644 --- a/examples/showcase/codegen/generators/queries.ts +++ b/examples/showcase/codegen/generators/queries.ts @@ -160,6 +160,9 @@ export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generat const generator: Generator = { name: "queries-file", + // Read by the api reference (`apiDocsFile()`): it documents the query helpers only when + // a generator in the run writes them. + emitsQueries: true, filter, generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { diff --git a/examples/showcase/codegen/templates/entity-type-names.ts b/examples/showcase/codegen/templates/entity-type-names.ts new file mode 100644 index 000000000..81124bb5d --- /dev/null +++ b/examples/showcase/codegen/templates/entity-type-names.ts @@ -0,0 +1,46 @@ +// The six type names the entity module exports for an entity. A client written against the +// generated entity (a typed API client, a form, a service layer) imports these by name, so +// the spelling is part of the generator's contract, not an accident of three templates. +// `entityTypeNames` is the one place that spelling lives: the templates that emit the names +// call it, and an owned generator that refers to them calls it too, so a rename cannot +// leave one side behind. `docs/ports/typescript.md` ("Types exported per entity") is the +// reader-facing table; test/entity-type-names.test.ts pins it to the golden output. + +import type { MetaData } from "@metaobjectsdev/metadata"; + +export interface EntityTypeNames { + /** The row type read back from the table: `InferSelectModel` of the Drizzle table. */ + row: string; + /** What an insert accepts: `InferInsertModel` of the Drizzle table. */ + insert: string; + /** `Partial`. */ + update: string; + /** The Zod insert schema's INPUT type, the argument of `create`. */ + create: string; + /** The Zod update schema's INPUT type: every settable field optional, the argument of `update`. */ + patch: string; + /** The client filter object: filterable fields, `sort`, `limit`, `offset`, `or`, `and`. */ + filter: string; +} + +export interface EntityTypeNamesOpts { + /** + * The row type is `Row` instead of `` when the entity is a TPH + * discriminator base, because its discriminated-union type owns the bare name. + */ + tphBase?: boolean; + /** Use this spelling instead of `obj.name` (a value object renamed on a name collision). */ + name?: string; +} + +export function entityTypeNames(obj: MetaData, opts: EntityTypeNamesOpts = {}): EntityTypeNames { + const base = opts.name ?? obj.name; + return { + row: opts.tphBase ? `${base}Row` : base, + insert: `${base}Insert`, + update: `${base}Update`, + create: `${base}Create`, + patch: `${base}Patch`, + filter: `${base}Filter`, + }; +} diff --git a/examples/showcase/codegen/templates/filter-type.ts b/examples/showcase/codegen/templates/filter-type.ts index c3e9cc7e7..e2fd030d3 100644 --- a/examples/showcase/codegen/templates/filter-type.ts +++ b/examples/showcase/codegen/templates/filter-type.ts @@ -16,6 +16,7 @@ import { opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; +import { entityTypeNames } from "./entity-type-names.js"; import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; // currency is integer minor units (allowlist `integer: true`, row schema `z.number().int()`). @@ -79,6 +80,7 @@ function renderFieldUnion(field: MetaField): string { * the server's per-subtype allowlist. */ export function renderFilterType(entity: MetaObject, exclude?: string): Code { + const filterName = entityTypeNames(entity).filter; // fields() returns effective fields, so inherited fields (from extends:/super:) are included in filter types. const allFields = entity.fields().filter((c) => c.name !== exclude); const filterableFieldsList = allFields.filter((c) => c.attr(FIELD_ATTR_FILTERABLE) === true); @@ -93,12 +95,12 @@ export function renderFilterType(entity: MetaObject, exclude?: string): Code { if (fieldLines.length === 0 && sortFieldNames.length === 0) { return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: string; - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } @@ -112,13 +114,13 @@ export type ${entity.name}Filter = { })(); return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: ${sortType}; ${fieldLines.join("\n")} - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } diff --git a/examples/showcase/codegen/templates/inferred-types.ts b/examples/showcase/codegen/templates/inferred-types.ts index 588ec5fde..1eec56f31 100644 --- a/examples/showcase/codegen/templates/inferred-types.ts +++ b/examples/showcase/codegen/templates/inferred-types.ts @@ -44,6 +44,7 @@ import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier, providedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { effectivePackage } from "@metaobjectsdev/codegen-ts/engine"; +import { entityTypeNames } from "./entity-type-names.js"; /** * Emit Drizzle's InferSelectModel / InferInsertModel aliases for an entity. @@ -71,7 +72,7 @@ export function renderInferredTypes( const insertSym = imp("t:InferInsertModel@drizzle-orm"); const docs = renderDocsFor(entity); const docsPrefix = docs ? `${docs}\n` : ""; - const rowName = tphBase ? `${entity.name}Row` : entity.name; + const { row: rowName, insert: insertName, update: updateName } = entityTypeNames(entity, { tphBase }); // #214 — a write-through entity read-view routes READS to the replica view, so the // read row type is emitted from the VIEW's read schema (z.infer, carrying // the derived fields) by the entity-file composer, NOT here. `skipRow` suppresses @@ -80,14 +81,14 @@ export function renderInferredTypes( // inferred from the write TABLE (derived-free, #213). if (skipRow) { return code` -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } return code` ${docsPrefix}export type ${rowName} = ${selectSym}; -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } diff --git a/examples/showcase/codegen/templates/zod-validators.ts b/examples/showcase/codegen/templates/zod-validators.ts index 5f8d45a2d..ea4945fea 100644 --- a/examples/showcase/codegen/templates/zod-validators.ts +++ b/examples/showcase/codegen/templates/zod-validators.ts @@ -9,6 +9,7 @@ // object-array field collapsed to z.array(z.string()) and the JSON Schema sent // downstream (e.g. to LLM tool_use input_schema) lost the nested object shape. +import { entityTypeNames } from "./entity-type-names.js"; import { code, joinCode, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; import { MetaObject, MetaField, type MetaRoot, stripPackage } from "@metaobjectsdev/metadata"; import { @@ -529,6 +530,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const insertSchemaName = `${objName}InsertSchema`; const updateSchemaName = `${objName}UpdateSchema`; const preservingSchemaName = `${objName}InsertPreservingSchema`; + const { create: createName, patch: patchName } = entityTypeNames(obj, { name: objName }); const docs = renderDocsFor(obj); const docsPrefix = docs ? `${docs}\n` : ""; @@ -561,11 +563,11 @@ ${joinCode(updateFieldLines, { on: ",\n" })} /** Typed create shape for ${objName}: the insert schema's INPUT (pre-transform) type. A * renamed/dropped/misspelt field is a compile error at every \`create${objName}\` call site; * the schema still validates at runtime. */ -export type ${objName}Create = ${z}.input; +export type ${createName} = ${z}.input; /** Typed patch shape for ${objName}: every settable field, optional (FR-035 PATCH). A * renamed/dropped field is a compile error at every \`update${objName}\` call site. */ -export type ${objName}Patch = ${z}.input;${preservingBlock} +export type ${patchName} = ${z}.input;${preservingBlock} `; } diff --git a/fixtures/metamodel-docs/expected/INDEX.md b/fixtures/metamodel-docs/expected/INDEX.md index afe8f0df5..46ff01d31 100644 --- a/fixtures/metamodel-docs/expected/INDEX.md +++ b/fixtures/metamodel-docs/expected/INDEX.md @@ -4,13 +4,13 @@ # MetaObjects Metamodel — Index -This documents the **metamodel itself** — the type / subtype / attribute -vocabulary the loader accepts — generated from the strict registry. It is -NOT documentation of a user's entities (that is the `docs` generator, `docsFile()`). +This page documents the **metamodel itself**: the type, subtype and attribute +vocabulary the loader accepts, generated from the strict registry. Your own +entities are documented by the `docs` generator (`docsFile()`). -Follow a link into `types/.md` for the full attribute table, allowed -children, and cardinality of a subtype. Universal documentation attributes -(`@description`/`@title`/…) apply to every node and are listed once in +Each row links to its type page, which has the full attribute table, the allowed +children and the cardinality of that subtype. Documentation attributes +(`@description`, `@title` and the rest) apply to every node and are listed once in [providers.md](providers.md) under `metaobjects-documentation`. | Type.subType | Description | Page | diff --git a/server/csharp/MetaObjects.Cli.Tests/EjectEndToEndTests.cs b/server/csharp/MetaObjects.Cli.Tests/EjectEndToEndTests.cs index d011beade..74c86d912 100644 --- a/server/csharp/MetaObjects.Cli.Tests/EjectEndToEndTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/EjectEndToEndTests.cs @@ -76,6 +76,12 @@ private static string RepoRoot() psi.Environment["MSBUILDDISABLENODEREUSE"] = "1"; psi.Environment["DOTNET_CLI_USE_MSBUILD_SERVER"] = "0"; psi.Environment["UseSharedCompilation"] = "false"; + // The generated web app's default host builder watches appsettings.json with an inotify + // instance, and the per-user limit (128 by default) is shared with every other process the + // user runs, including other test lanes. When it is spent the app dies in + // WebApplication.CreateBuilder with an IOException (exit 134) that has nothing to do with + // the code under test; polling needs no inotify instance. + psi.Environment["DOTNET_USE_POLLING_FILE_WATCHER"] = "1"; psi.ArgumentList.Add("run"); psi.ArgumentList.Add("--project"); psi.ArgumentList.Add(project); diff --git a/server/csharp/MetaObjects.Cli.Tests/EjectUndoTests.cs b/server/csharp/MetaObjects.Cli.Tests/EjectUndoTests.cs new file mode 100644 index 000000000..791b5344c --- /dev/null +++ b/server/csharp/MetaObjects.Cli.Tests/EjectUndoTests.cs @@ -0,0 +1,188 @@ +// `dotnet meta eject --undo [...]` — drop owned generator copies identical to the shipped +// ones, and the `gen` notice that says such copies exist. Same contract as the TypeScript port. + +using MetaObjects.Cli; +using MetaObjects.Codegen; +using Xunit; + +namespace MetaObjects.Cli.Tests; + +public sealed class EjectUndoTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "meta-undo-" + Guid.NewGuid().ToString("N")); + private string Owned(string rel) => Path.Combine(_root, rel.Replace('/', Path.DirectorySeparatorChar)); + + public EjectUndoTests() => Directory.CreateDirectory(_root); + public void Dispose() { try { Directory.Delete(_root, recursive: true); } catch { } } + + private void Eject(params string[] names) => + Assert.Equal(0, EjectCommand.Run(names, _root, force: false).ExitCode); + + private void Edit(string rel) => File.AppendAllText(Owned(rel), "\n// my own edit\nvar mine = 1;\n"); + + [Fact] + public void With_no_names_removes_every_identical_copy_and_keeps_the_edited_one() + { + Eject("entity", "filter-allowlist"); + Edit("codegen/generators/EntityGenerator.cs"); + + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Equal(0, result.ExitCode); + Assert.False(File.Exists(Owned("codegen/generators/FilterAllowlistGenerator.cs"))); + Assert.True(File.Exists(Owned("codegen/generators/EntityGenerator.cs"))); + Assert.Contains(result.Out, l => l.StartsWith("removed codegen/generators/FilterAllowlistGenerator.cs")); + Assert.DoesNotContain(result.Out, l => l.Contains("removed codegen/generators/EntityGenerator.cs")); + } + + [Fact] + public void It_prints_the_edit_that_points_the_entry_back_and_never_edits_Generators_cs() + { + Eject("filter-allowlist"); + var before = File.ReadAllText(Owned("codegen/Generators.cs")); + + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Equal(before, File.ReadAllText(Owned("codegen/Generators.cs"))); + var text = string.Join("\n", result.Out); + Assert.Contains("new Codegen.Generators.FilterAllowlistGenerator()", text); + Assert.Contains("new MetaObjects.Codegen.Generators.FilterAllowlistGenerator()", text); + } + + [Fact] + public void A_named_copy_that_differs_is_refused_with_exit_2_and_nothing_is_removed() + { + Eject("entity", "filter-allowlist"); + Edit("codegen/generators/EntityGenerator.cs"); + + var result = EjectCommand.Undo(["filter-allowlist", "entity"], _root, force: false); + + Assert.Equal(2, result.ExitCode); + Assert.True(File.Exists(Owned("codegen/generators/FilterAllowlistGenerator.cs"))); + Assert.True(File.Exists(Owned("codegen/generators/EntityGenerator.cs"))); + var err = string.Join("\n", result.Err); + Assert.Contains("entity DIFFERS", err); + Assert.Contains("--force", err); + } + + [Fact] + public void Force_removes_a_copy_that_differs() + { + Eject("entity"); + Edit("codegen/generators/EntityGenerator.cs"); + + var result = EjectCommand.Undo(["entity"], _root, force: true); + + Assert.Equal(0, result.ExitCode); + Assert.False(File.Exists(Owned("codegen/generators/EntityGenerator.cs"))); + } + + [Fact] + public void An_unknown_or_not_owned_name_refuses_the_whole_call() + { + Eject("filter-allowlist"); + + Assert.Equal(2, EjectCommand.Undo(["filter-allowlist", "no-such"], _root, force: false).ExitCode); + var notOwned = EjectCommand.Undo(["filter-allowlist", "entity"], _root, force: false); + Assert.Equal(2, notOwned.ExitCode); + Assert.Contains("entity", string.Join("\n", notOwned.Err)); + Assert.True(File.Exists(Owned("codegen/generators/FilterAllowlistGenerator.cs"))); + } + + [Fact] + public void Says_plainly_when_nothing_is_identical() + { + Eject("entity"); + Edit("codegen/generators/EntityGenerator.cs"); + + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Equal(0, result.ExitCode); + Assert.Contains("no owned copy is identical", string.Join("\n", result.Out)); + Assert.True(File.Exists(Owned("codegen/generators/EntityGenerator.cs"))); + } + + [Fact] + public void Says_plainly_when_nothing_is_owned() + { + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Equal(0, result.ExitCode); + Assert.Contains("no owned generator copies", string.Join("\n", result.Out)); + } + + [Fact] + public void Companions_go_with_the_generator_unless_a_remaining_owned_generator_uses_them() + { + var api = GeneratorRegistry.Get("api-docs")!; + Assert.NotEmpty(api.Companions); + Eject("api-docs"); + var companion = EjectableGenerators.OwnedRelativePath(api.Companions[0]); + Assert.True(File.Exists(Owned(companion))); + + Assert.Equal(0, EjectCommand.Undo([], _root, force: false).ExitCode); + Assert.False(File.Exists(Owned(companion))); + Assert.False(File.Exists(Owned("codegen/generators/ApiDocsGenerator.cs"))); + } + + [Fact] + public void An_edited_companion_is_kept_and_so_the_generator_is_not_identical() + { + var api = GeneratorRegistry.Get("api-docs")!; + Eject("api-docs"); + var companion = EjectableGenerators.OwnedRelativePath(api.Companions[0]); + Edit(companion); + + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Contains("no owned copy is identical", string.Join("\n", result.Out)); + Assert.True(File.Exists(Owned(companion))); + } + + [Fact] + public void Gen_notice_names_identical_copies_and_the_undo_command_and_is_silent_otherwise() + { + Assert.Null(OwnedCopy.IdenticalNotice(_root)); + + Eject("entity", "filter-allowlist"); + Edit("codegen/generators/EntityGenerator.cs"); + + var notice = OwnedCopy.IdenticalNotice(_root)!; + Assert.DoesNotContain('\n', notice); + Assert.Contains("filter-allowlist", notice); + Assert.DoesNotContain("entity", notice); + Assert.Contains("dotnet meta eject --undo", notice); + } + + [Fact] + public void A_changed_helper_runtime_keeps_its_generator_out_of_the_notice_and_out_of_no_name_undo() + { + Eject("routes"); + Assert.Contains("routes", OwnedCopy.IdenticalNotice(_root)!); + var runtimeFile = $"{HelperRuntime.OwnedDirectory}/{HelperRuntime.Files[0]}"; + Edit(runtimeFile); + + Assert.Null(OwnedCopy.IdenticalNotice(_root)); + var result = EjectCommand.Undo([], _root, force: false); + Assert.Equal(0, result.ExitCode); + Assert.True(File.Exists(Owned("codegen/generators/RoutesGenerator.cs"))); + Assert.True(File.Exists(Owned(runtimeFile))); + + var named = EjectCommand.Undo(["routes"], _root, force: false); + Assert.Equal(2, named.ExitCode); + Assert.Contains(HelperRuntime.Files[0], string.Join("\n", named.Err)); + Assert.True(File.Exists(Owned("codegen/generators/RoutesGenerator.cs"))); + } + + [Fact] + public void Undoing_the_last_runtime_user_removes_the_whole_identical_helper_runtime() + { + Eject("routes"); + + var result = EjectCommand.Undo([], _root, force: false); + + Assert.Equal(0, result.ExitCode); + Assert.False(Directory.Exists(Owned(HelperRuntime.OwnedDirectory))); + Assert.Contains(result.Out, l => l.StartsWith("The helper runtime copy is gone")); + } +} diff --git a/server/csharp/MetaObjects.Cli.Tests/GenUnknownAttrWarningTests.cs b/server/csharp/MetaObjects.Cli.Tests/GenUnknownAttrWarningTests.cs index ea7fa0ec6..6d7133cc7 100644 --- a/server/csharp/MetaObjects.Cli.Tests/GenUnknownAttrWarningTests.cs +++ b/server/csharp/MetaObjects.Cli.Tests/GenUnknownAttrWarningTests.cs @@ -48,6 +48,28 @@ public void Gen_warns_about_each_unknown_attribute_and_keeps_its_exit_code() Assert.Contains("dotnet meta verify", run.Stderr); } + [Fact] + public void Strict_gen_rejects_an_unknown_attribute_before_writing_anything() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.acme.json"), Typos); + var run = OwnedRunner.Run(_tmp, ["entity"], "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--strict"); + Assert.Equal(1, run.ExitCode); + Assert.Contains("ERR_UNKNOWN_ATTR", run.Stderr); + Assert.Contains("hint: " + CodegenCli.GenStrictHint, run.Stderr); + Assert.DoesNotContain("verify --lax", run.Stderr); + Assert.False(Directory.Exists(OutDir) && Directory.EnumerateFiles(OutDir, "*", SearchOption.AllDirectories).Any()); + } + + [Fact] + public void Strict_gen_on_a_clean_model_generates() + { + File.WriteAllText(Path.Combine(MetaDir, "meta.acme.json"), + Typos.Replace("\"@isAbstrakt\": true", "\"abstract\": true").Replace("@requird", "@required")); + var run = OwnedRunner.Run(_tmp, ["entity"], "gen", MetaDir, "--out", OutDir, "--namespace", "Acme", "--strict"); + Assert.Equal(0, run.ExitCode); + Assert.NotEmpty(Directory.EnumerateFiles(OutDir, "*.cs", SearchOption.AllDirectories)); + } + [Fact] public void A_clean_model_has_no_unknown_attribute_warnings() { diff --git a/server/csharp/MetaObjects.Cli.Tests/InitWithDocsTests.cs b/server/csharp/MetaObjects.Cli.Tests/InitWithDocsTests.cs new file mode 100644 index 000000000..175ec6937 --- /dev/null +++ b/server/csharp/MetaObjects.Cli.Tests/InitWithDocsTests.cs @@ -0,0 +1,52 @@ +// `dotnet meta init --with-docs` lists the api-docs generator in the scaffolded generator list. +// Without the flag nothing changes. + +using MetaObjects.Cli; +using Xunit; + +namespace MetaObjects.Cli.Tests; + +public sealed class InitWithDocsTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "meta-init-docs-" + Guid.NewGuid().ToString("N")); + private string List => File.ReadAllText(Path.Combine(_root, "codegen", "Generators.cs")); + + private IEnumerable ActiveEntries => List.Split('\n') + .Select(l => l.Trim()) + .SkipWhile(l => !l.StartsWith("public static IReadOnlyList All", StringComparison.Ordinal)) + .Where(l => l.StartsWith("new ", StringComparison.Ordinal)); + + public InitWithDocsTests() => Directory.CreateDirectory(_root); + public void Dispose() { try { Directory.Delete(_root, recursive: true); } catch { } } + + [Fact] + public void With_docs_lists_the_api_docs_generator_and_says_how_it_is_drift_checked() + { + var lines = InitCommand.Run(_root, withDocs: true); + + Assert.Contains(ActiveEntries, e => e.StartsWith("new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = \"docs/csharp\"", StringComparison.Ordinal)); + Assert.Contains("csharp", string.Join("\n", lines)); + Assert.Contains("outDir", string.Join("\n", lines)); + } + + [Fact] + public void Without_the_flag_the_list_is_unchanged() + { + InitCommand.Run(_root); + + Assert.Equal(CodegenScaffold.GeneratorsCs([]), List); + Assert.Empty(ActiveEntries); + } + + [Fact] + public void With_docs_never_edits_an_existing_list_and_prints_the_entry_to_add() + { + InitCommand.Run(_root); + var before = List; + + var lines = InitCommand.Run(_root, withDocs: true); + + Assert.Equal(before, List); + Assert.Contains(lines, l => l.Contains("ApiDocsGenerator { OutDir = \"docs/csharp\"")); + } +} diff --git a/server/csharp/MetaObjects.Cli/EjectCommand.cs b/server/csharp/MetaObjects.Cli/EjectCommand.cs index 7909ed3f4..c6be8be12 100644 --- a/server/csharp/MetaObjects.Cli/EjectCommand.cs +++ b/server/csharp/MetaObjects.Cli/EjectCommand.cs @@ -247,6 +247,117 @@ public static Outcome Run(IReadOnlyList names, string root, bool force) return new Outcome(0, outLines, []); } + /// + /// eject --undo: drop owned generator copies that are identical to the shipped ones. With no + /// every identical owned copy goes; with names only those, and a copy that + /// DIFFERS is refused (exit 2, nothing removed) unless . Every name is + /// validated before anything is deleted. Companion emitters go only when identical (or forced) and + /// no remaining owned generator is handed them. The helper runtime counts toward a copy being + /// identical, so it goes whole when no remaining owned generator's output imports it. It never edits + /// codegen/Generators.cs or the project: it prints the edit that points each entry back at the + /// shipped generator. + /// + public static Outcome Undo(IReadOnlyList names, string root, bool force) + { + var all = EjectableGenerators.Entries(); + List targets; + if (names.Count == 0) + { + targets = OwnedCopy.IdenticalCopies(root).ToList(); + if (targets.Count == 0) + { + var owned = all.Where(e => OwnedCopy.Status(root, e) is not null).ToList(); + return new Outcome(0, [owned.Count == 0 + ? "no owned generator copies under codegen/generators/; nothing to undo." + : "no owned copy is identical to its shipped generator, so nothing was removed: " + + string.Join("; ", owned.Select(e => $"{e.Name} {OwnedCopy.UndoStatus(root, e)}")) + "."], []); + } + } + else + { + var known = all.ToDictionary(e => e.Name, e => e, StringComparer.Ordinal); + var unknown = names.Where(n => !known.ContainsKey(n)).ToList(); + if (unknown.Count > 0) + return Outcome.Usage( + $"unknown or non-ejectable generator(s): {string.Join(", ", unknown)}. " + + $"Nothing was removed. Ejectable: {string.Join(", ", EjectableNames())}."); + targets = names.Distinct(StringComparer.Ordinal).Select(n => known[n]).ToList(); + var notOwned = targets.Where(e => OwnedCopy.Status(root, e) is null).Select(e => e.Name).ToList(); + if (notOwned.Count > 0) + return Outcome.Usage( + $"no owned copy of: {string.Join(", ", notOwned)} under codegen/generators/. Nothing was removed."); + var differs = targets.Where(e => OwnedCopy.UndoStatus(root, e) != "identical").ToList(); + if (differs.Count > 0 && !force) + return Outcome.Usage( + "not removed (nothing was changed): " + + string.Join("; ", differs.Select(e => $"{e.Name} {OwnedCopy.UndoStatus(root, e)}")) + + ". Undo drops only copies identical to the shipped generator; `dotnet meta gen --list` " + + "shows how each one differs. Pass --force to discard your edits."); + } + + var generatorsDir = Path.Combine(root, "codegen", "generators"); + var gone = targets.Select(t => t.Name).ToHashSet(StringComparer.Ordinal); + var remaining = all.Where(e => !gone.Contains(e.Name) && OwnedCopy.Status(root, e) is not null).ToList(); + var lines = new List(); + + foreach (var t in targets) + { + File.Delete(Path.Combine(generatorsDir, t.SourceFileName!)); + lines.Add($"removed codegen/generators/{t.SourceFileName} (\"{t.Name}\")"); + } + foreach (var sourcePath in targets.SelectMany(t => t.Companions).Distinct(StringComparer.Ordinal)) + { + var relative = EjectableGenerators.OwnedRelativePath(sourcePath); + var path = Path.Combine(root, relative.Replace('/', Path.DirectorySeparatorChar)); + if (!File.Exists(path)) continue; + if (remaining.FirstOrDefault(e => e.Companions.Contains(sourcePath)) is { } user) + lines.Add($" kept {relative} — \"{user.Name}\" is still owned and uses it"); + else if (!force && OwnedCopy.Compare(File.ReadAllText(path), + EjectableGenerators.RewriteForEject(EjectableGenerators.ReadSourcePath(sourcePath))).Verdict + != OwnedCopy.Verdict.Identical) + lines.Add($" kept {relative} — it differs from the shipped one, so it is yours"); + else + { + File.Delete(path); + lines.Add($" removed {relative}"); + } + } + + var runtimeUsers = targets.Where(t => HelperRuntime.UsedBy(EjectableGenerators.ReadSource(t.Name)!)).ToList(); + var runtimeDir = Path.Combine(root, HelperRuntime.OwnedDirectory.Replace('/', Path.DirectorySeparatorChar)); + var runtimeRemoved = false; + if (runtimeUsers.Count > 0 && Directory.Exists(runtimeDir)) + { + if (remaining.FirstOrDefault(e => HelperRuntime.UsedBy(EjectableGenerators.ReadSource(e.Name)!)) is { } user) + lines.Add($" kept {HelperRuntime.OwnedDirectory}/ — \"{user.Name}\" is still owned and uses it"); + else + { + foreach (var file in HelperRuntime.Files) + { + var path = Path.Combine(runtimeDir, file); + if (!File.Exists(path)) continue; + File.Delete(path); + lines.Add($" removed {HelperRuntime.OwnedDirectory}/{file}"); + runtimeRemoved = true; + } + if (!Directory.EnumerateFileSystemEntries(runtimeDir).Any()) Directory.Delete(runtimeDir); + } + } + if (Directory.Exists(generatorsDir) && !Directory.EnumerateFileSystemEntries(generatorsDir).Any()) + Directory.Delete(generatorsDir); + + lines.Add(""); + lines.Add("codegen/Generators.cs is yours — undo never edits it. Point each entry back at the shipped " + + "generator (the project does not compile until you do):"); + foreach (var t in targets) + lines.Add($" replace {EjectableGenerators.OwnedListEntry(t)} with {GeneratorRegistry.ListEntry(t)}"); + if (runtimeRemoved) + lines.Add($"The helper runtime copy is gone: drop the `` " + + $"line from the project that compiled it, and register the timestamp converter from " + + $"`{HelperRuntime.PackagedNamespace}` again."); + return new Outcome(0, lines, []); + } + /// /// Copy the emitters the ejected write their output through /// () next to the generators, each once however diff --git a/server/csharp/MetaObjects.Cli/InitCommand.cs b/server/csharp/MetaObjects.Cli/InitCommand.cs index f2dcbb7f7..4b1483eb2 100644 --- a/server/csharp/MetaObjects.Cli/InitCommand.cs +++ b/server/csharp/MetaObjects.Cli/InitCommand.cs @@ -121,11 +121,21 @@ public static IReadOnlyList WriteMissing(string root, IEnumerableThe init command's pure logic: write what is missing and report it. public static class InitCommand { - public static IReadOnlyList Run(string root) + /// The docs directory and the entry init --with-docs lists. + public const string DocsOutDir = "docs/csharp"; + + private static string DocsEntry(string root) => + "new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = \"" + DocsOutDir + "\", ProjectName = \"" + + (Path.GetFileName(Path.GetFullPath(root).TrimEnd(Path.DirectorySeparatorChar, Path.AltDirectorySeparatorChar)) is { Length: > 0 } n + ? n.Replace("\\", "").Replace("\"", "") : "project") + "\" }"; + + public static IReadOnlyList Run(string root, bool withDocs = false) { var csprojPath = Path.Combine(root, "codegen", "Codegen.csproj"); var csprojExisted = File.Exists(csprojPath); - var written = CodegenScaffold.WriteMissing(root, []); + var docsEntry = DocsEntry(root); + var listExisted = File.Exists(Path.Combine(root, "codegen", "Generators.cs")); + var written = CodegenScaffold.WriteMissing(root, withDocs ? [docsEntry] : []); var lines = new List(); foreach (var rel in CodegenScaffold.Files) lines.Add(written.Contains(rel) ? $" wrote {rel}" : $" kept {rel} (already yours)"); @@ -136,6 +146,21 @@ public static IReadOnlyList Run(string root) lines.Add( "codegen/Generators.cs is the generator list `dotnet meta gen` runs. It declares nothing yet, " + "so gen writes nothing until you add entries: `dotnet meta gen --list` is the catalog."); + if (withDocs) + { + if (listExisted) + { + lines.Add("codegen/Generators.cs already exists and is yours — init never edits it. Add:"); + lines.Add($" {docsEntry},"); + lines.Add("to the list there."); + } + else + lines.Add($"codegen/Generators.cs lists the api-docs generator, which writes {DocsOutDir}/api/csharp/."); + lines.Add( + $"`dotnet meta gen` writes {DocsOutDir}/; `dotnet meta verify` drift-checks it once " + + "\"csharp\": { \"outDir\": \"\" } is set in .metaobjects/config.json (init does not write that " + + $"file); keep {DocsOutDir}/ generated files only, verify compares the whole tree."); + } if (csprojExisted) lines.AddRange(EjectCommand.OlderPinWarning(csprojPath, "init")); return lines; } diff --git a/server/csharp/MetaObjects.Cli/OwnedCopy.cs b/server/csharp/MetaObjects.Cli/OwnedCopy.cs index 50a9203e5..309ab59d3 100644 --- a/server/csharp/MetaObjects.Cli/OwnedCopy.cs +++ b/server/csharp/MetaObjects.Cli/OwnedCopy.cs @@ -127,4 +127,41 @@ public static Comparison Compare(string owned, string reference) } return clauses.Count == 0 ? "identical" : string.Join("; ", clauses); } + + /// + /// , plus a "codegen/runtime/: …" clause when the generator's output + /// imports the helper runtime and the owned copy of it is not identical. Undo reads this: the + /// shipped generator imports the shipped runtime, so dropping such a copy would silently stop + /// the runtime edit reaching the generated code. + /// + public static string? UndoStatus(string root, MetaObjects.Codegen.GeneratorRegistryEntry entry) + { + var status = Status(root, entry); + if (status is null + || !MetaObjects.Codegen.HelperRuntime.UsedBy(MetaObjects.Codegen.EjectableGenerators.ReadSource(entry.Name)!) + || RuntimeStatus(root) is not { } runtime || runtime == "identical") + return status; + var clause = $"{MetaObjects.Codegen.HelperRuntime.OwnedDirectory}/: {runtime}"; + return status == "identical" ? clause : $"{status}; {clause}"; + } + + /// Every ejectable entry whose owned copy, the emitters eject handed over with it and the + /// helper runtime its output imports, is identical to the shipped one ( + /// reads identical). + public static IReadOnlyList IdenticalCopies(string root) => + MetaObjects.Codegen.EjectableGenerators.Entries().Where(e => UndoStatus(root, e) == "identical").ToList(); + + /// + /// The one advisory line dotnet meta gen prints when owned copies identical to the shipped + /// generators exist (they carry no change, only the upkeep of owning them), naming them and the + /// command that drops them. null when there are none, so a project without them pays nothing. + /// + public static string? IdenticalNotice(string root) + { + var names = IdenticalCopies(root).Select(e => e.Name).ToList(); + return names.Count == 0 + ? null + : $"note: owned generator copies identical to the shipped ones: {string.Join(", ", names)}. " + + "`dotnet meta eject --undo` drops them (it prints the codegen/Generators.cs edit; it changes nothing else)."; + } } diff --git a/server/csharp/MetaObjects.Cli/Program.cs b/server/csharp/MetaObjects.Cli/Program.cs index 488bc519c..eac1ca106 100644 --- a/server/csharp/MetaObjects.Cli/Program.cs +++ b/server/csharp/MetaObjects.Cli/Program.cs @@ -27,12 +27,15 @@ (helpRequested ? Console.Out : Console.Error).WriteLine( "usage: dotnet meta [options]\n" + " commands:\n" + - " init [--root ] write codegen/Generators.cs (the generator list),\n" + - " codegen/Program.cs and codegen/Codegen.csproj\n" + + " init [--root ] [--with-docs] write codegen/Generators.cs (the generator list),\n" + + " codegen/Program.cs and codegen/Codegen.csproj;\n" + + " --with-docs lists the api-docs generator in it\n" + " gen [] --out [--namespace ] [--emit-abstract-shapes]\n" + - " [--generators ] [--template-root ]\n" + + " [--generators ] [--template-root ] [--strict]\n" + " run the generator list in codegen/Generators.cs;\n" + - " --generators narrows it to the entries named\n" + + " --generators narrows it to the entries named;\n" + + " --strict rejects an unknown @attr before writing\n" + + " (as verify does, ADR-0023; default is a warning)\n" + " gen --list list available generators (stable names) and exit\n" + " verify [--templates [--prompts ]] [--codegen --out [--namespace ]] [--db] [--lax]\n" + " drift gates (ADR-0021 D2 subverbs):\n" + @@ -56,6 +59,13 @@ " eject ... [--force] [--root ] copy a reference generator into\n" + " codegen/generators/ to own (ADR-0034);\n" + " see `dotnet meta gen --list`\n" + + " eject --undo [...] [--force] [--root ]\n" + + " drop owned copies identical to the shipped\n" + + " generators (all of them, or the names given;\n" + + " a copy that differs needs --force). It prints\n" + + " the Generators.cs edit and never makes it.\n" + + " There is no `doctor` command: gen's notice\n" + + " and `gen --list` show identical copies\n" + " fmt [] [--check] rewrite metadata into canonical form (#304);\n" + " --check lists drift, exits non-zero, changes nothing\n" + " agent-docs see `npx meta agent-docs`\n" + @@ -131,6 +141,11 @@ static int RunGen(string[] rest) : Directory.GetCurrentDirectory(); if (RequireGeneratorList(projectRoot, "dotnet meta gen") is { } refused) return refused; + // Advisory: owned copies identical to the shipped generators carry nothing of the adopter's. + // One line on stderr, free when there are none; never changes the exit code. + try { if (OwnedCopy.IdenticalNotice(projectRoot) is { } notice) Console.Error.WriteLine(notice); } + catch (Exception) { /* advisory only: never fail a gen over the notice */ } + // Advisory: nudge a re-scaffold if the copied-in agent context predates this build. // Never throws, never changes the exit code (a missing/corrupt manifest is ignored). AgentContextStalenessCheck.WarnIfStale(Directory.GetCurrentDirectory()); @@ -179,12 +194,14 @@ static int RemovedDocs() static int RunInit(string[] rest) { string? root = null; + bool withDocs = false; for (int i = 0; i < rest.Length; i++) { if (rest[i] == "--root" && i + 1 < rest.Length) root = rest[++i]; - else return RefuseFlag("init", rest[i], ["--root"], []); + else if (rest[i] == "--with-docs") withDocs = true; + else return RefuseFlag("init", rest[i], ["--root"], ["--with-docs"]); } - var result = InitCommand.Run(root ?? Directory.GetCurrentDirectory()); + var result = InitCommand.Run(root ?? Directory.GetCurrentDirectory(), withDocs); foreach (var line in result) Console.WriteLine(line); return 0; } @@ -273,17 +290,20 @@ static int RunFmt(string[] rest) static int RunEject(string[] rest) { var names = new List(); - bool force = false; + bool force = false, undo = false; string? root = null; for (int i = 0; i < rest.Length; i++) { if (rest[i] == "--force") force = true; + else if (rest[i] == "--undo") undo = true; else if (rest[i] == "--root" && i + 1 < rest.Length) root = rest[++i]; else if (!rest[i].StartsWith('-')) names.Add(rest[i]); - else return RefuseFlag("eject", rest[i], ["--root"], ["--force"]); + else return RefuseFlag("eject", rest[i], ["--root"], ["--force", "--undo"]); } - var result = EjectCommand.Run(names, root ?? Directory.GetCurrentDirectory(), force); + var result = undo + ? EjectCommand.Undo(names, root ?? Directory.GetCurrentDirectory(), force) + : EjectCommand.Run(names, root ?? Directory.GetCurrentDirectory(), force); foreach (var line in result.Out) Console.WriteLine(line); foreach (var line in result.Err) Console.Error.WriteLine($"error: {line}"); return result.ExitCode; @@ -608,7 +628,7 @@ partial class Program { static readonly string[] GenValueFlags = ["--out", "--namespace", "--generators", "--template-root", "--template-spec", "--column-naming", "--baseline"]; - static readonly string[] GenBoolFlags = ["--list", "--emit-abstract-shapes"]; + static readonly string[] GenBoolFlags = ["--list", "--emit-abstract-shapes", "--strict"]; static readonly string[] VerifyValueFlags = ["--prompts", "--out", "--namespace", "--generators", "--template-root", "--column-naming"]; static readonly string[] VerifyBoolFlags = ["--templates", "--codegen", "--db", "--lax", "--no-field-lint", "--require-implementers"]; @@ -616,14 +636,23 @@ partial class Program /// Each command's usage, printed by dotnet meta <command> --help. static readonly Dictionary CommandUsage = new() { - ["gen"] = "usage: dotnet meta gen [] --out [--namespace ] [--generators ] [--template-root ] [--emit-abstract-shapes] [--baseline default|adopt]\n" + + ["gen"] = "usage: dotnet meta gen [] --out [--namespace ] [--generators ] [--template-root ] [--emit-abstract-shapes] [--baseline default|adopt] [--strict]\n" + " dotnet meta gen --list\n" + " runs the generator list in codegen/Generators.cs (`dotnet meta init` writes it);\n" + " --generators narrows that list to the entries named, and never adds one.\n" + + " --strict: an unknown @attr is an error (exit 1, nothing written), as in verify (ADR-0023);\n" + + " without it gen warns and generates. Owned copies identical to the shipped generators\n" + + " get one advisory line naming `dotnet meta eject --undo`.\n" + " column/table naming: \"naming\" in .metaobjects/config.json (ADR-0058), not a flag here", - ["init"] = "usage: dotnet meta init [--root ]", + ["init"] = "usage: dotnet meta init [--root ] [--with-docs]\n" + + " --with-docs lists the api-docs generator in codegen/Generators.cs (written to docs/csharp)", ["verify"] = "usage: dotnet meta verify [--templates [--prompts ]] [--codegen --out [--namespace ] [--generators ] [--template-root ]] [--db] [--lax] [--no-field-lint] [--require-implementers]", ["fmt"] = "usage: dotnet meta fmt [] [--check]", - ["eject"] = "usage: dotnet meta eject ... [--force] [--root ]", + ["eject"] = "usage: dotnet meta eject ... [--force] [--root ]\n" + + " dotnet meta eject --undo [...] [--force] [--root ]\n" + + " --undo drops owned copies identical to the shipped generators (every one, or the names given);\n" + + " a copy that differs is refused (exit 2) unless --force. It prints the codegen/Generators.cs edit\n" + + " that points the entry back at the shipped generator and never edits the project.\n" + + " There is no `doctor` command: gen's notice and `dotnet meta gen --list` show identical or stale copies.", }; } diff --git a/server/csharp/MetaObjects.Cli/VerifyCommand.cs b/server/csharp/MetaObjects.Cli/VerifyCommand.cs index 2c775f05e..34d3bc89f 100644 --- a/server/csharp/MetaObjects.Cli/VerifyCommand.cs +++ b/server/csharp/MetaObjects.Cli/VerifyCommand.cs @@ -66,11 +66,7 @@ public sealed record Outcome( /// Actionable hint (#96 / ADR-0023) printed when a strict load rejects an /// undeclared own @attr (ERR_UNKNOWN_ATTR) — names the three exits. /// - public const string UNKNOWN_ATTR_HINT = - "verify loads strict by default (ADR-0023): an unregistered or typo'd @attr is " + - "ERR_UNKNOWN_ATTR. To fix: register the attr on a metadata provider, OR move " + - "arbitrary author-supplied properties into an `attr.properties` bag, OR run " + - "'dotnet meta verify --lax' to keep the legacy open-attr load."; + public const string UNKNOWN_ATTR_HINT = MetaObjects.Codegen.CodegenCli.UnknownAttrHint; /// The one-line note bare verify prints advertising the subverbs. public const string SUBVERB_NOTE = diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiDocsCrossPortConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiDocsCrossPortConformanceTests.cs index 884ae66c3..9ac6520fb 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ApiDocsCrossPortConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiDocsCrossPortConformanceTests.cs @@ -38,6 +38,11 @@ private static string RepoRoot() return dir; } + // The C# port writes no model pages, so the back-link exists only federated to a base URL + // (ApiDocsGenerator.ModelBaseUrl); the manifest's relative apiCsharpToModel is the path math + // CSharpDocsPathsTests pins. Here the link is /. + private const string ModelBase = "https://docs.example.com/model"; + private static string CaseDir() => Path.Combine(RepoRoot(), "fixtures", "conformance", CaseName); [Fact] @@ -60,7 +65,7 @@ public void CsharpApiDocsSurface_MatchesTheSharedManifest() // ---- run the api-docs entry as gen runs it ---- var outcome = CodegenCli.RunGen( MetaObjects.Loader.MetaDataLoader.FromDirectory(inputDir), Path.Combine(tmp, "code"), "Generated", false, - [new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = outDir }], tmp); + [new MetaObjects.Codegen.Generators.ApiDocsGenerator { OutDir = outDir, ModelBaseUrl = ModelBase }], tmp); Assert.True(outcome.Ok, string.Join("\n", outcome.LoadErrors)); // ---- per-unit: the api/csharp page must exist AND carry the model back-link ---- @@ -68,7 +73,7 @@ public void CsharpApiDocsSurface_MatchesTheSharedManifest() { var node = unit.GetProperty("node").GetString()!; var apiCsharpPath = unit.GetProperty("apiCsharpPath").GetString()!; - var apiCsharpToModel = unit.GetProperty("apiCsharpToModel").GetString()!; + var apiCsharpToModel = ModelBase + "/" + unit.GetProperty("modelPath").GetString()!; var pageFile = Path.Combine(outDir, apiCsharpPath.Replace('/', Path.DirectorySeparatorChar)); diff --git a/server/csharp/MetaObjects.Codegen.Tests/ApiDocsRunTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ApiDocsRunTests.cs index 7416ae39d..4ac1cb114 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ApiDocsRunTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ApiDocsRunTests.cs @@ -88,6 +88,19 @@ public void The_entry_options_reach_the_pages() .Contains("https://docs.example.com/model/", StringComparison.Ordinal)); } + [Fact] + public void Without_a_model_base_url_the_pages_carry_no_model_link() + { + // The C# port writes no model pages, so a relative link would resolve to nothing. + var outcome = Gen(new ApiDocsGenerator { OutDir = Docs }); + + Assert.True(outcome.Ok, string.Join("\n", outcome.LoadErrors)); + var pages = FilesUnder(Docs).Where(p => p.StartsWith("api/csharp/") && !p.EndsWith("README.md") && !p.EndsWith("AGENT-API.md")).ToList(); + Assert.NotEmpty(pages); + Assert.All(pages, p => Assert.DoesNotContain("Model / metadata", + File.ReadAllText(Path.Combine(Docs, p.Replace('/', Path.DirectorySeparatorChar))))); + } + [Fact] public void Code_and_documentation_in_one_list_each_land_in_their_own_directory() { diff --git a/server/csharp/MetaObjects.Codegen.Tests/GeneratorAuthoringExampleTests.cs b/server/csharp/MetaObjects.Codegen.Tests/GeneratorAuthoringExampleTests.cs new file mode 100644 index 000000000..dce607ae8 --- /dev/null +++ b/server/csharp/MetaObjects.Codegen.Tests/GeneratorAuthoringExampleTests.cs @@ -0,0 +1,39 @@ +// The generator in docs/ports/csharp.md, "Write your own generator", compiled and run. If the +// document's example stops compiling or stops writing a file, this fails. + +using MetaObjects.Loader; +using Xunit; + +namespace MetaObjects.Codegen.Tests; + +// --- the example, as the document shows it (codegen/generators/EntityList.cs) --- +public sealed class EntityListGenerator : IGenerator +{ + public string Name => "entity-list"; + + public IEnumerable Generate(GenContext ctx) + { + var names = ctx.Entities.Select(e => e.Name).Order(StringComparer.Ordinal); + yield return new EmittedFile("EntityList.g.cs", + "// \npublic static class EntityList\n{\n" + + " public static readonly string[] Names = [" + string.Join(", ", names.Select(n => $"\"{n}\"")) + "];\n}\n"); + } +} + +public sealed class GeneratorAuthoringExampleTests : IDisposable +{ + private readonly string _root = Path.Combine(Path.GetTempPath(), "meta-authoring-" + Guid.NewGuid().ToString("N")); + public GeneratorAuthoringExampleTests() => Directory.CreateDirectory(_root); + public void Dispose() { try { Directory.Delete(_root, recursive: true); } catch { } } + + [Fact] + public void The_documented_generator_runs_and_writes_its_file() + { + var load = new MetaDataLoader().Load([new FileSource(CorpusPaths.FitnessMetadata)]); + var outcome = CodegenCli.RunGen(load, Path.Combine(_root, "out"), "Acme", false, [new EntityListGenerator()], _root); + + Assert.True(outcome.Ok, string.Join("\n", outcome.LoadErrors)); + var text = File.ReadAllText(Path.Combine(_root, "out", "EntityList.g.cs")); + Assert.Contains("public static class EntityList", text); + } +} diff --git a/server/csharp/MetaObjects.Codegen/CodegenCli.cs b/server/csharp/MetaObjects.Codegen/CodegenCli.cs index a9a2f9185..d4eb86f36 100644 --- a/server/csharp/MetaObjects.Codegen/CodegenCli.cs +++ b/server/csharp/MetaObjects.Codegen/CodegenCli.cs @@ -105,8 +105,24 @@ public static void ReportFailure(GenOutcome outcome, string command) : $"{command}: FAILED (metadata did not load cleanly)"); } - /// Trailing advice after gen's unknown-attribute warnings. gen has - /// no strict flag of its own; verify is the strict door (ADR-0023). + /// The hint verify and gen --strict print after an ERR_UNKNOWN_ATTR: the three + /// exits (ADR-0023). + public const string UnknownAttrHint = + "verify loads strict by default (ADR-0023): an unregistered or typo'd @attr is " + + "ERR_UNKNOWN_ATTR. To fix: register the attr on a metadata provider, OR move " + + "arbitrary author-supplied properties into an `attr.properties` bag, OR run " + + "'dotnet meta verify --lax' to keep the legacy open-attr load."; + + /// The hint gen --strict prints after an ERR_UNKNOWN_ATTR: gen's own way out is + /// to drop the flag, not verify --lax (ADR-0023). + public const string GenStrictHint = + "gen --strict loads strictly, as verify does (ADR-0023): an unregistered or typo'd @attr is " + + "ERR_UNKNOWN_ATTR and nothing was written. To fix: register the attr on a metadata provider, " + + "OR move arbitrary author-supplied properties into an `attr.properties` bag, " + + "OR drop --strict."; + + /// Trailing advice after gen's unknown-attribute warnings. Without --strict + /// gen loads leniently; gen --strict and verify are the strict doors (ADR-0023). public const string UnknownAttrGenAdvice = "gen loads leniently and generated anyway, but `dotnet meta verify` rejects this metadata " + "(ADR-0023): fix or remove the attribute - a typo'd one (`isAbstrakt`, `requird`) silently " + @@ -441,7 +457,7 @@ public static int Run(string[] args, IReadOnlyList generators) { Console.Error.WriteLine( "usage: gen [] --out [--namespace ] [--generators ]\n" + - " [--emit-abstract-shapes] [--baseline default|adopt] [--template-root ]\n" + + " [--emit-abstract-shapes] [--strict] [--baseline default|adopt] [--template-root ]\n" + " | verify --codegen [] --out [--namespace ] [--generators ]\n" + " [--template-root ] [--lax]\n" + " (column/table naming: \"naming\" in .metaobjects/config.json — ADR-0058)"); @@ -521,7 +537,7 @@ public static IReadOnlyList ComposeGenerators( private static readonly string[] GenValueFlags = ["--out", "--namespace", "--generators", "--template-root", "--template-spec", "--column-naming", "--baseline"]; - private static readonly string[] GenBoolFlags = ["--emit-abstract-shapes"]; + private static readonly string[] GenBoolFlags = ["--emit-abstract-shapes", "--strict"]; private static readonly string[] VerifyValueFlags = ["--out", "--namespace", "--generators", "--template-root", "--column-naming"]; private static readonly string[] VerifyBoolFlags = ["--codegen", "--lax"]; @@ -532,7 +548,7 @@ private sealed class OwnedArgs public string? MetadataDir, OutDir, TemplateRoot, TemplateSpecPath, ColumnNamingRaw, GeneratorsCsv; public string Namespace = "Generated"; public string Baseline = "default"; - public bool NamespaceExplicit, EmitAbstractShapes, Codegen, Lax; + public bool NamespaceExplicit, EmitAbstractShapes, Codegen, Lax, Strict; /// 1.0 docs flags given (), in order. public List LegacyDocs = []; /// The usage error the arguments earn, or null. @@ -581,6 +597,7 @@ public static OwnedArgs Parse(string[] rest, string command, string[] valueFlags if (flag == "--emit-abstract-shapes") a.EmitAbstractShapes = true; else if (flag == "--codegen") a.Codegen = true; else if (flag == "--lax") a.Lax = true; + else if (flag == "--strict") a.Strict = true; } else { @@ -725,8 +742,26 @@ private static int RunGenArgs(string[] rest, IReadOnlyList declared) Console.Error.WriteLine($"warning: {w}"); // ADR-0023: gen loads leniently; name what `verify` would reject instead of staying silent. - WarnUnknownAttrs(meta); - var outcome = RunGen(meta.Load(), a.OutDir, a.Namespace, a.EmitAbstractShapes, suite, projectRoot, a.Baseline, + // With --strict it loads as verify does and refuses before any file is written. + LoadResult genLoad; + if (a.Strict) + { + genLoad = meta.Load(strict: true); + if (genLoad.Errors.Count > 0) + { + foreach (var e in genLoad.Errors) Console.Error.WriteLine($" load error: {e.Code}: {e.Message}"); + if (genLoad.Errors.Any(e => e.Code == ErrorCode.ERR_UNKNOWN_ATTR)) + Console.Error.WriteLine($" hint: {GenStrictHint}"); + Console.Error.WriteLine($"{GenCommandName}: FAILED (metadata did not load cleanly; --strict)"); + return 1; + } + } + else + { + WarnUnknownAttrs(meta); + genLoad = meta.Load(); + } + var outcome = RunGen(genLoad, a.OutDir, a.Namespace, a.EmitAbstractShapes, suite, projectRoot, a.Baseline, includeNames: declared.Any(g => g.Name == "names"), // Only a run of the whole declared list can tell a stale file from one the // narrowing left out. diff --git a/server/csharp/MetaObjects.Codegen/Generators/ApiDocsGenerator.cs b/server/csharp/MetaObjects.Codegen/Generators/ApiDocsGenerator.cs index a0cf2ac07..822dbaaab 100644 --- a/server/csharp/MetaObjects.Codegen/Generators/ApiDocsGenerator.cs +++ b/server/csharp/MetaObjects.Codegen/Generators/ApiDocsGenerator.cs @@ -31,7 +31,9 @@ public sealed class ApiDocsGenerator : IGenerator /// The label the agent page carries in its header. public string ProjectName { get; init; } = "project"; - /// When set, the base URL the model pages' back-links federate to. + /// The base URL the model pages' back-links point at. The C# port writes no model pages of + /// its own, so the "Model / metadata" link is emitted only when this is set; without it the pages + /// carry no such link rather than one that resolves to nothing. public string? ModelBaseUrl { get; init; } public IEnumerable Generate(GenContext ctx) @@ -43,7 +45,9 @@ public IEnumerable Generate(GenContext ctx) foreach (var unit in model.Units) { var pagePath = DocsPaths.DocPageOutputPath(layout, unit.Package, unit.Node); - var modelHref = DocsPaths.ModelCrossHref(ApiSubDir + "/" + pagePath, pagePath, ModelBaseUrl); + var modelHref = string.IsNullOrEmpty(ModelBaseUrl) + ? null + : DocsPaths.ModelCrossHref(ApiSubDir + "/" + pagePath, pagePath, ModelBaseUrl); yield return Page(pagePath, renderer.RenderUnitPage(unit, modelHref)); } yield return Page("README.md", renderer.RenderIndex(model, layout)); diff --git a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/apidocs/KotlinApiDocsGenerator.kt b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/apidocs/KotlinApiDocsGenerator.kt index 5e47f4b4e..597ed9ec3 100644 --- a/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/apidocs/KotlinApiDocsGenerator.kt +++ b/server/java/codegen-kotlin/src/main/kotlin/com/metaobjects/generator/kotlin/apidocs/KotlinApiDocsGenerator.kt @@ -47,7 +47,10 @@ class KotlinApiDocsGenerator : FileEmittingGenerator() { } for (unit in model.units) { val pagePath = DocsPaths.docPageOutputPath(layout, unit.pkg, unit.node) - val modelHref = DocsPaths.modelCrossHref("$subDir/$pagePath", pagePath, modelBaseUrl) + // No model pages are written beside the api pages, so a relative link would be dead: + // the link is emitted only against a that hosts them. + val modelHref = if (modelBaseUrl.isNullOrEmpty()) null + else DocsPaths.modelCrossHref("$subDir/$pagePath", pagePath, modelBaseUrl) add(pagePath, renderer.renderUnitPage(unit, modelHref)) } add("README.md", renderer.renderIndex(model, layout)) @@ -65,7 +68,7 @@ class KotlinApiDocsGenerator : FileEmittingGenerator() { /** Arg: the page layout, `flat` (one file per unit, the default) or `package`. */ const val ARG_LAYOUT: String = "layout" - /** Arg: when set, the base URL the pages' links to the model pages are written against. */ + /** Arg: the base URL the pages' links to the model pages are written against; with none set, the pages carry no such link. */ const val ARG_MODEL_BASE_URL: String = "modelBaseUrl" /** The api-surface subdirectory of the cross-port docs contract. */ diff --git a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/apidocs/JavaApiDocsGenerator.java b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/apidocs/JavaApiDocsGenerator.java index ef7f2764c..b680088f0 100644 --- a/server/java/codegen-spring/src/main/java/com/metaobjects/generator/apidocs/JavaApiDocsGenerator.java +++ b/server/java/codegen-spring/src/main/java/com/metaobjects/generator/apidocs/JavaApiDocsGenerator.java @@ -41,7 +41,7 @@ public final class JavaApiDocsGenerator extends FileEmittingGenerator { /** Arg: the page layout, {@code flat} (one file per unit, the default) or {@code package}. */ public static final String ARG_LAYOUT = "layout"; - /** Arg: when set, the base URL the pages' links to the model pages are written against. */ + /** Arg: the base URL the pages' links to the model pages are written against; with none set, the pages carry no such link. */ public static final String ARG_MODEL_BASE_URL = "modelBaseUrl"; /** The api-surface subdirectory of the cross-port docs contract. */ @@ -67,7 +67,10 @@ protected List generate(MetaDataLoader loader) { Set paths = new HashSet<>(); for (ApiUnit unit : model.units()) { String pagePath = DocsPaths.docPageOutputPath(layout, unit.pkg(), unit.node()); - String modelHref = DocsPaths.modelCrossHref(subDir + "/" + pagePath, pagePath, modelBaseUrl); + // No model pages are written beside the api pages, so a relative link would be dead: + // the link is emitted only against a that hosts them. + String modelHref = modelBaseUrl == null || modelBaseUrl.isEmpty() ? null + : DocsPaths.modelCrossHref(subDir + "/" + pagePath, pagePath, modelBaseUrl); add(pages, paths, subDir, pagePath, renderer.renderUnitPage(unit, modelHref)); } add(pages, paths, subDir, "README.md", renderer.renderIndex(model, layout)); diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataEjectMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataEjectMojo.java index 1e3b5c2a6..a3bf34d19 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataEjectMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataEjectMojo.java @@ -118,7 +118,12 @@ public class MetaDataEjectMojo extends AbstractMojo { @Parameter(property = "runtimeDir") String runtimeDir; - private static final String CODEGEN_DIR = "codegen"; + /** Remove owned copies identical to the shipped generator instead of ejecting (see + * {@link #undo()}). With {@code -Dnames} only those; without, every identical copy. */ + @Parameter(property = "undo", defaultValue = "false") + boolean undo; + + static final String CODEGEN_DIR = "codegen"; @Override public void execute() throws MojoExecutionException, MojoFailureException { @@ -126,6 +131,10 @@ public void execute() throws MojoExecutionException, MojoFailureException { getLog().info(buildListOutput()); return; } + if (undo) { + undo(); + return; + } if (names == null || names.isBlank()) { throw new MojoFailureException( @@ -243,6 +252,107 @@ public void execute() throws MojoExecutionException, MojoFailureException { } } + // ------------------------------------------------------------------------------------ + // undo + // ------------------------------------------------------------------------------------ + + /** + * {@code -Dundo=true}: delete the owned copies that are identical to the shipped generator, so + * the build runs the stock one again. Without {@code -Dnames} that is every identical copy; + * with names, those only, and a copy that differs is refused with nothing removed unless + * {@code -Dforce=true}. Every name is validated before anything is deleted. It never edits + * {@code pom.xml}: it prints the {@code } to put back. The shared support and + * runtime copies stay (other owned generators import them) and are named. + */ + private void undo() throws MojoExecutionException, MojoFailureException { + Path codegenRoot = project.getBasedir().toPath().resolve(CODEGEN_DIR); + List all; + try { + all = OwnedStock.find(codegenRoot, + OwnedStock.runtimeRoots(project.getBasedir().toPath(), project.getModules()), + OwnedStock.classpath(getClass().getClassLoader())); + } catch (IOException ex) { + throw new MojoExecutionException("Could not compare the owned copies under " + codegenRoot, ex); + } + EjectSupport.Port explicitPort = port == null || port.isBlank() ? null : EjectSupport.Port.parse(port); + + List chosen = new ArrayList<>(); + if (names == null || names.isBlank()) { + chosen.addAll(OwnedStock.identical(all)); + if (chosen.isEmpty()) { + getLog().info("No owned generator copy under " + codegenRoot + + " is identical to the shipped one; nothing removed."); + return; + } + } else { + Map> catalog = EjectSupport.ejectableByStableName(); + List errors = new ArrayList<>(); + for (String n : List.of(names.split(",")).stream().map(String::trim) + .filter(s -> !s.isEmpty()).distinct().collect(Collectors.toList())) { + if (!catalog.containsKey(n)) { + errors.add(" unknown generator \"" + n + "\""); + continue; + } + List found = all.stream() + .filter(c -> c.entry.stableName.equals(n) + && (explicitPort == null || c.entry.port == explicitPort)) + .collect(Collectors.toList()); + if (found.isEmpty()) { + errors.add(" \"" + n + "\" has no owned copy under " + codegenRoot); + continue; + } + for (OwnedStock.Copy c : found) { + if (!c.identical() && !force) { + errors.add(" \"" + n + "\" differs from the shipped generator (" + + c.behind + " line(s) behind, " + c.ownedOnly + " line(s) of your own; differing: " + + String.join(", ", c.differing) + "); -Dforce=true removes it anyway"); + } else { + chosen.add(c); + } + } + } + if (!errors.isEmpty()) { + throw new MojoFailureException("Nothing was removed:\n" + String.join("\n", errors)); + } + } + + // A class copied with two generators (the extractor, with entity) goes only when no + // generator that stays still owns it. + Set kept = new LinkedHashSet<>(); + for (OwnedStock.Copy c : all) if (!chosen.contains(c)) kept.addAll(c.files); + Set remove = new LinkedHashSet<>(); + for (OwnedStock.Copy c : chosen) for (Path f : c.files) if (!kept.contains(f)) remove.add(f); + try { + for (Path f : remove) { + Files.deleteIfExists(f); + for (Path dir = f.getParent(); dir != null && !dir.equals(codegenRoot) + && Files.isDirectory(dir); dir = dir.getParent()) { + try (var s = Files.list(dir)) { + if (s.findAny().isPresent()) break; + } + Files.delete(dir); + } + } + } catch (IOException ex) { + throw new MojoExecutionException("Could not remove an owned copy under " + codegenRoot, ex); + } + + Set shared = new LinkedHashSet<>(); + for (OwnedStock.Copy c : chosen) { + getLog().info("Removed owned \"" + c.entry.stableName + "\" (" + + c.files.stream().map(f -> f.getFileName().toString()).collect(Collectors.joining(", ")) + + ")" + (c.identical() ? "." : ", which differed (-Dforce=true).")); + getLog().info(" edit your pom.xml: " + c.ownedClassname() + " -> " + + c.stockClassname() + ""); + shared.addAll(c.shared); + } + getLog().info("This goal never edits pom.xml; make the edit above for each generator removed."); + if (!shared.isEmpty()) { + getLog().info("Left in place, since other owned generators may import them (delete by hand if " + + "none does): " + String.join(", ", shared)); + } + } + // ------------------------------------------------------------------------------------ // templates // ------------------------------------------------------------------------------------ @@ -773,7 +883,7 @@ private String ownedSupportRows(Path codegenRoot) { /** The owned copy named {@code filename} under {@code codegen/src/main/}, or * {@code null}. It may live under ANY package the adopter chose. */ - private static Path findOwned(Path codegenRoot, String langDir, String filename) throws IOException { + static Path findOwned(Path codegenRoot, String langDir, String filename) throws IOException { if (codegenRoot == null) return null; Path srcRoot = codegenRoot.resolve("src/main/" + langDir); if (!Files.isDirectory(srcRoot)) return null; @@ -880,21 +990,8 @@ private static void fold(int[] totals, List differing, String name, Ejec /** The owned runtime copy of {@code simple} in this module or one of its children, or {@code null}. */ private Path findOwnedRuntime(String simple) throws IOException { if (project == null || project.getBasedir() == null) return null; - Path base = project.getBasedir().toPath(); - List roots = new ArrayList<>(); - roots.add(base.resolve("src/main/java")); - if (project.getModules() != null) { - for (String module : project.getModules()) roots.add(base.resolve(module).resolve("src/main/java")); - } - for (Path root : roots) { - if (!Files.isDirectory(root)) continue; - try (var walk = Files.walk(root)) { - for (Path p : (Iterable) walk.filter(f -> f.getFileName().toString().equals(simple + ".java"))::iterator) { - if (EjectSupport.isOwnedRuntime(Files.readString(p, StandardCharsets.UTF_8))) return p; - } - } - } - return null; + return OwnedStock.findOwnedRuntime( + OwnedStock.runtimeRoots(project.getBasedir().toPath(), project.getModules()), simple); } // ------------------------------------------------------------------------------------ diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataGeneratorMojo.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataGeneratorMojo.java index 706d888e4..a0338bf32 100644 --- a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataGeneratorMojo.java +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/MetaDataGeneratorMojo.java @@ -47,6 +47,7 @@ protected void executeGenerators(MetaDataLoader loader, List generato // used to exit 0 on it. refuseUnmetRequires(generatorImpls); refuseSharedGeneratorDirs(); + noteOwnedStock(); // Refuse before any generator runs — the one choke point that sees the WHOLE // entity set every route-emitting generator below (Java's codegen-spring AND @@ -128,6 +129,24 @@ protected void executeGenerators(MetaDataLoader loader, List generato registerGeneratedSourceRoots(); } + /** + * One advisory line when an owned generator copy under {@code codegen/} is identical to the + * shipped generator. Never changes the build outcome: a failed comparison says nothing. + */ + private void noteOwnedStock() { + if (project == null || project.getBasedir() == null) return; + try { + Path base = project.getBasedir().toPath(); + String line = OwnedStock.notice(OwnedStock.find( + base.resolve(MetaDataEjectMojo.CODEGEN_DIR), + OwnedStock.runtimeRoots(base, project.getModules()), + OwnedStock.classpath(getClass().getClassLoader()))); + if (line != null) getLog().info("MetaData Mojo > " + line); + } catch (IOException | RuntimeException ignored) { + // advisory only + } + } + /** * Delete the recorded files this run did not emit and may judge * ({@link OrphanSweep#isCandidate}): its own generators' files, and files whose diff --git a/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/OwnedStock.java b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/OwnedStock.java new file mode 100644 index 000000000..40193cf59 --- /dev/null +++ b/server/java/maven-plugin/src/main/java/com/metaobjects/mojo/OwnedStock.java @@ -0,0 +1,209 @@ +package com.metaobjects.mojo; + +import com.metaobjects.generator.GeneratorRegistry; + +import java.io.IOException; +import java.io.InputStream; +import java.nio.charset.StandardCharsets; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.ArrayList; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * The owned generator copies under {@code codegen/} that are identical to the stock generator + * this plugin ships: what {@code mvn metaobjects:eject -Dundo=true} removes, and what + * {@code metaobjects:generate} mentions in one advisory line. + * + *

Maven-free, like {@link EjectSupport}: it takes directories and a resource reader. "Identical" + * is {@link EjectSupport#compare} over every file the eject wrote for the generator: the generator, + * the classes copied with it and the page templates it reads (a file gone missing counts as + * differing), and the shared support and helper runtime copies it reaches. Undo never deletes the + * shared copies (other owned generators import them) and names them instead; but a change to one + * keeps the generator out of "identical", because the shipped generator imports the shipped + * support and runtime, so the change would stop reaching the generated code. + */ +final class OwnedStock { + + private OwnedStock() {} + + /** Reads a reference resource off the classpath. */ + interface Reader { + String read(String resourcePath) throws IOException; + } + + static Reader classpath(ClassLoader loader) { + return resource -> { + try (InputStream in = loader.getResourceAsStream(resource)) { + if (in == null) throw new IOException("classpath resource not found: " + resource); + return new String(in.readAllBytes(), StandardCharsets.UTF_8); + } + }; + } + + /** One owned copy found under {@code codegen/}. */ + static final class Copy { + final EjectSupport.Entry entry; + /** The package the copy lives in, read from where the adopter put it. */ + final String ownedPackage; + /** Every file of the copy that exists: the generator, its companions, its templates. */ + final List files; + /** Names of the files that differ from, or are missing against, the shipped ones. */ + final List differing; + final int behind; + final int ownedOnly; + /** Shared copies left in place by an undo: support classes and helper runtime. */ + final List shared; + + Copy(EjectSupport.Entry entry, String ownedPackage, List files, List differing, + int behind, int ownedOnly, List shared) { + this.entry = entry; + this.ownedPackage = ownedPackage; + this.files = files; + this.differing = differing; + this.behind = behind; + this.ownedOnly = ownedOnly; + this.shared = shared; + } + + boolean identical() { return differing.isEmpty(); } + + /** The {@code } the pom names today. */ + String ownedClassname() { return ownedPackage + "." + entry.simpleName(); } + + /** The {@code } that points back at the shipped generator. */ + String stockClassname() { return entry.classname; } + } + + /** The source roots an owned helper runtime may sit under: {@code base}'s and each module's + * {@code src/main/java}. */ + static List runtimeRoots(Path base, List modules) { + List roots = new ArrayList<>(); + roots.add(base.resolve("src/main/java")); + if (modules != null) { + for (String module : modules) roots.add(base.resolve(module).resolve("src/main/java")); + } + return roots; + } + + /** The owned runtime copy of {@code simple} under {@code roots}, recognised by its ownership + * marker, or {@code null}. */ + static Path findOwnedRuntime(List roots, String simple) throws IOException { + for (Path root : roots) { + if (!Files.isDirectory(root)) continue; + try (var walk = Files.walk(root)) { + for (Path p : (Iterable) walk.filter(f -> f.getFileName().toString().equals(simple + ".java"))::iterator) { + if (EjectSupport.isOwnedRuntime(Files.readString(p, StandardCharsets.UTF_8))) return p; + } + } + } + return null; + } + + /** Every owned copy under {@code codegenRoot}, in catalog order; empty when there is none. The + * helper runtime its output imports is looked for under {@code runtimeRoots}. */ + static List find(Path codegenRoot, List runtimeRoots, Reader reader) throws IOException { + List out = new ArrayList<>(); + if (codegenRoot == null || !Files.isDirectory(codegenRoot)) return out; + for (EjectSupport.Entry e : EjectSupport.allEntries()) { + if (e.resourcePath == null) continue; + String langDir = e.port == EjectSupport.Port.KOTLIN ? "kotlin" : "java"; + Path found = MetaDataEjectMojo.findOwned(codegenRoot, langDir, e.simpleName() + "." + e.port.extension); + if (found == null) continue; + Path srcRoot = codegenRoot.resolve("src/main/" + langDir); + String ownedPackage = srcRoot.relativize(found.getParent()).toString() + .replace(found.getFileSystem().getSeparator(), "."); + List files = new ArrayList<>(); + List differing = new ArrayList<>(); + int[] totals = new int[2]; + for (String source : e.sources()) { + String name = EjectSupport.simpleNameOf(source) + "." + e.port.extension; + Path file = found.resolveSibling(name); + String owned = ""; + if (Files.exists(file)) { + files.add(file); + owned = Files.readString(file, StandardCharsets.UTF_8); + } + String reference = EjectSupport.rewriteCopiedImports( + reader.read(e.resourceOf(source)), e.sources(), ownedPackage); + fold(totals, differing, name, EjectSupport.compare(owned, reference)); + } + if (e.port == EjectSupport.Port.JAVA) { + Path resourceRoot = codegenRoot.resolve("src/main/resources") + .resolve(EjectSupport.ownedTemplatePrefix(ownedPackage)); + for (String ref : e.resources) { + Path file = resourceRoot.resolve(ref + EjectSupport.TEMPLATE_EXTENSION); + String owned = ""; + if (Files.exists(file)) { + files.add(file); + owned = Files.readString(file, StandardCharsets.UTF_8); + } + fold(totals, differing, ref + EjectSupport.TEMPLATE_EXTENSION, EjectSupport.compare( + owned, reader.read(GeneratorRegistry.EJECT_TEMPLATE_RESOURCE_ROOT + + ref + EjectSupport.TEMPLATE_EXTENSION))); + } + Set support = new LinkedHashSet<>(); + for (String source : e.sources()) support.addAll(EjectSupport.supportFor(reader.read(e.resourceOf(source)))); + for (String simple : support) { + Path owned = MetaDataEjectMojo.findOwned(codegenRoot, "java", simple + ".java"); + if (owned == null) continue; + fold(totals, differing, simple + ".java", EjectSupport.compare( + Files.readString(owned, StandardCharsets.UTF_8), + reader.read(GeneratorRegistry.EJECT_SUPPORT_RESOURCE_ROOT + simple + ".java"))); + } + for (String simple : e.runtime) { + Path owned = findOwnedRuntime(runtimeRoots, simple); + if (owned == null) continue; + fold(totals, differing, simple + ".java", EjectSupport.compare( + Files.readString(owned, StandardCharsets.UTF_8), + reader.read(GeneratorRegistry.EJECT_RUNTIME_RESOURCE_ROOT + simple + ".java"))); + } + } + out.add(new Copy(e, ownedPackage, files, differing, totals[0], totals[1], + shared(codegenRoot, e, reader))); + } + return out; + } + + /** The shared copies an undo leaves: owned support classes the generator imports, and the + * helper runtime its output imports (named whether or not a copy exists). */ + private static List shared(Path codegenRoot, EjectSupport.Entry e, Reader reader) throws IOException { + Set out = new LinkedHashSet<>(); + if (e.port == EjectSupport.Port.JAVA) { + for (String source : e.sources()) { + for (String simple : EjectSupport.supportFor(reader.read(e.resourceOf(source)))) { + if (MetaDataEjectMojo.findOwned(codegenRoot, "java", simple + ".java") != null) out.add(simple); + } + } + out.addAll(e.runtime); + } + return new ArrayList<>(out); + } + + /** The copies of {@code copies} whose files are all identical to the shipped ones. */ + static List identical(List copies) { + return copies.stream().filter(Copy::identical).collect(Collectors.toList()); + } + + /** + * The advisory line {@code metaobjects:generate} logs, or {@code null} when no owned copy is + * identical to the shipped generator. + */ + static String notice(List copies) { + List same = identical(copies); + if (same.isEmpty()) return null; + String names = same.stream().map(c -> c.entry.stableName).distinct().collect(Collectors.joining(", ")); + return "Owned generator copies under codegen/ identical to the shipped ones: " + names + + ". They add nothing to what the plugin runs; drop them with " + + "`mvn metaobjects:eject -Dundo=true`."; + } + + private static void fold(int[] totals, List differing, String name, EjectSupport.Comparison cmp) { + totals[0] += cmp.behind; + totals[1] += cmp.ownedOnly; + if (cmp.verdict != EjectSupport.Verdict.IDENTICAL) differing.add(name); + } +} diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/ApiDocsGeneratorEntryTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/ApiDocsGeneratorEntryTest.java index 434a2374b..80a8fa19b 100644 --- a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/ApiDocsGeneratorEntryTest.java +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/ApiDocsGeneratorEntryTest.java @@ -71,7 +71,8 @@ public void theGenerateGoalWritesTheApiReferenceOfEachLanguageTheListNames() thr assertTrue(Files.exists(kotlin.resolve("AGENT-API.md"))); String kotlinPage = Files.readString(kotlin.resolve("Basket.md")); assertTrue(kotlinPage, kotlinPage.contains("```kotlin")); - assertTrue(kotlinPage, kotlinPage.contains("**Model / metadata:**")); + assertFalse("no : the model link would be dead, so it is omitted: " + kotlinPage, + kotlinPage.contains("**Model / metadata:**")); } @Test diff --git a/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectUndoTest.java b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectUndoTest.java new file mode 100644 index 000000000..9b8f0be13 --- /dev/null +++ b/server/java/maven-plugin/src/test/java/com/metaobjects/mojo/EjectUndoTest.java @@ -0,0 +1,183 @@ +package com.metaobjects.mojo; + +import org.apache.maven.plugin.MojoFailureException; +import org.apache.maven.plugin.logging.Log; +import org.apache.maven.project.MavenProject; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.mockito.ArgumentCaptor; +import org.mockito.Mockito; + +import java.io.IOException; +import java.nio.file.Files; +import java.nio.file.Path; +import java.util.List; +import java.util.stream.Collectors; + +import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; +import static org.junit.Assert.assertNotNull; +import static org.junit.Assert.assertNull; +import static org.junit.Assert.assertTrue; +import static org.junit.Assert.fail; + +/** {@code mvn metaobjects:eject -Dundo=true} and the {@code metaobjects:generate} notice. */ +public class EjectUndoTest { + + @Rule + public TemporaryFolder tempFolder = new TemporaryFolder(); + + private Path basedir; + private MavenProject project; + + @Before + public void setUp() throws IOException { + basedir = tempFolder.newFolder("adopter-app").toPath(); + project = Mockito.mock(MavenProject.class); + Mockito.when(project.getBasedir()).thenReturn(basedir.toFile()); + Mockito.when(project.getGroupId()).thenReturn("com.acme"); + Mockito.when(project.getArtifactId()).thenReturn("widgets"); + Mockito.when(project.getVersion()).thenReturn("1.0.0"); + Mockito.when(project.getDependencies()).thenReturn(List.of()); + } + + private MetaDataEjectMojo mojo(String names) { + MetaDataEjectMojo m = new MetaDataEjectMojo(); + m.project = project; + m.names = names; + return m; + } + + private Log undo(String names, boolean force) throws Exception { + MetaDataEjectMojo m = mojo(names); + m.undo = true; + m.force = force; + Log log = Mockito.mock(Log.class); + m.setLog(log); + m.execute(); + return log; + } + + private String logged(Log log) { + ArgumentCaptor c = ArgumentCaptor.forClass(CharSequence.class); + Mockito.verify(log, Mockito.atLeast(0)).info(c.capture()); + return c.getAllValues().stream().map(CharSequence::toString).collect(Collectors.joining("\n")); + } + + private List findOwned() throws IOException { + return OwnedStock.find(basedir.resolve("codegen"), OwnedStock.runtimeRoots(basedir, List.of()), + OwnedStock.classpath(getClass().getClassLoader())); + } + + /** The helper runtime eject copied for the owned generators, under {@code src/main/java}. */ + private List ownedRuntime() throws IOException { + try (var walk = Files.walk(basedir.resolve("src/main/java"))) { + return walk.filter(p -> p.toString().endsWith(".java")).sorted().collect(Collectors.toList()); + } + } + + private Path ownedDto() { + return basedir.resolve("codegen/src/main/java/com/acme/codegen/SpringDtoGenerator.java"); + } + + private void eject(String names) throws Exception { + MetaDataEjectMojo m = mojo(names); + m.port = "java"; + m.execute(); + } + + @Test + public void withNoNamesRemovesEveryCopyIdenticalToTheShippedOneAndPrintsThePomEdit() throws Exception { + eject("dto"); + assertTrue(Files.exists(ownedDto())); + String out = logged(undo(null, false)); + assertFalse(Files.exists(ownedDto())); + assertTrue(out, out.contains("com.acme.codegen.SpringDtoGenerator -> " + + "com.metaobjects.generator.spring.SpringDtoGenerator")); + assertTrue(out, out.contains("never edits pom.xml")); + assertTrue("pom.xml of the adopter is not touched", !Files.exists(basedir.resolve("pom.xml"))); + } + + @Test + public void withNoNamesAnEditedCopyStaysAndNothingIsRemoved() throws Exception { + eject("dto"); + Files.writeString(ownedDto(), Files.readString(ownedDto()) + "\n// mine\n"); + logged(undo(null, false)); + assertTrue(Files.exists(ownedDto())); + } + + @Test + public void aNamedEditedCopyIsRefusedAndNothingIsRemovedEvenForAnIdenticalNameBesideIt() throws Exception { + eject("dto,repository"); + Files.writeString(ownedDto(), Files.readString(ownedDto()) + "\n// mine\n"); + try { + undo("repository,dto", false); + fail("expected MojoFailureException"); + } catch (MojoFailureException expected) { + assertTrue(expected.getMessage(), expected.getMessage().contains("\"dto\" differs from the shipped")); + } + assertTrue(Files.exists(ownedDto())); + assertTrue(Files.exists(basedir.resolve("codegen/src/main/java/com/acme/codegen/SpringRepositoryGenerator.java"))); + } + + @Test + public void forceRemovesANamedEditedCopy() throws Exception { + eject("dto"); + Files.writeString(ownedDto(), Files.readString(ownedDto()) + "\n// mine\n"); + undo("dto", true); + assertFalse(Files.exists(ownedDto())); + } + + @Test + public void anUnknownOrUnownedNameFailsTheWholeCall() throws Exception { + eject("dto"); + try { + undo("dto,bogus", false); + fail("expected MojoFailureException"); + } catch (MojoFailureException expected) { + assertTrue(expected.getMessage().contains("unknown generator \"bogus\"")); + } + try { + undo("dto,repository", false); + fail("expected MojoFailureException"); + } catch (MojoFailureException expected) { + assertTrue(expected.getMessage().contains("\"repository\" has no owned copy")); + } + assertTrue(Files.exists(ownedDto())); + } + + @Test + public void theGenerateNoticeNamesIdenticalCopiesAndStaysSilentOtherwise() throws Exception { + assertNull(OwnedStock.notice(findOwned())); + eject("dto"); + String line = OwnedStock.notice(findOwned()); + assertNotNull(line); + assertTrue(line, line.contains("dto")); + assertTrue(line, line.contains("mvn metaobjects:eject -Dundo=true")); + Files.writeString(ownedDto(), Files.readString(ownedDto()) + "\n// mine\n"); + assertNull(OwnedStock.notice(findOwned())); + assertEquals(1, findOwned().stream() + .filter(c -> c.entry.stableName.equals("dto")).count()); + } + + @Test + public void aChangedHelperRuntimeKeepsTheGeneratorOutOfTheNoticeAndOutOfNoNameUndo() throws Exception { + eject("dto"); + List runtime = ownedRuntime(); + assertFalse("ejecting dto copies the helper runtime its output imports", runtime.isEmpty()); + assertNotNull(OwnedStock.notice(findOwned())); + Files.writeString(runtime.get(0), Files.readString(runtime.get(0)) + "\n// mine\n"); + assertNull(OwnedStock.notice(findOwned())); + undo(null, false); + assertTrue("the generator whose runtime changed is not undone", Files.exists(ownedDto())); + try { + undo("dto", false); + fail("expected MojoFailureException"); + } catch (MojoFailureException expected) { + assertTrue(expected.getMessage(), expected.getMessage().contains(runtime.get(0).getFileName().toString())); + } + assertTrue(Files.exists(ownedDto())); + } +} diff --git a/server/python/src/metaobjects/cli.py b/server/python/src/metaobjects/cli.py index a092c7103..23e02fe02 100644 --- a/server/python/src/metaobjects/cli.py +++ b/server/python/src/metaobjects/cli.py @@ -89,7 +89,7 @@ scan_requirements, summarise_requirements, ) -from metaobjects.codegen.config import GenConfig +from metaobjects.codegen.config import DEFAULT_API_PREFIX, GenConfig from metaobjects.codegen.overwrite_policy import ( has_hash_manifest, is_pristine_generated, @@ -609,12 +609,13 @@ def _load_root_from_collection( return result.root, [] -#: Trailing advice after ``gen``'s unknown-attribute warnings. ``gen`` has no strict -#: flag of its own; ``verify`` is the strict door (ADR-0023), so that is what it names. +#: Trailing advice after ``gen``'s unknown-attribute warnings. ``gen`` loads leniently +#: unless ``--strict``; ``verify`` is strict by default (ADR-0023), so the line names both. UNKNOWN_ATTR_GEN_ADVICE = ( "metaobjects gen loads leniently and generated anyway, but `metaobjects verify` " "rejects this metadata (ADR-0023): fix or remove the attribute - a typo'd one " - "(`isAbstrakt`, `requird`) silently changes what is generated." + "(`isAbstrakt`, `requird`) silently changes what is generated. " + "Pass `--strict` to make `gen` refuse it too." ) @@ -648,13 +649,35 @@ def _print_unknown_attr_warnings(findings: list[str]) -> None: print(f"warning: {UNKNOWN_ATTR_GEN_ADVICE}", file=sys.stderr) -def _strict_load_hint() -> str: - """Actionable next-steps when strict verify rejects an undeclared @attr.""" +def _print_identical_owned_notice(root: Path) -> None: + """ONE advisory line when owned generator copies are identical to the shipped ones: an + owned copy that was never edited only costs the adopter upgrades. Advisory: the exit + code is unchanged, and a project with no owned copies reads no file.""" + try: + names = [e.name for e in owned.identical_owned(root)] + except Exception: # noqa: BLE001 — advisory; a packaged module that cannot be read says nothing + return + if names: + print( + f"note: owned generator copy identical to the shipped one: {', '.join(names)}. " + "Run `metaobjects eject --undo` to drop the copies and track the shipped " + "generators again.", + file=sys.stderr, + ) + + +def _strict_load_hint(strict_flag: str = "verify") -> str: + """Actionable next-steps when a strict load rejects an undeclared @attr.""" + opt_out = ( + "OR drop `--strict` to keep the lenient load" + if strict_flag == "--strict" + else "OR re-run with `--lax` to keep the legacy open-attr load" + ) + who = "gen --strict" if strict_flag == "--strict" else "verify" return ( - "verify is strict (ADR-0023): every authored @attr must be declared. Fix: " + f"{who} is strict (ADR-0023): every authored @attr must be declared. Fix: " "register the attr on a metadata provider, OR move arbitrary author-supplied " - "properties into an `attr.properties` bag, OR re-run with `--lax` to keep the " - "legacy open-attr load." + f"properties into an `attr.properties` bag, {opt_out}." ) @@ -876,6 +899,7 @@ def _run_suite( baseline: str = "default", refused_out: list[str] | None = None, select: Callable[[str], bool] | None = None, + api_prefix: str = DEFAULT_API_PREFIX, overwrite_hand_edits: frozenset[str] = frozenset(), overwrite_paths_out: set[str] | None = None, ) -> list[str]: @@ -918,6 +942,7 @@ def _run_suite( emit_package_init=emit_package_init, gen_state_dir=gen_state_dir, column_naming=column_naming, + api_prefix=api_prefix, # Manifest keys are relative to the PROJECT, which is what the manifest is # anchored on — not to ``--out``, which varies per run while the manifest does # not. Passed alongside gen_state_dir because the two are one decision: state @@ -1249,9 +1274,17 @@ def _cmd_fmt(args: argparse.Namespace) -> int: def _cmd_eject(args: argparse.Namespace) -> int: - """Copy reference generators into ``codegen/generators/`` to own (ADR-0034 Am. 3).""" + """Copy reference generators into ``codegen/generators/`` to own (ADR-0034 Am. 3), or + with ``--undo`` drop the copies that are identical to the shipped ones.""" root = Path(args.root) if args.root else Path.cwd() - result = owned.eject(args.names, root, force=args.force) + if args.undo: + result = owned.undo(args.names, root, force=args.force) + elif not args.names: + print("error: name at least one generator to eject (`metaobjects gen --list` shows " + "them), or pass --undo", file=sys.stderr) + return 2 + else: + result = owned.eject(args.names, root, force=args.force) for line in result.out: print(line) for line in result.err: @@ -1259,6 +1292,57 @@ def _cmd_eject(args: argparse.Namespace) -> int: return result.code +#: The metadata a fresh ``init`` writes: a loadable, empty model to author into. +_INIT_METADATA = "metadata:\n package: app\n children: []\n" + +_INIT_CONFIG = """\ +# What `metaobjects gen` generates: each target names its outDir and its generators. +# `metaobjects gen --list` shows the catalog. +metadata: metaobjects +targets: + app: + outDir: generated + generators: [] +""" + +_INIT_DOCS_CONFIG = """\ + docs: + outDir: docs + generators: [api-docs] +""" + + +def _cmd_init(args: argparse.Namespace) -> int: + """Scaffold ``metaobjects.config.yaml`` and the ``metaobjects/`` metadata directory. + + Never overwrites: a file that exists is reported and left alone. ``--with-docs`` adds a + ``docs`` target running ``api-docs``; without it no documentation generator is wired. + """ + root = Path(args.root) if args.root else Path.cwd() + config = _INIT_CONFIG + (_INIT_DOCS_CONFIG if args.with_docs else "") + wrote: list[str] = [] + kept: list[str] = [] + for rel, text in ( + (_CONFIG_NAME, config), + ("metaobjects/meta.app.yaml", _INIT_METADATA), + ): + path = root / rel + if path.exists(): + kept.append(rel) + continue + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(text, encoding="utf-8") + wrote.append(rel) + for rel in wrote: + print(f"wrote {rel}") + for rel in kept: + print(f"kept {rel} (already exists)") + if args.with_docs and _CONFIG_NAME in kept: + print("note: add the docs target yourself: `docs: {outDir: docs, generators: [api-docs]}`") + print("next: describe your entities in metaobjects/, then run `metaobjects gen`.") + return 0 + + def _warn_if_agent_context_stale() -> None: """Print ONE advisory line to stderr if the scaffolded agent context is stale. @@ -1535,7 +1619,7 @@ def selected(gen: Generator, token: str) -> bool: written = _run_suite( root, out_dir, gens, t.entities, gen_state_dir=gen_state_dir, column_naming=column_naming, project_root=project_root, baseline=baseline, - refused_out=refused_out, select=select, + refused_out=refused_out, select=select, api_prefix=config.api_prefix, overwrite_hand_edits=overwrite, overwrite_paths_out=overwrite_paths_out, ) if spec_gens: @@ -1543,7 +1627,7 @@ def selected(gen: Generator, token: str) -> bool: root, out_dir, spec_gens, t.entities, emit_package_init=False, gen_state_dir=gen_state_dir, column_naming=column_naming, project_root=project_root, baseline=baseline, refused_out=refused_out, - select=select, + select=select, api_prefix=config.api_prefix, ) except ValueError as exc: # intra-target run_gen collision → clean error errors.append(f"target '{t.name}': {exc}") @@ -1609,17 +1693,23 @@ def _cmd_gen_config(args: argparse.Namespace) -> int: if collection is None: return 1 + # `--strict` is the load `verify` uses (ADR-0023): an unknown attribute is an error and + # nothing is written. Without it `gen` loads leniently and only warns. + strict = bool(getattr(args, "strict", False)) unknown_attrs: list[str] = [] root, load_errors = _load_root_from_collection( - collection, providers=providers, libraries=config.libraries, + collection, strict=strict, providers=providers, libraries=config.libraries, unknown_attr_out=unknown_attrs, ) if root is None: print("error: failed to load metadata:", file=sys.stderr) for msg in load_errors: print(f" {msg}", file=sys.stderr) + if strict and any("ERR_UNKNOWN_ATTR" in m for m in load_errors): + print(_strict_load_hint(strict_flag="--strict"), file=sys.stderr) return 1 _print_unknown_attr_warnings(unknown_attrs) + _print_identical_owned_notice(config.config_dir) template_root, templates_refusal = _template_root_for_config(args, config) if templates_refusal is not None: @@ -2815,6 +2905,14 @@ def _build_parser() -> argparse.ArgumentParser: "as `templateSpec: ` in metaobjects.config.yaml" ), ) + gen.add_argument( + "--strict", + action="store_true", + help=( + "strict-attr load, the one `verify` uses (ADR-0023): an undeclared attribute is " + "an error and nothing is written. Without it gen loads leniently and warns." + ), + ) gen.add_argument( "--templates", default=None, @@ -3025,12 +3123,28 @@ def _build_parser() -> argparse.ArgumentParser: "eject", help="copy reference generators into codegen/generators/ to own and edit", ) - eject_p.add_argument("names", nargs="+", metavar="NAME", help="stable generator names") + eject_p.add_argument("names", nargs="*", metavar="NAME", help="stable generator names") eject_p.add_argument("--force", action="store_true", - help="replace an existing owned copy with the reference") + help="replace an existing owned copy with the reference " + "(with --undo: remove a copy that differs from it)") + eject_p.add_argument( + "--undo", action="store_true", + help="drop owned copies identical to the shipped generator (all of them, or the " + "names given); a copy that differs is refused unless --force; prints the " + "config edit, never makes it", + ) eject_p.add_argument("--root", default=None, help="project root (default: cwd)") eject_p.set_defaults(func=_cmd_eject) + init_p = sub.add_parser( + "init", + help="scaffold metaobjects.config.yaml and metaobjects/ (never overwrites a file)", + ) + init_p.add_argument("--with-docs", action="store_true", + help="also wire the api-docs generator (a `docs` target)") + init_p.add_argument("--root", default=None, help="project root (default: cwd)") + init_p.set_defaults(func=_cmd_init) + agent_docs = sub.add_parser( "agent-docs", help=( diff --git a/server/python/src/metaobjects/codegen/config.py b/server/python/src/metaobjects/codegen/config.py index 6132121be..3b862adc2 100644 --- a/server/python/src/metaobjects/codegen/config.py +++ b/server/python/src/metaobjects/codegen/config.py @@ -5,6 +5,9 @@ from metaobjects.naming import DEFAULT_COLUMN_NAMING +#: What generated routers mount under when no ``apiPrefix`` is configured. +DEFAULT_API_PREFIX = "/api" + @dataclass class GenConfig: @@ -54,6 +57,11 @@ class GenConfig: # constant this generator emits names a different column than the one a row # actually lands in. column_naming: str = DEFAULT_COLUMN_NAMING + # The path the generated routers mount under (and the api docs name): ``/api/authors`` + # by default, ``/v1/authors`` for ``/v1``, ``/authors`` for ``""``. The CLI fills it from + # ``apiPrefix`` in ``metaobjects.config.yaml``, the Python spelling of the TypeScript + # config's key of the same name. + api_prefix: str = DEFAULT_API_PREFIX # First-time-on-existing-file behaviour. "default" refuses a file that cannot be # proved to be generated output; "adopt" records what is on disk as the baseline and # writes nothing — the one run a project with no committed manifest can perform (the diff --git a/server/python/src/metaobjects/codegen/eject.py b/server/python/src/metaobjects/codegen/eject.py index af4b7bb20..c135099a2 100644 --- a/server/python/src/metaobjects/codegen/eject.py +++ b/server/python/src/metaobjects/codegen/eject.py @@ -79,10 +79,7 @@ def owned_token(entry: GeneratorEntry) -> str: def packaged_source(entry: GeneratorEntry) -> str: - assert entry.source_module is not None - module = importlib.import_module(entry.source_module) - assert module.__file__ is not None - return Path(module.__file__).read_text(encoding="utf-8") + return packaged_module_path(entry).read_text(encoding="utf-8") def ejected_source(entry: GeneratorEntry) -> str: @@ -125,6 +122,7 @@ class EjectResult: def eject(names: list[str], root: Path, *, force: bool = False) -> EjectResult: """Copy each named generator. Every name is checked before any file is written.""" + names = list(dict.fromkeys(names)) known = {e.name: e for e in ejectable()} unknown = [n for n in names if n not in known] if unknown: @@ -175,6 +173,92 @@ def eject(names: list[str], root: Path, *, force: bool = False) -> EjectResult: return EjectResult(0, out, []) +def identical_owned(root: Path) -> list[GeneratorEntry]: + """The owned generator copies that are identical to the shipped one, generator file and + companion runtime both. ``gen`` names them and ``eject --undo`` drops them.""" + found: list[GeneratorEntry] = [] + for entry in ejectable(): + if owned_path(root, entry.name).exists(): + verdict = owned_verdict(root, entry) + if verdict is not None and verdict[0] == "identical": + found.append(entry) + return found + + +def undo(names: list[str], root: Path, *, force: bool = False) -> EjectResult: + """Drop owned generator copies identical to the shipped ones (``eject --undo``). + + With no *names*, every identical copy goes. With names, only those; a copy that differs + is refused (nothing removed) unless *force*. Every name is checked before any file is + removed. The config is never edited: the entry that points back at the shipped generator + is printed instead. A companion runtime copy goes only when it matches the package and + no remaining owned generator reads it.""" + names = list(dict.fromkeys(names)) + known = {e.name: e for e in ejectable()} + unknown = [n for n in names if n not in known] + if unknown: + return EjectResult(2, [], [ + f"unknown or non-ejectable generator(s): {', '.join(unknown)}; " + f"ejectable: {', '.join(sorted(known))}", + ]) + not_owned = [n for n in names if not owned_path(root, n).exists()] + if not_owned: + return EjectResult(2, [], [f"not an owned copy (nothing at {OWNED_DIR.as_posix()}): " + f"{', '.join(not_owned)}"]) + + if names: + chosen = [known[n] for n in names] + identical = identical_owned(root) + differing = [e for e in chosen if e not in identical] + if differing and not force: + return EjectResult(2, [], [ + f"{e.name} differs from the shipped generator; nothing removed. Inspect: " + f'diff -u "{owned_path(root, e.name)}" "{packaged_module_path(e)}" ' + "(pass --force to remove it anyway)" + for e in differing + ]) + else: + chosen = identical_owned(root) + if not chosen: + return EjectResult(0, [ + "nothing to undo: no owned generator copy is identical to the shipped one." + ], []) + + removing = {e.name for e in chosen} + out: list[str] = [] + wiring: list[str] = [] + for entry in chosen: + path = owned_path(root, entry.name) + path.unlink() + out.append(f"removed {path.relative_to(root).as_posix()}") + wiring.append(f" replace {owned_token(entry)} with \"{entry.name}\"") + for module in entry.runtime: + rt = runtime_path(root, module) + if not rt.exists(): + continue + rel = rt.relative_to(root).as_posix() + if any(owned_path(root, o.name).exists() and o.name not in removing and module in o.runtime + for o in ejectable()): + out.append(f" kept {rel}: another owned generator still reads it") + elif _compare(rt.read_text(encoding="utf-8"), + packaged_runtime_path(module).read_text(encoding="utf-8")) != "identical": + out.append(f" kept {rel}: it differs from the package") + else: + rt.unlink() + out.append(f" removed {rel}") + out.append("") + out.append("metaobjects.config.yaml is not edited. In `generators`:") + out.extend(wiring) + return EjectResult(0, out, []) + + +def packaged_module_path(entry: GeneratorEntry) -> Path: + assert entry.source_module is not None + module = importlib.import_module(entry.source_module) + assert module.__file__ is not None + return Path(module.__file__) + + def _normalised(text: str) -> Counter[str]: return Counter(line.rstrip() for line in text.splitlines() if line.strip()) diff --git a/server/python/src/metaobjects/codegen/generators/api_docs_generator.py b/server/python/src/metaobjects/codegen/generators/api_docs_generator.py index a0cf765a7..766413c97 100644 --- a/server/python/src/metaobjects/codegen/generators/api_docs_generator.py +++ b/server/python/src/metaobjects/codegen/generators/api_docs_generator.py @@ -49,7 +49,7 @@ def __init__( def generate(self, ctx: GenContext) -> list[EmittedFile]: if ctx.loaded_root is None: return [] - model = PythonApiModelBuilder().build(ctx.loaded_root, self.project_name) + model = PythonApiModelBuilder(ctx.config.api_prefix).build(ctx.loaded_root, self.project_name) renderer = PythonApiDocsRenderer() layout = Layout.PACKAGE @@ -59,8 +59,12 @@ def page(path: str, content: str) -> EmittedFile: files: list[EmittedFile] = [] for unit in model.units: page_path = doc_page_output_path(layout, unit.package, unit.node) - href = model_cross_href( - f"{self.api_subdir}/{page_path}", page_path, self.model_base_url + # The model page is written by no Python generator, so a relative link would be + # dead: link only when `modelBaseUrl` says where the model pages are published. + href = ( + model_cross_href(f"{self.api_subdir}/{page_path}", page_path, self.model_base_url) + if self.model_base_url + else None ) files.append(page(page_path, renderer.render_unit_page(unit, href))) files.append(page("README.md", renderer.render_index(model, layout))) diff --git a/server/python/src/metaobjects/codegen/generators/router_generator.py b/server/python/src/metaobjects/codegen/generators/router_generator.py index fd44da73b..b55a52ad8 100644 --- a/server/python/src/metaobjects/codegen/generators/router_generator.py +++ b/server/python/src/metaobjects/codegen/generators/router_generator.py @@ -10,7 +10,8 @@ FR-008 §2.3. Conforms to the cross-port REST API contract (see ``docs/features/api-contract.md``): -* Routes: ``/api/`` (e.g. ``/api/authors``). +* Routes: ``/`` (e.g. ``/api/authors``; the prefix + is ``apiPrefix`` in ``metaobjects.config.yaml``, default ``/api``). * 5 CRUD verbs: GET list, GET by id, POST create, PATCH+PUT update, DELETE. * ``?withCount=1`` switches list response to ``{"rows", "total"}``. * ``?sort=field:asc|desc`` parsed via a static per-entity allowlist @@ -70,6 +71,7 @@ from metaobjects.meta.persistence.source.source_constants import ( SOURCE_KIND_TABLE, ) +from metaobjects.codegen.config import DEFAULT_API_PREFIX from metaobjects.naming import DEFAULT_COLUMN_NAMING from metaobjects.naming_refs import ref_target from metaobjects.wire import WIRE_NATIVE, WireFormat, wire_of @@ -1204,6 +1206,7 @@ def _render_tph_router( object_index: dict[str, MetaObject], column_naming: str = DEFAULT_COLUMN_NAMING, wire: WireFormat = WireFormat(), + api_prefix: str = DEFAULT_API_PREFIX, ) -> str: """FR-017 TPH: emit the discriminator base's router — a polymorphic collection at the base path + a full per-subtype CRUD set at //. The repo @@ -1377,7 +1380,7 @@ def _render_tph_router( sub = st.entity.name parts.append(f"from .{sub} import {sub}Create, {sub}Patch") parts.append("") - parts.append(f'router = APIRouter(prefix="/api/{plural}", tags=["{plural}"])') + parts.append(f'router = APIRouter(prefix="{api_prefix}/{plural}", tags=["{plural}"])') parts.append("") parts.append("") parts.extend(_sort_type_and_tables_lines(sort_field_nodes)) @@ -1606,6 +1609,7 @@ def render_router( column_naming: str = DEFAULT_COLUMN_NAMING, wire: WireFormat | None = None, report_export: bool = False, + api_prefix: str = DEFAULT_API_PREFIX, ) -> str | None: """Render an entity as a FastAPI ``APIRouter`` module. @@ -1656,7 +1660,9 @@ def render_router( # answering the cross-port 405 envelope. Gated by # fixtures/api-contract-conformance/projection/. if is_read_only_routed(entity): - return self._render_readonly_router(entity, column_naming, wire, report_export) + return self._render_readonly_router( + entity, column_naming, wire, report_export, api_prefix + ) return None # FR-017 TPH: a discriminator base emits a polymorphic collection at the base @@ -1664,7 +1670,9 @@ def render_router( if object_index is not None: plan = tph_plan_for(entity, object_index) if plan is not None: - return self._render_tph_router(entity, plan, object_index, column_naming, wire) + return self._render_tph_router( + entity, plan, object_index, column_naming, wire, api_prefix + ) m2m: list[M2mDescriptor] = ( resolve_m2m_descriptors(entity, object_index, column_naming) @@ -1758,7 +1766,7 @@ def render_router( # both keyed by field.name), NOT the read/entity-shape model. parts.append(f"from .{short_name} import {short_name}Create, {short_name}Patch") parts.append("") - parts.append(f'router = APIRouter(prefix="/api/{plural}", tags=["{plural}"])') + parts.append(f'router = APIRouter(prefix="{api_prefix}/{plural}", tags=["{plural}"])') parts.append("") parts.append("") # Sort allowlist + parse helper — per-entity, closed over the allowlist set so @@ -1889,6 +1897,7 @@ def _render_readonly_router( column_naming: str = DEFAULT_COLUMN_NAMING, wire: WireFormat = WireFormat(), report_export: bool = False, + api_prefix: str = DEFAULT_API_PREFIX, ) -> str: """Render a read-only (`@kind: view` / `materializedView`) object as a FastAPI ``APIRouter``: GET list + GET by id, and the four write verbs answering 405. An @@ -1983,7 +1992,7 @@ def _render_readonly_router( parts.append(f"from .{snake}_export import export_csv, export_xml") parts.append(f"from .{EXPORT_WRITER_MODULE} import EXPORT_MAX_ROWS") parts.append("") - parts.append(f'router = APIRouter(prefix="/api/{plural}", tags=["{plural}"])') + parts.append(f'router = APIRouter(prefix="{api_prefix}/{plural}", tags=["{plural}"])') parts.append("") parts.append("") parts.extend(_sort_type_and_tables_lines(sort_field_nodes)) @@ -2064,7 +2073,8 @@ def emit(entity: MetaObject, c: GenContext) -> list[EmittedFile]: # column name within one run. wire = wire_of(c.loaded_root) if c.loaded_root is not None else None source = self.render_router( - entity, index, c.config.column_naming, wire, REPORT_EXPORT_GENERATOR in c.generators + entity, index, c.config.column_naming, wire, REPORT_EXPORT_GENERATOR in c.generators, + c.config.api_prefix, ) if source is None: return [] @@ -2086,11 +2096,14 @@ def render_router( entity: MetaObject, object_index: dict[str, MetaObject] | None = None, column_naming: str = DEFAULT_COLUMN_NAMING, + api_prefix: str = DEFAULT_API_PREFIX, ) -> str | None: """Module-level back-compat wrapper. Delegates to a default :class:`RouterGenerator` instance so existing callers (and tests) are unaffected. Subclass :class:`RouterGenerator` to customize.""" - return RouterGenerator().render_router(entity, object_index, column_naming) + return RouterGenerator().render_router( + entity, object_index, column_naming, api_prefix=api_prefix + ) def router_generator(report_max_rows: int = DEFAULT_REPORT_MAX_ROWS) -> Generator: diff --git a/server/python/src/metaobjects/codegen/metaobjects-config.schema.json b/server/python/src/metaobjects/codegen/metaobjects-config.schema.json index a28f470da..1e7ffffc6 100644 --- a/server/python/src/metaobjects/codegen/metaobjects-config.schema.json +++ b/server/python/src/metaobjects/codegen/metaobjects-config.schema.json @@ -17,6 +17,11 @@ "items": { "type": "string", "pattern": "^[^:]+:[^:]+$" }, "description": "Consumer provider refs as 'module:symbol', resolved relative to this config file's directory (no PYTHONPATH needed)." }, + "apiPrefix": { + "type": "string", + "pattern": "^(/.*)?$", + "description": "The path generated routers mount under, and the api docs name: '/api' (default) gives /api/authors, '/v1' gives /v1/authors, '' mounts at /authors." + }, "templates": { "type": "string", "minLength": 1, diff --git a/server/python/src/metaobjects/codegen/project_config.py b/server/python/src/metaobjects/codegen/project_config.py index c699597b0..3f98021ee 100644 --- a/server/python/src/metaobjects/codegen/project_config.py +++ b/server/python/src/metaobjects/codegen/project_config.py @@ -28,6 +28,7 @@ import yaml # type: ignore[import-untyped] # PyYAML ships no type stubs +from metaobjects.codegen.config import DEFAULT_API_PREFIX from metaobjects.naming import removed_naming_source_message #: Config filename looked up in the cwd when ``--config`` is not given. @@ -43,6 +44,10 @@ #: followed the rc.3 guide and added it gets the naming-migration message, not "unknown key". _REMOVED_COLUMN_NAMING_KEY = "columnNaming" +#: A mount path the generated router can carry verbatim inside a string literal: a leading +#: ``/`` then path characters only, so a quote or backslash cannot break the emitted source. +_API_PREFIX_RE = re.compile(r"/[A-Za-z0-9._/-]*") + #: Every key this loader reads at the top level, and the ONLY ones it accepts. #: ``metaobjects-config.schema.json`` declares ``additionalProperties: false``, so the #: shipped schema has always said an unknown key is invalid; naming the set here is what @@ -50,7 +55,7 @@ #: schema by ``test_schema_and_loader_accept_EXACTLY_the_same_keys``. TOP_LEVEL_KEYS: tuple[str, ...] = ( "metadata", "providers", "libraries", "targets", "requirementTests", "apiDocs", - "templates", + "templates", "apiPrefix", ) @@ -198,6 +203,9 @@ class ProjectConfig: #: absolute). ``None`` => ``--templates`` / ``--prompts``, else ``prompts/`` when it exists, #: else ``templates/``. templates: str | None = None + #: ``apiPrefix``: the path generated routers mount under, and the api docs name. Same key + #: as the TypeScript config; here the default is ``/api`` and ``""`` mounts at the root. + api_prefix: str = DEFAULT_API_PREFIX def metadata_dir(self) -> str: """The metadata dir resolved against ``config_dir`` (absolute path string).""" @@ -460,6 +468,15 @@ def load_project_config(path: Path) -> ProjectConfig: if templates is not None and (not isinstance(templates, str) or not templates): raise ConfigError(f"{path}: 'templates' must be a non-empty string (a directory path).") + api_prefix = raw.get("apiPrefix", DEFAULT_API_PREFIX) + if not isinstance(api_prefix, str) or (api_prefix and not _API_PREFIX_RE.fullmatch(api_prefix)): + raise ConfigError( + f"{path}: 'apiPrefix' must be \"\" (mount at the root) or a path starting with " + f"'/' made of letters, digits, '.', '_', '-' and '/', e.g. \"/api\" or \"/v1\"; " + f"got {api_prefix!r}." + ) + api_prefix = api_prefix.rstrip("/") + return ProjectConfig( config_dir=path.parent.resolve(), metadata=metadata, @@ -471,4 +488,5 @@ def load_project_config(path: Path) -> ProjectConfig: api_docs=api_docs, legacy_docs=legacy_docs, templates=templates, + api_prefix=api_prefix, ) diff --git a/server/python/src/metaobjects/codegen/runtime/api_docs_builder.py b/server/python/src/metaobjects/codegen/runtime/api_docs_builder.py index a783a9741..e5e2476c2 100644 --- a/server/python/src/metaobjects/codegen/runtime/api_docs_builder.py +++ b/server/python/src/metaobjects/codegen/runtime/api_docs_builder.py @@ -38,6 +38,7 @@ FieldShape, ) from metaobjects.codegen import type_map +from metaobjects.codegen.config import DEFAULT_API_PREFIX from metaobjects.codegen.model_walk import package_of from .m2m_codegen import ( build_object_index, @@ -94,6 +95,10 @@ def _is_email_kind(tmpl: MetaData) -> bool: class PythonApiModelBuilder: """Builds the :class:`ApiModel` SDK-surface IR from a loaded ``MetaRoot``.""" + def __init__(self, api_prefix: str = DEFAULT_API_PREFIX) -> None: + #: The path the documented routes mount under (``apiPrefix``; ``""`` = at the root). + self.api_prefix = api_prefix + def build(self, root: MetaData, project: str) -> ApiModel: objects = [ # ADR-0039 sanctioned own: top-level scan on the loader ROOT (never extended, own == effective) @@ -237,7 +242,7 @@ def _read_only_symbols(self, obj: MetaObject) -> list[ApiSymbol]: item-verb refusals are 405s, not endpoints, so no write verb is documented; a keyless object has no item route and its seam no ``find_by_id``.""" router_module = naming.router_module_name(obj.name) - base_path = "/api/" + naming.route_path(obj.name) + base_path = f"{self.api_prefix}/" + naming.route_path(obj.name) repo = naming.repository_class_name(obj.name) fields_const = naming.filter_fields_const(obj.name) item = has_item_route(obj) @@ -289,7 +294,7 @@ def _add_rest_symbols( object_index: dict[str, MetaObject], ) -> None: router_module = naming.router_module_name(entity.name) - base_path = "/api/" + naming.route_path(entity.name) + base_path = f"{self.api_prefix}/" + naming.route_path(entity.name) # The router registers the item routes with a per-entity path param # (`/{_id}`, e.g. `/{customer_id}`) — document that exact name, not # a generic `{id}`, so the docs match the generated FastAPI routes. diff --git a/server/python/tests/codegen/test_api_docs_builder.py b/server/python/tests/codegen/test_api_docs_builder.py index 72d6732c3..7ad98f0b4 100644 --- a/server/python/tests/codegen/test_api_docs_builder.py +++ b/server/python/tests/codegen/test_api_docs_builder.py @@ -115,3 +115,25 @@ def test_unit_page_carries_the_contract_back_link() -> None: assert "**Model / metadata:** [Order](../../../../acme/shop/Order.md)" in page assert "## Model" in page assert "## REST" in page + + +def test_api_prefix_is_the_documented_base_path() -> None: + case = _repo_root() / "fixtures" / "conformance" / "api-docs-cross-port" / "input" + root = MetaDataLoader.from_directory(str(case)).root + + def order_page(prefix: str | None) -> str: + builder = PythonApiModelBuilder() if prefix is None else PythonApiModelBuilder(prefix) + model = builder.build(root, "p") + return render_unit_page(next(u for u in model.units if u.node == "Order"), None) + + assert "/api/orders" in order_page(None) + assert "/v1/orders" in order_page("/v1") and "/api/orders" not in order_page("/v1") + bare = order_page("") + assert "`/orders" in bare or " /orders" in bare + assert "/api/" not in bare + + +def test_unit_page_omits_the_back_link_when_nothing_resolves() -> None: + model = _model() + order = next(u for u in model.units if u.node == "Order") + assert "Model / metadata" not in render_unit_page(order, None) diff --git a/server/python/tests/codegen/test_cli_gen_unknown_attr.py b/server/python/tests/codegen/test_cli_gen_unknown_attr.py index 24bbbfa4d..4ca9e4888 100644 --- a/server/python/tests/codegen/test_cli_gen_unknown_attr.py +++ b/server/python/tests/codegen/test_cli_gen_unknown_attr.py @@ -69,3 +69,30 @@ def test_gen_clean_model_prints_no_unknown_attr_warning(tmp_path: Path, capsys) rc = main(as_config(["gen", "--generators", "entity", meta_dir, "--out", str(tmp_path / "out")])) assert rc == 0 assert "ERR_UNKNOWN_ATTR" not in capsys.readouterr().err + + +def test_gen_strict_refuses_an_unknown_attribute_and_writes_nothing(tmp_path: Path, capsys) -> None: + _meta(tmp_path, TYPO_YAML) + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text("targets:\n models:\n outDir: gen/models\n generators: [entity]\n") + rc = main(["gen", "--strict", "--config", str(cfg)]) + assert rc != 0 + err = capsys.readouterr().err + assert "ERR_UNKNOWN_ATTR" in err and "requird" in err + assert "gen --strict is strict (ADR-0023)" in err + assert not (tmp_path / "gen").exists() + + +def test_gen_strict_passes_a_clean_model(tmp_path: Path) -> None: + _meta(tmp_path, CLEAN_YAML) + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text("targets:\n models:\n outDir: gen/models\n generators: [entity]\n") + assert main(["gen", "--strict", "--config", str(cfg)]) == 0 + + +def test_lenient_advice_names_the_strict_flag(tmp_path: Path, capsys) -> None: + _meta(tmp_path, TYPO_YAML) + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text("targets:\n models:\n outDir: gen/models\n generators: [entity]\n") + assert main(["gen", "--config", str(cfg)]) == 0 + assert "--strict" in capsys.readouterr().err diff --git a/server/python/tests/codegen/test_cli_init.py b/server/python/tests/codegen/test_cli_init.py new file mode 100644 index 000000000..811eabc27 --- /dev/null +++ b/server/python/tests/codegen/test_cli_init.py @@ -0,0 +1,69 @@ +"""`metaobjects init` scaffolds a config and a metadata directory, and never overwrites.""" +from __future__ import annotations + +from pathlib import Path + +from metaobjects.cli import main + + +def test_init_writes_a_config_with_no_docs_generator_and_a_loadable_model( + tmp_path: Path, capsys +) -> None: + assert main(["init", "--root", str(tmp_path)]) == 0 + out = capsys.readouterr().out + assert "wrote metaobjects.config.yaml" in out and "wrote metaobjects/meta.app.yaml" in out + assert "api-docs" not in (tmp_path / "metaobjects.config.yaml").read_text() + assert main(["gen", "--strict", "--config", str(tmp_path / "metaobjects.config.yaml")]) == 0 + + +def test_init_with_docs_wires_api_docs_and_gen_runs(tmp_path: Path) -> None: + assert main(["init", "--with-docs", "--root", str(tmp_path)]) == 0 + cfg = tmp_path / "metaobjects.config.yaml" + assert "generators: [api-docs]" in cfg.read_text() + assert main(["gen", "--config", str(cfg)]) == 0 + assert (tmp_path / "docs" / "api" / "python" / "README.md").is_file() + + +def test_init_never_overwrites_and_reports_what_it_kept(tmp_path: Path, capsys) -> None: + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text("mine: true\n") + assert main(["init", "--root", str(tmp_path)]) == 0 + assert cfg.read_text() == "mine: true\n" + out = capsys.readouterr().out + assert "kept metaobjects.config.yaml" in out + assert (tmp_path / "metaobjects" / "meta.app.yaml").is_file() + + +def test_api_prefix_in_the_config_reaches_routers_and_docs(tmp_path: Path) -> None: + assert main(["init", "--with-docs", "--root", str(tmp_path)]) == 0 + (tmp_path / "metaobjects" / "meta.app.yaml").write_text( + "metadata:\n package: app\n children:\n" + " - object.entity:\n name: Author\n children:\n" + " - source.rdb: { table: authors }\n" + " - field.long: { name: id }\n" + " - identity.primary: { name: pk, fields: [id] }\n" + ) + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text(cfg.read_text().replace( + "generators: []", "generators: [entity, filter-allowlist, routes]") + "apiPrefix: /v1\n") + assert main(["gen", "--config", str(cfg)]) == 0 + assert 'APIRouter(prefix="/v1/authors"' in (tmp_path / "generated" / "author_router.py").read_text() + assert "/v1/authors" in (tmp_path / "docs" / "api" / "python" / "app" / "Author.md").read_text() + assert main(["verify", "--codegen", "--config", str(cfg)]) == 0 + + +def test_api_prefix_must_be_empty_or_a_path(tmp_path: Path, capsys) -> None: + assert main(["init", "--root", str(tmp_path)]) == 0 + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text(cfg.read_text() + "apiPrefix: v1\n") + assert main(["gen", "--config", str(cfg)]) != 0 + assert "apiPrefix" in capsys.readouterr().err + + +def test_api_prefix_with_a_quote_or_backslash_is_refused(tmp_path: Path, capsys) -> None: + # The prefix is written into the generated router's source, so it may only hold path characters. + assert main(["init", "--root", str(tmp_path)]) == 0 + cfg = tmp_path / "metaobjects.config.yaml" + cfg.write_text(cfg.read_text() + "apiPrefix: '/v1\"x'\n") + assert main(["gen", "--config", str(cfg)]) != 0 + assert "apiPrefix" in capsys.readouterr().err diff --git a/server/python/tests/codegen/test_eject_undo.py b/server/python/tests/codegen/test_eject_undo.py new file mode 100644 index 000000000..b2ea79b39 --- /dev/null +++ b/server/python/tests/codegen/test_eject_undo.py @@ -0,0 +1,145 @@ +"""`metaobjects eject --undo` and the `gen` notice for owned copies identical to the shipped ones.""" +from __future__ import annotations + +from pathlib import Path + +from metaobjects.cli import main +from tests.codegen.test_eject import _project + +GEN = Path("codegen") / "generators" +RT = Path("codegen") / "runtime" + + +def _gen_cfg(tmp_path: Path, generators: str) -> Path: + return _project(tmp_path, generators) + + +def test_undo_with_no_names_removes_identical_copies_and_prints_the_config_edit( + tmp_path: Path, monkeypatch, capsys +) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity", "names"]) == 0 + capsys.readouterr() + assert main(["eject", "--undo"]) == 0 + out = capsys.readouterr().out + assert not (tmp_path / GEN / "entity.py").exists() + assert not (tmp_path / GEN / "names.py").exists() + assert 'replace codegen.generators.entity:entity_model with "entity"' in out + assert "metaobjects.config.yaml is not edited" in out + + +def test_undo_keeps_a_copy_that_differs_and_says_so(tmp_path: Path, monkeypatch, capsys) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity", "names"]) == 0 + (tmp_path / GEN / "names.py").write_text( + (tmp_path / GEN / "names.py").read_text() + "\nMINE = 1\n" + ) + capsys.readouterr() + assert main(["eject", "--undo"]) == 0 + assert not (tmp_path / GEN / "entity.py").exists() + assert (tmp_path / GEN / "names.py").exists() + + +def test_undo_with_nothing_identical_says_so(tmp_path: Path, monkeypatch, capsys) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "--undo"]) == 0 + assert "nothing to undo" in capsys.readouterr().out + + +def test_undo_by_name_refuses_a_differing_copy_and_removes_nothing( + tmp_path: Path, monkeypatch, capsys +) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity", "names"]) == 0 + (tmp_path / GEN / "names.py").write_text( + (tmp_path / GEN / "names.py").read_text() + "\nMINE = 1\n" + ) + capsys.readouterr() + assert main(["eject", "--undo", "entity", "names"]) == 2 + err = capsys.readouterr().err + assert "names differs" in err and "diff -u" in err and "--force" in err + assert (tmp_path / GEN / "entity.py").exists() + assert (tmp_path / GEN / "names.py").exists() + + +def test_undo_force_removes_a_differing_copy(tmp_path: Path, monkeypatch) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "names"]) == 0 + path = tmp_path / GEN / "names.py" + path.write_text(path.read_text() + "\nMINE = 1\n") + assert main(["eject", "--undo", "names", "--force"]) == 0 + assert not path.exists() + + +def test_undo_validates_every_name_before_removing(tmp_path: Path, monkeypatch, capsys) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity"]) == 0 + capsys.readouterr() + assert main(["eject", "--undo", "entity", "nope"]) == 2 + assert "nope" in capsys.readouterr().err + assert main(["eject", "--undo", "entity", "names"]) == 2 # names is not owned + assert (tmp_path / GEN / "entity.py").exists() + + +def test_undo_removes_identical_runtime_only_when_no_remaining_owner_uses_it( + tmp_path: Path, monkeypatch, capsys +) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "routes"]) == 0 + runtime = sorted((tmp_path / RT).glob("*.py")) + assert runtime + # A differing runtime copy is kept, and the generator still goes only if identical too. + runtime[0].write_text(runtime[0].read_text() + "\nMINE = 1\n") + capsys.readouterr() + assert main(["eject", "--undo", "routes"]) == 2 # companion differs => not identical + assert (tmp_path / GEN / "routes.py").exists() + assert main(["eject", "--undo", "routes", "--force"]) == 0 + out = capsys.readouterr().out + assert not (tmp_path / GEN / "routes.py").exists() + assert runtime[0].exists() and f"kept {RT.as_posix()}/{runtime[0].name}" in out + for other in runtime[1:]: + assert not other.exists() + + +def test_eject_without_names_or_undo_is_an_error(capsys) -> None: + assert main(["eject"]) == 2 + assert "--undo" in capsys.readouterr().err + + +def test_gen_names_identical_owned_copies_once_and_exits_zero( + tmp_path: Path, monkeypatch, capsys +) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity"]) == 0 + cfg = _gen_cfg(tmp_path, "codegen.generators.entity:entity_model") + capsys.readouterr() + assert main(["gen", "--config", str(cfg)]) == 0 + err = capsys.readouterr().err + notes = [ln for ln in err.splitlines() if "identical to the shipped" in ln] + assert len(notes) == 1 + assert "entity" in notes[0] and "metaobjects eject --undo" in notes[0] + + +def test_gen_is_silent_when_the_owned_copy_was_edited(tmp_path: Path, monkeypatch, capsys) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity"]) == 0 + path = tmp_path / GEN / "entity.py" + path.write_text(path.read_text() + "\nMINE = 1\n") + cfg = _gen_cfg(tmp_path, "codegen.generators.entity:entity_model") + capsys.readouterr() + assert main(["gen", "--config", str(cfg)]) == 0 + assert "identical to the shipped" not in capsys.readouterr().err + + +def test_gen_without_owned_copies_prints_no_notice(tmp_path: Path, capsys) -> None: + cfg = _gen_cfg(tmp_path, "entity") + assert main(["gen", "--config", str(cfg)]) == 0 + assert "identical to the shipped" not in capsys.readouterr().err + + +def test_undo_with_a_repeated_name_removes_it_once(tmp_path: Path, monkeypatch, capsys) -> None: + monkeypatch.chdir(tmp_path) + assert main(["eject", "entity"]) == 0 + capsys.readouterr() + assert main(["eject", "--undo", "entity", "entity"]) == 0 + assert not (tmp_path / GEN / "entity.py").exists() diff --git a/server/python/tests/codegen/test_router_generator.py b/server/python/tests/codegen/test_router_generator.py index 1a69c12e2..ecc534faa 100644 --- a/server/python/tests/codegen/test_router_generator.py +++ b/server/python/tests/codegen/test_router_generator.py @@ -94,6 +94,12 @@ def test_router_has_apirouter_with_pluralized_lowercase_prefix() -> None: assert "@generated by metaobjects" in out +def test_api_prefix_sets_the_mount_path() -> None: + entity = _author() + assert 'APIRouter(prefix="/v1/authors"' in render_router(entity, api_prefix="/v1") + assert 'APIRouter(prefix="/authors"' in render_router(entity, api_prefix="") + + def test_all_five_verbs_present() -> None: out = _require(_author()) # Two GETs (list + by-id), POST, PATCH+PUT on the same handler, DELETE. diff --git a/server/python/tests/conformance/test_api_docs_cross_port_conformance.py b/server/python/tests/conformance/test_api_docs_cross_port_conformance.py index e0892a4df..4b569ac3d 100644 --- a/server/python/tests/conformance/test_api_docs_cross_port_conformance.py +++ b/server/python/tests/conformance/test_api_docs_cross_port_conformance.py @@ -37,13 +37,14 @@ def _repo_root() -> Path: _CASE_DIR = _ROOT / "fixtures" / "conformance" / "api-docs-cross-port" -def _drive_python_docs(input_dir: Path, out_dir: Path) -> None: +def _drive_python_docs(input_dir: Path, out_dir: Path, model_base_url: str | None = None) -> None: """Run the Python api reference end-to-end: ``api-docs`` declared in a config target whose ``outDir`` is *out_dir*, run by ``metaobjects gen``.""" config = out_dir.parent / "metaobjects.config.yaml" config.write_text( f"metadata: {json.dumps(str(input_dir))}\n" - f"targets:\n docs:\n outDir: {json.dumps(str(out_dir))}\n generators: [api-docs]\n", + + (f"apiDocs:\n modelBaseUrl: {json.dumps(model_base_url)}\n" if model_base_url else "") + + f"targets:\n docs:\n outDir: {json.dumps(str(out_dir))}\n generators: [api-docs]\n", encoding="utf-8", ) assert cli.main(["gen", "--config", str(config)]) == 0 @@ -64,24 +65,30 @@ def test_python_api_docs_paths_and_back_links_match_shared_manifest(tmp_path: Pa out_dir = tmp_path / "docs-out" _drive_python_docs(input_dir, out_dir) - # ---- per-unit: the api/python page exists AND carries the contract back-link ---- + # ---- per-unit: the api/python page exists. Python writes no model page, so a relative + # back-link would be dead: the page carries none until `modelBaseUrl` says where the + # model pages are published (then the link is that base plus the manifest's modelPath). units = manifest["units"] assert units, "manifest must declare at least one unit" for unit in units: node = unit["node"] - api_python_path = unit["apiPythonPath"] - api_python_to_model = unit["apiPythonToModel"] - - page = out_dir / api_python_path + page = out_dir / unit["apiPythonPath"] assert page.is_file(), ( - f"Python api-docs surface did not produce '{api_python_path}' for unit " + f"Python api-docs surface did not produce '{unit['apiPythonPath']}' for unit " f"'{node}'. Searched under: {out_dir}" ) + assert "Model / metadata" not in page.read_text(encoding="utf-8"), ( + f"api/python page for '{node}' links a model page nothing writes" + ) - contents = page.read_text(encoding="utf-8") - expected_back_link = f"**Model / metadata:** [{node}]({api_python_to_model})" - assert expected_back_link in contents, ( - f"rendered api/python page for '{node}' must carry the contract back-link:\n" - f" {expected_back_link}\nactual page:\n{contents}" + federated = tmp_path / "federated" / "docs-out" + federated.parent.mkdir() + _drive_python_docs(input_dir, federated, "https://docs.example.test/model/") + for unit in units: + contents = (federated / unit["apiPythonPath"]).read_text(encoding="utf-8") + expected = ( + f"**Model / metadata:** [{unit['node']}]" + f"(https://docs.example.test/model/{unit['modelPath']})" ) + assert expected in contents, contents diff --git a/server/python/tests/test_api_docs_packages.py b/server/python/tests/test_api_docs_packages.py index 9ac1799cc..996f4649d 100644 --- a/server/python/tests/test_api_docs_packages.py +++ b/server/python/tests/test_api_docs_packages.py @@ -53,12 +53,13 @@ def test_pages_and_back_links_are_filed_under_each_package() -> None: config=GenConfig(out_dir="/tmp/out"), warn=lambda _m: None, ) - files = {f.path: f.content for f in ApiDocsGenerator(api_subdir="api").generate(ctx)} + files = {f.path: f.content for f in ApiDocsGenerator(api_subdir="api", model_base_url="https://docs.example.test").generate(ctx)} assert "api/alpha/lib/Widget.md" in files assert "api/beta/app/Order.md" in files assert not [p for p in files if p.endswith("Order.md") and "alpha" in p], sorted(files) # The back-link to the model page stays in the page's own package. - assert "alpha/lib/Widget.md" in files["api/alpha/lib/Widget.md"] - assert "beta/app/Order.md" in files["api/beta/app/Order.md"] + # (The link is emitted only when `modelBaseUrl` says where the model pages live.) + assert "https://docs.example.test/alpha/lib/Widget.md" in files["api/alpha/lib/Widget.md"] + assert "https://docs.example.test/beta/app/Order.md" in files["api/beta/app/Order.md"] assert "alpha" not in files["api/beta/app/Order.md"] diff --git a/server/typescript/packages/cli/src/commands/eject.ts b/server/typescript/packages/cli/src/commands/eject.ts index 65d318024..41769c911 100644 --- a/server/typescript/packages/cli/src/commands/eject.ts +++ b/server/typescript/packages/cli/src/commands/eject.ts @@ -15,9 +15,9 @@ // config-array edit has regressed across at least four filed issues). The primary // consumer performs two file edits and one install trivially when told exactly what // they are. `meta gen` plus `tsc` is the audit. -import { existsSync } from "node:fs"; -import { mkdir, writeFile, stat, readFile } from "node:fs/promises"; -import { join } from "node:path"; +import { existsSync, readdirSync } from "node:fs"; +import { mkdir, writeFile, stat, readFile, rm, rmdir } from "node:fs/promises"; +import { dirname, join } from "node:path"; import { cliVersion } from "../lib/version.js"; import * as coreTpl from "@metaobjectsdev/codegen-ts"; import * as reactTpl from "@metaobjectsdev/codegen-ts-react"; @@ -427,6 +427,13 @@ interface GeneratorListRow { * Absent when the project owns none of its renderers. */ templates?: GeneratorTemplateRow[]; + /** + * The files under `codegen/runtime/` that differ from the installed adapter package, for a + * generator whose output imports that adapter. Any makes `status` `differs`: the shipped + * generator imports the package, so undoing this copy would drop the adapter edit unseen. + * Absent when none differ. + */ + runtime?: string[]; } interface GeneratorTemplateRow { @@ -447,6 +454,7 @@ async function generatorListGroups(cwd: string): Promise { // Every renderer the project holds, against the package: attributed to a generator by what // that generator imports, so a renderer another generator owns never flags this one. const templateStatus = await templateCopyStatus(cwd, packagedTemplates()); + const changedRuntime = (await runtimeCopyStatus(cwd)).filter((r) => !isStockVerdict(r.verdict)).map((r) => r.path); for (const source of SOURCES) { const rows: GeneratorListRow[] = []; groups.push({ packageName: source.packageName, rows }); @@ -483,6 +491,10 @@ async function generatorListGroups(cwd: string): Promise { } if (renderers.length > 0) row.templates = renderers; if (renderers.some((r) => r.status === "differs")) row.status = "differs"; + if (ejectsRuntime(name) && changedRuntime.length > 0) { + row.runtime = changedRuntime; + row.status = "differs"; + } } } return groups; @@ -517,14 +529,19 @@ async function listOutput(cwd: string): Promise { else if (r.status === "reformatted") lines.push(` ${r.name} [owned — same content as the reference, your formatting]`); else { const differing = (r.templates ?? []).filter((t) => t.status === "differs"); + const companions = [ + ...(differing.length > 0 ? [`${differing.length} renderer(s)`] : []), + ...(r.runtime ? [`${r.runtime.length} runtime adapter file(s)`] : []), + ]; lines.push( generatorFileDiffers(r) ? ` ${r.name} [owned — DIFFERS: ${r.behind} line(s) behind, ${r.own} line(s) of your own]` - : ` ${r.name} [owned — DIFFERS: generator identical, ${differing.length} renderer(s) differ]`, + : ` ${r.name} [owned — DIFFERS: generator identical, ${companions.join(" and ")} differ]`, ); for (const t of differing) { lines.push(` renderer ${t.path} [DIFFERS: ${t.behind} line(s) behind, ${t.own} line(s) of your own]`); } + for (const path of r.runtime ?? []) lines.push(` runtime ${path} [DIFFERS]`); } } lines.push(""); @@ -594,6 +611,208 @@ async function listOutput(cwd: string): Promise { return lines.join("\n"); } +// --------------------------------------------------------------------------- +// --undo: take back an owned copy that is the shipped reference +// --------------------------------------------------------------------------- + +/** A copy comparison that found the shipped content, whatever the formatting. */ +function isStockVerdict(verdict: string): boolean { + return verdict === "identical" || verdict === "reformatted"; +} + +/** A generator row whose file, every renderer it reaches and its runtime adapter, are the shipped reference. */ +function isStockRow(r: GeneratorListRow): boolean { + return isStockVerdict(r.status); +} + +/** + * The advisory `meta gen` prints when owned generator copies are the shipped reference. + * + * An owned copy that never changed buys nothing: it freezes at the release it was copied + * from and misses every later fix, while the same generator imports from its package. Most + * were ejected while the stock generators could not be imported, and nothing noticed once + * they could. Undefined when there are none, and free when the project owns no copy. + */ +export async function stockCopyNotice(cwd: string): Promise { + if (!existsSync(join(cwd, OWNED_GENERATORS_DIR))) return undefined; + const stock = (await generatorListGroups(cwd)).flatMap((g) => g.rows).filter(isStockRow).map((r) => r.name); + if (stock.length === 0) return undefined; + const one = stock.length === 1; + return ( + `${stock.length} owned generator cop${one ? "y is" : "ies are"} the shipped reference, unchanged ` + + `(${stock.join(", ")}): ${one ? "it" : "they"} will miss the fixes later releases make. ` + + `\`meta eject --undo\` removes ${one ? "it" : "them"} and prints the import to restore.` + ); +} + +/** One owned copy `--undo` took back. */ +interface UndoneRow { + name: string; + path: string; + /** The edit in `metaobjects.config.ts` that runs the generator as shipped again. */ + wire: { import: string; replaces: string }; +} + +interface UndoKept { + name: string; + path: string; + reason: string; +} + +interface UndoPayload { + undone: UndoneRow[]; + /** Every file removed: the generators, their copied renderers and adapter source. */ + removed: string[]; + kept: UndoKept[]; + /** Files under `codegen/` that were left because they differ from the package. */ + keptCompanions: string[]; +} + +function diffCommand(name: string, packageName: string, path: string): string { + return `diff -u node_modules/${packageName}/src/reference/${name}.ts ${path}`; +} + +async function pruneEmptyDirs(cwd: string, dirs: readonly string[]): Promise { + for (const d of dirs) { + let dir = join(cwd, d); + // Stop at `codegen/`'s children: `codegen/` itself may hold the adopter's own files. + while (existsSync(dir) && readdirSync(dir).length === 0 && dir !== join(cwd, "codegen")) { + await rmdir(dir); + dir = dirname(dir); + } + } +} + +/** + * `meta eject --undo [...]`. Removes owned generator copies that are the shipped + * reference (file and every renderer it reaches), plus the renderer and adapter copies no + * remaining owned generator uses. It never edits `metaobjects.config.ts`, as eject never + * does: it names the import to swap back. + * + * Names are validated, and a customised copy refused, before anything is removed. With no + * names a customised copy is simply kept and said so. + */ +async function undoCommand(names: readonly string[], force: boolean, cwd: string, fmt: OutputFormat): Promise { + const bad = names.filter((n) => resolveSource(n) === undefined); + if (bad.length > 0) { + log.error( + `meta eject --undo takes generator names; not a generator: ${bad.join(", ")}. Nothing was removed. ` + + "A library or the docs-site theme is your own data once ejected, so it is never undone. " + + "Run `meta eject --list` to see what you own.", + ); + return 2; + } + const owned = (await generatorListGroups(cwd)).flatMap((g) => g.rows).filter((r) => r.status !== "available"); + const named = names.length > 0; + const targets = named ? owned.filter((r) => names.includes(r.name)) : owned.filter(isStockRow); + const kept: UndoKept[] = []; + const notOwned = names.filter((n) => !owned.some((r) => r.name === n)); + if (notOwned.length > 0) { + log.error( + `not an owned copy (nothing at ${OWNED_GENERATORS_DIR}/): ${notOwned.join(", ")}. Nothing was removed. ` + + "Run `meta eject --list` to see what you own.", + ); + return 2; + } + const refused = named && !force ? targets.filter((r) => !isStockRow(r)) : []; + if (refused.length > 0) { + for (const r of refused) { + const path = `${OWNED_GENERATORS_DIR}/${r.name}.ts`; + const changed = [ + ...(r.templates ?? []).filter((t) => t.status === "differs").map((t) => t.path), + ...(r.runtime ?? []), + ]; + log.error( + `${path} differs from the ${r.package} reference (${r.behind} line(s) behind it, ${r.own} of your own` + + `${changed.length > 0 ? `; changed with it: ${changed.join(", ")}` : ""}), ` + + `so undoing it would discard your changes. See them: ${diffCommand(r.name, r.package, path)}`, + ); + } + log.error("Nothing was removed. Pass --force to remove a customised copy anyway."); + return 2; + } + if (!named) { + for (const r of owned.filter((o) => !isStockRow(o))) { + kept.push({ + name: r.name, + path: `${OWNED_GENERATORS_DIR}/${r.name}.ts`, + reason: r.status === "owned" ? "owned, reference unreadable" : "customised — differs from the reference", + }); + } + } + + const gone = new Set(targets.map((r) => r.name)); + const remaining = owned.filter((r) => !gone.has(r.name)); + const stillReached = new Set(remaining.flatMap((r) => closureOf(r.name).templates.map((f) => f.path))); + const removed: string[] = []; + const keptCompanions: string[] = []; + + for (const r of targets) removed.push(`${OWNED_GENERATORS_DIR}/${r.name}.ts`); + const templateRows = await templateCopyStatus(cwd, packagedTemplates()); + const dropTemplates = new Set(targets.flatMap((r) => (r.templates ?? []).map((t) => t.path))); + for (const row of templateRows) { + if (!dropTemplates.has(row.path) || stillReached.has(row.path)) continue; + if (isStockVerdict(row.verdict)) removed.push(row.path); + else keptCompanions.push(row.path); + } + // The adapter source is one shared tree: it goes only when nothing left in the project emits + // code that imports it, and goes whole or not at all, so a forced undo never leaves half of it. + const adapterDropped = targets.some((r) => ejectsRuntime(r.name)) && !remaining.some((r) => ejectsRuntime(r.name)); + if (adapterDropped) { + const rows = await runtimeCopyStatus(cwd); + const changed = rows.filter((row) => !isStockVerdict(row.verdict)).map((row) => row.path); + if (changed.length === 0) removed.push(...rows.map((row) => row.path)); + else keptCompanions.push(...changed); + } + for (const rel of removed) await rm(join(cwd, rel), { force: true }); + await pruneEmptyDirs(cwd, [OWNED_TEMPLATES_DIR, OWNED_RUNTIME_DIR, OWNED_GENERATORS_DIR]); + + const undone: UndoneRow[] = targets.map((r) => { + const closure = closureOf(r.name); + const exportName = extractExportName(extractImportLine(closure.generator.content, r.name), r.name); + return { + name: r.name, + path: `${OWNED_GENERATORS_DIR}/${r.name}.ts`, + wire: { + import: `import { ${exportName} } from "${factoryModuleOf(r.name) ?? r.package}";`, + replaces: `import { ${exportName} } from "./${OWNED_GENERATORS_DIR}/${r.name}.js";`, + }, + }; + }); + + if (fmt !== "text") { + const payload: UndoPayload = { undone, removed, kept, keptCompanions }; + emitStructured(payload, fmt); + return 0; + } + if (undone.length === 0) { + log.info( + owned.length === 0 + ? "No owned generator copies: nothing to undo." + : "Nothing to undo: every owned copy differs from the shipped reference.", + ); + } else { + log.info(`Removed ${removed.length} file(s); ${undone.length} generator(s) run as shipped again:`); + for (const f of removed) log.info(` ${f}`); + log.info("In metaobjects.config.ts, REPLACE each import below (`meta gen` fails on the old one until you do):"); + for (const u of undone) log.info(` ${u.wire.replaces}\n -> ${u.wire.import}`); + if (adapterDropped && keptCompanions.length === 0) { + log.info( + `The shipped routes generator imports its HTTP adapter from ${HTTP_RUNTIME_PACKAGE} rather than ` + + `${OWNED_RUNTIME_DIR}/: install it, then re-run \`meta gen\`.`, + ); + } + } + for (const k of kept) log.info(`Kept ${k.path}: ${k.reason}.`); + if (keptCompanions.length > 0) { + log.info( + `Kept ${keptCompanions.length} file(s) under codegen/ that differ from the package: ` + + `${keptCompanions.join(", ")}. \`meta eject --list\` compares each.`, + ); + } + return 0; +} + // --------------------------------------------------------------------------- // the command // --------------------------------------------------------------------------- @@ -851,6 +1070,14 @@ export async function ejectCommand( return 2; } + if (flags.undo) { + if (flags.list) { + log.error("meta eject: --undo does not combine with --list."); + return 2; + } + return undoCommand(flags.names, flags.force, cwd, fmt); + } + if (flags.list) { if (fmt === "text") log.info(await listOutput(cwd)); else emitStructured(await listData(cwd), fmt); diff --git a/server/typescript/packages/cli/src/commands/gen.ts b/server/typescript/packages/cli/src/commands/gen.ts index b100ef26a..bdba49c35 100644 --- a/server/typescript/packages/cli/src/commands/gen.ts +++ b/server/typescript/packages/cli/src/commands/gen.ts @@ -21,8 +21,9 @@ import type { WriteStatus } from "@metaobjectsdev/codegen-ts"; import { packageOfResolutionKey } from "@metaobjectsdev/metadata"; import type { MetaRoot } from "@metaobjectsdev/metadata"; import type { Collection } from "@metaobjectsdev/sdk"; -import { reportLoadError } from "../lib/load-error.js"; -import { UNKNOWN_ATTR_GEN_ADVICE, unknownAttrWarnings } from "../lib/unknown-attr-warnings.js"; +import { describeLoadError, reportLoadError } from "../lib/load-error.js"; +import { STRICT_GEN_HINT, UNKNOWN_ATTR_GEN_ADVICE, unknownAttrWarnings } from "../lib/unknown-attr-warnings.js"; +import { stockCopyNotice } from "./eject.js"; import { buildCatalogListing, renderCatalogText, wiredGeneratorNames, ownedGeneratorNames, declaredDepsOf, @@ -143,6 +144,10 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat // projectRoot, not cwd, for the same reason its neighbour is — the manifest belongs to // whichever directory `resolveCollection` decided the metadata lives in. advise(warnIfManifestIgnored(projectRoot)); + // Owned generator copies that are the shipped reference, unchanged: they miss every later fix. + // Says nothing when the project owns none, and never affects the exit code. + const stockNotice = await stockCopyNotice(projectRoot).catch(() => undefined); + if (stockNotice !== undefined) { log.warn(stockNotice); advise(stockNotice); } // ADR-0034 Amendment 9: the config IS the list. With none there is nothing to generate, // documentation included, and a run that wrote nothing silently would read as a pass. @@ -186,9 +191,16 @@ export async function genCommand(args: string[], cwd: string, fmt: OutputFormat ...loadMemoryOptionsFrom(forgeConfig), }; try { - metadata = await loadMemory(genCollection.configDir, loadOptions); + // `--strict` is the load `meta verify` makes (ADR-0023): an unknown attribute stops the + // run before anything is written, instead of being warned about and generated anyway. + metadata = await loadMemory(genCollection.configDir, flags.strict ? { ...loadOptions, strict: true } : loadOptions); } catch (err) { reportLoadError(log, "failed to load metadata", err); + // The hint explains the strict-only rejection (an undeclared attribute); any other load + // failure is not something `--strict` caused, so it gets no hint. + if (flags.strict && describeLoadError(err).code === "ERR_UNKNOWN_ATTR") { + log.error(STRICT_GEN_HINT); + } return 1; } diff --git a/server/typescript/packages/cli/src/commands/init.ts b/server/typescript/packages/cli/src/commands/init.ts index 96262db78..107720a6e 100644 --- a/server/typescript/packages/cli/src/commands/init.ts +++ b/server/typescript/packages/cli/src/commands/init.ts @@ -90,9 +90,14 @@ dist/ *.tsbuildinfo `; -function buildMetaobjectsConfigBody(dialect: "sqlite" | "postgres" | "d1" = "sqlite"): string { +function buildMetaobjectsConfigBody(dialect: "sqlite" | "postgres" | "d1" = "sqlite", withDocs = false): string { + const docsImport = withDocs + ? `import { docsFile, apiDocsFile } from "@metaobjectsdev/codegen-ts/generators";\n` + : ""; + const docsTarget = withDocs ? ` targets: { docs: { outDir: "${DEFAULT_DOCS_DIR}" } },\n` : ""; + const generators = withDocs ? "[docsFile(), apiDocsFile()]" : "[]"; return `import { defineConfig } from "@metaobjectsdev/cli"; - +${docsImport} export default defineConfig({ outDir: "${SCAFFOLD_OUT_DIR}", dialect: "${dialect}", @@ -138,7 +143,7 @@ export default defineConfig({ // meta generator new openapi --scope model // // writes a working, commented generator into ./codegen/generators/ and adds it here. - generators: [], +${docsTarget} generators: ${generators}, }); `; } @@ -190,6 +195,12 @@ export interface InitOptions { * without acquiring a TypeScript project it will never use. */ configOnly?: boolean; + /** + * Wire the documentation generators (`docsFile()`, `apiDocsFile()`) and the `docs` + * target they write to, so the first `meta gen` produces documentation. Documentation + * is a generator like any other; this only chooses the two that need no further setup. + */ + withDocs?: boolean; } export interface InitResult { @@ -682,8 +693,13 @@ export async function init(opts: InitOptions): Promise { // Scaffold metaobjects.config.ts at the project root. Never overwrite if it exists. const forgeConfigPath = join(opts.cwd, "metaobjects.config.ts"); if (!(await fileExists(forgeConfigPath))) { - await writeFile(forgeConfigPath, buildMetaobjectsConfigBody(opts.d1 ? "d1" : "sqlite"), "utf8"); + await writeFile(forgeConfigPath, buildMetaobjectsConfigBody(opts.d1 ? "d1" : "sqlite", opts.withDocs === true), "utf8"); result.created.push("metaobjects.config.ts"); + } else if (opts.withDocs === true) { + result.warnings.push( + "--with-docs: metaobjects.config.ts already exists and was left alone. To write the docs, add " + + "`docsFile()` and `apiDocsFile()` from @metaobjectsdev/codegen-ts/generators to its `generators` list.", + ); } // Scaffold a minimal root .gitignore ONLY when the project has none — never @@ -914,6 +930,7 @@ export async function initCommand(args: string[], cwd: string): Promise wireRoot: flags.wireRoot, docsOnly: flags.docsOnly, configOnly: flags.configOnly, + withDocs: flags.withDocs, }); if (result.alreadyInitialized) { diff --git a/server/typescript/packages/cli/src/index.ts b/server/typescript/packages/cli/src/index.ts index f821f0b30..6e81cf7e3 100644 --- a/server/typescript/packages/cli/src/index.ts +++ b/server/typescript/packages/cli/src/index.ts @@ -29,6 +29,7 @@ USAGE: COMMANDS: init Scaffold a MetaObjects project in the current repo init --refresh-docs Refresh .metaobjects/AGENTS.md + CLAUDE.md after CLI upgrades + init --with-docs Scaffold with documentation wired (docsFile(), apiDocsFile()) so the first \`meta gen\` writes it init --config-only Write only .metaobjects/config.json — for a Maven- or pip-rooted project agent-docs Scaffold only the agent-context (.metaobjects/ + .claude/skills/) — canonical redirect target for all language ports gen [...] Codegen TS targets from your declared metadata @@ -36,6 +37,7 @@ COMMANDS: way to get any output MetaObjects does not ship (OpenAPI, JSON Schema, …) eject Copy a reference generator into codegen/generators/ to own it (any time after init) eject --list List every ejectable generator name, grouped by package + eject --undo Remove owned generator copies that are the shipped reference deps sync [...] Resolve declared dependencies (path, npm, python): sync the committed snapshot + sha256 lock (.metaobjects/deps/, .metaobjects/deps.lock.json) deps list One line per locked dependency: name, version, hash, node/package summary @@ -209,6 +211,9 @@ FLAGS: --generators Narrow the run to these DECLARED generators, by stable name, comma- separated (e.g. docs,api-docs to write only the documentation). A name metaobjects.config.ts does not declare is refused: it never adds one. + --strict Load the model strictly, as \`meta verify\` does (ADR-0023): an + unknown attribute (a typo'd \`@requird\`) is an error and nothing is + written, instead of a warning over output that silently changed. --format Output format (global flag; default toon off-TTY) [] Positional filter on entity names --help, -h Print this help @@ -231,6 +236,8 @@ NOTE: outDir, dialect, dbImport, extStyle are read from metaobjects.config.ts USAGE: meta eject ... Copy each generator into codegen/generators/.ts meta eject --list List every ejectable generator name, grouped by package + meta eject --undo [...] + Take back owned copies that are the shipped reference Eject a generator to CHANGE what it emits. To run one as shipped, import its factory from its package instead — \`meta gen --list\` prints the import line — and copy @@ -240,7 +247,15 @@ refuses the whole call and copies nothing. FLAGS: --list List ejectable generators instead of copying one - --force Overwrite an already-ejected file (default: never clobber) + --force Overwrite an already-ejected file (default: never clobber). + With --undo: remove a named copy even if you changed it + --undo Remove owned generator copies that are the shipped reference, + unchanged: with no names every such copy, with names those. A + copy you changed is kept (with no names) or refused (named, unless + --force). Their copied renderers and adapter source go too when + no remaining copy uses them. It never edits metaobjects.config.ts: + it prints the import to swap back. \`meta gen\` says when copies + qualify. There is no \`meta doctor\`; \`--list\` is the report. --format Output format (global flag). The structured form carries each file's wire lines, the consolidated install set and the config keys the ejected generators read. @@ -481,6 +496,8 @@ FLAGS: --no-wire-root Skip wiring root metaobjects.config.ts --config-only Write only .metaobjects/config.json (no TS scaffold) — declares metadata sources for the Node CLI from a Maven- or pip-rooted project + --with-docs Wire docsFile() and apiDocsFile() and a docs target in the scaffolded + config, so the first \`meta gen\` writes documentation --help, -h Print this help `, "agent-docs": `meta agent-docs — scaffold the agent-context (.metaobjects/ always-on files + .claude/skills/) diff --git a/server/typescript/packages/cli/src/lib/args.ts b/server/typescript/packages/cli/src/lib/args.ts index 80f626008..f2dbca752 100644 --- a/server/typescript/packages/cli/src/lib/args.ts +++ b/server/typescript/packages/cli/src/lib/args.ts @@ -18,6 +18,7 @@ export interface InitFlags { wireRoot: boolean; docsOnly: boolean; configOnly: boolean; + withDocs: boolean; } /** The flag table `parseInitArgs` parses. Exported so the help text can be gated against it. */ @@ -33,6 +34,7 @@ export const INIT_OPTIONS = { "no-wire-root": { type: "boolean", default: false }, "docs-only": { type: "boolean", default: false }, "config-only": { type: "boolean", default: false }, + "with-docs": { type: "boolean", default: false }, } as const; export function parseInitArgs(argv: string[]): InitFlags { @@ -54,6 +56,7 @@ export function parseInitArgs(argv: string[]): InitFlags { wireRoot: !values["no-wire-root"], docsOnly: !!values["docs-only"], configOnly: !!values["config-only"], + withDocs: !!values["with-docs"], }; } @@ -128,6 +131,12 @@ export interface GenFlags { * config does not declare is refused; a narrowing never adds a generator. */ generators?: string[]; + /** + * `--strict`: load the model strictly, as `meta verify` does (ADR-0023), so an unknown + * attribute is an error and nothing is written. Without it `gen` loads leniently and + * only warns. + */ + strict: boolean; } /** The flag table `parseGenArgs` parses. Exported so the help text can be gated against it. */ @@ -139,6 +148,7 @@ export const GEN_OPTIONS = { "no-antipatterns": { type: "boolean", default: false }, "limit": { type: "string" }, "generators": { type: "string" }, + "strict": { type: "boolean", default: false }, } as const; export function parseGenArgs(argv: string[]): GenFlags { @@ -166,6 +176,7 @@ export function parseGenArgs(argv: string[]): GenFlags { list: !!values.list, probe: !!values.probe, noAntipatterns: !!values["no-antipatterns"], + strict: !!values.strict, // Throws on a bad value; the command layer reports it and exits 2, exactly as // it does for --baseline above. limit: parseAdvisoryLimit(values.limit as string | undefined), @@ -795,12 +806,18 @@ export interface EjectFlags { list: boolean; /** Overwrite an already-ejected file; default false — eject never clobbers. */ force: boolean; + /** + * `--undo`: take back owned copies that are the shipped reference. With no names, every + * such copy; with names, those, and a copy that differs is refused unless `force`. + */ + undo: boolean; } /** The flag table `parseEjectArgs` parses. Exported so the help text can be gated against it. */ export const EJECT_OPTIONS = { "list": { type: "boolean", default: false }, "force": { type: "boolean", default: false }, + "undo": { type: "boolean", default: false }, } as const; export function parseEjectArgs(argv: string[]): EjectFlags { @@ -829,6 +846,7 @@ export function parseEjectArgs(argv: string[]): EjectFlags { names: positionals, list: !!values.list, force: !!values.force, + undo: !!values.undo, }; } diff --git a/server/typescript/packages/cli/src/lib/unknown-attr-warnings.ts b/server/typescript/packages/cli/src/lib/unknown-attr-warnings.ts index 369756283..9481ad153 100644 --- a/server/typescript/packages/cli/src/lib/unknown-attr-warnings.ts +++ b/server/typescript/packages/cli/src/lib/unknown-attr-warnings.ts @@ -2,14 +2,19 @@ import { relative, isAbsolute } from "node:path"; import { unknownAttributeFindings, type LoadMemoryOptions } from "@metaobjectsdev/sdk"; /** - * The trailing advice on every unknown-attribute warning `meta gen` prints. `gen` has no - * `--strict` flag of its own — `meta verify` is the strict door (ADR-0023), so that is what - * the warning names. + * The trailing advice on every unknown-attribute warning `meta gen` prints. `meta verify` + * and `meta gen --strict` are the strict doors (ADR-0023); a plain `gen` stays lenient. */ export const UNKNOWN_ATTR_GEN_ADVICE = "meta gen loads leniently and generated anyway, but `meta verify` rejects this metadata " + "(ADR-0023): fix or remove the attribute — a typo'd one (`isAbstrakt`, `@requird`) " + - "silently changes what is generated."; + "silently changes what is generated. `meta gen --strict` stops the run instead."; + +/** What `meta gen --strict` adds when the strict load rejects the model. Nothing was written. */ +export const STRICT_GEN_HINT = + "meta gen --strict loads strictly, as `meta verify` does (ADR-0023): every authored @attr must " + + "be declared, and nothing was written. Fix: register the attr on a metadata provider, OR move " + + "arbitrary author-supplied properties into an `attr.properties` bag, OR drop `--strict`."; /** * `meta gen`'s unknown-attribute warnings: one line per ADR-0023 finding a STRICT load of diff --git a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap index 9a85de3d0..9fac28ca2 100644 --- a/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap +++ b/server/typescript/packages/cli/test/__snapshots__/cli.test.ts.snap @@ -9,6 +9,7 @@ USAGE: COMMANDS: init Scaffold a MetaObjects project in the current repo init --refresh-docs Refresh .metaobjects/AGENTS.md + CLAUDE.md after CLI upgrades + init --with-docs Scaffold with documentation wired (docsFile(), apiDocsFile()) so the first \`meta gen\` writes it init --config-only Write only .metaobjects/config.json — for a Maven- or pip-rooted project agent-docs Scaffold only the agent-context (.metaobjects/ + .claude/skills/) — canonical redirect target for all language ports gen [...] Codegen TS targets from your declared metadata @@ -16,6 +17,7 @@ COMMANDS: way to get any output MetaObjects does not ship (OpenAPI, JSON Schema, …) eject Copy a reference generator into codegen/generators/ to own it (any time after init) eject --list List every ejectable generator name, grouped by package + eject --undo Remove owned generator copies that are the shipped reference deps sync [...] Resolve declared dependencies (path, npm, python): sync the committed snapshot + sha256 lock (.metaobjects/deps/, .metaobjects/deps.lock.json) deps list One line per locked dependency: name, version, hash, node/package summary diff --git a/server/typescript/packages/cli/test/cli.test.ts b/server/typescript/packages/cli/test/cli.test.ts index f5484f67d..41e992e88 100644 --- a/server/typescript/packages/cli/test/cli.test.ts +++ b/server/typescript/packages/cli/test/cli.test.ts @@ -6,7 +6,7 @@ import { tmpdir } from "node:os"; import { join } from "node:path"; // wireRoot defaults TRUE (root-wiring is on by default so the scaffolded always-on actually loads; --no-wire-root opts out). -const defaultInitFlags = { force: false, quiet: false, printOnly: false, refreshDocs: false, d1: false, servers: [], clients: [], noSkills: false, wireRoot: true, docsOnly: false, configOnly: false }; +const defaultInitFlags = { force: false, quiet: false, printOnly: false, refreshDocs: false, d1: false, servers: [], clients: [], noSkills: false, wireRoot: true, docsOnly: false, configOnly: false, withDocs: false }; describe("parseInitArgs", () => { test("default flags (wireRoot on by default, others off)", () => { @@ -27,6 +27,9 @@ describe("parseInitArgs", () => { test("--d1 toggles d1", () => { expect(parseInitArgs(["--d1"])).toEqual({ ...defaultInitFlags, d1: true }); }); + test("--with-docs toggles withDocs", () => { + expect(parseInitArgs(["--with-docs"])).toEqual({ ...defaultInitFlags, withDocs: true }); + }); test("--config-only toggles configOnly", () => { expect(parseInitArgs(["--config-only"])).toEqual({ ...defaultInitFlags, configOnly: true }); }); diff --git a/server/typescript/packages/cli/test/eject-undo.test.ts b/server/typescript/packages/cli/test/eject-undo.test.ts new file mode 100644 index 000000000..d0d71edae --- /dev/null +++ b/server/typescript/packages/cli/test/eject-undo.test.ts @@ -0,0 +1,296 @@ +// `meta eject --undo` takes back an ejected generator that never changed. A copy that is +// the shipped reference — byte for byte, or in the adopter's own formatting — freezes at the +// release it was copied from and misses every later fix, so a project that ejected during a +// period when the stock generators could not be imported keeps carrying it. Nothing used to +// notice. `meta gen` now says so, and `--undo` removes the copy. +import { afterEach, beforeEach, describe, expect, test } from "bun:test"; +import { existsSync, mkdtempSync, readdirSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { ejectCommand, stockCopyNotice } from "../src/commands/eject.js"; + +let logged: string[]; +let erred: string[]; +const origLog = console.log; +const origErr = console.error; + +beforeEach(() => { + logged = []; + erred = []; + console.log = (...a: unknown[]) => { logged.push(a.join(" ")); }; + console.error = (...a: unknown[]) => { erred.push(a.join(" ")); }; +}); +afterEach(() => { + console.log = origLog; + console.error = origErr; +}); + +function tmp(): string { + return mkdtempSync(join(tmpdir(), "meta-eject-undo-")); +} +const gen = (dir: string, name: string): string => join(dir, "codegen/generators", `${name}.ts`); +const text = (): string => [...logged, ...erred].join("\n"); +/** The adapter files eject copied under `codegen/runtime/`, project-relative. */ +const runtimeFiles = (dir: string): string[] => + existsSync(join(dir, "codegen/runtime")) + ? (readdirSync(join(dir, "codegen/runtime"), { recursive: true }) as string[]) + .filter((f) => f.endsWith(".ts")) + .map((f) => `codegen/runtime/${f.replaceAll("\\", "/")}`) + : []; + +describe("meta eject --undo", () => { + test("with no names, removes every owned copy that is the shipped reference", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["entity", "queries"], dir, "text")).toBe(0); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "entity"))).toBe(false); + expect(existsSync(gen(dir, "queries"))).toBe(false); + // Their copied template modules go with them, and the empty directories after. + expect(existsSync(join(dir, "codegen/templates"))).toBe(false); + expect(existsSync(join(dir, "codegen/generators"))).toBe(false); + // The config is the adopter's: undo names the edit, it never makes it. + expect(text()).toContain('import { entityFile } from "@metaobjectsdev/codegen-ts/generators"'); + expect(text()).toContain("metaobjects.config.ts"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a copy in the adopter's own formatting still counts as the reference", async () => { + const dir = tmp(); + try { + await ejectCommand(["barrel"], dir, "text"); + const file = gen(dir, "barrel"); + writeFileSync(file, readFileSync(file, "utf8").split("\n").map((l) => (/^\s*(import|export|const|return)\b.*;$/.test(l) ? ` ${l.slice(0, -1)}` : l)).join("\n"), "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(file)).toBe(false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("with no names, a customised copy is kept and said so", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "barrel"], dir, "text"); + writeFileSync(gen(dir, "barrel"), `${readFileSync(gen(dir, "barrel"), "utf8")}\n// MINE: a house rule\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "entity"))).toBe(false); + expect(existsSync(gen(dir, "barrel"))).toBe(true); + expect(text()).toContain("barrel"); + expect(text()).toMatch(/kept|customi[sz]ed|differs/i); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("naming a customised copy refuses the whole call and removes nothing", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "barrel"], dir, "text"); + writeFileSync(gen(dir, "barrel"), `${readFileSync(gen(dir, "barrel"), "utf8")}\n// MINE\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo", "entity", "barrel"], dir, "text")).toBe(2); + expect(existsSync(gen(dir, "entity"))).toBe(true); + expect(existsSync(gen(dir, "barrel"))).toBe(true); + expect(text()).toContain("diff -u"); + expect(text()).toContain("--force"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("--force removes a named customised copy", async () => { + const dir = tmp(); + try { + await ejectCommand(["barrel"], dir, "text"); + writeFileSync(gen(dir, "barrel"), `${readFileSync(gen(dir, "barrel"), "utf8")}\n// MINE\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo", "--force", "barrel"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "barrel"))).toBe(false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a template module another owned generator still reaches is kept", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "routes"], dir, "text"); + writeFileSync(gen(dir, "routes"), `${readFileSync(gen(dir, "routes"), "utf8")}\n// MINE\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "entity"))).toBe(false); + expect(existsSync(gen(dir, "routes"))).toBe(true); + // routes still imports whatever it shares with entity, so those files stay. + const imported = [...readFileSync(gen(dir, "routes"), "utf8").matchAll(/from "\.\.\/templates\/([^"]+)\.js"/g)]; + for (const m of imported) { + expect(existsSync(join(dir, "codegen/templates", `${m[1]}.ts`)), m[1]).toBe(true); + } + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a differing template module keeps its generator, which is reported", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity"], dir, "text"); + const tpl = readFileSync(gen(dir, "entity"), "utf8").match(/from "\.\.\/templates\/([^"]+)\.js"/)?.[1]; + expect(tpl).toBeDefined(); + const tplFile = join(dir, "codegen/templates", `${tpl}.ts`); + writeFileSync(tplFile, `${readFileSync(tplFile, "utf8")}\n// MINE\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "entity"))).toBe(true); + expect(existsSync(tplFile)).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("nothing identical: says so and changes nothing", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(text()).toMatch(/nothing to undo|no owned/i); + await ejectCommand(["barrel"], dir, "text"); + writeFileSync(gen(dir, "barrel"), `${readFileSync(gen(dir, "barrel"), "utf8")}\n// MINE\n`, "utf8"); + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "barrel"))).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("an unknown name, or a name that is not a generator, refuses and removes nothing", async () => { + const dir = tmp(); + try { + await ejectCommand(["barrel"], dir, "text"); + logged = []; erred = []; + expect(await ejectCommand(["--undo", "barrel", "nope"], dir, "text")).toBe(2); + expect(await ejectCommand(["--undo", "iam"], dir, "text")).toBe(2); + expect(existsSync(gen(dir, "barrel"))).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a named generator that was never ejected refuses, as a typo would, and removes nothing", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity"], dir, "text"); + logged = []; erred = []; + expect(await ejectCommand(["--undo", "barrel"], dir, "text")).toBe(2); + expect(text()).toContain("not an owned copy"); + expect(existsSync(gen(dir, "entity"))).toBe(true); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("a changed runtime adapter keeps the generator that reaches it, named or not", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["entity"], dir, "text")).toBe(0); + const [adapter] = runtimeFiles(dir); + expect(adapter).toBeDefined(); + const adapterPath = join(dir, adapter as string); + writeFileSync(adapterPath, `${readFileSync(adapterPath, "utf8")}\nexport const mine = 1;\n`, "utf8"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(existsSync(gen(dir, "entity"))).toBe(true); + expect(existsSync(adapterPath)).toBe(true); + logged = []; erred = []; + expect(await ejectCommand(["--undo", "entity"], dir, "text")).toBe(2); + expect(existsSync(gen(dir, "entity"))).toBe(true); + expect(text()).toContain(adapter as string); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("an unchanged runtime adapter goes whole with the last generator that reaches it", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["entity"], dir, "text")).toBe(0); + expect(runtimeFiles(dir).length).toBeGreaterThan(0); + expect(await ejectCommand(["--undo"], dir, "text")).toBe(0); + expect(runtimeFiles(dir)).toEqual([]); + expect(existsSync(gen(dir, "entity"))).toBe(false); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("--undo does not combine with --list", async () => { + const dir = tmp(); + try { + expect(await ejectCommand(["--undo", "--list"], dir, "text")).toBe(2); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("the structured form lists what was undone and the import to restore", async () => { + const dir = tmp(); + try { + await ejectCommand(["barrel"], dir, "text"); + logged = []; erred = []; + expect(await ejectCommand(["--undo"], dir, "json")).toBe(0); + const payload = JSON.parse(logged.join("\n")) as { + undone: Array<{ name: string; path: string; wire: { import: string } }>; + removed: string[]; + }; + expect(payload.undone.map((u) => u.name)).toEqual(["barrel"]); + expect(payload.undone[0]?.wire.import).toContain("@metaobjectsdev/codegen-ts"); + expect(payload.removed).toContain("codegen/generators/barrel.ts"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); + +describe("stockCopyNotice", () => { + test("is undefined when the project owns no generator copy", async () => { + const dir = tmp(); + try { + expect(await stockCopyNotice(dir)).toBeUndefined(); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("names the copies that are the shipped reference, and the command that removes them", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "barrel"], dir, "text"); + writeFileSync(gen(dir, "barrel"), `${readFileSync(gen(dir, "barrel"), "utf8")}\n// MINE\n`, "utf8"); + const notice = await stockCopyNotice(dir); + expect(notice).toContain("entity"); + expect(notice).not.toContain("barrel"); + expect(notice).toContain("meta eject --undo"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + + test("leaves out a copy whose runtime adapter changed", async () => { + const dir = tmp(); + try { + await ejectCommand(["entity", "barrel"], dir, "text"); + const [adapter] = runtimeFiles(dir); + const adapterPath = join(dir, adapter as string); + writeFileSync(adapterPath, `${readFileSync(adapterPath, "utf8")}\nexport const mine = 1;\n`, "utf8"); + const notice = await stockCopyNotice(dir); + expect(notice).toContain("barrel"); + expect(notice).not.toContain("entity"); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/cli/test/gen-unknown-attr-warning.test.ts b/server/typescript/packages/cli/test/gen-unknown-attr-warning.test.ts index 704faa437..4290ec63d 100644 --- a/server/typescript/packages/cli/test/gen-unknown-attr-warning.test.ts +++ b/server/typescript/packages/cli/test/gen-unknown-attr-warning.test.ts @@ -5,9 +5,10 @@ // `gen` now names each finding — attribute, node and file — as a warning, and keeps its // exit code. import { describe, test, expect, beforeEach, afterEach } from "bun:test"; -import { mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; +import { existsSync, mkdtempSync, mkdirSync, rmSync, writeFileSync } from "node:fs"; import { join, resolve } from "node:path"; import { run } from "../src/index.js"; +import { ejectCommand } from "../src/commands/eject.js"; const WORKSPACE_TMP = resolve(import.meta.dirname, "fixtures", "__tmp__"); @@ -121,3 +122,66 @@ describe("meta gen — unknown attributes are warned about, not swallowed", () = } }); }); + +describe("meta gen --strict — an unknown attribute is an error", () => { + test("fails with the strict load's error and writes nothing", async () => { + const root = setupRepo({ typos: true }); + try { + expect(await run(["gen", "--strict", "--cwd", root])).toBe(1); + const stderr = err.join("\n"); + expect(stderr).toContain("ERR_UNKNOWN_ATTR"); + expect(stderr).toContain("--strict"); + expect(existsSync(join(root, "generated"))).toBe(false); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("a clean model generates exactly as without it", async () => { + const root = setupRepo({ typos: false }); + try { + expect(await run(["gen", "--strict", "--cwd", root])).toBe(0); + expect(existsSync(join(root, "generated"))).toBe(true); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("without it the same model still generates, with the warning", async () => { + const root = setupRepo({ typos: true }); + try { + expect(await run(["gen", "--cwd", root])).toBe(0); + expect(existsSync(join(root, "generated"))).toBe(true); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); + +describe("meta gen — owned generator copies that are the shipped reference", () => { + test("names them once, with the command that removes them, and still exits 0", async () => { + const root = setupRepo({ typos: false }); + try { + mkdirSync(join(root, "codegen/generators"), { recursive: true }); + // `barrel` is the smallest reference generator; copied verbatim it is the stock one. + expect(await ejectCommand(["barrel"], root, "text")).toBe(0); + err = []; + expect(await run(["gen", "--cwd", root])).toBe(0); + const notice = err.filter((l) => l.includes("meta eject --undo")); + expect(notice).toHaveLength(1); + expect(notice[0]).toContain("barrel"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); + + test("says nothing when the project owns no generator copy", async () => { + const root = setupRepo({ typos: false }); + try { + expect(await run(["gen", "--cwd", root])).toBe(0); + expect(err.join("\n")).not.toContain("meta eject --undo"); + } finally { + rmSync(root, { recursive: true, force: true }); + } + }); +}); diff --git a/server/typescript/packages/cli/test/init.test.ts b/server/typescript/packages/cli/test/init.test.ts index 7a4bcf187..e78085e37 100644 --- a/server/typescript/packages/cli/test/init.test.ts +++ b/server/typescript/packages/cli/test/init.test.ts @@ -242,6 +242,37 @@ describe("init() — the owned-codegen tier is empty on purpose", () => { expect(configSrc).toContain("meta eject"); }); + test("--with-docs wires the docs generators and their target, and still copies no generator", async () => { + await init({ cwd, withDocs: true }); + const configSrc = readFileSync(join(cwd, "metaobjects.config.ts"), "utf8"); + expect(configSrc).toContain('import { docsFile, apiDocsFile } from "@metaobjectsdev/codegen-ts/generators";'); + expect(configSrc).toContain("generators: [docsFile(), apiDocsFile()]"); + expect(configSrc).toMatch(/targets: \{ docs: \{ outDir: "[^"]+" \} \}/); + expect(readdirSync(join(cwd, "codegen", "generators"))).toEqual([]); + }); + + test("--with-docs on a project that already has a config leaves it alone and says what to add", async () => { + writeFileSync(join(cwd, "metaobjects.config.ts"), "export default {};\n", "utf8"); + const result = await init({ cwd, withDocs: true }); + expect(readFileSync(join(cwd, "metaobjects.config.ts"), "utf8")).toBe("export default {};\n"); + expect(result.warnings.join("\n")).toContain("docsFile()"); + }); + + test("--with-docs writes documentation on the first `meta gen`", async () => { + const dir = mkGenProjectDir("with-docs-"); + try { + await init({ cwd: dir, withDocs: true, quiet: true }); + mkdirSync(join(dir, "metaobjects"), { recursive: true }); + writeFileSync(join(dir, "metaobjects", "meta.probe.json"), PROBE_ENTITY, "utf8"); + const { genCommand } = await import("../src/commands/gen.js"); + expect(await genCommand([], dir)).toBe(0); + expect(existsSync(join(dir, "docs", "generated"))).toBe(true); + expect(readdirSync(join(dir, "docs", "generated")).length).toBeGreaterThan(0); + } finally { + rmSync(dir, { recursive: true, force: true }); + } + }); + test("--print-only forecasts the same: the tier, not files in it", async () => { const result = await init({ cwd, printOnly: true }); expect(result.created).toContain("codegen/generators"); diff --git a/server/typescript/packages/cli/test/unit/args-gen.test.ts b/server/typescript/packages/cli/test/unit/args-gen.test.ts index 956fdfd85..c19bf66f0 100644 --- a/server/typescript/packages/cli/test/unit/args-gen.test.ts +++ b/server/typescript/packages/cli/test/unit/args-gen.test.ts @@ -10,10 +10,15 @@ describe("parseGenArgs", () => { baseline: "default", list: false, probe: false, noAntipatterns: false, + strict: false, limit: DEFAULT_ADVISORY_LIMIT, }); }); + test("--strict", () => { + expect(parseGenArgs(["--strict"]).strict).toBe(true); + }); + test("--list", () => { expect(parseGenArgs(["--list"]).list).toBe(true); }); diff --git a/server/typescript/packages/cli/test/unit/config-resolver.test.ts b/server/typescript/packages/cli/test/unit/config-resolver.test.ts index 2e368e3a7..78ed3469c 100644 --- a/server/typescript/packages/cli/test/unit/config-resolver.test.ts +++ b/server/typescript/packages/cli/test/unit/config-resolver.test.ts @@ -22,13 +22,13 @@ describe("resolveGenConfig", () => { // outDir/dialect/dbImport/extStyle. Only dryRun + entities come from flags. test("passes dryRun and entities through", () => { - const resolved = resolveGenConfig({ dryRun: true, entities: ["User", "Post"], baseline: "default", list: false, probe: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); + const resolved = resolveGenConfig({ dryRun: true, entities: ["User", "Post"], baseline: "default", list: false, probe: false, noAntipatterns: false, strict: false, limit: DEFAULT_ADVISORY_LIMIT }); expect(resolved.dryRun).toBe(true); expect(resolved.entities).toEqual(["User", "Post"]); }); test("defaults: dryRun false, entities empty", () => { - const resolved = resolveGenConfig({ dryRun: false, entities: [], baseline: "default", list: false, probe: false, noAntipatterns: false, limit: DEFAULT_ADVISORY_LIMIT }); + const resolved = resolveGenConfig({ dryRun: false, entities: [], baseline: "default", list: false, probe: false, noAntipatterns: false, strict: false, limit: DEFAULT_ADVISORY_LIMIT }); expect(resolved.dryRun).toBe(false); expect(resolved.entities).toEqual([]); }); diff --git a/server/typescript/packages/codegen-ts/src/entity-type-names.ts b/server/typescript/packages/codegen-ts/src/entity-type-names.ts new file mode 100644 index 000000000..81124bb5d --- /dev/null +++ b/server/typescript/packages/codegen-ts/src/entity-type-names.ts @@ -0,0 +1,46 @@ +// The six type names the entity module exports for an entity. A client written against the +// generated entity (a typed API client, a form, a service layer) imports these by name, so +// the spelling is part of the generator's contract, not an accident of three templates. +// `entityTypeNames` is the one place that spelling lives: the templates that emit the names +// call it, and an owned generator that refers to them calls it too, so a rename cannot +// leave one side behind. `docs/ports/typescript.md` ("Types exported per entity") is the +// reader-facing table; test/entity-type-names.test.ts pins it to the golden output. + +import type { MetaData } from "@metaobjectsdev/metadata"; + +export interface EntityTypeNames { + /** The row type read back from the table: `InferSelectModel` of the Drizzle table. */ + row: string; + /** What an insert accepts: `InferInsertModel` of the Drizzle table. */ + insert: string; + /** `Partial`. */ + update: string; + /** The Zod insert schema's INPUT type, the argument of `create`. */ + create: string; + /** The Zod update schema's INPUT type: every settable field optional, the argument of `update`. */ + patch: string; + /** The client filter object: filterable fields, `sort`, `limit`, `offset`, `or`, `and`. */ + filter: string; +} + +export interface EntityTypeNamesOpts { + /** + * The row type is `Row` instead of `` when the entity is a TPH + * discriminator base, because its discriminated-union type owns the bare name. + */ + tphBase?: boolean; + /** Use this spelling instead of `obj.name` (a value object renamed on a name collision). */ + name?: string; +} + +export function entityTypeNames(obj: MetaData, opts: EntityTypeNamesOpts = {}): EntityTypeNames { + const base = opts.name ?? obj.name; + return { + row: opts.tphBase ? `${base}Row` : base, + insert: `${base}Insert`, + update: `${base}Update`, + create: `${base}Create`, + patch: `${base}Patch`, + filter: `${base}Filter`, + }; +} diff --git a/server/typescript/packages/codegen-ts/src/generator.ts b/server/typescript/packages/codegen-ts/src/generator.ts index a42430072..93c86ad4e 100644 --- a/server/typescript/packages/codegen-ts/src/generator.ts +++ b/server/typescript/packages/codegen-ts/src/generator.ts @@ -133,7 +133,10 @@ export interface Generator { /** Named output target (registry key). Defaults to "default". */ target?: string; /** Marks the generator that produces entity modules — the runner uses its - * target as the entity-module target for cross-target import resolution. */ + * target as the entity-module target for cross-target import resolution, and + * aggregates it into `ctx.config.includeEntityModule`, which the api reference reads + * to document the model, its Zod schemas and its `relations()` only when something + * emits them. A copy ejected before the marker is matched by its name. */ emitsEntityModule?: boolean; /** FR-038 §8 — declare the output namespace this generator exclusively owns. * A full `meta gen` run already removes every file it recorded writing in a @@ -191,6 +194,11 @@ export interface Generator { * generators read to emit the `format` branch on a report list route. Without a marked * generator in the route file's target the routes are byte-identical to before export existed. */ emitsReportExport?: boolean; + /** Marks the generator that writes the `.queries.ts` data-access modules + * (`queriesFile()`). The runner aggregates it across the run into + * `ctx.config.includeQueries`, which the api reference reads to document the query helpers + * only when something emits them. A copy ejected before the marker is matched by its name. */ + emitsQueries?: boolean; /** Marks a generator that emits a CLIENT UI artifact — a form, a hook, a grid or * its columns. The runner aggregates it across the suite into * ResolvedGenConfig.includeUiTier, which `agent/ui.md` reads to decide whether diff --git a/server/typescript/packages/codegen-ts/src/generators/api-model.ts b/server/typescript/packages/codegen-ts/src/generators/api-model.ts index 8cda79f56..78c8a4a3c 100644 --- a/server/typescript/packages/codegen-ts/src/generators/api-model.ts +++ b/server/typescript/packages/codegen-ts/src/generators/api-model.ts @@ -290,6 +290,15 @@ export interface ApiModelContext { * run didn't configure). The Fastify REST surface is always documented (it is * the default-suite routes generator). Defaults to false. */ includeHonoRoutes?: boolean; + /** Whether a generator in the run writes the entity modules. The model, its Zod schemas and its + * `relations()` are documented only when it does; a caller that is not a run leaves it out + * and gets them. Default true. */ + includeEntityModule?: boolean; + /** Whether a generator in the run writes the query modules. The query helpers are documented + * only when it does; a caller that is not a run leaves it out and gets them. Default true. */ + includeQueries?: boolean; + /** Likewise for the Fastify REST routes. Default true. */ + includeRestRoutes?: boolean; /** The project's `apiPrefix`. Every generated route is mounted inside * `fastify.register(…, { prefix: apiPrefix })`, so it is part of the address a * documented endpoint is served at — a page that omits it is one path segment @@ -315,6 +324,11 @@ export function buildApiModel(root: MetaRoot, ctx: ApiModelContext): ApiModel { const apiPrefix = ctx.apiPrefix ?? ""; const relationMap = ctx.relationMap ?? buildRelationMap(root); const includeHono = ctx.includeHonoRoutes ?? false; + const surfaces = { + entity: ctx.includeEntityModule ?? true, + queries: ctx.includeQueries ?? true, + rest: ctx.includeRestRoutes ?? true, + }; const expose = ctx.expose ?? {}; const units: ApiUnitDoc[] = []; @@ -324,7 +338,7 @@ export function buildApiModel(root: MetaRoot, ctx: ApiModelContext): ApiModel { // every symbol documents is the derived one. `generatableObjects` is the same swap // `runGen` hands the generators, so the units are the objects code was emitted for. for (const obj of generatableObjects(root.objects(), root)) { - units.push(buildEntityUnit(obj, pkCtx, root, layout, relationMap, includeHono, apiPrefix, expose)); + units.push(buildEntityUnit(obj, pkCtx, root, layout, relationMap, includeHono, surfaces, apiPrefix, expose)); } for (const tmpl of templateOutputs(root)) { @@ -437,6 +451,7 @@ function buildEntityUnit( layout: OutputLayout, relationMap: RelationMap, includeHono: boolean, + surfaces: { entity: boolean; queries: boolean; rest: boolean }, apiPrefix: string, expose: RoutesExpose, ): ApiUnitDoc { @@ -449,7 +464,7 @@ function buildEntityUnit( const entityMod = entityModulePath(layout, obj, name); // --- model: the entity type/const the entity-file generator emits (bare name). --- - symbols.push({ + if (surfaces.entity) symbols.push({ name, kind: "model", importPath: entityMod, @@ -460,16 +475,16 @@ function buildEntityUnit( }); if (isQueryable(obj)) { - symbols.push(...dataAccessSymbols(obj, ctx, root, layout)); + if (surfaces.queries) symbols.push(...dataAccessSymbols(obj, ctx, root, layout)); // A read-only object's entity module exports a read schema only: no insert or update // schema exists to document. - if (!isReadOnlySurface(obj)) symbols.push(...validationSymbols(obj, entityMod)); + if (surfaces.entity && !isReadOnlySurface(obj)) symbols.push(...validationSymbols(obj, entityMod)); // REST needs no gate of its own: the routes generator's built-in filter is // `servesReadApi && !isTphSubtype` — exactly isQueryable — so every queryable // object gets routes. (Its `filter` option can narrow that further; this builder // reads the model, not the wired generator set, and cannot see it. See the module // header.) - if (expose.restMounted?.(obj) !== false) symbols.push(...restSymbols(obj, layout, root, apiPrefix, expose.rest)); + if (surfaces.rest && expose.restMounted?.(obj) !== false) symbols.push(...restSymbols(obj, layout, root, apiPrefix, expose.rest)); // The OPT-IN Hono variant mounts the SAME CRUD verbs under the same rules — // documented only when the adopter wired it. if (includeHono && expose.restHonoMounted?.(obj) !== false) { @@ -482,7 +497,7 @@ function buildEntityUnit( // keys by the entity whose module renders the block — the discriminator // base, for a TPH subtype; an abstract entity emits a value-object module // and never carries one. Independent of isQueryable. --- - const relationSym = relationSymbol(obj, entityMod, relationMap); + const relationSym = surfaces.entity ? relationSymbol(obj, entityMod, relationMap) : undefined; if (relationSym !== undefined) symbols.push(relationSym); // --- callable: call, only when the entity is backed by a stored-proc / @@ -1282,6 +1297,9 @@ export function buildApiModelForRun(ctx: GenContext): ApiModel { loadedRoot: ctx.loadedRoot, outputLayout: ctx.config.outputLayout ?? "flat", includeHonoRoutes: ctx.config.includeHonoRoutes ?? false, + ...(ctx.config.includeEntityModule !== undefined && { includeEntityModule: ctx.config.includeEntityModule }), + ...(ctx.config.includeQueries !== undefined && { includeQueries: ctx.config.includeQueries }), + ...(ctx.config.includeRestRoutes !== undefined && { includeRestRoutes: ctx.config.includeRestRoutes }), // The verbs the routes generators actually mount (`expose`), so a page never lists a // route the project does not serve. ...(ctx.config.routesExpose !== undefined && { expose: ctx.config.routesExpose }), diff --git a/server/typescript/packages/codegen-ts/src/generators/queries-file.ts b/server/typescript/packages/codegen-ts/src/generators/queries-file.ts index 80ee00a44..ca99779fe 100644 --- a/server/typescript/packages/codegen-ts/src/generators/queries-file.ts +++ b/server/typescript/packages/codegen-ts/src/generators/queries-file.ts @@ -40,6 +40,9 @@ export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generat const generator: Generator = { name: "queries-file", + // Read by the api reference (`apiDocsFile()`): it documents the query helpers only when + // a generator in the run writes them. + emitsQueries: true, filter, generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { diff --git a/server/typescript/packages/codegen-ts/src/index.ts b/server/typescript/packages/codegen-ts/src/index.ts index 2b37f7b30..3c03cc9e5 100644 --- a/server/typescript/packages/codegen-ts/src/index.ts +++ b/server/typescript/packages/codegen-ts/src/index.ts @@ -113,6 +113,7 @@ export { toCamelCase, toPascalCase, toSnakeCase } from "./naming.js"; export { enumValues } from "./enum-meta.js"; export { objectRefTarget } from "./model-walk.js"; export { servedPath } from "./api-surface.js"; +export { entityTypeNames, type EntityTypeNames, type EntityTypeNamesOpts } from "./entity-type-names.js"; // routesHandlerName is public for the same reason the routes-expose members below are: the // routes reference template tells an adopter retargeting to another HTTP framework to copy // `renderRoutesFile`'s body out of the package source, and that body names this. See the diff --git a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts index 129f70aae..a2667ae10 100644 --- a/server/typescript/packages/codegen-ts/src/metaobjects-config.ts +++ b/server/typescript/packages/codegen-ts/src/metaobjects-config.ts @@ -80,6 +80,18 @@ export interface ResolvedGenConfig { * api-docs reads this to AUTO-DETECT whether to document the Hono CRUD surface * (it otherwise mirrors the default Fastify-only suite). Undefined ⇒ false. */ includeHonoRoutes?: boolean; + /** Whether a generator in the run writes the `.ts` entity modules (the suite's + * `emitsEntityModule` markers, or an `entity-file` by name). api-docs documents the model, + * its Zod schemas and its `relations()` only then. Undefined ⇒ documented. */ + includeEntityModule?: boolean; + /** Whether a generator in the run writes the `.queries.ts` modules (the suite's + * `emitsQueries` markers, or a `queries-file` by name). api-docs documents the query + * helpers only then. Undefined ⇒ documented, which is what a caller that is not a run + * (a thin docs entrypoint) wants. */ + includeQueries?: boolean; + /** Likewise for the Fastify REST routes: a routes generator that mounts them (`mountsRoutes` + * on a non-Hono generator, or a `routes-file` by name). Undefined ⇒ documented. */ + includeRestRoutes?: boolean; /** The `expose` option the run's routes generators were built with (`runRoutesExpose`), per * routes surface, for the api reference to document only the verbs they mount. Absent documents all five. */ routesExpose?: RoutesExpose; diff --git a/server/typescript/packages/codegen-ts/src/reference/api-docs.ts b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts index 9cc1b4bda..db1b5d436 100644 --- a/server/typescript/packages/codegen-ts/src/reference/api-docs.ts +++ b/server/typescript/packages/codegen-ts/src/reference/api-docs.ts @@ -9,7 +9,11 @@ // targets: Markdown. The pages describe the code the stock TypeScript generators // emit (the entity module, queries, the Fastify routes and, when // `routesFileHono` is wired, the Hono ones, the prompt and output helpers); -// they import nothing. +// they import nothing. The entity module, the queries and the routes are +// described only when a generator in the run writes them +// (`emitsEntityModule`, `emitsQueries`, `mountsRoutes`, or a copy of the +// stock one by name); a generator of your own that writes them under +// another name needs the marker, or its surface is left out. // use-when: you want to change the API reference: which units get a page, where the // pages land, or which of the three forms are written. The packaged // `apiDocsFile()` writes these pages without this file; eject it only to diff --git a/server/typescript/packages/codegen-ts/src/reference/queries.ts b/server/typescript/packages/codegen-ts/src/reference/queries.ts index 7fde437cd..3be4632cb 100644 --- a/server/typescript/packages/codegen-ts/src/reference/queries.ts +++ b/server/typescript/packages/codegen-ts/src/reference/queries.ts @@ -160,6 +160,9 @@ export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generat const generator: Generator = { name: "queries-file", + // Read by the api reference (`apiDocsFile()`): it documents the query helpers only when + // a generator in the run writes them. + emitsQueries: true, filter, generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { diff --git a/server/typescript/packages/codegen-ts/src/runner.ts b/server/typescript/packages/codegen-ts/src/runner.ts index d60656a58..92192bd9a 100644 --- a/server/typescript/packages/codegen-ts/src/runner.ts +++ b/server/typescript/packages/codegen-ts/src/runner.ts @@ -1,6 +1,6 @@ import { join, relative, resolve, isAbsolute, dirname } from "node:path"; import { warnMissingPromptGenerators } from "./prompt-generator-gate.js"; -import { runEmitsHonoRoutes, runEmitsUiTier, runRoutesExpose, warnUnmarkedRoutesGenerators, warnUnmarkedUiGenerators } from "./ui-tier-gate.js"; +import { runEmitsEntityModule, runEmitsHonoRoutes, runEmitsQueries, runEmitsRestRoutes, runEmitsUiTier, runRoutesExpose, warnUnmarkedRoutesGenerators, warnUnmarkedUiGenerators } from "./ui-tier-gate.js"; import { warnRetiredCodegenAttrs } from "./retired-codegen-attrs.js"; import { tmpdir } from "node:os"; import { randomUUID } from "node:crypto"; @@ -664,6 +664,11 @@ export async function runGen(opts: RunGenOpts): Promise { // `MetaobjectsGenConfig extends Omit`, and it is the escape hatch for // an owned generator whose marker was dropped. const includeHonoRoutes = config.includeHonoRoutes ?? runEmitsHonoRoutes(config.generators); + // The api reference documents the entity module, the query helpers and the REST routes only + // when something in the run writes them. Run-scoped, like the Hono flag, and settable the same way. + const includeEntityModule = config.includeEntityModule ?? runEmitsEntityModule(config.generators); + const includeQueries = config.includeQueries ?? runEmitsQueries(config.generators); + const includeRestRoutes = config.includeRestRoutes ?? runEmitsRestRoutes(config.generators); // Same auto-detection for the CLIENT UI tier. `agent/ui.md` describes forms, grids // and the endpoints their hooks call; whether any of that is emitted is a generator @@ -875,6 +880,9 @@ export async function runGen(opts: RunGenOpts): Promise { dialect: config.dialect, outputLayout: selfTarget.outputLayout, includeHonoRoutes, + includeEntityModule, + includeQueries, + includeRestRoutes, includeNames: namesTargets.has(selfTarget.name), includeReportExport: reportExportTargets.has(selfTarget.name), includeUiTier, diff --git a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts index 57e6bc7d6..4cbc0624e 100644 --- a/server/typescript/packages/codegen-ts/src/templates/filter-type.ts +++ b/server/typescript/packages/codegen-ts/src/templates/filter-type.ts @@ -16,6 +16,7 @@ import { opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; +import { entityTypeNames } from "../entity-type-names.js"; import { enumValues } from "../enum-meta.js"; // currency is integer minor units (allowlist `integer: true`, row schema `z.number().int()`). @@ -79,6 +80,7 @@ function renderFieldUnion(field: MetaField): string { * the server's per-subtype allowlist. */ export function renderFilterType(entity: MetaObject, exclude?: string): Code { + const filterName = entityTypeNames(entity).filter; // fields() returns effective fields, so inherited fields (from extends:/super:) are included in filter types. const allFields = entity.fields().filter((c) => c.name !== exclude); const filterableFieldsList = allFields.filter((c) => c.attr(FIELD_ATTR_FILTERABLE) === true); @@ -93,12 +95,12 @@ export function renderFilterType(entity: MetaObject, exclude?: string): Code { if (fieldLines.length === 0 && sortFieldNames.length === 0) { return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: string; - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } @@ -112,13 +114,13 @@ export type ${entity.name}Filter = { })(); return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: ${sortType}; ${fieldLines.join("\n")} - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } diff --git a/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts b/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts index b880265b0..1ba0acd7f 100644 --- a/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts +++ b/server/typescript/packages/codegen-ts/src/templates/inferred-types.ts @@ -44,6 +44,7 @@ import { sharedEnumForField } from "../enum-shared.js"; import { sharedEnumImportSpecifier, providedEnumImportSpecifier } from "../enum-import.js"; import type { RenderContext } from "../render-context.js"; import { effectivePackage } from "../docs-paths.js"; +import { entityTypeNames } from "../entity-type-names.js"; /** * Emit Drizzle's InferSelectModel / InferInsertModel aliases for an entity. @@ -71,7 +72,7 @@ export function renderInferredTypes( const insertSym = imp("t:InferInsertModel@drizzle-orm"); const docs = renderDocsFor(entity); const docsPrefix = docs ? `${docs}\n` : ""; - const rowName = tphBase ? `${entity.name}Row` : entity.name; + const { row: rowName, insert: insertName, update: updateName } = entityTypeNames(entity, { tphBase }); // #214 — a write-through entity read-view routes READS to the replica view, so the // read row type is emitted from the VIEW's read schema (z.infer, carrying // the derived fields) by the entity-file composer, NOT here. `skipRow` suppresses @@ -80,14 +81,14 @@ export function renderInferredTypes( // inferred from the write TABLE (derived-free, #213). if (skipRow) { return code` -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } return code` ${docsPrefix}export type ${rowName} = ${selectSym}; -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } diff --git a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts index 246aac296..e6a526d36 100644 --- a/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts +++ b/server/typescript/packages/codegen-ts/src/templates/zod-validators.ts @@ -9,6 +9,7 @@ // object-array field collapsed to z.array(z.string()) and the JSON Schema sent // downstream (e.g. to LLM tool_use input_schema) lost the nested object shape. +import { entityTypeNames } from "../entity-type-names.js"; import { code, joinCode, imp, type Code } from "ts-poet"; import { MetaObject, MetaField, type MetaRoot, stripPackage } from "@metaobjectsdev/metadata"; import { @@ -529,6 +530,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const insertSchemaName = `${objName}InsertSchema`; const updateSchemaName = `${objName}UpdateSchema`; const preservingSchemaName = `${objName}InsertPreservingSchema`; + const { create: createName, patch: patchName } = entityTypeNames(obj, { name: objName }); const docs = renderDocsFor(obj); const docsPrefix = docs ? `${docs}\n` : ""; @@ -561,11 +563,11 @@ ${joinCode(updateFieldLines, { on: ",\n" })} /** Typed create shape for ${objName}: the insert schema's INPUT (pre-transform) type. A * renamed/dropped/misspelt field is a compile error at every \`create${objName}\` call site; * the schema still validates at runtime. */ -export type ${objName}Create = ${z}.input; +export type ${createName} = ${z}.input; /** Typed patch shape for ${objName}: every settable field, optional (FR-035 PATCH). A * renamed/dropped field is a compile error at every \`update${objName}\` call site. */ -export type ${objName}Patch = ${z}.input;${preservingBlock} +export type ${patchName} = ${z}.input;${preservingBlock} `; } diff --git a/server/typescript/packages/codegen-ts/src/ui-tier-gate.ts b/server/typescript/packages/codegen-ts/src/ui-tier-gate.ts index 52658d7b0..950a64277 100644 --- a/server/typescript/packages/codegen-ts/src/ui-tier-gate.ts +++ b/server/typescript/packages/codegen-ts/src/ui-tier-gate.ts @@ -79,6 +79,31 @@ export function runEmitsHonoRoutes(generators: readonly Generator[]): boolean { return generators.some((g) => g.emitsHonoRoutes === true); } +/** + * Does this run write the query modules? The api reference documents the query helpers only + * when it does: it used to describe the stock queries whatever the generator list held, so a + * project that wired none, or its own, got pages naming functions nothing exports. + * + * The marker, or the name `queriesFile()` carries: an owned copy keeps its name through + * `meta eject`, so a copy ejected before `emitsQueries` existed is still found. + */ +export function runEmitsQueries(generators: readonly Generator[]): boolean { + return generators.some((g) => g.emitsQueries === true || g.name === "queries-file"); +} + +/** Does this run write the entity modules (`.ts`: the model, its Zod schemas and its + * `relations()`)? Same reasoning, and same name fallback, as {@link runEmitsQueries}. */ +export function runEmitsEntityModule(generators: readonly Generator[]): boolean { + return generators.some((g) => g.emitsEntityModule === true || g.name === "entity-file"); +} + +/** Does this run mount the Fastify REST routes? Same reasoning, and same name fallback, as {@link runEmitsQueries}. */ +export function runEmitsRestRoutes(generators: readonly Generator[]): boolean { + return generators.some( + (g) => g.emitsHonoRoutes !== true && (g.mountsRoutes !== undefined || g.name === "routes-file"), + ); +} + /** * The `expose` option of each routes surface the run mounts, for the api reference to apply (a docs * page that lists DELETE for a route the adopter does not mount is a lie). diff --git a/server/typescript/packages/codegen-ts/test/api-docs-surfaces.test.ts b/server/typescript/packages/codegen-ts/test/api-docs-surfaces.test.ts new file mode 100644 index 000000000..814a9cfa1 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/api-docs-surfaces.test.ts @@ -0,0 +1,153 @@ +// The api reference described the stock queries and routes whatever the generator list held, +// so a project that wired none of them, or its own, got pages naming functions and endpoints +// nothing emits, with no drift to catch it. Each surface is now documented only when a +// generator in the run writes it. +import { describe, expect, test } from "bun:test"; +import { InMemoryStringSource, MetaDataLoader } from "@metaobjectsdev/metadata"; +import { buildApiModel } from "../src/generators/api-model.js"; +import { queriesFile } from "../src/generators/queries-file.js"; +import { routesFile } from "../src/generators/routes-file.js"; +import { routesFileHono } from "../src/generators/routes-file-hono.js"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { runGen } from "../src/runner.js"; +import { defineConfig } from "../src/metaobjects-config.js"; +import { apiDocsFile } from "../src/generators/api-docs-file.js"; +import { entityFile } from "../src/generators/entity-file.js"; +import { docsFile } from "../src/generators/docs-file.js"; +import { runEmitsEntityModule, runEmitsQueries, runEmitsRestRoutes } from "../src/ui-tier-gate.js"; + +async function loadRoot() { + const res = await new MetaDataLoader().load([ + new InMemoryStringSource( + JSON.stringify({ + "metadata.root": { + package: "acme::shop", + children: [{ + "object.entity": { + name: "Product", + children: [ + { "source.rdb": { "@table": "products" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ], + }, + }], + }, + }), + { id: "meta.json", format: "json" }, + ), + ]); + expect(res.errors).toEqual([]); + return res.root; +} + +const kinds = (model: ReturnType): Set => + new Set(model.units.find((u) => u.node === "Product")!.symbols.map((s) => s.kind)); + +describe("the api reference documents only the surfaces the run writes", () => { + test("a caller that is not a run still gets every surface", async () => { + const root = await loadRoot(); + const k = kinds(buildApiModel(root, { loadedRoot: root })); + expect(k.has("data-access")).toBe(true); + expect(k.has("rest")).toBe(true); + }); + + test("no queries generator: the query helpers are not documented", async () => { + const root = await loadRoot(); + const without = kinds(buildApiModel(root, { loadedRoot: root, includeQueries: false })); + expect(without.has("data-access")).toBe(false); + expect(without.has("rest")).toBe(true); + expect(without.has("model")).toBe(true); + }); + + test("no entity generator: the model, its schemas and relations() are not documented", async () => { + const root = await loadRoot(); + const without = kinds(buildApiModel(root, { loadedRoot: root, includeEntityModule: false })); + expect(without.has("model")).toBe(false); + expect(without.has("validation")).toBe(false); + expect(without.has("data-access")).toBe(true); + }); + + test("no routes generator: the REST endpoints are not documented", async () => { + const root = await loadRoot(); + const without = kinds(buildApiModel(root, { loadedRoot: root, includeRestRoutes: false })); + expect(without.has("rest")).toBe(false); + expect(without.has("model")).toBe(true); + }); +}); + +describe("what the run's generators say they write", () => { + test("queries: the shipped generator, found by its marker", () => { + expect(runEmitsQueries([])).toBe(false); + expect(runEmitsQueries([entityFile(), routesFile()])).toBe(false); + expect(runEmitsQueries([queriesFile()])).toBe(true); + }); + + test("queries: an owned copy ejected before the marker is found by its name", () => { + expect(runEmitsQueries([{ name: "queries-file", generate: () => [] }])).toBe(true); + expect(runEmitsQueries([{ name: "house-queries", generate: () => [] }])).toBe(false); + expect(runEmitsQueries([{ name: "house-queries", emitsQueries: true, generate: () => [] }])).toBe(true); + }); + + test("entity module: the shipped generator by its marker, an older owned copy by its name", () => { + expect(runEmitsEntityModule([])).toBe(false); + expect(runEmitsEntityModule([queriesFile(), routesFile()])).toBe(false); + expect(runEmitsEntityModule([entityFile()])).toBe(true); + expect(runEmitsEntityModule([{ name: "entity-file", generate: () => [] }])).toBe(true); + expect(runEmitsEntityModule([{ name: "house-entity", emitsEntityModule: true, generate: () => [] }])).toBe(true); + }); + + test("routes: Fastify counts, Hono alone does not", () => { + expect(runEmitsRestRoutes([])).toBe(false); + expect(runEmitsRestRoutes([routesFile()])).toBe(true); + expect(runEmitsRestRoutes([routesFileHono()])).toBe(false); + expect(runEmitsRestRoutes([routesFileHono(), routesFile()])).toBe(true); + }); +}); + +describe("through meta gen: the written pages follow the generator list", () => { + async function page(generators: Parameters[0]["generators"]): Promise { + const root = await loadRoot(); + const tmp = mkdtempSync(join(tmpdir(), "api-docs-surfaces-")); + try { + await runGen({ + config: defineConfig({ + outDir: join(tmp, "src"), extStyle: "none", dbImport: "../db", dialect: "sqlite", + targets: { docs: { outDir: join(tmp, "docs") } }, generators, + }), + metadata: root, projectRoot: tmp, + }); + return readFileSync(join(tmp, "docs", "api", "Product.md"), "utf8"); + } finally { rmSync(tmp, { recursive: true, force: true }); } + } + + test("entity and docs only: neither the queries nor the routes are described", async () => { + const p = await page([entityFile(), apiDocsFile()]); + expect(p).not.toContain("GET /products"); + expect(p).not.toContain("findProductById"); + }); + + test("with the queries and routes generators wired, both are", async () => { + const p = await page([entityFile(), queriesFile(), routesFile(), apiDocsFile()]); + expect(p).toContain("GET /products"); + expect(p).toContain("findProductById"); + expect(p).toContain("ProductInsertSchema"); + }); + + test("the meta init --with-docs scaffold (docs generators only) names nothing that is not generated", async () => { + const p = await page([docsFile(), apiDocsFile()]); + for (const symbol of ["interface Product", "ProductInsertSchema", "ProductUpdateSchema", "findProductById", "GET /products"]) { + expect(p).not.toContain(symbol); + } + }); + + test("a generator of your own that mounts routes and carries the marker counts", async () => { + const own = { ...routesFile(), name: "house-routes" }; + const p = await page([entityFile(), own, apiDocsFile()]); + expect(p).toContain("GET /products"); + expect(p).not.toContain("findProductById"); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/entity-type-names.test.ts b/server/typescript/packages/codegen-ts/test/entity-type-names.test.ts new file mode 100644 index 000000000..b12301cd3 --- /dev/null +++ b/server/typescript/packages/codegen-ts/test/entity-type-names.test.ts @@ -0,0 +1,48 @@ +// `entityTypeNames` is the contract for the six type names the entity module exports, and +// docs/ports/typescript.md lists them. The golden snapshot is the emitted truth: every name +// the helper returns for an entity must be exported by that entity's module, so the table +// cannot fall behind the generator. +import { describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { InMemoryStringSource, MetaDataLoader } from "@metaobjectsdev/metadata"; +import { entityTypeNames } from "../src/index.js"; + +const GOLDEN = join(import.meta.dirname, "golden", "__snapshots__", "sqlite"); + +async function entity(name: string) { + const res = await new MetaDataLoader().load([ + new InMemoryStringSource( + JSON.stringify({ + metadata: { package: "t", children: [{ "object.entity": { name, children: [{ "field.long": { name: "id" } }] } }] }, + }), + { id: "m.json", format: "json" }, + ), + ]); + expect(res.errors).toEqual([]); + return res.root.objects()[0]!; +} + +describe("entityTypeNames", () => { + test("spells the six names from the entity name", async () => { + expect(entityTypeNames(await entity("Subscriber"))).toEqual({ + row: "Subscriber", + insert: "SubscriberInsert", + update: "SubscriberUpdate", + create: "SubscriberCreate", + patch: "SubscriberPatch", + filter: "SubscriberFilter", + }); + }); + + test("a TPH discriminator base keeps the bare name for its union and reads rows as Row", async () => { + expect(entityTypeNames(await entity("Vehicle"), { tphBase: true }).row).toBe("VehicleRow"); + }); + + test("every name is a type exported by the golden Subscriber module", async () => { + const source = readFileSync(join(GOLDEN, "Subscriber.ts"), "utf8"); + for (const [role, name] of Object.entries(entityTypeNames(await entity("Subscriber")))) { + expect(source, role).toMatch(new RegExp(`^export type ${name}\\b`, "m")); + } + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/metaobjects-config.test.ts b/server/typescript/packages/codegen-ts/test/metaobjects-config.test.ts index cd3fdc403..78b919964 100644 --- a/server/typescript/packages/codegen-ts/test/metaobjects-config.test.ts +++ b/server/typescript/packages/codegen-ts/test/metaobjects-config.test.ts @@ -75,7 +75,7 @@ describe("defineConfig", () => { test("type-level: MetaobjectsGenConfig embeds ResolvedGenConfig's non-DB fields exactly (#194 — dbImport/dialect are OPTIONAL here, filled by the runner)", () => { // Everything except dbImport/dialect matches ResolvedGenConfig field-for-field. expectTypeOf>() - .toEqualTypeOf>(); + .toEqualTypeOf>(); // extStyle is OPTIONAL too: the config `meta init` scaffolds omits it and must still // typecheck; the runner fills DEFAULT_EXT_STYLE. expectTypeOf().toEqualTypeOf(); diff --git a/server/typescript/packages/metadata/src/metamodel-docs/render.ts b/server/typescript/packages/metadata/src/metamodel-docs/render.ts index 6e842f93b..87a974ba4 100644 --- a/server/typescript/packages/metadata/src/metamodel-docs/render.ts +++ b/server/typescript/packages/metadata/src/metamodel-docs/render.ts @@ -194,12 +194,12 @@ function renderIndex(defs: readonly TypeDefinition[]): string { const lines: string[] = [ header("Metamodel vocabulary index — every type.subType, its one-line description, and a link to its type page."), "# MetaObjects Metamodel — Index\n", - "This documents the **metamodel itself** — the type / subtype / attribute", - "vocabulary the loader accepts — generated from the strict registry. It is", - "NOT documentation of a user's entities (that is the `docs` generator, `docsFile()`).\n", - "Follow a link into `types/.md` for the full attribute table, allowed", - "children, and cardinality of a subtype. Universal documentation attributes", - "(`@description`/`@title`/…) apply to every node and are listed once in", + "This page documents the **metamodel itself**: the type, subtype and attribute", + "vocabulary the loader accepts, generated from the strict registry. Your own", + "entities are documented by the `docs` generator (`docsFile()`).\n", + "Each row links to its type page, which has the full attribute table, the allowed", + "children and the cardinality of that subtype. Documentation attributes", + "(`@description`, `@title` and the rest) apply to every node and are listed once in", "[providers.md](providers.md) under `metaobjects-documentation`.\n", "| Type.subType | Description | Page |", "| --- | --- | --- |", diff --git a/server/typescript/packages/test-generators/src/generators/queries.ts b/server/typescript/packages/test-generators/src/generators/queries.ts index 7fde437cd..3be4632cb 100644 --- a/server/typescript/packages/test-generators/src/generators/queries.ts +++ b/server/typescript/packages/test-generators/src/generators/queries.ts @@ -160,6 +160,9 @@ export const queriesFile = function queriesFile(opts?: QueriesFileOpts): Generat const generator: Generator = { name: "queries-file", + // Read by the api reference (`apiDocsFile()`): it documents the query helpers only when + // a generator in the run writes them. + emitsQueries: true, filter, generate: perEntity(async (entity, ctx) => { if (!ctx.renderContext) { diff --git a/server/typescript/packages/test-generators/src/templates/entity-type-names.ts b/server/typescript/packages/test-generators/src/templates/entity-type-names.ts new file mode 100644 index 000000000..81124bb5d --- /dev/null +++ b/server/typescript/packages/test-generators/src/templates/entity-type-names.ts @@ -0,0 +1,46 @@ +// The six type names the entity module exports for an entity. A client written against the +// generated entity (a typed API client, a form, a service layer) imports these by name, so +// the spelling is part of the generator's contract, not an accident of three templates. +// `entityTypeNames` is the one place that spelling lives: the templates that emit the names +// call it, and an owned generator that refers to them calls it too, so a rename cannot +// leave one side behind. `docs/ports/typescript.md` ("Types exported per entity") is the +// reader-facing table; test/entity-type-names.test.ts pins it to the golden output. + +import type { MetaData } from "@metaobjectsdev/metadata"; + +export interface EntityTypeNames { + /** The row type read back from the table: `InferSelectModel` of the Drizzle table. */ + row: string; + /** What an insert accepts: `InferInsertModel` of the Drizzle table. */ + insert: string; + /** `Partial`. */ + update: string; + /** The Zod insert schema's INPUT type, the argument of `create`. */ + create: string; + /** The Zod update schema's INPUT type: every settable field optional, the argument of `update`. */ + patch: string; + /** The client filter object: filterable fields, `sort`, `limit`, `offset`, `or`, `and`. */ + filter: string; +} + +export interface EntityTypeNamesOpts { + /** + * The row type is `Row` instead of `` when the entity is a TPH + * discriminator base, because its discriminated-union type owns the bare name. + */ + tphBase?: boolean; + /** Use this spelling instead of `obj.name` (a value object renamed on a name collision). */ + name?: string; +} + +export function entityTypeNames(obj: MetaData, opts: EntityTypeNamesOpts = {}): EntityTypeNames { + const base = opts.name ?? obj.name; + return { + row: opts.tphBase ? `${base}Row` : base, + insert: `${base}Insert`, + update: `${base}Update`, + create: `${base}Create`, + patch: `${base}Patch`, + filter: `${base}Filter`, + }; +} diff --git a/server/typescript/packages/test-generators/src/templates/filter-type.ts b/server/typescript/packages/test-generators/src/templates/filter-type.ts index c3e9cc7e7..e2fd030d3 100644 --- a/server/typescript/packages/test-generators/src/templates/filter-type.ts +++ b/server/typescript/packages/test-generators/src/templates/filter-type.ts @@ -16,6 +16,7 @@ import { opsForField, } from "@metaobjectsdev/metadata"; import { isSortableField } from "./filter-shared.js"; +import { entityTypeNames } from "./entity-type-names.js"; import { enumValues } from "@metaobjectsdev/codegen-ts/engine"; // currency is integer minor units (allowlist `integer: true`, row schema `z.number().int()`). @@ -79,6 +80,7 @@ function renderFieldUnion(field: MetaField): string { * the server's per-subtype allowlist. */ export function renderFilterType(entity: MetaObject, exclude?: string): Code { + const filterName = entityTypeNames(entity).filter; // fields() returns effective fields, so inherited fields (from extends:/super:) are included in filter types. const allFields = entity.fields().filter((c) => c.name !== exclude); const filterableFieldsList = allFields.filter((c) => c.attr(FIELD_ATTR_FILTERABLE) === true); @@ -93,12 +95,12 @@ export function renderFilterType(entity: MetaObject, exclude?: string): Code { if (fieldLines.length === 0 && sortFieldNames.length === 0) { return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: string; - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } @@ -112,13 +114,13 @@ export type ${entity.name}Filter = { })(); return code` -export type ${entity.name}Filter = { +export type ${filterName} = { limit?: number; offset?: number; sort?: ${sortType}; ${fieldLines.join("\n")} - or?: ${entity.name}Filter[]; - and?: ${entity.name}Filter[]; + or?: ${filterName}[]; + and?: ${filterName}[]; }; `; } diff --git a/server/typescript/packages/test-generators/src/templates/inferred-types.ts b/server/typescript/packages/test-generators/src/templates/inferred-types.ts index 588ec5fde..1eec56f31 100644 --- a/server/typescript/packages/test-generators/src/templates/inferred-types.ts +++ b/server/typescript/packages/test-generators/src/templates/inferred-types.ts @@ -44,6 +44,7 @@ import { sharedEnumForField } from "@metaobjectsdev/codegen-ts/engine"; import { sharedEnumImportSpecifier, providedEnumImportSpecifier } from "@metaobjectsdev/codegen-ts/engine"; import type { RenderContext } from "@metaobjectsdev/codegen-ts/engine"; import { effectivePackage } from "@metaobjectsdev/codegen-ts/engine"; +import { entityTypeNames } from "./entity-type-names.js"; /** * Emit Drizzle's InferSelectModel / InferInsertModel aliases for an entity. @@ -71,7 +72,7 @@ export function renderInferredTypes( const insertSym = imp("t:InferInsertModel@drizzle-orm"); const docs = renderDocsFor(entity); const docsPrefix = docs ? `${docs}\n` : ""; - const rowName = tphBase ? `${entity.name}Row` : entity.name; + const { row: rowName, insert: insertName, update: updateName } = entityTypeNames(entity, { tphBase }); // #214 — a write-through entity read-view routes READS to the replica view, so the // read row type is emitted from the VIEW's read schema (z.infer, carrying // the derived fields) by the entity-file composer, NOT here. `skipRow` suppresses @@ -80,14 +81,14 @@ export function renderInferredTypes( // inferred from the write TABLE (derived-free, #213). if (skipRow) { return code` -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } return code` ${docsPrefix}export type ${rowName} = ${selectSym}; -export type ${entity.name}Insert = ${insertSym}; -export type ${entity.name}Update = Partial<${entity.name}Insert>; +export type ${insertName} = ${insertSym}; +export type ${updateName} = Partial<${insertName}>; `; } diff --git a/server/typescript/packages/test-generators/src/templates/zod-validators.ts b/server/typescript/packages/test-generators/src/templates/zod-validators.ts index 5f8d45a2d..ea4945fea 100644 --- a/server/typescript/packages/test-generators/src/templates/zod-validators.ts +++ b/server/typescript/packages/test-generators/src/templates/zod-validators.ts @@ -9,6 +9,7 @@ // object-array field collapsed to z.array(z.string()) and the JSON Schema sent // downstream (e.g. to LLM tool_use input_schema) lost the nested object shape. +import { entityTypeNames } from "./entity-type-names.js"; import { code, joinCode, imp, type Code } from "@metaobjectsdev/codegen-ts/engine"; import { MetaObject, MetaField, type MetaRoot, stripPackage } from "@metaobjectsdev/metadata"; import { @@ -529,6 +530,7 @@ export function renderZodValidators(obj: MetaObject, ctx?: RenderContext): Code const insertSchemaName = `${objName}InsertSchema`; const updateSchemaName = `${objName}UpdateSchema`; const preservingSchemaName = `${objName}InsertPreservingSchema`; + const { create: createName, patch: patchName } = entityTypeNames(obj, { name: objName }); const docs = renderDocsFor(obj); const docsPrefix = docs ? `${docs}\n` : ""; @@ -561,11 +563,11 @@ ${joinCode(updateFieldLines, { on: ",\n" })} /** Typed create shape for ${objName}: the insert schema's INPUT (pre-transform) type. A * renamed/dropped/misspelt field is a compile error at every \`create${objName}\` call site; * the schema still validates at runtime. */ -export type ${objName}Create = ${z}.input; +export type ${createName} = ${z}.input; /** Typed patch shape for ${objName}: every settable field, optional (FR-035 PATCH). A * renamed/dropped field is a compile error at every \`update${objName}\` call site. */ -export type ${objName}Patch = ${z}.input;${preservingBlock} +export type ${patchName} = ${z}.input;${preservingBlock} `; } diff --git a/site-reference/index.html b/site-reference/index.html index 98acef2fd..243b50bef 100644 --- a/site-reference/index.html +++ b/site-reference/index.html @@ -45,12 +45,12 @@

MetaObjects Metamodel — Index

-

This documents the metamodel itself — the type / subtype / attribute -vocabulary the loader accepts — generated from the strict registry. It is -NOT documentation of a user's entities (that is the docs generator, docsFile()).

-

Follow a link into types/<family>.md for the full attribute table, allowed -children, and cardinality of a subtype. Universal documentation attributes -(@description/@title/…) apply to every node and are listed once in +

This page documents the metamodel itself: the type, subtype and attribute +vocabulary the loader accepts, generated from the strict registry. Your own +entities are documented by the docs generator (docsFile()).

+

Each row links to its type page, which has the full attribute table, the allowed +children and the cardinality of that subtype. Documentation attributes +(@description, @title and the rest) apply to every node and are listed once in providers.md under metaobjects-documentation.