Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,29 @@ here.**

## [Unreleased]

### Added

- **Report export: `GET /<segment>?format=csv|json|xml` on a served report, in all five ports.**
A new standalone `report-export` catalog generator (importable as shipped, ejectable in every
port, drift-checked by `verify --codegen`, never implied by a library) generates one export
module per served report plus one shared writer module per project, and makes the routes
generator emit the `format` branch. Without `format`, or with no export generator wired, a
response is byte-identical to before. CSV is RFC 4180 with CRLF, a header row of the derived
field names, wire-form values and a formula-injection guard on every string cell (the render
escapers' rule); JSON is the list route's body (the array, or the `withCount` envelope) plus
`Content-Disposition`; XML is `<{Report}><row><{field}>...` with a pinnable null form
(`xmlNulls`). A format the route does not wire answers
`400 {"error":"invalid_format","expected":[...]}`, `withCount` with `format=csv` or `format=xml`
answers `400 export_with_count`, and more rows than the generator option `maxRows` (default 100,000)
answers `400 {"error":"export_too_large","maxRows":N}` before a byte is written; nothing is ever
truncated. With `format` set `maxRows` governs and the list route's `report_too_large` cap does
not. The export adds no dependency an adopter must install in any port, and `xlsx` is reserved
(refused today like any unwired format). Pinned by six `report/scenarios/export-*.yaml`
scenarios run by every port's generated report lane. See
[docs/features/reporting.md](docs/features/reporting.md#exporting-a-report),
[docs/features/api-contract.md](docs/features/api-contract.md#report-export) and
[ADR-0031 Amendment 1](spec/decisions/ADR-0031-serialization-vs-data-download-boundary.md).

## [1.1.4] — 2026-10-26

_npm `1.1.4` (full lockstep across all 14 `@metaobjectsdev/*` publish candidates), PyPI `1.1.4`, NuGet `1.1.4` and Maven Central `8.1.4`._
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,7 @@ first-week wedge plan — and `meta init` picks up from there.
|---|---|---|---|
| TypeScript | **npm `1.1.4`** — the `@metaobjectsdev/*` packages | [`docs/ports/typescript.md`](docs/ports/typescript.md) | [`server/typescript/`](server/typescript/) · [`client/web/`](client/web/) |
| Java | **Maven Central `8.1.4`** (`com.metaobjects:*` — the JVM major is npm major + 7) — loader + OMDB + render + Maven plugin all shipped; full conformance green | [`docs/ports/java.md`](docs/ports/java.md) | [`server/java/`](server/java/) |
| Kotlin | **Maven Central `8.1.4`** — codegen tier on top of Java: 16 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs); 24 / 24 persistence-conformance | [`docs/ports/kotlin.md`](docs/ports/kotlin.md) | [`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/) |
| Kotlin | **Maven Central `8.1.4`** — codegen tier on top of Java: 17 generators (entity, Exposed table, names, relations, repository, output-parser, output-prompt, render-helper, extractor, filter-allowlist, validator, Spring config, storedProc, Spring controller, requirement-tests, api-docs, report-export); 24 / 24 persistence-conformance | [`docs/ports/kotlin.md`](docs/ports/kotlin.md) | [`server/java/codegen-kotlin/`](server/java/codegen-kotlin/) · [`server/java/metadata-ktx/`](server/java/metadata-ktx/) |
| C# | **NuGet `1.1.4`** — loader + conformance + EF Core codegen + render engine + `dotnet meta` CLI all shipped | [`docs/ports/csharp.md`](docs/ports/csharp.md) | [`server/csharp/`](server/csharp/) |
| Python | **PyPI `1.1.4`** — loader + conformance + render + entity-model codegen + ObjectManager runtime shipped; schema migrations are TS-owned (ADR-0015) | [`docs/ports/python.md`](docs/ports/python.md) | [`server/python/`](server/python/) |

Expand Down
12 changes: 6 additions & 6 deletions docs/CONFORMANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor
| [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ |
| [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ |
| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 59 (53 query + 6 migration) | all 59; the 24 `report-*` and the `-15` decimal scenarios also on SQLite, MySQL and D1, `decimal-exact` also on MySQL (a scenario's `engines:` key is exact) | 53 query (migrations TS-only, ADR-0015); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (via Exposed); the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query; the `report-*` and `engines:` scenarios also on SQLite and MySQL | 53 query (pg8000); the `report-*` and `engines:` scenarios also on SQLite (`sqlite3`) and MySQL (PyMySQL) |
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 106 (31 core + 10 tph + 9 m2m + 4 jsonb + 2 write-through + 12 projection + 27 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 117 (31 core + 10 tph + 9 m2m + 5 jsonb + 2 write-through + 12 projection + 37 report + 11 decimal) | ✓ (Fastify reference + generated lane); `report/` and `decimal/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC); `report/` and `decimal/` generated controllers over OMDB on Postgres, SQLite and MySQL | ✓ (embedded HTTP + Exposed); `report/` and `decimal/` also on SQLite and MySQL | ✓ (Kestrel + Npgsql); `report/` and `decimal/` also on SQLite and MySQL | ✓ (FastAPI + pg8000); `report/` and `decimal/` also on SQLite and MySQL |
| [`fixtures/decimal-wire/`](../fixtures/decimal-wire/) | 1 vector file (format, parse, filter operand, 15-digit read) | ✓ (`runtime-ts` and the generated helper) | ✓ (`CanonicalDecimal`) | ✓ (the generated wire file) | ✓ (`DecimalWire`) | ✓ (`decimal_wire`) |
| [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) |
| [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ |
Expand Down Expand Up @@ -348,9 +348,9 @@ REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall cloc
`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the
write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates).

### `fixtures/api-contract-conformance/` (106)
### `fixtures/api-contract-conformance/` (117)

All 106 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
All 117 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port
REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD
routes answer identically over HTTP — list / get / create / patch+put / delete,
plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1`
Expand All @@ -364,12 +364,12 @@ URL grammar `?filter[<field>][<op>]=<value>` (FR-009). On top of the 31 core
scenarios the corpus carries seven sub-corpora — `tph/` (10, single-table
inheritance; a subtype decimal filters and refuses extra digits), `m2m/` (9 — 3 plain, 5 gating TPH x M:N together, the combination
each corpus alone could not reach, and 1 pinning the collection-URL spelling),
`jsonb/` (4, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
`jsonb/` (5, typed value-object columns; numbers inside them: a double stays a JSON number, a decimal is the canonical string),
`write-through/` (2, table-write + view-read entities), `projection/` (12, a
read-only view answers reads and refuses writes with 405; a projection with no declared
identity has no item route; a projection keyed on a field not named `id` is addressed by
it, whether its identity names `@fields` or derives them; decimal and float fields filter), `decimal/` (11, ADR-0019 Amendment 1: a decimal, array element and undeclared-precision column included, is one
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (27,
canonical string, read exactly, filtered exactly and refused past its declared digits) and `report/` (37,
FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on
its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps
a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java,
Expand Down Expand Up @@ -555,7 +555,7 @@ files into a real Cube.
## Orphaned fixtures (tested but not yet documented)

The fixtures in the nine corpora mapped above (metamodel 413 + yaml 16 + verify 37
+ render 16 + persistence 59 + api-contract 106 + source-resolution 25 + scope 10 +
+ render 16 + persistence 59 + api-contract 117 + source-resolution 25 + scope 10 +
dependency 23) each map to a feature doc, and so do the two requirement corpora and the
`cube-model` corpus, whose case lists live in their own READMEs. None are orphaned today. The remaining
corpora in the totals table gate tooling contracts (registry manifests, provider
Expand Down
1 change: 1 addition & 0 deletions docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,7 @@ this tree is documentation, not the source of truth.
| Adopt a design MetaObjects already ships — users/groups/roles, an LLM trace envelope — instead of authoring it (`libraries`, `meta eject <library>`) | [`features/libraries.md`](features/libraries.md) |
| Record what the system is supposed to do, and stop agents reviving retired features | [`features/requirements.md`](features/requirements.md) |
| Declare what a dashboard groups by and counts (`dimension`, `measure`, `segment`, `object.report`; load-time checked; a view-backed report becomes a SQL view served by a generated read-only list route and a TypeScript list hook) | [`features/reporting.md`](features/reporting.md) |
| Download a report as CSV, JSON or XML (`?format=`, the `report-export` generator, its cap and null forms) | [`features/reporting.md`](features/reporting.md#exporting-a-report) |
| Export the reporting vocabulary to Cube (`cube-model`: cubes, joins, rollups; the `cube` lane that checks it against a real Cube) | [`features/cube-export.md`](features/cube-export.md) |
| Wire prompt construction (FR-004) | [`features/templates-and-payloads.md`](features/templates-and-payloads.md) |
| Share a metadata shape across multiple instances (abstracts, `extends:`) | [`features/abstracts-and-inheritance.md`](features/abstracts-and-inheritance.md) |
Expand Down
62 changes: 61 additions & 1 deletion docs/features/api-contract.md
Original file line number Diff line number Diff line change
Expand Up @@ -266,6 +266,9 @@ each handler they register instead; an `HTTPException` is rethrown to your
A `limit` below 1, a negative `offset` or a value that is not an integer is
`400 { "error": "pagination.invalid_value", "param": "limit" | "offset", "value", "expected" }` on
a report in every port. See [reporting.md](reporting.md#how-a-report-is-served).
- **A report list can be exported: `?format=csv|json|xml`.** See
[Report export](#report-export) below. Without `format` the response is byte-identical to the list
route described here.
- **No page-size maximum on an entity, and the default is per port.** The contract pins what
an explicit `limit`/`offset` returns (`list-with-pagination`); it does not pin a
default or a ceiling, and the ports differ. With `limit` **omitted**, the
Expand All @@ -281,6 +284,63 @@ each handler they register instead; an `HTTPException` is rethrown to your
`[<row>...]` to `{ rows: [<row>...], total: <N> }` (needed for grid
pagination). The grid hook always sends `withCount=1`.

### Report export

One selector on the existing report list route: `GET <apiPrefix>/<segment>?format=csv|json|xml`.
The route accepts `format` only when a `report-export` generator is wired for the project (see
[reporting.md](reporting.md#exporting-a-report)); a project that wires none sees no change at all,
and `?format=` is then ignored exactly as any unknown query parameter is today. **Accept headers are
never consulted**: the selector is the query parameter, in every port.

| Request | Answer |
|---|---|
| no `format` | Exactly the list route above: same body, same headers, same caps. |
| `format=json` | The list route's JSON body, unchanged, plus `Content-Disposition`: the array, or with `withCount=1` the `{ rows, total }` envelope. `Content-Type: application/json`. |
| `format=csv` | RFC 4180 CSV. `Content-Type: text/csv; charset=utf-8`. |
| `format=xml` | `application/xml`, shape below. |
| any other value (`xlsx`, `pdf`, `CSV`, empty) | `400 { "error": "invalid_format", "expected": ["csv", "json", "xml"] }`. `expected` lists the formats this route wires, in that order. `xlsx` is reserved for a later generator (`report-export-xlsx`) and is refused today like any unknown value. Matching is case-sensitive. |
| `format=csv` or `format=xml` and `withCount` truthy | `400 { "error": "export_with_count" }` (`message` is free prose). Only JSON carries the `{ rows, total }` envelope. |
| more rows than `maxRows` | `400 { "error": "export_too_large", "maxRows": N }` before a byte of the body is written. |

- **Header comparison.** Header names are case-insensitive. A runner compares `Content-Type` with the whitespace around `;` normalized (`text/csv;charset=utf-8` equals `text/csv; charset=utf-8`), because some servers re-spell it; every other header is compared exactly.
- **Headers.** Every successful export carries
`Content-Disposition: attachment; filename="<segment>.<ext>"` with `<ext>` one of `csv`, `json`,
`xml`. A response without `format` carries none.
- **Filters, sort, `limit` and `offset`** use the list route's parser, allowlists and 400 envelopes
unchanged. In export mode an absent `limit` means every row up to `maxRows`; an explicit `limit` or
`offset` is honoured.
- **One cap, two names.** `maxRows` is an option of the `report-export` generator, default 100,000.
The route fetches `maxRows + 1` rows and refuses with `export_too_large` rather than truncate, and
an explicit `limit` above `maxRows` is refused the same way. With `?format` set the export cap
governs and `report_too_large` never answers; without it the list route's own cap
(`report_too_large`, default 10,000) is untouched. A report is materialised up to the cap; there is
no streaming in this version.
- **CSV.** UTF-8, no BOM unless the generated constant `EXPORT_CSV_BOM` is flipped to true, CRLF after
every record including the last. One header row with the report's derived field names, in derived-field
order (dimensions, then measures, as `reportShape` lists them). A value is written in its wire
form: currency as integer minor units, a date as `YYYY-MM-DD`, a decimal as the `wire` setting spells
it, a boolean as `true`/`false`, null as an empty cell. A cell containing `"`, `,`, CR or LF is
quoted with embedded quotes doubled. **The formula-injection guard applies to every string-valued
cell** (`field.string`, an enum symbol, a uri): a value starting with `=`, `+`, `-`, `@`, TAB or CR is
prefixed with `'` before quoting. This is the render escapers' rule
(`render-csv-injection` fixture). Numbers, decimals, dates and booleans are never guarded, so a
negative number stays a number.
- **XML.** `<{ReportName}><row><{field}>value</{field}>...</row>...</{ReportName}>`, no XML declaration,
no indentation, UTF-8, values in the same wire form as CSV, text and attribute content escaped
(`&amp; &lt; &gt;`). There is no injection guard in XML. A null follows the generator option
`xmlNulls`: `"nil"` writes the field as an empty element with `xsi:nil="true"` and declares
`xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"` on the root; `"empty"` writes an empty element
and declares nothing. **The default is each port's own standard** (ADR-0019 Amendment 2): see the
per-port table in [reporting.md](reporting.md#exporting-a-report). The corpus pins `"nil"` in every lane
and compares the body as an XML infoset (names, attributes, text, order), not byte for byte.
- **JSON** is whatever the list route's standard serializer writes in that port, so a `wire` setting
applies to it exactly as it does to the list route.

The report-export scenarios (`report/scenarios/export-*.yaml`) use three assertion keys the rest of the
corpus does not: `headers` on `expect` (a map of lower-cased header name to its exact value, or `null` for
"the header is absent"), `text` on the body (the exact body, compared as UTF-8 text), and `xml` on the body
(compared as an XML infoset). A request may also carry `headers` (a map sent with the request).

### Base-URL policy

The two sides are configured **separately, and deliberately so**.
Expand Down Expand Up @@ -544,7 +604,7 @@ the empty rows, `isNull=true` matches nothing, and a sort on it orders the same
engine.

Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/)
(30 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
(37 scenarios), **generated lane only, on all five ports**, for the reason `projection/`
gives. The columns, their types and the per-port generated files are in
[reporting.md](reporting.md#how-a-report-is-served).

Expand Down
11 changes: 11 additions & 0 deletions docs/features/own-your-codegen.md
Original file line number Diff line number Diff line change
Expand Up @@ -1143,6 +1143,17 @@ from which the goal infers it. `requirement-tests` is one such name:
`KotlinRequirementTestsGenerator.kt`, each with its default rendering. The identity
function, the digest and the renderer and filter hook types stay in the package.

### `report-export` (all ports)

`report-export` is a standalone catalog generator, never implied by a library, that adds the
`?format=csv|json|xml` download to served report list routes. Every port ships it and ejects it
with its own command (`meta eject report-export`, `metaobjects eject`, `mvn metaobjects:eject
-Dnames=report-export`, `dotnet meta eject report-export`). The copy holds the CSV and XML writers,
so the guard, quoting and null form are yours to change. The routes generator emits the `format`
branch only when the generator is wired: every port's runner detects it from the generator list (on
Java and Kotlin the Maven plugin hands that list to `routes`). Options and defaults:
[reporting.md](reporting.md#exporting-a-report).

### Java `entity` and `extractor`

```bash
Expand Down
Loading
Loading