Skip to content

feat(migrate-ts): decide Postgres drift against a shadow database - #510

Merged
dmealing merged 8 commits into
mainfrom
fm/mo-pg-default-canon
Oct 29, 2026
Merged

dmealing merged 8 commits into
mainfrom
fm/mo-pg-default-canon

Conversation

@dmealing

@dmealing dmealing commented Oct 29, 2026 •

Copy link
Copy Markdown
Member

What Changed

  • Postgres drift (meta verify --db, meta migrate --from-db / --apply) now applies the expected schema to a scratch (shadow) database, introspects it with the live-database introspector, and diffs the two catalogs. The scratch database comes from --shadow-db, SHADOW_DATABASE_URL, or migrate.shadowDatabaseUrl, else a CREATE DATABASE that is always dropped afterwards. The rolled-back temp-object probe and the string normaliser remain as fallbacks, and every run reports which path ran.
  • Introspection reads identity and comments on Postgres. Identity add/drop plans as change-column-identity (dropping it needs --allow drop-identity-default), comment changes as change-comment (clearing one needs --allow drop-comment). The snapshot moves to formatVersion 4; v3 snapshots still load.
  • Column DEFAULTs that Postgres re-spells (NOW(), current_date, timestamptz date literals, '1e3', '007', composite expressions) are compared through the expr-canon probe instead of text, and a column rename that carries a CHECK is decided on the scratch copy. Real changes are still reported. The Postgres emitter no longer writes DEFAULT beside GENERATED BY DEFAULT AS IDENTITY.
Validation

Risk Assessment

⚠️ Medium: The latest fix round (drop-comment gate, v4 snapshot unknown marker, quoted-expression default parsing, new tests) checks out with no concrete defect found, but I did not review the earlier shadow-path machinery and the change is large, touching migration and drift logic.

Testing

I ran the real CLI against a disposable Postgres 16 container and a throwaway project. Authored default spellings read clean through both verify and migrate. Real default drift (seq 8 vs 7, starts_at now()+1 day) was reported by both commands, and an explicit unreachable shadow database is a hard error in both. Plain migrate with --shadow-db exits 2. A role without CREATEDB falls back to the temp-object probe, says so, and still catches drift. An undeclared comment is skipped unless --allow drop-comment is passed, while verify still reports it. The PG-backed integration suites passed (154 in migrate-ts, 27 in cli), after I fixed the migration-slug collision between test files. The earlier baseline ts-fast and ts-unit run passed, and scripts/ci-local.sh --quick finished green with 4 lanes skipped. I did not drive the SQLite and offline-normaliser fallbacks live, and the committed-snapshot old-format check was covered only by the existing suites.

  • Live validation: ✅ go - 7 of 8 scenarios driven live against the product
Scenario Result Live Evidence
Authored defaults NOW(), current_date, date literal, '1e3', 'open' read clean through verify and migrate --from-db on the shadow path ✅ pass live r5-default-spellings-and-drift.txt
Real default drift (seq 8 vs 7, starts_at now()+1 day, status, dbl) is reported by verify and planned by migrate --dry-run, with the same verdict ✅ pass live r5-default-spellings-and-drift.txt
Explicit unreachable --shadow-db or SHADOW_DATABASE_URL is a hard error naming the cause in verify and migrate; plain migrate refuses --shadow-db ✅ pass live r5-explicit-shadow-and-cleanup.txt
A reachable explicit shadow database is used, left empty, and no mo_shadow_ database is left after the run ✅ pass live r5-explicit-shadow-and-cleanup.txt
A role without CREATEDB falls back to the temp-object probe, says which path ran and why, and still detects default drift ✅ pass live r5-probe-fallback-no-createdb.txt
A shape with no special rule (column type change) is reported when it really differs; an undeclared comment is skipped by migrate without --allow drop-comment but still reported by verify ✅ pass live r5-comment-gate-and-type-drift.txt
PG-backed migrate-ts and cli integration suites (shadow round trip, schema-wide diff, rename carry, create-from-shadow, snapshot versions) pass against live Postgres ✅ pass live r5-migrate-ts-pg-integration.txt; r5-cli-pg-integration.txt
SQLite and offline string-normaliser fallback paths ⏸️ untested no Not driven through the CLI. The change is Postgres-specific, so the SQLite and offline paths were left to the existing unit suites.
Evidence: Authored default spellings clean, then real default drift (verify and migrate)
\### clean verify (authored NOW(), current_date, date literal, 1e3 as authored)
meta: Postgres drift compared against a scratch (shadow) database.
meta: meta verify — schema in sync with postgresql://postgres:***@127.0.0.1:47188/postgres.
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 0
summary: "2 gate(s) ran, all clean"
schemaDrift:
  changes: []
notRepresented[2]: "per-gate drift detail for every gate but schema (which template variable drifted, which generated file differs, which migration failed to replay) — printed as text on stderr; this payload carries each gate's pass/fail verdict, and the schema gate's differences in schemaDrift",the loader's own warnings and the agent-context/manifest advisories — printed as text on stderr
\### clean migrate --from-db --dry-run
changes: []
written: []
summary: no schema changes
help[1]: metadata and schema are in sync — nothing to do
notes[1]: Postgres drift compared against a scratch (shadow) database.
\### real drift: seq 8, starts_at now()+1day, status 'closed', dbl 2000
Time: 2.450 ms
meta: Postgres drift compared against a scratch (shadow) database.
meta: meta verify — schema drift vs postgresql://postgres:***@127.0.0.1:47188/postgres (4 change(s)):
meta:   ~ column ev.seq
meta:   ~ column ev.starts_at
meta:   ~ column ev.dbl
meta:   ~ column ev.status
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 1
summary: 1 of 2 gate(s) failed (schema)
changes[1]{kind,count}:
  change-column-default,4
written: []
summary: 4 change-column-default; preview only (nothing written)
help[1]: re-run without --dry-run to write the migration
notes[1]: Postgres drift compared against a scratch (shadow) database.
sql:
  up: "ALTER TABLE \"ev\" ALTER COLUMN \"seq\" SET DEFAULT '7';\n\nALTER TABLE \"ev\" ALTER COLUMN \"starts_at\" SET DEFAULT now();\n\nALTER TABLE \"ev\" ALTER COLUMN \"dbl\" SET DEFAULT '1000';\n\nALTER TABLE \"ev\" ALTER COLUMN \"status\" SET DEFAULT 'open';"
  down: "ALTER TABLE \"ev\" ALTER COLUMN \"status\" SET DEFAULT 'closed';\n\nALTER TABLE \"ev\" ALTER COLUMN \"dbl\" SET DEFAULT '2000';\n\nALTER TABLE \"ev\" ALTER COLUMN \"starts_at\" SET DEFAULT (now() + '1 day'::interval);\n\nALTER TABLE \"ev\" ALTER COLUMN \"seq\" SET DEFAULT '8';"
Evidence: Explicit shadow errors, plain-migrate refusal, shadow cleanup
\### restored verify:
exit=0
\### unreachable explicit shadow (verify)
meta: verify: failed to introspect postgresql://postgres:***@127.0.0.1:47188/postgres: shadow database: could not be opened: postgresql://nobody:***@127.0.0.1:1/nope: connect ECONNREFUSED 127.0.0.1:1
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 1
exit=1
\### unreachable explicit shadow (migrate --from-db)
meta: migrate: unexpected error: shadow database: could not be opened: postgresql://nobody:***@127.0.0.1:1/nope: connect ECONNREFUSED 127.0.0.1:1
error: "migrate: unexpected error: shadow database: could not be opened: postgresql://nobody:***@127.0.0.1:1/nope: connect ECONNREFUSED 127.0.0.1:1"
hint: run `meta migrate --help` for usage
exit=1
\### plain migrate with --shadow-db
meta: migrate: --shadow-db needs --from-db (or --apply): without it migrate compares the metadata with the committed snapshot and reads no database
error: "migrate: --shadow-db needs --from-db"
hint: re-run with --from-db to compare the metadata with the database
exit=2
\### explicit reachable shadow db
meta: Postgres drift compared against a scratch (shadow) database.
meta: meta verify — schema in sync with postgresql://postgres:***@127.0.0.1:47188/postgres.
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 0
shadow tables left:
0
\### leftover mo_shadow_ DBs:
0
\### SHADOW_DATABASE_URL env unreachable
meta: verify: failed to introspect postgresql://postgres:***@127.0.0.1:47188/postgres: shadow database: could not be opened: postgresql://nobody:***@127.0.0.1:1/nope: connect ECONNREFUSED 127.0.0.1:1
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 1
Evidence: Probe fallback for a role without CREATEDB
\### no-CREATEDB role: clean
meta: Postgres expressions compared via the temp-object probe on the target database (no scratch database was available): it needs write privileges and takes brief locks on the target. Pass --shadow-db <url> to avoid that. The scratch database was not used: could not be opened: CREATE DATABASE on the --db server failed: permission denied to create database.
meta: meta verify — schema in sync with postgresql://lim:***@127.0.0.1:47188/postgres.
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 0
\### no-CREATEDB role: real drift
meta: Postgres expressions compared via the temp-object probe on the target database (no scratch database was available): it needs write privileges and takes brief locks on the target. Pass --shadow-db <url> to avoid that. The scratch database was not used: could not be opened: CREATE DATABASE on the --db server failed: permission denied to create database.
meta: meta verify — schema drift vs postgresql://lim:***@127.0.0.1:47188/postgres (2 change(s)):
meta:   ~ column ev.seq
meta:   ~ column ev.starts_at
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 1
changes[1]{kind,count}:
summary: 2 change-column-default; preview only (nothing written)
notes[1]: "Postgres expressions compared via the temp-object probe on the target database (no scratch database was available): it needs write privileges and takes brief locks on the target. Pass --shadow-db <url> to avoid that. The scratch database was not used: could not be opened: CREATE DATABASE on the --db server failed: permission denied to create database."
  up: "ALTER TABLE \"ev\" ALTER COLUMN \"seq\" SET DEFAULT '7';\n\nALTER TABLE \"ev\" ALTER COLUMN \"starts_at\" SET DEFAULT NOW();"
  down: "ALTER TABLE \"ev\" ALTER COLUMN \"starts_at\" SET DEFAULT (now() + '1 day'::interval);\n\nALTER TABLE \"ev\" ALTER COLUMN \"seq\" SET DEFAULT '8';"
\### restored
exitCode: 0
Evidence: Comment gate and column type drift
\### undeclared comment + type change: verify
meta: Postgres drift compared against a scratch (shadow) database.
meta: meta verify — schema drift vs postgresql://postgres:***@127.0.0.1:47188/postgres (2 change(s)):
meta:   ~ column ev.seq (bigint → integer)
meta:   ~ comment ev (comment)
meta: meta verify — not run: templates (--templates), codegen (--codegen), deps (--deps), replay (--replay)
exitCode: 1
\### migrate dry-run (no allow)
changes[1]{kind,count}:
  change-column-type,1
summary: 1 change-column-type; not applied
notes[2]: "kept the comment on ev, which the metadata does not declare; pass --allow drop-comment to clear it.",Postgres drift compared against a scratch (shadow) database.
\### migrate dry-run --allow drop-comment
  change-column-type,1
  change-comment,1
summary: "1 change-column-type, 1 change-comment; not applied"
\### restored
exitCode: 0
Evidence: migrate-ts PG integration result
bun test v1.3.14 (0d9b296a)

 154 pass
 0 fail
 491 expect() calls
Ran 154 tests across 27 files. [8.71s]
Evidence: cli PG integration result
 27 pass
 0 fail
Ran 27 tests across 6 files. [4.37s]
Evidence: ci-local --quick log (exit 0)
metaobjects local CI  (mode: quick — TS + drift gates only)

── ▶ leak-scan (security) ──────────────────────────────────────────────
leak-scan: clean
── ✓ leak-scan (security) (0s)

── ▶ pom-version parity ──────────────────────────────────────────────
check-pom-versions: ✓ 3 parent + 1 property pom(s) match reactor 8.1.4
── ✓ pom-version parity (0s)

── ▶ bun-version parity ──────────────────────────────────────────────
check-bun-version: bun 1.3.14 matches the workflow pin.
── ✓ bun-version parity (0s)

── ▶ uv.lock version parity ──────────────────────────────────────────────
check-uv-lock-version: uv.lock agrees with pyproject.toml (1.1.4)
── ✓ uv.lock version parity (0s)

── ▶ publish-intent parity ──────────────────────────────────────────────
publish-intent parity: OK (lockstep 1.1.4; 2 declared source-only)
── ✓ publish-intent parity (0s)

── ▶ publish-set parity ──────────────────────────────────────────────
   1. @metaobjectsdev/metadata  (server/typescript/packages/metadata)
   2. @metaobjectsdev/render  (server/typescript/packages/render)
   3. @metaobjectsdev/docs-site  (server/typescript/packages/docs-site)
   4. @metaobjectsdev/migrate-ts  (server/typescript/packages/migrate-ts)
   5. @metaobjectsdev/codegen-ts  (server/typescript/packages/codegen-ts)
   6. @metaobjectsdev/runtime-ts  (server/typescript/packages/runtime-ts)
   7. @metaobjectsdev/sdk  (server/typescript/packages/sdk)
   8. @metaobjectsdev/runtime-web  (client/web/packages/runtime-web)
   9. @metaobjectsdev/codegen-ts-react  (server/typescript/packages/codegen-ts-react)
  10. @metaobjectsdev/codegen-ts-tanstack  (server/typescript/packages/codegen-ts-tanstack)
  11. @metaobjectsdev/react  (client/web/packages/react)
  12. @metaobjectsdev/tanstack  (client/web/packages/tanstack)
  13. @metaobjectsdev/cli  (server/typescript/packages/cli)
  14. @metaobjectsdev/ai-runtime  (server/typescript/packages/ai-runtime)
publish set: OK (lockstep 1.1.4; 14 packages, tier-ordered, closed over sibling deps)
ok:   happy path: private + off-lockstep excluded, order [metadata, render, docs-site, cli]
ok:   untiered member: threw on "publish order undeclared"
ok:   unclosed set: threw on "not closed over its own runtime dependencies"
ok:   inverted tier order: threw on "publishes a dependency AFTER its dependent"
ok:   regression: docs-site (#2) publishes before cli (#12) in the real tree
ok:   regression: docs-site has a declared tier

publish-set tests: all passed
── ✓ publish-set parity (0s)

── ▶ no committed pre-release version ──────────────────────────────────────────────
check-no-prerelease-versions: ✓ no pre-release version in any version declaration
── ✓ no committed pre-release version (0s)

── ▶ script-name hook collisions ──────────────────────────────────────────────
script-name-hooks: ✓ 10 root scripts, no lifecycle-hook collisions
── ✓ script-name hook collisions (0s)

── ▶ ci lane selection ──────────────────────────────────────────────
ok:   [server/typescript/packages/cli/src/x.ts] -> 'ts'
ok:   [client/web/packages/react/src/y.tsx] -> 'ts'
ok:   [server/java/metadata/src/main/java/A.java] -> 'java'
ok:   [server/java/codegen-kotlin/src/main/kotlin/B.kt] -> 'java'
ok:   [server/python/src/metaobjects/loader.py] -> 'python'
ok:   [server/csharp/MetaObjects/Loader.cs] -> 'csharp'
ok:   [fixtures/conformance/foo/meta.json] -> 'ts java python csharp'
ok:   [spec/metamodel.md] -> 'ts java python csharp'
ok:   [scripts/ci-local.sh] -> 'ts java python csharp'
ok:   [.github/workflows/local-ci.yml] -> 'ts java python csharp'
ok:   [agent-context/skills/x.md] -> 'ts java python csharp'
ok:   [weird-new-toplevel/file.txt] -> 'ts java python csharp'
ok:   [docs/features/cli.md] -> ''
ok:   [README.md CHANGELOG.md] -> ''
ok:   [server/typescript/a.ts server/python/b.py] -> 'ts python'
ok:   [docs/x.md fixtures/y.json] -> 'ts java python csharp'
ALL PASS
ok:   union all -> 'ts java python csharp'
ok:   union reorders to canonical -> 'ts java python csharp'
ok:   union deduplicates -> 'ts java'
ok:   union single -> 'python'
ok:   union empty -> ''
ok:   union drops unknown tokens -> 'ts'
ok:   every lane green -> nothing to re-run -> ''
ok:   docs-only run skips a red java -> java still not green -> 'java'
ok:   one red ts lane selects the whole ts port -> 'ts'
ok:   skipped is walked past to the real verdict -> ''
ok:   cancelled is not green -> 'csharp'
ok:   a lane with no verdict in the window is not green -> 'python'
ok:   no history at all -> everything -> 'ts java python csharp'
ok:   every port-testing job in .github/workflows/local-ci.yml is mapped by lane_port -> ''
ok:   every lane in lane_port exists as a job in .github/workflows/local-ci.yml -> ''
all checks passed
── ✓ ci lane selection (0s)

── ▶ extract field kinds ──────────────────────────────────────────────
extract FieldKind: 4 ports agree on [STRING, INT, LONG, DOUBLE, BOOLEAN, ENUM, OBJECT] (1 sanctioned deviation(s): csharp)
── ✓ extract field kinds (0s)

── ▶ no-magic gate wired (5 ports) ──────────────────────────────────────────────
no-magic gate coverage: 5 ports wired; 13 shapes tracked, 0 known gaps
── ✓ no-magic gate wired (5 ports) (0s)

── ▶ test-file references resolve ──────────────────────────────────────────────
test-file references: 32 named, all resolve
── ✓ test-file references resolve (0s)

── ▶ metamodel-version bump ──────────────────────────────────────────────

  ⚠ metamodel-version: 18 prose field(s) changed since v1.1.4:
    - field.base @formExclude: description
    - field.boolean @formExclude: description
    - field.currency @formExclude: description
    - field.date @formExclude: description
    - field.decimal @formExclude: description
    - field.double @formExclude: description
    - field.enum @formExclude: description
    - field.float @formExclude: description
    - field.inet @formExclude: description
    - field.int @formExclude: description
    - field.long @formExclude: description
    - field.map @formExclude: description
    … and 6 more

    Did the RULE change, or only its wording? A rule can change with no
    machine-readable footprint — #210 retired assembly origins from object.value
    and its only manifest edit was a `rules` string. If the rule changed, this is
    a metamodel MAJOR: run
      node scripts/check-metamodel-version.mjs --set <version>

  metamodel-version: none change since v1.1.4; declared 1.2 — ok.
ok:   no change classifies as nothing
ok:   a removed subtype is BREAKING
ok:   an added subtype is ADDITIVE
ok:   a removed attr is BREAKING
ok:   an added OPTIONAL attr is ADDITIVE
ok:   an added REQUIRED attr is BREAKING
ok:   optional → required is BREAKING
ok:   required → optional is ADDITIVE
ok:   a changed valueType is BREAKING
ok:   a changed isArray is BREAKING
ok:   an enum member REMOVE

... [473730 bytes truncated] ...

    if (true) {

[Survived] ConditionalExpression
src/super-resolve.ts:295:11
-         if (parsed !== undefined) {
+         if (true) {

[Survived] ConditionalExpression
src/super-resolve.ts:297:13
-           if (ownerNode !== undefined && ownerNode !== "over-drop") resolveNode(ownerNode);
+           if (true) resolveNode(ownerNode);

[Survived] LogicalOperator
src/super-resolve.ts:297:13
-           if (ownerNode !== undefined && ownerNode !== "over-drop") resolveNode(ownerNode);
+           if (ownerNode !== undefined || ownerNode !== "over-drop") resolveNode(ownerNode);

[Survived] ConditionalExpression
src/super-resolve.ts:297:13
-           if (ownerNode !== undefined && ownerNode !== "over-drop") resolveNode(ownerNode);
+           if (true && ownerNode !== "over-drop") resolveNode(ownerNode);

[Survived] ConditionalExpression
src/super-resolve.ts:297:40
-           if (ownerNode !== undefined && ownerNode !== "over-drop") resolveNode(ownerNode);
+           if (ownerNode !== undefined && true) resolveNode(ownerNode);

[Survived] StringLiteral
src/super-resolve.ts:297:54
-           if (ownerNode !== undefined && ownerNode !== "over-drop") resolveNode(ownerNode);
+           if (ownerNode !== undefined && ownerNode !== "") resolveNode(ownerNode);

[Survived] ObjectLiteral
src/super-resolve.ts:333:21
-             referrer: { type: node.type, subType: node.subType },
+             referrer: {},

[Survived] ObjectLiteral
src/super-resolve.ts:332:19
-             target: { type: target.type, subType: target.subType },
+             target: {},

[Survived] StringLiteral
src/super-resolve.ts:351:15
-           kind: "unresolved",
+           kind: "",

Ran 0.99 tests per mutant on average.
--------------------|------------------|----------|-----------|------------|----------|----------|
                    | % Mutation score |          |           |            |          |          |
File                |  total | covered | # killed | # timeout | # survived | # no cov | # errors |
--------------------|--------|---------|----------|-----------|------------|----------|----------|
All files           |  54.62 |   54.62 |      951 |        13 |        801 |        0 |        0 |
 parser-core.ts     |  45.50 |   45.50 |      475 |         0 |        569 |        0 |        0 |
 provider.ts        |  61.54 |   61.54 |       39 |         1 |         25 |        0 |        0 |
 serializer-json.ts |  64.49 |   64.49 |      199 |         8 |        114 |        0 |        0 |
 subtype-rules.ts   |  69.23 |   69.23 |      117 |         0 |         52 |        0 |        0 |
 super-resolve.ts   |  75.30 |   75.30 |      121 |         4 |         41 |        0 |        0 |
--------------------|--------|---------|----------|-----------|------------|----------|----------|
^[[32m01:14:06 (3195323) INFO MutationTestReportHelper^[[39m Final mutation score of 54.62 is greater than or equal to break threshold 35
^[[32m01:14:06 (3195323) INFO MutationTestExecutor^[[39m Done in 7 minutes and 40 seconds.
── ✓ completeness-gate (mutation) (7m42s)

── ▶ doc-template drift ──────────────────────────────────────────────
── ✓ doc-template drift (0s)

── ▶ embedded-library drift ──────────────────────────────────────────────
wrote 6 embedded library file(s) + 2 manifest(s) to server/typescript/packages/metadata/src/library/embedded-library.generated.ts, server/java/metadata/src/main/java/com/metaobjects/library/EmbeddedLibrary.java and server/csharp/MetaObjects/Library/EmbeddedLibrary.cs
── ✓ embedded-library drift (0s)

── ⊘ --quick: SKIPPING the other-language ports (csharp/java/python/kotlin),
        the full Java reactor, the docker integration suite and the cube live
        check. Run the full `scripts/ci-local.sh` (no flag) before opening/merging
        a PR or tagging.

══════════════════════ SUMMARY ══════════════════════
  ✓ leak-scan (security)
  ✓ pom-version parity
  ✓ bun-version parity
  ✓ uv.lock version parity
  ✓ publish-intent parity
  ✓ publish-set parity
  ✓ no committed pre-release version
  ✓ script-name hook collisions
  ✓ ci lane selection
  ✓ extract field kinds
  ✓ no-magic gate wired (5 ports)
  ✓ test-file references resolve
  ✓ metamodel-version bump  (1s)
  ✓ positioning claims
  ✓ hosted assess prompt builds
  ✓ migration guides vs registry
  ✓ dependencies installed
  ✓ peer-range bounds
  ✓ owned template copies current
  ✓ reference templates lint
  ✓ shipped doc examples load  (1s)
  ✓ no retired $apiPrefix
  ✓ site payload is true  (19s)
  ✓ site reference is fresh  (1s)
  ✓ release tag gate  (9s)
  ✓ scripts/ typecheck  (2s)
  ✓ requirements ledger verifies  (2s)
  ✓ requirements cover vocabulary  (1s)
  ✓ metamodel scaffolder  (1s)
  ✓ requirement harness
  ✓ fixture-lint
  ✓ ts build + typecheck  (17s)
  ✓ conformance: typescript  (3m02s)
  ✓ ts unit suites  (49s)
  ✓ completeness-gate (mutation)  (7m42s)
  ✓ doc-template drift
  ✓ embedded-library drift
  ⊘ csharp/java/python/kotlin conformance (--quick)
  ⊘ java-reactor (--quick)
  ⊘ integration-tests (--quick)
  ⊘ cube-model live check (--quick)
── slowest steps ──
     7m42s  completeness-gate (mutation)
     3m02s  conformance: typescript
       49s  ts unit suites
       19s  site payload is true
       17s  ts build + typecheck
        9s  release tag gate
        2s  requirements ledger verifies
        2s  scripts/ typecheck
        1s  metamodel scaffolder
        1s  metamodel-version bump
        1s  requirements cover vocabulary
        1s  shipped doc examples load
  total wall time: 12m27s
══════════════════════════════════════════════════════
LOCAL CI PASSED (with 4 skipped — see ⊘ above).
exit=0
  • Outcome: ⚠️ 2 infos across 2 runs (1h31m49s)

Pipeline

Updates from git push no-mistakes

✅ **Intent** - passed

✅ No issues found.

🔧 **Rebase** - 1 issue found → auto-fixed ✅
  • ⚠️ CHANGELOG.md - merge conflict rebasing onto origin/main

🔧 Fix applied.
✅ Re-checked - no issues remain.

✅ **Review** - 8 findings across 2 rounds, all fixed (1 declined)

Every finding below was resolved in a later round; none remains open.

  • ✅ verify --db committed-snapshot check never used the shadow - fixed: it now reuses the run's shadow, and the target probe runs only when no scratch database is available.
  • ✅ A failing explicit shadow database was silently ignored - fixed: an explicit --shadow-db / SHADOW_DATABASE_URL / migrate.shadowDatabaseUrl that cannot be opened or applied is a hard error naming the cause; only the automatic CREATE DATABASE path falls back, and it says why.
  • ✅ Views, comments and identity were model-carried rather than shadow-decided - fixed: the introspector now reads comments, identity and view bodies from both catalogs and they are compared catalog-to-catalog; the narrowing text was removed from the docs and CHANGELOG.
  • ✅ Views applied to the shadow could discard the table comparison - fixed: views stay applied (they are now compared from the catalog) without a failing view discarding the table comparison.
  • ✅ migrate and baseline --from-db did not report which path ran - fixed: every Postgres run reports it, clean runs included.
  • ✅ Rename-carry untested on the shadow path - fixed: shadow-path tests added.
  • ✅ --no-shadow-db was not in the requirement - fixed: flag and shadowDisabled removed; order is setting, CREATE DATABASE, probe, normaliser.
  • ✅ Snapshots written before this change would turn verify --db red on identity/comments - fixed: those properties are treated as unknown, not absent, in an older snapshot; new snapshots record them; CHANGELOG upgrade note added.
  • ✅ change-comment could clear a DBA-written comment silently - fixed: gated behind --allow drop-comment; without it migrate reports and skips it, and verify --db still reports the drift.
  • ✅ No test applied a plan that creates a table from a shadow catalog - fixed: create-from-shadow round-trip test added (including an expression default).
  • ↩️ Declined by the maintainer: removing the shadow branch of baseline --from-db. It stays shadow-first, because the probe writes to and locks the target and may only run when no scratch database is available.
✅ **Test** - failures from round 1 fixed; final live validation go

Round 1 found meta migrate --dry-run missing real column-default drift that verify --db reported (seq 8 vs 7, starts_at now()+1 day vs now()), and an unreachable explicit --shadow-db being silently ignored (live validation: no-go). Both were fixed: migrate and verify now reach the same verdict on the same database, with a live-Postgres regression test for those cases, and the explicit-shadow hard error applies to meta migrate as well as verify. A test-isolation flake in the new cli integration test (shared migration slug) was also fixed. The final result is below.

  • Live validation: ✅ go - 7 of 8 scenarios driven live against the product
Scenario Result Live Evidence
Authored defaults NOW(), current_date, date literal, '1e3', 'open' read clean through verify and migrate --from-db on the shadow path ✅ pass live r5-default-spellings-and-drift.txt
Real default drift (seq 8 vs 7, starts_at now()+1 day, status, dbl) is reported by verify and planned by migrate --dry-run, with the same verdict ✅ pass live r5-default-spellings-and-drift.txt
Explicit unreachable --shadow-db or SHADOW_DATABASE_URL is a hard error naming the cause in verify and migrate; plain migrate refuses --shadow-db ✅ pass live r5-explicit-shadow-and-cleanup.txt
A reachable explicit shadow database is used, left empty, and no mo_shadow_ database is left after the run ✅ pass live r5-explicit-shadow-and-cleanup.txt
A role without CREATEDB falls back to the temp-object probe, says which path ran and why, and still detects default drift ✅ pass live r5-probe-fallback-no-createdb.txt
A shape with no special rule (column type change) is reported when it really differs; an undeclared comment is skipped by migrate without --allow drop-comment but still reported by verify ✅ pass live r5-comment-gate-and-type-drift.txt
PG-backed migrate-ts and cli integration suites (shadow round trip, schema-wide diff, rename carry, create-from-shadow, snapshot versions) pass against live Postgres ✅ pass live r5-migrate-ts-pg-integration.txt; r5-cli-pg-integration.txt
SQLite and offline string-normaliser fallback paths ⏸️ untested no Not driven through the CLI. The change is Postgres-specific, so the SQLite and offline paths were left to the existing unit suites.
  • scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains
  • Live meta migrate --from-db --apply, meta verify and meta migrate --from-db --dry-run against Postgres 16, using authored defaults NOW(), current_date, a date literal, '1e3', 'open', and 7
  • Real default drift (seq 8 vs 7, starts_at now()+1 day vs now(), status 'closed', dbl 2000), checked through both verify and migrate
  • Unreachable explicit shadow via --shadow-db and via SHADOW_DATABASE_URL; plain migrate --dry-run --shadow-db
  • Reachable explicit shadow database; checked it was left empty and that no mo_shadow_* database remained
  • Role without CREATEDB: probe fallback, path message, and drift still detected
  • Undeclared table comment plus column type drift: --allow drop-comment gate
  • bun test in migrate-ts: pg-, postgres-, lifecycle-pg, apply-pg and declared-rename-carries-constraints (154 pass, 0 fail)
  • bun test in cli: migrate-postgres, verify-db-drift, verify-db-scope, verify-db-view-recreate, shadow-default-drift-pg, shadow-catalog-pg (27 pass, 0 fail)
  • Baseline scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains (ran before this step, passed)
  • scripts/ci-local.sh --quick (exit 0, LOCAL CI PASSED, 4 lanes skipped)
✅ **Document** - 2 findings, resolved
  • ✅ CHANGELOG.md heading - resolved: entries stay under ## [Unreleased]; the heading is renamed only when the release is cut.
  • ✅ drop-unmanaged missing from the migrate help --allow list - fixed: added, help snapshot updated.
✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

@dmealing
dmealing merged commit 8a727b1 into main Oct 29, 2026
1 check passed
@dmealing
dmealing deleted the fm/mo-pg-default-canon branch October 29, 2026 05:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant