Repository navigation
feat(migrate-ts): decide Postgres drift against a shadow database - #510
Merged
Merged
Conversation
…nd the column-rename CHECK carry
…tch (shadow) database first
…ct(shadow) against introspect(live)
…ed-expression defaults
…efuse --shadow-db
…DME, CHANGELOG 1.2.0 heading, drift table row.
…naged to migrate --allow help and snapshot
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Changed
meta verify --db,meta migrate --from-db/--apply) now applies the expected schema to a scratch (shadow) database, introspects it with the live-database introspector, and diffs the two catalogs. The scratch database comes from--shadow-db,SHADOW_DATABASE_URL, ormigrate.shadowDatabaseUrl, else aCREATE DATABASEthat is always dropped afterwards. The rolled-back temp-object probe and the string normaliser remain as fallbacks, and every run reports which path ran.change-column-identity(dropping it needs--allow drop-identity-default), comment changes aschange-comment(clearing one needs--allow drop-comment). The snapshot moves toformatVersion4; v3 snapshots still load.DEFAULTs that Postgres re-spells (NOW(),current_date, timestamptz date literals,'1e3','007', composite expressions) are compared through the expr-canon probe instead of text, and a column rename that carries a CHECK is decided on the scratch copy. Real changes are still reported. The Postgres emitter no longer writesDEFAULTbesideGENERATED BY DEFAULT AS IDENTITY.Validation
Risk Assessment
Testing
I ran the real CLI against a disposable Postgres 16 container and a throwaway project. Authored default spellings read clean through both verify and migrate. Real default drift (seq 8 vs 7, starts_at now()+1 day) was reported by both commands, and an explicit unreachable shadow database is a hard error in both. Plain migrate with
--shadow-dbexits 2. A role without CREATEDB falls back to the temp-object probe, says so, and still catches drift. An undeclared comment is skipped unless--allow drop-commentis passed, while verify still reports it. The PG-backed integration suites passed (154 in migrate-ts, 27 in cli), after I fixed the migration-slug collision between test files. The earlier baselinets-fastandts-unitrun passed, andscripts/ci-local.sh --quickfinished green with 4 lanes skipped. I did not drive the SQLite and offline-normaliser fallbacks live, and the committed-snapshot old-format check was covered only by the existing suites.Evidence: Authored default spellings clean, then real default drift (verify and migrate)
Evidence: Explicit shadow errors, plain-migrate refusal, shadow cleanup
Evidence: Probe fallback for a role without CREATEDB
Evidence: Comment gate and column type drift
Evidence: migrate-ts PG integration result
Evidence: cli PG integration result
Evidence: ci-local --quick log (exit 0)
Pipeline
Updates from git push no-mistakes
✅ **Intent** - passed
✅ No issues found.
🔧 **Rebase** - 1 issue found → auto-fixed ✅
CHANGELOG.md- merge conflict rebasing onto origin/main🔧 Fix applied.
✅ Re-checked - no issues remain.
✅ **Review** - 8 findings across 2 rounds, all fixed (1 declined)
Every finding below was resolved in a later round; none remains open.
verify --dbcommitted-snapshot check never used the shadow - fixed: it now reuses the run's shadow, and the target probe runs only when no scratch database is available.--shadow-db/SHADOW_DATABASE_URL/migrate.shadowDatabaseUrlthat cannot be opened or applied is a hard error naming the cause; only the automatic CREATE DATABASE path falls back, and it says why.migrateandbaseline --from-dbdid not report which path ran - fixed: every Postgres run reports it, clean runs included.--no-shadow-dbwas not in the requirement - fixed: flag andshadowDisabledremoved; order is setting, CREATE DATABASE, probe, normaliser.verify --dbred on identity/comments - fixed: those properties are treated as unknown, not absent, in an older snapshot; new snapshots record them; CHANGELOG upgrade note added.change-commentcould clear a DBA-written comment silently - fixed: gated behind--allow drop-comment; without it migrate reports and skips it, andverify --dbstill reports the drift.baseline --from-db. It stays shadow-first, because the probe writes to and locks the target and may only run when no scratch database is available.✅ **Test** - failures from round 1 fixed; final live validation go
Round 1 found
meta migrate --dry-runmissing real column-default drift thatverify --dbreported (seq 8 vs 7,starts_atnow()+1 day vs now()), and an unreachable explicit--shadow-dbbeing silently ignored (live validation: no-go). Both were fixed: migrate and verify now reach the same verdict on the same database, with a live-Postgres regression test for those cases, and the explicit-shadow hard error applies tometa migrateas well asverify. A test-isolation flake in the new cli integration test (shared migration slug) was also fixed. The final result is below.scripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchainsLivemeta migrate --from-db --apply,meta verifyandmeta migrate --from-db --dry-runagainst Postgres 16, using authored defaults NOW(), current_date, a date literal, '1e3', 'open', and 7Real default drift (seq 8 vs 7, starts_at now()+1 day vs now(), status 'closed', dbl 2000), checked through both verify and migrateUnreachable explicit shadow via--shadow-dband viaSHADOW_DATABASE_URL; plainmigrate --dry-run --shadow-dbReachable explicit shadow database; checked it was left empty and that nomo_shadow_*database remainedRole without CREATEDB: probe fallback, path message, and drift still detectedUndeclared table comment plus column type drift:--allow drop-commentgatebun testin migrate-ts: pg-, postgres-, lifecycle-pg, apply-pg and declared-rename-carries-constraints (154 pass, 0 fail)bun testin cli: migrate-postgres, verify-db-drift, verify-db-scope, verify-db-view-recreate, shadow-default-drift-pg, shadow-catalog-pg (27 pass, 0 fail)Baselinescripts/ci-local.sh --only ts-fast --only ts-unit --strict-toolchains(ran before this step, passed)scripts/ci-local.sh --quick(exit 0, LOCAL CI PASSED, 4 lanes skipped)✅ **Document** - 2 findings, resolved
CHANGELOG.mdheading - resolved: entries stay under## [Unreleased]; the heading is renamed only when the release is cut.drop-unmanagedmissing from the migrate help--allowlist - fixed: added, help snapshot updated.✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.