Skip to content

Restore PostgreSQL CI headroom with batched reflection and smaller partitions - #4737

Merged
3 commits merged into
mainfrom
work/pgtimeout-1024
Oct 24, 2026
Merged

3 commits merged into
mainfrom
work/pgtimeout-1024

Conversation

@shiny-code-app

@shiny-code-app shiny-code-app Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor

Repeated native PostgreSQL jobs reach their 20-minute deadline without a failing assertion and block the merge train. Batch the invariant inspector's columns, indexes, primary keys and check constraints into fresh per-verification catalog reads. All existing invariant checks and write-fence refusals remain in place. The same-store ABBA comparison reduces SQL statements from 239 to 20 and median verification time from 0.246 to 0.031 seconds; a representative recovery scenario falls from 53.0 to 33.4 seconds.

Use seven partitions in postgres_integration through its existing generic discovery and partitioning. With batching alone, train candidate #4750's slowest native job still took 16:49. Replaying its measured 965 test costs projects worst test execution from 15:20 at six partitions to 12:19 at seven; eight projects 12:17, so seven is the smallest useful adjustment. Hosted performance at seven remains to be measured. All tests remain included exactly once and the 20-minute guard and 15-minute advisory stay unchanged. No other CI job changes. One additional runner job adds setup overhead; registered logical capacity is not proof of physical CPU capacity.

Validation: whole affected safety/schema/store modules (234 tests), native refusals (15), the original ordered 161-case shard, and affected native/schema compatibility modules passed against this runtime implementation. The latter selected 849 including a proposed budget probe subsequently removed; no existing test was changed or removed. Seven-way discovery verifies all 965 current source IDs exactly once. The 50 CLI/sharding/runner/security tests, actionlint, Ruff, formatting, role-word checks and mypy pass. Inspection NOT RUN/UNKNOWN under the Supervisor's explicit no-IDE instruction. Full new-head CI and independent final review are recorded in PR comments before checked Ready.

Closes #4188. PGTIMEOUT-1024, merge-path repair with explicit PR-cap exemption. Original stalled #4722 sources #4700/#4707/#4711 already landed through replacement #4731; no duplicate requeue is needed. Timing findings are on #4188 for CIMEASURE-1024.

Removed: repeated per-table invariant catalog reflection; no existing tests or assertions.
Net lines: +36 (+46/-10 across three files).

@shiny-code-app

Copy link
Copy Markdown
Contributor Author

Review result: weighed
Head: ef3e9e3

Own independent review of record: Anthropic claude-opus-5-5[1m], confirmed by the reviewer CLI, completed on final head ef3e9e3. No implementation defects found in either round. Reviewed schema metadata shapes, fresh-read behavior, all invariant/refusal classes, absent/retired relations, callers and existing native refusal coverage.

Acted on the first review's test-design finding by removing the proposed catalog-budget test. The temporary probe exposed the tracked baseline's227 catalog queries, but its schema-size threshold could fail after an intended retirement. Existing behavior tests plus measured ABBA evidence qualify this optimization without a new brittle gate. No existing test or assertion changed. Shortened the existing testing paragraph.

Final review notes declined as non-defects: empty metadata filters can cause a bounded whole-schema read on a failing/retired-relation path, but do not restore per-table query growth or relax any refusal; a speculative extra branch is unnecessary for this repair. The operational testing paragraph retains the existing fresh-read contract and is shorter, not mutable runtime authority. Existing native tests exercise the adapter against actual PostgreSQL type/index/predicate/key/fence drift. Hosted job duration proof remains mandatory before Ready.

Before/after evidence: same-store/service ABBA, six old and six new, invariant SQL 239 → 20, catalog227 → 8, median 0.245669 → 0.031282s. Same recovery scenario 53.016 → 33.370s; unchanged ordered hosted shard of 161 tests passes locally in 250.353s. 234 whole safety/schema/store tests and 15 native refusal checks pass. Ruff/format/role-word and 1762-file mypy pass. Broader native-module run and current-head CI remain pending. IDE inspection NOT RUN/UNKNOWN because this brief forbids launching or touching JetBrains apps.

@shiny-code-app

Copy link
Copy Markdown
Contributor Author

Ready for the merge train

Head: ef3e9e3

@shiny-code-app shiny-code-app Bot added the ready-to-merge Every Code PR is ready for repository owner merge review label Oct 24, 2026
@shiny-code-app

shiny-code-app Bot commented Oct 24, 2026 •

Copy link
Copy Markdown
Contributor Author

Launchplane completed the merge-train step for this pull request.

Launchplane landed this pull request through protected batch PR #4759.

Launchplane manages this comment and will update it as the train moves.

@shiny-code-app shiny-code-app Bot changed the title Restore PostgreSQL CI headroom with batched invariant reflection Restore PostgreSQL CI headroom with batched reflection and smaller partitions Oct 24, 2026
@shiny-code-app

Copy link
Copy Markdown
Contributor Author

Review result: weighed
Head: 55cb802

Independent final review weighed for 55cb802: Anthropic claude-opus-5-5[1m], confirmed by CLI, through model-review helper. This follows the fresh design assessment after the earlier two review rounds.

No implementation defects found. Reviewer verified fresh per-call metadata, unchanged schema-refusal helpers, SQLAlchemy reflection equivalence and full native inventory partitioning. Missing-table exception wording remains a fail-closed nondefect. Empty-filter reflection is a bounded nondefect and the outer store rejects missing tables first. An unprofiled sibling reflection path is outside this measured repair; no speculative change.

Acted on residual operational risk by requiring actual new-head source CI and train-candidate PostgreSQL timings before completion. Seven partitions redistribute existing work and add setup overhead; the replay is an estimate and physical contention remains unknown. Seven has essentially the same projected worst case as eight while using one fewer slot. The 20-minute deadline and 15-minute advisory stay unchanged, no tests/assertions are removed, and no other CI job changes. Existing (2,3,6) sharding tests test generic properties and were correctly left unchanged rather than pinning workflow text.

Local new-head actionlint, 50 CLI/sharding/runner/security tests, seven-way exact-once discovery, Ruff/format, role-word check and 1762-file mypy passed. Prior whole affected safety/native tests cover the unchanged runtime implementation. IDE inspection NOT RUN/UNKNOWN under this brief's explicit ban. Actual hosted timings remain pending.

@shiny-code-app
shiny-code-app Bot marked this pull request as ready for review October 24, 2026 20:13
@shiny-code-app

Copy link
Copy Markdown
Contributor Author

Ready for the merge train

Head: 55cb802

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-to-merge Every Code PR is ready for repository owner merge review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PostgreSQL candidate jobs reach the deadline after all tests pass

0 participants