Repository navigation
test(docker): pin q3c with a build ARG (default 2.0.6) and add a q3c version comparison script - #161
Merged
Merged
Conversation
tests/docker/Dockerfile installed postgresql-16-q3c from the PGDG apt repository, unpinned. That now resolves to 2.0.6, while production runs 2.0.1, so the integration tests no longer ran against the production q3c. Build q3c from the upstream tag instead, selected by a Q3C_VERSION build ARG that defaults to 2.0.1. docker-compose passes Q3C_VERSION through from the environment when it is set, so the suite can run against another version with `Q3C_VERSION=2.0.6 uv run pytest tests/integration`. compare_q3c_versions.py loads one seeded point set into two servers and compares q3c_ang2ipix() row by row and q3c_radial_query() per cone, the latter also against the exact angular distance. It is the check behind the 2.0.1 -> 2.0.6 upgrade decision for the q3c expression indexes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
🟡 Changes recommended
The comparison can exit successfully despite ground-truth misses or extras, and exact-pole rim sampling collapses distinct position angles, leaving intended polar coverage incomplete.
2 open issues
What changed in this PR
This PR aligns the Docker integration-test environment with production’s q3c version and adds a tool to assess spatial-index and query compatibility across versions.
Changes:
- Build q3c from an upstream tag, defaulting to 2.0.1, with a Compose environment override.
- Add a seeded comparison script for pixel-index values, radial-query results, and exact-distance checks.
- Reorder index entries in schema documentation without changing their definitions.
| File | Description |
|---|---|
tests/docker/Dockerfile |
Build the selected q3c version in a separate build stage. |
tests/docker/docker-compose.test.yml |
Pass through the optional Q3C_VERSION build argument. |
tests/docker/compare_q3c_versions.py |
Compare q3c behavior across two PostgreSQL servers. |
docs/tbls/public.target.md |
Reorder an index entry; no schema change. |
docs/tbls/public.sky.md |
Reorder an index entry; no schema change. |
docs/tbls/public.fluxstd.md |
Reorder an index entry; no schema change. |
Review effort: Balanced
2.0.6 is what the unpinned apt package already installed and the version production is moving to. Q3C_VERSION=2.0.1 still builds the production version. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
offset_points() divided a cos(dec0) factor through atan2, so at an exact pole every position angle collapsed onto two longitudes, and its arcsin for dec lost about 1e-9 deg near the poles. Build the point from its east/north/up components with atan2 only; positions now land within 2e-14 deg of the requested distance everywhere. A radial query that misses an interior point or returns an exterior one now fails the run even when both servers agree on the wrong answer. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Summary
tests/docker/installedpostgresql-16-q3cfrom PGDG apt, unpinned. It now resolves to 2.0.6 (2.0.6-1.pgdg13+2), while production runs 2.0.1, so the tested q3c version had changed with no one deciding it.Q3C_VERSIONbuild ARG that defaults to 2.0.6, the version production is moving to.docker-compose.test.ymlpassesQ3C_VERSIONthrough from the environment when it is set, so the production version can still be tested:tests/docker/compare_q3c_versions.pyloads one seeded point set into two servers. It comparesq3c_ang2ipix()row by row, and comparesq3c_radial_query()per cone, both between the two servers and against the exact angular distance. The usage is in the module docstring.Results: q3c 2.0.1 vs 2.0.6, PostgreSQL 16.16, linux/amd64
q3c_ang2ipix: 0 mismatches out of 7,405,697 rows. The set holds 5M uniform points plus points next to ra=0/360, at and next to the poles, around the cube-face boundaries, on integer/half-integer grids and their ulp neighbours, and non-finite values (NULL in both versions). Reading the source agrees:q3c_ang2ipix_xy,q3c_xiyi2ipix,init_q3c1and the SQL wrapper are unchanged apart from formatting and aconst. So the existing*_q3c_ang2ipix_idxindexes stay valid.q3c_radial_query: identical on 31 of 33 cones of radius 0.7°. The exceptions are the two cones centred on or next to a cube vertex, (45, +35.26) and (225, -35.26439), where 2.0.1 misses 2206 and 2027 points inside the cone and 2.0.6 returns them all. Every other cone, poles included, has no ground-truth errors in either version. A separate probe of all 8 vertices found the 2.0.1 bug for centres within about 0.04° of a vertex and radii of about 0.66–0.72°. In that window it misses about 10% of the cone. Upstream fixed it in 2.0.6 (segasai/q3c@3533f250, "Prevent issues next to cube vertices"). The upgrade changes query results there, and it changes them to the correct ones.Q3C_VERSION=2.0.1. Each run confirmedq3c_version()inside the container.Notes
.github/workflows/test_database.ymlstill installs q3c from apt, currently 2.0.6. This PR leaves it unchanged.preparehelper against PostgreSQL's$(LIBS), so the build stage installs a few-devlibraries. The final image does not include them.🤖 Generated with Claude Code