Skip to content

test(docker): pin q3c with a build ARG (default 2.0.6) and add a q3c version comparison script - #161

Merged
monodera merged 6 commits into
mainfrom
u/monodera/q3c-version-arg
Nov 17, 2026
Merged

monodera merged 6 commits into
mainfrom
u/monodera/q3c-version-arg

Conversation

@monodera

@monodera monodera commented Nov 17, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • tests/docker/ installed postgresql-16-q3c from PGDG apt, unpinned. It now resolves to 2.0.6 (2.0.6-1.pgdg13+2), while production runs 2.0.1, so the tested q3c version had changed with no one deciding it.
  • q3c is now built from the upstream tag, selected by a Q3C_VERSION build ARG that defaults to 2.0.6, the version production is moving to. docker-compose.test.yml passes Q3C_VERSION through from the environment when it is set, so the production version can still be tested:
    Q3C_VERSION=2.0.1 uv run pytest tests/integration -v
  • tests/docker/compare_q3c_versions.py loads one seeded point set into two servers. It compares q3c_ang2ipix() row by row, and compares q3c_radial_query() per cone, both between the two servers and against the exact angular distance. The usage is in the module docstring.

Results: q3c 2.0.1 vs 2.0.6, PostgreSQL 16.16, linux/amd64

  • q3c_ang2ipix: 0 mismatches out of 7,405,697 rows. The set holds 5M uniform points plus points next to ra=0/360, at and next to the poles, around the cube-face boundaries, on integer/half-integer grids and their ulp neighbours, and non-finite values (NULL in both versions). Reading the source agrees: q3c_ang2ipix_xy, q3c_xiyi2ipix, init_q3c1 and the SQL wrapper are unchanged apart from formatting and a const. So the existing *_q3c_ang2ipix_idx indexes stay valid.
  • q3c_radial_query: identical on 31 of 33 cones of radius 0.7°. The exceptions are the two cones centred on or next to a cube vertex, (45, +35.26) and (225, -35.26439), where 2.0.1 misses 2206 and 2027 points inside the cone and 2.0.6 returns them all. Every other cone, poles included, has no ground-truth errors in either version. A separate probe of all 8 vertices found the 2.0.1 bug for centres within about 0.04° of a vertex and radii of about 0.66–0.72°. In that window it misses about 10% of the cone. Upstream fixed it in 2.0.6 (segasai/q3c@3533f250, "Prevent issues next to cube vertices"). The upgrade changes query results there, and it changes them to the correct ones.
  • Integration tests: 45 passed on the default build (2.0.6), and 45 passed with Q3C_VERSION=2.0.1. Each run confirmed q3c_version() inside the container.

Notes

  • .github/workflows/test_database.yml still installs q3c from apt, currently 2.0.6. This PR leaves it unchanged.
  • The 2.0.1 Makefile links its prepare helper against PostgreSQL's $(LIBS), so the build stage installs a few -dev libraries. The final image does not include them.

🤖 Generated with Claude Code

monodera and others added 2 commits November 17, 2026 11:45
tests/docker/Dockerfile installed postgresql-16-q3c from the PGDG apt
repository, unpinned. That now resolves to 2.0.6, while production runs
2.0.1, so the integration tests no longer ran against the production q3c.

Build q3c from the upstream tag instead, selected by a Q3C_VERSION build
ARG that defaults to 2.0.1. docker-compose passes Q3C_VERSION through from
the environment when it is set, so the suite can run against another
version with `Q3C_VERSION=2.0.6 uv run pytest tests/integration`.

compare_q3c_versions.py loads one seeded point set into two servers and
compares q3c_ang2ipix() row by row and q3c_radial_query() per cone, the
latter also against the exact angular distance. It is the check behind
the 2.0.1 -> 2.0.6 upgrade decision for the q3c expression indexes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The comparison can exit successfully despite ground-truth misses or extras, and exact-pole rim sampling collapses distinct position angles, leaving intended polar coverage incomplete.

2 open issues
What changed in this PR

This PR aligns the Docker integration-test environment with production’s q3c version and adds a tool to assess spatial-index and query compatibility across versions.

Changes:

  • Build q3c from an upstream tag, defaulting to 2.0.1, with a Compose environment override.
  • Add a seeded comparison script for pixel-index values, radial-query results, and exact-distance checks.
  • Reorder index entries in schema documentation without changing their definitions.
File Description
tests/​docker/​Dockerfile Build the selected q3c version in a separate build stage.
tests/​docker/​docker-compose.test.yml Pass through the optional Q3C_VERSION build argument.
tests/​docker/​compare_q3c_versions.py Compare q3c behavior across two PostgreSQL servers.
docs/​tbls/​public.target.md Reorder an index entry; no schema change.
docs/​tbls/​public.sky.md Reorder an index entry; no schema change.
docs/​tbls/​public.fluxstd.md Reorder an index entry; no schema change.

Review effort: Balanced

Comment thread tests/docker/compare_q3c_versions.py Resolved
Comment thread tests/docker/compare_q3c_versions.py Resolved
2.0.6 is what the unpinned apt package already installed and the version
production is moving to. Q3C_VERSION=2.0.1 still builds the production
version.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@monodera monodera changed the title test(docker): pin q3c to a build ARG and add a q3c version comparison script test(docker): pin q3c with a build ARG (default 2.0.6) and add a q3c version comparison script Nov 17, 2026
github-actions Bot and others added 3 commits November 17, 2026 02:56
offset_points() divided a cos(dec0) factor through atan2, so at an exact
pole every position angle collapsed onto two longitudes, and its arcsin
for dec lost about 1e-9 deg near the poles. Build the point from its
east/north/up components with atan2 only; positions now land within
2e-14 deg of the requested distance everywhere.

A radial query that misses an interior point or returns an exterior one
now fails the run even when both servers agree on the wrong answer.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@monodera
monodera merged commit dc906a6 into main Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants