Nextflow-compatible image, faster streaming reads, release 0.5.1

[Claptar]

Two independent fixes, released together as 0.5.1.

---

1. The image could not be used from a Nextflow process

Three problems, one of them a hard failure:

ENTRYPOINT ["adata"]. Nextflow requires /bin/bash to be the container entrypoint — it invokes docker run IMG /bin/bash -ue .command.sh, which the entrypoint mangled into adata /bin/bash -ue .command.sh. Reproduced against the old behaviour:

Error: No such command '/bin/bash'.

Apptainer users were accidentally fine, since singularity exec ignores the entrypoint — likely why this went unnoticed. The old docker.entrypointOverride escape hatch is gone from the current Nextflow config reference.

No ps. Nextflow needs bash (>=3.0), ps, awk, date, grep, sed, tail and tee in the task container to collect metrics. procps is not in bookworm-slim, so every task silently lost its trace row.

Bind-mounted runtimes. Apptainer bind-mounts the host $HOME, so a user's ~/.local/lib/python3.12/site-packages could shadow the image's virtualenv; and Nextflow is commonly configured with -u $(id -u):$(id -g), which leaves no writable $HOME.

What changed

Dockerfile — dropped ENTRYPOINT for CMD ["adata", "--help"]; added procps and mawk; kept curl/unzip/ca-certificates rather than purging them; set PYTHONNOUSERSITE=1, XDG_CACHE_HOME and UV_COMPILE_BYTECODE=1. A build-time RUN asserts the required tool set, so base-image drift fails the build rather than every task.

.dockerignore — new. There wasn't one, so COPY . . pulled the host's .venv, .git, .pytest_cache and .claude/worktrees/ (two full repo copies) into every local build. CI never hit it because a fresh checkout has none of them. The apt and duckdb layers also moved ahead of COPY . ., so a source edit no longer re-downloads duckdb.

Docs — README.md and docs/index.md both documented docker run IMAGE view file.h5ad, which the entrypoint change makes wrong. Both now name the command.

Breaking change

The image no longer sets an entrypoint, so the command must be named explicitly:

-docker run --rm -v /data:/data quay.io/cellgeni/adata-cli:0.5.1 view /data/f.h5ad
+docker run --rm -v /data:/data quay.io/cellgeni/adata-cli:0.5.1 adata view /data/f.h5ad

This is why it ships as 0.5.1 rather than by moving the 0.5.0 tag: anyone pinned to 0.5.0 would otherwise break silently.

---

2. Copying a row-chunked store was dominated by read latency

_chunk_step returned the source's chunk height verbatim, so a store chunked (1, n_cols) was copied one row per read. On a local disk that is merely wasteful; on Lustre or NFS every read is a round-trip costing milliseconds, so a million-row copy spent nearly all of its time waiting.

Reads are now grown to a 32 MiB budget and rounded down to a whole number of source chunks, since a partial read still decompresses the whole chunk:

shape	source chunks	step before	step after	read size
(1_000_000, 30_000) f32	(1, 30_000)	1 row	279 rows	32 MiB
(10_000_000,) i64	(65_536,)	65,536 rows	4,194,304 rows	32 MiB

Sizing needs the dtype, which h5py misreports for variable-length strings — itemsize is 8 there because the value is a pointer, not the text. VLEN_ELEMENT_BYTES is assumed instead, so the row count is not overestimated by an order of magnitude.

Known limit, deliberately kept: the step is floored at one whole chunk, so TARGET_READ_BYTES is a target rather than a cap. A source whose own chunk already exceeds the budget — say (1000, 1_000_000) f32 chunked whole — reads 3.8 GiB in one go. That matches the previous behaviour exactly, so nothing regresses, and it is now commented at the call site rather than left as an implied guarantee.

_chunk_step is private with a single production call site, updated in the same commit.

---

Review

Codex flagged that XDG_CACHE_HOME was self-defeating as first written — uv sync honours it, so the build left /tmp/.cache root-owned and mode 0755, and a task under an arbitrary UID could not write to the very path the image advertises. Correct, confirmed, and fixed in 9a77450 by clearing and recreating the directory 1777 in the same layer as the sync. Thread resolved.

Verification

Built the image and ran a real pipeline under Nextflow 26.04.6 with docker.runOptions = '-u $(id -u):$(id -g)'. It completes, and the trace is populated — direct evidence that ps works:

task_id  hash       name     status     exit  realtime  %cpu    peak_rss  peak_vmem
1        7b/f03e58  INSPECT  COMPLETED  0     438ms     298.3%  12.8 MB   17.7 MB

Also confirmed on the final image: entrypoint is []; all eight required tools resolve; $XDG_CACHE_HOME is writable as UID 12345 and 999; a deliberately poisoned ~/.local/.../zarr.py is ignored in favour of the venv; no .git/.claude/.pytest_cache in the image. Full suite: 1037 passed.

Left out, deliberately

• quay-on-tag.yml still builds amd64 only. Fine for the farm, but local Apple Silicon builds now diverge in architecture from the published image. A one-line platforms: addition fixes it if wanted.
• OMP_NUM_THREADS is not pinned in the image. NumPy's BLAS sizes its pool to the whole host, which oversubscribes a shared LSF node — but that belongs in a pipeline's env scope, not baked in. Noted in a Dockerfile comment.

Releasing

quay-on-tag.yml only builds on a tag push and has no workflow_dispatch, so after merge:

git checkout main && git pull
git tag 0.5.1 && git push origin 0.5.1

The tag must match pyproject.toml exactly or publish.yml's check-version job fails. That push also publishes 0.5.1 to PyPI and moves the latest image tag.

🤖 Generated with Claude Code

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-21T10:42:40.070686Z	dcd4d24	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[github-actions]

Test Results (py3.12)

833 tests  +8   833 ✅ +8   1m 23s ⏱️ ±0s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 9a77450. ± Comparison against base commit 7f4df63.

♻️ This comment has been updated with latest results.

[github-actions]

Test Results (py3.13)

833 tests  +8   833 ✅ +8   1m 21s ⏱️ +10s
  1 suites ±0     0 💤 ±0 
  1 files   ±0     0 ❌ ±0 

Results for commit 9a77450. ± Comparison against base commit 7f4df63.

♻️ This comment has been updated with latest results.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dcd4d2479a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".