Archives

[matth-love]

Description

Summary

Make archive extraction safe for shared-cache and multiprocessing workflows by centralizing archive materialization in Fetchez and migrating Transformez's archive consumers to the new extraction utilities.

The previous implementation extracted VDatum grids directly into the shared cache with zipfile.extract(), and other compressed resources used independent extraction paths. When multiple Transformez/Globato processes shared a cache, one process could truncate and rewrite a resource while another was reading it, causing GDAL/NetCDF read failures and repeated cache repair.

Fetchez now provides p_f_extract() as the common archive materialization primitive. It uses inter-process locking, validates reusable extracted resources, stages archive contents outside the final destination, and atomically publishes completed files. The existing p_f_unzip(), p_f_untar(), and p_f_gunzip() helpers delegate to this implementation.

Transformez now uses the shared extraction path for all of its archive-derived resources:

• VDatum grid members are extracted through fetchez.utils.p_f_extract().
• GZIP resources fetched by ReferenceFetcher use p_f_extract().
• VDatum package installation uses p_f_extract() for archive extraction.
• HTDP package installation uses p_f_extract() for archive extraction.
• Extracted resources are represented as Path objects throughout the filesystem-facing code.

Regression coverage verifies the Transformez/Fetchez integration and preserves the intended shared-cache behavior, while the multi-process extraction tests in Fetchez verify the locking and atomic-publication guarantees themselves.

This removes the in-place archive extraction that caused the reported VDatum/shared-cache race and establishes Fetchez as the single safe archive-materialization layer for Transformez.

Closes #183 .

---

Checklist

• PR title is descriptive
• PR body contains links to related and resolved issues (e.g. closes #1)
• If needed, CHANGELOG.md updated
• If needed, docs and/or README.md updated
• If needed, unit tests added
• All checks passing
• At least one approval

---

🔍 Docs preview: https://transformez--189.org.readthedocs.build/en/189/