Add pipelines dehydrate with verified, portable component refs (v0.1.28)

[Volv-G]

(AI-assisted)

What

Adds a first-class tangle sdk pipelines dehydrate PIPELINE -o OUTPUT [--mode auto|digest|name|url|file] [--components-dir DIR] command, the inverse of pipelines hydrate. It is backed by the shared PipelineDehydrator and does no traversal of its own. The PR also fixes the dehydrator correctness bugs that exposing the command revealed. Version 0.1.28.

Visible behaviour changes

• auto verifies content. A published digest is emitted only when its own published spec semantically matches the inline component, not merely because the digest exists. This affects pipeline-runs export --dehydrate: a mismatching digest now extracts a local file instead.
• Explicit digest / name always write a local copy plus one <output stem>.components.yaml resolve config per output, referenced as resolve://./<stem>.components.yaml#<fragment>. Each fragment is an ordered list: the verified primary first, then the local copy. name pins the owner found via ComponentInspector.inspect_by_digest. It pins the author, not the version; hydration still resolves that owner's latest candidate.
• Every mode fully dehydrates nested graphs. Explicit file/url used to leave inline subgraph specs in place, producing a partially hydrated document.
• Subgraph filenames are content-addressed: name_N.yaml becomes name-<digest>.yaml, and the subgraphs/ directory location is unchanged.
• New resolve-config marker fallback_on_error: true. A marked entry that raises falls through to the next entry. Unmarked, single and last entries propagate errors exactly as before, so hand-authored configs are unaffected.

Evidence

A silent cross-output corruption bug, pre-existing on master. Subgraph files were named <name>_<counter>.yaml in a shared subgraphs/ directory. Dehydrating pipeline A to a.yaml and then a different pipeline B to b.yaml in the same directory made both write subgraphs/judge_0.yaml, and a.yaml then rehydrated to B's content. It was reproduced and is now covered by a regression test. Genuinely distinct specs still get separate files: a real case differing only in a component_yaml_path provenance annotation stays two files.

Other bugs found and fixed during review, each reproduced first:

• A stale or foreign carried digest that existed in the library was emitted even though it resolves to different content. This affected AUTO as well as DIGEST.
• The NAME primary was unreachable against the real client, because find_existing_components returns rows without specs.
• Equal specs with different verified primaries hit a fragment-ID collision.
• A reused dehydrator instance leaked one output's cached files into another output's bundle.
• The new command accepted null/false/[] input and wrote {}.
• Path("gs://…") collapsed the // into a local gs:/… path, and the command wrote there.
• The standalone helper ignored an explicit base_url when no client was passed.

How

• pipeline_dehydrator.py:
  • Semantic verification of a carried digest's own spec.
  • The portable resolve-config writer.
  • Content-addressed subgraph filenames, computed after relative child refs are rewritten against a placeholder in subgraphs/, and processed deepest-first so a parent's address covers its children's names.
  • A separate per-call subgraph map, reset along with all other per-call state.
  • client_factory forwarded to TangleCliHandler.
  • Client creation handled at each operation boundary via self._get_client().
• pipeline_hydrator.py: the fallback_on_error marker only.
• pipelines.py: dehydrate_pipeline_file and DehydrateResult. It loads input with the strict load_pipeline_file, rejects raw :// URIs before Path() normalization using the engine's own URI rule (so Windows drive paths stay local), and never echoes the URI value, which may carry credentials.
• pipelines_cli.py: the command. str-typed path arguments so URI detection sees the raw value, LazyTangleApiClient, and the same option and config plumbing as hydrate.

Two semantics were deliberately accepted by the human:

• Deprecation successors are followed by design, even when their content differs; the local copy covers availability, not content drift.
• Tangle rejects duplicate digests, so multi-publisher same-digest handling is out of scope.

Failure modes

• Offline or unauthorized library: verification degrades to local copies.
• Malformed --header/--auth-header: deliberately a loud error, not a silent local-only result.
• Output always required: the resolve config needs a local output. Without one, dehydration raises ResolveManifestUnavailableError rather than writing into the cwd.

Testing

• Python 3.10 full: 2065 passed
• Python 3.12 full: 2065 passed
• tests/test_packaging.py: 10/10 passed, identical on pristine origin/master
• ruff: clean on all touched files; the repo-wide count is still the pre-existing 36
• git diff --check: clean
• New test files: test_dehydrator_portable_refs.py, test_pipelines_dehydrate_cli.py and test_dehydrator_subgraph_addressing.py. NAME and CLI tests drive the real TangleApiClient, stubbed only at its HTTP seams.
• Round trips: nested bundles are relocated and then rehydrated offline.
• Mutation testing: every load-bearing guard was deliberately broken and each break made tests fail. Unobservable or redundant guards were removed rather than justified.

Independent review by pi-135: six rounds, final verdict NO BLOCKING FINDINGS. Review page: https://piforge-preview.quick.shopify.io/?s=tangle-pipeline-crud&p=portable-dehydrator-references-independent-corre-bb128a37

Review focus

• The fallback_on_error hydrator contract: the one shared-behaviour change for resolve configs.
• AUTO's new content check on the shipped export path.
• Content-addressed subgraph naming, and the ordering "relativize, then address".

Known non-blocking follow-ups, not in this PR:

• Offline delay: with the library unreachable, the shared client retries each lookup for about 61 seconds before the local fallback, so a verification-specific retry budget or circuit breaker would help.
• Orphaned files: counter-named subgraph files left by earlier runs are not cleaned up.
• Write safety: file writes remain non-atomic and follow symlinks, as before.