Skip to content

Write root metadata.labels from @pipeline(labels=...) (v0.1.27) - #70

Merged
Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/tangle-cli-pipeline-metadata-lab-918b429
Sep 28, 2026
Merged

Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/tangle-cli-pipeline-metadata-lab-918b429

Conversation

@Volv-G

@Volv-G Volv-G commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator

(AI-assisted)

What

@pipeline(labels={...}) writes the compiled pipeline's root metadata.labels block. GraphBuilder carried only annotations, so five corpus pipelines could not be authored in Python at all.

Evidence

Fifteen corpus files carry a root metadata.labels; ten are components, leaving five pipelines:

pipeline metadata key order labels
relevance-tools/…/search_signals/pipeline.yaml labels, annotations team / domain / stage
…/join_features_from_bigquery_to_featureset_pipeline.yaml labels only team / domain / stage
…/storefront_searcharray_train_dsat_daily_pulse_e2e.pipeline.yaml labels, annotations team / domain / stage
…/l1_tangentable_pipeline.yaml annotations, labels team / stage / domain / tangentable
…/experiments/legacy_config/real_upi_searcharray_smoke/pipeline.yaml labels, annotations team / domain / stage

Every key and value is a str. The corpus is 4:1 labels-first, so that is the canonical order here; l1_tangentable_pipeline.yaml is the one shape this will not reproduce byte-for-byte.

String-only, per the schemas. Both the dehydrated and the pipeline schema type metadata.labels as additionalProperties: {"type": "string"} — strictly narrower than metadata.annotations, which the dehydrated schema also lets be a number, boolean or null. The value policy follows the schema rather than a guess.

Purely descriptive — no readers. Grepping labels across tangle_cli source returns nothing: no compiler, hydrator or client code reads them. They appear only in the two schemas and in the generated MetadataSpec.labels, and the OpenAPI spec has no label query parameters, so there is not even server-side filtering today.

No task-level labels. Zero occurrences in the corpus, and neither schema gives task metadata a labels property — task metadata is annotations-only. Deliberately out of scope rather than added silently.

How

  • PIPELINE_LABELS_POLICY in schema_validation.py, reusing the existing check_annotations machinery. AnnotationPolicy.label names the surface, so diagnostics read labels value for key 'team' must be a string; got int.
  • reject_reserved_key_prefix is deliberately off: system/ is documented as reserved for Tangle's own annotations, and nothing reserves a label prefix, so enabling it would refuse a document both schemas accept. Pinned by a test, and one line to flip if that changes.
  • New InvalidPipelineLabelsError so a caller can tell which metadata block it got wrong.
  • labels threaded PipelineFn → GraphBuilder → emit_pipeline, written before annotations whichever order the keywords were passed.
  • Not inherited into subpipeline children, consistent with root annotations, and not part of compile identity or the overrides fingerprint (that fingerprint covers config overrides only).

No pipeline_labels compile keyword is added. Annotations got one because the varying part of that block comes from a caller's per-environment config; nothing sets labels that way, pipeline_annotations is not even called from tangle-deploy, and the rules are already shared, so adding one later is small.

Failure modes

  • Strictly additive: a pipeline that does not pass labels= compiles byte-for-byte as before, including labels={} and labels=None.
  • An author who wants annotations first in metadata cannot get it. One corpus pipeline is written that way; the block's meaning is unaffected.

Review focus

  • The labels-before-annotations ordering, given the 4:1 corpus split.
  • Leaving the system/ prefix legal on labels.
  • Whether skipping pipeline_labels is the right call.

Tophatting

@pipeline(
    "Search signals",
    labels={"team": "discovery", "domain": "search-signals", "stage": "analysis"},
    annotations={"version": "1.0.0"},
)
def search_signals() -> Out[str]:
    ...

compiles to the corpus block exactly:

name: Search signals
metadata:
  labels:
    team: discovery
    domain: search-signals
    stage: analysis
  annotations:
    version: 1.0.0

Checklist

  • 20 new tests in tests/test_pipeline_labels.py: each of the four distinct corpus label blocks round-tripping, the rendered YAML text, labels-before-annotations regardless of keyword order, author key order preserved, no-labels byte-identity across omitted/{}/None, two subpipeline cases, and the validation matrix.
  • The subpipeline leak test compares a labelled and an unlabelled compile with the child-<hash8> token masked — child sidecar hashes fold in the output directory, so a naive cross-directory name comparison fails for an unrelated reason. With the hash masked, the only delta is the parent's metadata block.
  • 7 mutations, all caught: annotations-first ordering, empty block emitted, labels sorted, value strictness dropped, labels not threaded into the builder, caller dict aliased instead of copied, non-mapping accepted.
  • Full frozen suite 1986 passed on 3.10 and 3.12; ruff clean on the six touched source files and the test; packaging guards 10 passed; git diff --check clean.
  • uv.lock hand-edited on the tangle-cli self-entry only (one line); uv lock was not run.
  • pyright not run: it is not a repo dependency and the internal npm proxy has no package for it.
  • Diagnostics name the key and the type and never echo a value, pinned by a test that plants a recognisable string in three rejected shapes.

GraphBuilder carried only annotations, so the five corpus pipelines with
a root metadata.labels block could not be authored in Python.

Both schemas type metadata.labels as additionalProperties string, which
is narrower than metadata.annotations, so the value policy is strict
str -> str. Reuses check_annotations under a new PIPELINE_LABELS_POLICY,
with the reserved system/ prefix rule deliberately off: that prefix is
reserved for Tangle's own annotations, not for labels.

labels is written before annotations, matching the corpus majority.
Root only, descriptive only, and outside compile identity.
@Volv-G
Volv-G requested a review from Ark-kun as a code owner September 28, 2026 14:01
@Volv-G
Volv-G merged commit 52e1941 into master Sep 28, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant