Skip to content

Declare graph inputs and outputs from the pipeline body (v0.1.23) - #66

Merged
Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/tangle-cli-v0-1-23-public-graph-e27384d
Sep 25, 2026
Merged

Volv-G merged 1 commit into
masterfrom
piforge/tangle-pipeline-crud/tangle-cli-v0-1-23-public-graph-e27384d

Conversation

@Volv-G

@Volv-G Volv-G commented Sep 25, 2026

Copy link
Copy Markdown
Collaborator

(AI-assisted)

What

A pipeline's inputs come from its In[T] parameters and its outputs from the return annotation. That covers graphs whose I/O is Python-shaped, and nothing else. There is no spelling for an input named "Pipeline Creation Time", for an exact Tangle type string like Json, for an input that only exists under a condition, or for a per-input editor.position — which is why downstream code reaches into current_builder().inputs / .output_values behind private helpers (84 call sites of one such pair in relevance-tools).

graph_input() / graph_output() declare the same entries on the active trace, as a public validated API:

@pipeline("Daily Pulse")
def daily_pulse() -> Out[str]:
    created = graph_input(
        "Pipeline Creation Time", "String", default="Use as a cache busting mechanism"
    )
    limit = graph_input("query_limit", "Integer", optional=True, when=has_limit)
    scrape = SCRAPE.named("Scrape")(created=created, limit=limit)
    graph_output("scrape_date", scrape.scrape_date, "String")
    return scrape

The signature route stays the recommended default — it is typed and IDE-checked. These are for porting existing YAML and for shapes the signature genuinely cannot express.

How

  • Declaration order is signature parameters first, then body order, which is the order the entries appear in the document.
  • default implies optional: true unless optional=False, matching what a defaulted In[T] parameter already emits.
  • Names are unique, checked against earlier declarations, In[...] parameters, and the name the return annotation contributes (Out[T]'s output_name or an Outputs field). The last of those is enforced in the tracer, since the return output is added after the body runs.
  • Outputs wire to a handle, never a constant — a task output or a graph input, the same rule Outputs enforces.
  • when=False declares nothing and returns None, so a conditional input is one assignment rather than an if/else around the call site.
  • position=(x, y) reuses 0.1.22's canonical editor.position serializer; annotations={...} is validated under the same caller policy as pipeline_annotations. This is how a graph input finally gets an editor position from Python — the gap called out in the 0.1.22 README.
  • default must be a string, because InputSpec.default is string | null in the schema. Caught at the call with a field-named message instead of a schema path later.
  • New InvalidGraphIoError(CompileError); diagnostics name the field and never echo the value. python_pipeline/__init__ stays light — no eager trace import.

Bug fixed along the way

graph_output() beside a returned Out[T] was silently lossy: emit prefers the multi-output map, while the returned edge lived only in the legacy single-output shims, so the returned output vanished from outputValues. The returned output now joins the map, declared last. Regression test included.

Failure modes

Migration consequences for code moving off a hand-rolled builder helper (relevance-tools' _gin / _gout). The emitted document is identical at 28 of 32 call sites; the four exceptions are deliberate:

  • Three sites gain optional: true — those passing default= with no explicit optional. _gin never implemented the implication. This is a semantic change, not cosmetic: optional is read at submit time for required-input validation, so those inputs start behaving the way their defaults already implied. Pass optional=False to keep the old emission exactly.
  • One site swaps key order — the only call passing both optional= and default= now emits default before optional. Content is unchanged. That order is what the corpus uses 28 times against 3 for the reverse, and what the In[T] path already emits.

Other notes:

  • A non-string default is now rejected on this surface. The In[T] signature path still writes param.default raw and can therefore still emit a schema-invalid document; that is a separate pre-existing bug and gets its own PR rather than riding along here.
  • Declaring the same name twice is an error rather than last-write-wins, unlike annotations: an input name is an identity in the graph, not a value.

Review focus

  • The duplicate-name checks spanning three sources (body declarations, In[...] parameters, return annotation).
  • The output_values join in trace.py — the fix for the lossy case, and the only change to existing behaviour.
  • That the migration deltas above are the ones we want.

Tophatting

uv run --frozen pytest tests/test_graph_io.py

21 tests on real compile output: non-identifier names and exact type strings, declaration order, the optional/default rules including optional=False, all three duplicate-name collisions, constant-output rejection, the returned-output regression, position=/annotations=, subpipeline child binding from the parent call site, declaration outside a trace, and no-echo diagnostics.

Checklist

  • Full suite green locally on Python 3.13: 1844 passed.
  • Version bumped to 0.1.23 in pyproject.toml, packages/tangle-cli/src/tangle_cli/__init__.py, tests/test_packaging.py, and the uv.lock editable self-entry (lock diff is the one-line version change only).
  • README documents the API, the rules, and the migration deltas above.

A pipeline's inputs come from its `In[T]` parameters and its outputs from
the return annotation, which only covers graphs whose I/O is Python-shaped.
A name that is not an identifier ("Pipeline Creation Time"), an exact Tangle
type string, an input declared conditionally, or a per-input editor position
has no spelling at all — which is why downstream code reaches into
`current_builder().inputs` / `.output_values` behind private helpers.

`graph_input()` / `graph_output()` declare the same entries on the active
trace as a public, validated API: unique names checked against `In[...]`
parameters and the return annotation, `default` implying `optional: true`
the way a defaulted parameter already does, handles-not-constants for
outputs, `when=` for conditional declaration, and `position=(x, y)` reusing
the canonical editor.position serializer.

Also fixes a latent bug: `graph_output()` beside a returned `Out[T]` was
silently lossy, because emit prefers the multi-output map while the returned
edge lived only in the legacy single-output shims. The returned output now
joins the map, declared last.
@Volv-G
Volv-G requested a review from Ark-kun as a code owner September 25, 2026 05:07
@Volv-G
Volv-G merged commit f443303 into master Sep 25, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant