TODO: a flagship example — AFTER the code is clean

Boris, 2026-10-07: "We will do flagship later AFTER CODE IS clean ... for now just put a
placeholder somehow ... and a TODO gh issue."

This is that issue. It replaces #17, which proposed a specific flagship (a NobsMed graph IR in,
competing report projections out) and was dropped.

The gap, stated plainly

There is no example in this repo that a stranger would be convinced by. Measured:

greeting.py        2 steps, trims whitespace. Its OWN docstring says you would never
                   need this library for it.
contestable.py     the right SHAPE — 4 judgement-call stages, 2 strategies differing in
                   one — and deliberately has no score, by a test.
parallel / counter / subgraph / ladder/*   teaching rungs, one concept each.

So the README's argument — a stage worth declaring is a stage worth arguing about — is made
entirely by examples that are not worth arguing about. That is the single biggest credibility gap
in the repo, and it is not a code defect.

Blocked on, deliberately

#9 (GraphImplementation + about carrying a path), #10 (GraphSpec.problem), #11
(run_count), #1 item 3 (object silently disabling a check). A flagship written on top of a
vocabulary that is about to be renamed gets rewritten; a flagship is also the worst place to
discover that about is unusable across a nesting boundary.

The criteria any candidate must satisfy

Carried over from #17, because this is the part of it worth keeping:

the input is already a public artifact — no API key, no service, no secrets. A reader drops
a file in and runs it.
no domain to learn. An example whose domain costs 500 lines to understand teaches the
domain, not the library.
contestable in the SELECTION, not the rendering — two competent people would choose
differently and you cannot assert which is right, only measure which does better.
evaluable offline. Arm A needs no LLM; an LLM arm is optional and comes second.

What "done" looks like

One example, importable by name, shipping a Dataset and ≥2 strategies differing in exactly one
stage, whose diff_diagram() highlights that stage, and whose eval_battle(..., run_count=3)
prints three numbers per arm. Linked from the README in place of the placeholder that now sits
there.

README: say there is no flagship example, and open #24 for it (#25)

⭐ LEADING CANDIDATE — a workflow that finds incoherence in other workflows

Boris, 2026-10-07: "could be an AI workflow for doing a coherence check on other AI workflows
......... so super dog fooding ... sort of like instead of a coherence check we are doing a llm as
judge grader for coherence quality" — and "other developers can have option to turn on this
funny coherence checker idea anyways."

⛔ The strongest objection first, because the framing decides everything

A rubric-scored "coherence quality: 7/10" is the exact thing this library forbids. #12 says it
about this shape:

"I don't know what result I want" → no. A boundary plus an eval suite produces a number
over an unstated objective, and a number reads as signal.

examples/contestable.py carries a test forbidding a score for that reason, and
coherence_check()'s whole position is that every finding is a provable defect. An LLM rating
coherence is a verdict the reader cannot check — which is what we accuse others of.

⭐ The framing that works: `coherence_check()` IS the oracle, and defects are INJECTED

start      examples/greeting.py — clean, 0 findings
inject     wire `raw_name` into `compose` instead of `clean_name`
           (both are `str` — the crossed-wire case the README leads with)
oracle     coherence_check() names the exact rule and the exact node
ask        the LLM arm: find the incoherence
score      precision / recall against a KNOWN answer

gradeable offline, because the grader is deterministic — the LLM is the thing under test,
not the judge of record
the dataset generates itself — 15 real designs × 30 named defects
contestable in the prompt strategy, which is a genuine judgement call: how do you describe a
declaration to a model so it spots a crossed wire? Two competent people write that differently.

And it attacks something `coherence_check()` has ALREADY admitted it cannot reach

#1 item 1, closed as a documented boundary:

a step returning the wrong value of the right type — cite_that_cites_nothing returns the
draft untouched. check() → CLEAN. No action. This is the boundary between what a spec
checks and what an eval measures.

An LLM judge is the right tool for exactly that boundary. So this does not duplicate the 13 rules
— it goes after what the rules documented as out of scope. That is a far better story than
"we grade coherence."

Against this issue's own criteria


public artifact, no API key	✅ the input is a `GraphSpec` — data, in-repo
no domain to learn	✅ the strongest point. The domain IS this library. Exactly what #17's medical version failed.
contestable in the SELECTION	✅ the prompt strategy, not the rendering
evaluable offline	⚠️ only in the framing above. Deterministic oracle, LLM under test. A rubric grader fails this outright.

Two dependencies, and one is not optional

⛔ needs potential_incoherence enumerated first. You cannot score "did the LLM find the
defect" without a machine-comparable name for the defect. The 30-member enum is the scoring
key, not a side quest.
reopens GraphSpec.problem — and who owns a brief across a boundary #10's rubric refusal, usefully. That was ruled out because "a rubric belongs
where something can run it, and there is no step-level runner." This gives rubrics a home — a
graph boundary, which is where GraphSpec.problem — and who owns a brief across a boundary #10 said they would belong.

Opt-in, per Boris

The LLM arm is off by default: it needs a model, and every other example in this repo runs
with no API key. Arm A is the deterministic checker, which is also the oracle; the LLM arm is a
flag a developer turns on. That keeps the quickstart keyless and makes the comparison honest —
you are measuring the LLM against a known answer, not against another opinion.

The risk to watch

Hall of mirrors. A stranger asking "why would I want this?" does not want "to check
workflows like itself." The injected-defect framing survives that — it is a measurable claim
about finding real bugs. A rubric grader does not.

Say who this is for, in line 5 (#31)

Navigation Menu

Repository navigation

TODO: a flagship example — AFTER the code is clean #24

The gap, stated plainly

Blocked on, deliberately

The criteria any candidate must satisfy

What "done" looks like

⭐ LEADING CANDIDATE — a workflow that finds incoherence in other workflows

⛔ The strongest objection first, because the framing decides everything

⭐ The framing that works: `coherence_check()` IS the oracle, and defects are INJECTED

And it attacks something `coherence_check()` has ALREADY admitted it cannot reach

Against this issue's own criteria

Two dependencies, and one is not optional

Opt-in, per Boris

The risk to watch

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

TODO: a flagship example — AFTER the code is clean #24

Description

The gap, stated plainly

Blocked on, deliberately

The criteria any candidate must satisfy

What "done" looks like

Activity

borisdev commented on Oct 7, 2026

⭐ LEADING CANDIDATE — a workflow that finds incoherence in other workflows

⛔ The strongest objection first, because the framing decides everything

⭐ The framing that works: coherence_check() IS the oracle, and defects are INJECTED

And it attacks something coherence_check() has ALREADY admitted it cannot reach

Against this issue's own criteria

Two dependencies, and one is not optional

Opt-in, per Boris

The risk to watch

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

⭐ The framing that works: `coherence_check()` IS the oracle, and defects are INJECTED

And it attacks something `coherence_check()` has ALREADY admitted it cannot reach