Skip to content

Reads gate: type thin-evidence species (one strain or 1-3 runs) as UNVALIDATED instead of holding them - #327

Merged
hyphaltip merged 7 commits into
mainfrom
gate-thin-evidence
Nov 10, 2026
Merged

hyphaltip merged 7 commits into
mainfrom
gate-thin-evidence

Conversation

@hyphaltip

Copy link
Copy Markdown
Member

Curator request (2026-11-09): Pneumocystis and other species with only one strain in the assembly DB, or only a few runs, should still be typed from reads instead of being held. Curator ruling: type them, cap the decision at UNVALIDATED and show the reason.

Why the old floors existed

min_runs=4 and min_loci=2 date from the first wave driver (c97fe2c), and nothing records a rationale. The 4 works as a sample-size floor for the test fractions: with 1 run, "typed ≥ 75%" can only be 0 or 1. min_loci=2 guards against a panel cut from one strain's locus, which can miss divergent strains. Both risks remain, so these species are now labelled rather than hidden.

What

  • wave_driver.py
    • plan-one: a species now holds only when an idiomorph has no locus (too_few_loci) or no run (too_few_runs_0). Below the old floors it writes plan/<slug>/thin with:
      • single_locus: an idiomorph's panel comes from one assembled locus;
      • few_runs_<n>: 1 to 3 runs.
    • Decision: decide_species(..., thin) caps the decision at UNVALIDATED, with the thin tokens first. A HOLD stays a HOLD (the tokens are added) and a NOT_TYPABLE cause is unchanged.
    • Carrying the tokens: gate records them as thin=a+b in the reasons. regate, curate and apply-causes keep them.
  • merge_screen.py: the re-gating of species that gained runs keeps the cap. Before this fix, a later screen merge would have lifted a thin species back up.
  • build_reads_site.py: wording for the two reasons ("thin evidence: only 2 runs available to type"; "…the panel of an idiomorph comes from a single assembled locus (one strain)…").
  • Docs: CONTRACT.md (Waves) and the module docstring.

Data: the held species, re-planned and typed

Re-plan of the 22 species held for too_few_loci or too_few_runs_1-3. Left out: the members of the complexes already typed as groups (C. gattii, Blumeria, Sporothrix) and the two species with 0 runs.

  • Code and inputs: HPCC frozen worktree run-37856af, v0.8.1 inputs (waves 1 and 2 had been planned on the v0.7.0 tables).
  • Runs: stage thin per wave, 204 runs, all done, none to rerun (HPCC results/2026-11-09_thin_evidence_replan/).
  • Merge: append-only, by orchestration/merge_thin_2026-11-09.py. Existing rows are untouched (git diff --numstat shows only added rows). Every changed gate row is in wave*/thin_changes.tsv. The new folder wave5_basidio holds C. amylolentus, and its two Malassezia stay panel_build_failed.
Now Species
UNVALIDATED few_runs_<n> Pneumocystis jirovecii (1 of 2 typed), Trichophyton tonsurans (3 of 3, 3 of 3 agree), Penicillium solitum (3 of 3), Coniothyrium glycines (1 of 1, agrees), Ogataea parapolymorpha (3 of 3), Aspergillus minisclerotigenes (1 of 2), A. quadrilineatus (2 of 2 both, so also cause_unchecked)
UNVALIDATED single_locus Madurella mycetomatis (19 of 20), Hymenoscyphus fraxineus (35 of 41), Candidozyma pseudohaemuli (11 of 12), Fusarium sambucinum (8 of 8), A. westerdijkiae (4 of 5), Rhizopus stolonifer (14 of 19), Cryptococcus amylolentus (7 of 11), F. andiyazi (5 of 9), Pseudocercospora fijiensis (5 of 9), Morchella importuna (9 of 15, 6 both)
still HOLD (typing numbers) Ogataea angusta (0 of 2), Morchella sextelata (3 of 8), Mucor piriformis (0 of 9), M. racemosus (2 of 20): the one-strain panel does not type the other strains
still HOLD (no run) Candida boidinii, Fusarium virguliforme, Candida ipomoeae

A. quadrilineatus (2 of 2 both) is likely homothallic like A. nidulans. It could get a causes.tsv row after a look at its assemblies. That is not done here.

Checks

  • Full suite on the HPCC at the data commit 4300707: 1974 passed, 8 skipped (mafft loaded). At the final head ac41325: gate, merge_screen and site tests 44 passed.
  • Site build (HPCC): 250 species (was 247), 22,342 runs (+204). P. jirovecii and T. tonsurans are "unvalidated" with the thin-evidence reason first; M. piriformis is "withheld".
  • Not done: a headless-browser check of the rebuilt pages, and topping up the new UNVALIDATED species to the cap of 700 (runs_full gives at most 50; only Hymenoscyphus reaches 41).

🤖 Generated with Claude Code

hyphaltip and others added 7 commits November 9, 2026 21:47
… and capped at UNVALIDATED instead of held

Curator 2026-11-09: species with a single strain in the assembly DB (single_locus) or fewer than 4 runs
(few_runs_<n>) go on to typing; the decision is capped at UNVALIDATED with those reasons first. Hard floors:
an idiomorph with no locus, or no runs at all, still hold the species.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… UNVALIDATED (thin), 4 HOLD on their numbers

Re-plan of the species held for too_few_loci / too_few_runs_1-3 with the thin-evidence gate (run-37856af, v0.8.1 inputs),
stage thin per wave (results/2026-11-09_thin_evidence_replan on the HPCC), merged append-only by
orchestration/merge_thin_2026-11-09.py; changed rows in wave*/thin_changes.tsv. New committed folder wave5_basidio
(C. amylolentus; the two Malassezia stay panel_build_failed).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eld for too_few_runs_0

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hyphaltip
hyphaltip merged commit 1446589 into main Nov 10, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant