Skip to content

Reads: A. fumigatus population MAT report (4,849 isolates), MAT frequency, ENA FTP fallback - #326

Merged
hyphaltip merged 4 commits into
mainfrom
sketch-store-ena-ftp
Nov 10, 2026
Merged

hyphaltip merged 4 commits into
mainfrom
sketch-store-ena-ftp

Conversation

@hyphaltip

Copy link
Copy Markdown
Member

Summary

A population report of A. fumigatus mating type from the sketch store (every public isolate plus the lab collection), shown on a new page of the reads site, plus the two pieces it needs.

Headline: MAT1-1 50.4% per clone (95% CI 48.2–52.5%, 2,077 clones; exact binomial p = 0.76), 51.2% per isolate, across 4,849 isolates (4,736 kept). No country or study departs from 50% at p < 0.01 after clone correction; three near-clonal lineages are skewed (clonal expansion, not a species-wide shift). Sketch and reads-type calls agree 4,614 times and disagree 0 times where both give one idiomorph.

Commits

  • 263ef54 ENA FTP fallback (sketch_one.sh): ENA answered HTTPS with 403 for 17 ENA-only runs from the HPCC while FTP served them; the fallback recovered 15 (2 remain ena_partial).
  • ad1100b mat_frequency.py: MAT1-1 fraction per isolate and per clone group (ANI 0.9995), by source, study, country and lineage (ANI 0.998), with Wilson 95% CI and an exact binomial test. Leaves out curator exclusions (STORE/exclude.tsv), other species (mean ANI < 0.95) and QC mixtures, with reasons.
  • 0087aa8 site report: export_report.py (run on the HPCC) writes datasets/reads/sketch_store/reports/aspergillus_fumigatus/ (population.json 145 KB, runs_public.tsv 438 KB, committed); build_reads_site.py copies it; new web/intro/reads/population.html; the reads overview and the A. fumigatus species report link to it.

Data decisions (curator, 2026-11-09)

  • Local Population_Genomics isolates count in every number but are never named: the run table is public runs only, PCoA points carry no identifiers (tested).
  • The report is a separate page, not a section of the species report.
  • Moose_3-1.1 is excluded (low-quality sequencing; mean ANI 0.749).

Findings worth a look

  • 63 isolates filed as A. fumigatus are another species (mean ANI ~0.83): PRJNA1186192 (44, CLSI voriconazole/isavuconazole breakpoints), PRJNA1381729 (6, in vivo breakpoint model), and one run under PRJDB43097 (an A. flavus project). A PopSeq relabelling issue is to follow.
  • study_titles.tsv predates those two BioProjects, so they show no title until it is rebuilt from PopSeq.

Test plan

  • Full suite: 1955 passed, 18 skipped (pixi run -e test python -m pytest tests -q)
  • New tests: test_sketch_store_mat_frequency.py, test_sketch_store_export_report.py (local names never leak), test_build_reads_site_script.py (report copied and indexed; none -> empty list)
  • Page in a headless browser (local http.server): 0 console errors, 0 failed requests, all sections filled (17 country groups, 36 studies, 20 lineages, 4,569 PCoA points, 21 screens), no local names, no horizontal scroll at 390 px; overview table and species-page link render

🤖 Generated with Claude Code

hyphaltip and others added 4 commits November 9, 2026 20:34
On 2026-11-09 ENA answered HTTPS with 403 for 17 ENA-only A. fumigatus runs
from the HPCC (after earlier retries) while FTP served the same files. When the
HTTPS stream delivers nothing, sketch_one.sh now retries the same paths over
ftp://. Recovered 15 of the 17; 2 remain ena_partial.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ency.py)

Reads the store's MAT calls, distance QC (mean ANI, clone groups), k-mer QC and
the curator exclusion list (STORE/exclude.tsv). Call per sample: reads-type when
single, else the sketch call. Leaves out curator exclusions, other species (mean
ANI < 0.95) and QC mixtures, with the reason in samples.tsv. Reports the MAT1-1
fraction with Wilson 95% CI and an exact binomial test against 0.5, per sample
and clone-corrected (one vote per clone group at ANI 0.9995), overall and by
source, study, country and lineage (ANI 0.998 groups).

First run (A. fumigatus, 4,721 samples kept of 4,834): MAT1-1 0.513 per sample,
0.503 (0.482-0.525) over 2,153 clone groups. Fixture test covers call choice,
exclusions, clone correction and the statistics. README documents the script,
exclude.tsv and the FTP fallback.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- export_report.py (run on the HPCC): population.json + runs_public.tsv of a
  sketch store: per-isolate and clone-corrected MAT1-1 fraction, by country,
  study, source and lineage, sketch vs reads-type concordance, outlier kinds and
  isolates of another species, PCoA points, target screens. Local isolates are
  counted but never named (curator decision 2026-11-09).
- reports/aspergillus_fumigatus/: 4,849 isolates; MAT1-1 50.4% per clone
  (95% CI 48.2-52.5%, 2,077 clones), 51.2% per isolate.
- build_reads_site.py copies each report and lists it in index.json; new page
  reads/population.html; the overview and the species report link to it.
- Checked in a headless browser: 0 console errors, 0 failed requests, no local
  names, no horizontal scroll at phone width. Full suite 1955 passed, 18 skipped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@hyphaltip
hyphaltip merged commit c35e093 into main Nov 10, 2026
2 checks passed
@hyphaltip

Copy link
Copy Markdown
Member Author

Correction to this PR's description (2026-11-10): the 50 runs of PRJNA1186192 (44) and PRJNA1381729 (6) are not another species. Their k-mers match Af293 at ANI 0.991-1.000; they hold a few hundred reads each (likely targeted sequencing filed as WGS), so they are unlike every other sample for lack of data. The isolates of another species are 6 public runs (A. oryzae/flavus-type, A. nidulans, A. fischeri, A. lentulus, 2 unresolved) plus local ones (Moose_3-1.1 is A. nidulans). The MAT frequency does not change (same isolates left out). The page and the exclusion reasons are corrected on branch afum-study-titles; catalog fix proposed in stajichlab/PopSeq#37.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant