Skip to content

fix(canu): make canu failing to run fatal instead of continuing (#92) - #93

Merged
gbouras13 merged 3 commits into
mainfrom
fix/canu-error-handling
Oct 22, 2026
Merged

gbouras13 merged 3 commits into
mainfrom
fix/canu-error-handling

Conversation

@gbouras13

@gbouras13 gbouras13 commented Oct 22, 2026 •

Copy link
Copy Markdown
Owner

Refs #92. This is the plassembler side. The root cause in the image is BusyBox's expr rejecting the expr -- that canu's job scripts use, and that is being fixed separately by adding coreutils to the bioconda recipe. So this PR doesn't close the issue.

Problem

Any non-zero exit from canu -correct was reported as:

Canu failed to correct any reads. This probably means there is low depth, don't be too concerned.

and plassembler went on with uncorrected reads. In #92 canu could not run at all: meryl-count failed on every attempt. Going on with uncorrected reads left Unicycler assembling 110 spurious contigs, which the depth filter then had to discard. Once canu worked there were none, and the run took 2.5× less time.

Change

canu ends every failed run with the same ABORT: Don't panic … banner, including the low-coverage stop that a plasmid-free sample is expected to hit. So the banner can't decide. The line before it can:

canu prints meaning now
-- <stage> jobs failed, tried N times, giving up. a compute job never produced its output fatal
a CRASH: banner canu's own internal error fatal
Read coverage (N) lower than allowed. too few reads carries on, saying so
anything else not recognised carries on, showing canu's stderr

This matches failure lines rather than whitelisting harmless stops. canu has ~200 caExit sites, and missing one harmless stop would turn plasmid-free samples into failed runs.

On the fatal path the canu directory is kept, because the run exits before remove_directory. The message includes the tail of canu's stderr, which names each job's own log. For #92 that is correction/0-mercounts/meryl-count.000001.out, which contains expr: syntax error.

canu now runs directly from run_canu.py, the same way qc.py handles chopper. The two canu special cases in ExternalTool.run_tool are removed.

Verification

I ran real canu 2.3 with plassembler's exact arguments on tests/test_data/end_to_end/abaumanii_plasmid.fastq.gz:

scenario exit job-failure line low-coverage line
normal 0 – –
expr wrapper rejecting -- like BusyBox 1 ✓ –
50 reads (3.7×) 1 – ✓
empty input 1 – ✓
20 reads at the fallback genomeSize 0 – –
30× non-overlapping random reads 0 – –

None of the thin-data cases prints the job-failure line. I also drove the fixed run_canu_correct against real canu under the CLI's exiting ERROR sink. The BusyBox case exits 1, low coverage warns and carries on, and a normal run is silent. In canu's source the three lines are identical in 2.2 and 2.3.

tests/test_run_canu.py adds 12 tests that replay real canu 2.3 stderr (fixtures in tests/test_data/canu/) through a stub canu. 10 of them fail against main, and the other 2 are the success cases, which are meant to pass on both. The non-slow suite passes (102), and ruff format --check and ruff check are clean. I didn't run the slow tests locally, because this machine has no bioinformatics tool stack.

Also bumps the version to 1.8.6, with release notes crediting the reporter.

gbouras13 and others added 3 commits October 22, 2026 15:21
Any non-zero exit from canu -correct was reported as "This probably
means there is low depth, don't be too concerned", and plassembler went
on with uncorrected reads. In #92 that hid a canu that could not run at
all: in the Biocontainers image, canu's job scripts call `expr -- ...`,
which BusyBox's expr rejects, so meryl-count failed every time. Going on
with uncorrected reads left Unicycler assembling 110 spurious contigs.

canu ends every failed run with the same "ABORT: Don't panic" banner,
including the coverage stop a plasmid-free sample is expected to hit, so
the banner cannot decide. The lines before it can:

- "-- <stage> jobs failed, tried N times, giving up." is how canu reports
  any compute job that never produced its output. Jobs only start once
  the reads pass canu's coverage checks, so this is canu's environment
  failing. Fatal.
- A "CRASH:" banner is canu's own internal error. Fatal.
- "Read coverage (N) lower than allowed." is the thin-reads stop. Carries
  on as before, now saying why.
- Anything else carries on too, but shows canu's stderr instead of
  reassurance. canu has close to 200 ways of stopping, and one this does
  not know must not turn a plasmid-free sample into a failed run.

Checked against real canu 2.3 runs with plassembler's own arguments: a
busybox-style expr reproduces #92 and is now fatal; too few reads, no
reads, thin reads and non-overlapping reads either stop on coverage or
exit 0, and none of them prints the job-failure line. The signatures read
the same in canu 2.2.

The fatal path keeps the canu directory, which holds each job's own log -
for #92, the meryl-count log with "expr: syntax error" in it. The same
handling covers canu assembly with --canu_flag. canu now runs directly
rather than through ExternalTool.run_tool, so its two special cases there
are removed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Left unreleased: no version bump.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@gbouras13
gbouras13 merged commit 2199dbc into main Oct 22, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant