Skip to content

blast-jobs: BLAST job service for the MCP's BLAST tools - #6

Merged
christabone merged 12 commits into
masterfrom
feat/blast-jobs
Oct 12, 2026
Merged

christabone merged 12 commits into
masterfrom
feat/blast-jobs

Conversation

@christabone

Copy link
Copy Markdown
Member

blast-jobs: BLAST job service for the MCP's BLAST tools

A sibling crate blast-jobs/ that implements the internal job API the MCP's blast_submit / blast_status / blast_results tools call, by running pinned NCBI BLAST+ 2.17.0 (official tarball, SHA-256 verified) against a catalogue-described database directory, isolated in its own container.

What ships

  • Config + catalogue-driven database mapping (no site specifics in Rust); typed per-program advanced flag table with an explicit deny list; per-program residue caps and wall limits; masking defaults matching the website.
  • On-disk job store with restart recovery and 24 h retention; FIFO queue with a limit; N workers.
  • axum API: POST /internal/jobs, GET /internal/jobs/{id}, GET /internal/jobs/{id}/results.tsv (6 std stitle, ≤ 16 MiB incl. header, X-Blast-Jobs-Truncated), GET /internal/health; bearer auth (constant-time), 429 + Retry-After on a full queue, 413 body limit.
  • Dockerfile (multi-stage, non-root, HEALTHCHECK), image smoke test, real-BLAST integration test (self-skips without BLAST_JOBS_REAL_BLAST), dedicated path-filtered workflow with contents: read.
  • Benchmark harness (blast-jobs/bench/) with public RefSeq query sequences.
  • MCP side: blast_results now reports results_truncated; five per-field 400 codes mapped to their arguments (handoff §7 updated).

Verification

  • blast-jobs: 137 tests with the fake BLAST, 8 real BLAST+ 2.17.0 end-to-end tests, image 332 MB with smoke test; fmt, clippy, cargo deny, shellcheck, actionlint clean.
  • mcp: lib 57, tools_blast 13, conformance 4 (23 tools); catalog unchanged.
  • Process: design reviewed by Opus and Fable (two revisions), five tasks each with an independent review and fix round, Fable whole-branch review with one fix wave and a scoped re-review; 43 hostile advanced strings verified rejected or contained.

Not in this PR

  • Production rollout and benchmark (private side, docker_production).
  • The MCP end-to-end ignored test against a live service (follow-up).

Dockerfile: rust:1.90-bookworm --locked build; NCBI BLAST+ 2.17.0 x64 tarball from the versioned
HTTPS path pinned by SHA-256 3888112d...a411b; bookworm-slim runtime with curl and libgomp1, the
seven binaries in /opt/ncbi-blast/bin, -version checks at build time, uid 10002, HEALTHCHECK on
/internal/health?liveness=1.

tests/real_blast.rs (skips unless BLAST_JOBS_REAL_BLAST names a BLAST+ bin dir): builds the Notch
fixture with the archive's makeblastdb flags (-parse_seqids) and checks a known hit for blastn,
blastp, blastx, tblastn and tblastx, -task blastn-short via advanced, and the dust/seg defaults.
tests/image-smoke.sh: blast-jobs check plus one real search and the health check inside the image.

CI: blast-jobs-integration and blast-jobs-image, on master pushes and blast-jobs/ changes only.
Docs: docs/blast-jobs.md, deploy samples, handoff section 7 points at blast-jobs.
…AST test pins 2.17.0

Move blast-jobs-integration and blast-jobs-image to .github/workflows/blast-jobs.yml with on.paths
triggers and contents: read, dropping dorny/paths-filter. Pass the tarball pin through env. The
real-BLAST test no longer falls back to PATH: every binary comes from BLAST_JOBS_REAL_BLAST and must
report 2.17.0+, or the test fails.
- bench/run.sh: cleanup kills the run_job workers (and the curl or sleep
  each is waiting on) before removing $tmp, so an interrupted run leaves
  no pollers behind; pids are cleared once reaped.
- Rejections of evalue, max_hits, matrix, megablast and genetic_code use
  invalid_<field> codes, and the MCP maps each to the tool argument of
  that name; max_hits and genetic_code are read signed so a negative
  value is coded rather than a serde failure; genetic_code is ignored
  unchecked for blastn and blastp.
- Integer advanced flags are bounded to a C int (max_hsps stays clamped),
  so an oversized value is a 400 rather than a failed job.
- scrub replaces backtick-quoted and key=/path tokens.
- Catalogue::load refuses a database_root with whitespace or quotes.
- ci.yml: top-level permissions: contents: read.
- Remove validate::validate and State::as_str (unused); one SERVICE_OWNED
  (advanced) and one QUERY_FILE (command).
- The wall-limit test's /proc scan runs on Linux only.
- Docs: field codes, truncation header, lcase_masking has no effect, no
  slow-client timeout, BLAST_BASE_URL example, handoff downloads wording.
results_tsv reads X-Blast-Jobs-Truncated and returns it with the body.
blast_results adds results_truncated (the envelope already owns
`truncated` for its own size cut) and, when true, a summary sentence
saying total_hits is a lower bound and how to narrow the search.
@christabone
christabone merged commit ebd86f8 into master Oct 12, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant