Repository navigation
Fix zero protein counts on PRIDE (proteins-above-lod) - #43
Merged
Merged
Conversation
Adds a new `get_unique_samples()` function and `pyprideap unique-samples` CLI command that extracts unique sample identifiers from a dataset. Supports local file input or PRIDE accession download, with output to stdout or file. Control/QC samples are excluded by default (toggle with --include-controls). Closes #23 https://claude.ai/code/session_01FRn28g9LvUfggGWSzm7hud
Changed default behavior: all samples (including controls) are returned by default. Use --exclude-controls flag or exclude_controls=True to filter out control/QC samples. https://claude.ai/code/session_01FRn28g9LvUfggGWSzm7hud
Import specific functions directly from their submodules at the top of cli.py instead of lazily importing the full package inside each command. https://claude.ai/code/session_01FRn28g9LvUfggGWSzm7hud
Skip SampleID when its cardinality matches the feature count (indicating it is assay-indexed rather than sample-level). Also recognize SampleId (lowercase 'd') used by SomaScan datasets. https://claude.ai/code/session_01FRn28g9LvUfggGWSzm7hud
Add unique-samples command to list unique sample identifiers
CI typecheck has failed on every dev push since 2026-10-06 (the April runs
passed). The dev extras are unpinned, so CI picks up pandas 3 /
pandas-stubs 3.0.5 / numpy 2.5 / mypy 2.4, which report 25 errors in code
that had not changed; later merges added 8 more.
- returns of pandas expressions that the stubs type as Any are cast to
the declared return type (no behaviour change)
- dict-or-Series parameters converted with pd.Series(...) are cast so the
union narrows (normalization)
- Series.replace("", pd.NA) -> replace({"": pd.NA}); Hashable-keyed status
lookup; reused variable names in the summary table and dilution QC
renamed; manifest tabs typed as list[dict[str, Any]]
Fix mypy errors from current pandas-stubs (unblocks #39)
- unique-samples unpacked _download_pad_files() as a list, but it returns (files, sdrf_path) since #38, so `unique-samples -a PADxxxxxx` crashed (caught by mypy on main) - get_unique_samples(exclude_controls=True) matched SampleType without normalising separators, so NEGATIVE_CONTROL / PLATE_CONTROL / SAMPLE_CONTROL were kept; it now uses control_sample_mask like filter_controls - CLI tests for unique-samples with a mocked PRIDE download
Release 1.2.0
Five PAD datasets show 0 proteins on PRIDE because proteins-above-lod returned an empty list. Causes, from the deposited files: - Olink exports with both LODNPX and LOD (count-based) were compared with LOD: no NPX ever exceeded it (PAD000037). The reader now uses LODNPX. - hand-rearranged NPX files with shifted columns put text in NPX/LOD and the LOD comparison raised (PAD000024). NPX/LOD/LODNPX are coerced to numbers; broken rows become NaN. - table-only ADATs without ^HEADER/^COL_DATA markers were rejected (PAD000045 v4.0 file). They are parsed as a TABLE_BEGIN body. - SomaScan ADATs with study samples only (no buffers) have no LOD source (PAD000003, PAD000011, PAD000045 1.3k file). All measured protein reagents (Type == "Protein") are returned with a warning; Olink without any LOD still returns none. Counting also now uses the report's LOD resolver (moved to processing.lod.resolve_lod_with_source; SomaScan uses eLOD, not NCLOD on buffers) and only study samples, so controls below LOD no longer lower the per-assay share (PAD000002 1858 -> 1943, PAD000004 149 -> 186).
|
ⓘ Qodo reviews are paused because the subscription is no longer active. Ask your workspace admin to reactivate the subscription to resume reviews. Manage billing |
|
Note Currently processing new changes in this PR. This may take a few minutes, please wait... ⚙️ Run configuration
📒 Files selected for processing (5)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Five datasets show 0 proteins on the PRIDE affinity page. The PRIDE API's
noOfProteinscomes from this library'sproteins-above-lod, which returned an empty list for them. I downloaded the deposited files and found these causes:.npx.csvLODNPXand a count-basedLOD(median 1,002); NPX was compared withLOD, so 0 of 1,319 assays passedLODNPXMDD_rearranged_NPX.npx.csvPlateID.1–3,QC.1–3); text inNPX/LODmade the comparison raiseNPX/LOD/LODNPXcoerced to numbers; broken rows become NaN^HEADER/^COL_DATAmarkers; the reader rejected it^TABLE_BEGINbodyType == "Protein") with a warningWhy fall back to measured proteins for SomaScan: in datasets that do have buffers, nearly every SomaScan assay is above eLOD: 94.9–99.7% per dilution on PAD000021. Olink datasets without any LOD still return none, because many Olink assays really are below LOD.
Also in this PR
get_proteins_above_lodnow usesresolve_lod_with_source, moved toprocessing/lod.pyand re-exported fromviz.qc.compute. SomaScan therefore uses eLOD rather than an NCLOD computed on buffer samples, matching Use one LOD resolver across the QC report (SomaScan eLOD, FixedLOD shown as active) #30.Other PRIDE values will change on regeneration
Several current counts were produced by an older pyprideap build and are stale; for example,
mainalready gives 2,193 for PAD000022 against 1,882 on PRIDE. On local data, the rest move by −8 to +36: PAD000001 1,347→1,361, PAD000006 2,447→2,458, PAD000013 6,384→6,415, PAD000021 6,364→6,400.Tests
Typefilter for SomaScan control reagents;LODNPXpreference and text coercion;pytest171 passed.Deployment
This targets
mainso it can be part of 1.2.0, which isn't tagged yet. After release, rebuilding pride-affinity-ws regenerates the cached counts (#2 there). PRIDE'snoOfProteinsis filled from that endpoint, so whatever job writes it needs to run again.Separately, PAD000024's deposited file has shifted columns, which the curators may want to check with the submitter.
Summary by CodeRabbit