Repository navigation
Conversation
findMissingResidues() built an array indexed by residue number and slid it against each SEQRES sequence looking for an offset where the occupied slots matched. Numbering conventions that skip numbers without any residue being absent, as Kabat, EU and IMGT numbering all do, put a phantom empty slot in that array for every skipped number. Once the array grew longer than the sequence the range of candidate offsets was empty, so no alignment was even attempted and the chain was silently left with its gaps unfilled. Replace the layout with a dynamic programming alignment of the residues to the sequence. Residue numbers are now only a hint: a gap is cheaper to open where the numbering is discontinuous, which is where the depositor has told us residues are absent. Also warn when a chain contains the residues its sequence lists but cannot be aligned, rather than doing nothing silently. Fixes openmm#154, openmm#170, openmm#219. The missing residues now found agree residue for residue with the REMARK 465 records of 1A0Q, 1AO7, 6RPA, 6RPB, 5J7S, 5UDC, 4JSV and 1BHL, and no missing residues are reported for 1IGT, 1VII, 1UBQ, 3NIR, 2HMI, 1AKE or 4HHB. The first five of those found none at all before this change.
|
Short answer to this: Claude is wrong. The PDB spec is pretty clear about this:
You don't get the use an arbitrary, domain based numbering system. The only reason you're allowed to have a skip is if a residue is present in the molecule but you don't have any coordinates for it. A file that does it differently is incorrect. PDBx/mmCIF tried to deal with this better by having separate |
|
I think this is known to be an issue where the files in the PDB disagree with the documentation. I can see why you keep the principled approach though. |
Fixes #219. Fixes #170. Fixes #154.
These three issues have the same root cause, so one change closes all of them.
For transparency all code here was written by Claude but at first glance seemed
close enough to what I was thinking of doing and tests seem sensible. The
wordy description below has the same source.
This maybe overkill and make things more fragile but if this looks promising happy
to follow up.
The problem
findMissingResidues()builds an array indexed by residue number and slides itagainst each SEQRES sequence looking for an offset at which every occupied slot
matches. That only works when residue numbers run
1, 2, 3, ….Kabat and EU numbering for antibodies, IMGT numbering, and other domain-based
conventions deliberately skip numbers without any residue being absent. Each
skipped number becomes a phantom empty slot. Once the array is longer than the
sequence,
range(len(sequence.residues) - len(chainWithGaps[chain]) + 1)is anempty range: the loop body never runs, no alignment is attempted, and the
chain silently drops out of the matching step.
missingResiduesgets no entriesfor it,
addMissingAtoms()builds nothing, andfindMissingAtoms()then caps thelast observed residue with OXT — so a chain truncated mid-sequence comes out
looking well-formed. No exception, no warning.
1IGT chain B is a good illustration: 444 observed residues and 444 SEQRES entries
that agree residue for residue, but 8 insertion codes and 23 numbering gaps make
the computed span 482, so the offset range is −37 and the chain is never examined.
In #219 the conclusion was that 6RPA's numbering is inconsistent with its SEQRES.
It isn't — IMGT numbering skips numbers by design, and the reporter was right
"the SEQRES represents properly the structure".
The insertion-code shim added in #184 addresses only part of this. Making its
bump conditional on the number not having already advanced — the obvious minim
fix — takes 1IGT chain B from span 482 to 481, not 444. I measured it; it removes
one phantom slot out of 38. The number-indexed layout is the problem, not the
The change
Align the observed residue list to the sequence, and use residue numbers onl
a hint for where to place gaps.
New
_alignResiduesToSequence(residues, sequence)returns a strictly increasingmap from residue index to sequence index, or
None. Notes on the design:name, preserving the current
a == bsemantics. No new tolerance for renamed ornon-SEQRES residues.
skipped sequence elements is fixed at
len(sequence) - len(residues)and ontheir placement is free. A linear gap penalty is therefore a constant and does
nothing; scoring gap openings prefers a few continuous missing loops over
scattered single residues.
discontinuous, since that is where the depositor has told us residues are absent.
This is what makes the result exact rather than merely plausible: without it
missing-residue counts are already correct everywhere I tested, but placement
drifts by one inside runs of repeated residues — 6RPA D and 6RPB F/K/P gave
[MET, GLY, ARG]where the deposition says[MET, ARG, GLY].vectorised numpy operations. 0.05 s for a 2000-residue chain, 0.19 s at 4000.
The sequence-major assignment order is unchanged: sequences are considered in file
order and each is assigned to the first chain it aligns to. That ordering is
load-bearing, since chain IDs are not unique — it is what stops a one-residue
ligand chain from claiming the 212-element sequence of the polymer chain
Lishares an ID with, as happens in 1A0Q.
findMissingResidues()now also warns when a chain contains exactly the residuesone of the sequences lists but still cannot be aligned, rather than doing noth
silently. It stays quiet when the names themselves disagree, since that means the
structure genuinely does not contain what the sequence lists.
chainWithGaps,chainSequenceandchainOffsetwere function-local and nevreached
self, andmissingResidueskeeps its(chain.index, indexInChain) -> [names]contract, so_addAtomsToTopology,findMissingAtoms,ui.pyandManual.htmlare untouched. No public API change, no new dependency.Structures tested
Compared residue by residue, not just by count, against the deposited
REMARK 465records. Exact agreement on all 16.6RPA, 6RPB, 1AO7 and 5J7S are the structures from #219; 1A0Q is #154; 5UDC and
antibodies generally are #170. 4JSV, 1BHL and 5UDC already worked and are included
to catch regressions — 5UDC appears to have been fixed incidentally by #184, s
its insertion codes happen to land such that the array does not overrun.
1IGT deserves a note: nothing is missing from it, so the reported result is empty
both before and after. What changes is that chains B and D are now aligned to
sequences instead of being skipped, which is what the test asserts.
Also verified directly: non-numeric residue IDs (previously an unguarded
int(r.id)), single-residue chains, chains longer than their sequence, chainsno possible match, and empty chains.
Tests
New
pdbfixer/tests/test_find_missing_residues.py, 7 tests. No existing test wasmodified. Full suite: 31 passed.
Four tests are offline and use two new fixtures:
data/insertion_codes.pdb— a hand-written chain with Kabat-style numbering(
51, 52, 52A, 52B, 53), a numbering jump from 57 to 60 with nothing absent, andone genuine two-residue gap. Asserts the exact
missingResiduesdict, and tbuilding the structure yields the 14 sequence residues and nothing else.
data/unalignable_order.pdb— exercises the warning.polymer produce no warnings.
None of the five existing fixtures contained a single insertion code (I checke
column 27 of every ATOM record), which is why new ones were needed.
Three parametrized tests download 1IGT, 1A0Q and 1VII from RCSB. I marked them
skipif(os.getenv("GITHUB_ACTION") is not None), following the existing pattetest_cli.py— happy to drop that if you would rather they run in CI, sincetest_charge_and_solvate.pyalready downloads there.Not addressed here
Related defects in the same family — treating
residue.idas a positional indthat I have deliberately left alone:
_addAtomsToTopologyderives IDs for newly built residues by arithmetic onint(residue.id), which collides or runs backwards on Kabat-numbered chainsCosmetic: names, geometry and connectivity are correct, but the output numbering
of a repaired antibody is not meaningful.
applyMutationskeysresSeq_to_residueonint(residue.id), collapsing100and
100Aonto one key.findNonstandardResiduesmatches MODRES on(chain.id, residue.id, residue.name)with the insertion code dropped on both sides.
One pre-existing behaviour is worth flagging because strict name matching pres
it rather than fixing it (#350): calling
applyMutations()beforefindMissingResidues()renames a residue in the topology but not in
self.sequences, so the chain nolonger matches its sequence and its gaps are silently not filled — the whole chain,
not just the mutated region. On master and on this branch alike:
Scoring mismatches instead of forbidding them would fix that, with a match-fraction
threshold so a small ligand chain cannot claim a polymer's sequence. I left it
to keep this change to one behaviour, but say the word and I will add it.