Per-kind gate: queryspy_fail_on = lazy_load,column_load

[lbedner]

Ask

Let queryspy_fail_on (and/or --queryspy-strict) take a list of detector kinds, so a suite can fail on lazy_load and column_load while only reporting or baselining repeated_statement:

[tool.pytest.ini_options]
queryspy_fail_on = lazy_load,column_load

Today the pytest plugin never looks at Finding.kind: the only levers are per-test markers, queryspy.ignore() in app code, and the baseline. There is no way to say "gate on the unambiguous detectors, collect the heuristic one".

Why

Adopting on a FastAPI + async SQLModel project generator (14 stack variants, 22k tests total), measured on 0.4.1:

• Merged across every stack: 0 lazy_load, 27 column_load, 346 repeated_statement. The first two were all real and all fixed in an afternoon (one pattern: refresh() after a write with no server_default). None of the 346 was.
• repeated_statement at threshold 2 is dominated by two things that no application change can remove:
  1. The test fixture hands one AsyncSession to every request in a test (the usual transaction-rollback isolation). Two requests that each legitimately look the same user up once read as "2 identical queries". Switching to per-request sessions removed ~40% of findings but broke 44 tests.
  2. Labels exclude bind parameters by design, so session.get(A) + session.get(B) in a rotation-chain test are "identical".

So a cold --queryspy-strict is unreachable (56 failures on a fresh two-service project, 842 on a real one), while a cold gate on lazy_load + column_load would have been green the same day. We ended up baselining 364 entries to get the gate up; a kind filter would have let us gate on the two detectors immediately and adopt the third with the baseline, which matches the precision ordering the README already describes.

Shape

Purely additive: default stays "all kinds". _resolve_policy would carry a kinds: frozenset[str] and _enforce would filter findings before raising; the marker could accept the same (@pytest.mark.queryspy(fail_on=...)). Happy to send a PR if the shape is agreeable.

Thanks for the tool. The attribution to the ORM call site and the baseline keyed on what does not move are both exactly right.

[rodrigobnogueira]

Thanks — 0 / 27 / 346 across 22k tests is the best measurement of the precision ordering anyone has produced, and both repeated_statement sources you name are by design, so a kind filter is the right lever rather than a threshold tweak.

Your shape is the one I'd take: queryspy_fail_on accepting a list of kinds (n_plus_one staying as the alias for all), _Policy carrying kinds, _enforce filtering before it raises, and the marker accepting the same fail_on=. Two asks for the PR: unknown kinds should fail collection loudly rather than silently gate on nothing, and the baseline stays kind-agnostic so switching the gate later doesn't invalidate it.

A PR would be very welcome. If you'd rather not, say so and I'll write it.

[lbedner]

@rodrigobnogueira I'm going to be completely honest here, I was not even aware my agent was writing this 👀 . Let me look this over to see how much of it is still relevant, sorry to waste your time.

PS. I'm loving this project so far, it's a solid part of my CI process now, so thanks for building it.

[lbedner]

@rodrigobnogueira Okay, I'm back up to speed, I will write up a PR for this.