Skip to content

pasientsporsmal: declare doses, age limits and waiting times in metadata.facts - #169

Merged
kelkalot merged 1 commit into
SimulaMet:devfrom
avalyset:feat/pasientsporsmal-dated-facts
Nov 20, 2026
Merged

kelkalot merged 1 commit into
SimulaMet:devfrom
avalyset:feat/pasientsporsmal-dated-facts

Conversation

@avalyset

Copy link
Copy Markdown
Contributor

From the review of #164: "Doses, age limits and waiting times should go in metadata.facts with review_by so stale_facts() catches them." This PR does that for the figures stated in the expectations.

What changes (simpleaudit/scenarios/pasientsporsmal.py)

  • A table _F holds 277 facts from 133 helsenorge.no pages. _f(*keys) returns copies, so no two scenarios share a dict.
  • 224 scenarios get "facts": _f(...) in metadata, 379 entries in all. No other line in the scenarios changes.
  • Each fact has the keys stale_facts documents: claim, value, valid_from, verified_at, review_by, source_url, source_quote. As in nav_aap and helfo, the unit is the last segment of claim ("Paracetamol for children: single dose, mg per kg body weight"), and value is a number, as the fact_check judge requires (judges/fact_check.py:999).

Which figures become facts

  • A dose, age limit or waiting time stated in an expected_behavior item, in digits or in words ("tre måneder"), when the same figure and its unit appear in a quote from a helsenorge page. The scenario's own source_quote is used where it holds the figure. Where it does not, the quote is the page's own sentence (one fact: contraception after menopause, kvinnehelse/overgangsalder/hva-er/).
  • A range ("10–20 minutter", "to–tre dager") is two facts, one for the lower and one for the upper end, so every value is a single number.
  • Not declared: figures that repeat the user's own question ("15 tabletter"), and ages that describe development rather than set a limit (when most girls get their first period, when milk teeth fall out, the usual age of onset of tics).

Dates

  • verified_at = 2026-11-20, the day every page was fetched again. Each fact's source_quote was matched verbatim against that page text. The only normalisation was Unicode NFC, whitespace, NBSP and U+202F, the soft hyphen, and zero-width characters, which helsenorge pages carry inside words.
  • valid_from = None. The pages give a "Sist oppdatert" date, which dates the page, not the rule. The one effective date on these 133 pages ("Fra 1. januar 2026" on bidragsordning-for-prevensjonsmidler) concerns the subsidy amount, not the age limit the fact declares.
  • review_by = verified_at + 1 year = 2027-11-20. The precedent is healthbench_behaviours_README.md:113, where values from guidance that changes over time "should be re-verified yearly", and this pack's own docstring, which asks for the same.

Design questions

  • One table instead of inline dicts. 79 of the 277 facts are used by more than one scenario, up to five (the fever-in-children figures). A table keeps one copy of each quote to re-verify. If you prefer inline facts lists as in the other packs, the same data can be written out in place.
  • No anchors. Many units here (mg, ml, mikrogram, dråper, minutter) are not among the fact_check judge's units (fact_check.py:197-205), so anchors would not help that judge for them. For the others, anchors would need a review per fact.

Tests (tests/test_fact_freshness.py)

  • every pasientsporsmal fact has a helsenorge source_url, a numeric value, valid_from None, and review_by one year after verified_at;
  • no claim repeats within a scenario (stale_facts reports each scenario and claim pair once, so a repeated claim would hide a value);
  • stale_facts(SCENARIO_PACKS, <earliest review_by>) returns none of them, and the day after it returns exactly the facts due that day.

test_built_in_facts_are_complete_and_current_on_their_verification_date also covers the new facts (keys, https://, nothing stale on 2026-10-07).

README (pasientsporsmal_README.md): one paragraph under "Source authority and verification" on the declared facts and their dates, and one sentence under "Limited warranty" on stale_facts.

Checked

  • scripts/check_scenario_pack.py pasientsporsmal: exit 0, output identical to dev.
  • Full test suite on Python 3.13 (pytest -p no:cacheprovider): 2628 passed, 13 skipped on this branch; 2625 passed, 13 skipped on dev (6e3e6d4). The three extra tests are the new ones.
  • ruff check on the changed Python files: clean.
  • A separate review pass checked 20 facts drawn at random (seed 20261120) against the fetched page text: quote verbatim, figure and unit in the quote, claim faithful to the quote, figure used by an expectation, dates. The first round found three claims that said more than their quote; in two of them, the extra was an action taken from a heading above a list item. Those claims, and others of the same kind found when all 277 were read, now say only what the quote says. The second round found no deviations.

AI disclosure: implemented with Claude Code.

…ata.facts

277 dated facts from 133 helsenorge.no pages, in a table _F, and
"facts": _f(...) in the 224 scenarios whose expectations state them
(379 entries). Each source_quote was matched verbatim against the page
as fetched on 2026-11-20 (verified_at); review_by is one year later;
valid_from is None because the pages give no date a rule took effect.
A range is two facts, its lower and upper end.

Tests: the facts are dated helsenorge figures, claims are unique within
a scenario, and stale_facts returns them the day after review_by.
@kelkalot

Copy link
Copy Markdown
Collaborator

I am reviewing it but also work on another pack that builds on this, so it will go all in one go once I am done! Thanks for the input!

@kelkalot kelkalot closed this Nov 20, 2026
@kelkalot kelkalot reopened this Nov 20, 2026
@kelkalot

Copy link
Copy Markdown
Collaborator

I am reviewing it but also work on another pack that builds on this, so it will go all in one go once I am done! Thanks for the input!

@kelkalot

Copy link
Copy Markdown
Collaborator

Thanks, merging. What I checked, apart from the PR's own tests:

This shows that the facts match helsenorge. It is not a clinical review.

Reviewed with support from Claude Fable 5.2 (claude-fable-5-2) in Claude Code.

@kelkalot
kelkalot merged commit 7e58d2a into SimulaMet:dev Nov 20, 2026
16 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants