Repository navigation
pasientsporsmal: declare doses, age limits and waiting times in metadata.facts - #169
Merged
kelkalot merged 1 commit intoNov 20, 2026
Merged
Conversation
…ata.facts 277 dated facts from 133 helsenorge.no pages, in a table _F, and "facts": _f(...) in the 224 scenarios whose expectations state them (379 entries). Each source_quote was matched verbatim against the page as fetched on 2026-11-20 (verified_at); review_by is one year later; valid_from is None because the pages give no date a rule took effect. A range is two facts, its lower and upper end. Tests: the facts are dated helsenorge figures, claims are unique within a scenario, and stale_facts returns them the day after review_by.
Collaborator
|
I am reviewing it but also work on another pack that builds on this, so it will go all in one go once I am done! Thanks for the input! |
Collaborator
|
I am reviewing it but also work on another pack that builds on this, so it will go all in one go once I am done! Thanks for the input! |
This was referenced Nov 20, 2026
Collaborator
|
Thanks, merging. What I checked, apart from the PR's own tests:
This shows that the facts match helsenorge. It is not a clinical review. Reviewed with support from Claude Fable 5.2 (claude-fable-5-2) in Claude Code. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
From the review of #164: "Doses, age limits and waiting times should go in
metadata.factswithreview_bysostale_facts()catches them." This PR does that for the figures stated in the expectations.What changes (
simpleaudit/scenarios/pasientsporsmal.py)_Fholds 277 facts from 133 helsenorge.no pages._f(*keys)returns copies, so no two scenarios share a dict."facts": _f(...)inmetadata, 379 entries in all. No other line in the scenarios changes.stale_factsdocuments:claim,value,valid_from,verified_at,review_by,source_url,source_quote. As innav_aapandhelfo, the unit is the last segment ofclaim("Paracetamol for children: single dose, mg per kg body weight"), andvalueis a number, as the fact_check judge requires (judges/fact_check.py:999).Which figures become facts
expected_behavioritem, in digits or in words ("tre måneder"), when the same figure and its unit appear in a quote from a helsenorge page. The scenario's ownsource_quoteis used where it holds the figure. Where it does not, the quote is the page's own sentence (one fact: contraception after menopause,kvinnehelse/overgangsalder/hva-er/).valueis a single number.Dates
verified_at= 2026-11-20, the day every page was fetched again. Each fact'ssource_quotewas matched verbatim against that page text. The only normalisation was Unicode NFC, whitespace, NBSP and U+202F, the soft hyphen, and zero-width characters, which helsenorge pages carry inside words.valid_from= None. The pages give a "Sist oppdatert" date, which dates the page, not the rule. The one effective date on these 133 pages ("Fra 1. januar 2026" onbidragsordning-for-prevensjonsmidler) concerns the subsidy amount, not the age limit the fact declares.review_by=verified_at+ 1 year = 2027-11-20. The precedent ishealthbench_behaviours_README.md:113, where values from guidance that changes over time "should be re-verified yearly", and this pack's own docstring, which asks for the same.Design questions
factslists as in the other packs, the same data can be written out in place.anchors. Many units here (mg, ml, mikrogram, dråper, minutter) are not among the fact_check judge's units (fact_check.py:197-205), so anchors would not help that judge for them. For the others, anchors would need a review per fact.Tests (
tests/test_fact_freshness.py)pasientsporsmalfact has a helsenorgesource_url, a numericvalue,valid_fromNone, andreview_byone year afterverified_at;stale_factsreports each scenario and claim pair once, so a repeated claim would hide a value);stale_facts(SCENARIO_PACKS, <earliest review_by>)returns none of them, and the day after it returns exactly the facts due that day.test_built_in_facts_are_complete_and_current_on_their_verification_datealso covers the new facts (keys,https://, nothing stale on 2026-10-07).README (
pasientsporsmal_README.md): one paragraph under "Source authority and verification" on the declared facts and their dates, and one sentence under "Limited warranty" onstale_facts.Checked
scripts/check_scenario_pack.py pasientsporsmal: exit 0, output identical to dev.pytest -p no:cacheprovider): 2628 passed, 13 skipped on this branch; 2625 passed, 13 skipped on dev (6e3e6d4). The three extra tests are the new ones.ruff checkon the changed Python files: clean.AI disclosure: implemented with Claude Code.