You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test_compare_deterministic_markovchain: tolerance should scale with ensemble standard error; fix failure path #558
test_compare_deterministic_markovchain in tests/test_tox_markovchain.py asserts that the ensemble mean of 1000 stochastic runs is within 5% of the deterministic trajectory. For nonlinear frameworks (anything with a force of infection) the stochastic mean differs from the deterministic trajectory through Jensen's inequality and fade-out, so this is not a property of the method but an approximation that holds at large population sizes. With the SIR demo at 100 people there is 14% extinction and a 17% gap in final infections.
The test also has a fixed 5% tolerance unrelated to the ensemble's standard error, an unused do_hypothesis_test helper, the ration typo and broken index handling in its failure path, and a seed built from hash(demo) which is salted per process.
Implementation plan
Replace the fixed ratio tolerance with a test on the standardised difference: |mean_mc - det| / (sd_mc / sqrt(N)) below a threshold chosen for the number of comparisons being made (or use the existing t-test helper with a Bonferroni-adjusted alpha).
Restrict the comparison to times and compartments where the deterministic value is large enough that the large-population approximation holds, and document that choice in the test.
Fix the failure path (typo, boolean mask indexing, f-string) so a failure reports the first offending timepoint correctly.
Seed from (i + seed + demo_index) or SeedSequence([seed, i]) rather than hash(demo).
Reduce N where possible so the test is cheap enough to run on every CI job, or mark it as slow.
Activity