You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Common random numbers across scenarios in run_sampled_sims #555
_run_sampled_sim() creates one generator per attempt (rng_attempt) and passes the same object to parset.sample(), progset.sample() and then to every run_sim() call across the list of progset_instructions. Because the generator is stateful, the second scenario starts where the first scenario's random stream ended. Verified: two scenarios run from the same sample have different trajectories before the program start year, when they should be identical.
The code comment ("it's important to START from the same seeds") describes the intended behaviour, not the actual one. Without common random numbers (CRN) the variance of scenario differences is much larger than it needs to be, which is the main thing an uncertainty analysis of interventions cares about.
Even with the fix below, CRN is only partial: Generator.multinomial consumes a variable number of uniforms depending on n, so streams desynchronise after the scenarios diverge.
Implementation plan
In _run_sampled_sim(), draw a seed per attempt, and for each instruction set create np.random.default_rng(seed) afresh before calling run_sim(). Parameter and program sampling should use a separate child seed so that the model stream is identical across scenarios regardless of how much sampling consumed.
Use np.random.SeedSequence(rand_seed).spawn(n_samples) in run_sampled_sims() and again per attempt, rather than rng.integers(1e15), which is the documented way to derive independent child streams.
Add a test: run two instruction sets from one sample and assert trajectories are identical up to the first year at which the instructions differ.
(Follow-on, larger) Full CRN: give each compartment its own Generator spawned from the model seed, so a draw in one compartment never affects the stream of another. This also answers the question on Stochastic transitions #474 about per-compartment generators. Draws per compartment per timestep are then synchronised across scenarios as long as n is the same, and partially otherwise. Measure the variance reduction on the SIR demo before and after.
Activity