Skip to content

[FEAT] Record the seed each random draw used as random_state_, and handle random_state as documented - #1605

Open
scks1 wants to merge 10 commits into
casact:mainfrom
scks1:fix-random-state-1568-on-ee2c24d1
Open

scks1 wants to merge 10 commits into
casact:mainfrom
scks1:fix-random-state-1568-on-ee2c24d1

Conversation

@scks1

@scks1 scks1 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

BootstrapODPSample and the Benktander family now build their generators with sklearn.utils.check_random_state, so random_state behaves as the docstrings say:

  • None follows the global np.random state, so np.random.seed repeats a run.
  • 0 reproduces the ultimates as well as the triangles. The process-variance seed used to be None for 0.
  • A RandomState instance works. It used to raise a TypeError.

With random_state=None, fit draws one integer, uses it for every draw and keeps it as random_state_, so passing it back repeats the run. On a triangle with several columns, each column draws its own integer, so the columns stay independent as on main, and np.random.seed before fit repeats the run. Integer seeds give the same results as before, with every column sharing the seed.

This also fixes BornhuetterFerguson(apriori_sigma=0.5) on a sparse triangle, which raises AttributeError: 'function' object has no attribute 'RandomState' on main.

Tests: six in test_bootstrap.py and two in test_benktander.py, on the numpy and sparse backends.

Related GitHub Issue(s)

Closes #1568

AI/LLM Usage

This change was made with Árbol, an AI assistant running on Muse, working under my direction. It wrote the change and the tests, ran the checks and drafted this description. I reviewed the diff and the results before submitting.

Additional Context for Reviewers

Three questions:

  1. The resampled triangle's .random_state is now random_state_, the seed the draws used, where it was the constructor value. Should it keep the constructor value?
  2. With random_state=None, predict now reuses the seed fit drew instead of drawing a new one. Is that what you want?
  3. With several columns, an integer seed gives every column the same draws, as on main, while None gives independent columns. Should an integer seed give independent columns too? That would change seeded multi-column results.

The cupy branch is untested (no GPU here). The bootstrap, methods, development and utilities tests pass with the uv.lock versions, and ruff is clean.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

…nted

BootstrapODPSample and the Benktander family build their generators with
sklearn.utils.check_random_state through a small helper, so None draws from
the global np.random generator, an integer seeds a new RandomState and a
RandomState instance is used as it is. The process-variance step offsets an
integer seed by 1 without a truthiness test, so seed 0 now reproduces the
ultimates as well as the resampled triangles. Integer seeds other than 0
give the same draws as before.

Modules without a random.RandomState of their own, such as sparse, now draw
through numpy, so apriori_sigma no longer raises AttributeError on a sparse
triangle.

Part of casact#1568.

Developed with an AI assistant (Árbol, running Muse).
When random_state is None, BootstrapODPSample.fit and the Benktander family's
fit draw one integer from the global np.random generator, use it for every
draw and keep it as random_state_, so passing it back as random_state repeats
the run. On a triangle with several columns and random_state=None, each
column's child estimator draws its own integer, so the columns' residual
draws stay independent as before, and np.random.seed before fit repeats the
run. An integer or a RandomState instance passed in is kept as it is and
shared by every column. The resampled Triangle carries random_state_ for the
process-variance step, and Benktander.predict reuses it for the apriori draws.
When apriori_sigma is 0, no draw is made and random_state_ is random_state.

Closes casact#1568.

Developed with an AI assistant (Árbol, running Muse).
@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 17.9% of exported symbols fully typed (303 / 1691)

Known Ambiguous Unknown Total
Project (head) 303 96 1292 1691

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 330
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 14)

Known Ambiguous Unknown Total
Patch 0 0 14 14
Patch symbol details
Symbol Status Change
chainladder.adjustments.tests.test_bootstrap.test_check_random_state_on_a_module_with_its_own_generator ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_different_seeds_give_different_draws ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_accepts_random_state_instance ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_given_is_recorded ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_none_independent_columns ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_none_records_the_seed ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_none_uses_global_generator ❌ unknown new
chainladder.adjustments.tests.test_bootstrap.test_random_state_zero_reproduces_ultimates ❌ unknown new
chainladder.methods.tests.test_benktander.test_apriori_sigma_predict_redraws_without_a_seed ❌ unknown new
chainladder.methods.tests.test_benktander.test_apriori_sigma_random_state_forms ❌ unknown new
chainladder.methods.tests.test_benktander.test_apriori_sigma_records_the_seed ❌ unknown new
chainladder.methods.tests.test_capecod.test_capecod_apriori_sigma_predict_redraws_without_a_seed ❌ unknown new
chainladder.utils.utility_functions.RandomStateMixin ❌ unknown new
chainladder.utils.utility_functions.RandomStateMixin.random_state_ ❌ unknown new

@codecov

codecov Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.06%. Comparing base (a59efc9) to head (9e41fda).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1605      +/-   ##
==========================================
+ Coverage   95.03%   95.06%   +0.03%     
==========================================
  Files         100      100              
  Lines        6460     6508      +48     
  Branches      819      824       +5     
==========================================
+ Hits         6139     6187      +48     
  Misses        203      203              
  Partials      118      118              
Flag Coverage Δ
unittests 95.06% <100.00%> (+0.03%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Developed with an AI assistant (Árbol, running Muse).
@henrydingliu

Copy link
Copy Markdown
Member

is the codecov miss legit? i can start the review if it's a false alarm (which does happen occasionally)?

@scks1

scks1 commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

It's real, but it's only the GPU path. The three missed lines and the partial one are the cupy branch of _check_random_state, which runs only when the array module has its own random.RandomState. CI installs the test and docs extras, and cupy is only in performance, so CI can't reach that branch.

I've pushed f6a4507, a test that covers that branch with a stand-in module, so it needs no cupy. In a local coverage run it covers the lines Codecov flagged, and the bootstrap, Benktander and Cape Cod tests pass. Codecov should clear once it reruns on the new head.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

thanks for adding the tests!

as I try to comprehend the overall approach of this PR, i'm wondering if we are only inches away from a dedicated EstimatorWithRandom (will need a better name) mixin? the shared methods, the identical random_state_ estimated attribute... maybe such a mixin already exists out there (maybe in actsim)? also let me know if i'm suffering from some delusions of grandeur and the mixin would overcomplicate rather than simplify

@scks1

scks1 commented Nov 14, 2026

Copy link
Copy Markdown
Contributor Author

Thanks! I don't think one exists yet. actsim's develop branch has none: its StochasticSimulator takes seed and ClaimSimulator takes random_seed, and both build an np.random.default_rng Generator, where chainladder uses RandomState through sklearn's check_random_state. sklearn has no random-state mixin either.

I'm happy to add one in this PR. Modeled on DisposalMixin, a RandomStateMixin would hold the two shared steps: one sets random_state_ from _record_seed, and one returns _check_random_state(self.random_state_, xp). BootstrapODPSample and Benktander would use it, and BornhuetterFerguson, CapeCod and ExpectedLoss would get it through Benktander. Some handling would stay in each class:

  • Benktander records a seed only when apriori_sigma != 0.
  • BootstrapODPSample passes a seed to each column.
  • The process-variance seed offset runs on the transformed Triangle, not the estimator.
  • Each class keeps its random_state_ paragraph in its docstring.

If that shape works for you, I'll push it here. A better name is welcome.

The three questions in the description are still open whenever you get to them.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

can you clarify if those three questions are in regards to benktander or bootstrap? i think 2 and 3 are benktander and 1 is bootstrap but wanted to confirm

@scks1

scks1 commented Nov 17, 2026

Copy link
Copy Markdown
Contributor Author

Two of the three are about BootstrapODPSample and one is about Benktander, so 3 is the bootstrap one rather than Benktander.

  1. Bootstrap. transform sets the resampled triangle's random_state to random_state_.
  2. Benktander, and the methods built on it. When apriori_sigma is not 0, predict draws its aprioris through _get_benktander_aprioris, which seeds from random_state_, so with random_state=None it reuses the seed fit drew.
  3. Bootstrap. With several columns, fit gives each column its own seed when random_state=None and the same seed when it is an integer. Benktander draws one apriori per index row, so columns don't come into it there.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

@scks1 sorry i forgot about this. can you address the conflict and we can close this out?

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

No problem. I merged main into the branch and fixed the conflict in test_bootstrap.py. Both sides only added tests at the end of the file, so the merge keeps the tests from this PR and the new hat matrix fallback test from main. The PR's own changes are the same as before. The bootstrap and Benktander tests pass locally on the merged branch.

The three questions in the description stay as the PR has them unless you want a change. I left the RandomStateMixin out of this push so the PR keeps its current scope, and I can open it as a follow-up if you want it.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

Two of the three are about BootstrapODPSample and one is about Benktander, so 3 is the bootstrap one rather than Benktander.

  1. Bootstrap. transform sets the resampled triangle's random_state to random_state_.

to check my understanding, is this only an issue when random_state is None?

  1. Benktander, and the methods built on it. When apriori_sigma is not 0, predict draws its aprioris through _get_benktander_aprioris, which seeds from random_state_, so with random_state=None it reuses the seed fit drew.

my vote is for redrawing when random_state=None. my rationale is this:

  • we should enable maximum flexibility when possible
  • reusing random_state_ when random_state=None means the user has no flexibility to get new draws on predict
  • however, redrawing on predict when random_state=None doesn't preclude the user from setting the same seed, we just have to document it in the docstring
  1. Bootstrap. With several columns, fit gives each column its own seed when random_state=None and the same seed when it is an integer. Benktander draws one apriori per index row, so columns don't come into it there.

(This comment was written with Árbol, an AI assistant, under my direction.)

in the end state we should let the user choose. not sure if it's convenient to fit into this PR. we can keep the existing inconsistency and open up a new issue

lastly, I'd prefer to get RandomStateMixin first. I get nervous whenever we have too many orphan private functions in utils.

Add RandomStateMixin to utils/utility_functions.py, next to
_record_seed and _check_random_state. Its _record_random_state sets
random_state_ in fit, and its _get_random_state returns the generator
for the recorded seed on the triangle's array module.

BootstrapODPSample and Benktander use it in place of their own calls,
and BornhuetterFerguson, CapeCod and ExpectedLoss get it through
Benktander. Benktander passes draws=False when apriori_sigma is 0.
The per-column seeds in BootstrapODPSample, the process-variance seed
offset on the transformed Triangle and each class's random_state_
docstring stay as they were. Results do not change.

Developed with an AI assistant (Árbol, running Muse).
With apriori_sigma above 0 and random_state=None, predict used to reuse
the seed fit recorded as random_state_, so every predict repeated the
fit's aprioris. predict now asks RandomStateMixin._get_random_state for a
generator with redraw=True, which draws a new seed through _record_seed
when random_state is None. An integer seed still gives the same aprioris
in fit and every predict, and a RandomState instance keeps advancing.

The random_state docstrings of Benktander, BornhuetterFerguson, CapeCod
and ExpectedLoss describe the predict behavior. New tests cover predict
for all four classes. fit and random_state_ do not change.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks. Yes, on 1 the mismatch only happens with random_state=None. I checked all three forms on raa with BootstrapODPSample(n_sims=50) and printed whether the resampled triangle's random_state is the constructor value.

int 42           triangle.random_state is constructor value: True
RandomState(42)  triangle.random_state is constructor value: True
None             triangle.random_state is constructor value: False

An integer and a RandomState instance pass through unchanged. With None the triangle carries the integer that fit drew, which is random_state_. Passing that integer back repeats the run. After np.random.seed(7) the recorded seed was 327741615, and the summed Chainladder ultimate came to 10732763.568327516 both times. So the triangle's random_state differs from the constructor only in the case where the constructor gives no seed to keep.

On 2, the second commit below makes predict redraw when random_state=None. On 3, the column behavior stays as it is.

Two commits on top of the merge are ready for the next push, with the mixin first as you asked.

  1. RandomStateMixin in utils/utility_functions.py. BootstrapODPSample and Benktander use its _record_random_state in fit and its _get_random_state for their draws, in place of the separate private calls. BornhuetterFerguson, CapeCod and ExpectedLoss get it through Benktander. Results do not change.
  2. predict on Benktander and the methods built on it calls _get_random_state(xp, redraw=True). With random_state=None that draws a new seed through _record_seed, so np.random.seed still repeats a run. An integer still gives the same aprioris in fit and every predict. New tests in test_benktander.py cover all four classes.

The random_state docstring in all four classes now ends like this.

If None, fit draws an integer seed from the RandomState instance used by np.random and keeps it as random_state_, and each predict draws a new seed the same way, so its aprioris differ from the fit's. Pass an integer to get the same aprioris from fit and every predict. A RandomState instance keeps advancing, so each predict continues its sequence.

Should I open the issue for letting users choose independent columns in BootstrapODPSample?

(This comment was written with Árbol, an AI assistant, under my direction.)

Comment thread chainladder/utils/utility_functions.py Outdated
return _axis_orders[_get_axis_number(axis)]


def _check_random_state(seed, xp=np):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should these three functions stay inside the mixin class?

Comment thread chainladder/utils/utility_functions.py Outdated
return _axis_orders[_get_axis_number(axis)]


def _check_random_state(seed, xp=np):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you please add type hinting?



def test_random_state_zero_reproduces_ultimates(raa):
"""

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for this and the next two tests, do we have and/or need the converse test, i.e. different local/global seeds lead to different results?

``np.random.seed`` repeats the run, and that an integer seed is still shared
by every column.
"""
tri = cl.load_sample("raa")

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this test can work on sparse, please use the test fixture

Comment thread chainladder/adjustments/bootstrap.py Outdated
_process_variance_seed,
)

random_state = _check_random_state(_process_variance_seed(self.random_state), xp)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the process variance seed recorded (when self.random_state=None)? does it need to be?

)


def test_capecod_apriori_sigma_predict_redraws_without_a_seed(

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a dedicated test_capecod.py

raise ValueError("sample_weight is required.")

super().fit(X, y, sample_weight)
self._record_random_state(draws=self.apriori_sigma != 0)

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the relationship between apriori_sigma, random_state and the redraw parameter seems needlessly complicated. do we really need that redraw parameter? is self.random_state is None not identical to redraw in all cases?

…hints

_check_random_state, _process_variance_seed and _record_seed become
static methods of RandomStateMixin, with type hints. The bootstrap
process-variance step and its test call them through the class.

Developed with an AI assistant (Árbol, running Muse).
A new test checks that different integer seeds, different global
np.random seeds and RandomState instances built from different seeds
give different resampled triangles and ultimates. The independent-columns
test now takes the raa fixture, so it also runs on the sparse backend.

Developed with an AI assistant (Árbol, running Muse).
Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review, Henry. I pushed a commit for each point and answer all seven comments below.

Helpers in the mixin. _check_random_state, _process_variance_seed and _record_seed are now static methods of RandomStateMixin. Nothing outside this PR imported them. The bootstrap process-variance step runs on the returned triangle instead of the estimator, so it calls them through the class.

Type hints. All five mixin methods now carry them.

Converse tests. They were missing. test_different_seeds_give_different_draws compares seeds 0 and 1, global seeds 1 and 2 with random_state=None, and RandomState(42) with RandomState(43). Each pair gives different resampled triangles and different ultimates.

Code
import numpy as np
import chainladder as cl

raa = cl.load_sample("raa")


def total_ultimate(random_state):
    sims = cl.BootstrapODPSample(n_sims=20, random_state=random_state).fit_transform(raa)
    return round(float(np.nansum(cl.Chainladder().fit(sims).ultimate_.values)), 2)


print("seed 0", total_ultimate(0), "seed 1", total_ultimate(1))
np.random.seed(1)
a = total_ultimate(None)
np.random.seed(2)
print("global seed 1", a, "global seed 2", total_ultimate(None))
print("RandomState(42)", total_ultimate(np.random.RandomState(42)), "RandomState(43)", total_ultimate(np.random.RandomState(43)))

Output

seed 0 4427953.31 seed 1 4444521.21
global seed 1 4340281.86 global seed 2 4244723.72
RandomState(42) 4247318.02 RandomState(43) 4251353.75

Sparse fixture. test_random_state_none_independent_columns now takes the raa fixture and passes on both backends.

Process-variance seed. fit records it. With random_state=None, fit draws an integer and stores it as random_state_. transform copies that integer to the returned triangle, and the process-variance draws use the integer plus 1. On raa with n_sims=50, random_state_ came out as 1872583848 and the process-variance seed as 1872583849. A refit with random_state=1872583848 repeated ultimate_ and process_variance_ exactly. So I don't think it needs an attribute of its own. A RandomState instance behaves differently. The process-variance draws continue the instance, so two Chainladder fits on the same resampled triangle give different ultimates. That matches how an instance behaves everywhere else.

Code
import numpy as np
import chainladder as cl
from chainladder.utils.utility_functions import RandomStateMixin

raa = cl.load_sample("raa")
np.random.seed(2)
boot = cl.BootstrapODPSample(n_sims=50).fit(raa)
sims = boot.transform(raa)
print("random_state_", boot.random_state_, "triangle random_state", sims.random_state)
print("process-variance seed", RandomStateMixin._process_variance_seed(sims.random_state))

first = cl.Chainladder().fit(sims)
again = cl.BootstrapODPSample(n_sims=50, random_state=boot.random_state_).fit_transform(raa)
repeat = cl.Chainladder().fit(again)
print("refit repeats ultimate_", np.array_equal(first.ultimate_.values, repeat.ultimate_.values, equal_nan=True))
print("refit repeats process_variance_", np.array_equal(first.process_variance_.values, repeat.process_variance_.values, equal_nan=True))

inst = cl.BootstrapODPSample(n_sims=50, random_state=np.random.RandomState(9)).fit_transform(raa)
one = cl.Chainladder().fit(inst).ultimate_.values
two = cl.Chainladder().fit(inst).ultimate_.values
print("RandomState instance, two fits on the same sims match", np.array_equal(one, two, equal_nan=True))

Output

random_state_ 1872583848 triangle random_state 1872583848
process-variance seed 1872583849
refit repeats ultimate_ True
refit repeats process_variance_ True
RandomState instance, two fits on the same sims match False

test_capecod.py. The CapeCod predict test now lives there.

The redraw parameter. self.random_state is None doesn't match it. fit and predict both see random_state=None, but they need different seeds. fit must draw from the seed it records in random_state_, so that passing that seed back repeats the fit. predict needs a fresh seed. A check on random_state is None alone can't tell the two calls apart. I tried that version, and a refit with the recorded random_state_ no longer repeated the fit's aprioris. The other cases never reach the branch. apriori_sigma=0 makes no draws, and an integer or a RandomState instance skips it. CapeCod goes through Benktander.fit and Benktander.predict, so it follows the same rule.

Code
import numpy as np
import chainladder as cl
from chainladder.utils.utility_functions import RandomStateMixin as M

gen = cl.load_sample("genins")
weight = gen.latest_diagonal * 0 + 1


def refit_repeats():
    model = cl.BornhuetterFerguson(apriori_sigma=0.5).fit(gen, sample_weight=weight)
    seed = model.random_state_
    repeat = cl.BornhuetterFerguson(apriori_sigma=0.5, random_state=seed).fit(gen, sample_weight=weight)
    return np.array_equal(model.expectation_.values, repeat.expectation_.values)


print("with the redraw flag", refit_repeats())


def without_flag(self, xp=np, redraw=False):
    if self.random_state is None:
        return M._check_random_state(M._record_seed(None), xp)
    return M._check_random_state(getattr(self, "random_state_", self.random_state), xp)


M._get_random_state = without_flag
print("with random_state is None only", refit_repeats())

Output

with the redraw flag True
with random_state is None only False
estimator            random_state    sigma  predict == fit
Benktander           None            0.5    False
Benktander           int 7           0.5    True
Benktander           RandomState(7)  0.5    False
CapeCod              None            0.5    False
CapeCod              int 7           0.5    True
CapeCod              RandomState(7)  0.5    False
any of the four      any             0.0    True
Code
import numpy as np
import chainladder as cl

gen = cl.load_sample("genins")
weight = gen.latest_diagonal * 0 + 1
premium = gen.latest_diagonal * 0 + 8e6
forms = {"None": lambda: None, "int 7": lambda: 7, "RandomState(7)": lambda: np.random.RandomState(7)}

for estimator, sw in [(cl.Benktander, weight), (cl.BornhuetterFerguson, weight), (cl.ExpectedLoss, weight), (cl.CapeCod, premium)]:
    for label, make in forms.items():
        for sigma in (0.5, 0.0):
            model = estimator(apriori_sigma=sigma, random_state=make()).fit(gen, sample_weight=sw)
            fit = model.expectation_.set_backend("numpy").values
            predicted = model.predict(gen, sample_weight=sw).expectation_.set_backend("numpy").values
            print(f"{estimator.__name__:20s} {label:15s} {sigma}  {np.array_equal(fit, predicted)}")

Output

Benktander           None            0.5  False
Benktander           None            0.0  True
Benktander           int 7           0.5  True
Benktander           int 7           0.0  True
Benktander           RandomState(7)  0.5  False
Benktander           RandomState(7)  0.0  True
BornhuetterFerguson  None            0.5  False
BornhuetterFerguson  None            0.0  True
BornhuetterFerguson  int 7           0.5  True
BornhuetterFerguson  int 7           0.0  True
BornhuetterFerguson  RandomState(7)  0.5  False
BornhuetterFerguson  RandomState(7)  0.0  True
ExpectedLoss         None            0.5  False
ExpectedLoss         None            0.0  True
ExpectedLoss         int 7           0.5  True
ExpectedLoss         int 7           0.0  True
ExpectedLoss         RandomState(7)  0.5  False
ExpectedLoss         RandomState(7)  0.0  True
CapeCod              None            0.5  False
CapeCod              None            0.0  True
CapeCod              int 7           0.5  True
CapeCod              int 7           0.0  True
CapeCod              RandomState(7)  0.5  False
CapeCod              RandomState(7)  0.0  True

The flag can still go if each caller passes its own seed. fit would pass random_state_ and predict would pass _record_seed(self.random_state). I checked that version against the current code for Benktander, BornhuetterFerguson, ExpectedLoss and CapeCod with all three seed forms. It gave identical aprioris in the fit and in two predicts each time. Would you like that change in this PR?

Code
import numpy as np
import chainladder as cl
from chainladder.utils.utility_functions import RandomStateMixin as M

gen = cl.load_sample("genins")
weight = gen.latest_diagonal * 0 + 1
premium = gen.latest_diagonal * 0 + 8e6
current = cl.Benktander._get_benktander_aprioris


def seeded(self, X, sample_weight, redraw=False):
    """Draw the aprioris from a seed the caller picks, with no redraw branch."""
    seed = M._record_seed(self.random_state) if redraw else self.random_state_
    state = M._check_random_state(seed, X.get_array_module())
    cov = self.apriori_sigma / self.apriori
    sigma_log = np.sqrt(np.log1p(cov**2))
    mu_log = np.log(self.apriori) - 0.5 * sigma_log**2
    draws = state.lognormal(mu_log, sigma_log, X.shape[0])
    apriori = sample_weight * draws.reshape(X.shape[0], -1)[..., None, None]
    apriori.index = X.index.copy()
    apriori.columns = sample_weight.columns
    return apriori


def run(estimator, sw, random_state, method):
    cl.Benktander._get_benktander_aprioris = method
    np.random.seed(5)
    model = estimator(apriori_sigma=0.5, random_state=random_state()).fit(gen, sample_weight=sw)
    out = [model.expectation_.set_backend("numpy").values]
    for _ in range(2):
        out.append(model.predict(gen, sample_weight=sw).expectation_.set_backend("numpy").values)
    cl.Benktander._get_benktander_aprioris = current
    return out


forms = {"None": lambda: None, "int 7": lambda: 7, "RandomState(7)": lambda: np.random.RandomState(7)}
for estimator, sw in [(cl.Benktander, weight), (cl.BornhuetterFerguson, weight), (cl.ExpectedLoss, weight), (cl.CapeCod, premium)]:
    for label, make in forms.items():
        a, b = run(estimator, sw, make, current), run(estimator, sw, make, seeded)
        print(f"{estimator.__name__:20s} {label:15s} {all(np.array_equal(x, y) for x, y in zip(a, b))}")

Output

Benktander           None            True
Benktander           int 7           True
Benktander           RandomState(7)  True
BornhuetterFerguson  None            True
BornhuetterFerguson  int 7           True
BornhuetterFerguson  RandomState(7)  True
ExpectedLoss         None            True
ExpectedLoss         int 7           True
ExpectedLoss         RandomState(7)  True
CapeCod              None            True
CapeCod              int 7           True
CapeCod              RandomState(7)  True

The full suite passes with 1735 tests passed and 9 skipped.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu henrydingliu left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for including the code and output along with the narrative. it's a lot easier to process!

fit must draw from the seed it records in random_state_

this feels like a new scenario. when would we call fit twice? i'm looking through both existing tests and new tests from this PR and not finding any example. i'd venture that the overall design of these estimators is to fit once and maybe transform/predict multiple times. i believe this means that _record_random_state can key off of self.random_state completely?

Comment thread chainladder/utils/utility_functions.py Outdated
An integer seed is offset by 1 so the two sets of draws differ, including
for seed 0. A ``RandomState`` instance or ``None`` is passed through.
"""
if isinstance(seed, (int, np.integer)) and not isinstance(seed, bool):

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seed is never a bool per type hinting?

Benktander.fit now draws its aprioris from the seed it records as
random_state_, and Benktander.predict draws from random_state. With
random_state=None, predict therefore gets fresh aprioris from the global
generator, and the fit stays repeatable from random_state_. The
_get_random_state redraw parameter is no longer needed.

_process_variance_seed no longer excludes bool, which the type hints
already rule out.

Developed with an AI assistant (Árbol, running Muse).
@scks1

scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, Henry. I worded that line badly. Nothing in the PR calls fit twice. The case I meant uses two estimators. The first fits with random_state=None and records the integer it drew as random_state_. A second estimator built with random_state=first.random_state_ then repeats the first one's draws. That is the #1568 fix, and test_random_state_none_records_the_seed checks it for the bootstrap.

That case still needs fit to draw from random_state_. With random_state=None, a generator keyed off random_state alone is the global one, so the fit's draws no longer come from the integer it records.

random_state_ repeats the run True
keyed off random_state alone False
Code
import numpy as np
import chainladder as cl
from chainladder.utils.utility_functions import RandomStateMixin as M

raa = cl.load_sample("raa")


def repeats():
    np.random.seed(3)
    first = cl.BootstrapODPSample(n_sims=20).fit(raa)
    again = cl.BootstrapODPSample(n_sims=20, random_state=first.random_state_).fit(raa)
    a, b = first.resampled_triangles_.values, again.resampled_triangles_.values
    return np.array_equal(a, b, equal_nan=True)


print("random_state_ repeats the run", repeats())
M._get_random_state = lambda self, xp=np, redraw=False: M._check_random_state(
    self.random_state, xp
)
print("keyed off random_state alone", repeats())

The redraw flag can go, though. In a local commit, Benktander.fit passes random_state_ to the apriori draw and Benktander.predict passes random_state. With None, predict draws fresh aprioris from the global generator. An integer or a RandomState instance behaves as before. Keying off random_state alone fails 4 tests in the bootstrap, Benktander, CapeCod and predict test files, both cases of test_random_state_none_records_the_seed and both of test_apriori_sigma_records_the_seed. The local commit passes all 93 tests in those files and 1735 in the full suite. The example from 6994622454 prints the same lines on it as on 214f68f.

random_state_ 1872583848 triangle random_state 1872583848
process-variance seed 1872583849
refit repeats ultimate_ True
refit repeats process_variance_ True
RandomState instance, two fits on the same sims match False
Example from 6994622454
import numpy as np
import chainladder as cl
from chainladder.utils.utility_functions import RandomStateMixin

raa = cl.load_sample("raa")
np.random.seed(2)
boot = cl.BootstrapODPSample(n_sims=50).fit(raa)
sims = boot.transform(raa)
print("random_state_", boot.random_state_, "triangle random_state", sims.random_state)
print("process-variance seed", RandomStateMixin._process_variance_seed(sims.random_state))

first = cl.Chainladder().fit(sims)
again = cl.BootstrapODPSample(n_sims=50, random_state=boot.random_state_).fit_transform(raa)
repeat = cl.Chainladder().fit(again)
print("refit repeats ultimate_", np.array_equal(first.ultimate_.values, repeat.ultimate_.values, equal_nan=True))
print("refit repeats process_variance_", np.array_equal(first.process_variance_.values, repeat.process_variance_.values, equal_nan=True))

inst = cl.BootstrapODPSample(n_sims=50, random_state=np.random.RandomState(9)).fit_transform(raa)
one = cl.Chainladder().fit(inst).ultimate_.values
two = cl.Chainladder().fit(inst).ultimate_.values
print("RandomState instance, two fits on the same sims match", np.array_equal(one, two, equal_nan=True))

On the bool guard, you're right. The hints say int, and bool is a subclass of int, so random_state=True seeds the main draws as 1. With the guard gone, the process-variance draws get 2 instead of reusing 1. The same commit drops it.

# 214f68fb
process-variance seed for True: True
first draw, main vs process variance: 0.417022004702574 0.417022004702574
# local commit
process-variance seed for True: 2
first draw, main vs process variance: 0.417022004702574 0.43599490214200376
Snippet
import numpy as np
from chainladder.utils.utility_functions import RandomStateMixin as M

seed = M._process_variance_seed(True)
print("process-variance seed for True:", seed)
print("first draw, main vs process variance:",
      M._check_random_state(True, np).rand(), M._check_random_state(seed, np).rand())

I can push it to this PR if that design looks right to you.

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu

Copy link
Copy Markdown
Member

sounds good. please push

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEAT] Record the seed each random draw used, and handle random_state=0 and RandomState instances as documented

2 participants