Skip to content

[DOCS] Add a gallery example comparing two tail estimators on shared bootstrap draws - #1672

Merged
henrydingliu merged 2 commits into
casact:mainfrom
scks1:docs-paired-bootstrap-1664
Nov 19, 2026
Merged

henrydingliu merged 2 commits into
casact:mainfrom
scks1:docs-paired-bootstrap-1664

Conversation

@scks1

@scks1 scks1 commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary of Changes

Adds docs/gallery/plot_paired_bootstrap.ipynb, a gallery example that fits two tail estimators to the same BootstrapODPSample draws and looks at the difference on each draw. It compares TailCurve with TailBondy on genins, once on shared draws and once on draws from a second seed. It shows the identity Var(B − C) = Var B + Var C − 2 Cov(B, C), the ratio near √(1 − ρ), and that more draws shrink the standard error of the mean difference but leave the gap between the two standard deviations in place. The gallery index gets a card and a toctree entry, and docs/images/plot_paired_bootstrap.png is the card image.

Related GitHub Issue(s)

Closes #1664

AI/LLM Usage

I used Árbol, an AI assistant running on Muse, under my direction to write the notebook, run it and draft this description. I reviewed the notebook and its numbers.

Additional Context for Reviewers

Results on main 687e329, 1,000 draws with seeds 42 and 7:

TailBondy minus TailCurve mean sd
same draws -696,715 1,540,822
independent draws -702,384 4,728,455

The correlation of the two totals across shared draws is 0.885, √(1 − ρ) is 0.338 and the observed ratio of standard deviations is 0.326.

draws ratio standard error of the shared-draw mean
200 0.303 105,259
1,000 0.326 48,725
2,000 0.331 35,350
  • The notebook runs in about 17 seconds and peaks near 680 MB.
  • pytest --nbmake passes on the notebook, and ruff 0.16.1 check and format pass.
  • A full jb build docs with execution off succeeds. The new page adds no warnings, and the link to plot_bootstrap_comparison, the shapland2016 citation and the gallery card all resolve.
  • The example relies on one fit_transform call feeding both tail estimators. [FEAT] Record the seed each random draw used as random_state_, and handle random_state as documented #1605 changes how estimators record random_state, and nothing in this notebook depends on that change.

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

…p draws

The example fits TailCurve and TailBondy to the same 1,000
BootstrapODPSample draws of genins and to draws from a second seed. The
shared-draw difference in total IBNR has a standard deviation of
1,540,822 against 4,728,455 for independent draws, a ratio of 0.326,
close to sqrt(1 - rho) at a correlation of 0.885. A second table runs
200, 1,000 and 2,000 draws to show that the ratio stays near a third
while the standard error of the mean difference falls from 105,259 to
35,350.

Developed with an AI assistant (Árbol, running Muse).
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 18.1% of exported symbols fully typed (305 / 1682)

Known Ambiguous Unknown Total
Project (head) 305 96 1281 1682

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 329
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): no exported symbol type-completeness changes detected.

@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.03%. Comparing base (687e329) to head (eb9e6d5).

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1672   +/-   ##
=======================================
  Coverage   95.03%   95.03%           
=======================================
  Files         100      100           
  Lines        6471     6471           
  Branches      819      819           
=======================================
  Hits         6150     6150           
  Misses        203      203           
  Partials      118      118           
Flag Coverage Δ
unittests 95.03% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@henrydingliu

Copy link
Copy Markdown
Member
image

we typically associate method with cl, bf, etc. this is really comparison of two tail estimators. can you please change the title?

image

for clarity, can you please specify that the "shared draws" is sims and "draws from a second seed" is other?

image

this code block didn't get any text explanation. also, this block can probably be moved up, before the draws, since it's establishing deterministic baselines for both methods.

image

this section can be broken into two. first sub-section shows the mystery of a higher st dev of independent draws (the 4.7M st dev isn't printed in a code block). second sub-section explains the math.

image

the skinny vs fat bell curves only show the result. try adding a scatter plot, bondy on X, curve on Y.

The page now compares two tail estimators, TailBondy and TailCurve, and
its title and gallery card say so. The deterministic baselines come
first, with a sentence on what they are for. The text names `sims` as
the shared draws and `other` as the draws from a second seed.

The old discussion section is now two. The first prints the standard
deviation of each method and of each difference, 1,540,822 on shared
draws against 4,728,455 on independent draws. The second gives the
variance identity and the correlations, 0.885 on shared draws and
0.011 on independent draws. A scatter plot of TailCurve against
TailBondy total IBNR, with a line of equal totals, shows the two
correlations before the histogram.

Developed with an AI assistant (Árbol, running Muse).
@scks1 scks1 changed the title [DOCS] Add a gallery example comparing two methods on shared bootstrap draws [DOCS] Add a gallery example comparing two tail estimators on shared bootstrap draws Nov 19, 2026
@scks1

scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the close read. eb9e6d5 takes all five points in order.

  1. The page is now "Comparing TailBondy and TailCurve on the Same Bootstrap Draws", and the gallery card reads "Comparing Tail Estimators on Shared Bootstrap Draws". I'll rename the PR to "[DOCS] Add a gallery example comparing two tail estimators on shared bootstrap draws".
  2. The text before the draws now names sims as the shared draws and other as the draws from a second seed.
  3. The baseline block moved above the draws. A sentence before it says it fits each tail estimator to the original genins and that the bootstrap differences should center near the difference between the two totals.
  4. The old section is now two. "Independent draws give a wider difference" has a new cell that prints each standard deviation, 1,540,822 on shared draws against 4,728,455 on independent draws. "The variance of a difference" keeps the identity and adds the correlation on independent draws, 0.011 against 0.885 on shared draws.
  5. A scatter plot follows the math, with TailBondy on X and TailCurve on Y and a dashed line of equal totals. Shared draws form a narrow band along the line and independent draws form a round cloud. The histogram stays after it.
The standard deviations and correlations
import numpy as np

import chainladder as cl

genins = cl.load_sample("genins")


def total_ibnr(triangles, tail):
    developed = tail.fit_transform(cl.Development().fit_transform(triangles))
    return cl.Chainladder().fit(developed).ibnr_.sum("origin").values.flatten()


sims = cl.BootstrapODPSample(n_sims=1000, random_state=42).fit_transform(genins)
other = cl.BootstrapODPSample(n_sims=1000, random_state=7).fit_transform(genins)

curve = total_ibnr(sims, cl.TailCurve())
bondy = total_ibnr(sims, cl.TailBondy())
bondy_other = total_ibnr(other, cl.TailBondy())

print("sd, same draws", round((bondy - curve).std(ddof=1)))
print("sd, independent draws", round((bondy_other - curve).std(ddof=1)))
print("correlation, same draws", round(np.corrcoef(curve, bondy)[0, 1], 3))
print("correlation, independent draws", round(np.corrcoef(curve, bondy_other)[0, 1], 3))
sd, same draws 1540822
sd, independent draws 4728455
correlation, same draws 0.885
correlation, independent draws 0.011

(This comment was written with Árbol, an AI assistant, under my direction.)

@henrydingliu
henrydingliu merged commit c857a76 into casact:main Nov 19, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[DOC] Gallery example: compare two methods on the same bootstrap draws

2 participants