Skip to content

[FIX] Reject stepped and non-contiguous origin and development slices - #1677

Open
scks1 wants to merge 2 commits into
casact:mainfrom
scks1:fix-noncontig-origin-dev-slice
Open

scks1 wants to merge 2 commits into
casact:mainfrom
scks1:fix-noncontig-origin-dev-slice

Conversation

@scks1

@scks1 scks1 commented Nov 19, 2026

Copy link
Copy Markdown
Contributor

Summary of Changes

This PR fixes the slicing bug from #1649. The origin dtype check in _compatibility_check stays a separate fix.

_contig_slice turns evenly spaced positions into a stepped slice. The check in get_idx only asked whether the origin and development keys were slices, so [0, 2, 4] and ::2 passed. Boolean masks go through _slice, which had no check, so raa[raa.origin != "1985"] returned a triangle with a missing year. Each of these built a triangle with gaps between periods.

The PR adds _LocBase._is_unit_slice. It accepts a slice with a step of None, 1 or -1. get_idx, __setitem__ and _slice use it on the origin and development axes and raise the existing fancy-indexing error for anything else. A step of -1 keeps adjacent periods in reverse order, and sort_axis and its tests rely on it.

Two library paths pick every step-th development age on purpose. Grain conversion of a cumulative triangle keeps every fourth age to go from quarterly to annual development, and _slice_valuation serves valuation masks and latest_diagonal. Both pass a private allow_gaps=True to get_idx and keep their behavior. Valuation masks therefore work as on main.

The stricter check also turned up a gap in dropna. It compared the string labels of pattern triangles, so "120-132" sorted between "12-24" and "72-84". A JSON round trip of a fitted MunichAdjustment on mcl gave ldf_ nine columns with a gap instead of six. dropna now trims development by position.

Related GitHub Issue(s)

Refs #1649

AI/LLM Usage

This change was made with Árbol, an AI assistant running on Muse, working under my direction. It wrote the fix and the tests, ran the checks below and drafted this description.

Additional Context for Reviewers

  1. Each pick on main 687e329 and on this branch.
Pick on raa main this branch
iloc[..., ::2, :] 5 origins with gaps raises
iloc[..., [0, 2, 4], :] 3 origins with gaps raises
iloc[..., ::-1, :] 10 origins reversed 10 origins reversed
iloc[..., :, ::2] 5 ages with gaps raises
raa[raa.origin != "1985"] 9 origins with a gap raises
raa[raa.development % 24 == 0] 5 ages with gaps raises
raa[raa.origin >= "1985"] 6 origins 6 origins

On this branch the script prints

iloc[..., ::2, :]: ValueError: Fancy indexing on origin/development is not supported.
iloc[..., [0, 2, 4], :]: ValueError: Fancy indexing on origin/development is not supported.
iloc[..., ::-1, :]: (1, 1, 10, 10)
iloc[..., :, ::2]: ValueError: Fancy indexing on origin/development is not supported.
origin != 1985: ValueError: Fancy indexing on origin/development is not supported.
development % 24 == 0: ValueError: Fancy indexing on origin/development is not supported.
origin >= 1985: (1, 1, 6, 10)

and on main it prints

iloc[..., ::2, :]: (1, 1, 5, 10)
iloc[..., [0, 2, 4], :]: (1, 1, 3, 10)
iloc[..., ::-1, :]: (1, 1, 10, 10)
iloc[..., :, ::2]: (1, 1, 10, 5)
origin != 1985: (1, 1, 9, 10)
development % 24 == 0: (1, 1, 10, 5)
origin >= 1985: (1, 1, 6, 10)
Code
import chainladder as cl

raa = cl.load_sample("raa")
keys = {
    "iloc[..., ::2, :]": lambda: raa.iloc[..., ::2, :],
    "iloc[..., [0, 2, 4], :]": lambda: raa.iloc[..., [0, 2, 4], :],
    "iloc[..., ::-1, :]": lambda: raa.iloc[..., ::-1, :],
    "iloc[..., :, ::2]": lambda: raa.iloc[..., :, ::2],
    "origin != 1985": lambda: raa[raa.origin != "1985"],
    "development % 24 == 0": lambda: raa[raa.development % 24 == 0],
    "origin >= 1985": lambda: raa[raa.origin >= "1985"],
}
for name, pick in keys.items():
    try:
        print(f"{name}: {pick().shape}")
    except ValueError as err:
        print(f"{name}: ValueError: {err}")
  1. The dropna gap on main shows up in a JSON round trip.
['12-24', '24-36', '36-48', '48-60', '60-72', '72-84']
['12-24', '24-36', '36-48', '48-60', '60-72', '72-84', '120-132', '132-144', '144-156']

On this branch both lines read ['12-24', '24-36', '36-48', '48-60', '60-72', '72-84'].

Code
import chainladder as cl

x = cl.MunichAdjustment(paid_to_incurred=("paid", "incurred")).fit_transform(
    cl.load_sample("mcl")
)
y = cl.read_json(x.to_json())
print(list(x.ldf_.development))
print(list(y.ldf_.development))
  1. New tests in test_slicing.py cover stepped iloc keys on both axes, non-contiguous boolean masks, a contiguous mask, setting through stepped keys, reversal, and quarterly to annual grain. test_json_pattern_keeps_development in test_utilities.py covers the dropna gap. The full suite gives 1756 passed and 9 skipped on this branch, against 1739 passed and 9 skipped on main.

  2. Two questions for you. Should reversed ::-1 slices stay allowed? Should user-facing stepped development picks such as iloc[..., :, ::2] raise, or should they stay open for users who want a coarser view?

Submitter's Checklist

  • I have reviewed and am adhering to the standards outlined in the project Governing Doc.
  • The PR subject title summarizes the changes, with one proper prefix ([FIX], [FEAT], [DOCS], [TST], [CHORE], or [BRK]).
  • I am a human (not a bot), and this PR form is written by a human.

Reviewer's Checklist

  • The implementation addresses the associated issue(s).
  • The implementation is appropriate, maintainable, and follows ARCHITECTURE.md.
  • PR subject title has the proper prefix and the subject is appropriate.
  • Relevant issue(s) are linked.
  • AI/LLM usage is disclosed and appropriate.
  • Documentation and tests are appropriate.
  • CI tests passed, or any failures are acceptable.
  • Leave a comment with the final recommendation (e.g. approve as is, request a secondary review, or flag an area for more review).

_contig_slice turns evenly spaced positions into a stepped slice, so
iloc picks such as [0, 2, 4] or ::2 and boolean masks such as
origin != "1985" passed the fancy-indexing check and built triangles
with gaps between periods. get_idx, __setitem__ and _slice now accept
only slices with a step of None, 1 or -1 on the origin and development
axes and raise the existing fancy-indexing error otherwise. A step of
-1 keeps adjacent periods and sort_axis relies on it.

Two library paths pick every step-th development age on purpose. Grain
conversion of a cumulative triangle and _slice_valuation pass a private
allow_gaps flag to get_idx and keep their behavior.

dropna compared the string labels of pattern triangles, so "120-132"
sorted between "12-24" and "72-84". A JSON round trip of a fitted
MunichAdjustment on the mcl sample therefore gave ldf_ nine columns
with a gap instead of six. dropna now trims development by position.

Refs casact#1649.

Developed with an AI assistant (Árbol, running Muse).
@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Pyright Type Completeness

View the full pyright --verifytypes output for this commit

Project (full chainladder package, at this PR's head): 18.1% of exported symbols fully typed (305 / 1688)

Known Ambiguous Unknown Total
Project (head) 305 96 1287 1688

Other symbols referenced but not exported by chainladder: 24

Known Ambiguous Unknown Total
Other (head) 11 1 12 24

Symbols without documentation:

  • Functions without docstring: 329
  • Functions without default param: 0
  • Classes without docstring: 10

Patch (exported symbols added or changed by this PR): 0.0% fully typed (0 / 6)

Known Ambiguous Unknown Total
Patch 0 0 6 6
Patch symbol details
Symbol Status Change
chainladder.core.tests.test_slicing.test_get_idx_stepped_development_raises ❌ unknown new
chainladder.core.tests.test_slicing.test_get_idx_stepped_origin_raises ❌ unknown new
chainladder.core.tests.test_slicing.test_non_contiguous_boolean_mask_raises ❌ unknown new
chainladder.core.tests.test_slicing.test_reversed_and_coarser_grain_still_work ❌ unknown new
chainladder.core.tests.test_slicing.test_set_stepped_origin_development_raises ❌ unknown new
chainladder.utils.tests.test_utilities.test_json_pattern_keeps_development ❌ unknown new

Developed with an AI assistant (Árbol, running Muse).
@codecov

codecov Bot commented Nov 19, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.04%. Comparing base (687e329) to head (f3685ed).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #1677   +/-   ##
=======================================
  Coverage   95.03%   95.04%           
=======================================
  Files         100      100           
  Lines        6471     6480    +9     
  Branches      819      821    +2     
=======================================
+ Hits         6150     6159    +9     
  Misses        203      203           
  Partials      118      118           
Flag Coverage Δ
unittests 95.04% <100.00%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant