Skip to content

registry: stitch the 1998 grain re-code at scale 0.2 - #123

Merged
mspinola merged 1 commit into
mainfrom
claude/grain-1998-recode
Oct 10, 2026
Merged

mspinola merged 1 commit into
mainfrom
claude/grain-1998-recode

Conversation

@mspinola

Copy link
Copy Markdown
Owner

Summary

  • ZC, ZS, ZW, ZO and KE gain a hist_codes predecessor at scale 0.2: 002601, 005601, 001601, 004601, 001611. The five CBOT/KCBT grains re-coded at the 1997/1998 year boundary and the registry carried only the new codes, so the store's grain history began 1998-01-06 although the CFTC publishes it from 1986-01-15.
  • The old codes are the same contracts reported in thousands of bushels rather than contracts (5,000 bu each). Evidence, measured against the CFTC deacot1986_2016 bundle:
    • every position column is divisible by 5 on 99.3 to 100% of pre-1998 rows and on 0% after the switch
    • open interest drops 5.0x (ZC), 5.5x (ZS), 5.0x (ZW), 5.7x (KE) at the seam; oats 3.8x, which is a real OI move (its rows are still 100% divisible by 5)
    • reportable trader counts do not move: corn 366 to 345, wheat 164 to 169, oats 31 to 30
  • No overlap week (old ends 1997-12-30, new starts 1998-01-06), so the overlap rule from get_cot: overlap weeks keep the larger scaled open interest #122 is not exercised.
  • Registry comment carries the evidence inline; CHANGELOG Unreleased / Added.

Test plan

  • tests/test_registry.py::test_grain_predecessors_scaled_to_contracts: the five mappings at 0.2, and ZM / ZL / ZR still single-code
  • Full suite passes; ruff check src tests scripts clean
  • End-to-end against the real producer into a scratch COTDATA_STORE: cftc.update(codes=[the ten codes]) then get_cot for each symbol gives 1986-01-15 to 2026-10-06, no duplicate dates, one code label, and a level-continuous seam (ZW OI 93,742 on 1997-12-30 to 97,105 on 1998-01-06; KE 45,071 to 45,252)

Deployment

The producer writes the five new per-code files on its next --cot run after the Windows box pulls this. Until then get_cot returns the post-1998 series exactly as before (a missing predecessor file is skipped). The 08:10 catch-up syncs both replicas unconditionally, so the files reach the Mac the morning after the pull.

🤖 Generated with Claude Code

ZC, ZS, ZW, ZO and KE gain a hist_codes predecessor (002601, 005601,
001601, 004601, 001611) at scale 0.2. The five CBOT/KCBT grains re-coded
at the 1997/1998 year boundary and the registry carried only the new
codes, so the store's grain history began 1998-01-06 although the CFTC
publishes it from 1986-01-15.

The old codes are the same contracts reported in thousands of bushels
rather than contracts (5,000 bu each): every position column is divisible
by 5 on 99.3 to 100% of pre-1998 rows and on none after, open interest
drops 5.0 to 5.7x at the seam (oats 3.8x, a real move), and trader counts
do not move. No overlap week.

get_cot now returns the primary frame untouched when no predecessor file
is present, so a symbol with hist_codes behaves exactly as before until
the producer writes the new files.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mspinola
mspinola merged commit 713ea03 into main Oct 10, 2026
5 checks passed
@mspinola
mspinola deleted the claude/grain-1998-recode branch October 10, 2026 15:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant