[dbccpage]

🤖 Generated with Claude Code

Jeremy's rule: customers keep their types, so floats must be cacheable. A float may enter the cache, but it may never pretend to be exact. When a float cannot be handled exactly, it is refused by name and the refusal is visible as an event.

The invariant is that there is no path from a binary float to an exact aggregate. It holds on the event path and, since the second commit, on every cold-rebuild path too.

What changes

Event and repair path (first commit, 18f71e3)

Site	Before	After
_to_decimal(float)	Converted with Decimal(str(x)), so 0.1 became Decimal('0.1')	Raises FloatNotExact (subclass of TypeError, reason float_not_exact)
Repair checks (single table, joins, normal forms)	A float in a group, aggregate or predicate column gave INVALIDATE invalid_evidence_type, the same reason as bool	Gives INVALIDATE float_not_exact; bool keeps invalid_evidence_type
SQLite cold SUM over a column not declared decimal	SMPL_DEC_SUM summed str(x): 0.1 + 0.2 gave Decimal('0.3'), while SQLite's own SUM is 0.30000000000000004	SMPL_STRICT_SUM refuses REAL as float_not_exact
_values_equal	Two floats were compared through str()	IEEE equality between two floats. A float never equals an exact number

Cold-rebuild paths (second commit, 10f1088): fixes from the external review

#	Review finding	Reproduced	Fix
P0	A float entering a verified-decimal column kept SMPL_DEC_SUM, so the rebuild gave Decimal('0.3')	Yes	Sticky per-column float taint revokes exact-decimal eligibility for good. Later decimals cannot re-admit it. On DuckDB, DECIMAL becomes DOUBLE
P0	fetch_table_rows turned floats into Decimal for join, shared-pool and per-shape pool rebuilds	Yes, all three gave {'gold': Decimal('0.3')}	Floats stay floats on every fetch. A refused join rebuild empties and invalidates its cache, so no stale value survives. Sibling pool shapes still rebuild, and the call raises once, naming the refused shapes
P1	SMPL_STRICT_SUM summed numeric TEXT as decimals (0.3), while SQLite gives 0.30000000000000004	Yes	TEXT and BLOB are refused as invalid_evidence_type, as the exact fold refuses them
P1	process_or_rebuild raised FloatNotExact out of the call after INVALIDATE	Yes	Returns (UNSUPPORTED "uncacheable_float_aggregate", cache, False) with cache.uncacheable_reason = "float_not_exact" and the cache emptied. The caller answers from the origin query. A later clean rebuild clears the mark
P2	Float keys normalized like exact numbers (0.1 keyed like Decimal('0.1'), 1.0 like 1), contradicting _values_equal	Yes	Float row keys are a tagged FloatKey, so they match only IEEE-equal floats. Float GROUP BY values on the SQL path are refused, like the exact fold's group keys

Behaviour change. A join over a REAL column that is equal to an int or Decimal key in another table no longer matches. [Edit: a float join or primary key is refused as float_not_exact on every join path, by JoinAuxState.from_tables since the fourth commit. FloatKey now only keys AggregateCache values built directly with float keys.]

Source provenance (third commit, f33fe1a): fixes from the second review

The invariant is now stated precisely: a cached answer equals the origin query under the admitted semantics, or the query is refused.

#	Review finding	Reproduced	Fix
P1	SQLite affinity stores "2" in an INTEGER column as 2, so the cold SUM accepted what the exact fold refuses	Yes, Decimal('3')	Provenance is the adapter's observed source kinds, not storage affinity. A SUM/AVG column whose observed kinds aren't all exact numbers is refused as invalid_evidence_type. Group columns holding float, bool or mixed kinds are refused the same way
P1	A direct SQL 0.2 beside an admitted 0.1 still summed to Decimal('0.3')	Yes	On SQLite, writes the adapter did not make are detected through total_changes, and exact-decimal eligibility is revoked for good. DuckDB has no such counter, so the guarantee there covers adapter-observed writes only. The substrate says so in its own docs
P1	The SQL join matched REAL 10.0 to INTEGER 10, while the Python path refused it	Yes, SQL gave gold = 5	Join keys must share one kind on both sides, both as written and as stored. Float keys are refused as float_not_exact and TEXT '10' vs INTEGER 10 as mixed_join_key_types, on the SQL, manager and pool paths. Same-kind joins are tested equal to the origin SQL on every path
P2	Float-predicate TypeError escaped process_or_rebuild	Yes	SourceTypeRefused(TypeError) carries a reason, and FloatNotExact subclasses it. The outcome is UNSUPPORTED uncacheable_float_aggregate for floats and uncacheable_source_type otherwise, with the specific reason in cache.uncacheable_reason

Cost of the direct-write check. After any unobserved write, a decimal column is no longer certified, even if that write was exact. Integer columns are unaffected.

Review fixes (fourth commit, 299abcd)

Each finding was reproduced on 188213c, and each new test in tests/test_float_cold_paths.py fails there and passes here.

#	Finding	Reproduced	Fix
P0	RegisteredShapesRegistry with a substrate: the rebuild's refusal escaped process_write_event and left the runtime state seeded. The next int INSERT repaired it	Yes. The store served 15 where origin's SUM is 15.5	A SourceTypeRefused from the rebuild is an outcome. The runtime state is emptied and marked uncacheable, the stored value stays invalidated, and later events invalidate instead of repairing
P1	JoinCacheManager.rebuild with a float predicate value or float group key raised plain TypeError. The old values stayed servable and the aux state stayed live	Yes, the stale {'gold': 99} stayed	predicates.scalar, _checked_group_atom, _comparable_extremum_val and JoinAuxState.from_tables raise FloatNotExact for floats. The manager and the pool seed drop their old state on any rebuild failure
P1	After a refusal, a disjoint-column UPDATE returned PRESERVE with cache.values == {}	Yes	process_event never PRESERVEs or REPAIRs a cache marked uncacheable. It invalidates, so the next rebuild either refuses again or clears the mark
P1	The extremum cold SQL path read MIN/MAX over REAL through Decimal(str(x))	Yes, Decimal('0.1')	Refused as float_not_exact, as the fold and event paths already refuse it. Extremum and top-k refusals in process_or_rebuild get the UNSUPPORTED outcome
P2	Pool on_invalidate rebuild: a float refusal raised out of process_event and lost the event's decisions	Yes	_rebuild_invalidated keeps the refusal as an outcome, since the shapes are already marked
P2	FloatKey keys could not be stored (Unsupported cache wire type), and checkpoints dropped uncacheable_reason	Yes	float_key wire type, and the mark is saved and restored with the checkpoint
P3	SQLite INTEGER affinity stores 2.0 as 2	Already refused by f33fe1a	Second guard: a float-tainted column sums through SMPL_FLOAT_SUM

FloatNotExact and SourceTypeRefused move to predicates.py, the lowest layer, and are re-exported from smplcache.smplcache. FloatNotExact is also a ValueError, because the join-key and extremum sites it replaced raised ValueError.

Evidence (local, Python 3.13)

• Fourth commit (299abcd)

  • Full suite: pytest -q --ignore=tests/test_live_containers.py --ignore=tests/production_native.py, 2264 passed.
  • black --check . and mypy smplcache demo scripts/check_sql_coverage.py are clean, and scripts/check_sql_coverage.py and the CLI smoke test pass.
  • 13 new or tightened tests: all fail on 188213c and pass here.

• Full suite

  • Command: .venv/bin/python -m pytest -q -p no:cacheprovider --ignore=tests/production_native.py
  • Result: exit 0. The skips are the live-container tests.

• New tests

  • tests/test_source_provenance.py: 11 tests. Each fix fails its tests when injected away (no aggregate provenance: 2 fail; no tripwire: 1; no mixed-key refusal: 1; float-only outcome: 2), and all 11 pass after reverting.
  • tests/test_float_cold_paths.py: 16 tests, run on SQLite and DuckDB where relevant.
  • tests/test_float_not_exact.py: 17 tests.

• Types and formatting: mypy smplcache reports no issues in 40 files, and black --check is clean.

• Injections into the new fixes, each reverted afterwards (16/16 pass after reverting):

  Injection	Result
  No revocation	1 failed, the SQLite transition test. On DuckDB the DOUBLE change still refuses, as a second guard
  Float→Decimal restored in the fetch	4 failed: the fetch, manager and both pool tests
  Text admitted to the strict sum	1 failed
  Uncacheable outcome removed	3 failed

Not in this PR

• Saved-state reload in debezium.py and store_adapter.py still reads state JSON with Decimal(str(v)). Those values are written by smplcache itself, not taken from source floats.
• The native CI job. test_production_read_contention_during_publication[bounded] is timing-flaky on main. It failed on 49a3a5c and d52e174, and passed on 9e55b2f and 3a838a2.
  • Cause: the bounded worker drains about 13 generations/s against the writer's 20 commits/s, so the backlog takes 29 to 41 s against a 30 s limit.
  • The fix awaits Jeremy's choice and will come in its own PR.

https://claude.ai/code/session_0194kAmYv4pLMyrspzKEWExZ

---

Generated by Claude Code

[dbccpage]

Live Database Containers is red, but the failure is not from this PR.

• The failure: 1 failed, 53 passed. The failing test is tests/production_native.py::test_production_read_contention_during_publication[bounded], with "native source did not catch up within 30 seconds".
• Same on main: that test fails the same way on main, before this diff [Edit: the test is timing-flaky on main, not deterministically red. It failed on 49a3a5c and d52e174, and passed on 9e55b2f and on the current 3a838a2 (Reconnect dead worker sessions and retry strict reads that raced the worker #43).] I reproduced it locally against Postgres 16 with wal2json.
• Cause: the bounded worker publishes one signed generation per WAL transaction. That is about 13 generations/s against a writer doing about 20 commits/s. The backlog of about 440 drains in 29 to 41 s, against a 30 s catch_up limit, so a run passes or fails depending on where in that range it lands.
• Fix: none yet. The fix changes the publication protocol, so it waits on the maintainer's choice and will come in its own PR.
• Not re-run: re-running would only spend Actions minutes on a known flake.

---

Generated by Claude Code