diff --git a/.claude/board/AGENT_LOG.md b/.claude/board/AGENT_LOG.md index 4b0d56433..d0e50a981 100644 --- a/.claude/board/AGENT_LOG.md +++ b/.claude/board/AGENT_LOG.md @@ -14,6 +14,11 @@ - Card `substrate-mapper`, three units: lance 13 `dataset/mem_wal` (5 files, 187 items), lancedb 0.40 MemWAL/LSM (4 files, ~301 items), lance-graph persist sink (`persist_sink.rs`, `batch_writer.rs`, `blw_rows.rs`, `blw_tenant.rs`, 76 items). - Ledger: spot-checks 2/2 per map against source (`set_lsm_write_spec` :73, `execute_lsm_merge_insert` :666, `WalSink` :534, `recover_and_apply` :677, and the `LsmWriteSpec` public API); 0 wrong. One stale upstream doc found by the orchestrator, not by a card: `LsmWriteSpec`'s doc calls the `merge_insert` dispatch a follow-up, but `table/merge.rs:320` already dispatches it. - Cost: about 587k, 591k and 606k subagent tokens. The maps fed the pre-registration; no conclusion came from a card. +## 2026-10-14 — D-HOLO H6c harvest: one substrate-mapper run + +- Card `substrate-mapper` (run as a general-purpose agent with the card pasted; the card is not in this session's agent list), unit: the merged `holo_tile_forms_probe.rs`, 93 items. +- Ledger: spot-checks 3/3 (`Prod::binop` L127, `Hyb::settle` L359, `time` L622); 0 wrong, 0 missed. About 515k subagent tokens. +- Outcome: the map located the sections copied into `holo_tile_crossover_probe.rs`; no conclusion came from the card. ## 2026-10-14 — D-HOLO H6b harvest: four substrate-mapper runs, one reply each diff --git a/.claude/board/LATEST_STATE.md b/.claude/board/LATEST_STATE.md index 228813310..b5d09bad5 100644 --- a/.claude/board/LATEST_STATE.md +++ b/.claude/board/LATEST_STATE.md @@ -1,3 +1,9 @@ +## 2026-10-15 — main after #1565 (merged `2379ddc2`); D-HL-5 probes on main + +- **On `main` (#1564, merged `703d6a94`):** six D-HL-5 thought-lane probes in `cognitive-shader-driver/examples` and the givens layer (hydration writes thought 0; contradicting hypotheses refused and recorded). Probes only; no production wiring. +- **On `main` (#1565):** the kanban R1 test covers all nine verdicts from `CognitiveWork`. +- **In PR:** #1566 (D-HOLO H14 lane storage, measured on v4), #1567 (D-WAL-6 version clock wired into `LanceCycleWriter`). + ## 2026-10-15 — main after #1562 (merged `31ef1802`); D-WAL on main - **On `main` (#1562):** D-WAL-1/2/3 for `LanceCycleWriter` vs the lancedb 0.40 MemWAL path (probe, harness, example behind `lancedb-sdk`); entry `2026-10-15-d-wal-shootout.md`. No production wiring of MemWAL. @@ -21,6 +27,11 @@ - **Contract:** `RbacBinding::with_class_field_defaults(Vec<(u16, WideFieldMask)>)`. A role the authority declared no field mask for gets the class default instead of every field, so the union over a user's roles can no longer open a sensitive field through a mask-less role. Policy data keyed by classid; it never changes a declared mask and is dropped outside the plug. HubSPO-rs D-IAM-3 (POC). Tests: `rbac_plug::a_class_default_replaces_the_open_default_and_nothing_else`, `authorize::plug_tests::a_class_default_keeps_sensitive_fields_out_of_the_union`; both red with the lookup removed. - **Open (closed since):** OGAR's authority now sets the default (`GrantSource::default_field_mask_of`, OGAR #358); the output stage is `walk_rails_authorized` (above). +## 2026-10-14 — main after #1553 (merged) + +- **On `main` (#1553, merged `a1abdf8f`):** D-HOLO H6b measured; H6c pre-registered. +- **In PR:** D-HOLO H6c measured (`holo_tile_crossover_probe`): crossover at g = 8–12 holds; declared construction passes the structured rule; no switch certified under the max-over-fixtures rule. + ## 2026-10-14 — main after #1552 (merged) - **On `main` (#1552, merged `98c03dd1`):** the Haiku harvest tier (six cards, supervision rules); policy only. diff --git a/.claude/board/PR_ARC_INVENTORY.md b/.claude/board/PR_ARC_INVENTORY.md index 8133ed9bb..a84bcbd06 100644 --- a/.claude/board/PR_ARC_INVENTORY.md +++ b/.claude/board/PR_ARC_INVENTORY.md @@ -1,3 +1,17 @@ +## 2026-10-15 — lance-graph PR #1565 (merged `2379ddc2`, branch `ccr-9f3b8075-74u47l-r1-test`, head `39ad8b1a`, 1 commit) — contract: the R1 test checks every verdict from `CognitiveWork` + +- **Added:** `kanban.rs` R1 test walks all nine (effect, counterfactual) verdicts from `CognitiveWork`: none reopens the goal, the one accepted verdict moves on to `Evaluation`, the other eight leave the column unchanged. +- **Locked:** moving on to `Evaluation` is not a goal revision; the goal is revised in the postactional phase. The old comment ("refused whatever the verdict") was false since #1560. +- **Deferred / open:** whether an accepted verdict should move `CognitiveWork` mid-action or wait for the action to finish. +- **Confidence:** high; disable run red (routing `NoIncrease` to `Plan`). + +## 2026-10-15 — lance-graph PR #1564 (merged `703d6a94`, branch `ccr-9f3b8075-74u47l-thought-lanes`, head `83b5cf58`, 25 commits) — D-HL-5 thought lanes: layer probes and the givens layer + +- **Added:** six in-memory probes in `cognitive-shader-driver/examples`, each pre-registered in `ce64-hypothesis-layers-v1.md` before it was written: `thought_layers_probe`, `delta_granularity_probe`, `lane_trie_probe`, `identity_session_probe`, `coverage_orientation_probe`, `givens_layer_probe`. Entries `2026-10-14-{delta-granularity-economics,lane-trie-economics,identity-sessions-morton,coverage-orientation}.md`, `2026-10-15-givens-layer.md`; STATUS D-HL-5 row. +- **Locked:** only hydration writes layer 0; a hypothesis contradicting a given is refused and recorded; bits 59..63 hydrate as one `EpistemicState5`, capped at `Related` by the proofs that ran; one read-only table per (classid, sealed version) behind a `LazyLock`. Count queries keep both per-thought masks and per-row words and route per query (1.1 ns per perspective count). v4: hydration 34 ns per row, given-checked write 73 ns per cell, merged point read 2.3 µs. +- **Deferred / open:** the merged point read (about 21 overlay entries per row) is the open cost; persistence and the H1–H6 decision rule are not run. +- **Confidence:** medium; tests written first and red against stubs, disable runs red; timings on one v4 VM. + ## 2026-10-15 — lance-graph PR #1562 (merged `31ef1802`, branch `ccr-9f3b8075-74u47l-wal-shootout`, head `e38779f5`, 19 commits) — D-WAL: LanceCycleWriter vs MemWAL measured; capped session cache, version clock, copy-free temporal projections - **Added:** plan `wal-shootout-v1.md`; probe `tests/wal_shootout_probe.rs` (V1–V4, behind `lancedb-sdk`), harness `tests/wal_common/`, examples `wal_shootout` and `wal_rss_probe`; `LanceCycleWriter` session capped at `SESSION_METADATA_CACHE_BYTES` (8 MiB) on create and open; contract `temporal_pov::{VersionClock, ClockError}`; `temporal.rs` index forms (`deinterlace_indices`, `local_trajectories_indices`, `local_trajectory_indices_of`); entry `2026-10-15-d-wal-shootout.md`; STATUS rows D-WAL-1..7, D-LNC-7. @@ -12,6 +26,13 @@ - **Deferred / open:** D-MTP-2..5 (named `LanceSpace`, recursion via the external driver, the trie width fix upstream, PeTTa in the probes); the order witness is probe-local. - **Confidence:** high for equality (disable runs red as pre-registered); timings are best of 5 over three runs on one VM. +## 2026-10-14 — lance-graph PR #1553 (merged `a1abdf8f`, branch `ccr-9f3b8075-74u47l-holo-h6b`, head `c25573e3`, 3 commits) — D-HOLO H6b measured; H6c pre-registered + +- **Added:** example `holo_tile_forms_probe` (P / I / D / H on the 256 × 256 Context × Operation tile, 14 patterns incl. the doughnut, row-oracle tests); H6b results; H6c pre-registration; board entry `2026-10-14-d-holo-h6b.md`; the #1552 arc entries. +- **Locked:** H6b structured rule NOT MET (the 65,536-cell build dominates at N = 100; P 0.21–0.73× D per query); the hybrid's byte switch refuted by sparse and dense (1.8–4.8× D); P's cost follows its group count. +- **Deferred / open:** the terminal selector stays held; H6c measures the group switch, the crossover and declared construction. +- **Confidence:** medium; three disable runs red, three timing runs, ~40 % VM noise. + ## 2026-10-14 — lance-graph PR #1552 (merged `98c03dd1`, branch `ccr-9f3b8075-74u47l-haiku-harvest`) — Haiku harvest tier: six cards and their supervision rules - **Added:** cards `.claude/agents/{harvest-scout,census-counter,quote-harvester,substrate-mapper,spec-editor,guarded-executor}.md`; `tiered-agent-execution-protocol.md` § Haiku harvest tier (card table, brief sizing, supervision rules, mapping a large surface); CLAUDE.md Model Policy rewritten; BOOT files, worker template, cca2a and v3 BOOT aligned; AGENT_LOG entries for the first runs; #1551 board entries. diff --git a/.claude/board/STATUS_BOARD.md b/.claude/board/STATUS_BOARD.md index 36ba8ec58..f9bc36199 100644 --- a/.claude/board/STATUS_BOARD.md +++ b/.claude/board/STATUS_BOARD.md @@ -37,11 +37,12 @@ Plan: `.claude/plans/relalg-v1.md`. Crate: `crates/lance-graph-relalg` (own work | **D-HOLO-H5** | Coordinate-native vs coordinate-crossing: prefix count, perspectives, reverse `memberOf` with inverse build/update cost | Measured 2026-10-14, corrected after review: Q1 CONFIRMED at N = 100 (31–37×), FAILS at N = 1 (1.3–1.4×, the occupancy build dominates); Q2 passes only with row words stored; plane inverse refuted on Q3b (break-even > 1,000) | == row oracle; Range ≥ 5× sweep; break-even N reported; permuted-tile run disagrees | | **D-HOLO-H6** | Closure of address restrictions as interval sets vs masks vs value sweep | Measured 2026-10-14: HOLDS; symbolic cheapest in all 24 cells | == row oracle; symbolic cost flat in area, linear in interval count | | **D-HOLO-H6b** | Product normal form vs Morton intervals vs dense mask vs hybrid on the Context × Operation reading of the `u8:u8` tile; construction, normalization, ∩/∪/¬, emptiness, equality, bytes, reuse at N = 1..1,000 | Measured 2026-10-14: structured rule NOT MET (build dominates at N = 100; P 0.21–0.73× D per query); hybrid passes checkerboard and random 50 %, fragmented 1.19–1.33; byte switch refuted by sparse/dense (H 1.8–4.8× D); group count, not bytes or density, tracks P's cost | == row oracle; structured patterns ≤ 0.5× dense at N = 100; hybrid ≤ 1.25× dense on adversarial patterns; component count vs density as predictor reported | -| **D-HOLO-H6c** | Group-count crossover (g = 1..64, blocked vs interleaved), a group-count switch for the hybrid, and construction from a declaration with the same declaration for every arm | Queued (pre-registered, not run) | == row oracle; P/D per query crosses 1 between g = 4 and 16 in both layouts; H' ≤ 1.25× D everywhere and never ≥ 2×; declared structured rule P or H ≤ 0.5× D at N = 100 | +| **D-HOLO-H6c** | Group-count crossover (g = 1..64, blocked vs interleaved), a group-count switch for the hybrid, and construction from a declaration with the same declaration for every arm | Measured 2026-10-14: Q1 HOLDS (crossover g = 8–12, both layouts); Q2 not shown (G = 8 worst query 1.21–2.28× D vs byte switch 9.8–15.1×; max-over-fixtures rule noise-bound); Q3 PASS (declared build: P or H 0.32–0.42× D at N = 100) | == row oracle; P/D per query crosses 1 between g = 4 and 16 in both layouts; H' ≤ 1.25× D everywhere and never ≥ 2×; declared structured rule P or H ≤ 0.5× D at N = 100 | | **D-HOLO-H8** | One formula in three domains: logical algebra, physical realization and semantic leakage compared separately | Queued (pre-registered, not run) | no semantic leakage; families grow only with representation and value algebra | | **D-HOLO-H9** | Relational emptiness decided symbolically (empty operand, disjoint domains, contradictory constraints) vs by fold | Queued (pre-registered, not run) | zero false empties; fraction decided symbolically reported | | **D-HOLO-H11** | Stacked logic: a chain of NAL polyfill bricks at depth 1–8 composed before one evaluation | Queued (pre-registered, not run; after H6b) | answers no single brick gives, == oracle; cost below d separate runs at d = 4; no confidence inflation, gate disable must inflate | | **D-HOLO-H12** | Semantic map of inference functions across OpenNARS, ONA, the NAL polyfill (and a MeTTa NAL if pinnable) | Queued (pre-registered, not run) | last system adds zero nodes in both orders; polyfill gaps named; differences classified as distinction or defect | +| **D-HOLO-H14** | Per classid x thought x 64k tile: full 512-byte row writes vs a connected lane delta (row key + lane mask + changed lanes in one record), and per-(thought, lane) tables indexed directly by the row address (`Lane_l[index]`, no traversal; presence masks folded, materialization charged only at a terminal); session thought masks and thought lane masks kept as separate levels | Measured 2026-10-15 (v4, one class): C folds 10–256× faster than B on F1/F2; C merge-on-read 2.8–6.1× slower than B; C default and B default both not met; returned to the operator with the crossover; H14b (positional per-row lane mask for C) open | == dense reference; C default iff at k <= 4 its fold queries are >= 2x faster than B, its reads with terminal materialization <= 1.25x B and bytes <= B; else B default iff read and session read <= 1.25x A and bytes <= 0.5x A; mixed result returns to the operator | | **D-RA-1** | `Rel` + `normalize` + `symbolic` + R2IL lowering | Shipped #1546 `c21bd165` (11 tests, 11 disable runs red; four comparisons on one lane 8 → 2 ops; 70/2,000 small descriptions refused as written, all run rewritten) | == row oracle over 400 generated descriptions × 5 measures | | **D-RA-2** | Rules into the contract set IR (D-CMR-1/-6); relalg keeps only lowering or is deleted | Queued (maintainer decision) | relalg tests pass unchanged against the contract IR | | **D-RA-3** | Datalog rules as algebra values; names are labels resolved through the label codebook, never strings in a rule | Queued (after D-RA-2) | codebook-built rules == hand-written `Rel`; unknown label refused at resolution | diff --git a/.claude/board/entries/2026-10-14-d-holo-h6c.md b/.claude/board/entries/2026-10-14-d-holo-h6c.md new file mode 100644 index 000000000..f2d77b3dc --- /dev/null +++ b/.claude/board/entries/2026-10-14-d-holo-h6c.md @@ -0,0 +1,17 @@ +# 2026-10-14 — D-HOLO-H6c — group-count crossover, group switch, declared construction (measured) + +**Status:** MEASURED (v4, avx512f, three runs). Plan `.claude/plans/d-holo-address-indexed-algebra-v1.md`, +§ Results: H6c. Probe: `crates/lance-graph-mask-risc/examples/holo_tile_crossover_probe.rs`. + +**Outcome:** +- Q1 HOLDS: P/D per query crosses 1 between g = 8 and 12 (once 12 and 16), in both layouts; P is + 0.14–0.49× D up to 4 groups and 9.4–12.4× D at 64. +- Q2 not shown: the group switch at G = 8 is far better than H6b's byte switch (worst query + 1.21–2.28× vs 9.8–15.1× D), but the registered max-over-38-fixtures rule gives undecided / fail / + pass across the three runs; the worst fixture changes every run. +- Q3 PASS: built from a declaration (0.3–0.7 µs, vs ~0.3 ms for the cell sweep), P or H is + 0.32–0.42× D at N = 100 on row, column, rectangle and prefix in every run; P is cheaper from + the first query. + +**Open:** the selector stays held. A noise-robust criterion for Q2 (per-fixture median over runs), +and whether mid-query conversion to D should be allowed (the doughnut at G = 2 pays 1.8–1.9× for it). diff --git a/.claude/board/entries/2026-10-15-d-holo-h14-lane-storage.md b/.claude/board/entries/2026-10-15-d-holo-h14-lane-storage.md new file mode 100644 index 000000000..916ee5e9e --- /dev/null +++ b/.claude/board/entries/2026-10-15-d-holo-h14-lane-storage.md @@ -0,0 +1,18 @@ +# D-HOLO-H14 — full row vs connected delta vs inverted fold (v4, 2026-10-15) + +**Status:** MEASURED. Plan: `.claude/plans/d-holo-address-indexed-algebra-v1.md` (H14, two amendments, results). +**Probe:** `crates/lance-graph-mask-risc/examples/holo_lane_storage_probe.rs`; `tier: v4 (avx512f)`; one class timed. + +- C (per-lane tables indexed by the row address, presence masks folded) wins + every counting query: F1 10–24× faster than B, F2 39–256× (0.017–0.59 ns + per aperture row against B's 0.86–109). +- C loses every merge-on-read: full-row read 259–1,529 ns against B's + 51–1,005 and A's 38–152; the merge probes every touched lane table. +- Bytes handed to the writer: C ≈ B (0.7–1.2×), both far below A at low k. +- Rule: C default not met (reads), B default over A not met (B's reads at + 100 % dirty up to 4.6× A at k = 4). Returns to the operator; crossover stated in the + plan. + +**OPEN:** H14b — C plus a positional per-row lane mask, so merge-on-read +reads only the k changed tables; 16 classes untimed; C's seal at low dirt is +page-fault bound. diff --git a/.claude/board/entries/README.md b/.claude/board/entries/README.md index 1640b6b30..0fc798c6a 100644 --- a/.claude/board/entries/README.md +++ b/.claude/board/entries/README.md @@ -25,13 +25,14 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately opposite directions; the stranding this convention prevents shows up in exactly one of them, never both. -341 entries, 2026-08-06 .. 2026-10-15. +343 entries, 2026-08-06 .. 2026-10-15. | date | entry id | finding | file | |---|---|---|---| | 2026-10-15 | `ledger-correction-wire-contract` | | [2026-10-15-ledger-correction-wire-contract.md](2026-10-15-ledger-correction-wire-contract.md) | | 2026-10-15 | `givens-layer` | | [2026-10-15-givens-layer.md](2026-10-15-givens-layer.md) | | 2026-10-15 | `D-WAL` | `LanceCycleWriter` vs the lancedb 0.40 MemWAL write path (measured) | [2026-10-15-d-wal-shootout.md](2026-10-15-d-wal-shootout.md) | +| 2026-10-15 | `D-HOLO-H14` | | [2026-10-15-d-holo-h14-lane-storage.md](2026-10-15-d-holo-h14-lane-storage.md) | | 2026-10-14 | `revision-layer-invariants` | | [2026-10-14-revision-layer-invariants.md](2026-10-14-revision-layer-invariants.md) | | 2026-10-14 | `D-MTP-0` | where Hyperon/MeTTa attaches to the masking-algebra polyfill | [2026-10-14-metta-polyfill-map.md](2026-10-14-metta-polyfill-map.md) | | 2026-10-14 | `ledger-revision-after-1553` | | [2026-10-14-ledger-revision-after-1553.md](2026-10-14-ledger-revision-after-1553.md) | @@ -41,6 +42,7 @@ exactly one of them, never both. | 2026-10-14 | `delta-granularity-economics` | | [2026-10-14-delta-granularity-economics.md](2026-10-14-delta-granularity-economics.md) | | 2026-10-14 | `D-MTP-1b` | the hop as a range (measured) | [2026-10-14-d-mtp-1b-range-hop.md](2026-10-14-d-mtp-1b-range-hop.md) | | 2026-10-14 | `D-MTP-1` | the MeTTa two-hop join through Quack Semijoin (measured) | [2026-10-14-d-mtp-1-join.md](2026-10-14-d-mtp-1-join.md) | +| 2026-10-14 | `D-HOLO-H6c` | group-count crossover, group switch, declared construction (measured) | [2026-10-14-d-holo-h6c.md](2026-10-14-d-holo-h6c.md) | | 2026-10-14 | `D-HOLO-H6b` | normal forms on the Context × Operation tile (measured) | [2026-10-14-d-holo-h6b.md](2026-10-14-d-holo-h6b.md) | | 2026-10-14 | `D-HOLO` | | [2026-10-14-d-holo-falsifiers.md](2026-10-14-d-holo-falsifiers.md) | | 2026-10-14 | `coverage-orientation` | | [2026-10-14-coverage-orientation.md](2026-10-14-coverage-orientation.md) | diff --git a/.claude/plans/d-holo-address-indexed-algebra-v1.md b/.claude/plans/d-holo-address-indexed-algebra-v1.md index 5262565e6..7ced213d8 100644 --- a/.claude/plans/d-holo-address-indexed-algebra-v1.md +++ b/.claude/plans/d-holo-address-indexed-algebra-v1.md @@ -1,6 +1,6 @@ # D-HOLO — address-indexed algebra: falsifiers H1, H5, H6 (v1) -**Status:** MEASURED 2026-10-14 (H1, H5, H6, H6b); H6c, H8, H9, H11, H12 +**Status:** MEASURED 2026-10-14 (H1, H5, H6, H6b, H6c); H8, H9, H11, H12, H14 PRE-REGISTERED, not run. The pre-registration below is unchanged; results are in § Results (2026-10-14), the next falsifiers in § Next falsifiers. @@ -35,11 +35,12 @@ is run. - [x] D-HOLO-H6: closure of address restrictions probe, same - [x] Results section, board entry, status rows - [x] D-HOLO-H6b: normal forms on the Context × Operation tile (with the connectedness column) -- [ ] D-HOLO-H6c: group-count switch, the 2–60 group crossover, declared construction +- [x] D-HOLO-H6c: group-count switch, the 2–60 group crossover, declared construction - [ ] D-HOLO-H8: semantic diversity does not add instruction families - [ ] D-HOLO-H9: relational emptiness without a fold - [ ] D-HOLO-H11: stacked logic (pre-registered after H6b; runs after it) - [ ] D-HOLO-H12: semantic map of inference functions across NARS systems +- [ ] D-HOLO-H14: full-row writes vs lane deltas that stay in their row (pre-registered 2026-10-15); amended 2026-10-15: arm C is the inverted fold and a candidate ## Accounting rules (apply to every falsifier) @@ -795,3 +796,290 @@ because D is swept while P is declared does not count. - Every Q1 fixture really has g groups (P's group count equals g) and is neither empty nor full. Disable: assigning every context the first mask must fail it. +## Results: H6c (2026-10-14) + +Probe `crates/lance-graph-mask-risc/examples/holo_tile_crossover_probe.rs`, v4 +(avx512f), three runs. The arms, query and patterns are copied from the H6b +probe; the hybrid gains a switch policy. Five tests (declared == swept build in +P, D and I; Q1 fixtures have exactly g groups; the group switch goes both ways; +every arm and switch matches the row oracle on the 14 H6b patterns and 24 Q1 +fixtures; a rectangle's Morton decomposition is exact). The three pre-registered +disables each turned a test red: an off-by-one in the declared range end, the +switch ignoring G, every context given the first mask. + +| Question | Result, three runs | Verdict | +|---|---|---| +| Q1: P/D per query crosses 1 between g = 4 and 16 in both layouts | blocked: 8 → 12 in all runs; interleaved: 8 → 12 twice, 12 → 16 once | HOLDS; layouts agree within one step | +| Q2: a group switch ≤ 1.25× D everywhere, never ≥ 2× (G nearest the crossover: 8) | G = 8: max query 1.68 / 2.28 / 1.21; max total 1.88 / 1.72 / 1.20 | NOT shown safe (undecided, fails, passes) | +| Q2: the byte switch | max query 9.8–15.1× D | NOT safe in every run | +| Q3: declared build, P or H ≤ 0.5× D at N = 100 on row, column, rectangle, prefix | P 0.32–0.76, H 0.38–0.83; the better of the two 0.32–0.42 | PASS in all runs | + +**Q1.** P's query cost grows with the group count; D's does not. P is 0.14–0.49× +D up to g = 4, 0.40–0.63× at g = 6–8, and 0.92–1.86× at g = 12–16. At g = 64 it is +9.4–12.4× D. Single operations sit above the full query: at g = 12, ∩ and ∪ are +1.29–1.94× D, and ¬ already crosses 1 at g = 6–8 in the interleaved layout +(1.03–1.38×). + +**Q2.** The group switches at G = 2–16 beat the byte switch by a wide margin +(worst query 1.21–2.88× against 9.8–15.1×); G = 32 keeps P past the crossover +and reaches 3.1–7.3×. But the maximum over 38 fixtures moves from run to +run (worst fixture: doughnut, fragmented, g = 12 blocked, g = 48 interleaved), +which is what a maximum over noisy ratios does at about 40 % noise. The rule as +registered cannot certify any switch here. One effect repeats in two runs: at +G = 2 the doughnut (2 groups) costs 1.8–1.9× D per query, because its +intermediate results exceed 2 groups and are converted to D mid-query. + +**Q3.** Built from a declaration, P costs about 0.3–0.7 µs to build, against +about 0.3 ms for the 65,536-cell sweep (H6b), and P's total is below D's from +the first query (N ≤ 1) on every declared pattern. I's declared build (Morton +decomposition of the rectangles, 2–10 µs) makes I the most expensive arm +except on the prefix. The "swept D" column in the probe's output builds through +the declaration's membership function, which re-collects the rectangles on +every cell; it is not comparable with H6b's sweep and is not used in any ratio. + +**What this changes.** The H6b structured rule fails only because of the sweep: +with declared construction it passes in every run. P is the cheaper form up to +about 8 groups and loses beyond about 12. A group-count switch near G = 8 fixes +H6b's byte-switch failure, but the registered max-over-fixtures criterion is +too noise-sensitive to certify it at 14 + 24 fixtures. The selector stays held. +Open: a noise-robust criterion (per-fixture median over runs, or a repeated +maximum) and whether switching intermediate results mid-query should be +allowed at all. + + +## Next falsifier: H14 — full-row writes vs lane deltas that stay in their row (PRE-REGISTERED 2026-10-15, NOT RUN) + +**Question (operator, 2026-10-15).** For one graph (classid) x thought (rung) x +64k-row tile, is a dirty row written as (a) the full 512-byte SoA row, or (b) a +32-lane index plus the changed 16-byte lanes? D-HL-5 measured the in-memory +tile (slot delta 12 + 16k bytes per dirty row, 0.054 of a row at k = 1); this +falsifier owns the persistence choice. Written before the probe exists; a +change after the run is a new dated section. + +**DECISION (operator, 2026-10-15).** Two levels of awareness stay distinct and +are never folded into each other: +- a **session** holds several thoughts: one thought mask per session; +- a **thought** holds several lanes: one lane mask per (thought, row). +A lane is never stored apart from its row. Either the whole row is written, or +the delta carries its row key and lane mask in the same record, so a read never +has to re-join lanes to rows. Storage that splits lanes from their rows creates +a dissociation that a later read must re-merge, and that re-merge is a +materialization cost. SCOPE: H14. REVISIT WHEN: an arm without the connection +wins on read and bytes together. + +**Arms.** +- A (full row): per dirty (thought, row), the 512-byte row image. +- B (connected delta): per dirty (thought, row), one record = row key, lane + mask (u32), the changed lanes packed in lane order. Lanes exist only inside + their record. +- C (dissociated, control): one table per (thought, lane), keyed by row. It is + kept only to measure the re-merge it forces; it cannot become the default + under the decision above. +All three hold the same session thought masks; the session level is not part +of any arm's lane storage. + +**Grid.** Classes 1 and 16; thoughts per session 1, 8, 64; 65,536 rows per +tile; dirty fraction 1 %, 10 %, 100 %; changed lanes per dirty row k = 1, 4, +16, 31. Fixtures deterministic (seeded), identical for every arm. + +**Metrics.** Bytes stored and bytes one append would write (Arrow batch size, +computed exactly); seal ns per dirty row; full-row read ns per row (base merged +with the thought's record); session read ns per row (every thought of one +session over a 4,096-row aperture); for C, the re-merge ns per row reported on +its own. v4 only, tier printed by the program; ns per row or per record, never +whole-call milliseconds. The Lance write itself is not timed here (D-WAL owns +it); this falsifier reports what each arm would hand to it. + +**Rule.** B is the default if, at k <= 4 on every (dirty, thoughts) cell, B's +full-row read is <= 1.25x A's, its session read <= 1.25x A's, and its bytes <= +0.5x A's. Otherwise A is the default. C is reported, never chosen. If B passes +only at some k, the crossover k* is reported and the decision returns to the +operator. + +**Tests and disable runs (before any timing).** +- Every arm reads the same bytes as a dense reference tile, row by row + (disable: drop the merge in B's read). +- The session mask names exactly the thoughts that wrote in the session + (disable: derive it from the lane masks of one thought). +- A thought's lane mask equals the lanes that differ from the base at that row + (disable: store every lane). +- B never yields a lane without its row key; C's re-merge returns the same rows + as A (disable: skip a lane table in the re-merge). + +### H14 amendment before any run (2026-10-15): arm C is the inverted fold, and a candidate + +**Correction (operator, 2026-10-15).** "If you use c) with the inverted +holographic addresses you just make the fold the default and computation +happens inside the inverted fold, only pay for materialisierung." The text +above treated C as a dissociated control that "cannot become the default" and +is "reported, never chosen". Both statements are SUPERSEDED. Nothing has been +run, so this is a pre-registration change, not a result change. + +**Why C does not dissociate.** The decision above forbids storage that a read +must re-join to its rows. Per-lane planes over the tile's own row addresses +need no re-join: every plane lives in the same 65,536-row coordinate space, so +combining lanes, thoughts and the aperture is an address-aligned mask fold +(`And` / `Or` / `Count` in mask-risc), not a join. The row connection is +carried by the address instead of by co-locating lanes in one record. Lanes +are assembled into a row only when a terminal asks for a row, and that +assembly is the one materialization C pays. + +**C (inverted fold), replacing the control.** Per (thought, lane) one plane: +- the set of rows whose lane differs from the base, as an address mask + (carrier chosen by density: dense bitmap, sparse list or runs; the probe + prints which carrier each cell used, since a dense 8 KiB plane per + (thought, lane) is 64 x 31 x 8 KiB = 15.9 MB per tile and would lose on + bytes at 1 % dirty); +- the changed 16-byte values for those rows, in address order. +Queries stay inside the fold until a named terminal. The session thought mask +is unchanged and still not part of any arm's lane storage. + +**Relation to H5.** H5 refuted a maintained inverse built in addition to the +primary store (the plane inverse did not pay back its build within 1,000 +queries). Here the inverse planes are the primary store: there is no second +structure to build, only the seal cost of writing into planes, which is +measured below. If C's seal cost per dirty row exceeds B's by more than the +H5 plane build ratio, that is reported as the same failure, not hidden. + +**Added queries (all arms, same fixtures).** Each answered without building a +row, except where a terminal is named: +- F1: rows thought t changed in any lane of a lane set L, inside an aperture, + as a count; +- F2: rows changed by any thought of the session in lane l, inside an + aperture, as a count; +- F3: F1's rows, materialized as full 512-byte rows (terminal). +A and B answer F1 and F2 by scanning their records; C by folding planes. + +**Metrics added.** For C, fold ns per row of the aperture (F1, F2) and +terminal materialization ns per emitted row (F3, and the full-row and session +reads), reported separately so the materialization cost is visible on its +own. Bytes as before, per carrier. + +**Revised rule (replaces "C is reported, never chosen").** +- C is the default if, at k <= 4 on every (dirty, thoughts) cell: F1 and F2 + on C are at least 2x faster than on B; the full-row read and session read on + C, including terminal materialization, are <= 1.25x B's; and C's bytes are + <= B's. +- Otherwise the A-vs-B rule above decides between A and B. +- If C wins the fold queries but loses the materializing reads, both numbers + are reported, the crossover (how many rows a query materializes before C + stops winning) is stated, and the decision returns to the operator. + +**Tests and disable runs added (before any timing).** +- F1, F2 and F3 return the same rows and bytes on every arm as on the dense + reference (disable: drop one lane plane from C's fold). +- C's materialization counter is zero after F1 and F2 and equals the emitted + row count after F3 (disable: assemble rows inside the fold). +- C's planes name exactly the rows whose lane differs from the base + (disable: store every row of a touched lane). + +### H14 correction to the amendment (2026-10-15): the address is the register, nothing is traversed + +**Correction (operator, 2026-10-15).** "You don't traverse, the address is +already the loci register. Instead of `Table1[index][a][b]`, +`Table1[index][a] Table2[index][b]`." The amendment above stored C's changed +values "in address order", compacted. Finding a row's value in a compacted run +needs a rank or a search over the plane, which is a traversal. That part is +SUPERSEDED. + +**C, as corrected.** One table per (thought, lane), indexed by the row address +itself: `Lane_l[index]` is the 16-byte value of lane l at that row. The +address is the position; reading lanes a and b of row i is two positional +reads at the same i, with no search, no rank and no join. The presence mask +per (thought, lane) stays, and it is what F1 and F2 fold over; it is never +used to locate a value. + +Contrast with B: B's record is `Table[index][a][b]`, one row key with its lanes +packed behind a lane mask, so reading lane b means walking the record (a +popcount rank over the lane mask). The probe reports that rank as part of B's +read cost. + +**Bytes.** Logically every (thought, lane) table spans the tile, which is +65,536 x 16 B = 1 MiB. Two rules keep that from becoming the stored size: +- only (thought, lane) pairs that changed at least one row get a table; + an absent table falls through to the base row (the zero-fallback); +- the bytes handed to the writer are the dirty positions only (Lance sparse + write, merge on read), reported per cell next to the resident size. +The resident size is reported on its own so the cost of positional tables in +memory is visible, not hidden behind the persisted size. + +**Test added.** Reading lane l of row i on C touches exactly position i of one +table: an access counter equals the number of lanes read, independent of how +many rows the table holds (disable: store C's values compacted and look them +up by rank, which makes the counter grow with the plane). + +**Addendum (operator, 2026-10-15): "it's merge on read but for the join."** +Combining lane tables is not a separate join step. It is the same +merge-on-read that overlays dirty bytes on the base: for row i, start from the +base row and overlay position i of each touched (thought, lane) table. Base +over delta and lane next to lane are one operation at one index. So: +- the original metric "for C, the re-merge ns per row reported on its own" is + SUPERSEDED; C's full-row read is measured as one merge-on-read and reported + with the same metric as A and B; +- "terminal materialization" for C means exactly this merge-on-read, run only + for the rows a terminal asks for; F1 and F2 never run it; +- the original DECISION's concern (a dissociation that a later read must + re-merge at extra cost) does not apply to positional tables, because the + merge is the read itself. It still applies to any layout that would need a + key lookup or a sort to line lanes up, which is why compacted runs stay + superseded. + +### H14 results (v4, 2026-10-15): C wins every fold, loses every merge-on-read; returns to the operator + +Run: `cargo run --release -p lance-graph-mask-risc --example +holo_lane_storage_probe` on an AVX-512 host (the program printed +`tier: v4 (avx512f)`). One class per run; 16 classes were not timed (the +64-thought cells need about 8 GB for one class on a 15 GB host). Bytes scale +linearly with classes; the timing effect of more classes is untested. +Seven oracle tests pass against a dense reference; five disable runs each +turned their test red (drop B's merge, store every lane in B, skip a lane +plane in C's fold, build rows inside C's fold, session mask from one thought). + +Measured, ns per row unless noted (full table in the board entry): + +| quantity | A full row | B connected delta | C inverted fold | +|---|---|---|---| +| F1 (one thought, 4 lanes, per aperture row) | 0.5–38.8 | 0.43–1.6 | 0.027–0.075 | +| F2 (all thoughts, one lane, per aperture row) | 1.2–272 | 0.86–109 | 0.017–0.59 | +| full-row read (merge-on-read for C) | 38–152 | 51–1,005 | 259–1,529 | +| session read (per thought-row) | 6.7–61 | 11.5–173 | 57–539 | +| seal per dirty row, 1 % dirty | 31–292 | 13–30 | 2,454–21,106 | +| seal per dirty row, 100 % dirty | 271–389 | 5–59 | 32–222 | +| bytes handed to the writer | 516 B per dirty row | 12 + 16k | ≈ B (0.7–1.2× B) | +| resident | as written | as written | 31 MiB per thought, logical | + +Rule at k <= 4 on every (dirty, thoughts) cell: +- C's folds are at least 2× faster than B's on F1 and F2: **yes**, in every + cell (F1 10–24×, F2 39–256×). F2 is the session-wide count and + grows with the number of thoughts on all three arms: C ORs one presence + window per thought, 0.017–0.023 ns per aperture row at 1 thought, + 0.058–0.072 at 8, 0.38–0.59 at 64. +- C's reads with merge-on-read within 1.25× of B's: **no**, in every cell + (2.8–6.1× slower). +- C default: not met. B default over A: not met (B's full-row read is up to + 4.6× A's at 100 % dirty with k = 4, and its session read 1.4–2.2× A's at + 1 % dirty). + +Per the pre-registered rule this returns to the operator with both numbers. +The crossover, from F3 (fold, then materialize the rows the fold names): +C beats B on F3 at 1 % dirty in every cell (C 59–136 vs B 87–690 ns per +emitted row) and at 10 % dirty when k = 1; it loses in most other cells. A +query on C pays about 0.4 ns per aperture row less than B for F1 and about +200–450 ns more per materialized row, so C wins a mixed query while it +materializes fewer than about 1 row in 500–1,000 of the aperture for a +single-thought count, and between about 1 in 8 and 1 in 3 for a +session-wide count at 64 thoughts. + +What the numbers point at (HYPOTHESIS, not measured here): +- C's read cost is the merge itself, not the positional access: the merge + probes every touched lane table of the thought (31 tables at every k in + this fixture, because the changed lanes vary by row), and each probe is a + separate 1 MiB table, so a row read touches up to 31 cache lines in 31 + tables. A per-row lane mask held positionally beside the tables + (`Mask[i]`, 4 bytes per row) would let the merge read only the k tables + that changed, keep the fold, and keep the address as the register. That is + B's lane mask without B's packed record, and it is the next arm to try + (H14b). +- C's seal at 1 % dirty is page faults: the positional tables are zeroed + lazily, so each scattered dirty row touches a fresh page. At 100 % dirty + the same seal costs 32–222 ns. Persisted bytes are unaffected. diff --git a/crates/lance-graph-mask-risc/Cargo.toml b/crates/lance-graph-mask-risc/Cargo.toml index 703921e15..1abcd91b5 100644 --- a/crates/lance-graph-mask-risc/Cargo.toml +++ b/crates/lance-graph-mask-risc/Cargo.toml @@ -43,3 +43,11 @@ test = true [[example]] name = "holo_tile_forms_probe" test = true + +[[example]] +name = "holo_tile_crossover_probe" +test = true + +[[example]] +name = "holo_lane_storage_probe" +test = true diff --git a/crates/lance-graph-mask-risc/examples/holo_lane_storage_probe.rs b/crates/lance-graph-mask-risc/examples/holo_lane_storage_probe.rs new file mode 100644 index 000000000..87bfbbf11 --- /dev/null +++ b/crates/lance-graph-mask-risc/examples/holo_lane_storage_probe.rs @@ -0,0 +1,758 @@ +//! D-HOLO H14: how a dirty row of one (classid x thought x 64k tile) is held. +//! +//! Pre-registered in `.claude/plans/d-holo-address-indexed-algebra-v1.md` +//! ("Next falsifier: H14", with the two 2026-10-15 amendments). Arms: +//! - A, full row: per dirty (thought, row) the 512-byte row image. +//! - B, connected delta: per dirty (thought, row) one record: row key, lane +//! mask, the changed 16-byte lanes packed in lane order (`Table[i][a][b]`). +//! - C, inverted fold: per (thought, lane) one table indexed directly by the +//! row address (`Lane_l[i]`) plus a presence mask. Lanes of row i are read +//! at position i of each table; counting queries fold presence masks. +//! +//! The session level (which thoughts wrote) is one u64 mask, shared by all +//! arms and not part of any arm's lane storage. +//! +//! Run (release, the tier is printed): `cargo run --release --example +//! holo_lane_storage_probe`. Tests: `cargo test --release --example +//! holo_lane_storage_probe`. +#![allow(dead_code)] + +use std::hint::black_box; +use std::time::Instant; + +use ndarray::simd::mask_or; + +const N: usize = 65_536; +const WORDS: usize = N / 64; +const ROW: usize = 512; +const LANE: usize = 16; +const LANES: usize = 32; +const NONE: u32 = u32::MAX; +const APERTURE: usize = 4_096; +/// The lane set of F1. +const F1_LANES: [usize; 4] = [1, 8, 15, 22]; + +type Row = [u8; ROW]; +type Val = [u8; LANE]; + +fn tier() -> &'static str { + if cfg!(target_feature = "avx512f") { + "v4 (avx512f)" + } else if cfg!(target_feature = "avx2") { + "v3 (avx2): timings not reportable" + } else { + "below v3: timings not reportable" + } +} + +fn mix(mut z: u64) -> u64 { + z = z.wrapping_add(0x9E37_79B9_7F4A_7C15); + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) +} + +fn lane_bytes(row: &Row, l: usize) -> &Val { + row[l * LANE..(l + 1) * LANE].try_into().unwrap() +} + +fn base() -> Vec { + (0..N) + .map(|r| { + let mut row = [0u8; ROW]; + for (w, c) in row.as_chunks_mut::<8>().0.iter_mut().enumerate() { + *c = mix((r * 64 + w) as u64).to_le_bytes(); + } + row + }) + .collect() +} + +/// One thought's writes: dirty rows ascending, each with its lane mask and +/// the new values in lane order. +struct Writes { + rows: Vec, + masks: Vec, + vals: Vec, +} + +/// The j-th lane a dirty row writes: distinct for j < 31, never lane 0 (key). +fn lane_of(t: usize, r: usize, j: usize) -> usize { + 1 + (r * 7 + t * 13 + j * 5) % (LANES - 1) +} + +fn writes(base: &[Row], t: usize, per_10k: u64, k: usize) -> Writes { + let mut w = Writes { + rows: Vec::new(), + masks: Vec::new(), + vals: Vec::new(), + }; + for (r, b) in base.iter().enumerate() { + if mix(((t as u64) << 32) ^ r as u64 ^ 0xA5A5) % 10_000 >= per_10k { + continue; + } + let mut mask = 0u32; + for j in 0..k { + mask |= 1 << lane_of(t, r, j); + } + w.rows.push(r as u32); + w.masks.push(mask); + for l in BitIter(mask) { + let x = mix(((t as u64) << 40) ^ ((r as u64) << 8) ^ l as u64); + let mut v = [0u8; LANE]; + v[..8].copy_from_slice(&x.to_le_bytes()); + v[8..].copy_from_slice(&x.rotate_left(17).to_le_bytes()); + if &v == lane_bytes(b, l) { + v[0] ^= 1; + } + w.vals.push(v); + } + } + w +} + +struct BitIter(u32); +impl Iterator for BitIter { + type Item = usize; + fn next(&mut self) -> Option { + if self.0 == 0 { + return None; + } + let t = self.0.trailing_zeros() as usize; + self.0 &= self.0 - 1; + Some(t) + } +} + +/// Which thoughts of a session wrote anything: one u64. +fn session_mask(all: &[Writes]) -> u64 { + all.iter() + .enumerate() + .filter(|(_, w)| !w.rows.is_empty()) + .fold(0, |m, (t, _)| m | 1 << t) +} + +// ---------------------------------------------------------------- A + +struct ArmA { + index: Vec, + images: Vec, +} + +impl ArmA { + fn seal(base: &[Row], w: &Writes) -> Self { + let mut a = ArmA { + index: vec![NONE; N], + images: Vec::with_capacity(w.rows.len()), + }; + let mut v = 0; + for (&r, &m) in w.rows.iter().zip(&w.masks) { + let mut img = base[r as usize]; + for l in BitIter(m) { + img[l * LANE..(l + 1) * LANE].copy_from_slice(&w.vals[v]); + v += 1; + } + a.index[r as usize] = a.images.len() as u32; + a.images.push(img); + } + a + } + fn bytes(&self) -> usize { + self.images.len() * (4 + ROW) + } + fn read(&self, base: &[Row], r: usize) -> Row { + match self.index[r] { + NONE => base[r], + i => self.images[i as usize], + } + } + /// A holds no lane mask: a changed lane is found by comparing with base. + fn changed(&self, base: &[Row], r: usize, l: usize) -> bool { + match self.index[r] { + NONE => false, + i => lane_bytes(&self.images[i as usize], l) != lane_bytes(&base[r], l), + } + } +} + +// ---------------------------------------------------------------- B + +struct ArmB { + index: Vec, + rows: Vec, + masks: Vec, + starts: Vec, + words: Vec, +} + +impl ArmB { + fn seal(w: &Writes) -> Self { + let mut b = ArmB { + index: vec![NONE; N], + rows: Vec::with_capacity(w.rows.len()), + masks: Vec::with_capacity(w.rows.len()), + starts: Vec::with_capacity(w.rows.len()), + words: Vec::with_capacity(w.vals.len()), + }; + let mut v = 0; + for (&r, &m) in w.rows.iter().zip(&w.masks) { + let n = m.count_ones() as usize; + b.index[r as usize] = b.rows.len() as u32; + b.rows.push(r); + b.masks.push(m); + b.starts.push(b.words.len() as u32); + b.words.extend_from_slice(&w.vals[v..v + n]); + v += n; + } + b + } + fn bytes(&self) -> usize { + self.rows.len() * 12 + self.words.len() * LANE + } + fn read(&self, base: &[Row], r: usize) -> Row { + let mut out = base[r]; + let i = self.index[r]; + if i != NONE { + let i = i as usize; + let w0 = self.starts[i] as usize; + for (w, l) in BitIter(self.masks[i]).enumerate() { + out[l * LANE..(l + 1) * LANE].copy_from_slice(&self.words[w0 + w]); + } + } + out + } + fn mask(&self, r: usize) -> u32 { + match self.index[r] { + NONE => 0, + i => self.masks[i as usize], + } + } +} + +// ---------------------------------------------------------------- C + +struct Plane { + present: Vec, + /// Positional: the value of row i is `values[i]`. + values: Vec, + count: usize, +} + +struct ArmC { + planes: Vec>, + touched: Vec, +} + +#[cfg(test)] +thread_local! { + static VALUE_READS: std::cell::Cell = const { std::cell::Cell::new(0) }; + static MERGES: std::cell::Cell = const { std::cell::Cell::new(0) }; +} + +impl ArmC { + fn seal(w: &Writes) -> Self { + let mut c = ArmC { + planes: (0..LANES).map(|_| None).collect(), + touched: Vec::new(), + }; + let mut v = 0; + for (&r, &m) in w.rows.iter().zip(&w.masks) { + for l in BitIter(m) { + let p = c.planes[l].get_or_insert_with(|| Plane { + present: vec![0; WORDS], + values: vec![[0; LANE]; N], + count: 0, + }); + p.present[r as usize / 64] |= 1 << (r % 64); + p.values[r as usize] = w.vals[v]; + p.count += 1; + v += 1; + } + } + c.touched = (0..LANES as u8) + .filter(|&l| c.planes[l as usize].is_some()) + .collect(); + c + } + /// Bytes handed to the writer: per touched lane the dirty positions + /// (as a bitmap or a u32 list, whichever is smaller) and their values. + fn bytes(&self) -> usize { + self.planes + .iter() + .flatten() + .map(|p| (WORDS * 8).min(4 * p.count) + LANE * p.count) + .sum() + } + fn resident(&self) -> usize { + self.touched.len() * (WORDS * 8 + N * LANE) + } + fn value(&self, l: usize, r: usize) -> Option<&Val> { + let p = self.planes[l].as_ref()?; + if p.present[r / 64] >> (r % 64) & 1 == 0 { + return None; + } + #[cfg(test)] + VALUE_READS.with(|c| c.set(c.get() + 1)); + Some(&p.values[r]) + } + /// Merge-on-read at one address: the base row overlaid with position r + /// of every touched lane table. + fn read(&self, base: &[Row], r: usize) -> Row { + #[cfg(test)] + MERGES.with(|c| c.set(c.get() + 1)); + let mut out = base[r]; + for &l in &self.touched { + if let Some(v) = self.value(l as usize, r) { + out[l as usize * LANE..(l as usize + 1) * LANE].copy_from_slice(v); + } + } + out + } + /// The presence words of `lanes` over an aperture, ORed: no row is built. + fn fold_lanes(&self, lanes: &[usize], a0: usize, acc: &mut [u64]) { + let (w0, w1) = (a0 / 64, (a0 + APERTURE) / 64); + acc.fill(0); + let mut tmp = vec![0u64; w1 - w0]; + for &l in lanes { + if let Some(p) = &self.planes[l] { + mask_or(acc, &p.present[w0..w1], &mut tmp); + acc.copy_from_slice(&tmp); + } + } + } +} + +fn popcount(words: &[u64]) -> u64 { + words.iter().map(|w| u64::from(w.count_ones())).sum() +} + +// ---------------------------------------------------------------- queries + +fn lmask(lanes: &[usize]) -> u32 { + lanes.iter().fold(0, |m, &l| m | 1 << l) +} + +fn f1_a(a: &ArmA, base: &[Row], a0: usize) -> u64 { + (a0..a0 + APERTURE) + .filter(|&r| F1_LANES.iter().any(|&l| a.changed(base, r, l))) + .count() as u64 +} +fn f1_b(b: &ArmB, a0: usize) -> u64 { + let m = lmask(&F1_LANES); + (a0..a0 + APERTURE).filter(|&r| b.mask(r) & m != 0).count() as u64 +} +fn f1_c(c: &ArmC, a0: usize, acc: &mut [u64]) -> u64 { + c.fold_lanes(&F1_LANES, a0, acc); + popcount(acc) +} +fn f2_a(arms: &[ArmA], base: &[Row], l: usize, a0: usize) -> u64 { + (a0..a0 + APERTURE) + .filter(|&r| arms.iter().any(|a| a.changed(base, r, l))) + .count() as u64 +} +fn f2_b(arms: &[ArmB], l: usize, a0: usize) -> u64 { + (a0..a0 + APERTURE) + .filter(|&r| arms.iter().any(|b| b.mask(r) >> l & 1 != 0)) + .count() as u64 +} +fn f2_c(arms: &[ArmC], l: usize, a0: usize, acc: &mut [u64], tmp: &mut [u64]) -> u64 { + let (w0, w1) = (a0 / 64, (a0 + APERTURE) / 64); + acc.fill(0); + for c in arms { + if let Some(p) = &c.planes[l] { + mask_or(acc, &p.present[w0..w1], tmp); + acc.copy_from_slice(tmp); + } + } + popcount(acc) +} +/// F3 on C: fold first, then merge-on-read only the rows the fold names. +fn f3_c(c: &ArmC, base: &[Row], a0: usize, acc: &mut [u64], out: &mut Vec) { + c.fold_lanes(&F1_LANES, a0, acc); + out.clear(); + for (wi, &w) in acc.iter().enumerate() { + for b in BitIter64(w) { + out.push(c.read(base, a0 + wi * 64 + b)); + } + } +} +fn f3_b(b: &ArmB, base: &[Row], a0: usize, out: &mut Vec) { + let m = lmask(&F1_LANES); + out.clear(); + for r in a0..a0 + APERTURE { + if b.mask(r) & m != 0 { + out.push(b.read(base, r)); + } + } +} +fn f3_a(a: &ArmA, base: &[Row], a0: usize, out: &mut Vec) { + out.clear(); + for r in a0..a0 + APERTURE { + if F1_LANES.iter().any(|&l| a.changed(base, r, l)) { + out.push(a.read(base, r)); + } + } +} + +struct BitIter64(u64); +impl Iterator for BitIter64 { + type Item = usize; + fn next(&mut self) -> Option { + if self.0 == 0 { + return None; + } + let t = self.0.trailing_zeros() as usize; + self.0 &= self.0 - 1; + Some(t) + } +} + +// ---------------------------------------------------------------- timing + +/// Median of 11 samples; each sample is the mean over at least 3 calls and +/// 2 ms. Returns ns per call. +fn time(mut f: F) -> f64 { + f(); + let mut v: Vec = (0..11) + .map(|_| { + let t = Instant::now(); + let mut n = 0u32; + while n < 3 || t.elapsed().as_micros() < 2_000 { + f(); + n += 1; + } + t.elapsed().as_nanos() as f64 / f64::from(n) + }) + .collect(); + v.sort_by(|a, b| a.total_cmp(b)); + v[5] +} + +fn main() { + println!("D-HOLO H14: full row (A) vs connected delta (B) vs inverted fold (C)"); + println!("tier: {}", tier()); + println!("rows per tile {N}; aperture {APERTURE}; timings ns, median of 11\n"); + let base = base(); + println!( + "{:>3} {:>5} {:>3} | {:>9} {:>9} {:>9} {:>10} | {:>6} {:>6} {:>6} | {:>6} {:>6} {:>6} | {:>6} {:>6} {:>6} | {:>7} {:>7} {:>7} | {:>7} {:>7} {:>7} | {:>6} {:>6} {:>6}", + "T", "dirty", "k", "A KiB", "B KiB", "C KiB", "C res KiB", + "sealA", "sealB", "sealC", "readA", "readB", "readC", "sessA", "sessB", "sessC", + "F1 A", "F1 B", "F1 C", "F2 A", "F2 B", "F2 C", "F3 A", "F3 B", "F3 C" + ); + let mut b_rule = true; + let mut c_rule = true; + let mut c_fold_wins = true; + let mut c_read_ok = true; + let mut acc = vec![0u64; APERTURE / 64]; + let mut tmp = vec![0u64; APERTURE / 64]; + let mut out: Vec = Vec::with_capacity(APERTURE); + for t_n in [1usize, 8, 64] { + for (label, per_10k) in [("1%", 100u64), ("10%", 1_000), ("100%", 10_000)] { + for k in [1usize, 4, 16, 31] { + let ws: Vec = (0..t_n).map(|t| writes(&base, t, per_10k, k)).collect(); + let dirty: usize = ws.iter().map(|w| w.rows.len()).sum(); + let session = session_mask(&ws); + let a: Vec = ws.iter().map(|w| ArmA::seal(&base, w)).collect(); + let b: Vec = ws.iter().map(ArmB::seal).collect(); + let c: Vec = ws.iter().map(ArmC::seal).collect(); + let (ba, bb): (usize, usize) = ( + a.iter().map(ArmA::bytes).sum(), + b.iter().map(ArmB::bytes).sum(), + ); + let bc: usize = c.iter().map(ArmC::bytes).sum(); + let rc: usize = c.iter().map(ArmC::resident).sum(); + + // Seal, ns per dirty (thought, row). Timed on thought 0 only + // and scaled by its dirty count, so a 64-thought cell does + // not rebuild gigabytes eleven times. + let d0 = ws[0].rows.len().max(1) as f64; + let sa = time(|| { + black_box(ArmA::seal(&base, black_box(&ws[0]))); + }) / d0; + let sb = time(|| { + black_box(ArmB::seal(black_box(&ws[0]))); + }) / d0; + let sc = time(|| { + black_box(ArmC::seal(black_box(&ws[0]))); + }) / d0; + + // Full-row read: 65,536 random (thought, row) pairs. + let probe: Vec<(u16, u32)> = (0..N as u64) + .map(|i| { + let x = mix(i ^ 0x5151); + ((x % t_n as u64) as u16, ((x >> 32) % N as u64) as u32) + }) + .collect(); + let per = probe.len() as f64; + let ra = time(|| { + for &(t, r) in &probe { + black_box(a[t as usize].read(&base, r as usize)); + } + }) / per; + let rb = time(|| { + for &(t, r) in &probe { + black_box(b[t as usize].read(&base, r as usize)); + } + }) / per; + let rcn = time(|| { + for &(t, r) in &probe { + black_box(c[t as usize].read(&base, r as usize)); + } + }) / per; + + // Session read: every thought of the session over one aperture. + let a0 = 8 * APERTURE; + let thoughts: Vec = (0..t_n).filter(|t| session >> t & 1 == 1).collect(); + let sper = (thoughts.len() * APERTURE).max(1) as f64; + let qa = time(|| { + for r in a0..a0 + APERTURE { + for &t in &thoughts { + black_box(a[t].read(&base, r)); + } + } + }) / sper; + let qb = time(|| { + for r in a0..a0 + APERTURE { + for &t in &thoughts { + black_box(b[t].read(&base, r)); + } + } + }) / sper; + let qc = time(|| { + for r in a0..a0 + APERTURE { + for &t in &thoughts { + black_box(c[t].read(&base, r)); + } + } + }) / sper; + + // F1 / F2 / F3, ns per aperture row (F3 per emitted row). + let ap = APERTURE as f64; + let want1 = f1_b(&b[0], a0); + assert_eq!(f1_a(&a[0], &base, a0), want1, "F1 A"); + assert_eq!(f1_c(&c[0], a0, &mut acc), want1, "F1 C"); + let want2 = f2_b(&b, 5, a0); + assert_eq!(f2_a(&a, &base, 5, a0), want2, "F2 A"); + assert_eq!(f2_c(&c, 5, a0, &mut acc, &mut tmp), want2, "F2 C"); + let f1a = time(|| { + black_box(f1_a(&a[0], &base, black_box(a0))); + }) / ap; + let f1b = time(|| { + black_box(f1_b(&b[0], black_box(a0))); + }) / ap; + let f1c = time(|| { + black_box(f1_c(&c[0], black_box(a0), &mut acc)); + }) / ap; + let f2a = time(|| { + black_box(f2_a(&a, &base, 5, black_box(a0))); + }) / ap; + let f2b = time(|| { + black_box(f2_b(&b, 5, black_box(a0))); + }) / ap; + let f2c = time(|| { + black_box(f2_c(&c, 5, black_box(a0), &mut acc, &mut tmp)); + }) / ap; + let emitted = want1.max(1) as f64; + let f3a = time(|| { + f3_a(&a[0], &base, black_box(a0), &mut out); + black_box(&out); + }) / emitted; + let f3b = time(|| { + f3_b(&b[0], &base, black_box(a0), &mut out); + black_box(&out); + }) / emitted; + let f3c = time(|| { + f3_c(&c[0], &base, black_box(a0), &mut acc, &mut out); + black_box(&out); + }) / emitted; + + if k <= 4 { + let b_ok = rb <= 1.25 * ra && qb <= 1.25 * qa && bb as f64 <= 0.5 * ba as f64; + let folds = f1b >= 2.0 * f1c && f2b >= 2.0 * f2c; + let reads = rcn <= 1.25 * rb && qc <= 1.25 * qb; + let c_ok = folds && reads && bc <= bb; + b_rule &= b_ok; + c_rule &= c_ok; + c_fold_wins &= folds; + c_read_ok &= reads; + } + let _ = dirty; + println!( + "{t_n:>3} {label:>5} {k:>3} | {:>9} {:>9} {:>9} {:>10} | {sa:>6.1} {sb:>6.1} {sc:>6.1} | {ra:>6.2} {rb:>6.2} {rcn:>6.2} | {qa:>6.2} {qb:>6.2} {qc:>6.2} | {f1a:>7.3} {f1b:>7.3} {f1c:>7.3} | {f2a:>7.3} {f2b:>7.3} {f2c:>7.3} | {f3a:>6.1} {f3b:>6.1} {f3c:>6.1}", + ba >> 10, bb >> 10, bc >> 10, rc >> 10 + ); + } + } + } + println!("\nseal: ns per dirty row; read/sess: ns per row read; F1/F2: ns per aperture row; F3: ns per emitted row"); + println!("rule at k <= 4, every (dirty, thoughts) cell:"); + println!( + " C folds >= 2x faster than B on F1 and F2: {}", + if c_fold_wins { "yes" } else { "no" } + ); + println!( + " C reads (merge-on-read) <= 1.25x B: {}", + if c_read_ok { "yes" } else { "no" } + ); + println!(" C default: {}", if c_rule { "HOLDS" } else { "not met" }); + println!( + " B default over A: {}", + if b_rule { "HOLDS" } else { "not met" } + ); +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fixture(t_n: usize, per_10k: u64, k: usize) -> (Vec, Vec) { + let b = base(); + let ws = (0..t_n).map(|t| writes(&b, t, per_10k, k)).collect(); + (b, ws) + } + + /// The dense reference: each thought's rows written in full. + fn dense(base: &[Row], w: &Writes) -> Vec { + let mut d = base.to_vec(); + let mut v = 0; + for (&r, &m) in w.rows.iter().zip(&w.masks) { + for l in BitIter(m) { + d[r as usize][l * LANE..(l + 1) * LANE].copy_from_slice(&w.vals[v]); + v += 1; + } + } + d + } + + #[test] + fn every_arm_reads_the_dense_reference() { + for k in [1, 4, 31] { + let (base, ws) = fixture(2, 1_000, k); + for w in &ws { + let d = dense(&base, w); + let (a, b, c) = (ArmA::seal(&base, w), ArmB::seal(w), ArmC::seal(w)); + let mut changed = 0; + for r in 0..N { + assert_eq!(a.read(&base, r), d[r], "A row {r} k {k}"); + assert_eq!(b.read(&base, r), d[r], "B row {r} k {k}"); + assert_eq!(c.read(&base, r), d[r], "C row {r} k {k}"); + changed += usize::from(d[r] != base[r]); + } + assert_eq!(changed, w.rows.len(), "every dirty row changes, no other"); + } + } + } + + #[test] + fn the_session_mask_names_exactly_the_writing_thoughts() { + let b = base(); + let ws: Vec = (0..4) + .map(|t| writes(&b, t, if t == 2 { 0 } else { 1_000 }, 4)) + .collect(); + assert!(ws[2].rows.is_empty() && !ws[3].rows.is_empty()); + assert_eq!(session_mask(&ws), 0b1011); + } + + #[test] + fn a_lane_mask_is_the_set_of_lanes_that_differ_from_the_base() { + let (base, ws) = fixture(1, 1_000, 4); + let b = ArmB::seal(&ws[0]); + let d = dense(&base, &ws[0]); + for r in 0..N { + let diff = (0..LANES).fold(0u32, |m, l| { + m | u32::from(lane_bytes(&d[r], l) != lane_bytes(&base[r], l)) << l + }); + assert_eq!(b.mask(r), diff, "row {r}"); + } + assert!(b.masks.iter().all(|m| m.count_ones() == 4 && m & 1 == 0)); + } + + #[test] + fn c_planes_name_exactly_the_rows_whose_lane_differs() { + let (base, ws) = fixture(1, 1_000, 4); + let c = ArmC::seal(&ws[0]); + let d = dense(&base, &ws[0]); + for l in 0..LANES { + for r in 0..N { + let differs = lane_bytes(&d[r], l) != lane_bytes(&base[r], l); + let present = c.planes[l] + .as_ref() + .is_some_and(|p| p.present[r / 64] >> (r % 64) & 1 == 1); + assert_eq!(present, differs, "lane {l} row {r}"); + } + } + } + + #[test] + fn folds_and_terminals_agree_on_every_arm() { + let (base, ws) = fixture(3, 1_000, 4); + let a: Vec = ws.iter().map(|w| ArmA::seal(&base, w)).collect(); + let b: Vec = ws.iter().map(ArmB::seal).collect(); + let c: Vec = ws.iter().map(ArmC::seal).collect(); + let d0 = dense(&base, &ws[0]); + let (mut acc, mut tmp) = (vec![0u64; APERTURE / 64], vec![0u64; APERTURE / 64]); + for a0 in [0, 3 * APERTURE, N - APERTURE] { + let want1 = (a0..a0 + APERTURE) + .filter(|&r| { + F1_LANES + .iter() + .any(|&l| lane_bytes(&d0[r], l) != lane_bytes(&base[r], l)) + }) + .count() as u64; + assert!(want1 > 0, "fixture changes F1 lanes in the aperture"); + assert_eq!(f1_a(&a[0], &base, a0), want1); + assert_eq!(f1_b(&b[0], a0), want1); + assert_eq!(f1_c(&c[0], a0, &mut acc), want1); + for l in [1, 5, 30] { + let want2 = f2_b(&b, l, a0); + assert_eq!(f2_a(&a, &base, l, a0), want2, "lane {l}"); + assert_eq!(f2_c(&c, l, a0, &mut acc, &mut tmp), want2, "lane {l}"); + } + let (mut oa, mut ob, mut oc) = (Vec::new(), Vec::new(), Vec::new()); + f3_a(&a[0], &base, a0, &mut oa); + f3_b(&b[0], &base, a0, &mut ob); + f3_c(&c[0], &base, a0, &mut acc, &mut oc); + assert_eq!(oa.len() as u64, want1); + assert_eq!(oa, ob); + assert_eq!(oa, oc); + } + } + + #[test] + fn c_counts_without_building_rows_and_builds_only_what_f3_emits() { + let (base, ws) = fixture(2, 1_000, 4); + let c: Vec = ws.iter().map(ArmC::seal).collect(); + let (mut acc, mut tmp) = (vec![0u64; APERTURE / 64], vec![0u64; APERTURE / 64]); + MERGES.with(|m| m.set(0)); + let n1 = f1_c(&c[0], 0, &mut acc); + let _ = f2_c(&c, 5, 0, &mut acc, &mut tmp); + assert_eq!(MERGES.with(|m| m.get()), 0, "F1 and F2 build no row"); + let mut out = Vec::new(); + f3_c(&c[0], &base, 0, &mut acc, &mut out); + assert!(n1 > 0); + assert_eq!(MERGES.with(|m| m.get()), n1, "one merge per emitted row"); + } + + #[test] + fn reading_a_lane_touches_one_position() { + let (_, ws) = fixture(1, 10_000, 31); + let c = ArmC::seal(&ws[0]); + VALUE_READS.with(|v| v.set(0)); + let mut found = 0; + for r in [0usize, 777, N - 1] { + for l in 1..LANES { + found += u64::from(c.value(l, r).is_some()); + } + } + assert_eq!( + found, + 3 * 31, + "every lane of a fully written row is present" + ); + assert_eq!( + VALUE_READS.with(|v| v.get()), + found, + "one value access per lane read" + ); + } +} diff --git a/crates/lance-graph-mask-risc/examples/holo_tile_crossover_probe.rs b/crates/lance-graph-mask-risc/examples/holo_tile_crossover_probe.rs new file mode 100644 index 000000000..12a2469d4 --- /dev/null +++ b/crates/lance-graph-mask-risc/examples/holo_tile_crossover_probe.rs @@ -0,0 +1,1251 @@ +//! D-HOLO H6c: the group-count crossover, a group-count switch for the hybrid, +//! and construction from a declaration. +//! +//! Pre-registered in `.claude/plans/d-holo-address-indexed-algebra-v1.md` +//! (§ Next falsifier: H6c), after H6b. The tile, the four arms (P, I, D, H), +//! the query and the row oracle are copied unchanged from +//! `holo_tile_forms_probe.rs`; the hybrid gains a switch policy. +//! +//! - Q1: fixtures with exactly g distinct operation masks, blocked or +//! interleaved over the contexts; P vs D per query as g grows. +//! - Q2: the hybrid switching on P's group count (G = 2..32) beside the byte +//! switch of H6b, over the 14 H6b patterns and the Q1 family. +//! - Q3: every arm built from the same declaration (rectangles or a Morton +//! prefix) instead of a 65,536-cell sweep; H6b's structured rule rerun. +//! +//! `cargo run --release -p lance-graph-mask-risc --example holo_tile_crossover_probe` +//! `cargo test -p lance-graph-mask-risc --example holo_tile_crossover_probe` + +// The arms and patterns are copied whole from the H6b probe; this probe does +// not use every method or field of them. +#![allow(dead_code)] + +use std::hint::black_box; +use std::time::Instant; + +use ndarray::simd::{mask_and, mask_any, mask_not, mask_or}; + +const SIDE: usize = 256; +const CELLS: usize = SIDE * SIDE; +const N: u32 = CELLS as u32; +const DENSE_WORDS: usize = CELLS / 64; +const DENSE_BYTES: usize = DENSE_WORDS * 8; +/// Bytes of one product-form group: a context mask and an operation mask. +const GROUP_BYTES: usize = 64; + +fn tier() -> &'static str { + if cfg!(target_feature = "avx512f") { + "v4 (avx512f)" + } else if cfg!(target_feature = "avx2") { + "v3 (avx2): timings not reportable" + } else { + "below v3: timings not reportable" + } +} + +// ---------------------------------------------------------------- 256-bit masks + +type M256 = [u64; 4]; +const EMPTY: M256 = [0; 4]; + +fn bit(m: &M256, i: usize) -> bool { + m[i / 64] >> (i % 64) & 1 == 1 +} +fn set(m: &mut M256, i: usize) { + m[i / 64] |= 1 << (i % 64); +} +fn and(a: &M256, b: &M256) -> M256 { + [a[0] & b[0], a[1] & b[1], a[2] & b[2], a[3] & b[3]] +} +fn or(a: &M256, b: &M256) -> M256 { + [a[0] | b[0], a[1] | b[1], a[2] | b[2], a[3] | b[3]] +} +fn not(a: &M256) -> M256 { + [!a[0], !a[1], !a[2], !a[3]] +} +fn is_zero(a: &M256) -> bool { + a.iter().all(|&w| w == 0) +} + +// ---------------------------------------------------------------- P: product form + +/// Canonical product form: groups with non-zero, pairwise disjoint context +/// masks and non-zero, distinct operation masks, sorted by operation mask. +#[derive(Clone, PartialEq, Eq, Debug)] +struct Prod { + groups: Vec<(M256, M256)>, +} + +impl Prod { + /// Canonicalize cells with disjoint context masks: drop empty ones, merge + /// equal operation masks, sort. + fn normalize(mut cells: Vec<(M256, M256)>) -> Prod { + cells.retain(|(c, o)| !is_zero(c) && !is_zero(o)); + cells.sort_unstable_by_key(|a| a.1); + let mut groups: Vec<(M256, M256)> = Vec::with_capacity(cells.len()); + for (c, o) in cells { + match groups.last_mut() { + Some(last) if last.1 == o => last.0 = or(&last.0, &c), + _ => groups.push((c, o)), + } + } + Prod { groups } + } + + fn build(f: &dyn Fn(usize, usize) -> bool) -> Prod { + let mut cells = Vec::with_capacity(SIDE); + for c in 0..SIDE { + let mut ops = EMPTY; + for o in 0..SIDE { + if f(c, o) { + set(&mut ops, o); + } + } + let mut ctx = EMPTY; + set(&mut ctx, c); + cells.push((ctx, ops)); + } + Prod::normalize(cells) + } + + /// The groups plus the implicit group of contexts with no operations. + fn with_rest(&self) -> Vec<(M256, M256)> { + let mut covered = EMPTY; + for (c, _) in &self.groups { + covered = or(&covered, c); + } + let mut v = self.groups.clone(); + let rest = not(&covered); + if !is_zero(&rest) { + v.push((rest, EMPTY)); + } + v + } + + /// `f` over the common refinement of both context partitions. + fn binop(&self, b: &Prod, f: fn(&M256, &M256) -> M256) -> Prod { + let (ga, gb) = (self.with_rest(), b.with_rest()); + let mut cells = Vec::with_capacity(ga.len() + gb.len()); + for (ca, oa) in &ga { + for (cb, ob) in &gb { + let c = and(ca, cb); + if !is_zero(&c) { + cells.push((c, f(oa, ob))); + } + } + } + Prod::normalize(cells) + } + + fn intersect(&self, b: &Prod) -> Prod { + self.binop(b, and) + } + fn union(&self, b: &Prod) -> Prod { + self.binop(b, or) + } + fn complement(&self) -> Prod { + let cells = self + .with_rest() + .into_iter() + .map(|(c, o)| (c, not(&o))) + .collect(); + Prod::normalize(cells) + } + fn is_empty(&self) -> bool { + self.groups.is_empty() + } + fn contains(&self, c: usize, o: usize) -> bool { + self.groups.iter().any(|(cm, om)| bit(cm, c) && bit(om, o)) + } + fn bytes(&self) -> usize { + self.groups.len() * GROUP_BYTES + } + fn canonical(&self) -> bool { + let g = &self.groups; + let disjoint = g + .iter() + .enumerate() + .all(|(i, a)| g[i + 1..].iter().all(|b| is_zero(&and(&a.0, &b.0)))); + disjoint + && g.iter().all(|(c, o)| !is_zero(c) && !is_zero(o)) + && g.windows(2).all(|w| w[0].1 < w[1].1) + } + fn to_dense(&self) -> Dense { + let mut d = Dense::zero(); + for (cm, om) in &self.groups { + for c in 0..SIDE { + if bit(cm, c) { + d.w[c * 4..c * 4 + 4].copy_from_slice(om); + } + } + } + d + } +} + +// ---------------------------------------------------------------- I: Morton intervals + +fn spread(v: u32) -> u32 { + let mut x = v & 0xFF; + x = (x | (x << 4)) & 0x0F0F; + x = (x | (x << 2)) & 0x3333; + (x | (x << 1)) & 0x5555 +} +fn gather(v: u32) -> u32 { + let mut x = v & 0x5555; + x = (x | (x >> 1)) & 0x3333; + x = (x | (x >> 2)) & 0x0F0F; + (x | (x >> 4)) & 0x00FF +} +/// Morton code with x = operation, y = context. +fn morton(c: usize, o: usize) -> u32 { + spread(o as u32) | (spread(c as u32) << 1) +} +fn unmorton(code: u32) -> (usize, usize) { + (gather(code >> 1) as usize, gather(code) as usize) +} + +/// A sorted set of disjoint, non-adjacent half-open Morton intervals. +#[derive(Clone, PartialEq, Eq, Debug)] +struct Iv { + s: Vec<(u32, u32)>, +} + +fn push(out: &mut Vec<(u32, u32)>, lo: u32, hi: u32) { + if lo >= hi { + return; + } + if let Some(last) = out.last_mut() { + if last.1 >= lo { + last.1 = last.1.max(hi); + return; + } + } + out.push((lo, hi)); +} + +impl Iv { + fn build(f: &dyn Fn(usize, usize) -> bool) -> Iv { + let mut s = Vec::new(); + for code in 0..N { + let (c, o) = unmorton(code); + if f(c, o) { + push(&mut s, code, code + 1); + } + } + Iv { s } + } + fn union(&self, b: &Iv) -> Iv { + let (a, b) = (&self.s, &b.s); + let mut out = Vec::with_capacity(a.len() + b.len()); + let (mut i, mut j) = (0, 0); + while i < a.len() || j < b.len() { + let take_a = j >= b.len() || (i < a.len() && a[i].0 <= b[j].0); + let (lo, hi) = if take_a { a[i] } else { b[j] }; + if take_a { + i += 1; + } else { + j += 1; + } + push(&mut out, lo, hi); + } + Iv { s: out } + } + fn intersect(&self, b: &Iv) -> Iv { + let (a, b) = (&self.s, &b.s); + let mut out = Vec::new(); + let (mut i, mut j) = (0, 0); + while i < a.len() && j < b.len() { + let lo = a[i].0.max(b[j].0); + let hi = a[i].1.min(b[j].1); + push(&mut out, lo, hi); + if a[i].1 < b[j].1 { + i += 1; + } else { + j += 1; + } + } + Iv { s: out } + } + fn complement(&self) -> Iv { + let mut out = Vec::with_capacity(self.s.len() + 1); + let mut at = 0; + for &(lo, hi) in &self.s { + push(&mut out, at, lo); + at = hi; + } + push(&mut out, at, N); + Iv { s: out } + } + fn is_empty(&self) -> bool { + self.s.is_empty() + } + fn contains(&self, c: usize, o: usize) -> bool { + let code = morton(c, o); + let k = self.s.partition_point(|&(_, hi)| hi <= code); + k < self.s.len() && self.s[k].0 <= code + } + fn bytes(&self) -> usize { + self.s.len() * 8 + } + fn canonical(&self) -> bool { + self.s.iter().all(|&(lo, hi)| lo < hi && hi <= N) + && self.s.windows(2).all(|w| w[0].1 < w[1].0) + } +} + +// ---------------------------------------------------------------- D: dense mask + +#[derive(Clone, PartialEq, Eq, Debug)] +struct Dense { + w: Box<[u64]>, +} + +impl Dense { + fn zero() -> Dense { + Dense { + w: vec![0u64; DENSE_WORDS].into_boxed_slice(), + } + } + fn build(f: &dyn Fn(usize, usize) -> bool) -> Dense { + let mut d = Dense::zero(); + for c in 0..SIDE { + for o in 0..SIDE { + if f(c, o) { + let i = c * SIDE + o; + d.w[i / 64] |= 1 << (i % 64); + } + } + } + d + } + fn intersect(&self, b: &Dense) -> Dense { + let mut d = Dense::zero(); + mask_and(&self.w, &b.w, &mut d.w); + d + } + fn union(&self, b: &Dense) -> Dense { + let mut d = Dense::zero(); + mask_or(&self.w, &b.w, &mut d.w); + d + } + fn complement(&self) -> Dense { + let mut d = Dense::zero(); + mask_not(&self.w, CELLS, &mut d.w); + d + } + fn is_empty(&self) -> bool { + !mask_any(&self.w) + } + fn contains(&self, c: usize, o: usize) -> bool { + let i = c * SIDE + o; + self.w[i / 64] >> (i % 64) & 1 == 1 + } + fn bytes(&self) -> usize { + DENSE_BYTES + } +} + +// ---------------------------------------------------------------- H: hybrid + +/// When the hybrid keeps the product form. +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Switch { + /// H6b: P while it takes fewer bytes than the dense mask. + Bytes, + /// H6c: P while it has at most this many groups. + Groups(usize), +} + +impl Switch { + fn keeps(self, p: &Prod) -> bool { + match self { + Switch::Bytes => p.bytes() < DENSE_BYTES, + Switch::Groups(g) => p.groups.len() <= g, + } + } + fn label(self) -> String { + match self { + Switch::Bytes => String::from("bytes"), + Switch::Groups(g) => format!("G={g}"), + } + } +} + +#[derive(Clone, Debug)] +enum Form { + P(Prod), + D(Dense), +} + +#[derive(Clone, Debug)] +struct Hyb { + form: Form, + switch: Switch, +} + +impl Hyb { + fn settle(p: Prod, switch: Switch) -> Hyb { + let form = if switch.keeps(&p) { + Form::P(p) + } else { + Form::D(p.to_dense()) + }; + Hyb { form, switch } + } + fn build(f: &dyn Fn(usize, usize) -> bool, switch: Switch) -> Hyb { + Hyb::settle(Prod::build(f), switch) + } + fn dense(&self) -> Dense { + match &self.form { + Form::P(p) => p.to_dense(), + Form::D(d) => d.clone(), + } + } + fn binop(&self, b: &Hyb, pf: fn(&Prod, &Prod) -> Prod, df: fn(&Dense, &Dense) -> Dense) -> Hyb { + let s = self.switch; + match (&self.form, &b.form) { + (Form::P(x), Form::P(y)) => Hyb::settle(pf(x, y), s), + (Form::D(x), Form::D(y)) => Hyb { + form: Form::D(df(x, y)), + switch: s, + }, + _ => Hyb { + form: Form::D(df(&self.dense(), &b.dense())), + switch: s, + }, + } + } + fn intersect(&self, b: &Hyb) -> Hyb { + self.binop(b, Prod::intersect, Dense::intersect) + } + fn union(&self, b: &Hyb) -> Hyb { + self.binop(b, Prod::union, Dense::union) + } + fn complement(&self) -> Hyb { + match &self.form { + Form::P(p) => Hyb::settle(p.complement(), self.switch), + Form::D(d) => Hyb { + form: Form::D(d.complement()), + switch: self.switch, + }, + } + } + fn is_empty(&self) -> bool { + match &self.form { + Form::P(p) => p.is_empty(), + Form::D(d) => d.is_empty(), + } + } + fn equal(&self, b: &Hyb) -> bool { + match (&self.form, &b.form) { + (Form::P(x), Form::P(y)) => x == y, + _ => self.dense() == b.dense(), + } + } + fn contains(&self, c: usize, o: usize) -> bool { + match &self.form { + Form::P(p) => p.contains(c, o), + Form::D(d) => d.contains(c, o), + } + } + fn is_dense(&self) -> bool { + matches!(self.form, Form::D(_)) + } +} + +// ---------------------------------------------------------------- the query + +/// The per-query formula: X = (A ∩ ¬B) ∪ (B ∩ ¬A), then emptiness of X and +/// equality of X with A. Exercises ∩, ∪ and ¬, plus both tests. +macro_rules! query { + ($a:expr, $b:expr, $eq:expr) => {{ + let (a, b) = ($a, $b); + let x = a + .intersect(&b.complement()) + .union(&b.intersect(&a.complement())); + let e = x.is_empty(); + let q = $eq(&x, a); + (x, e, q) + }}; +} + +// ---------------------------------------------------------------- patterns + +fn mix(mut z: u64) -> u64 { + z = z.wrapping_add(0x9E37_79B9_7F4A_7C15); + z = (z ^ (z >> 30)).wrapping_mul(0xBF58_476D_1CE4_E5B9); + z = (z ^ (z >> 27)).wrapping_mul(0x94D0_49BB_1331_11EB); + z ^ (z >> 31) +} +fn coin(seed: u64, c: usize, o: usize, per_10k: u64) -> bool { + mix(seed ^ ((c * SIDE + o) as u64)) % 10_000 < per_10k +} +fn in_rect(c: usize, o: usize, c0: usize, c1: usize, o0: usize, o1: usize) -> bool { + (c0..c1).contains(&c) && (o0..o1).contains(&o) +} + +type Cell = fn(usize, usize) -> bool; + +struct Pattern { + name: &'static str, + structured: bool, + adversarial: bool, + a: Cell, + b: Cell, +} + +fn patterns() -> Vec { + let p = |name, structured, adversarial, a: Cell, b: Cell| Pattern { + name, + structured, + adversarial, + a, + b, + }; + vec![ + p( + "row", + true, + false, + |c, o| c == 37 && (10..200).contains(&o), + |c, o| c == 37 && (100..250).contains(&o), + ), + p( + "column", + true, + false, + |c, o| o == 91 && (5..230).contains(&c), + |c, o| o == 91 && (40..256).contains(&c), + ), + p( + "rectangle", + true, + false, + |c, o| in_rect(c, o, 20, 120, 30, 170), + |c, o| in_rect(c, o, 60, 200, 100, 240), + ), + p( + "prefix", + true, + false, + |c, o| morton(c, o) >> 12 == 3, + |c, o| morton(c, o) >> 14 == 0, + ), + p("diagonal", false, false, |c, o| c == o, |c, o| c == o + 1), + p( + "checkerboard", + false, + true, + |c, o| (c + o) % 2 == 0, + |c, o| (c ^ o) & 2 == 0, + ), + p( + "random 1%", + false, + false, + |c, o| coin(1, c, o, 100), + |c, o| coin(2, c, o, 100), + ), + p( + "random 50%", + false, + true, + |c, o| coin(3, c, o, 5_000), + |c, o| coin(4, c, o, 5_000), + ), + p( + "sparse", + false, + false, + |c, o| coin(5, c, o, 10), + |c, o| coin(6, c, o, 10), + ), + p( + "dense", + false, + false, + |c, o| !coin(7, c, o, 10), + |c, o| !coin(8, c, o, 10), + ), + p( + "complementary", + true, + false, + |c, o| in_rect(c, o, 20, 120, 30, 170), + |c, o| !in_rect(c, o, 20, 120, 30, 170), + ), + p( + "disjoint", + true, + false, + |c, o| in_rect(c, o, 0, 100, 0, 100), + |c, o| in_rect(c, o, 150, 256, 120, 256), + ), + p( + "fragmented", + false, + true, + |c, o| o < c, + |c, o| o < c * 3 / 4, + ), + p( + "doughnut", + false, + false, + |c, o| in_rect(c, o, 40, 200, 40, 200) && !in_rect(c, o, 90, 150, 90, 150), + |c, o| in_rect(c, o, 100, 240, 0, 256), + ), + ] +} + +// ---------------------------------------------------------------- timing + +fn time(mut f: F) -> f64 { + f(); + let mut v: Vec = (0..11) + .map(|_| { + let t = Instant::now(); + let mut n = 0u32; + while n < 3 || t.elapsed().as_micros() < 2_000 { + f(); + n += 1; + } + t.elapsed().as_nanos() as f64 / f64::from(n) + }) + .collect(); + v.sort_by(|a, b| a.total_cmp(b)); + v[5] +} + +// ---------------------------------------------------------------- Q1 fixtures + +/// The first adjacent pair of grid points where P/D goes from below 1 to at +/// least 1, as `(g below, g above)`. A ratio above 1 that comes before every +/// point below 1 is not a crossing: noise can put one there. +fn upward_crossing(points: &[(usize, f64)]) -> Option<(usize, usize)> { + points + .windows(2) + .find(|w| w[0].1 < 1.0 && w[1].1 >= 1.0) + .map(|w| (w[0].0, w[1].0)) +} + +const GS: [usize; 12] = [1, 2, 3, 4, 6, 8, 12, 16, 24, 32, 48, 64]; + +#[derive(Clone, Copy, PartialEq, Eq, Debug)] +enum Layout { + Blocked, + Interleaved, +} + +/// One operation mask per context: `g` distinct seeded masks of density 0.5, +/// assigned in contiguous blocks or round robin. +fn group_rows(g: usize, layout: Layout, seed: u64) -> Vec { + let masks: Vec = (0..g as u64) + .map(|k| { + let mut m = EMPTY; + for (w, word) in m.iter_mut().enumerate() { + *word = mix(seed.wrapping_mul(0x1000).wrapping_add(k * 4 + w as u64)); + } + m + }) + .collect(); + (0..SIDE) + .map(|c| match layout { + Layout::Blocked => masks[c * g / SIDE], + Layout::Interleaved => masks[c % g], + }) + .collect() +} + +// ---------------------------------------------------------------- Q3 declarations + +/// A half-open rectangle: contexts `c0..c1`, operations `o0..o1`. +type Rect = (usize, usize, usize, usize); + +/// A pattern as a declaration: disjoint rectangles, or one aligned Morton block. +#[derive(Clone, Debug)] +enum Decl { + Rects(Vec), + Prefix { shift: u32, value: u32 }, +} + +/// The operation mask with bits `o0..o1` set. +fn range_mask(o0: usize, o1: usize) -> M256 { + let mut m = EMPTY; + for (w, word) in m.iter_mut().enumerate() { + let (lo, hi) = (w * 64, w * 64 + 64); + let (a, b) = (o0.max(lo), o1.min(hi)); + if a < b { + let n = b - a; + let bits = if n == 64 { + u64::MAX + } else { + ((1u64 << n) - 1) << (a - lo) + }; + *word |= bits; + } + } + m +} + +impl Decl { + /// The declaration as rectangles. An aligned Morton block of 4^k codes is + /// a 2^k × 2^k square. + fn rects(&self) -> Vec { + match self { + Decl::Rects(r) => r.clone(), + Decl::Prefix { shift, value } => { + let (c, o) = unmorton(value << shift); + let side = 1usize << (shift / 2); + vec![(c, c + side, o, o + side)] + } + } + } + fn contains(&self, c: usize, o: usize) -> bool { + self.rects() + .iter() + .any(|&(c0, c1, o0, o1)| in_rect(c, o, c0, c1, o0, o1)) + } +} + +/// P from a declaration: cut the contexts at every rectangle edge, OR the +/// operation masks of the rectangles covering each piece, then normalize. +fn prod_declared(d: &Decl) -> Prod { + let rects = d.rects(); + let mut cuts: Vec = rects.iter().flat_map(|r| [r.0, r.1]).collect(); + cuts.extend([0, SIDE]); + cuts.sort_unstable(); + cuts.dedup(); + let mut cells = Vec::with_capacity(cuts.len()); + for w in cuts.windows(2) { + let (lo, hi) = (w[0], w[1]); + let mut ops = EMPTY; + for &(c0, c1, o0, o1) in &rects { + if c0 <= lo && hi <= c1 { + ops = or(&ops, &range_mask(o0, o1)); + } + } + cells.push((range_mask(lo, hi), ops)); + } + Prod::normalize(cells) +} + +/// D from a declaration: each covered row gets its operation range word by word. +fn dense_declared(d: &Decl) -> Dense { + let mut out = Dense::zero(); + for (c0, c1, o0, o1) in d.rects() { + let m = range_mask(o0, o1); + for c in c0..c1 { + for (w, &bits) in m.iter().enumerate() { + out.w[c * 4 + w] |= bits; + } + } + } + out +} + +/// I from a declaration: each rectangle as its maximal aligned Morton blocks, +/// unioned; a prefix is its one interval. +fn iv_declared(d: &Decl) -> Iv { + match d { + Decl::Prefix { shift, value } => Iv { + s: vec![(value << shift, (value + 1) << shift)], + }, + Decl::Rects(rects) => rects + .iter() + .fold(Iv { s: Vec::new() }, |acc, &r| acc.union(&box_iv(r))), + } +} + +fn box_iv(r: Rect) -> Iv { + fn rec(r: Rect, c: usize, o: usize, side: usize, base: u32, out: &mut Vec<(u32, u32)>) { + let (c0, c1, o0, o1) = r; + if c >= c1 || c + side <= c0 || o >= o1 || o + side <= o0 { + return; + } + if c0 <= c && c + side <= c1 && o0 <= o && o + side <= o1 { + push(out, base, base + (side * side) as u32); + return; + } + let h = side / 2; + let q = (h * h) as u32; + rec(r, c, o, h, base, out); + rec(r, c, o + h, h, base + q, out); + rec(r, c + h, o, h, base + 2 * q, out); + rec(r, c + h, o + h, h, base + 3 * q, out); + } + let mut s = Vec::new(); + rec(r, 0, 0, SIDE, 0, &mut s); + Iv { s } +} + +/// The H6b patterns that have a declaration: (name, structured, A, B). +fn declared() -> Vec<(&'static str, bool, Decl, Decl)> { + let r = |v: &[Rect]| Decl::Rects(v.to_vec()); + // The complement of [20,120) × [30,170) as four disjoint rectangles. + let not_rect = r(&[ + (0, 20, 0, 256), + (120, 256, 0, 256), + (20, 120, 0, 30), + (20, 120, 170, 256), + ]); + vec![ + ( + "row", + true, + r(&[(37, 38, 10, 200)]), + r(&[(37, 38, 100, 250)]), + ), + ( + "column", + true, + r(&[(5, 230, 91, 92)]), + r(&[(40, 256, 91, 92)]), + ), + ( + "rectangle", + true, + r(&[(20, 120, 30, 170)]), + r(&[(60, 200, 100, 240)]), + ), + ( + "prefix", + true, + Decl::Prefix { + shift: 12, + value: 3, + }, + Decl::Prefix { + shift: 14, + value: 0, + }, + ), + ("complementary", false, r(&[(20, 120, 30, 170)]), not_rect), + ( + "disjoint", + false, + r(&[(0, 100, 0, 100)]), + r(&[(150, 256, 120, 256)]), + ), + ( + "doughnut", + false, + r(&[ + (40, 90, 40, 200), + (150, 200, 40, 200), + (90, 150, 40, 90), + (90, 150, 150, 200), + ]), + r(&[(100, 240, 0, 256)]), + ), + ] +} + +// ---------------------------------------------------------------- reporting + +/// One fixture for Q2: a name and its two membership functions. +struct Fixture { + name: String, + a: Box bool>, + b: Box bool>, +} + +fn fixtures() -> Vec { + let mut v: Vec = patterns() + .into_iter() + .map(|p| Fixture { + name: p.name.to_string(), + a: Box::new(p.a), + b: Box::new(p.b), + }) + .collect(); + for &g in &GS { + for layout in [Layout::Blocked, Layout::Interleaved] { + let (ra, rb) = (group_rows(g, layout, 11), group_rows(g, layout, 12)); + v.push(Fixture { + name: format!("g={g} {layout:?}"), + a: Box::new(move |c, o| bit(&ra[c], o)), + b: Box::new(move |c, o| bit(&rb[c], o)), + }); + } + } + v +} + +const SWITCHES: [Switch; 6] = [ + Switch::Bytes, + Switch::Groups(2), + Switch::Groups(4), + Switch::Groups(8), + Switch::Groups(16), + Switch::Groups(32), +]; + +fn main() { + println!("D-HOLO H6c: group-count crossover, group-count switch, declared construction"); + println!("tier: {}", tier()); + println!("timings: ns, median of 11\n"); + + // ------------------------------------------------------------ Q1 + println!( + "Q1: P vs D per query as the group count grows (A, B: same g and layout, seeds 11, 12)" + ); + println!("g layout P groups | P query D query P/D | P/D ∩ ∪ ¬"); + let mut crossover = Vec::new(); + for layout in [Layout::Blocked, Layout::Interleaved] { + let mut last_below: Option = None; + let mut first_above: Option = None; + let mut points: Vec<(usize, f64)> = Vec::new(); + for &g in &GS { + let (ra, rb) = (group_rows(g, layout, 11), group_rows(g, layout, 12)); + let fa = |c: usize, o: usize| bit(&ra[c], o); + let fb = |c: usize, o: usize| bit(&rb[c], o); + let (pa, pb) = (Prod::build(&fa), Prod::build(&fb)); + let (da, db) = (Dense::build(&fa), Dense::build(&fb)); + assert_eq!(pa.groups.len(), g, "g={g} {layout:?}: fixture group count"); + let (px, ..) = query!(&pa, &pb, |x: &Prod, y: &Prod| x == y); + let (dx, ..) = query!(&da, &db, |x: &Dense, y: &Dense| x == y); + assert_eq!(px.to_dense(), dx, "g={g} {layout:?}: P and D disagree"); + let pq = time(|| { + black_box(query!( + black_box(&pa), + black_box(&pb), + |x: &Prod, y: &Prod| x == y + )); + }); + let dq = time(|| { + black_box(query!( + black_box(&da), + black_box(&db), + |x: &Dense, y: &Dense| x == y + )); + }); + let pi = time(|| { + black_box(black_box(&pa).intersect(black_box(&pb))); + }); + let di = time(|| { + black_box(black_box(&da).intersect(black_box(&db))); + }); + let pu = time(|| { + black_box(black_box(&pa).union(black_box(&pb))); + }); + let du = time(|| { + black_box(black_box(&da).union(black_box(&db))); + }); + let pn = time(|| { + black_box(black_box(&pa).complement()); + }); + let dn = time(|| { + black_box(black_box(&da).complement()); + }); + let r = pq / dq; + points.push((g, r)); + if r < 1.0 { + last_below = Some(g); + } else if first_above.is_none() { + first_above = Some(g); + } + println!( + "{g:<4} {:<12} {:>8} | {pq:>9.0} {dq:>9.0} {r:>5.2} | {:>5.2} {:>5.2} {:>5.2}", + format!("{layout:?}"), + pa.groups.len(), + pi / di, + pu / du, + pn / dn + ); + } + let up = upward_crossing(&points); + println!(" {layout:?}: last g with P/D < 1: {last_below:?}; first g with P/D ≥ 1: {first_above:?}; first upward crossing: {up:?}"); + crossover.push((layout, up)); + } + let in_range = crossover + .iter() + .all(|&(_, up)| matches!(up, Some((b, a)) if b >= 4 && a <= 16)); + println!( + " prediction (crossover between g = 4 and 16 in both layouts): {}", + if in_range { "HOLDS" } else { "FALSIFIED" } + ); + let step = |g: Option| g.and_then(|g| GS.iter().position(|&x| x == g)); + let agree = match ( + step(crossover[0].1.map(|(_, a)| a)), + step(crossover[1].1.map(|(_, a)| a)), + ) { + (Some(a), Some(b)) => a.abs_diff(b) <= 1, + _ => false, + }; + println!(" layouts agree within one step of the g grid: {agree}\n"); + + // ------------------------------------------------------------ Q2 + println!("Q2: hybrid switch policies vs D, over 14 H6b patterns + {} Q1 fixtures (N = 100, sweep build)", GS.len() * 2); + let fx = fixtures(); + let mut dq = Vec::new(); + let mut dt = Vec::new(); + for f in &fx { + let (da, db) = (Dense::build(&*f.a), Dense::build(&*f.b)); + let q = time(|| { + black_box(query!( + black_box(&da), + black_box(&db), + |x: &Dense, y: &Dense| x == y + )); + }); + let b = time(|| { + black_box((Dense::build(&*f.a), Dense::build(&*f.b))); + }); + dq.push(q); + dt.push(b + 100.0 * q); + } + println!("switch | max total/D max query/D | worst fixture (query) | verdict"); + for &s in &SWITCHES { + let mut tot = Vec::new(); + let mut qr = Vec::new(); + for (k, f) in fx.iter().enumerate() { + let (ha, hb) = (Hyb::build(&*f.a, s), Hyb::build(&*f.b, s)); + let q = time(|| { + black_box(query!( + black_box(&ha), + black_box(&hb), + |x: &Hyb, y: &Hyb| x.equal(y) + )); + }); + let b = time(|| { + black_box((Hyb::build(&*f.a, s), Hyb::build(&*f.b, s))); + }); + tot.push((b + 100.0 * q) / dt[k]); + qr.push(q / dq[k]); + } + let mt = tot.iter().copied().fold(0.0, f64::max); + let (wk, mq) = + qr.iter() + .copied() + .enumerate() + .fold((0, 0.0), |a, (k, v)| if v > a.1 { (k, v) } else { a }); + let verdict = if mt <= 1.25 && mq <= 1.25 { + "safe (≤ 1.25)" + } else if mt >= 2.0 || mq >= 2.0 { + "NOT safe (≥ 2× somewhere)" + } else { + "undecided (1.25–2)" + }; + println!( + "{:<7} | {mt:>11.2} {mq:>12.2} | {:<28} | {verdict}", + s.label(), + fx[wk].name + ); + } + println!(); + + // ------------------------------------------------------------ Q3 + println!("Q3: construction from a declaration (every arm, D included); ns and ratio to D"); + println!("pattern build P / I / D (swept D) | query P / I / D | total N=100 P/D H/D | N where P < D"); + for (name, structured, a, b) in declared() { + let (pa, pb) = (prod_declared(&a), prod_declared(&b)); + let (ia, ib) = (iv_declared(&a), iv_declared(&b)); + let (da, db) = (dense_declared(&a), dense_declared(&b)); + let s = Switch::Groups(8); + let (ha, hb) = (Hyb::settle(pa.clone(), s), Hyb::settle(pb.clone(), s)); + let bp = time(|| { + black_box((prod_declared(black_box(&a)), prod_declared(black_box(&b)))); + }); + let bi = time(|| { + black_box((iv_declared(black_box(&a)), iv_declared(black_box(&b)))); + }); + let bd = time(|| { + black_box((dense_declared(black_box(&a)), dense_declared(black_box(&b)))); + }); + let bh = time(|| { + black_box(( + Hyb::settle(prod_declared(black_box(&a)), s), + Hyb::settle(prod_declared(black_box(&b)), s), + )); + }); + let fa = |c: usize, o: usize| a.contains(c, o); + let fb = |c: usize, o: usize| b.contains(c, o); + let bsw = time(|| { + black_box((Dense::build(&fa), Dense::build(&fb))); + }); + let qp = time(|| { + black_box(query!( + black_box(&pa), + black_box(&pb), + |x: &Prod, y: &Prod| x == y + )); + }); + let qi = time(|| { + black_box(query!(black_box(&ia), black_box(&ib), |x: &Iv, y: &Iv| x == y)); + }); + let qd = time(|| { + black_box(query!( + black_box(&da), + black_box(&db), + |x: &Dense, y: &Dense| x == y + )); + }); + let qh = time(|| { + black_box(query!( + black_box(&ha), + black_box(&hb), + |x: &Hyb, y: &Hyb| x.equal(y) + )); + }); + let tot = |b: f64, q: f64| b + 100.0 * q; + let (rp, rh) = (tot(bp, qp) / tot(bd, qd), tot(bh, qh) / tot(bd, qd)); + let n_star = if qp < qd { + format!("{:.0}", ((bp - bd) / (qd - qp)).max(0.0).ceil()) + } else if bp <= bd { + String::from("N ≤ crossover only") + } else { + String::from("never") + }; + let rule = if structured { + format!( + " structured: {}", + if rp.min(rh) <= 0.5 { + "PASS (≤ 0.5)" + } else if rp.min(rh) > 0.5 && rp.min(rh) < 2.0 { + "undecided" + } else { + "FAIL" + } + ) + } else { + String::new() + }; + println!( + "{name:<13} {bp:>6.0} {bi:>6.0} {bd:>6.0} ({bsw:>7.0}) | {qp:>6.0} {qi:>6.0} {qd:>6.0} | {rp:>5.2} {rh:>5.2} | {n_star}{rule}" + ); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn a_ratio_above_one_before_every_point_below_is_not_a_crossing() { + // Codex example: above 1 at g = 1, then below 1 through g = 64. + let pts: Vec<(usize, f64)> = GS + .iter() + .map(|&g| (g, if g == 1 { 1.2 } else { 0.8 })) + .collect(); + assert_eq!(upward_crossing(&pts), None); + // An ordinary crossing between 8 and 12 is found. + let pts: Vec<(usize, f64)> = GS + .iter() + .map(|&g| (g, if g <= 8 { 0.8 } else { 1.2 })) + .collect(); + assert_eq!(upward_crossing(&pts), Some((8, 12))); + } + + fn same_cells(a: &dyn Fn(usize, usize) -> bool, b: &dyn Fn(usize, usize) -> bool) -> bool { + (0..CELLS).all(|i| a(i / SIDE, i % SIDE) == b(i / SIDE, i % SIDE)) + } + + /// FAILS IF: a declaration builds a different set from the swept build of + /// the same membership function, in any arm (P: group list; D: words; I: + /// intervals), or a declaration does not describe its H6b pattern. + #[test] + fn declared_build_equals_swept_build() { + let pats = patterns(); + for (name, _, a, b) in declared() { + let p = pats.iter().find(|p| p.name == name).unwrap(); + for (d, f) in [(&a, p.a), (&b, p.b)] { + let g = |c: usize, o: usize| d.contains(c, o); + assert!( + same_cells(&g, &f), + "{name}: declaration differs from the pattern" + ); + assert_eq!(prod_declared(d), Prod::build(&f), "{name}: P"); + assert_eq!(dense_declared(d), Dense::build(&f), "{name}: D"); + assert_eq!(iv_declared(d), Iv::build(&f), "{name}: I"); + } + } + } + + /// FAILS IF: a Q1 fixture does not have exactly g groups, or is empty or + /// full, or the two layouts coincide for g > 1. + #[test] + fn group_fixtures_have_g_groups() { + for &g in &GS { + for layout in [Layout::Blocked, Layout::Interleaved] { + let rows = group_rows(g, layout, 11); + let p = Prod::build(&|c, o| bit(&rows[c], o)); + assert_eq!(p.groups.len(), g, "g={g} {layout:?}"); + let n: usize = rows + .iter() + .map(|m| m.iter().map(|w| w.count_ones() as usize).sum::()) + .sum(); + assert!(n > 0 && n < CELLS, "g={g} {layout:?}: empty or full"); + } + if g > 1 { + assert_ne!( + group_rows(g, Layout::Blocked, 11), + group_rows(g, Layout::Interleaved, 11) + ); + } + } + } + + /// FAILS IF: the group-count hybrid never switches or always switches: at + /// G = 8 it must keep P at g = 1 and fall back to D at g = 64. + #[test] + fn group_switch_switches_both_ways() { + for layout in [Layout::Blocked, Layout::Interleaved] { + let one = group_rows(1, layout, 11); + let many = group_rows(64, layout, 11); + assert!(!Hyb::build(&|c, o| bit(&one[c], o), Switch::Groups(8)).is_dense()); + assert!(Hyb::build(&|c, o| bit(&many[c], o), Switch::Groups(8)).is_dense()); + } + } + + /// FAILS IF: on a Q1 fixture or an H6b pattern, P, D or a hybrid under any + /// switch differs from the row oracle on the query, cell by cell. + #[test] + fn every_arm_matches_the_oracle() { + for f in fixtures() { + let x: Vec = (0..CELLS) + .map(|i| (f.a)(i / SIDE, i % SIDE) != (f.b)(i / SIDE, i % SIDE)) + .collect(); + let e = x.iter().all(|&v| !v); + let q = (0..CELLS).all(|i| x[i] == (f.a)(i / SIDE, i % SIDE)); + let (pa, pb) = (Prod::build(&*f.a), Prod::build(&*f.b)); + let (px, pe, pq) = query!(&pa, &pb, |x: &Prod, y: &Prod| x == y); + assert_eq!((pe, pq), (e, q), "{}: P flags", f.name); + for &s in &SWITCHES { + let (ha, hb) = (Hyb::build(&*f.a, s), Hyb::build(&*f.b, s)); + let (hx, he, hq) = query!(&ha, &hb, |x: &Hyb, y: &Hyb| x.equal(y)); + assert_eq!((he, hq), (e, q), "{} {s:?}: H flags", f.name); + for (i, &want) in x.iter().enumerate() { + assert_eq!(hx.contains(i / SIDE, i % SIDE), want, "{} {s:?}", f.name); + } + } + for (i, &want) in x.iter().enumerate() { + assert_eq!(px.contains(i / SIDE, i % SIDE), want, "{}: P", f.name); + } + } + } + + /// FAILS IF: a rectangle's Morton decomposition is not exactly the rectangle. + #[test] + fn box_iv_is_exact() { + for r in [ + (0, 256, 0, 256), + (37, 38, 10, 200), + (3, 250, 7, 9), + (20, 120, 30, 170), + ] { + let iv = box_iv(r); + assert!(iv.canonical()); + for c in 0..SIDE { + for o in 0..SIDE { + assert_eq!( + iv.contains(c, o), + in_rect(c, o, r.0, r.1, r.2, r.3), + "{r:?} at ({c}, {o})" + ); + } + } + } + } +}