Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .claude/board/entries/2026-10-28-d-drca-2-dr256-where-dr3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# D-DRCA-2: DR256::WHERE × DR#3

**Status:** MEASURED, lab only. Code: `crates/dr-math-binding-probe/src/where_dr3.rs`, test `tests/dr256_where_dr3.rs`, costs `examples/dr256_where_stages.rs`; results in the crate README, section D-DRCA-2. Builds on D-DRCA-1 (`hhtl16`, #1834). Contract: `.claude/plans/d-drdt-dr-dt-ownership-amendment-v1.md` (#1835).

## Result

- One Quack WHERE (`Draft::bind` + `Filter::sql_where`, mask-risc `Keep`, 65,536 addresses) projects into a 64-slot DR frame by `map.len()` probes through the counted `LocalMap`. No new predicate evaluator, routing engine, mask store or epistemic sidecar.
- Epistemic absence (`$null`, NaN, SQL NULL, unset validity, K3 UNKNOWN) is one logical value. WHERE selects TRUE; the unselected slots are FALSE or UNKNOWN and no complement is offered. A negation is its own WHERE, so `NOT UNKNOWN` stays UNKNOWN. A three-valued property keeps value + validity. No Epi5 state is derived from absence.
- Every read is re-admitted against owner, source, codebook, frame, register frame, axis, DR method (the ClassId-bound `MethodSignature`), focus, scope and criteria; each refusal is pinned by a test against an independent source-ID oracle.
- Projection, admission and folds allocate nothing. Measured back to back on the same fixture, the single WHERE halves the fresh read (41.5–43.2 µs → 20.6 µs). Warm reuse pays from N = 2 (2.0×), 7.9–8.0× at N = 8; at N = 64 fresh and rebind exceed 1,000 µs (failed gate), warm stays at 24–25 µs.
- Sixteen disable runs each turn a test red. The #1792 control still reproduces (`reused R2 1032 vs fresh 31`).

## Open

A detached `Keep` projected under a NEW seal is accepted: the seal attests the caller's claim, not mask provenance (independent gate, D-NFR S5: a producer-side frame on Quack's bound query). `Criteria` is caller-supplied. One 64-slot frame per table; no DR256 resident register; no scattered-mask reuse.
3 changes: 2 additions & 1 deletion .claude/board/entries/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,14 +25,15 @@ index row, (3) no duplicate entry id. Checks 1 and 2 are deliberately
opposite directions; the stranding this convention prevents shows up in
exactly one of them, never both.

493 entries, 2026-08-06 .. 2026-10-28.
494 entries, 2026-08-06 .. 2026-10-28.

| date | entry id | finding | file |
|---|---|---|---|
| 2026-10-28 | `D-SAG-6-1` | | [2026-10-28-d-sag-6-1-audit-reference-ledger.md](2026-10-28-d-sag-6-1-audit-reference-ledger.md) |
| 2026-10-28 | `D-DT64-6` | | [2026-10-28-d-dt64-6-routing-bindstore-behavior.md](2026-10-28-d-dt64-6-routing-bindstore-behavior.md) |
| 2026-10-28 | `D-DT64-6-1a` | | [2026-10-28-d-dt64-6-ogit-predicates-impl.md](2026-10-28-d-dt64-6-ogit-predicates-impl.md) |
| 2026-10-28 | `D-DRDT` | | [2026-10-28-d-drdt-dr-dt-ownership-amendment.md](2026-10-28-d-drdt-dr-dt-ownership-amendment.md) |
| 2026-10-28 | `D-DRCA-2` | | [2026-10-28-d-drca-2-dr256-where-dr3.md](2026-10-28-d-drca-2-dr256-where-dr3.md) |
| 2026-10-28 | `D-DRCA-1` | | [2026-10-28-d-drca-1-hhtl16-dr-projection.md](2026-10-28-d-drca-1-hhtl16-dr-projection.md) |
| 2026-10-27 | `harvest-integration-composite-key-and-simd-scope` | | [2026-10-27-harvest-integration-composite-key-and-simd-scope.md](2026-10-27-harvest-integration-composite-key-and-simd-scope.md) |
| 2026-10-26 | `D-ONNX-INTAKE-0` | | [2026-10-26-onnx-frontend-intake-bakeoff.md](2026-10-26-onnx-frontend-intake-bakeoff.md) |
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/dr-math-binding-probe.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ on:
- "crates/dr-math-binding-probe/**"
- "crates/lance-graph-contract/**"
- "crates/lance-graph-act-r/**"
- "crates/lance-graph-quack/**"
- "crates/lance-graph-mask-risc/**"
- ".github/workflows/dr-math-binding-probe.yml"
- "rust-toolchain.toml"
push:
Expand All @@ -16,6 +18,8 @@ on:
- "crates/dr-math-binding-probe/**"
- "crates/lance-graph-contract/**"
- "crates/lance-graph-act-r/**"
- "crates/lance-graph-quack/**"
- "crates/lance-graph-mask-risc/**"
- ".github/workflows/dr-math-binding-probe.yml"
- "rust-toolchain.toml"
workflow_dispatch:
Expand Down
5 changes: 5 additions & 0 deletions crates/dr-math-binding-probe/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,8 @@ sha2 = "0.10"

[dev-dependencies]
lance-graph-act-r = { path = "../lance-graph-act-r" }
# D-DRCA-2 (tests/dr256_where_dr3.rs, examples/dr256_where_stages.rs): the
# real WHERE path. Quack binds and lowers, mask-risc executes the `Keep`
# terminal; the library itself never names either.
lance-graph-quack = { path = "../lance-graph-quack" }
lance-graph-mask-risc = { path = "../lance-graph-mask-risc" }
70 changes: 70 additions & 0 deletions crates/dr-math-binding-probe/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,3 +65,73 @@ Disable runs, each turning its named test red and restored from the commit: iden
Warm reads made 0 global→local lookups. Single host, single run; reported, not gates.

**Not done:** DR256::WHERE selection over 65,536 addresses and the stay / rebind / masked-read choice; focus larger than 64; any SIMD kernel (the column read is a scalar per-row gather, the largest warm cost after the folds); a real source owner, ACL or ClassView registry; DT method semantics (DR #4 handoff, amendment plan).

## D-DRCA-2: DR256::WHERE × DR#3 (`where_dr3`)

Lab test of the composition `global selection → optional amortized global→local mapping → HHTL16 Cartesian projection → native fold`. Module `src/where_dr3.rs`, test `tests/dr256_where_dr3.rs`, costs `examples/dr256_where_stages.rs`.

- **WHERE is Quack's, once.** `table("node").where_eq("class", "employee").where_eq("sel", 1).bind(..)` (`Draft::bind`, three-valued through `Filter::sql_where`), lowered and executed by mask-risc's `Keep` terminal over 65,536 addresses. Its output, the rows where the criteria are TRUE, is the only selection input. The module evaluates no predicate and depends on neither engine.
- **DT is the existing ClassId-bound schema.** The register's `MethodSignature` (owner `ClassId`, axes, truth, population) is what a selection is projected for and re-admitted against.
- **Global → local.** `project` tests bit `global(l)` of the WHERE mask for each local slot of the table's `LocalMap`: `map.len()` probes, never a pass over the universe, no map search. A selected identity outside the frame is not projected. The frame must carry `LocalMap::digest()` as its `coordinates` revision, so a selection cannot be projected through one coordinate assignment and read under another.
- **Admission.** Every read goes through `LocalSelection::admit`, which re-checks the global source seal (owner, source revision, codebook, address space), the local frame, register frame identity, identity axis, DR method, focus, scope and criteria. The only readable form is the `Admitted` it returns. A `LocalSelection` is a caller-owned value; there is no store or cache.
- **Folds.** The selection is a `u64` over the identity axis, ANDed into reads of the borrowed rows (`selected_count`, `selected_column`, `selected_column_k3`, `selected_cell`). No Cartesian product, no copied row.

**Epistemic absence.** `$null`, NaN, SQL NULL, an unset Arrow validity bit and K3 UNKNOWN are one logical value: no TRUE/FALSE judgment has been established. It does not say the entity or relationship does not exist, and no finer epistemic state (CausalEdge64 Epi5, under its own evidence contract) is derived from it. Consequences here:

- WHERE selects TRUE. A slot that is not selected is FALSE **or** UNKNOWN, and `Admitted` offers no complement, because the complement of a selection would read UNKNOWN as FALSE.
- `NOT UNKNOWN` is UNKNOWN. A negated selection is a second WHERE over the negated criteria through the same `sql_where`, with its own criteria identity; reusing one selection as the other is refused.
- A three-valued property keeps the register's own value + validity form (`BoundDr64::validity`). `selected_column_k3` returns `(value, validity)` over the selection, with the value masked by validity, so a stale payload under an UNKNOWN cell is never TRUE. The Boolean folds read payload bits as presence and refuse a three-valued or bag register (the rule `hhtl16::cooccurrence` and `relcomp::Chain::new` apply).
- Physical encodings (Quack validity planes, Arrow bitmaps, NaN payloads) are adapter concerns; none adds a semantic category.

Oracle: the source is plain Rust values per global address (`class`, `sel: Option<i32>`, `live`). Verdicts are a `match` on `Option`, local positions are `position()` in the frame's source-order id list, and registers are built from those positions, not through the map. It shares no code with Quack, mask-risc, `hhtl16` or `where_dr3`. NULL rows carry the payload the predicate tests for, so an engine that read NULL payloads would keep them.

Thirteen tests against that oracle:

- the global WHERE, whole mask over all 65,536 addresses, and its in-frame selection (TRUE, FALSE and UNKNOWN all occur in the frame; neither FALSE nor UNKNOWN is selected; most selected identities lie outside the frame);
- equal local ordinals of two tables: the employee selection projected through the department map selects no department, and reading it against a department-row register is refused (`AxisMismatch`, `DifferentFrame`);
- selected Identity × Property cell by cell, by column and by count, and the same relationships through the YZ rotation;
- UNKNOWN: the negated WHERE keeps exactly the oracle's FALSE rows, a NULL row is kept by neither WHERE (globally and in the frame), the complement of a selection differs from its negation, one selection is refused as the other, the three-valued column matches the oracle in value + validity form including with a stale payload under an UNKNOWN cell, and the Boolean folds refuse that register;
- a source change after WHERE (refused, then rebound to the oracle);
- a scope or focus change with identical coordinates (refused; the unchanged register admits);
- selected employees' sites through a witnessed FK (`Chain`), then a changed referenced property (refused, rebound);
- a reseal at equal numbers and a new coordinate assignment (refused; the old map is refused under the new frame, and the new map projects to the new ordinals);
- a foreign owner at equal numbers, other criteria, another DR method, and a new codebook (refused; the stale binding run silently keeps nobody);
- empty and full selections, plus a malformed width and an id outside the universe;
- the #1792 shape: a detached `Keep` reused after its criteria input changed answers silently with the old population; projected under the old seal it is refused;
- projection under a stale global seal, a foreign owner's seal or a resealed local frame;
- projection, admission and folds allocate nothing.

Sixteen disable runs, each restored from the commit and each turning at least one test red: the two projection-time seal checks, `MapNotFrame`, the complement projected as the selection, the eight admission guards (global, local, frame issue, axis, method, focus, scope, criteria), the three-valued column reporting UNKNOWN as known, its value not masked by validity, `selected_count` ignoring the selection, and the Boolean folds accepting a three-valued register.

The original #1792 control is unchanged and still reproduces: `cargo run --release --example p2_quack_where -- --controls` in `crates/drf-focus-probe` prints `reused R2 1032 vs fresh 31`.

`cargo run --release --example dr256_where_stages` on a 4-vCPU shared Xeon @ 2.10 GHz (avx512f), median of 21 runs. The single-WHERE path was measured back to back against the previous two-program path (TRUE and FALSE `Keep`, commit `cbc7930`) on the same fixture, alternating old/new three times; the two quiet pairs are shown (the third pair was disturbed by host load in both arms).

| stage | previous (two WHERE programs) | single WHERE | allocs |
|---|---|---|---|
| WHERE `Keep`, 65,536 rows | 41.7 / 40.2 µs | 20.1 / 20.0 µs | 0 |
| end-to-end fresh read | 43.2 / 41.5 µs | 20.6 / 20.6 µs | 0 |
| warm total, N = 8 | 42.9 / 42.4 µs | 21.4 / 21.0 µs | 0 |

Other stages of the single-WHERE path (two runs): `Draft::bind` 465–513 ns (10 allocs), `lower` 472–482 ns (7 allocs), `project` 912–926 ns, `admit` 76–82 ns, warm read 84–91 ns, `selected_count` 14–16 ns, all 0 allocs. Cold: `LocalMap::prepare` 891–963 ns (3 allocs); encoding the 65,536-row fixture into planes and lanes ~600 µs (fixture, once).

Totals for N compatible reads, single WHERE (µs, quiet pairs; fresh = WHERE + project + admit + read each time; warm = WHERE + project once, then admit + read; rebind = the owner advances before every read, the old selection is refused, then WHERE + project + admit + read):

| N | fresh | warm | rebind | fresh / warm |
|---|---|---|---|---|
| 1 | 20.7 / 21.1 | 20.6 / 20.7 | 20.8 / 21.1 | 1.00 / 1.02 |
| 2 | 42.4 / 42.4 | 21.1 / 20.6 | 41.3 / 41.2 | 2.01 / 2.05 |
| 4 | 83.2 / 80.1 | 21.0 / 20.1 | 85.7 / 81.7 | 3.96 / 3.99 |
| 8 | 170.8 / 165.1 | 21.4 / 21.0 | 169.4 / 168.6 | 7.96 / 7.86 |
| 64 | exceeds 1,000 µs (failed gate) | 25.2 / 24.1 | exceeds 1,000 µs (failed gate) | — |

One WHERE halves the fresh read. Warm reuse breaks even at N = 1 and pays from N = 2; at N = 64 only warm reuse stays inside the fold budget. Rebind costs the same as fresh. Microbenchmarks, reported, not gates.

**Open limitations.**

- **The seal attests the caller's claim, not the mask's provenance** (independent gate, D-NFR S5). A detached `Keep` projected under a new seal after its source changed is accepted (pinned by the #1792-shape test, so a fix turns it red). Closing it needs the producer to bind a mask to the seal it was computed under; Quack's bound `Query` has no frame today.
- `Criteria` is a caller-supplied id; Quack exposes no digest of a bound query.
- The global codebook revision rides in `FrameRevisions::relation_map`; the owner, ACL and scope are the lab `FrameStore`, `Scope { id }` equality.
- A negated selection costs its own WHERE, which is the price of `NOT UNKNOWN` staying UNKNOWN.
- One 64-slot frame per table; no larger focus, no DR256 resident register, no reuse of a scattered (`ScatterOrU32`) mask.
- `project` is 64 scalar probes (~1 µs); admission (~76–82 ns) is not broken down.
Loading
Loading