[AdaWorldAPI]

On a float lane, a clear presence bit and a NaN payload now represent the same state:

presence bit i == 0   ⇔   values[i] is NaN

"Is NaN" means x != x, not a particular bit pattern. Every NaN counts (quiet, signalling-pattern, either sign, any payload), so a backend that rewrites NaN payloads, as wasm can, cannot break the invariant. Because the two encodings agree, ndarray can treat NaN as an ordinary computed value. Callers no longer need to scrub the lane before handing it over. This also makes the float lane usable for SQL with NULL = NaN.

Added (src/simd_masking_ops.rs, re-exported from ndarray::simd)

function	what it does
not_nan_f64_to_mask, not_nan_f32_to_mask	Derives the presence mask from the lane, using the ordered simd_eq of each value against itself.
nan_where_clear_f64, nan_where_clear_f32	Writes NaN into every row whose presence bit is clear. Rows with the bit set are left untouched. Cost is one word read per 64 rows plus one store per clear bit.
presence_nan_mismatches_f64, presence_nan_mismatches_f32	Checks the invariant. Returns the number of rows where the mask and the lane disagree, popcount(presence XOR derived) over live rows, without allocating. 0 means the invariant holds.
{lt,le,gt,ge,eq,ne}_{f64,f32}_to_mask	Returns the SQL TRUE plane of a comparison. A NaN row is NULL and never matches, including for <>, unlike IEEE !=. A NaN threshold is the NULL literal and matches no row.
mask_not_under(pred, presence)	SQL NOT under three-valued logic: presence & !pred gives the FALSE plane, and UNKNOWN rows appear in neither plane. A bare complement is wrong here, because !(NaN > 5) is true in IEEE while NOT (NULL > 5) is UNKNOWN in SQL.

F64Mask8::to_bitmask is added on the AVX-512, AVX2/AVX, NEON, wasm and scalar backends. Nightly already had it.

presence in mask_not_under must be the presence of the whole predicate: the AND of its operands' presence planes, and empty for a NULL literal. A unit test exposed this: when the column's own plane was used, NOT (x > NULL) reported every present row as FALSE. The requirement is now in the function's docs.

Lab probe: examples/sm9_presence_probe.rs

The same rule applied to an integer lane, as a follow-up to #380.

-0 is NaN. In #380's sign-magnitude 9-bit code, NaN is the codeword "sign set, magnitude 0", written -0. This probe keeps that meaning and only changes where the state is stored:

	#380	this probe
NaN (NULL)	codeword -0 in the payload	presence bit clear, payload 0
zero	codeword +0	presence bit set, payload +0
values	−255..−1, +1..+255	−255..+255, all of them

A clear presence bit with 0 underneath is #380's -0. It is the same NaN, not a second, different one. Moving it out of the payload into the presence plane is what frees the payload: no 9-bit code is spent on NaN any more. The writer stores +0 (magnitude 0, sign clear) under a clear presence bit, so a missing row and a zero row hold the same payload bytes and only the presence bit tells them apart. A reader decodes presence-clear as NaN (nan_where_clear), whatever the payload holds.

• Masked dot product: lanes split by sign_a ^ sign_b into two unsigned u8 × u8 accumulators. NULL rows are excluded by the presence AND, and n is the popcount of the same mask. It is checked against a decoded oracle on all 512 × 512 (value or NULL) pairs, on random blocks with a filter, and with a garbage payload under a clear bit.

• Decoding applies nan_where_clear, so the float bijection holds on the decoded lane: presence_nan_mismatches == 0.

• tanh exactness against i8 Sym (−127..=127), over 2,000,001 points. tanh is odd, so the sign-magnitude table is indexed by magnitude and the sign plane passes through untouched:

  path	max error ratio (i8 / sign-magnitude)	RMS ratio
  output quantization only	2.01×	1.94×
  256-entry table, input range 2..8	1.92–1.99×	1.97–2.02×

  The largest |x| still below the top output code moves from 3.11 to 3.46. That is the gain of the one extra bit, about 2×.

• 3σ gate (test): 200,000 independent uniform inputs over [−4, 4], table path at r = 4. The tolerance 0.010 is fixed in advance, between the two formats' measured 99.73% error quantiles (about 0.0079 and 0.0157). Measured coverage: sign-magnitude 1.0000, i8 0.9519. The 3σ level is 0.9973, so sign-magnitude passes and i8 fails. The paired z of the error difference is 246.9. Quantizing the sign-magnitude table to even codes turns the gate red (coverage 0.9046).

• Scope: scalar reference only; the SIMD cost of the u8 × u8 accumulation is not measured. In the two earlier disable runs, removing the presence AND failed 3 of the then 6 tests and replacing the sign XOR with AND failed 2.

Tests

• Parity group 0x13xx: covers both directions, the mismatch count and the round trip. It runs at 12 lengths across every 8- and 16-lane group boundary, with leftover set bits beyond the last row and with a surplus output word.

• Parity groups 0x139xx–0x13Cxx: cover 6 operators × 9 thresholds (NaN, ±0, ±inf, subnormal, ±1, MAX), for both f64 and f32, plus the FALSE plane.

• The reference never does a float comparison. It detects NaN from the exponent and mantissa bits, and orders values by a sign-magnitude key in which -0 == +0. (That is the float -0.0, an ordinary zero, not the sign-magnitude -0 above.)

• Every backend run is green, 21/21 groups: native AVX-512, AVX2 (x86-64-v3), wasm, wasm without SIMD, wasm with relaxed SIMD, NEON under qemu, AVX under qemu, x86-64-v2 under qemu, and nightly.

• Disable runs: each guard was broken in turn, the parity program was run, and the guard was restored. Every run failed, as intended:

  guard removed	error code
  f64 tail mask	0x1301
  x == x replaced by x < inf	0x1304
  partial final chunk skipped when writing NaN	0x1322
  set bits beyond the last row counted	0x1341
  NULL-row gate in <>	0x13915
  NULL threshold allowed to match	0x13905
  NOT as a bare complement	0x13b00

• Unit tests, doctests and lint: 13 unit tests and 19 doctests pass, plus the probe's 7 tests. cargo clippy --lib --tests -D warnings is clean, as are clippy on the parity crate and on the probe. cargo fmt --check is clean for every line this PR changes; the parity crate still has older formatting diffs that this PR does not touch.

Not in this PR

• A masked float fold (n, mean, M2 with a fixed order), and bit-identical float results across backends.
• Tests for three-valued AND/OR of two nullable predicates. Those combine with the existing mask_and and mask_or.
• GROUP BY on float keys. Since NaN != NaN, grouping has to use the presence bit as the key.
• Known differences from PostgreSQL and DuckDB: a NaN stored as a real value reads back as NULL.
• The legacy quantize_f32_to_i8 and quantize_f32_to_u8 (src/hpc/quantized.rs) turn NaN into a real code through a saturating cast, so a NULL = NaN lane passed through them loses its NULLs. Not changed here.
• lut9, the SIMD form of the magnitude-indexed table, is in simd: simd_stride.rs (lut_u8, lut9) + FACET STRIDE rule and guard #382 (src/simd_stride.rs).

🤖 Generated with Claude Code

https://claude.ai/code/session_01XAqKMPh2Seh2WLccCksAdY

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration

• Source mode: Auto
• Repository: Unavailable
• Branch: Unavailable
• File: No configuration file from this repository applied
• Commit: Unavailable
• Layers: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 17ff5c0a-8d55-4f6d-9172-9edd6e0521ea

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Warning

Billing warning: we have not been able to collect payment for this subscription for more than 72 hours. Please update the payment method or pay any pending invoices in Billing to avoid service interruption.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-27T14:38:56.059138Z	a6803bc	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.