Skip to content

Parameter matrix test against NumPy; strides as NumPy's everywhere - #3

Merged
hokb merged 2 commits into
mainfrom
claude/project-thread-f4qooq
Nov 5, 2026
Merged

hokb merged 2 commits into
mainfrom
claude/project-thread-f4qooq

Conversation

@hokb

@hokb hokb commented Nov 5, 2026 •

Copy link
Copy Markdown
Member

Adds python/tests/test_matrix.py, copied from viproc (hokb/viproc#62), and ports the fixes for what it found. Every op runs on NumPy and on numpy2x with the same inputs and must give the same values (bit for bit), dtype, shape, strides, warnings and errors.

What the matrix covers. Shapes: 0-d; empty arrays of 1, 2, 3 and 8 dimensions; vectors; matrices; 3-d and 8-d arrays with extent-1 axes. Layouts: C, F, transposed, stepped, reversed. Seven dtypes. Special values: NaN, ±inf, -0.0 and integer extremes. Inputs are either ready or pending (direct path vs. task). Op groups: element-wise unary and binary ops with broadcasting and Python scalars, reductions over axes with and without keepdims, overflowing casts, rounding and clipping, and indexing with ints, slices, index arrays and masks. 4,804 cases run in about 2:30 min.

Fixes. Before them, 1,467 cases failed; now 0 do.

  • New arrays without elements get all strides 0, as in NumPy >= 2.5 (Layout::contiguous/keep_order, Layout::fresh in the issue paths). Such layouts count as C- and F-contiguous whatever their strides.
  • Reduction results are laid out in the input's axis order, and extent-1 axes take the stride of their position (reduction_result). mean/var divide the sum in place, as NumPy's _methods does.
  • asarray of a NumPy array keeps its strides where the copy can: empty and dense arrays exactly, strided ones densely with reversed axes kept reversed. NumPy's loops round differently on other strides: exp of a reversed float64 vector differed in the last bit.
  • The scalar shortcut passes plain 0-d arrays as borrowed 0-d ndarrays instead of NumPy scalars, because complex64 square differs in the last bit between the two.
  • The fast gather path now requires C strides on extent-1 axes too.

Tests. The full pytest suite passes: 6,947 passed.

Per-op cost (O1, callgrind, measured in viproc). These numbers come from the viproc branch, not from this port:

  • x * y: 4,844 instructions per op (main: 4,849)
  • x[idx]: 30,039 (main: 29,956)
  • x.sum(axis=0): 6,632 (main: 6,349)

🤖 Generated with Claude Code

https://claude.ai/code/session_01JT2yHir7oms2ybh1SL3Qu7

Every op runs on NumPy and on numpy2x with the same inputs: shapes (0-d,
empty arrays of 1, 2, 3 and 8 dimensions, vectors, matrices, 3-d and 8-d
arrays), layouts (C, F, transposed, stepped, reversed), dtypes, special
values (NaN, +-inf, -0.0, integer extremes), ready or pending inputs;
element-wise unary and binary ops with broadcasting and Python scalars,
reductions over axes with and without keepdims, overflowing casts,
rounding and clipping, indexing with ints, slices, index arrays and
masks. Values bit for bit, dtype, shape, strides, warnings, errors.

Copied from viproc (hokb/viproc#62). It finds deviations numpy2x still
has (strides of empty results, reduction result layouts, imports of
reversed arrays, the scalar shortcut on 0-d arrays); the fixes follow.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JT2yHir7oms2ybh1SL3Qu7
The parameter matrix (python/tests/test_matrix.py) found 1,467
deviations from NumPy; all are fixed, as in viproc (hokb/viproc#62):

- New arrays without elements get all strides 0 (NumPy >= 2.5):
  Layout::contiguous/keep_order and Layout::fresh in the issue paths.
  Such layouts count as C- and F-contiguous whatever their strides.
- Reduction results are laid out in the input's axis order, extent-1
  axes taking the stride of their position (reduction_result); mean/var
  divide the sum in place, as NumPy's _methods.
- asarray of a NumPy array keeps its strides where the copy can: empty
  and dense arrays exactly, strided ones densely with reversed axes kept
  reversed (NumPy's loops round differently on other strides).
- The scalar shortcut passes plain 0-d arrays as borrowed 0-d ndarrays,
  not as NumPy scalars (complex64 square differs in the last bit).
- The fast gather path requires C strides on extent-1 axes too.

The matrix now also builds its pending NumPy references the way the
pending numpy2x operands are made (np.where), and labels each failure
with its op.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JT2yHir7oms2ybh1SL3Qu7
@hokb hokb changed the title Parameter matrix test against NumPy Parameter matrix test against NumPy; strides as NumPy's everywhere Nov 5, 2026
@hokb
hokb marked this pull request as ready for review November 5, 2026 09:43
@hokb

hokb commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Instructions go up quite significantly ? Why this ? If we implement the layout calculations in C - similar or identical to how NumPy does it - where is the additional overhead coming from ?

hokb commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

The layout calculation itself is cheap (in C++, as NumPy's iterator does it). The extra ~280 instructions per x.sum(axis=0) come from three places, measured with callgrind (per op):

  • reduction_result is computed on every call, before the kernel-cache lookup: ~135 instead of ~80 for the old Layout::contiguous. The old code was not doing the same work, it was simply wrong for F-ordered and transposed inputs.
  • Layout::c_contiguous is no longer inlined. The new rule for empty arrays (contiguous whatever the strides) changed its body, and it now costs ~67 as an out-of-line call per op.
  • Layout::fresh in Runtime::issue checks every new output for an empty extent, a second loop over the shape (~60).

None of this is inherent. I am changing it now:

  1. The result layout gets cached together with the reduce kernel. The cache key already contains the input layout and axes, so a hit costs a copy.
  2. The empty check reuses nbytes(), which the issue path computes anyway for the buffer: a zero there means all strides become 0, with no extra loop.
  3. c_contiguous goes back to being inline.

Numbers follow with the push.


Generated by Claude Code

@hokb
hokb merged commit b2dd959 into main Nov 5, 2026
5 checks passed

hokb commented Nov 5, 2026

Copy link
Copy Markdown
Member Author

Done in e7f1a10. The cost is gone.

What changed:

  • The result layouts (without and with keepdims) are cached together with the fast reduce kernel.
  • The empty check reuses nbytes().
  • Layout::empty is inline.

O1 instructions per op (callgrind, this branch vs. main):

op this branch main
x * y (8 elements) 4,688 4,744
x.sum(axis=0) (10 x 10) 6,112 6,562
x.sum(axis=0), F-ordered 6,013 6,113
x[idx] (1,000 / 334) 28,322 28,333
s * 2.0 (scalar) 1,074 1,061

Repeated runs vary by about ±100 per op, so every case is at or below main within noise. The full suite passes: 6,947 passed. The same change is in viproc (hokb/viproc#62, f1ec705).


Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants