Skip to content

Fix wrapper input handling (NumPy arrays, MATLAB row/column vectors, mexAll GSL paths); add CI and wrapper tests; drop Python 2 - #21

Merged
benfulcher merged 5 commits into
masterfrom
fix/wrappers-ci
Oct 21, 2026
Merged

benfulcher merged 5 commits into
masterfrom
fix/wrappers-ci

Conversation

@benfulcher

@benfulcher benfulcher commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Summary

This PR fixes the wrapper input handling found in the October 2026 correctness audit (#3, finding 14, and #4's "Code issues found"), adds the repo's first CI, and turns the Python test into real value checks, with a matching MATLAB test (#3, finding 13). The Python single-feature functions now accept NumPy arrays and any other sequence of numbers and raise TypeError for anything else, instead of SystemError. The MATLAB functions accept row or column vectors, and mexAll finds GSL itself instead of assuming /usr/local. As decided in #19, it also drops the Python 2 build.

No feature values change: the wrappers call the same C functions on the same z-scored data. This PR does not touch C/src or C/Makefile, and the README diff is limited to the usage notes these fixes make obsolete.

Closes #14
Closes #15
Part of #19 (Python 2 only; packaging and the Twitter badge are still open)

Fixed

  • Python: SystemError for NumPy arrays and non-numeric input (Wrapper input handling: MATLAB column vectors, mexAll GSL paths, Python NumPy arrays #15). Python/catchaMouse16_wrap_P3.c accepted only lists and tuples. For anything else it returned NULL without setting an exception, and it ignored PyFloat_AsDouble errors, so a non-numeric item became −1 with a pending exception; both surfaced as SystemError. The wrapper now copies float64 buffers (such as NumPy arrays, including non-contiguous ones) directly and converts any other one-dimensional sequence or iterable element by element. Strings, bytes, multi-dimensional arrays, scalars and non-numeric items raise TypeError with a message that names the bad element, for example "expected a sequence of numbers, but element 2 is str". An empty series gives NaN instead of z-scoring zero values. Memory comes from PyMem_Malloc with error checks; the redundant second copy of the input and the unused int-valued wrapper are gone. Memory use stays flat over 180,000 calls, including calls that raise.
  • Python: catchaMouse16_all. It called list(data), which for a NumPy array gives a list of NumPy scalars, so it worked, but each of the 16 features then converted those objects one by one. It now passes sequences and arrays straight through, and reads only one-shot iterators (such as generators) into a list. Bad input raises the same TypeError.
  • Python build. setup_P3.py imports setuptools rather than distutils, which was removed from the standard library in Python 3.12. When gsl-config is on the PATH, it adds GSL's include and library folders (with an rpath), so Homebrew and conda installs build without setting CFLAGS or LDFLAGS. The documented python3 setup_P3.py build then install still works (checked with setuptools 84).
  • Python 2 (Decide: README install questions (Python 2, packaging, Twitter badge) #19). Python/setup.py and the Python 2 wrapper Python/catchaMouse16_wrap.c, which only setup.py used, are removed. I kept the name setup_P3.py because the GitBook install instructions use it.
  • MATLAB: row vs column vectors (Wrapper input handling: MATLAB column vectors, mexAll GSL paths, Python NumPy arrays #15). catchaMouse16_all passed data' to the MEX functions, which required a column vector, so it failed on a column vector and the single features failed on a row vector. The MEX wrapper (Matlab/M_wrapper.c) now accepts any vector, since the data are contiguous either way, and returns NaN for an empty input. catchaMouse16_all checks that its input is a real vector and passes double(data(:)). A matrix still raises an error, now with the ID catchaMouse16:notVector.
  • MATLAB: mexAll GSL paths (Wrapper input handling: MATLAB column vectors, mexAll GSL paths, Python NumPy arrays #15). mexAll hard-coded -I/usr/local/include and -L/usr/local/lib, which fails for Apple-silicon Homebrew and conda installs. It is now a function. mexAll(prefix) uses prefix/include and prefix/lib. Plain mexAll uses gsl-config, also looking in $CONDA_PREFIX/bin and Homebrew's bin folders because MATLAB's PATH often lacks them, then falls back to /opt/homebrew, /usr/local and /usr. If GSL is not found, it gives a clear error. It adds an rpath for GSL's library folder so conda's @rpath libraries load, works from any folder, writes the MEX files next to itself, and no longer compiles C/src/main.c (the command-line program) into every MEX file.
  • CI (Add CI and real tests for the Python and MATLAB wrappers #14). .github/workflows/ci.yml runs make and make test in C/ with gcc and clang on Ubuntu and clang on macOS (the FFT bug in Fix FFT twiddle factors on GCC builds #2 was GCC-only), with GSL from apt or Homebrew. It passes only CC to make (and Homebrew's GSL folders through CPATH/LIBRARY_PATH on macOS), so it doesn't depend on the Makefile's warning flags, which Build with -Wall instead of -w #17 is changing. It also builds and installs the Python extension with setup_P3.py, as the README describes, and runs the Python tests on Ubuntu (Python 3.9 and 3.13) and macOS (Python 3.13).
  • Python tests (Add CI and real tests for the Python and MATLAB wrappers #14). Python/test_features.py used to print values for features picked via dir()[:16]. It now asserts the hctsa v1.03 reference values from C/tests/test_features.c, with the same tolerance (relative 1e-9): the 14 test_ts.csv values, plus ST_LocalExtrema on wn299 and rw437, DN_RemovePoints on int1000 and SC_FluctAnal NaN on wn150. As in the C test, CO_AddNoise and the SC_FluctAnal standard error are only checked to be finite at N = 900. It also checks the feature names against featureList.txt, that catchaMouse16_all matches the single features, and the input handling: a NumPy array, a list, a tuple, a list of NumPy floats, a non-contiguous array and a generator all give identical values, integer input works, bad input raises TypeError (single features and catchaMouse16_all), and N = 8 and empty series give NaN rather than a crash. It runs with pytest or as python3 test_features.py, and needs NumPy but no longer pandas.
  • MATLAB test (Add CI and real tests for the Python and MATLAB wrappers #14). The new Matlab/test_catchaMouse16.m checks the same reference values with row and column vectors, for the single features and for catchaMouse16_all, plus N = 8, empty input and bad input (a matrix, a char array, complex data). It needs a MATLAB licence, so it is not in CI.
  • README. The Python quick start passes a NumPy array (no .tolist()), and the MATLAB example passes the same vector to both functions. The mexAll row describes how it finds GSL. The Python 2 and distutils notes are replaced by "The Python build needs setuptools", and the README mentions both test scripts. Nothing else in the README changes.

How it was tested

  • macOS arm64 (Darwin 27), GSL 2.8 (conda-forge), Apple clang 21 and Homebrew GCC 16. make test in C/: 20/20 checks pass with both compilers (unchanged; this PR doesn't touch C/).
  • Python 3.12 (uv), setuptools 84, NumPy 2: built with setup_P3.py (both build_ext --inplace and the README's build then install in a fresh venv). test_features.py: 7/7 tests pass under pytest with -W error, and as a script. The new test run against master's wrapper fails 6 of 7 tests (SystemError for NumPy input, no TypeError for bad input), so it catches the old behavior.
  • MATLAB R2026a: mexAll compiled all 16 MEX files with GSL found through gsl-config, and again with mexAll('<conda prefix>'), both run from another folder. mexAll('/nonexistent') and mexAll with no GSL on the PATH give the "not found" errors. test_catchaMouse16: all 45 checks pass. The /opt/homebrew fallback was not tested locally (no Homebrew GSL on this machine), but CI's macOS job uses Homebrew GSL for C and Python.
  • CI (run 39563198126): all 6 jobs pass. make test gives 20 passed, 0 failed with gcc 13.3 and clang 18.1 on Ubuntu 24.04 (GSL 2.7.1) and Apple clang 21 on macOS arm64 (Homebrew GSL 2.8). The Python tests give 7 passed on Ubuntu with Python 3.9 and 3.13 and on macOS with Python 3.13.

🤖 Generated with Claude Code

benfulcher and others added 5 commits October 21, 2026 18:35
Python 2 has been end-of-life since 2020, and Ben decided to drop it (#19).
Python/setup.py was its only build script and the only user of
Python/catchaMouse16_wrap.c, so both go; setup_P3.py remains the build
script.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The single-feature functions raised SystemError for NumPy arrays and for
non-numeric items (#15): the C wrapper accepted only lists and tuples,
returned NULL without setting an exception for anything else, and ignored
PyFloat_AsDouble errors, so a bad item became -1 with a pending exception.

The wrapper now reads float64 buffers (e.g., NumPy arrays) directly and
converts any other one-dimensional sequence or iterable element by element,
raising TypeError for strings, multi-dimensional arrays and non-numeric
items. An empty series gives NaN instead of z-scoring zero values. It uses
PyMem_Malloc with error checks and drops a redundant copy of the input and
the unused int-valued wrapper.

catchaMouse16_all passed list(data) to each feature, which for a NumPy
array is a list of NumPy scalars; it now passes sequences through, and reads
only one-shot iterators (generators) into a list.

setup_P3.py imports setuptools instead of distutils (removed from the
standard library in Python 3.12), and adds GSL's include and library
folders from gsl-config when it is available, so Homebrew and conda installs
build without setting CFLAGS or LDFLAGS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
catchaMouse16_all transposed its input, so it failed on a column vector
("Input must be a column vector."), while the single-feature MEX functions
failed on a row vector (#15). The MEX wrapper now accepts any vector (the
data are contiguous either way) and returns NaN for an empty input, and
catchaMouse16_all passes data(:) after checking the input is a real vector.

mexAll hard-coded -I/usr/local/include and -L/usr/local/lib, which fails
for Apple-silicon Homebrew (/opt/homebrew) and conda installs. It is now a
function that takes an optional GSL install prefix and otherwise uses
gsl-config (also looked for in CONDA_PREFIX and the Homebrew bin folders,
since MATLAB's PATH often lacks them), then /opt/homebrew, /usr/local and
/usr. It adds an rpath for GSL's library folder so conda's @rpath libraries
load, works from any folder, writes the MEX files next to itself, and no
longer compiles the command-line program's main.c into every MEX file.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing ran the C value checks from #3 automatically, the Python test only
printed values (picking features via dir()[:16]), and MATLAB had no test
(#14).

The GitHub Actions workflow runs make test in C/ with gcc and clang on
Ubuntu and clang on macOS (the FFT bug fixed in #2 was GCC-only), with GSL
from apt or Homebrew. It passes only CC to make, so it doesn't depend on
the Makefile's warning flags. It also builds and installs the Python
extension with setup_P3.py, as the README says, and runs the Python tests
on Ubuntu (Python 3.9 and 3.13) and macOS.

Python/test_features.py now asserts the hctsa v1.03 reference values from
C/tests/test_features.c (test_ts.csv and the C test series), with the same
tolerance, and checks the input handling: NumPy arrays, lists, tuples and
generators give identical values, bad input raises TypeError, and N = 8 and
empty series give NaN rather than a crash. It runs with pytest or as a
script.

Matlab/test_catchaMouse16.m checks the same reference values for row and
column vectors, catchaMouse16_all, short and empty series, and bad input.
It needs a MATLAB licence, so it is not in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Python features now take NumPy arrays, the MATLAB functions take row or
column vectors, mexAll finds GSL itself, the Python 2 build is gone, and
the Python build needs setuptools rather than distutils. Remove the
workarounds the README described for the old behavior and mention the
test scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@benfulcher
benfulcher merged commit 42ff3a9 into master Oct 21, 2026
6 checks passed
@benfulcher
benfulcher deleted the fix/wrappers-ci branch October 21, 2026 09:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wrapper input handling: MATLAB column vectors, mexAll GSL paths, Python NumPy arrays Add CI and real tests for the Python and MATLAB wrappers

1 participant