Skip to content

Align feature descriptions with the catch22 C implementation - #4

Merged
benfulcher merged 3 commits into
mainfrom
fix/feature-descriptions-audit
Oct 9, 2026
Merged

benfulcher merged 3 commits into
mainfrom
fix/feature-descriptions-audit

Conversation

@benfulcher

@benfulcher benfulcher commented Oct 9, 2026 •

Copy link
Copy Markdown
Contributor

I checked each feature page against the C implementation that pycatch22 wraps (the same code as catch22/C on main), and ran pycatch22 wherever a page's claim could be tested numerically.

Corrections

  • centroid_freq: the feature returns the median frequency of the power spectrum (the first frequency at which the cumulative power passes half the total), not its centroid. The page now explains why "centroid" is a misnomer. A centroid is the power-weighted mean frequency, which a high-frequency tail pulls upward; the median barely moves. For an AR(1) process with φ = 0.9, centroid_freq returns 0.107, while the true spectral centroid is 0.27. The page also gives the units: radians per sample, from 0 to π.
  • low_freq_power: the code doesn't divide by the total power. The output reads as a fraction only because z-scoring makes the total power about 1. The "Welch" estimate uses a single rectangular window over the whole series, so it is in effect a periodogram.
  • acf_timescale: the 1/e crossing is linearly interpolated, so outputs aren't integers (uncorrelated noise gives about 0.63, not 1). A note now says that the integer example outputs come from an earlier version. The hctsa equivalent is now given as CO_FirstCrossing(x_z,'ac',1/exp(1),'continuous') (the hctsa feature firstCrossing_1e_acf_point), linked to its source. It was previously given as the integer-lag 'discrete' version, firstCrossing_1e_acf_tau.
  • ami_timescale: removed "(potentially nonlinear)". The feature is a transform of the linear ACF, and its first minimum falls near the ACF's first zero crossing. Added the cap at N/2.
  • acf_first_min and periodicity: the pages now say what is computed and returned, including the Wang et al. peak conditions and the return value of 0.
  • outlier_timing_*: added the step size of 0.01 SD. Also added the trimming step: thresholds exceeded by 2% or fewer of the points are dropped before the final median.
  • transition_variance: the series is downsampled before it is converted to symbols. Each matrix entry is the joint probability of a pair of successive symbols, not a row-normalised transition probability.
  • rs_range / dfa:
    • rs_range subtracts a least-squares line from each window, not the line joining the window's endpoints, and doesn't divide by the SD.
    • Added the range of timescales and the rule that both features return 0 when fewer than 12 timescales remain.
    • The output is the proportion of timescales before the split point, not the timescale itself.
    • "Multifractal" is replaced with "crossover".
  • embedding_dist: added the cap on τ at N/10. The distances are between embedded points one step apart, and the exponential isn't fitted: its mean is set to the mean distance.
  • high_fluctuation: fixed an inverted sentence and added a note that the 0.04σ threshold is small.
  • ami2: added how the histogram estimate is computed.
  • Minor fixes: typos, Unicode "fi" ligatures in feature names, and PD_PeriodicityWang_th0_01.

Quirks in the C code: now being fixed

Three of the quirks I found while auditing turned out to be bugs introduced when the code was ported to C. They are being fixed so that the C matches MATLAB (hctsa), in DynamicsAndNeuralSystems/catch22#63, with downstream PRs DynamicsAndNeuralSystems/pycatch22#41, hendersontrent/Rcatch22#61 and brendanjohnharris/Catch22.jl#45 (draft):

  • periodicity and ami_timescale returned the lag minus one. From v0.6.0 they return the lag itself.
  • stretch_high and stretch_decreasing counted runs not at the start of the series one too long. From v0.6.0 they count runs exactly, and stretch_decreasing treats zero differences as non-increases, as MATLAB does. This PR's stretch_decreasing page already describes that behaviour.

These pages describe what the features compute, which is the corrected behaviour. Once v0.6.0 is released, some example outputs will shift slightly (periodicity and ami_timescale up by 1, the stretch features usually down by 1), so they should be refreshed then.

Still open, and not changed here:

  • embedding_dist mixes two quantities. It compares histogram bin probabilities with exponential density values, so the result depends on the bin width. catch22 inherited this from hctsa as it was in 2018; hctsa itself now uses 'pdf' normalisation.
  • pycatch22's short names differ from featureList.txt. pycatch22 uses transition_matrix where featureList.txt and these docs use transition_variance.

🤖 Generated with Claude Code

benfulcher and others added 2 commits October 9, 2026 17:06
Audit of each feature page against the pycatch22/catch22 C source:
- centroid_freq: describe it as the median frequency of the power
  spectrum and explain why "centroid" is a misnomer; give units
- low_freq_power: no explicit normalisation (z-scoring makes total ~1);
  Welch estimate is a single rectangular window (a periodogram)
- acf_timescale: crossing is linearly interpolated; examples predate this
- acf_first_min, periodicity: state what is computed and returned
- ami_timescale: sensitive only to linear autocorrelation; relation to
  the ACF zero-crossing
- outlier_timing: threshold step and the 2% trimming of thresholds
- transition_variance: downsample before symbolising; entries are joint
  pair probabilities
- self-affine scaling: least-squares detrending for rs_range, timescale
  range, output is a proportion, "multifractal" -> crossover
- embedding_dist: tau cap, one-step distances, exponential not fitted
- typos, fi ligatures and PD_PeriodicityWang_th0_01 name

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…v0.6)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@benfulcher

Copy link
Copy Markdown
Contributor Author

Update: the code quirks listed under "Not changed here" are now being fixed so the C matches MATLAB/hctsa: DynamicsAndNeuralSystems/catch22#63, with downstream PRs DynamicsAndNeuralSystems/pycatch22#41, hendersontrent/Rcatch22#61 and brendanjohnharris/Catch22.jl#45 (draft).

I've pushed one more commit here so that stretch_decreasing is described with the MATLAB tie handling: zero differences count as non-increases. Once v0.6.0 is released, some of the example outputs on these pages will change slightly: periodicity and ami_timescale go up by 1, and the stretch features usually go down by 1. Those numbers should be refreshed at that point rather than now.

…ing_1e_acf_point)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@benfulcher
benfulcher merged commit 7c6c73b into main Oct 9, 2026
4 checks passed
@benfulcher
benfulcher deleted the fix/feature-descriptions-audit branch October 9, 2026 09:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant