Skip to content

Fix 1.1 release audit findings and complete documentation follow-ups - #286

Open
jeremymanning wants to merge 49 commits into
masterfrom
fix/1.1-release-review
Open

Fix 1.1 release audit findings and complete documentation follow-ups#286
jeremymanning wants to merge 49 commits into
masterfrom
fix/1.1-release-review

Conversation

@jeremymanning

@jeremymanning jeremymanning commented Sep 6, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

This PR fixes scoring errors, cache-write races, lost Delay features, inconsistent font selection, NumPy 2-incompatible optional minimums, and documentation gaps found during the 1.1 review. The full report follows.

Closes #284.
Closes #285.

HyperTools 1.1 release review — 2026-09-05

Reviewed draft source: 96ac8b7f43c132f6f455ad1be3ffc98e84adead5 (master and v1.1.0 at review start). Changes are submitted on fix/1.1-release-review; master and the release tag are not modified.

Findings fixed in the PR

Priority Finding Fix and evidence
High Forecast backtests reused an instance fitted on dataset 1 for dataset 2. On a sine series followed by a positive quadratic series, AutoRegressor() produced negative quadratic forecasts and MAE 2551.68, versus 0.00246 when passing the class. Deep-copy the model for each dataset; preserve caller state. Regression tests compare actual returned forecasts for class, instance, and dictionary forms.
High Scoring accepted previously fitted forecasters/imputers even when their learned state could contain the held-out values. Imputation scoring also fitted caller-owned instances. Require unfitted instances for scoring, copy them before fitting, and document the distinction from ordinary fitted-model replay. Real fitted/unfitted model tests cover both forms.
Medium Delay silently lost features when distinct pandas column labels had identical string representations (1 and '1'); a 2-column, 2-lag input produced only 2 output columns. Reject colliding labels with a renaming instruction; tests include mixed-type and duplicate labels.
Medium URL-cache temporary names used only the process ID. Threads caching the same URL collided: 82 of 100 concurrent writes failed with FileNotFoundError. Use a unique temporary file per write, clean it on error, and atomically replace the destination. Test 100 real writes across 12 threads and verify payload/metadata and cleanup.
Medium System-installed Noto Sans took precedence over the bundled Regular face, contradicting deterministic font selection and failing the existing font regression test on this machine. Register bundled faces ahead of equal-scoring system faces. A fresh interpreter with another real same-family font proves the bundled file wins.
Medium Optional dependency minimums allowed gensim 4.3 and scikit-image 0.22, predating NumPy 2 support despite the library requiring NumPy>=2. Raise floors to gensim>=4.4.0 and scikit-image>=0.23.2 in extras/dev/docs. Real minimum-version feature tests pass under NumPy 2.3.5.
Documentation Public plotting/predict/impute docstrings described shipped features as 1.2; dependency prose implied ARIMA imputation. Correct version labels and separate forecasting from imputation support.
Documentation The “convert now” forecast example still hand-wrote URL download/cache logic after the native cache landed. Use hyp.load(ARCHIVE, cache=True), regenerate and execute the tutorial. Its committed video remains byte-identical.
Tooling The browser verifier expected docs-notebooks/master, searched highlighted HTML for contiguous pip install, and demanded an autoplay call in deliberately paused Plotly animations. Validate versioned notebook links, rendered code text, loaded frames/play controls, and execute a real transition in Chromium. Allow evidence/build paths outside the checkout.

Source and regression-test map

  • Forecast/imputation ownership: hypertools/predict/backtest.py, hypertools/impute/backtest.py; tests/test_predict_backtest.py, tests/test_impute_backtest.py.
  • Cache atomicity: hypertools/io/sources.py; tests/test_load_url_cache.py.
  • Delay collisions: hypertools/manip/delay.py; tests/test_manip_delay.py.
  • Font precedence: hypertools/plot/fonts.py; tests/test_fonts_bold.py.
  • Dependency compatibility: pyproject.toml, docs/doc_requirements.txt; real minimum-version runs of tests/test_gensim_text.py and tests/test_density.py, plus packaging/optional-import checks.
  • Documentation: dispatcher docstrings, docs/optional_dependencies.rst, readme.md, CHANGELOG.md, examples/animate_forecast.py, and its executed tutorial notebook.
  • Browser verification: scripts/verify_docs_playwright.py.

Review coverage and validation

  • Reviewed 1.1.0: audit follow-ups (gallery repeats, tutorial coverage, native usage) #284 and Fold-in candidates from the 1.1 examples/tutorials audit (native features to add) #285 bodies against code, tests, tutorial/example sources, API documentation, and release evidence. The implemented API choices include Smooth(center=False) (instead of the proposed conflicting align=), alignment_score, and matplotlib-only companion=; broader animated panels and launch-example visual rewrites remain the explicitly deferred scope.
  • Combined behavioral suite: 4885 passed, 19 skipped, 2 deselected in 15m45s, with the local LSL configuration below. The subsequent optional-floor changes affect metadata only and were checked separately against the actual minimum packages and rebuilt metadata.
  • Focused behavioral/docstring checks: 94 passed. Native-example/tutorial gates: 232 passed, 8 skipped.
  • An unrestricted earlier full run: 4862 passed, 19 skipped, 18 failed. One failure was the bundled-font bug fixed above (that run had already imported the old code). The other 17 were LSL discovery on the host network. With isolated loopback discovery, all 63 LSL/audit tests passed, 1 skipped. No LSL tests or assertions were weakened.
  • The LSL run used a temporary LSLAPICFG with a private SessionID, KnownPeers = {127.0.0.1}, and machine-scoped discovery. These are the upstream-supported LSL configuration settings; no user/global network settings were changed.
  • A clean source-copy Sphinx build with -W --keep-going executed 51/51 gallery examples and passed. Rebuilt again after updating the forecast example/tutorial. Post-build processing injected versioned Colab links and updated all 51 thumbnails.
  • 166 HTML pages, no missing internal file/anchor targets in the final post-processed build; 25 tutorial notebooks, no saved error outputs. All 51 generated notebooks passed the release-install checker.
  • Real Chromium checks passed on 8 representative pages: gallery, static plots, matplotlib videos, live Plotly animation, plot tutorial, and alignment tutorial. Screenshots were inspected for layout and nonblank plots.
  • The revised forecast tutorial was freshly executed; launch clips and the forecast video are unchanged in the PR. The remaining tutorials were reviewed through stored outputs and gates, not all re-executed during this review.
  • Release/packaging/native checks: 256 passed, 6 skipped in the initial source checkout; the network-blocked manifest check was rerun unrestricted, yielding 10/10 release-readiness checks passed.
  • Optional minimums: gensim 4.4.0 and scikit-image 0.23.2 were installed into a temporary target directory, leaving the main environment dependencies unchanged. Real Word2Vec training and marching-cubes generation succeeded under NumPy 2.3.5; the corresponding feature suites passed 79 tests, 1 skipped. Upstream evidence: gensim 4.4.0 adds NumPy 2 support, and scikit-image 0.23 release notes describe NumPy 2 compatibility/builds. The older floors could retain incompatible binary builds; latest-version CI did not test this boundary.
  • Rebuilt metadata and optional-import tests: 20 passed after refreshing the editable metadata without changing installed dependencies.
  • Ruff and git diff --check passed. Existing matrix CI and tag CI on 96ac8b7f were green when inspected. PR CI validates the new branch separately.
  • Draft wheel and sdist package contents were compared byte-for-byte with the original release commit. All 110 package files in each artifact match that commit; they do not contain this PR's fixes yet.

Issues and public-release disposition

The PR resolves the remaining implementation/documentation defects discovered while checking #284 and #285. Both issues are linked for closure on merge, rather than closed before fixes reach master.

Before public release, merge the PR and perform the normal release re-cut from the resulting commit: rebuild and republish the gallery/notebook manifest, rebuild wheel/sdist, update the draft assets/tag, and confirm the release/tag gates. The manifest intentionally pins an exact source commit. The already-green draft at 96ac8b7f cannot stand in for these checks after the fixes merge.

Large-download tests remain excluded by the project's default not bigdata marker. A passing test suite is not a guarantee that every third-party service, model, or platform combination is defect-free.

Detailed local logs and browser evidence are under /tmp/hypertools-review/ (not shipped in the package).

@jeremymanning
jeremymanning marked this pull request as ready for review September 6, 2026 03:36
jeremymanning and others added 14 commits September 5, 2026 23:54
@jeremymanning @claude
refactor(backtest): hoist the Forecaster/Imputer imports out of the s…
f116dd7 
…coring loops

The ultrareview of PR #286 flagged both 'from .common import ...' lines
as loop-body imports with no circular-import reason (common.py does not
import backtest.py). Module-scope imports match every sibling module.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(predict,align): reject repeated metrics, blame t= for holdout=Tru…
f1a1e09 
…e, attribute the unscored warning, score ragged return_score input

Release-review findings on the 1.1 draft:
- resolve_metrics() now raises ValueError naming a metric listed twice
  (case-insensitively) instead of a TypeError deep in build_scores(),
  for both predict(holdout=) and impute(truth=).
- holdout=True with t=0 reports t=0 as the problem.
- the 'left N scored value(s) missing' warning uses external_stacklevel()
  like every other user-facing warning in these modules.
- align(return_score=True) works on ragged input that align() trims; the
  'before' score is computed on the row-trimmed input (documented).
Seven regression tests, each failing on the unfixed library.

Also: RELEASE_CHECKLIST.md rewritten for the 1.1.0 re-cut (tag exists as a
draft and must be moved; gallery namespace is republished wholesale; the
example smoke gate is a manual step), and the session note for this review.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
test: replace the three spy/fake-object tests added since 1.0 with re…
e47968f 
…al observables; export HypertoolsOfflineError

- tests/predict/test_common.py: the all-identical-timestamps case is proven
  to come from the live _infer_step via the exception's own traceback and
  message, not a monkeypatched spy.
- tests/predict/test_predict_multiindex.py: grouping helpers are checked
  through their real outputs and the returned per-group models, not by
  wrapping them with observers.
- tests/test_names_display.py: notebook display timing is observed on a
  real in-process IPython.InteractiveShell with the json renderer captured
  through IPython's own capture_output; go.Figure.show is no longer patched.
- HypertoolsOfflineError is importable from hypertools and hypertools.io
  like the other three exceptions (public-API pin test updated).
- scripts/generate_baseline_screenshots.py points at the roadmap note that
  exists; trailing whitespace stripped from six tracked files so
  'git diff --check v1.0.0..HEAD' is clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
test(examples): scan every example and tutorial for the native-usage …
a59f2e2 
…defect markers, allowlist the deliberate ax= demos, gate the rebuilt tutorials' output cells

Issue #284 claimed the DEFECT_MARKERS scan was tightened with a per-file
allowlist for deliberate ax= demos and that every tutorial's measured
output cells were recorded; the release review found the scan covered
only the six launch scripts and six launch notebooks.

- scan_for_defects(root) walks all 51 examples and every non-install
  code cell of all 25 tutorials (install cells hold the legitimate
  find_spec('hypertools') guard); a planted-marker test proves it.
- new ax= marker (hand-built axes, raw ax.plot/scatter, plt.subplots)
  with a counted DEFECT_ALLOWLIST so a new use in an allowlisted file
  still fails and a stale entry is reported.
- EXPECTED_VISIBLE_OUTPUTS gains the eight rebuilt tutorials; the
  ran-every-cell / right-cells / no-error-output tests cover all 14.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot,io,tools): release-review findings in the 1.1 animation, tit…
ef887ca 
…le, colour, loader and text paths

Plot (22 findings, 53 regression tests):
- palette= lists shorter than the dataset count cycle again with no hue
  (1.0 behaviour); empty palettes raise ValueError, not StopIteration;
  per-dataset {category: color} dict entries merge by name; NaN in a
  continuous hue no longer poisons the colour range; legend_kwargs
  fontsize is honoured with font=; blend categories are RGB.
- dataset_fade= and on_frame artist mutations reach the LineCollections
  drawn under a continuous hue; loop=True accepts the documented
  2(n+1)-1 rotations list; companion= panels and {index} titles advance
  monotonically under order='serial'; window_bounds.start reflects the
  comet-head window; bad companion=/dataset_fade= values name the kwarg;
  a raising on_frame during .save() surfaces its own exception; a raising
  title= leaves no orphaned animation.
- title_wrap= applies to dynamic titles and keeps explicit newlines;
  plotly draws newline titles as line breaks and reserves top margin per
  title line and size; nested tuple labels= annotate; bare-string labels,
  bad label_anchor=, non-string title entries/callables, title_color vs
  title_kwargs colour conflicts and bad {index} formats raise clearly.

IO / tools (10 findings):
- load(offline=True) opens no connection: URLs skip the seaborn listing,
  the listing fetch has a timeout and a remembered failure, uncacheable
  sources raise HypertoolsOfflineError.
- yahoo: bars carry the exchange-local trading day (gmtoffset applied).
- synthetic datasets accept RandomState, Generator, SeedSequence and
  np.integer seeds everywhere; a reused SeedSequence is reproducible;
  n_datasets rejects non-integral values; streaming=True on a
  non-Hugging-Face source raises instead of returning everything.
- text2mat: a flat list of strings is one dataset (was [(N,d),(0,d),...]
  since 1.0), ragged nested lists work, mixed inputs raise; a dict
  semantic= spec with a gensim vectorizer warns and skips.
- text_windows accepts numpy integers.
- format_data warnings are attributed to the caller's line
  (external_stacklevel), so notebooks stop printing the library path.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(manip,align): fit() returns self so sklearn-style chains work
1be634d 
Manipulator.fit() and Aligner.fit() returned None, so
Smooth().fit(x).transform(y) and HyperAlign().fit(xs).transform(ys)
raised AttributeError; Imputer.fit() already returned self. Both now
return the fitted instance on every path (documented), with a chaining
test over Normalize/ZScore/Smooth/Resample/Delay, HyperAlign/Procrustes/
NullAlign and PPCA on real data.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs: release-review documentation fixes for 1.1.0
f0e56c9 
- CHANGELOG: 'Fixed during the release review' subsection (every fix
  since the draft), predict(metrics=/per_column=/return_forecasts=),
  impute(return_imputed=), load(**source_kwargs) documented; the
  unsupported '138 examples executed' claim reworded to what the tests do.
- readme: dependency floors match pyproject exactly (pillow added);
  '1.0 API' wording updated for 1.x.
- api.rst: HypertoolsOfflineError and HypertoolsTrustError listed with
  their autosummary stubs.
- conf.py: gallery pages' furo view/edit links point at the source
  example under examples/ (auto_examples/ is gitignored, so they 404'd);
  stale chemtrails/precog comment fixed.
- tutorials.rst + market_sectors prose use model='HyperAlign' like the code.
- notebooks (sources only; re-executed separately): 'hyper' marked as a
  deprecated alias (analyze, plot); Normalize(mode='isotropic') section
  (manip); alignment illustration restored (align); legend= form for the
  hue demo (text); hyp.load('wikipedia:...') replaces the wikipedia-api
  cells (wikipedia_embeddings).
- scripts/execute_tutorial.py scrubs the executing user's home directory
  from stored outputs so notebooks stop leaking developer paths.
- tests/AGENTS.md no longer claims the hierarchy guide's doctests run in
  the suite; hypertools/io/lsl.py numpydoc underline fixed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(lsl): fold synthetic_outlet's custom 'Signal definition' section…
223d468 
… into the summary so numpydoc stops warning

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot,predict): panels, series-mode, forecast and hierarchy findin…
384fe99 
…gs from the 1.1 release review

- forecasters carry min_history (ARIMA from its order); fit raises a
  clear ValueError on a shorter history; the animated forecast schedule
  waits for enough revealed rows, so predict='ARIMA' (and model lists)
  no longer crash under animate= with a statsmodels IndexError.
- a datetime-like t= works inside hyp.plot; predict= collections work on
  MultiIndex frames; dated column-MultiIndex frames draw dates for every
  leaf under ndims=1; forecast_hue= is one value per dataset with a model
  collection; series-mode bundles match hyp.predict's shape.
- panels=: predict+truth in both panel_fit modes; shared mode keeps
  DataFrame index/column names and accepts 3-column frames; nested hue=/
  labels= narrow per panel; ndims>3 draws 3-D panels; save_path is
  normalised and validated up front; plotly panels use the one-shot
  display wrapper.
- ndims=1: per-column fmt lists; date-aware xlim on both backends; no
  'dataset 1' y label; a 3-D ax= with ndims<=2 raises; TimedeltaIndex is
  drawn in a readable unit.
- a trailing Smooth(center=False) that introduces NaN rows is reported
  as such with the min_periods=1 hint.
- docstrings: font= weights, HyperAnimation.drawn_extent/.save.
- CHANGELOG entries for all of the above.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(tutorials): re-execute the eight edited tutorials on the reviewe…
00e5a00 
…d library; gate and test updates

analyze, plot, manip, align, text, wikipedia_embeddings, projectile_kalman
and conversation_trajectories re-executed with scripts/execute_tutorial.py
(home directory scrubbed from stored outputs; no /Users/ path remains in
any tutorial). manip's output-cell gate entry gains the new isotropic
Normalize cell; the nested-hue length test asserts the new message that
names the offending sub-list; tests/test_load_offline.py loses a trailing
blank line. Full suite before these two test updates: 5171 passed, 2 failed
(these two), 19 skipped.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(tutorials): re-rendered wikipedia_embeddings video from the re-e…
f775b37 
…xecuted notebook

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): 1.1 release review session log (findings, fixes, verific…
63aa9cf 
…ation)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(windows): POSIX paths in the example scanner; cache replace toler…
9340300 
…ates Windows' transient access-denied

Every Windows job on PR #286 (run 34014888620) failed on the same ten
tests: the widened native-usage scanner compared 'docs\tutorials\x'
against its forward-slash allowlist and roster, and the URL cache's
os.replace raised PermissionError (WinError 5) when twelve threads
replaced one entry at once. The scanner now reports POSIX-separated
relative paths on every host; the cache retries the rename briefly and
accepts a concurrent writer's identical file. macOS and Linux jobs were
green on the same commit.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): three skill-compounder notes from the 1.1 release review…
d20fdde 
… (Windows paths, Windows os.replace, detached verification pipeline)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning jeremymanning mentioned this pull request Sep 6, 2026
Open
39 tasks
@jeremymanning

jeremymanning commented Sep 6, 2026

Copy link
Copy Markdown
Member Author

Second review pass on this branch (2026-09-06): the cloud ultrareview of the PR diff, then nine local reviewers over the full v1.0.0..HEAD diff (plot, io/tools/manip, predict/impute/align/core, packaging/CI, documentation) and item-by-item verification of #284 and #285. Every confirmed finding is fixed on the branch with a regression test; the fixes are listed in the CHANGELOG's "Fixed during the release review" subsection. The commits since the original report:

  • f116dd7 module-scope imports in the backtest loops (the ultrareview nit)
  • f1a1e09 duplicate metrics=, holdout=True, t=0 message, warning attribution, ragged return_score
  • e47968f three spy/fake-object tests rewritten against real observables; HypertoolsOfflineError exported
  • a59f2e2 native-usage scanner covers every example and tutorial, with the ax= allowlist and output-cell gates the issue described
  • ef887ca 32 plot/io/text fixes: offline=True was still opening connections, yahoo: dates east of UTC, synthetic seed types, short palette= lists (a 1.0 regression), dataset_fade= under a continuous hue, loop=True rotations, companion= under serial order, title_wrap on dynamic titles, plotly newline titles, text2mat on a flat list of strings
  • 1be634d fit() returns self on the manipulator and aligner bases
  • f0e56c9 documentation: changelog completeness and accuracy, readme floors, API exceptions, gallery edit links, tutorial prose and code
  • 384fe99 ARIMA under animate= (per-model min_history), datetime t= inside plot, predict= collections on MultiIndex frames, panels= with truth=/labels/nested hue, series-mode fmt=/xlim=/y label, TimedeltaIndex
  • 00e5a00, f775b37 eight tutorials re-executed on the reviewed library, home paths scrubbed from stored outputs
  • 9340300 Windows: POSIX paths in the scanner, retrying os.replace in the URL cache
  • 63aa9cf, d20fdde session and project notes

Local verification on the final library: full suite 5171 passed / 0 failed (after the two stale expectations were updated), ruff clean, sphinx -W over the full gallery with zero warnings, the example smoke gate 344 passed. PR CI on the branch head d20fdde: every job green (release-gate skipped by design on a branch).

After merge the release is re-cut from the merge commit; RELEASE_CHECKLIST.md now describes moving the draft tag, republishing the gallery namespace and replacing the draft release assets.

@jeremymanning jeremymanning mentioned this pull request Sep 6, 2026
Open
38 tasks
jeremymanning and others added 10 commits September 6, 2026 03:39
@jeremymanning @claude
docs(notes): end state of the 1.1 release review session
0e4f57b 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot): a marker+line format string keeps its marker in the legend
45fe779 
The matplotlib backend draws 's--' as a smoothed line artist (which
carries the legend label) plus a markers-only artist at the raw sample
points labelled _nolegend_, so the legend handle showed only the dashes
(reported from the 1.1 feature tour, section 9.2). The line artist now
carries the marker with markevery=[]: its legend handle shows marker and
line while it still draws no markers along the interpolated vertices.
Pixel-level regression test in tests/test_plot_fmt_split_legend.py.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): feature-tour follow-up (legend marker fix, mixture demo …
087e338 
…data)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot,reduce): aspect-aware panels=True grid, no second pipeline f…
abe9295 
…it under return_model, quiet third-party reducer warnings

Reported from the 1.1 feature tour (section 9.8): a three-panel grid came
out 2x2 with a hole and, in a 9x3.2 in figure, each square 3-D axes
shrank to the short cell height; the cell also printed fifteen warnings.

- panels=True picks the grid from the figure's aspect ratio and prefers
  a grid with no spare cell (three panels form a row; four 2x2; six 2x3;
  five still 2x3 with one hidden cell). Explicit grids are unchanged.
- return_model=True reuses the pipeline analyze() fitted for the figure
  (the cluster stage, which runs on the reduced scores, is appended as a
  fitted step) instead of refitting every stage, so a UMAP/Isomap plot,
  and every panel grid built with it, fits and warns once.
- a seeded UMAP passes the n_jobs=1 umap forces anyway, so umap stops
  warning about a seed hypertools injected; a caller's own n_jobs= still
  reaches umap (and its warning).
- Isomap fits silence scipy's SparseEfficiencyWarning burst from
  sklearn's internal graph completion; sklearn's own connected-components
  data warning still reaches the user.
Seventeen new tests; two older panel tests updated to the new grid rule
(the hidden-spare property now covered with five panels).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): 9.8 panels follow-up
edc7d28 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot): truth= legend glyph shows its markers; 2-D density grids s…
1862086 
…pan the whole scene

- The truth curve keeps marker='o' with markevery=[] so its 'truth'
  legend entry is line + marker, no longer identical to the observed
  trace's entry (feature tour 9.11).
- New density.scene_bounds_2d: every 2-D KDE grid covers all datasets'
  padded bounds plus the unit frame square, on both backends, so a wide
  flat cloud's glow fades out instead of stopping in a hard band inside
  the frame (feature tour 9.14).
- Tests for both; projectile_kalman and stock_forecasting tutorials
  re-executed (truth= legends), which also refreshed the stock snapshot
  CSVs by one trading day as the notebook is designed to.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
feat(plot): panels= parity on plotly, hyp.subplots(backend='plotly') …
22219eb 
…+ ax=<cell>, per-panel colorbars on matplotlib

- plotly_backend.transplant_panel moves a drawn single-axes figure into a
  make_subplots cell whole: traces, the 2-D axis layout (unit frame,
  hidden ticks, DataFrame-column labels, or axis_scale='data' axes), the
  frame square and labels= annotations re-referenced to the cell, its own
  legend (plotly multiple legends) and its own colorbar beside the cell;
  3-D cells back the camera off so the cube stays inside a narrow cell.
- make_panel_grid reserves a gutter beside every cell for those legends
  and colorbars (default-sized grids widen by it; explicit size= verbatim).
- hyp.subplots(backend='plotly') returns the grid figure plus PlotlyCell
  handles that hyp.plot(..., ax=cell) draws into (title= becomes the cell
  title); several cell calls display the grid once per notebook cell.
- matplotlib: a colorbar drawn into a caller-supplied ax= (every panels=
  cell, every hyp.subplots axes) uses fig.colorbar(ax=...) instead of the
  figure-widening placement, so panels no longer stack their colorbars
  over the last panel or trip tight_layout warnings.
- Tests: tests/test_subplots_plotly.py (new), 11 more in
  tests/test_plot_panels.py. CHANGELOG entries; RELEASE_CHECKLIST.md
  brought up to date (suite size, Colab tour smoke, announce step).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot,predict): red-team round 1 -- panel grid backend, density gr…
a5fb653 
…id resolution, sparse ARIMA orders, plotly panel titles/fonts/left colorbars

- panels= on matplotlib under an active plotly preference built its grid
  with subplots()' new backend='auto' default; the grid is now explicitly
  matplotlib.
- 2-D density grids pad by four kernel widths past the data instead of
  spanning the whole scene: the glow still fades out inside the grid, and
  a small cloud beside a 10,000x larger one keeps its resolution (the
  scene-wide grid sampled it to all zeros).
- ARIMA.min_history_for accepts statsmodels' sparse lag orders
  (order=([1, 3], 0, 0)), counting the highest lag, as the fitter does.
- plotly panels: titles go through the single-axes title path (newlines,
  title_wrap=, title_kwargs=) and become cell annotations; the panel's
  font= travels with its legend and becomes the grid default; a
  location='left' colorbar stays on the cell's left; drawing into a 3-D
  cell twice keeps the earlier labels= annotations.
- Tests for each; a rendered-grid test now checks ink in both cells.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(predict,plot): red-team round 2 -- fitted-model reuse floor, plot…
e9dd085 
…ly cell title placement/replacement, per-cell fonts

- predict_new only holds the refit path to the fit-time minimum history;
  a model with an applier (ARIMA) reuses its learned parameters on any
  context the new data offers.
- plotly cells: the title annotation follows the title's own x/y/anchors
  mapped into the cell, is replaced (not stacked) when the cell is drawn
  into again, and reserves the multi-line top margin the single-axes
  path computed; the panel's font= is materialized on the cell's legend,
  title, axis titles/ticks and colorbar under explicit overrides.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): panels parity + red-team rounds 1-2 log; skillnotes
74a888d 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
jeremymanning and others added 23 commits September 7, 2026 02:09
@jeremymanning @claude
fix(plot): forecast-schedule projection ignores timer noise from 2- a…
b1382d8 
…nd 3-row fits

The slow-schedule notice extrapolated a per-row slope from the first two
timed fits, one row apart at 2 and 3 rows (tens of milliseconds each); on
a slow macOS CI runner that projected 10 s for a 30-row schedule and
tripped the 'small schedule stays silent' test. project_schedule_cost now
fits every timed length by least squares, and the schedule waits for a
timed fit of at least PROJECTION_MIN_ROWS (10) rows before projecting.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot): forecast styling and legend entries, plotly panel geometry…
ab5d7ce 
…, and the figure-review gaps

Jeremy's three findings from the 1.1 feature tour:

- A collection of models (predict=[...]) keeps each dataset's colour and
  takes a linestyle per model (forecast.FORECAST_MODEL_LINESTYLES);
  forecast_palette= on a collection colours by model instead. The old
  per-model 'husl' default reused the first dataset's colour, so two
  datasets under two models were indistinguishable pairs.
- Every predict= form lists its forecast once in the legend under the
  model's name, static and animated, on both backends, in the order
  data / forecasts / truth: proxy glyphs (matplotlib Line2D handles,
  plotly data-free traces tagged meta['hyp_legend_entry']), neutral
  gray when one model's forecasts span several colours. Plotly truth=
  traces mark every observation instead of every antialiased vertex.
- The plotly panels= / hyp.subplots grid is laid out like tight_layout:
  square centred 3-D cells, PANEL_GAP_PX gaps, title room per row, and
  a camera back-off only for cells narrower than tall
  (SCENE_CUBE_WIDTH_PER_HEIGHT 1.4 -> 0.92: plotly sizes a scene by
  its domain height; the old constant was measured against the cube).
  Cube widths now match the matplotlib grid's within a few pixels.

Three-role review of the tour's 51 figures (expected / observed /
adjudicated by separate agents) then found eleven more gaps, all fixed
with tests: ax= axes and matplotlib panels= cells draw in the palette
(they kept their figure's default cycle) and a second call into the
same axes or plotly figure continues it; default-size panels= figures
widen for their legends/colorbars; 3-D axis labels are inside the tight
bbox (_AxisLabelExtent); recoloured forecasts keep their trace's alpha
and the legend glyph never drops below 0.8; the 2-D frame square has a
12.5 % margin (UNIT_FRAME_SCALE) on both backends; legend_kwargs loc=
drops the outside-right anchor; plotly legend keys use a constant item
size; hyp.subplots(backend='plotly') grows its legend gutters only when
a cell brings a legend or colorbar (ensure_panel_gutter).

Tests: test_plot_forecast_legend_style, test_plot_panels_geometry (real
renders, cube fill within 6 %), test_figure_review_gaps; existing suites
retargeted to the new rules. Tutorials projectile_kalman and
stock_forecasting re-executed (their legend names no longer fold the
forecast in). CHANGELOG, docstrings and the session note updated.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(plot): Codex round-3 findings -- collection ownership under regro…
f9ccc75 
…uping, fitted multi-dataset animation, plotly forecast_fmt, legend_colors contract, panel layout

Two majors: a collection of models under hue=/cluster= regrouping only
resolved forecast ownership when the forecast and run counts differed,
so one dataset split into two runs under two models drew Kalman on the
earlier run and ARIMA on the final one, and the animated modes indexed
the reveal schedule by forecast (model-major) index instead of source
dataset, raising IndexError with forecast_trail=; a forecaster fitted
on several datasets could not animate because the schedule forecasts
one history at a time (Forecaster.for_dataset binds the view it needs).

Minors: plotly honours a colour letter and markers in forecast_fmt
(traces and legend keys); plotly animations keep a recoloured
forecast's alpha; a second call into the same plotly cell continues the
palette; plotly legend keys compare RGB, not RGBA; legend_colors pairs
define the legend outright on both backends and a plain colour list
recolours the final legend after the forecast/truth entries; an
attached right colorbar is padded past the panel legend's overhang;
plotly grid gutter rebuilds keep multi-line title room and explicit
legend_kwargs positions are translated into the cell; the animation
guide's forecast sections describe the 1.1 rules; the tight-bbox test
asserts four-sided containment.

Tests: tests/test_plot_review_round3.py (19), every scenario from the
review's own reproduction scripts.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning
docs(notes): round-3 outcome, round-4 scheduling
49441a3
@jeremymanning @claude
fix(plot): Codex round-4 findings -- pinned forecast_fmt colours, mix…
687e98c 
…ture-hue overlay legends, composed calls, plotly cell furniture

- A forecast_fmt colour letter pins the forecast colour in regrouped
  animations on both backends (only forecast_hue/cluster/palette counted
  before), and plotly's per-frame colours keep a recoloured forecast's
  alpha.
- Mixture-hue legends list forecasts and truth: a matrix hue clears
  legend= and builds swatch entries, which the overlay gating did not
  count as a legend; plotly_draw(legend_explicit=) separates explicit
  legend_colors pairs (which suppress the entries) from swatches.
- Repeated calls into one axes, figure or grid cell compose: the
  overlays style themselves from this call's lines, truth artists are
  unlabelled and tagged, and the legend is rebuilt by role (data entries,
  one key per model over every call's forecasts, one truth) on both
  backends.
- Plotly cells track their furniture per cell, so a legend and a
  colorbar from separate calls sit side by side and the gutter fits the
  busiest cell; ensure_panel_layout rebuilds the rows for a taller title
  immediately.
- A marker-only forecast_fmt draws markers on plotly; animated plotly
  collection traces tag their source dataset.

Tests: tests/test_plot_review_round4.py (15), from the review's own
reproduction script.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning
docs(notes): round-4 pushed, round-5 blocked by the Codex usage limit…
f4f7ab8 
…; index-confusion note
@jeremymanning @claude
feat(api): hyp.set_autoinstall() switches on-demand extra installatio…
4c4b288 
…n; weed out stale install instructions

- set_autoinstall (class in _shared.lazy_import, exported at the top level) turns
  the on-demand installation of optional extras on or off, called directly for
  the session or as a context manager for one block, like set_interactive_backend;
  HYPERTOOLS_AUTO_INSTALL only sets the starting value.
- New static gate: every library module that imports a package an extra provides
  calls lazy_import for it in the same file (three stated exemptions).
- Stale 'install it first' prose removed from the plot() reducer docstring, the
  autoencoder/gensim gallery examples and the LSL tutorial; the reduce() torch
  error says the on-demand install was tried.
- Docs: API reference 'Set autoinstall' section + stub, optional_dependencies
  'Turning it off' rewritten around the call, index/readme/CHANGELOG updated.
- scripts/execute_tutorial.py clears a skipped install cell's outputs; the two
  tutorials that shipped a pip notice with a local path are cleared, and a gate
  forbids published install cells from carrying output.
- tests/test_lsl_streaming.py: the by-type test uses a unique stream type (other
  processes on the host advertised idle EEG outlets).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix: address the paused Codex release audit (offline built-ins, expor…
3f4b087 
…t install policy, panel bundles/partitioning, doctest gate)

Findings 1-6 and the observations in notes/release_audit_2026-09-07_paused.md
(original text kept; an 'Updates by the Claude session' section records each fix):

- load(<built-in>, offline=True) serves only a hash-valid cached copy and raises
  HypertoolsOfflineError for a missing/corrupt one without downloading or deleting
  (real socket.connect audit-hook tests).
- set_autoinstall(False) reaches the plotly animation-export worker: the parent
  passes its effective setting through lazy_import.subprocess_env (tested in a
  real interpreter without kaleido); the _pip_install spy test is replaced by a
  real-observable one.
- panels=: shared-fit bundles expose the fitted pipeline (top level and per
  panel); every per-dataset / per-forecast argument is sliced per panel and a
  multi-dataset forecaster is bound with Forecaster.for_dataset (33 tests).
- A list of {category: color} dicts works once hue= regroups the datasets.
- alignment_score rejects 1-D / non-numeric / NaN / all-constant input clearly.
- hypertools.load docstring example is self-contained; docs/conf.py takes
  HYPERTOOLS_DOCS_PLOT_GALLERY=0 as a real boolean and the docs-clean CI job
  runs the sphinx doctest builder; docs/doc_requirements.txt floors match
  pyproject; HypertoolsOfflineError docstring covers built-ins.

Local: pytest 5363 passed, ruff/packaging clean, sphinx -W 0 warnings,
doctest 316/0, smoke gate 344.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix: Codex round 6 findings (overlapping set_autoinstall scopes, expo…
c700c85 
…rt exception type, panel palette slots) + plotly fmt colour and 2-D panel cells

- set_autoinstall: a lock-guarded scope stack replaces the saved global; a
  with-block removes only its own setting, the newest in force decides
  (process-global, documented); deterministic and two-thread tests.
- Plotly animation export re-raises the worker's ImportError (missing extra
  with installation off) / HypertoolsIOError (no Chrome) as that type via a
  .worker-error.json marker instead of a RuntimeError wrapper.
- panels=: one forecast_palette slot per LABEL (labels sharing a colour kept);
  every per-dataset roster entry has a behavioural partitioning test.
- Plotly honours the colour letter of a data fmt= string like matplotlib;
  panels= on 2-/1-column data without ndims= draws 2-D cells on both backends.
- tests/test_lsl_streaming.py: the under-load child budget is 900 s (the CI
  ubuntu 3.13 job hit 300 s where the same run's 3.10 job took 38 s).
- Audit notes carry Codex's round-6 section and the matching UPDATE entries.

Local: pytest 5460 passed, ruff/packaging clean, sphinx -W 0 warnings,
doctest 316/0, smoke gate 344.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): round-6 fixes pushed as c700c85; Codex round 7 launched
c88bc03 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): Codex round 7 cut off by the usage limit; detached relau…
339ce05 
…nch at 05:44

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): CI green on c700c85
081f7c9 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix: Codex round 7 (panel cells from analyzed data, palette slots con…
14e0965 
…sumed, bounded set_autoinstall scopes) + nits; HypertoolsTrustError exported

- panels=: every mode fits through one probe and draws each panel from the
  analyzed rows, so cell dimensionality follows manip/pipeline/reduce output
  (Delay -> PCA ndims=3 on 2-column data draws 3-D cells again); requested
  ndims and per-panel pipelines kept, no second fit.
- Composition palette offsets count only cycle-coloured datasets (fmt colour
  letters, explicit color=, hue= consume none), same on both backends.
- set_autoinstall: a new setting replaces a superseded one no block holds
  open (bounded records, no retained handles); block semantics unchanged.
- HypertoolsTrustError importable from hypertools; api.rst documents it and
  io.synthetic_outlet under their public names (viewcode backlinks resolve).
- Tests: tests/test_plot_review_round7.py (32), lazy_import retention +
  hand-off asserts, export ON-branch exception type.

Local (this tree): pytest 5493 passed, ruff/packaging clean; sphinx/doctest/
smoke to be re-run on resume (build interrupted for a machine suspend).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
docs(notes): resumed; docs pipeline green on 14e0965
1cad1e6 
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix: Codex round 8 (construct-then-enter scope race, panel cluster re…
650808f 
…play, plotly offset, mixed-width panels) + transient TLS-drop classifier

- set_autoinstall: live handles as weakref records plus a baseline folded in
  by the callback of a handle that dies unentered; newest call by order
  decides; a block only removes its own record; nothing retained.
- panels=: cells replay the probe's fitted clustering (seeded memberships
  equal the individual call's; no extra clusterer fits); bundles carry
  models['cluster_labels']; plotly composition keeps the palette offset
  across a color=/categorical-hue call; 1-/3-column independent panels draw
  the series on the 3-D cell's floor.
- tests/_netskip: an SSLError with a TLS-drop cause is transient; a
  certificate failure is not (CI ubuntu 3.11 Dropbox drop).
- HypertoolsTrustError exported; tests/test_plot_review_round8.py (52).

Local: pytest 5547 passed, ruff/packaging clean, sphinx -W 0, doctest 316/0,
smoke 344.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
feat: sorted image palettes, data-matrix palettes (palette_* options)…
8066308 
…, datawrangler-based input coercion (wave 0), Codex round 9 fixes

- Palettes: image colours sorted by value when used as a plot palette
  (palette_sort= / ?sort=; lead colour stays salient); a t x k matrix is a
  palette via hyp.reduce (palette_reduce/manip/normalize/align) -> scaled,
  sorted, MatrixColormap (exact interpolation); most-saturated anchor leads a
  per-dataset entry. Docs (api.rst, tutorials.rst, plot.ipynb section), tests.
- Datatype wave 0: format_data / get_type / as_dataframe / predict+impute
  normalisation / is_stream on dw.zoo predicates and dw.wrangle; polars
  DataFrame/LazyFrame/Series accepted; 46 real-polars tests (4 strict xfails
  name the wave-1 sites).
- Codex round 9: shared cluster colours kept per panel, narrow panels
  forecast in their own space, marker-only hue consumes no palette slot,
  live certificate failures never skipped (tests/test_plot_review_round9.py, 57).

Checkpoint commit: per-suite runs green; the full pipeline runs before the push.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
feat: datawrangler-based datatype handling everywhere (wave 1), manip…
25b3ba6 
… aligns unnamed+named lists by position, palette lead colour fix

- plot: index/column capture, hue=/labels=/truth=/matrix palette=/panels= and
  axis/legend labels through the shared predicates (polars DataFrame/
  LazyFrame/Series accepted; a 1-column label DataFrame hue no longer raises).
- manip (+ Manipulator classes, Pipeline steps), align, stack, damage,
  apply_model, Normalizer, save/load, text2mat, impute backtest truth=/mask=:
  funnels with backend='pandas', post-funnel re-checks removed, predicates
  instead of pandas/numpy isinstance ladders; tests/test_datatype_gate.py keeps
  hand-rolled type checks out (allowlisted option/model checks).
- manip([array, named frame]) aligns by position like plot/reduce/align
  (failed inside datawrangler before, pandas included); named frames with
  different labels raise a clear ValueError.
- palette_lead_color: a bare colour ('red') resolves again (the matrix-lead
  branch had swallowed it); most-saturated anchor leads a matrix palette.
- tests: test_polars_inputs.py (xfails lifted), test_polars_inputs_wave1.py (51).

Local: pytest 5754 passed, ruff/packaging clean, sphinx -W 0, doctest 316/0,
smoke 344.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix: MatrixColormap honours the Colormap contract (Codex round 10); C…
767e327 
…I-only test fixes (readme.md case, live-source-gate mode)

- MatrixColormap builds the parent's channel segment data (as from_list), so
  integer sampling, resampled(), reversed(), bad/masked input and alpha arrays
  work; contract test added.
- tests/test_format_data.py opens the repo's lowercase readme.md (macOS
  resolved README.md, the Linux CI jobs did not); the classifier test pins
  both guard modes (the live-source-gate job sets
  HYPERTOOLS_REQUIRE_LIVE_SOURCES=1 and re-raises by design).

Local: pytest 5755 passed, ruff/packaging clean, sphinx -W 0, doctest 316/0,
smoke 344.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning @claude
fix(review): Codex round 11 -- manip mixed lists keep indices, Matrix…
faedf14 
…Colormap under/over/bad per element, exact image interpolation

- manip: a list mixing unnamed arrays with named DataFrames now aligns on
  positional COLUMN labels only (frames keep their index; named-frame lists
  are left untouched; unequal widths untouched) -- R11-2
- MatrixColormap: under/over/bad applied per element, so masked/NaN and
  out-of-range values render like any LinearSegmentedColormap -- R11-1
- image palettes above 256 colours interpolate exactly instead of through
  the 256-entry LUT -- R11-3
- polars forecast_hue under panels= matches the pandas result -- R11-4
- prose fixes and stronger assertions in the palette/polars tests;
  datatype gate allowlist narrowed for the forecast_hue exemption
- notes: UPDATE sections after Codex round 11; CHANGELOG bullet

Local verification (full_verify19): pytest 5761 passed / 19 skipped,
ruff + diff --check clean, packaging 13, sphinx -W 0 warnings, doctest
316/0, example smoke gate 344 passed.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@jeremymanning
chore: checkpoint local release work before renewed audit
0a2cc0d
@jeremymanning
chore: checkpoint reviewed release fixes before backtest alignment
2e9669d
@jeremymanning
fix: score forecasts at observed times and isolate animation models
c033e85
@jeremymanning
fix: prevent overflow in integer forecast timestamps
4e825a4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

1 participant

@jeremymanning