Skip to content

Let the finite-difference problems solve at a precision of their own, and by FFT - #785

Merged
pancetta merged 1 commit into
masterfrom
fd-solve-precision
Oct 8, 2026
Merged

pancetta merged 1 commit into
masterfrom
fd-solve-precision

Conversation

@pancetta

@pancetta pancetta commented Oct 8, 2026

Copy link
Copy Markdown
Member

Part of moving the reduced-precision study (#784) into pySDC proper rather than keeping it in project-level problem twins. The goal it serves: showing that an expensive implicit solver may run, and return its result, at significantly reduced precision while pySDC still converges to full precision. This PR gives the library a solver that genuinely does so, independently of how the state is stored.

What changes

GenericNDimFinDiff, and through it heatNd_unforced, heatNd_forced and advectionNd, gains two options:

  • solve_precision: the precision the implicit solve runs at, independently of the state's dtype. None, the default, runs the existing solve unchanged. When set, the solve genuinely works in that precision: 'direct', 'GMRES' and 'CG' on a copy of the operator held at it, and the result is widened back to dtype.
  • solver_type='FFT', for periodic grids. The operator is circulant, so the FFT diagonalises it and a solve is two transforms and a division. That is exact, works for non-symmetric operators such as advection, and is the cheapest solve on a GPU.

Half precision is only reachable through 'FFT', since SciPy and CuPy sparse matrices stop at single precision:

  • On a GPU the transforms are cuFFT's complex32 ones (power-of-two grids), with every value stored and every product rounded in half precision.
  • On a CPU there is no half-precision FFT, so float16 is emulated: values are rounded through it around a single-precision transform. The docstring says so; it is optimistic.
  • Range: the right-hand side is scaled to a maximum of one and the transforms are unitary, which keeps everything inside float16's range. An unscaled 1024² transform reaches 1e6; float16 stops at 65504.

Complex states solve at complex precision. That is ParaDiag's case: complex128 state, complex64 solve, complex factor.

Two fixes along the way:

  • NaN from reduced-precision Krylov solves: such a solve is not asked for less than 100·ε of its own precision. Asked for more, fp32 CG on a T4 divided by a dot product that had rounded to zero and returned NaN.
  • advectionNd now passes dtype and useGPU through. It never did, so the GPU project's statement that every FD problem became GPU-capable did not hold for advection.

Tests

New pySDC/tests/test_problems/test_FD_solve_precision.py:

  • Accuracy at every precision, for every solver type: heat (symmetric) and advection (non-symmetric), in 1D and 2D, against a direct double-precision solve. Each check has a lower bound as well as an upper one: a solve that silently stayed in double would pass the upper bound, so only the lower bound shows the reduced precision is real.
  • The complex ParaDiag case, and the two misconfigurations: float16 with a sparse solver, and 'FFT' on a non-periodic grid.
  • What the option is for: delta-form SDC (delta_implicit, linear_implicit=True) with a half-precision FFT solve reaches the double-precision residual floor. Stock generic_implicit with the same solve stalls above 1e-6. That second case is the control.
  • GPU (cupy): fp32 CG and genuine fp16 FFT on the device.

Local: black and ruff over pySDC, and 1594 base tests across problems, sweepers, transfers, tutorials and the problem helper. On 4×T4 through etc/modal_gpu_tests.py: the 17 cupy tests of test_problems pass.

🤖 Generated with Claude Code

… and by FFT

GenericNDimFinDiff gains `solve_precision`, the precision the implicit solve
runs at independently of the state's `dtype`, and `solver_type='FFT'` for
periodic grids, where the operator is circulant and a solve is two transforms
and a division. With `solve_precision=None`, the default, the existing solve
runs unchanged.

Set, the solve genuinely works in that precision: the sparse solvers on a copy
of the operator held at it, the FFT with transforms at it, and the result is
widened back to `dtype`. Sparse matrices stop at single precision, so float16
needs 'FFT'; on a GPU those are cuFFT's half-precision transforms, on a CPU an
emulation, documented as such. Complex states solve at complex precision,
which is ParaDiag's case. A Krylov solve at reduced precision is not asked for
less than 100 eps of its precision: asked for more, CG divides by a dot product
that has rounded to zero and returns NaN, as it did on a T4.

advectionNd now passes `dtype` and `useGPU` through, which it never did, so the
claim that every FD problem runs on GPUs holds for advection too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 8, 2026

Copy link
Copy Markdown

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  pySDC/implementations/problem_classes
  AdvectionEquation_ND_FD.py
  HeatEquation_ND_FD.py
  generic_ND_FD.py 456, 499
Project Total  

This report was generated by python-coverage-comment-action

@pancetta
pancetta merged commit 121512d into master Oct 8, 2026
73 checks passed
@pancetta
pancetta deleted the fd-solve-precision branch October 8, 2026 20:49
pancetta added a commit that referenced this pull request Oct 8, 2026
… of it (#786)

* Write down the correction-solve contract, and keep the sweeper's side of it

The delta form exists so that the implicit solve -- typically the expensive,
often external part -- can work in, and return, a much lower precision while
the iteration still converges to full precision. What the sweeper and the
solver guarantee each other for that to hold is now documented on Problem,
replacing the note on the two optional methods.

The sweeper's side, on the linear route: it hands the solve its right-hand
side at unit size and multiplies the result back, which is exact and keeps any
format clear of underflow -- the correction it hands over shrinks with the
iteration, below float16's smallest subnormal by 1e-10 -- and it widens a
result that comes back in any precision before scaling it, so scaling never
happens at the solver's precision. The affine part f(0, t) is removed after the
scaling: the stock solve adds it back unscaled, and the two only cancel at the
same scale, which an existing test caught. On the nonlinear route the scaled
unknown would enter f, so there the solver scales internally, as documented.

Tested with a solver that rounds to float16, solves in single precision and
returns float16: in delta form it reaches the double-precision floor, and
fails at 1.2e-5 with the scaling removed; stock SDC with it stalls. On a T4,
delta-form SDC with #785's genuine half-precision FFT solve reaches 4.9e-14
against 6.2e-14 in double.

The docstrings write the collocation residual as r, keeping epsilon for a
format's precision, as the DeltaSDC README now does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Test the contract's claim on a GPU, with genuine half-precision arithmetic

test_half_precision_solve_reaches_double_precision_in_delta_form ran on the
CPU only, where a float16 FFT solve is an emulation around a single-precision
transform. It now also runs under the cupy marker, where the solve is cuFFT's
half-precision arithmetic -- the case the correction-solve contract is written
for: delta-form SDC must still reach the double-precision floor, and stock SDC
with the same solve must still stall. One parametrisation, the same assertions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
pancetta added a commit that referenced this pull request Oct 8, 2026
Brings in #785 (solve_precision and the FFT solve on the finite-difference
problems) and #786 (the correction-solve contract), which the project moves
onto next.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
pancetta added a commit that referenced this pull request Oct 8, 2026
…lve contract

#785 gave the finite-difference problems a solve precision of their own and an
FFT solve, and #786 made the delta-form sweeper scale the linear route's
right-hand side for any solver. The project's own versions of both go:

- heat_delta keeps only what no library solver has, half precision on a sparse
  solver, which it emulates; every other precision is the library's genuine
  solve. Its CG and FFT routes and its `normalize` knob are gone.
- heat_paradiag goes entirely: ParaDiag runs on the stock heat problem with
  dtype='complex128' and the library's solve_precision. paradiag.py keeps the
  transform at reduced precision, and circulant_eigenvalues goes with it.
- run_gpu.py uses heatNd_unforced and heatNd_forced directly.
- The project's FFT tests, which the library's test_FD_solve_precision now
  covers, and the GPU fingerprint of the FFT solve go.

The heat ladder's "unnormalised fp16 solve" control no longer fails, because
the sweeper now scales for that solver -- the contract working. It is removed
from the table, and lives on in the library's test with the scaling switched
off. The heat figure's control becomes stock SDC with the same half-precision
solve (it stalls at 0.25); Allen-Cahn keeps its unscaled solver, since on the
nonlinear route the solver scales itself. The README says so.

Every iteration count in the heat table is unchanged; the fp32 rows' floors move
within round-off (7.2e-14 to 8.8e-14), their solves now genuine rather than
emulated. ParaDiag's table is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant