Bump Cython to >=3.3,<3.4 - #2771
Conversation
github-actions
Bot
added
cuda.bindings
juenglin
added
dependencies
|
/ok to test 6f24210 |
|
mdboom
left a comment
mdboom
left a comment
There was a problem hiding this comment.
From the description:
Python 3.15 support path: Cython 3.3 adapts to 3.15 and its Limited API; our test dependency groups already carry python_version < '3.15' guards to be removed once 3.15 is officially supported.
This is related to some dependencies having not released 3.15 support or wheels yet (at the time we started Python 3.15 support). Seems totally unrelated to this PR, but we should file separate PRs to check the status of those dependencies and see if they support Python 3.15 now.
Free-threading improvements: critical sections on auto-generated functions, prange/parallel() without releasing the GIL, pymutex.locked().
These are Cython improvements? They don't seem to be related to this PR or be relevant to us since we don't use those features.
| assert int(val.frame.pArray[1]) == 0 | ||
| assert int(val.frame.pArray[2]) == 0 | ||
| val.frame.pArray = [1, 2, 3] | ||
| val.frame.pArray = [cuda.CUarray(x) for x in (1, 2, 3)] |
There was a problem hiding this comment.
This is a breaking API change. We will need to fix the cybind-generated code so this still works as-is.
| # init=False: Cython 3.3 wraps the dataclass-generated __init__ in a | ||
| # critical_section and then warns (promoted to an error by our | ||
| # warning_errors=True build) that the __post_init__ call inside it is | ||
| # unprotected Python attribute access. A hand-written __init__ sidesteps | ||
| # that; construction of a not-yet-shared object needs no lock anyway. |
There was a problem hiding this comment.
Useful comment for the review, but probably not to exist in the codebase forever. I'd just remove it.
|
We should do a benchmark run on this and make sure there are no major regressions as well. |
2 participants
Description
Bumps the Cython build/test pin from
>=3.2.5,<3.3to>=3.3,<3.4acrosscuda_bindings,cuda_core,cuda_pathfinder, and docs environment files.No motivating issue -- routine dependency upgrade.
Motivation
python_version < '3.15'guards to be removed once 3.15 is officially supported.prange/parallel()without releasing the GIL,pymutex.locked().Required code changes
Cython 3.3 added a new
critical_sectionmisuse warning (cython/cython#6766), promoted to a build error by ourwarning_errors=Truebuilds. It flagged three real patterns incuda_core:cuda/core/_memory/_buffer.pyx,cuda/core/_memoryview.pyx: narrowed@cython.critical_sectioncoverage to only thecdef-attribute check-and-set; Python-level construction/reads now happen outside the lock, since the lock never actually protected them (see cython/cython#7324 for upstream discussion of this exact limitation).cuda/core/_device_resources.pyx:WorkqueueResourceOptionsnow uses@dataclass(init=False)with a hand-written__init__, because Cython 3.3 unconditionally wraps dataclass-generated__init__in a critical section and then warns about its own generated call to__post_init__.Two
cuda_bindingstests (test_eglFrame,test_cudart_eglFrame) were updated to passCUarray/cudaArray_tobjects instead of raw ints, because Cython 3.3 now enforces declared container item types (list[CUarray]) at runtime. The setter's type annotation comes from generated code; the code generator may want to revisit it separately.Build times
Clean full-rebuild times (cache disabled, median of 3 runs) are essentially unchanged between Cython 3.2.9 and 3.3.0 for both
cuda_bindingsandcuda_coreon this hardware.Test parity
Identical results before/after on a local GPU (CUDA 13.2):
cuda_bindings: 552 passed / 26 skipped (+ 9 passed intests/cython)cuda_core: 3977 passed / 237 skipped / 4 xfailed (+ 1 passed intests/cython)Known follow-up (out of scope here)
Relocking
pixi.lockforcuda_bindings/cuda_coreto reflect the new Cython pin incidentally triggers a full re-solve of the unconstraineddefaultenvironment, which also bumps its CUDA toolchain from 12.9 to 13.3 (pre-existing lock drift relative toci/versions.yml, unrelated to Cython). That drift and its long-term fix are tracked separately in #2770;pixi.lockfiles are intentionally left untouched in this PR.Checklist