Skip to content

Speed up import isal.igzip by not importing the typing module - #319

Open
ptweezy wants to merge 1 commit into
pycompression:developfrom
ptweezy:lazy-typing-import
Open

ptweezy wants to merge 1 commit into
pycompression:developfrom
ptweezy:lazy-typing-import

Conversation

@ptweezy

@ptweezy ptweezy commented Nov 5, 2026 •

Copy link
Copy Markdown
Contributor

import isal.igzip and import isal.igzip_threaded also import the typing module, but the two modules use the names that they import from it only in annotations. typing in turn imports other modules, for example, functools on Python 3.14 and re, enum, and contextlib on Python 3.11, so every program that imports isal.igzip pays for them. python -m isal.igzip loads most of these modules anyway through argparse and shutil, so on the command line this PR saves only the import of typing itself.

With this PR, neither module imports typing:

  • igzip.py annotates the mtime parameter of compress() as int | None instead of Optional[SupportsInt]. Type checkers now report an mtime that isn't an int, but at run time compress() still converts mtime with int().
  • igzip_threaded.py uses list, tuple, and X | None in its annotations instead of List, Tuple, and Optional.

The test_import_does_not_load_cli_modules test is now test_import_does_not_load_slow_modules, and it also checks that typing isn't imported. That check fails on develop.

I measured the wall-clock time of new processes that run python -c "import isal.igzip", in paired rounds against develop. The Linux numbers are from Docker on WSL2, and the Windows numbers are from native builds. The table shows the medians in milliseconds:

Platform develop This PR
Linux, CPython 3.14 12.8 9.8
Linux, CPython 3.11 14.0 8.4
Linux, PyPy 3.11 30.9 21.0
Windows, CPython 3.14 40.5 34.0
Windows, CPython 3.11 40.9 27.4

On Linux with CPython 3.14, python -c pass takes 7.6 ms, so importing isal.igzip adds about 5.2 ms to the process on develop and 2.1 ms with this PR. On the five platforms in the table, python -m isal.igzip -c on a 64 KiB file finishes 1.3 to 2.4 ms sooner. A program that imports typing for other reasons saves nothing. xopen and dnaio both import typing, for example.

The test suite passes on this branch on Windows with CPython 3.14 and 3.11.

Checklist

  • Pull request details were added to CHANGELOG.rst
  • Documentation was updated (if needed)

@ptweezy

ptweezy commented Nov 5, 2026

Copy link
Copy Markdown
Contributor Author

This is very much a speedup that's tiny, but being me, I find each of them satisfying and I like how they add up when you consider users x runs

@rhpvorderman rhpvorderman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, one minor thing so it can be done with even less code.

Comment thread src/isal/igzip.py Resolved
Comment thread src/isal/igzip.py Resolved
@ptweezy
ptweezy force-pushed the lazy-typing-import branch from 3d815c7 to 7420b66 Compare November 6, 2026 15:53
Only type checkers need typing, and importing it also loads functools and
copyreg on Python 3.14, and re, enum, contextlib and warnings on 3.11.
igzip_threaded now uses built-in generics, and igzip annotates the mtime
argument of compress() as int | None instead of Optional[SupportsInt].
Under callgrind, import isal.igzip costs 10.5M instead of 28.4M
instructions on top of interpreter startup on Python 3.14, and 5.4M
instead of 38.6M on 3.11.
@ptweezy
ptweezy force-pushed the lazy-typing-import branch from 7420b66 to 4f3faf7 Compare November 6, 2026 15:55
@ptweezy
ptweezy requested a review from rhpvorderman November 6, 2026 15:57

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants