Skip to content

Add support for setting/getting ITK default number of threads - #967

Open
matham wants to merge 1 commit into
ANTsX:mainfrom
matham:threads
Open

matham wants to merge 1 commit into
ANTsX:mainfrom
matham:threads

Conversation

@matham

@matham matham commented Nov 21, 2026

Copy link
Copy Markdown
Contributor

This adds functions to Python to allow setting/getting the default number of ITK threads.

The number of threads used by ITK strongly affects the performance of ants - I did some testing below, setting the number of threads via the environmental variable and you can see how much it changes the duration of registration. Currently the only way to change the number of threads is to set ITK_GLOBAL_DEFAULT_NUMBER_OF_THREADS before ants is imported. However, you may want to set different number of threads based on the algorithm (e.g. affine seems to benefits from more threads while SyN doesn't above like 8). Also, when running ants via a GUI for end users, you typically want to allow them to set the number of threads as you go, which is after ants is imported, which isn't currently possible.

To test performance, I ran rigid/affine/SyN registration on multiple Windows 11 PCs via antspy with the following options --dimensionality 3 -r [0000026FFFCEF8E8,0000026FB83DAB48,1] --output [F:\data_T16\20260802\MNP_424M_W\results\registration_10um\MNP_424M_W_] --transform Rigid[0.1] --metric MI[0000026FFFCEF8E8,0000026FB83DAB48,1,32,Regular,1] --convergence [1000x500x250x100,1e-06,10] --shrink-factors 8x4x2x1 --smoothing-sigmas 3x2x1x0vox -x [NA,NA] --transform Affine[0.1] --metric MI[0000026FFFCEF8E8,0000026FB83DAB48,1,32,Regular,1] --convergence [1000x500x250x100,1e-06,10] --shrink-factors 8x4x2x1 --smoothing-sigmas 3x2x1x0vox -x [NA,NA] --transform SyN[0.2,3,0] --metric CC[0000026FFFCEF8E8,0000026FB83DAB48,1,2,Regular,1] --convergence [100x100x10x10,1e-06,10] --shrink-factors 8x4x2x1 --smoothing-sigmas 3x2x1x0vox -x [NA,NA] --float 0 --write-composite-transform 0 -v 1 --collapse-output-transforms 0. Where the moving volume was 1218x795x1877 at 10um resolution and the fixed template was 456x320x542 at 25um.

Besides the obvious I'm posting the results here so I can reference it later 😅 But it is quite interesting. These show the median duration of the given stage/level in seconds.

Xeon(R) w9-3475X (2.21 GHz) 72 cores (2 processors) 768GB DDR5
image

Xeon(R) Gold 6154 CPU @ 3.00GHz 72 cores (2 processors) DDR4 384GB:
image
8x2 means that I ran two runs of 8 threads at the same time, just to compare the effect of running two registrations at the same time.

Xeon(R) CPU E5-2690 v3 @ 2.60GHz 48 cores (2 processors) 256GB DDR4:
image

i7-7820X CPU @ 3.60GHz 20 cores 128GB DDR4:
image

I also looked at how down-sampling is affected by the number of threads. Where I down-sampled a volume of approximately 2000x10,000x6000 voxels from (4, 2.03, 2.03) to (10, 10, 10) microns resolution by first down-sampling each z-plane independently using 20 threads in parallel and then down-sampled the result in z. All using Gaussian interpolation. This was on the Xeon(R) CPU E5-2690 v3 @ 2.60GHz 48 cores (2 processors) 256GB DDR4:
image

Next I used the registration result to warp a 25um resolution volume containing labels to 10um resolution using the genericLabel interpolation. I had noticed this took much much longer then when doing the same with analog data so I was curious how this was effected by threads. I ran it on i7-7820X CPU @ 3.60GHz 20 cores 128GB DDR4:
image

I am mystified why the i7, arguably the weakest computer, has the best performance, excluding the first one, which is the most powerful computer. These are runs of one registration, but it very much aligns with my experience of running registration on these computers. In particularly I'm very puzzled why the Xeon(R) Gold 6154, which is quite powerful, is the slowest by a very significant amount. The python version and packages are very similar across them (I installed Python via conda and the packages via pip).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant