Repository navigation
perf: take MOD and MULMOD remainder digits without a branch on ARM64 - #186
Conversation
The first Moller-Granlund correction of a 128-by-64 reciprocal digit applies to about two digits in three, so a branch on it mispredicts. Chains that keep only the remainder (MOD by one limb, MULMOD by a 64-bit modulus, the 257-bit remainder of ADDMOD) now use URem2By1, which selects the corrected remainder. The quotient paths keep UDivRem2By1 as it was: a select there made DIV slower. The zkEVM build keeps the branch, which costs fewer guest steps. The seed table index in Reciprocal2By1 is a byte, so the lookup needs no bounds check.
LukaszRozmej
left a comment
There was a problem hiding this comment.
LGTM. URem2By1 is UDivRem2By1 without the quotient, and the quotient adjustments never feed r1. For normalised d, (byte)(d >> 55) is equal to the old (d >> 55) - 256. ReciprocalDivisionTests checks every seed entry at every shift against BigInteger, and the arm64 runners (with and without HW intrinsics) run it.
Nit: before this change, a divisor that is not normalised threw IndexOutOfRange from the bounds check. Now Release builds silently read a wrong seed, because only Debug.Assert guards it. That can happen only if there is a bug in a caller, so this is not blocking.
🤖 AI agent (Claude Code / Opus 5.5) on behalf of @LukaszRozmej
|
@LukaszRozmej Thanks. I kept it without a Release check. Every caller normalises the divisor before the lookup (shift by its leading zero count), the |
LukaszRozmej
left a comment
There was a problem hiding this comment.
Reviewed commit 67f76443021444825df2cae1bc59d03d1a6ce50e.
Reviewed remainder-only corrections against the existing quotient/remainder algorithm, normalized seed indexing, std/zkVM selection, BigInteger vectors and benchmarks. No new correctness finding; reported ARM64/guest measurements were not rerun.
Scope (5 changed files)
src/Nethermind.Int256.Benchmark/ReciprocalDivisionBench.cssrc/Nethermind.Int256.Tests/ReciprocalDivisionTests.cssrc/Nethermind.Int256/UInt256.divmod.cssrc/Nethermind.Int256/UInt256.std.cssrc/Nethermind.Int256/UInt256.zkevm.cs
Findings
None.
Optimization
Optimization pass completed; no additional actionable opportunity identified.
Validation: Static review of the PR diff, relevant callers/contracts and existing threads. No project tests, native builds or benchmarks rerun.
🤖 AI agent (Codex / GPT-6) on behalf of @LukaszRozmej
Changes
URem2By1: a remainder-only copy ofUDivRem2By1(Moller-Granlund 128-by-64 reciprocal digit). The first correction is a select (r1 = r1 > low ? r1 + d : r1) in the standard build, so the running remainder does not wait for a mispredicted branch. The zkEVM build keeps the branch (DivRemBranchlessCorrection => false), which costs fewer guest steps.URem2By1:RemainderBy64Reciprocal(MOD by one limb, and the operand reduction of MULMOD by a 64-bit modulus),Remainder128By64Bits(the product reduction of MULMOD by a 64-bit modulus) andRemainder257By64Bits(ADDMOD by a 64-bit modulus). These paths run only without the x64 hardware divide.UDivRem2By1does not change. A select there (quotient live) made DIV 6-13% slower on Neoverse-N2, so the quotient paths keep the branch.Reciprocal2By1indexes the seed table with(byte)(d >> 55): for a normaliseddthis is the old(d >> 55) - 256, and the byte index needs no bounds check. ADebug.Assertkeeps the normalisation precondition.ReciprocalDivisionBench(DIV, MOD and MULMOD by a 33- to 64-bit modulus, nine operand shapes) andReciprocalDivisionTests.Why the select: for random digits the first correction applies to 64-75% of digits (Python model of the digit, 20,000 digits per divisor class: 1e18, random 64-bit, 2- to 16-bit), so a branch on it is close to unpredictable.
Performance (arm64)
GitHub Actions
ubuntu-24.04-arm, Neoverse-N2, 4 cores, .NET 10.0.15. Base (bench: add reciprocal division shapes) and head built in the same job and run on the same runner as A1 B1 A2 B2, BenchmarkDotNet--warmupCount 3 --iterationCount 10. 0 B allocated in every case. Mean ns, error in parentheses.MULMOD by a 33- to 64-bit odd modulus (
MultiplyModBy64; the shape names the x, y operands):MOD:
DIV (code path not changed except the seed index):
A second run of the same jobs (https://github.com/NethermindEth/int256/actions/runs/43654848126) gives the same numbers within 0.5 points. The DIV-by-one-limb shapes are +0.6..+1.0% in both runs; their code is the same apart from the byte index, so I read this as a layout effect, but I cannot prove it.
Runs: https://github.com/NethermindEth/int256/actions/runs/43654808112 (arm64). In one of the four jobs of the second run the first B1 benchmark reported NA (harness); B2 of the same job and both rounds of the first run are complete.
x64: every changed call site is on the path without the hardware divide (
X86Base.X64.IsSupportedis false), so x64 runs the same code apart from the byte index. An x64 run (AMD EPYC,ubuntu-latest) of an earlier version of this change showed DIV, MOD and MULMOD flat (within +-0.7% in both rounds).zkVM guests
One run per arm (the executors are deterministic), the 11 blocks of
Nethermind.Stateless.ZiskGuest/inputs.json, Nethermind master0d776d3b096with the zkEVM build ofNethermind.Int256.dllfrom int256 base115878bor this PR swapped into each guest before bflat links it. Every output is the same as the expected output ininputs.jsonon every arm (Airbender:make verifyagainst the published digest; OpenVM: the public values). The guests take only the byte index (the select is off in the zkEVM build).ZisK
ziskemu steps.
Total cost: 194152241037 -> 194152188267 (-0.0000%).
SP1
SP1 cycles (
nethermindeth/sp1-runner:v6.9.0).OpenVM
OpenVM instret (
execute_metered_cost).Total cost: 148330045774 -> 148328230499 (-0.0012%).
Airbender
Airbender cycles.
Runs: https://github.com/NethermindEth/nethermind/actions/runs/43655824720 (all four guests, this PR head before a comment and a test were added) and https://github.com/NethermindEth/nethermind/actions/runs/43652715912 (OpenVM).
Tests
ReciprocalDivisionTests: DIV, MOD, MULMOD and ADDMOD againstBigIntegerfor divisors that cover all 256 seed-table entries (top nine bits of the normalised divisor) with low bits at both ends and random,ulong.MaxValueand2^63, and 4,000 random one-limb divisors, with dividends next to a multiple of the divisor and next to the maximum quotient. A wrong seed index (d >> 56) fails it.DOTNET_EnableHWIntrinsic=0, and the zkEVM build (602,346 / 602,346 / 602,320 tests).