Skip to content

Alternative: MeshLib's own fetched MRMimalloc instead of the dependencies' mimalloc - #7236

Closed
Fedr wants to merge 43 commits into
masterfrom
mimalloc-split-override
Closed

Alternative: MeshLib's own fetched MRMimalloc instead of the dependencies' mimalloc#7236
Fedr wants to merge 43 commits into
masterfrom
mimalloc-split-override

Conversation

@Fedr

@Fedr Fedr commented Oct 23, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

An alternative to how master (#7240) gets the mimalloc of MeshLib libraries on Linux and macOS. Instead of the dependencies' mimalloc built without the malloc override, MeshLib fetches mimalloc 3.5.3 (MESHLIB_PACKAGE_mimalloc in thirdparty/cpm/package-lock.cmake) and builds it in its own CMake tree as MRMimalloc. Everything else is as in master: MR::Allocator, MRMimallocOverride, mr_enable_mimalloc and MRMESH_NO_MIMALLOC.

What changes vs master

  • mr_find_mimalloc (cmake/Modules/Mimalloc.cmake), on Linux and macOS, builds mimalloc with FetchContent:
    • shared, without the override, local-dynamic TLS on Linux, release-built in every build type;
    • named MRMimalloc, so it cannot collide with another libmimalloc in the process, and installed next to MRMesh;
    • its headers isolated from the directory's include paths while it builds, and marked system for consumers.
  • MRMimallocOverride links MRMimalloc. Its comments name MRMimalloc instead of the dependencies' mimalloc.
  • MRMeshConfig.cmake imports MRMimalloc from next to MRMesh for code built against installed MeshLib.
  • MimallocSharedLibrary also counts libMRMimalloc., and ThirdpartyList.md names MRMimalloc.
  • Windows and macOS Homebrew/Emscripten (MRMESH_NO_MIMALLOC) are as in master.

The dependencies' mimalloc (CPM, vcpkg) stays: it still provides mimalloc.h to code that includes MeshLib headers outside MeshLib's CMake targets (MRPch, the mrbind bindings, installed packages). It has to be the same version as MRMimalloc.

Trade-offs vs master

  • Pros:
    • The library name can't collide with a libmimalloc already loaded in a process.
    • Any mimalloc version can be pinned independently of vcpkg's port.
  • Cons:
    • mimalloc is fetched over the network at configure time.
    • The process uses a second copy of mimalloc's headers, which must stay the same version as the dependencies' copy.

Performance

Booleans with this PR's earlier version (the same MRMimalloc and MRMimallocOverride) matched the dependencies' override mimalloc within about ±2% on 8 Linux, 3 Windows and 2 macOS CI jobs. Rebuilding MRMimalloc with initial-exec or TLSDESC TLS instead of local-dynamic changed nothing measurable.

🤖 Generated with Claude Code

Fedr and others added 26 commits October 22, 2026 13:38
@Fedr @claude
BitSet blocks use mi_stl_allocator
79d0483 
MRMesh links the same shared mimalloc as the executables, so bit sets get
mimalloc even where malloc is not redirected; Emscripten keeps std::allocator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Hide Blocks members from bindings; redirect test includes mimalloc.h
ba1d1b4 
mrbind drops custom allocators, so the generated C bindings passed a plain
std::vector to fromBlocks. mimalloc.h now comes in through MRBitSet.h, which
clashed with the test's own noexcept-less declarations on MSVC.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Windows bindings link mimalloc for BitSet's inline allocator
aaf4ce0 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRBitSet.h auto-links mimalloc on MSVC
3a917ad 
Consumers that link only MRMesh by name (the C++ examples, Windows
distribution users) failed with unresolved __imp_mi_free.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr
Merge branch 'master' of https://github.com/MeshInspector/MeshLib int…
fd61625 
…o bitset-mimalloc
@Fedr @claude
MeshLib never overrides malloc; MR_MIMALLOC_OVERRIDE for applications
b271e31 
MeshLib libraries link a non-override mimalloc (dlopen-safe thread-locals on
Linux), so mrmeshpy and other bindings load into any process; a malloc-
overriding libmimalloc loaded late crashed Python on Linux and macOS.
MRTest and MeshViewer keep the system allocator. An application sets
MR_MIMALLOC_OVERRIDE=ON: on macOS and Linux MeshLib then builds its own
override copy (mimalloc-override) shared by the libraries and
mr_enable_mimalloc executables; Windows keeps vcpkg's mimalloc[override],
whose redirect engages only in mr_enable_mimalloc executables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MeshLib never overrides malloc; MR_MIMALLOC_OVERRIDE for applications
af36ac1 
MeshLib libraries link a non-override mimalloc (dlopen-safe thread-locals on
Linux), so mrmeshpy and other bindings load into any process; a malloc-
overriding libmimalloc loaded late crashed Python on Linux and macOS.
MRTest and MeshViewer keep the system allocator. An application sets
MR_MIMALLOC_OVERRIDE=ON: on macOS and Linux MeshLib then builds its own
override copy (mimalloc-override) shared by the libraries and
mr_enable_mimalloc executables; Windows keeps vcpkg's mimalloc[override],
whose redirect engages only in mr_enable_mimalloc executables.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
No mimalloc auto-link in MeshLib's own CMake builds
59da8ce 
McpGateway reuses MRPch's precompiled header, which carried the pragma,
without linking mimalloc: LNK1104 on mimalloc-debug.dll.lib in Debug.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
macOS Homebrew builds get the non-override mimalloc from thirdparty/C…
ec56057 
…MakeLists.txt

They do not run CPM stage1, so find_package(mimalloc) found nothing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMeshConfig finds the mimalloc bundled with MeshLib distributions
fe63faf 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Install the override mimalloc next to MeshLib libraries
6f78172 
MeshInspector's AppImage could not find libmimalloc-override.so.3: the
fetched target was built but not installed with the application.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Override mimalloc also gets dlopen-safe thread-locals
1965660 
MeshInspector's start-and-exit test spawns a plain python3 that imports the
Python modules of the MeshInspector build, which link libmimalloc-override:
it segfaulted on import.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMeshConfig imports the bundled mimalloc itself
0c59741 
Its own CMake config, copied into distributions with the thirdparty libs,
points at a non-existent lib/MeshLib/include.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Local-dynamic TLS for the override mimalloc on Linux only
003f117 
On macOS mimalloc turns MI_LOCAL_DYNAMIC_TLS into -ftls-model=local-dynamic,
and MeshInspector's MRTest then aborted in dyld at launch on macOS 13 arm64.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MR_MIMALLOC_OVERRIDE: MeshLib libraries use standard allocators, as b…
27fc851 
…efore

Applications overriding malloc (MeshInspector) build MeshLib exactly as on
master: BitSet uses std::allocator, MeshLib libraries do not link mimalloc,
and only mr_enable_mimalloc executables get the override build. So plain
Python can still import such builds' modules (it segfaulted on Rocky Linux 8
with libmimalloc-override linked into them). Drops the local-dynamic TLS of
the override build, which did not help.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
mrbind bindings see MR_MIMALLOC_OVERRIDE from MRMesh's config_cmake.h
5ed6051 
They do not include config_cmake.h, so in MeshInspector builds mrmeshpy
compiled BitSet with mi_stl_allocator against an MRMesh using std::allocator:
"symbol not found in flat namespace '_mi_new_n'" on macOS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Override mimalloc built with release checks in Debug builds too
7ca8101 
MeshInspector's macOS Debug (Homebrew) start-and-exit test hung at exit,
twice; master uses Homebrew's release-built mimalloc there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Override mimalloc configured as Release, not just MI_BUILD_RELEASE
0c082ea 
mimalloc's CMake passes MI_DEBUG=2, MI_GUARDED=2 and MI_STATS=2 explicitly
for a Debug build type, so the define did not change anything.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MeshInspector builds exactly as on master; MeshLib builds its own non…
196d2db 
…-override MRMimalloc

The dependency trees (CPM, vcpkg, Homebrew) are back to master's malloc-
overriding mimalloc, and with MR_MIMALLOC_OVERRIDE (MeshInspector) MRTest and
MeshViewer override malloc again, as on master. Only MeshLib's own builds
differ: their libraries allocate BitSet blocks with a non-override mimalloc
of the same version that MeshLib builds itself on macOS and Linux, named
MRMimalloc so it does not collide with the dependencies' libmimalloc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMimalloc: own headers only, and system includes for consumers
70c4c1a 
The fetched mimalloc compiled against Homebrew's 3.5.1 headers from the
global include_directories, and its mi_stl_allocator failed the C bindings'
-Werror -Wdeprecated-copy through a non-system -I.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMeshConfig defines the mimalloc target before including MRMeshTargets
3fb4c7b 
The generated targets file rejects referenced imported targets that do not
exist yet ("referenced, but are missing: mimalloc"); MRMimalloc is found in
MR_MAIN_LIB_DIR instead of next to the imported MRMesh.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMeshConfig: thirdparty include dir for mimalloc only if it exists
7b21b8d 
The vcpkg tarball keeps thirdparty headers in the top-level include/.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMeshConfig looks for MRMimalloc next to the relocated thirdparty li…
83abfc8 
…b dir

In the extracted .deb, MRMesh's PACKAGE_MR_MAIN_LIB_DIR is the absolute
/usr/local/lib/MeshLib, while meshlib-config.cmake's directories relocate.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MR::Allocator<T> in MRAllocator.h: mimalloc or std::allocator by build
795f4cf 
BitSet::Blocks is now simply std::vector<block_type, Allocator<block_type>>.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Drop leftovers of earlier attempts from the PR
2a35442 
- generate.mk no longer links mimalloc for the Windows bindings: MRAllocator.h
  auto-links it on MSVC outside MeshLib's CMake builds.
- MRMimalloc builds with MeshLib's build type again (the forced Release was for
  the removed in-tree override copy), and mr_find_mimalloc has no TARGET guard.
- The test keeps its name MRMimallocRedirectTests.cpp; bits() keeps `auto`;
  MRCudaSolarRadiation.cpp keeps the unused vector it had on master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Linux: MR_MIMALLOC_OVERRIDE executables forward malloc to the librari…
9d50996 
…es' MRMimalloc

MRMimallocOverride, a thin shared library linked first by mr_enable_mimalloc executables,
defines malloc, new and the rest of glibc's allocation entry points (the Linux list of
mimalloc's alloc-override.c) as calls to MRMimalloc's mi_* functions. MeshLib libraries keep
linking only the non-override MRMimalloc and use mi_stl_allocator also with
MR_MIMALLOC_OVERRIDE=ON, so every process has exactly one mimalloc heap.
Linux CI builds with MR_MIMALLOC_OVERRIDE=ON for this prototype.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fedr and others added 2 commits October 23, 2026 12:19
@Fedr @claude
Temporary: CI for PRs into bitset-mimalloc
c21214e 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMimallocOverride: one C++ file, no C language needed (MeshInspector…
849ac8e 
…'s project has none)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr
Fedr changed the base branch from bitset-mimalloc to master October 23, 2026 10:19
Fedr and others added 2 commits October 23, 2026 13:19
@Fedr @claude
Revert the temporary CI trigger for PRs into bitset-mimalloc: the PR …
fe556cf 
…targets master

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MR::Allocator is mimalloc everywhere except Emscripten; macOS MRMimal…
b606b6f 
…locOverride

macOS: MRMimallocOverride interposes malloc, new and the malloc zone API with MRMimalloc,
a port of mimalloc's alloc-override.c and alloc-override-zone.c to its public API.
Windows: MeshLib libraries link vcpkg's mimalloc also with MR_MIMALLOC_OVERRIDE=ON, the
same DLL the executables' redirect uses. So MR_MIMALLOC_OVERRIDE no longer changes MeshLib
headers, and mrbind needs no -DMR_MIMALLOC_OVERRIDE.
macOS and Windows CMake CI build with MR_MIMALLOC_OVERRIDE=ON for this prototype.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr Fedr removed disable-build-windows disable-build-macos disable-build-macos-crossplatform Skip the self-hosted macOS x86_64 cross build and its distro test labels Oct 23, 2026
@Fedr Fedr changed the title Linux prototype: MR_MIMALLOC_OVERRIDE executables forward malloc to the libraries' MRMimalloc One mimalloc per process: MR::Allocator is mimalloc except in Emscripten, MR_MIMALLOC_OVERRIDE executables forward malloc to it Oct 23, 2026
Fedr and others added 5 commits October 23, 2026 13:42
@Fedr @claude
macOS MRMimallocOverride: every interpose replacement in its own image
c6016a0 
dyld applied the interposes of malloc_size (local) but not of calloc (mi_calloc in
MRMimalloc), so the Objective-C runtime aborted on malloc_size(p) = 0 at launch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MRMimalloc with release settings when it is the process allocator (MR…
db0f921 
…_MIMALLOC_OVERRIDE)

MeshViewer segfaulted at launch in the macOS arm64 Debug job: mimalloc's Debug build
type turns on MI_DEBUG=2, MI_GUARDED=2 (guard pages) and padding.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
macOS MRMimallocOverride: no realpath interpose
87ceaf6 
Unlike in mimalloc's own single-image override, MRMimalloc's calls are interposed too:
mi_realpath calls realpath, so the interpose recursed forever (MeshViewer segfaulted in
Debug and hung in Release). The system realpath allocates with the interposed malloc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Mimalloc.cmake: the reason for release-built MRMimalloc with MR_MIMAL…
7959b1d 
…LOC_OVERRIDE

The macOS crash was the realpath interpose, not mimalloc's Debug build; the release build
stays, as the dependencies' override mimalloc was release-built in Debug builds too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MimallocSharedLibrary: mimalloc.h and the library of the same major a…
8399d6d 
…nd minor version

In Homebrew builds Homebrew's include directory (-I) comes before MRMimalloc's (-isystem),
so its mimalloc.h 3.5.4 shadowed the fetched 3.5.3 one.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr
Fedr marked this pull request as ready for review October 23, 2026 13:39
Fedr and others added 2 commits October 23, 2026 16:40
@Fedr @claude
Drop leftovers: MR_MIMALLOC_IN_LIBRARIES, MR_MIMALLOC_OVERRIDE in con…
adbfb48 
…fig_cmake.h, override mimalloc in the Linux and macOS dependencies

- MR_MIMALLOC_IN_LIBRARIES was just NOT EMSCRIPTEN.
- MR_MIMALLOC_OVERRIDE changes no MeshLib header any more: only MRTest's tests read it,
  as a private definition.
- Nothing links the dependencies' override mimalloc on Linux and macOS: no CPM build,
  no mimalloc[override] in vcpkg-linux.txt and on vcpkg osx/linux (with its ports-unix
  overlay), no Homebrew mimalloc. The pin in package-lock.cmake stays for MRMimalloc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Merge origin/master into mimalloc-split-override
058560a 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
Restore the dependencies' mimalloc on Linux and macOS: it provides mi…
cf9a960 
…malloc.h

Without it, MRPch's precompiled header (MRBitSet.h -> MRAllocator.h) did not find
mimalloc.h in every Clang and macOS job; the bindings and installed packages also take
mimalloc.h from the dependencies' include directory. MeshLib still links only its own
MRMimalloc. MR_MIMALLOC_OVERRIDE stays out of config_cmake.h.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr Fedr mentioned this pull request Oct 23, 2026
Merged
@Fedr
Fedr marked this pull request as draft October 23, 2026 21:07
Fedr and others added 2 commits October 24, 2026 18:05
@Fedr @claude
Merge origin/master (vcpkg 2026.10.16) into mimalloc-split-override
f21f448 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
macOS Homebrew builds use std::allocator (MR_STD_ALLOCATOR)
d9a0b28 
Homebrew's include dir (-I) comes before MRMimalloc's (-isystem), so its mimalloc.h of any
version was compiled against the fetched MRMimalloc: Homebrew's 3.6.0 header failed
MimallocSharedLibrary (305 vs 306). On Homebrew, MeshLib libraries now use std::allocator, and
mr_enable_mimalloc executables keep Homebrew's override mimalloc as on master.
MR_STD_ALLOCATOR is MRMesh's public definition (for code built against installed MeshLib) and
in config_cmake.h (for the bindings); the version check is exact again.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr Fedr changed the title One mimalloc per process: MR::Allocator is mimalloc except in Emscripten, MR_MIMALLOC_OVERRIDE executables forward malloc to it One mimalloc per process: MR::Allocator is mimalloc except in Emscripten and macOS Homebrew builds, MR_MIMALLOC_OVERRIDE executables forward malloc to it Oct 24, 2026
Fedr and others added 3 commits October 24, 2026 22:29
@Fedr @claude
Merge origin/master (MeshLib#7240) into mimalloc-split-override
306c406 
Keeps this PR's own MRMimalloc design in every file both changed; the dependencies' mimalloc,
here only the mimalloc.h provider, is now built without the override as in master.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
MR_STD_ALLOCATOR is the option MRMESH_NO_MIMALLOC, as in master
b2b7478 
ON by default in Emscripten and macOS Homebrew builds, named like the other optional dependencies.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr @claude
No MR_MIMALLOC_OVERRIDE: MRTest and MeshViewer always override malloc…
20bdd18 
…, as on master

Overriding malloc in an executable forwards it to the libraries' own MRMimalloc, so it is
always safe: mr_enable_mimalloc needs no option, MRMimallocOverride is always built on macOS
and Linux, MRMimalloc is always release-built, and MRTest and MeshViewer call
mr_enable_mimalloc again (MSBuild: MimallocRedirect.props), so every CI job tests the override.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr Fedr changed the title One mimalloc per process: MR::Allocator is mimalloc except in Emscripten and macOS Homebrew builds, MR_MIMALLOC_OVERRIDE executables forward malloc to it Alternative: MeshLib's own fetched MRMimalloc instead of the dependencies' mimalloc Oct 24, 2026
@Fedr Fedr closed this Oct 25, 2026
@Fedr
Fedr deleted the mimalloc-split-override branch October 25, 2026 08:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@Grantim Grantim Grantim approved these changes

Assignees

No one assigned

Labels

disable-build-emscripten full-ci run all steps

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@Fedr @Grantim