Skip to content

BitSet blocks with the dependencies' non-override mimalloc; one mimalloc per process - #7240

Merged
Fedr merged 6 commits into
masterfrom
mimalloc-from-deps
Oct 24, 2026
Merged

Fedr merged 6 commits into
masterfrom
mimalloc-from-deps

Conversation

@Fedr

@Fedr Fedr commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

BitSet stores its blocks with mimalloc explicitly, and executables that override malloc (mr_enable_mimalloc, also MeshViewer and MRTest, as on master) forward it to the same mimalloc, so every process has exactly one mimalloc heap. The mimalloc comes from the dependencies, built without the malloc override, so MeshLib libraries can be loaded late into any process (Python, .NET, Java).

An alternative to #7236, which fetches and builds its own MRMimalloc instead.

Allocator

  • BitSet::Blocks is std::vector<block_type, Allocator<block_type>>. The new MRAllocator.h defines MR::Allocator<T> as mi_stl_allocator<T>.
  • It is std::allocator<T> with the new option MRMESH_NO_MIMALLOC, named like the other optional dependencies (MRIOEXTRAS_NO_*, MRVIEWER_NO_*). It defaults to ON in Emscripten builds, where malloc is mimalloc already with MR_EMSCRIPTEN_MIMALLOC, and in macOS Homebrew builds, because Homebrew's mimalloc overrides malloc. It is a public compile definition of MRMesh, so code built against installed MeshLib sees it too, and an entry in config_cmake.h.
  • MRMesh links the dependencies' mimalloc PUBLIC through mr_find_mimalloc().
  • On MSVC, MRAllocator.h auto-links mimalloc(-debug).dll.lib, so consumers that link only MRMesh by name resolve the inline mi_free.

Dependencies

Build mimalloc
Ubuntu (thirdparty/cpm/stage1) 3.5.3 shared, MI_OVERRIDE=OFF, local-dynamic TLS, release-built
vcpkg Linux and macOS (stock port, vcpkg 2026.10.16) 3.5.3 shared, no override feature (it is Windows-only in vcpkg.json), release-only like everything in MeshLib's Linux/macOS triplets, local-dynamic TLS from the Linux triplets
vcpkg Windows stock mimalloc[override], as before (the override feature is now Windows-only in vcpkg.json)
macOS Homebrew Homebrew's override mimalloc, used only by mr_enable_mimalloc executables, as on master

The libraries never load an override mimalloc, which would crash a plain Python loading it late. Release-only builds matter because the same mimalloc can be the allocator of the whole process (below).

Executables

mr_enable_mimalloc makes malloc use the libraries' mimalloc; it needs no option, as overriding is always safe now:

  • Linux: the thin MRMimallocOverride library (source/MRMimallocOverride/MRMimallocOverride.cpp), first in the link order. It defines the Linux list of mimalloc 3.5.3's src/alloc-override.c (malloc & co, glibc's __libc_*) and every operator new/delete as calls to mi_*.
  • macOS vcpkg: MRMimallocOverride (MRMimallocOverrideMacOS.cpp), a port of mimalloc's macOS override (alloc-override.c with MI_OSX_INTERPOSE plus src/prim/osx/alloc-override-zone.c) to the public API. It uses dyld __interpose tuples and a mimalloc malloc zone. Two things differ from mimalloc's single-image override:
    • dyld applied only interposes whose replacement is in the interposing image, so each replacement is a local wrapper.
    • The library's own calls are interposed too, so realpath is left out: mi_realpath calls it, which recursed.
  • MRMESH_NO_MIMALLOC (macOS Homebrew): the dependencies' mimalloc, Homebrew's override one, as on master (the libraries use std::allocator).
  • Windows: the redirect of the same vcpkg mimalloc.dll, unchanged.

MRMimallocOverride is built on macOS and Linux unless MRMESH_NO_MIMALLOC. MeshViewer and MRTest call mr_enable_mimalloc again (MSBuild: MimallocRedirect.props), as on master, so every CI job tests the override.

Packages and bindings

  • MRMeshConfig.cmake defines the mimalloc target for code built against installed MeshLib: vcpkg's package on Windows; elsewhere the dependencies' libmimalloc copied with MeshLib (the thirdparty lib dir in the .deb, vcpkg's lib dir in vcpkg installs), with the copied thirdparty include dir.
  • bits() and fromBlocks are hidden from the bindings (MR_BIND_IGNORE), because mrbind drops custom allocators (see this C bindings error). generate.mk reads MRMESH_NO_MIMALLOC from MRMesh's config_cmake.h, so the bindings agree with MRMesh on BitSet::Blocks.
  • MRCuda builds its result blocks as BitSet::Blocks, and DynamicArray accepts a std::vector with any allocator.

Tests

MRMimallocRedirectTests.cpp:

  • MimallocRedirectActive: malloc and new return mimalloc memory. On Windows the redirect is active, and on macOS the default malloc zone is mimalloc's.
  • MimallocBitSet: BitSet blocks come from mimalloc, also those allocated inside MRMesh.
  • MimallocSharedLibrary (Linux, macOS): exactly one libmimalloc* is loaded, plus libMRMimallocOverride (unless MRMESH_NO_MIMALLOC), and mimalloc.h matches the library's version.

The file is skipped where malloc is not mimalloc (Emscripten without MR_EMSCRIPTEN_MIMALLOC, MRMESH_NO_MIMALLOC outside macOS Homebrew).

Performance

Boolean timing of an overriding executable built with this PR matches master (the dependencies' override mimalloc replacing malloc) on 8 Linux, 3 Windows and 2 macOS CI jobs, interleaved on the same machine (two samples of the Linux and Windows jobs). Measured with #7236's MRMimalloc, the same mimalloc 3.5.3 built the same way, local-dynamic TLS costs nothing measurable.

🤖 Generated with Claude Code

…loc per process

MR::Allocator (BitSet::Blocks) is mi_stl_allocator, linked from the dependencies' mimalloc
built without the malloc override: thirdparty/cpm on Ubuntu, the ports-unix overlay on vcpkg
Linux/macOS (local-dynamic TLS on Linux, release-only), vcpkg's mimalloc[override] DLL on
Windows. macOS Homebrew builds use std::allocator (MR_STD_ALLOCATOR), as Emscripten does.

With MR_MIMALLOC_OVERRIDE=ON, executables forward malloc and new to the same mimalloc through
MRMimallocOverride (Linux: definitions; macOS: dyld interposes and a malloc zone), or use the
redirect of the same DLL on Windows. CMake CI builds with MR_MIMALLOC_OVERRIDE=ON for now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr Fedr added full-ci run all steps test-pip-build Build Python wheels (and discard them) labels Oct 23, 2026
Fedr and others added 4 commits October 24, 2026 00:09
On Windows the global target made the mimalloc that mr_enable_mimalloc prepends the same
item as MRMesh's interface one, which CMake then placed after MRMesh, so mimalloc-redirect
initialized after ucrtbase and malloc was not redirected (MimallocRedirectActive). Targets
in other directories resolve MRMesh's interface mimalloc without it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…malloc imported from the copied thirdparty libs

- Code built against installed MeshLib does not see config_cmake.h (MR_USE_CMAKE_CONFIGURE_FILE
  is MeshLib's own), so the macOS Homebrew C++ examples compiled mi_stl_allocator and did not
  link (_mi_free). MRMesh now exports MR_STD_ALLOCATOR=1, the same value as config_cmake.h.
- The .deb has no CMake config of mimalloc next to its copied thirdparty lib and include dirs,
  so MRMeshConfig imports libmimalloc from there (or next to MRMesh in vcpkg installs) as #7228 did.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…talls)

Linux vcpkg installs put vcpkg's lib dir at the prefix, not next to MRMesh (lib/MeshLib).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vcpkg 2026.10.16 ships mimalloc 3.5.3, so the ports-unix overlay is gone (as in master):
the non-override build comes from vcpkg.json's Windows-only override feature, release-only
from MeshLib's Linux/macOS triplets, and the Linux triplets add MI_LOCAL_DYNAMIC_TLS for mimalloc.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
-DMESHLIB_BUILD_GENERATED_C_BINDINGS=OFF
-DMRVIEWER_NO_GTK=ON
-DMRVIEWER_WITH_BUNDLED_CURL=ON
-DMR_MIMALLOC_OVERRIDE=ON

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why override here? does it only takes effect on MeshViewer and TestApp?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it only changed MeshLib's own executables (MRTest, MeshViewer), to test the override path in CI. It is gone now: MR_MIMALLOC_OVERRIDE is removed, and MRTest and MeshViewer always override malloc again, as on master, so every CI job tests it.

Comment thread source/MRMesh/CMakeLists.txt Outdated
target_link_libraries(${PROJECT_NAME} PRIVATE libzip::zip)

# MR::Allocator, see MRAllocator.h
IF(MR_STD_ALLOCATOR)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it just client property that we always disable on our side? (mb in this case it is better to use common naming MR_NO_MIMALLOC like we do for other optional thirdparties?)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is not a client property: it is ON by default in Emscripten builds (malloc is already mimalloc there) and in macOS Homebrew builds (Homebrew's mimalloc overrides malloc, and its mimalloc.h of any version comes first in the include path). Renamed to the option MRMESH_NO_MIMALLOC with those defaults, like MRIOEXTRAS_NO_* and MRVIEWER_NO_*.

Comment thread cmake/Modules/Mimalloc.cmake Outdated
find_package(mimalloc 3.5 CONFIG REQUIRED)
endif()

if(MR_MIMALLOC_OVERRIDE AND NOT WIN32)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems a bit strange to have too different flags: MR_MIMALLOC_OVERRIDE and MR_STD_ALLOCATOR

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, MR_MIMALLOC_OVERRIDE is removed: an overriding executable now forwards malloc to the libraries' own mimalloc, so overriding is always safe and mr_enable_mimalloc needs no option. MRMESH_NO_MIMALLOC is the only flag left.

…ALLOCATOR

Overriding malloc in an executable is always safe now (it forwards to the libraries' own
mimalloc), so mr_enable_mimalloc works without an option, MRMimallocOverride is always
built on macOS and Linux, and MRTest and MeshViewer override malloc again as on master
(CMake and MimallocRedirect.props); CI needs no -DMR_MIMALLOC_OVERRIDE=ON.
MR_STD_ALLOCATOR becomes the option MRMESH_NO_MIMALLOC, named like the other optional
dependencies, ON by default in Emscripten and macOS Homebrew builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Fedr
Fedr merged commit 03b05d6 into master Oct 24, 2026
174 checks passed
@Fedr
Fedr deleted the mimalloc-from-deps branch October 24, 2026 19:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

full-ci run all steps test-pip-build Build Python wheels (and discard them)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants