pooled_buffers is still a preallocation: one zeroed slab instead of N zeroed Vecs

[proggeramlug]

Split out of #88, which is otherwise addressed by #89.

Problem

BufferPool::new builds and zeroes every buffer at construction:

inner: Rc::new(RefCell::new((0..count).map(|_| vec![0; size]).collect())),

So a Loop that has not opened a socket still pays for its whole read pool — 4 MiB at the default 256 × 16 KiB, 1 MiB at a 64-buffer host profile. Measured as an RSS delta across Loop::new, one Loop per process: setting pooled_buffers = 0 saves 1 008 KiB on a 64 × 16 KiB profile, so those pages really are resident.

It is the same shape slots.rs already names — "building every slot at construction makes that number a preallocation instead of a ceiling" — and the same one #75 fixed for handles and #89 fixed for ring slots.

The obvious fix is wrong

Minting buffers on demand up to count fails the allocation contract, and correctly so. I tried it (measured −0.97 MiB) and dropped it: it moves an allocation into the first read, which fails

• turnloop-contract/tests/allocations.rs::file_readiness_survives_pool_backpressure_without_allocations_or_spin ("lease return storage", left: 1, right: 0)
• turnloop-contract/tests/allocations.rs::extra_child_descriptor_traffic_allocates_nothing_after_spawn (left: 2, right: 0)

"Operations allocate nothing" is a contract this project advertises, and warming the pool inside those tests to get past it would be weakening a real gate rather than fixing anything.

Suggested fix

Keep the buffers preallocated — contract intact — but allocate the pool as one contiguous alloc_zeroed slab of count * size bytes instead of count separate vec![0; size].

The distinction that makes this work is allocator behaviour, not Rust semantics: vec![0u8; 16384] does reach alloc_zeroed, but a 16 KiB request is served from malloc's bins and has to be memset, so it is resident immediately. A single count * size request (1 MiB, 4 MiB) is large enough to be served by fresh mmap, which is zero-filled lazily by the OS — so the slab costs address space at construction and resident pages only as buffers are actually used.

Nothing is allocated at run time, so every allocation contract keeps passing unchanged.

Cost

BufLease would hold a slab offset rather than a Vec<u8>. Its data: Option<Vec<u8>> is already private and there is no into_vec, so the public API (as_slice, writable, set_len, release) does not change — but the pool and lease become unsafe aliasing code over a shared slab, on a hot path. That is why it is its own issue rather than part of #89: it wants its own review, and probably a loom model or at least a test that two live leases hand out disjoint, non-overlapping regions.

Why it is worth it

Perry's A/B against its previous tokio wait driver has turnloop winning every memory row except the idle floor, where it started 3.61 MiB above tokio. #89 takes that to +0.36 MiB. This last megabyte is what flips the remaining row, and it is the difference between "turnloop wins on memory above ~22 connections" and "turnloop wins on memory, full stop".

[proggeramlug]

The suggested fix above is not robust enough, and here is the measurement that shows why

I proposed "one contiguous alloc_zeroed slab" on the grounds that a large zeroed allocation is mmap-backed and lazily faulted. That reasoning is right, but it rests on the allocator's mmap threshold, which is a tunable rather than a contract — and the threshold is not the same on the two platforms Perry ships.

Measured by sweeping pooled_buffer_size at a fixed 64 buffers, one Loop per process:

pooled_buffer_size	total pool	macOS resident	Linux/glibc pool cost
8 KiB	512 KiB	976 KiB	+512 KiB
16 KiB	1 MiB	1 472 KiB	+984 KiB
64 KiB	4 MiB	512 KiB	+3 032 KiB
128 KiB	8 MiB	496 KiB	+252 KiB
256 KiB	16 MiB	560 KiB	+280 KiB

macOS crosses its threshold somewhere at or below 64 KiB; glibc's default M_MMAP_THRESHOLD is exactly 128 KiB, so at 64 KiB Linux is not merely unimproved, it is three times worse than today. A change validated on macOS alone would have been a silent 2 MiB regression on Linux.

And glibc's threshold is dynamic: it grows when freed mmap'd blocks are observed, up to 32 MiB. A process that frees a large mapping can push later pool allocations back onto sbrk, where they are resident again. So even at 128 KiB this is a heuristic that happens to hold, not a property.

What that means for the fix

If the pool is going to be preallocated-but-not-resident, it should say so to the OS directly — mmap the slab rather than asking alloc_zeroed for something large enough that malloc probably maps it. Then the behaviour is a property of the code instead of a property of whichever allocator the host links, and it holds under a raised threshold, under jemalloc/mimalloc, and under a musl build whose thresholds differ again.

That is more work than the alloc_zeroed version I first suggested, and it wants a fallback path for targets where mapping is not available (wasm). Worth it: this is the last memory row where turnloop loses, and a fix that quietly depends on M_MMAP_THRESHOLD would be the kind of thing that regresses two releases later with nothing pointing at it.

Meanwhile

A host can get most of this today by setting pooled_buffer_size to 128 KiB, which is above both platforms' thresholds, keeps pooled_buffers (and therefore the events_per_turn coupling) unchanged, and makes reads larger rather than smaller. That is a workaround with a documented reason, not a fix, and it deserves a comment saying exactly which allocator behaviour it is leaning on.

[proggeramlug]

Retracting the workaround: 128 KiB measured WORSE in a real process

I suggested at the end of my last comment that "a host can get most of this today by setting pooled_buffer_size to 128 KiB". Do not do that. I measured it end to end and it regresses every memory row.

Perry's A/B, node:http server, 5 interleaved rounds, 0 invalid samples, the only change between the two runs being pooled_buffer_size 16 KiB -> 128 KiB:

metric	16 KiB	128 KiB
RSS before connections	20 976 KiB (+2.6 % vs tokio)	22 544 KiB (+10.3 %)
RSS peak, 1 connection	39 920 KiB (−1.3 %)	41 328 KiB (+2.1 %)
RSS peak, 64 connections	41 904 KiB (−3.4 %)	43 440 KiB (+0.2 %)
RSS peak, 1024 connections	60 976 KiB (−18.6 %)	62 320 KiB (−16.8 %)

It turned two rows that had just become wins back into losses.

Why the probe was wrong

My numbers came from a probe that built one Loop in an otherwise empty process. There, a 128 KiB alloc_zeroed is served by fresh mmap and the pages are never faulted, so the pool looked free. In a real Perry server — a large, busy, fragmented heap — the same request is evidently satisfied from an existing region the allocator already holds, and the zeroing is real work against resident pages.

That is precisely the fragility I flagged one comment earlier and then under-weighted: the behaviour depends on allocator state, not just request size. A trivial process is the best case for it, which makes a probe in one systematically optimistic. I should have measured in the real binary before recommending it, and the lesson generalises past this issue — an allocator-shaped hypothesis has to be tested against a real heap.

What that means for the fix

It strengthens the case for doing this properly rather than by tuning. An explicit mmap of the pool slab does not care how fragmented the host's heap is or what M_MMAP_THRESHOLD currently says — the pages are untouched because nothing touched them, by construction. That is the whole difference between a property and a coincidence.

The alloc_zeroed-a-big-block variant in the original issue text has the same defect as the config workaround and should not be implemented either.

[proggeramlug]

Measured: shrinking the pool does not shrink the process, so the premise here is unproven

Before writing the mmap slab I tested the assumption underneath it — that the preallocated pool is where a host's idle RSS actually goes. In Perry's A/B, changing only pooled_buffers, 3 rounds, 0 invalid samples:

pooled_buffers	pool	RSS before connections
64	1 MiB	21 056 KiB [21 040–21 056]
8	128 KiB	21 728 KiB [21 728–21 744]

Removing 896 KiB of pool made the process 672 KiB larger. The ranges do not overlap, and the tokio arm was stable at ~20 480 KiB across both runs, so this is not drift.

Whatever dominates a real process's floor, it is not the number of pooled buffers. A slab would have been unsafe aliasing code, platform-specific mapping and a wasm fallback, all justified by a model the measurement contradicts.

Where that leaves the issue

The observation that started it still holds in isolation: a one-Loop probe shows Config::default()'s 256 × 16 KiB pool costing ~4 MiB, and pooled_buffers = 0 removing it. What is now doubtful is that the isolated number predicts anything about a host.

That is the third time in this issue that a memory model of mine survived a probe and died against the real binary — first the alloc_zeroed-a-big-block idea, then the 128 KiB pooled_buffer_size workaround (which measured worse on both platforms), now the premise itself. The common factor is that a trivial process is the best case for every allocator trick, so a probe in one is systematically optimistic.

Suggested disposition: leave this open as an observation rather than a planned fix, and require a real-host measurement before anyone implements it. If someone does want the pool to stop being a preallocation, the first deliverable should be a measurement showing that the pool costs a host something — not a probe, and not a design.

#89 (ring slots allocated zeroed) is unaffected: it was measured the same way but confirmed in the real binary, where it moved Perry's floor from +17.8 % to +2.9 % against tokio and flipped two RSS rows to wins.

[proggeramlug]

Closing per the measurement in the last comment: removing 896 KiB of pool (pooled_buffers 64 → 8) made Perry's real node:http process 672 KiB larger, and both proposed fixes (one alloc_zeroed slab, pooled_buffer_size = 128 KiB) measured worse in a real host. The one-Loop probe is systematically optimistic, so the premise — that the preallocated pool is what a host's idle RSS pays for — is unproven.

Reopen with a real-host measurement showing the pool costs a host something before designing a fix (an explicit mmap slab with a wasm fallback would be the shape). The remaining #88 cost, WorkPort's ring, is being addressed separately.