Skip to content

events: match queries one slab at a time - #1087

Merged
tamirms merged 6 commits into
mainfrom
tamirms/events-slab-matcher
Nov 4, 2026
Merged

tamirms merged 6 commits into
mainfrom
tamirms/events-slab-matcher

Conversation

@tamirms

@tamirms tamirms commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

What

Rewrites how the events store matches a getEvents query. It now walks the query's window one slab at a time (65,536 consecutive event ids, roaring's container size) instead of combining whole-chunk bitmaps before the first fetch.

Tests: TestMatches_FetchesOnlyTrueCandidates requires the fetched ids to be exactly the query's matches; the matrices in slab_match_test.go compare against an oracle computed without the index; TestSlabStepperSkipsCandidateFreeSlabs pins which slabs the walk opens; TestTermPlansAtMostSevenTerms pins that no plan has more than seven terms, so FastAnd gets at most the eight inputs roaring checks before it allocates; roaring_contract_test.go (from #1086) pins, against roaring alone and under -race, the properties the shared index bitmaps rely on, including that a FastAnd of the walk's shape that comes out empty allocates only its result, for every mix of array, bitmap and run terms; TestMatches_ConcurrentIngestBorrowSafety runs walks while ingest keeps writing.

Why

A query is a list of filters, each an AND of terms (contract id, topics, event type), and the candidates are the ids that satisfy any filter inside the window. Today the engine intersects each filter's term bitmaps across the whole chunk, unions the results, and only then restricts to the window, all before the first event is fetched. Its cost tracks the terms' size across the chunk rather than the window or the page: a page of 1,000 events from a popular contract with a topic constraint first intersects that contract's millions of ids.

This PR turns each filter into plans, each a plain AND of terms (a topic-count range becomes one plan per count), and walks the window slab by slab in the requested direction. On each slab, every plan is one FastAnd over the slab's ids in the window and the plan's terms, and the plans' results are ORed and handed to the fetch. Before opening a slab, the walk asks each term for its next id (NextValue / PreviousValue) and jumps to the first slab that can hold a candidate, so a rare term opens only the slabs where it has events. The matching work scales with the window and the page, the walk holds one slab's results at a time, and both directions share one path. Every plan's FastAnd reuses one argument slice, so with the roaring pinned in #1086 a plan that matches nothing in a slab allocates only its empty result. The first batch is the caller's page size, so a full page arrives in one fetch. The hot store reads that fetch from RocksDB 512 ids at a time, because RocksDB pins every block a call touches until the call returns: one call for a sparse page of 10,000 events could pin about 313 MiB of block cache, and 512 ids pin about 16 MiB.

Known limitations

🤖 Generated with Claude Code

@tamirms
tamirms added this pull request to stack #1089 October 5, 2026 16:24
@tamirms
tamirms removed this pull request from stack #1089 October 8, 2026 08:18
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from 76598f8 to 86fe919 Compare October 8, 2026 08:18
@tamirms
tamirms added this pull request to stack #1102 October 8, 2026 08:19
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 3 times, most recently from 2dc83cf to ca3d216 Compare October 9, 2026 06:02
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 2 times, most recently from 60d5c4b to baa1bd4 Compare October 15, 2026 19:15
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from baa1bd4 to af5b344 Compare October 19, 2026 13:00
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from af5b344 to 13d4c27 Compare October 21, 2026 08:48
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 3 times, most recently from 9114839 to 6153b88 Compare October 21, 2026 09:52
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 3 times, most recently from f183b8a to 02fe44f Compare October 21, 2026 14:07
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 2 times, most recently from 84d7da5 to edea772 Compare October 21, 2026 17:05
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from edea772 to 2ab5c47 Compare October 21, 2026 17:15
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 3 times, most recently from 5891e7f to cf8bc60 Compare October 23, 2026 05:27
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch 3 times, most recently from 5353af6 to 4f9cc53 Compare October 23, 2026 07:09
Base automatically changed from tamirms/roaring-2.26-mphf-mmap to main October 24, 2026 02:49
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from 4f9cc53 to d9ef98c Compare October 24, 2026 02:49
@tamirms
tamirms removed this pull request from stack #1102 October 27, 2026 10:19
@tamirms
tamirms added this pull request to stack #1157 October 27, 2026 10:21
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from d9ef98c to 70911f3 Compare October 28, 2026 08:05
@tamirms
tamirms marked this pull request as ready for review October 28, 2026 08:37
@tamirms
tamirms requested a balanced review from Copilot October 28, 2026 08:37

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Batch reads continue after request cancellation, and the concurrency test can pass without exercising snapshot traversal during ingestion.

2 open issues
What changed in this PR

Replaces whole-chunk event bitmap matching with a shared, bidirectional slab walk, limiting candidate evaluation to relevant portions of the query window.

Changes:

  • Expands filters into term plans and skips slabs that cannot contain candidates.
  • Honors page-sized first fetches while splitting hot-store reads into bounded RocksDB batches.
  • Adds oracle-based matching, slab-skipping, batching, and snapshot-safety coverage.
File Description
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​slab_match.go Implements slab evaluation, directional traversal, and candidate streaming.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​slab_match_test.go Adds matching matrices, candidate-fetch checks, and slab-skipping tests.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​matches_differential_test.go Adds in-memory fixtures and concurrent-ingest snapshot coverage.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​match.go Introduces term plans, integrates slab matching, and honors first-batch hints.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​match_test.go Updates matching coverage and adds plan-width and batch-size checks.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​hot_store.go Splits event fetches into bounded RocksDB reads.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​hot_store_test.go Extends large-batch coverage across multiple read batches.
cmd/​stellar-rpc/​internal/​rpcv2/​stores/​event/​concurrent_bitmaps.go Clarifies read-only bitmap and snapshot-lifetime documentation.

Review effort: Balanced

@Shaptic Shaptic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly minor notes, can be merged without resolving any of them, but wanted to throw some thoughts out

tamirms added a commit that referenced this pull request Nov 4, 2026
…ed-matches

Brings in #1087's ctx check per fetch slice and its comment trim. The two
conflicts are resolved by taking #1087's shorter comments for ensureSlab,
appendUpTo and TestMatches_RareTermsSpanSlabs, keeping the ctx parameters.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tamirms and others added 6 commits November 4, 2026 09:50
A getEvents query was answered by intersecting each filter's term bitmaps
across the whole chunk, unioning the results across filters, and only then
restricting to the window. Its cost tracked the terms' size across the
chunk, not the window the client asked for or the page it would receive.

The engine now walks the window one slab at a time: 65,536 consecutive
event ids, roaring's container size, in the requested direction. Each
filter becomes plans that are plain ANDs of terms. On each slab, every plan
is one FastAnd over the slab's ids in the window and the plan's terms, and
the plans' results are ORed and handed to the fetch. Before opening a slab,
the walk asks each term for its next id and jumps to the first slab that
can hold a candidate, so a rare term opens only the slabs it appears in.
Work scales with the window and the page, memory per query with one
container, and both directions share one path. The first batch is the
caller's page size, so a full page arrives in one fetch.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QHnF5BhsuoxpWxGmatQzmt
A page now arrives in one fetch, so the hot store passed up to 10,001
ids to one BatchMultiGet. RocksDB pins every block that call touches
until it returns, and a sparse v1 page could pin about 313 MiB of
block cache. FetchEvents now calls BatchMultiGet once per
matchBatchSize ids, which bounds the pin to about 16 MiB. Event rows
never change once written, so the slices need no shared snapshot. The
cold store keeps one read per page.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwsd8qTEGeSb7poFEYcgg9
slabPlan.eval built its FastAnd argument slice with make on every call,
which goes to the heap for plans of four or more terms: at least 48
bytes per plan per slab, about 2 MB per page at 256 filters. The
stepper now sizes one buffer for its widest plan and eval reuses it,
so an empty slab allocates only FastAnd's result.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwsd8qTEGeSb7poFEYcgg9
A plan is at most the contract id, the event type, four topics and one
topic-count bucket, so with the slab FastAnd gets at most eight inputs,
the width the pinned roaring checks before it allocates. The test walks
every filter shape and topic-count constraint and requires that bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Wwsd8qTEGeSb7poFEYcgg9
A page now arrives in one fetch, so one hot FetchEvents can make about 20
BatchMultiGet calls, and a canceled request still made all of them.
FetchEvents now checks ctx before each call, so a cancel costs at most
the call in flight, as it did when the matcher fetched 512 ids at a time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Comments only.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tamirms
tamirms force-pushed the tamirms/events-slab-matcher branch from 218187d to c9ed941 Compare November 4, 2026 09:50
@tamirms
tamirms merged commit e1c84dd into main Nov 4, 2026
18 checks passed
@tamirms
tamirms deleted the tamirms/events-slab-matcher branch November 4, 2026 10:16
@tamirms
tamirms restored the tamirms/events-slab-matcher branch November 4, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants