Skip to content

events: add in-memory events index metrics - #1198

Draft
urvisavla wants to merge 1 commit into
stellar:mainfrom
urvisavla:inmemory-index-metrics
Draft

urvisavla wants to merge 1 commit into
stellar:mainfrom
urvisavla:inmemory-index-metrics

Conversation

@urvisavla

Copy link
Copy Markdown
Contributor

What

Adds two gauges for the in-memory events index of the open hot chunks.

Metric Label Meaning
stellar_rpc_events_inmemory_index_terms state Terms in the index
stellar_rpc_events_inmemory_index_dense_terms state Terms that are bitmaps and not lists

state has two values:

  • ingesting: the chunk that ingest writes to.
  • completed: the sum of all other open hot chunks.

Why

The in-memory index is the main memory cost of a hot chunk. These gauges show its size and how many terms are dense, so that we can tune it.

Scope

In scope:

  • rpcv2 only.
  • One new log line when a hot chunk with terms opens.
  • ConcurrentBitmaps.AddTo now increments an atomic counter when a term becomes dense. This is the only change on the ingest path.

Not in scope:

  • The cold index. Frozen chunks are not counted.
  • The size of the index in bytes.

Behavior

  • The collector reads the chunks at scrape time. No ingest code updates the gauges.
  • The ingesting chunk is the newest published chunk. This is correct because only startup and the chunk boundary publish a chunk, and the boundary publishes the next chunk id.
  • Stats reads the dense count before the term count. AddTo adds a term to the map before it counts the term as dense, so dense_terms is never larger than terms.
  • Both state values are always exported. They are zero until startup opens the query registry.

🤖 Generated with Claude Code

Add two gauges for the in-memory events index of the open hot chunks:
stellar_rpc_events_inmemory_index_terms and
stellar_rpc_events_inmemory_index_dense_terms, each with a state label
(ingesting, completed).

A collector reads the published hot chunks at scrape time. A hot chunk
that has terms logs its warmup time and term count when it opens.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant