Skip to content

fix(chain): run one authority-index scan at a time and keep proven history - #3901

Open
branarakic-agent wants to merge 11 commits into
testnet-canaryfrom
fix/authority-index-walk-bound
Open

branarakic-agent wants to merge 11 commits into
testnet-canaryfrom
fix/authority-index-walk-bound

Conversation

@branarakic-agent

@branarakic-agent branarakic-agent commented Nov 13, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • A core node now runs at most one Context Graph authority-index scan per contract and request class. A slow catch-up no longer turns into one more walk over the same pages for every reader that arrives while it runs.
  • A scan whose anchor is out of date (it waited behind another scan, or its endpoint lags) is refused for retry. It no longer throws the stored index away and restarts the walk from the contract's deployment block. Only a cursor that no anchor reaches for five minutes is rebuilt.
  • A deploy-block lookup that falls back to genesis reuses the deployment block the stored index was built from, so it no longer triggers a rebuild either.

What we saw

On a long-running core node on testnet, Context Graph authority-index reads took the node's whole chain RPC budget and kept it for days, through several worker restarts. Every minute the request governor admitted its full rate with about 170 requests queued. Other chain reads then missed their admission deadlines: proof ticks, StorageACK checks, syncs, and the chain event log tick. With the tick stalled, the log fast path was off too, so every authority read fell through to the live scan.

Three things combined:

  1. Scans at different anchors ran side by side. The single flight in ContextGraphAuthorityIndex only joins a scan to the same anchor on the same endpoint, and every scan runs under the index lifecycle, so it outlives the caller that started it. Under a throttled budget, a scan from a far-behind cursor takes longer than readers wait. Each new reader started one more walk over the same pages, and each loser of a page commit read another block hash to re-admit the winner.
  2. The durable cursor kept going back to the deployment block. Admission rebuilds the whole index when a scan's anchor sits more than 50 blocks below the stored cursor, or when the scan's deploy-block lookup differs from the stored one. Both happen without any chain change: a scan anchored before another scan's commit, a lagging endpoint, or the deploy-block lookup's genesis fallback, which the adapter never caches. Base Sepolia now makes a block every 0.2 s, so 50 blocks is about 10 s of lag. On the node we looked at, the stored cursor climbed through the history and dropped back to the first page several times a day.
  3. A walk's projection was stale when it landed, so the readers waiting on it each refreshed again.

Changes

  • ContextGraphAuthorityIndex runs one physical scan per scope and request class. A scan to another anchor or endpoint waits for the running one, then resumes from the cursor it left, which is a page or two. The wait is bounded by the caller's own signal, and a caller that gives up leaves nothing queued. The class is the caller's RPC request class, passed explicitly by the EVM reader (scanClass): a scan's requests keep the class of the caller that started it, so a foreground read does not wait out, or join, a walk that the core's background refresh started on the background share of the budget. A scan that waits keeps the index lifecycle it entered under: a clear() while it waits rejects it, and nothing prepared before the clear starts after it. New whenScanIdle(scope, signal) and durableDeploymentBlockNumber(scope).
  • Admission asks the index (rebuildOutOfReach) before it rebuilds a checkpoint whose cursor is more than 50 blocks above the scan's anchor. The index refuses for retry with cursor-ahead until the cursor has stayed out of reach for five minutes (the head tolerance the projection cache already uses). It keeps one record per key: the token first seen out of reach, and when. Any commit or any admission against a chain view clears it; a new token or a clock that stepped back starts it again. The next read at a current anchor admits the record and clears the clock, which covers the first read after a restart meeting a lagging endpoint. A restored or copied store, or a chain reset at the same address, is still rebuilt, five minutes later.
  • The EVM authority-index reader waits for a running scan before its head read, so it doesn't pay for an anchor that would be stale by the time the scan lets it in. It passes the caller's signal to the scan wait. The adapter's deploy-block lookup now says whether it pinned the block (resolveContractDeployBlockLookup, pinned: false for the uncached genesis fallback), and the reader hands that lookup to the index as it is (deploymentBlockPinned).
  • The index decides the scan's lower bound once, with the first admission. For an unpinned lookup it resumes the stored checkpoint from the block it was reduced from, if that block is at or below the anchor and admission keeps the checkpoint; otherwise the rebuild starts from the lookup's block (a stored block above the anchor, or history admission rejects, means a chain reset at the same address). If a later reload after a lost commit finds that checkpoint gone, the scan ends with a retryable error instead of changing its bounds mid-scan, and the next scan starts from the lookup. Log anchors prove coverage from deploymentCoverageFromBlockNumber.
  • The chain event log's page source now refuses a range it does not hold with the typed reason log-uncovered, and the reader takes that range to the live scan on the same provider instead of failing the attempt. That covers a rebuild below the stored deployment block.

Tests

  • New context-graph-authority-index-walks.unit.test.ts: 60 readers, one a second, each giving up after 4 s, through a real RpcRequestGovernor at 2 requests a second, over 22 pages of history.
    • On testnet-canary: up to 29 walks in flight at once; 181 of the 241 admitted requests were block-hash re-admissions; after 120 s the cursor had covered 7 of the 22 pages.
    • With this change: one walk, 129 requests, and the cursor ends at the last anchor less the holdback.
  • The same run from a far-behind cursor, as after a restart mid-walk.
  • A read anchored 200 blocks below a cursor the index just proved is refused for retry and the history is kept.
  • After a restart, a first read at a lagging anchor is refused and the next read at a current anchor resumes the stored history. A cursor that no anchor reaches for five minutes is rebuilt.
  • A chain reset under a running node is rebuilt once the cursor's last proof is five minutes old.
  • A caller that gives up while waiting for the running scan starts no scan.
  • A foreground read is not held behind a walk the background refresh started, on the same endpoint or another one.
  • clear() with one scan running and one queued, plus an idle wait: all three reject, and the queued read reads no page and commits nothing.
  • A scan in one request class that completes after a reorg rejection in the other does not make the rejected checkpoint exportable again.
  • The reader keeps the stored deployment block when the lookup is unpinned: no invalidation and no page reads. When the chain was reset below that block, it rebuilds from genesis.
  • When admission rejects the stored history (a reset chain whose head is past the old deployment block, with a graph created below it), the rebuild starts from the lookup's bound and the graph is in the view. A checkpoint admission keeps still resumes from the stored block.
  • On the log path the log refuses that rebuild as log-uncovered, and the reader rebuilds live: with one endpoint there is no failed attempt, and the graph below the old deployment block is included.
  • When another writer drops the stored checkpoint mid-scan, that scan ends retryably and the next one rebuilds from the lookup's block, keeping the graph below the old deployment block. With trusted-core seeding the same case ends retryably instead of reseeding with the old bounds.
  • A read exactly at the cursor admits it without a commit and resets the rebuild clock.
  • The out-of-reach clock starts again when the wall clock steps back.
  • The adapter lookup reports pinned for a search or a request-free cache hit, and not for the genesis fallback.
  • Admission: the index can hold an out-of-reach checkpoint for retry, and it is not asked within the ordinary skew bound.
  • Four existing tests drove two scans of one scope side by side. They now run them in order. The CAS-winner test uses two index instances on one store, so the cross-writer race keeps its coverage.

packages/chain unit tests, tsc and type tests, pnpm run lint and the file-size check pass locally, as do the agent tests that build a real adapter with the index.

Not changed

  • The freshness rules of the projection cache and of bounded reads. While a walk runs, readers still wait and fail closed at their own deadlines, as before.
  • A hash mismatch at the stored cursor still rebuilds the index: that is the fork proof.
  • Block-count windows: the 50-block reorg holdback and the 2,000-block pages. On Base Sepolia they now span about 10 s and about 7 minutes of chain, so a full walk from the deployment block is several thousand pages.

…story

A core node's Context Graph authority-index reads could hold the whole
chain RPC budget for days. Reads to different anchors each started their
own paged scan from the durable cursor, every scan outlived the caller
that started it, and a scan anchored before another scan's commit, a
lagging endpoint, or a deploy-block lookup that fell back to genesis could
send the durable cursor back to the contract's deployment block.

- The index runs one physical scan per scope and request class. A scan to
  another anchor or endpoint waits for it, bounded by its caller's signal,
  and resumes from the cursor it left. A foreground read does not wait out
  a walk the background refresh started on the background budget.
- The reader waits for a running scan before its head read and passes its
  caller's signal to the scan wait.
- Admission keeps a record this index committed or admitted in the last
  five minutes when a scan's anchor sits far below it, and refuses that
  scan for retry. Unproven records still rebuild as before.
- A deploy-block lookup that answers the unpinned 0 reuses the deployment
  block of the stored checkpoint.

Adds a test that drives 60 readers through a throttled RpcRequestGovernor:
on canary up to 29 walks ran at once and the cursor covered 7 of 22 pages
in 120 s; now one walk reaches the head.
…minutes

Base Sepolia now makes a block every 0.2 s, so the 50-block bound past
which admission rebuilt a cursor that sits above a scan's anchor is about
10 s of endpoint lag. The first scan after a restart has not proven the
stored cursor yet, so one lagging endpoint could still replace the whole
history with a walk from the deployment block.

Admission now asks the index before that rebuild. The index refuses for
retry until the cursor has stayed out of reach for the head tolerance
(five minutes) with no commit or admission proving it in that time; any
read at a current anchor admits the record and clears the clock. A copied
store or a chain reset at the same address is still rebuilt, five minutes
later.
// The index runs one scan per scope. Wait for a running one BEFORE the
// head read: an anchor read now is out of date once that scan lets
// this read in, and a walk that outlasts its readers cost one per reader.
await dependencies.index.whenScanIdle(

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Give scan admission and anchor preparation one owner

What's wrong
This adds a scheduling protocol across the reader/index boundary: wait for idle, prepare an anchor, then enter another admission loop. The duplicated coordination makes callers responsible for scan internals and requires lifecycle and cancellation handling at both stages. A single admission operation could remove that protocol.

Example
When two readers finish whenScanIdle together, both prepare anchors before one waits again inside #snapshot. Understanding scan admission therefore requires following both layers and their separate cancellation checks.

Suggested direction
Let the scoped scheduler invoke a caller-supplied preparation callback after acquiring admission. Keep lane management internal instead of exposing an idle-then-start protocol to readers.

For Agents
Refactor whenScanIdle, #snapshot, and rescanFinalizedProjection around one admission operation. Preserve request-class separation, compatible scan coalescing, lifecycle ownership, and waiter-local cancellation. Verify concurrent readers and abandoned waiters retain those behaviors.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We kept these two steps apart on purpose. #snapshot is the barrier that keeps one scan per scope and class. whenScanIdle is only an early wait that saves the head read, and lanes stay internal to the index. Holding the scan turn across the head read would make readers that could share one scan take turns instead. Since abea968 both waits capture the index lifecycle when they start, so a clear() stops them the same way.

- A scan that waits behind another, and whenScanIdle, keep the index
  lifecycle they entered under. A clear() while they wait rejects them,
  so nothing prepared before a Hub rotation or reconfiguration starts
  after it.
- The request class is part of the scan key, so a foreground read never
  joins a background walk on the same endpoint.
- The deploy-block lookup now says whether it pinned the block. The
  reader uses the stored checkpoint's deployment block only for an
  unpinned lookup and an anchor at or above that block, so a chain reset
  at the same address still rebuilds from genesis.
The per-lane registry already held the identity the single flight keyed
scans by, so a matching caller found the running scan there and then
looked it up again in KeyedSingleFlight. The active entry now carries the
scan's result: a caller to the same anchor on the same endpoint joins it,
any other caller in the lane waits for it, and clear() drops both at once.
The index no longer uses the single flight; Activity still drains the
physical work.

@otReviewAgent otReviewAgent left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review Agent reviewed this PR and has nothing new to post. 3 earlier findings were not re-posted because the last message in those threads was ours when the review started: packages/chain/src/evm-context-graph-authority-deployment.ts:42, packages/chain/test/context-graph-authority-index-bootstrap.unit.test.ts:576, packages/chain/src/context-graph-authority-index.ts:523.

- The stored deployment block now bounds only the resume of a checkpoint
  that admission keeps. The reader passes the lookup's own bound as
  rebuildFromBlockNumber, and the index rebuilds from it when admission
  does not keep the stored checkpoint. A chain reset at the same address
  therefore no longer skips creation events below the old deployment block.
- Out-of-reach recovery is one record per key: the token first seen out of
  reach, and when. A commit or an admission clears it, a new token or a
  clock that stepped back starts it again, and it rebuilds after the head
  tolerance. The separate proof map is gone.
- Adds a cross-class test for a scan completing after a reorg rejection, a
  clock-step test and unit tests for the deployment bounds.
resolveContractDeployBlockLookup is the EVM adapter's protected deploy-block
lookup with its pinned flag, beside resolveContractDeployBlock and
resolveContractDeployBlockNumber, which are already exempt. The mock has no
chain, deploy block or eth_getCode to resolve.
- The reader hands the deploy-block lookup to the index as it is
  (deploymentBlockNumber plus deploymentBlockPinned). Next to admission,
  the index resumes a stored checkpoint from its own deployment block for
  an unpinned lookup, if that block is at or below the anchor and admission
  keeps the checkpoint, and otherwise rebuilds from the lookup's block. The
  reader-side bounds helper and its rebuildFromBlockNumber pair are gone.
- A log-backed scan passes the first block the log covers. A rebuild that
  needs blocks below it is refused as rebuild-required, and the reader
  rebuilds live on the same provider instead of failing the attempt.
- Adds tests for the log-to-live rebuild, the refusal, and a read at the
  cursor resetting the rebuild clock without a commit.
);
// The index runs one scan per scope and class. Wait for it BEFORE the head
// read, which would be out of date by the time that scan lets this read in.
await dependencies.index.whenScanIdle(ownScope, options.signal === undefined

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Own scan preparation and admission in one scheduling operation

What's wrong
This introduces a two-stage scheduling protocol across the reader and index. whenScanIdle observes availability without reserving it, so #snapshot still needs its own arbitration loop. The reader must now understand the index's scheduling policy, combine cancellation signals, and repeat lifecycle checks between the two stages. That spreads ownership of one operation across two layers and makes future read paths harder to implement consistently.

Example
Two readers can both finish whenScanIdle, then independently resolve deployment and head information before #snapshot serializes them again. Understanding scheduling now requires following both arbitration points.

Suggested direction
Provide a coordinated operation that accepts lazy scan preparation and acquires the scope's slot before resolving its anchor. Keep scheduling and cancellation bookkeeping behind that boundary instead of requiring callers to perform an idle check followed by a separately scheduled scan.

For Agents
Review whenScanIdle, #snapshot, and rescanFinalizedProjection. Consolidate preparation and scan admission behind one coordinator. Preserve same-anchor joining, request-class isolation, lifecycle cancellation, and caller cancellation without abandoned queued scans; validate with the existing walk and head-read scenarios.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same point as the earlier thread, and we are keeping the two steps for the same reasons. #snapshot is the only arbiter of one scan per scope and class. whenScanIdle only saves the head read a reader would otherwise spend while it waits. Holding the slot across preparation would serialize those head reads, and readers that end up at the same anchor would stop sharing one scan. Both waits take the index lifecycle when they start, so a clear() stops either one. Leaving this open for the lead.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Own anchor preparation and scan scheduling in one operation

What's wrong
This exposes the index's scheduling protocol to its reader while retaining a second scheduling loop inside #snapshot. The idle check does not reserve the lane, so both layers must coordinate lane identity, cancellation, and lifecycle checks across intervening asynchronous work. This is a structural opportunity to remove the two-stage protocol rather than maintain it in two places.

Example
Two readers waiting behind one scan can both leave whenScanIdle, prepare their anchors, and then contend again inside #snapshot. Understanding scheduling therefore requires following both layers.

Suggested direction
Provide a coordinated prepare-and-scan operation that acquires the lane before invoking an anchor-preparation callback. Keep lane waiting, registration, and lifecycle handling behind that boundary so readers do not need a separate idle-check protocol.

For Agents
Consolidate lane scheduling across the reader and index. Preserve foreground/background isolation, same-anchor joining, caller-local cancellation, and lifecycle-owned scans. Verify concurrent callers and clear-during-wait behavior.

- Any admission that keeps no checkpoint now resets the scan's lower
  bound and write horizon to the lookup's block: the first admission, and
  a reload after a lost commit when another writer dropped the checkpoint
  (a chain reset at the same address). The rebuild no longer starts from
  the stored deployment block that history was reduced with.
- The chain event log's page source refuses a range it does not hold with
  the typed reason log-uncovered, and the reader takes that range to the
  live scan on the same provider. The index's page-floor input and its
  rebuild-required reason are gone.
- The scan picks its deployment bound once, with its first admission and
  before any seed session starts. If a reload after a lost commit later
  finds that checkpoint gone while the scan runs on the stored block, the
  scan ends with a retryable error instead of changing its bounds mid-scan.
  A seed session therefore never imports a checkpoint for the old bound,
  and the next scan rebuilds from the lookup.
- The index no longer reads the ambient RPC request context: the scan
  input and whenScanIdle take an explicit scanClass, which the EVM reader
  sets from its caller's request class.
// reset at the same address), this scan's bounds and any seed session
// are already set for the old block: end it, and let the next scan
// start over from the lookup.
if (admittedAtStart) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 Bug: Repository fallback is mistaken for a lost checkpoint

What's wrong
This guard also runs when bootstrap intentionally switches to the plain-scope fallback repository. If an unpinned lookup caused the initial trusted checkpoint’s deployment block to be adopted, an empty fallback repository triggers this exception. Subsequent retries retain the same trusted checkpoint and fail identically, preventing the configured local-history recovery.

Example
Keep a valid trusted checkpoint reduced from block 10, but old enough to require reseeding. Use an unpinned lookup returning block 0, enable localHistoryFallback, leave the plain-scope repository empty, and make trusted peers unavailable. Every scan throws 'dropped during this scan' before reading any history. Retrying repeats the failure because the trusted checkpoint remains unchanged.

Suggested direction
Initialize fallback admission and scan bounds for the plain-scope repository independently, while preserving the protection against losing a checkpoint within the same repository.

For Agents
Inspect the admit closure and bootstrap fallback repository switch in context-graph-authority-index.ts. Distinguish entering an independent repository from losing an adopted checkpoint. Verify that an unpinned lookup with a stale trusted checkpoint, unavailable peers, and an empty plain repository successfully rebuilds through local-history fallback.

});
// Admission returns a checkpoint only after matching its cursor hash.
markReachable(admitted);
if (admitted.kind !== 'checkpoint' && deploymentBlockNumber !== requestedDeploymentBlockNumber) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Separate bound selection from admission during a scan

What's wrong
The new admission wrapper serves two different phases: selecting scan bounds and validating records after those bounds and the seed session are established. Its behavior changes through captured mutable state. Readers must track call order and the lifetime of downstream bounds to understand whether admission can mutate the deployment block or terminate the scan. These phases should be explicit at the boundary.

Example
When admission returns a tombstone for an adopted stored deployment block, the first call changes deploymentBlockNumber to the lookup bound; a later call throws instead. The distinction depends on an external boolean rather than the operation's explicit inputs.

Suggested direction
Have initialization return the admitted record and effective deployment block together, then keep that block immutable. Make subsequent admission explicitly validate against the established bound and request a retry when rebuilding would require another bound. This removes admittedAtStart and the phase-dependent mutation from the shared closure.

For Agents
Separate initial bound resolution from admission during an established scan. Preserve stored-bound resume, lookup-bound rebuild, and retry when adopted history disappears. Use the existing reset, CAS-loss, and bootstrap scenarios to verify equivalent behavior.

return EVENTS.filter((entry) => entry.blockNumber >= fromBlock && entry.blockNumber <= toBlock);
},
}),
scanClass: 'background',

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Verify scan-class isolation through the EVM reader

What's wrong
These tests manually select the background lane, bypassing the new reader wiring. They cannot catch lost request-class propagation, which would make foreground authority reads wait behind a long background history scan.

Example
Removing scanClass forwarding from authorityIndexScanBoundsV1 would leave these direct index tests passing while background scans default to the foreground lane.

Suggested direction
Add a regression test through the production reader that exercises request-context extraction and forwarding to both whenScanIdle and the scan input.

For Agents
Extend the EVM reader tests with a background refresh under withRpcRequestContext. Hold its first page, then prove a foreground read on the same provider completes before releasing it.

The trusted-core bootstrap's fallback to local history moves a scan to
the plain-scope repository. The scan now sets its bounds again from that
repository's record, so an empty plain scope starts from the lookup's
block. Before, a scan that had resumed the trust-domain checkpoint from
its stored block ended retryably at the fallback, and every later scan
did the same.

Choosing the bounds and checking records against them are now separate
steps: startFromRepository picks the bound and admits the record the
scan starts from; readmit checks a CAS winner against the bound already
set and ends the scan retryably when that checkpoint is gone.

Adds a test through the EVM reader that a read in one request class is
answered while a walk in the other class holds a page on the same
provider, in both directions.
if (stored > bound && stored <= finalizedNumber) bound = stored;
}
const admitted = await admit(initial, bound);
deploymentBlockNumber = admitted.kind === 'checkpoint' ? bound : requestedDeploymentBlockNumber;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Represent each repository scan with an explicit, immutable context

What's wrong
The new startFromRepository helper returns only a record while secretly changing two enclosing scan bounds. Meanwhile, admit, readmit, markReachable, and rebuildOutOfReach capture repository state that bootstrap fallback reassigns. This materially increases temporal coupling: the repository, bounds, recovery clock, and seed session must remain aligned through assignments spread across the function, without an explicit model expressing that invariant.

Example
With an unpinned lookup at block 0, a trusted checkpoint can establish block 10 as the scan bound. Bootstrap fallback then changes the repository and silently resets both bounds through startFromRepository(). Understanding this transition requires tracing repository assignments, captured variables, seed-session cleanup, and the separate readmit path.

Suggested direction
Have initialization return an explicit context containing the repository identity, admitted record, deployment bound, and persistence horizon. Run paging against that fixed context; bootstrap fallback should start a fresh context for the plain repository. This removes repository rebinding and hidden bound mutations from the admission closures.

For Agents
Refactor #scan initialization and bootstrap fallback into repository-bound scan attempts. Preserve unpinned resume/rebuild behavior, per-repository recovery clocks, CAS-loss retries, and trust-domain separation. Run the existing unpinned fallback and lost-checkpoint cases in the walks and bootstrap suites.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in a565bfa. #scan now works through a fixed RepositoryScan per repository (key, repository, deployment bound and persistence horizon), which startRepositoryScan(key) returns together with the admitted record. Paging, commits and readmit read only that context, and admit, markReachable and the out-of-reach clock take its key and repository explicitly instead of reading reassigned variables. The bootstrap fallback starts a new RepositoryScan on the plain scope rather than rebinding the repository and bounds; the seed session stays bound to the first one and is closed before the switch. The walks, bootstrap, index and over-log suites pass unchanged, including the unpinned fallback and lost-checkpoint cases.

A scan now works on one repository at a time through a RepositoryScan:
the repository key, the repository, the deployment bound and the
persistence horizon, fixed when the scan starts there.
startRepositoryScan returns it together with the admitted record.
Paging, commits and readmit read only that context, and admission and
the out-of-reach clock take its key and repository explicitly instead
of reading variables the bootstrap fallback reassigned. The fallback to
local history starts a new RepositoryScan on the plain scope.

No behaviour change.
// An unpinned lookup resumes the stored checkpoint from the block it was
// reduced from, if that block is on this side of the anchor and admission
// keeps the history; otherwise the scan starts from the lookup's block.
const startRepositoryScan = async (key: string): Promise<Readonly<{

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Issue: Put the new recovery protocol behind the admission boundary

What's wrong
The change builds a substantial second admission layer inside #scan, even though checkpoint admission already has a dedicated module. Understanding whether a durable observation can be adopted now requires following the admission helper plus five local closures and commit-side bookkeeping. The RepositoryScan value groups fields but leaves their transition rules distributed through the paging orchestration.

Example
With an unpinned lookup at block 0 and stored history starting at block 1000, initial admission may preserve 1000 or rebuild from 0. Losing that checkpoint mid-scan instead requires a retry, while switching to the plain repository requires fresh bounds. These related transitions are currently coordinated by separate closures and assignments inside the paging function.

Suggested direction
Extend the admission collaborator with a repository-scoped session that owns the effective bounds and recovery transitions. Have the paging loop consume that session rather than manually passing repository keys and bounds through several admission wrappers and separately notifying successful commits.

For Agents
Extract the recovery/session boundary from #scan into the admission collaborator. Keep requested and effective deployment bounds, token timing, and winner adoption together. Preserve the existing resume, rejected-history rebuild, checkpoint-loss retry, and bootstrap-fallback scenarios.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are keeping this as it is in this PR. The last two rounds already put these rules in two named steps over one fixed RepositoryScan: startRepositoryScan picks the bound and admits the starting record, and readmit checks a CAS winner against that bound or ends the scan for retry. Moving them into a session in the admission module would change the API of that module, which admits one record today, and restructure the same code a third time late in a fix whose behaviour is settled. It is a reasonable follow-up if the lead wants it; leaving this open.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants