Skip to content

MST/2 T06: GitRetentionPort, locator compaction, frame-eviction fallback and space pressure #17

Description

@Ivanbeethoven

Context

MST/2 owns derived data (metadata pages, chunk maps, cached frame payloads)
that lives alongside Git-owned content. Today the collector is deliberately
fail-closed: NoopReaper in src/ceres/snapshot/retention.rs reports
success without deleting anything, because the deployment has no way to tell
whether a given piece of raw content is still needed by Git.

Spec 10 §7 makes that a hard rule, not a temporary wart:

MST/2 collector must not delete Git-owned raw blobs. … Without that
bidirectional integration, only MST/2-derived frame eviction, orphan temp
cleanup and protected metadata-graph GC are allowed; physical deletion of
shared raw blobs must stay disabled
.

So this issue is the precondition for ever turning physical reclamation on.

Scope

  1. GitRetentionPort — the bidirectional check. Before Git's own GC
    considers a raw object for deletion, it must be able to ask "does any
    snapshot / prepare / active reader reference this?" within one coordinated
    retention boundary. Concretely:
    • a query the Git GC path calls with the object id and its planned action;
    • a negative answer that is proven (no prepare root, no live lease, no
      LIVE retention node reachable from a root, not referenced by a
      still-running locator generation) — otherwise the answer is "keep".
  2. Enablement switch. A single configuration flag that states which
    reclamation capability this deployment actually has (MST2-derived only, or
    MST2-derived + Git-shared blobs) so an operator cannot accidentally turn on
    the stronger mode without the port being wired. Startup should refuse the
    stronger mode when the port is absent.
  3. Locator compaction order (spec 10 §4): write the new location → verify →
    atomically bump the locator generation → wait for all old readers/leases →
    delete the old location. A compaction must be abortable at any step without
    losing the object.
  4. Frame eviction fallback (spec 10 §3): an evicted or missing frame must be
    rebuilt from the fixed view; a cache miss must never surface as a gap, a
    short read, or an error to the client.
  5. Space pressure (spec 10 §9): under a disk/frame budget the system
    refuses or pauses new prepares rather than deleting active-lease data to make
    room.

Acceptance

  • A test proving that a shared raw blob referenced by a live lease is not
    deleted while the stronger mode is on, and is eligible once the lease is
    released and the roots are gone.
  • A test that the stronger mode cannot be enabled without the port compiled in
    and configured (startup refusal, not a warning).
  • A compaction test that kills the process between "write new location" and
    "bump generation" and asserts the object is still readable afterwards.
  • A frame-eviction test: delete a cached frame payload, request the same
    members, assert the bytes are rebuilt and identical (identity encoding).
  • GC-04/05/09 from spec 16 §6 (Git double retention, frame fallback, space
    shortage).

References

  • Spec 10 §§3, 4, 7, 9; spec 16 §6
  • src/ceres/snapshot/retention.rs (Reaper, NoopReaper),
    src/ceres/snapshot/chunks.rs (derived projection today)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions