Segment and checkpoint large rollouts with safe stale-session cleanup #437

Objective

Replace unbounded single-file growth for new sessions with a crash-consistent, segmented, checkpointed rollout format that can resume multi-gigabyte histories with bounded memory. Add conservative stale-session detection and cleanup only after the persistence health contract is stable.

Finish Line

New multi-gigabyte rollouts resume with bounded memory and crash consistency while V1 and live-session safety remain intact.

Current Status

State: Blocked
Next action: Design the versioned locator/manifest/reader contract and crash matrix after V1 writer health semantics are stable.
Blocked by: #436
Waiting for: #436 to establish durable-prefix, backpressure, and persistence-health contracts.
Last verified: July 18, 2026 against current whole-file resume and file-oriented archive/list behavior.

Scope

• In: V2 bundle locator, atomic manifest, bounded segments, sequence/checksum metadata, streaming reader, checkpoints, list/archive/state DB integration, lease/clean-close markers, diagnostic-first cleanup.
• Out: automatic V1 migration, automatic deletion of normal user history, cloud archival, speculative compression/encryption redesign.

Acceptance Criteria

• V1 JSONL remains fully readable and is never automatically rewritten during resume.
• New V2 sessions rotate bounded segments with an atomic committed prefix and exactly one listed thread.
• Crash injection at every rotation/manifest step leaves a valid resumable prefix.
• Resume streams checkpoint plus suffix rather than reading the entire history into one String.
• List, resume, archive, state DB repair, and current-rollout lookup share one version-aware locator.
• Lease/heartbeat and clean-close markers never classify an active session as stale.
• Automatic cleanup is limited to verified transient artifacts; user-history cleanup requires an explicit action.
• A logical 4.31 GB soak stays within the agreed memory ceiling and resumes correctly.

Relationships

This will be attached as a sub-issue of the agent reliability parent and blocked by the V1 writer recovery issue.

Validation

• Low-threshold segment rotation and crash-consistency tests.
• V1/V2 listing, resume, archive, and state DB compatibility tests.
• Streaming-memory benchmark and multi-gigabyte logical soak.
• Cleanup fixtures for live leases, expired locks, malformed tails, orphaned temp manifests, and archived sessions.

Decisions

• Introduce V2 only after readers/locators can understand it.
• Migration is explicit and free-space preflighted.
• No automatic deletion of resumable user history.

Open Questions

• Select the segment target and checkpoint cadence from benchmark evidence; 64 MiB is a starting hypothesis, not a committed interface.