Skip to content

fix(retention): drain backlog and retire unheld flow families - #145

Open
wolfy-j wants to merge 1 commit into
masterfrom
fix/retention-drains-backlog
Open

wolfy-j wants to merge 1 commit into
masterfrom
fix/retention-drains-backlog

Conversation

@wolfy-j

@wolfy-j wolfy-j commented Nov 14, 2026

Copy link
Copy Markdown
Contributor

Retention deleted one batch per interval, so any load above that rate grew the commit and data tables without bound, and terminal flow families (flows, nodes, results, inputs and outputs) were never deleted.

  • The diagnostics sweeper drains its backlog in bounded transactions per pass, selecting through age-leading indexes; a protected head page no longer hides later eligible rows. One log line per productive pass, none when idle.
  • Terminal flow families age out after flow_retention_days (default follows the diagnostic window, 30-day minimum; zero disables). Children are deleted in bounded pages, leaves first, with a durable fence for restart recovery.
  • Owners keep their flows: create accepts retain = true, and release drops the hold (actor-checked, idempotent). Held families are never aged out. Every family created before this migration is held, so installed owners lose nothing they still read.

Migration 10 backfills family identity on populated databases. Tested on SQLite and PostgreSQL (1,582 each), including a populated upgrade.

@wolfy-j
wolfy-j requested a review from skhaz November 14, 2026 06:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants