retro

Capture Codex, Claude Code, VS Code GitHub Copilot Chat, and installed source extensions such as OpenCode into durable local artifacts, evaluate them with signals, mine them into prompt-time memory, and report on cost and behavior via a static dashboard. Local-first, no cloud, evidence-linked.

Project wiki and onboarding guides: https://sajjadgg.github.io/retro/

This project answers two questions that "another memory store" doesn't:

How did this rollout actually go? (signals: activity, outcome, cost, risk)
What can the next session reuse from it? (mining: skills, procedures, failure triggers)

What it does

Capture — one global, per-user archive of Codex, Claude Code and VS Code Copilot Chat sessions (plus installed source extensions), refreshed by retro sync or a periodic schedule, with immutable raw copies.
Catalog — consolidates worktrees, clones and symlinked paths into one project, tags scratch/automated/eval noise with explainable rules, and rolls subagents up under their parent.
Signals — 50 evidence-linked readings per session across activity, collaboration, efficiency, friction, outcome, risk, usage/cost, work kind and data quality.
Memory — mines reusable skills, procedures and failure triggers into a SQLite/FTS index for prompt-time retrieval.
Tasks and evals — turns real trajectories into frozen, time-consistent benchmark tasks with isolated repository environments and hidden graders.
Experiments — freezes agent/model/seed configurations, runs them through Ghostlab on demand, and records every trial's grade and failure evidence.
Dashboard — a static, offline dashboard over all of the above, plus an opt-in localhost server for experiment controls and private grader inspection.

The full design is split across specs in specs/. The headline ones:

full_rollout_capture_feature_spec.md — the capture contract.
rollout_signals_spec.md — what signals are and how they aggregate.
rollout_mining_methods.md — the catalog of mining methods.
memory_storage_backend_spec.md — SQLite memory index over flat files.
rollout_dashboard_spec.md — the dashboard.
ccusage_comparison_spec.md — gap analysis vs. ccusage.
global_rollout_archive_migration_plan.md — global archive, migration, and periodic sync design.
CHANGELOG.md — release history and upgrade notes.

Install
Global setup and periodic capture
Session capture hooks
Architecture at a glance
Source extensions
Storage layout
Capture
Session catalog: ignoring noise and consolidating projects
Signals
Mining
Time-consistent benchmarks
Memory backend
Dashboard
Pricing snapshot
Release and publishing
Environment
Notes and guarantees

Install

Requires Python ≥ 3.9.

From PyPI:

python3 -m pip install --upgrade retro-ai
retro --help

The distribution is named retro-ai; it installs the retro command through the package's console entry point. Inside a virtual environment, the command is added to that environment's bin/ directory. For a persistent command managed outside a project environment, use:

python3 -m pip install --user --upgrade retro-ai

If pip warns that its scripts directory is not on PATH, add the directory shown in that warning to your shell configuration. On macOS with the system Python 3.9, that is commonly:

export PATH="$HOME/Library/Python/3.9/bin:$PATH"

Then reopen the terminal and run retro --help.

From a clone:

python3 -m venv .venv
.venv/bin/pip install -e .

The PyPI distribution is named retro-ai because the retro project name is already occupied on PyPI; the Python import package and CLI command are both named retro.

Global setup and periodic capture

Retro uses one archive for the current OS user, regardless of the directory where retro is invoked. Configure it and install the macOS user LaunchAgent explicitly:

retro setup \
  --archive-root /Volumes/External/retro/rollout-memory \
  --dashboard-dir /Volumes/External/retro/dashboard \
  --periodic 15m \
  --derived-every 6h

pip install never installs a background job by itself. retro setup writes the user config and installs the job. Use a stable pipx or per-user runtime; the scheduler refuses project/worktree .venv executables that may disappear.

retro config show
retro doctor
retro sync
retro schedule status
retro schedule run-now
retro schedule uninstall

Root precedence is --root, RETRO_ROOT, the legacy RETRO_ARTIFACT_ROOT, user config, then the platform per-user data directory. Archive-coupled state and the cross-process lock live under <archive>/state/. Small configuration stays under the platform user data directory. On macOS, scheduler logs deliberately stay on the internal disk at ~/Library/Logs/retro/, so an unmounted external archive cannot hide the failure.

When an archive is under /Volumes/<name>, scheduled sync skips without creating a replacement directory if the volume is not mounted. macOS may also deny background jobs access to removable volumes; retro doctor and retro schedule status identify the Python.app bundle to grant Full Disk Access in System Settings.

retro sync imports only new or changed local source sessions, preserves a raw revision when an upstream source was rewritten rather than appended, updates signals for changed sessions, and atomically switches the dashboard generation. It uses a cross-process archive lock, warns below 5 GiB free, and refuses a scheduled run below 2 GiB. Scheduled capture runs every 15 minutes; expensive signal and dashboard work is coalesced and runs at most every 6 hours by default. Manual retro sync refreshes pending derived work immediately.

Sync records capture → catalog → render → signals → memory → dashboard in <archive>/state/sync.json. Each stage has status and timing data, while the pending section retains only unfinished inputs, so a dashboard failure does not cause completed signal work to run again.

After moving an older archive, preview and repair absolute archive-internal normalized references:

retro archive repair-refs --dry-run
retro archive repair-refs

Archive migration

Migrations are checksummed, resumable, and refuse conflicting overwrites:

retro archive plan \
  --from /path/to/old-a/rollout-memory \
  --from /path/to/old-b/rollout-memory \
  --into "$HOME/Library/Application Support/retro/rollout-memory"

retro archive migrate --plan /path/to/plan.json --rebuild-derived
retro archive verify --plan /path/to/plan.json
retro archive cutover --plan /path/to/plan.json --link-path /path/to/legacy/rollout-memory

Cutover is allowed only after a successful derived rebuild and a newer verification. The original legacy archive is moved to a retained source-backup directory before its old path becomes a compatibility symlink.

Session capture hooks

retro capture start|end writes the immutable repo_start.json / repo_end.json sidecars that Git-backed task selection requires as exact-base proof. Install them as agent-host session hooks so every session gets them without a manual step:

retro capture hooks status                 # per-host installation state
retro capture hooks install --dry-run      # report planned edits only
retro capture hooks install                # all supported hosts
retro capture hooks install --host claude --host codex
retro capture hooks uninstall              # remove only Retro's entries

Host	User-level file	Events	Reference
Claude Code	`~/.claude/settings.json` (honours `CLAUDE_CONFIG_DIR`)	`SessionStart`, `SessionEnd`	hooks reference
Codex CLI	`~/.codex/hooks.json` (honours `CODEX_HOME`)	`SessionStart`, `SessionEnd`	hooks guide
GitHub Copilot CLI	`~/.copilot/hooks/retro-capture.json` (honours `COPILOT_HOME`)	`sessionStart`, `sessionEnd`	hooks reference

Installation is opt-in, never implicit: it is off by default in retro setup and is enabled with retro setup --capture-hooks, because it edits agent configuration outside the archive. Each edited file is copied to a timestamped .bak first and rewritten atomically. Other hooks in the same file are preserved, Retro's own entries are replaced rather than duplicated on re-install, and uninstall removes only the entries Retro added, deleting a file only when nothing else is left in it.

The installed command runs the hidden retro capture hook entry point. It reads the host's JSON payload on stdin, takes the session id (session_id or sessionId) and cwd, and writes the sidecar under raw/<host>/<session-id>/. It is fail-open by design: it never exits non-zero, never writes to stdout (Claude Code injects SessionStart stdout into the model context), returns immediately when cwd is not inside a Git repository, skips an already-captured phase instead of rewriting evidence, and appends failures to capture-hooks.log in the Retro log directory. Set RETRO_HOOK_COMMAND to override the resolved interpreter or console-script path recorded in the hook.

The Copilot file covers Copilot CLI and Copilot sessions on the VS Code Agent Host, which share the same hook implementation; the VS Code Local harness reads the same user-level directory but has no SessionEnd event, so only the start capture can fire there. Two host limits apply to the end capture only: Claude Code treats SessionEnd as observational and does not wait for it, and Codex caps SessionEnd hooks at three seconds. The start capture, which task selection needs as exact-base proof, is unaffected.

retro doctor reports the per-host hook state and start-capture coverage: the share of sessions started in the last 30 days, in a Git repository, that carry repo_start.json.

Architecture at a glance

                  ┌──────────────────────────────────────────────────────────┐
                  │                       retro CLI                       │
                  ├──────────────┬─────────────┬──────────────┬───────────┤
   ~/.claude  ──▶ │  importers   │   catalog   │   signals    │  memory   │
   ~/.codex   ──▶ │  (claude,    │ (identity,  │ + mining over│  index    │
 VS Code User ──▶ │ codex,       │ tags, repos)│ included only│  + weave  │
                  │ copilot)     │             │              │           │
                  └──────┬───────┴──────┬──────┴───────┬──────┴─────┬─────┘
                         ▼              ▼              ▼            ▼
                       raw/        normalized/      catalog/     memories/
                  (immutable)    (events.jsonl)   (sessions)   (files + SQLite)
                         │
                         ▼
                     rendered/   ← markdown view (any time)
                         │
                         ▼
                    retro dashboard build
                  (static HTML + KPIs + per-model cost + signals + drill-down)

Each stage is independent — you can re-render markdown, recompute signals, re-mine, rebuild the memory index, or rebuild the dashboard from whatever rollout-memory/ already has on disk.

Source extensions

Retro extension API 1 lets an installed package add a source host without editing Retro. Source packages register one adapter through the retro.sources Python entry-point group. Retro loads providers lazily and keeps the canonical Layout, NormalizedEvent, raw evidence, signals, mining, memory, and dashboard pipeline in core.

Inspect the active provider set:

retro extensions list
retro extensions doctor

Built-in Claude Code, Codex, and VS Code Copilot integrations use the same runtime as installed providers. Imports run in a staging layout; Retro validates the provider result and atomically promotes the raw and normalized artifacts while holding the archive lock. A broken optional provider does not prevent built-in providers from loading. Duplicate host names fail closed and are reported by extensions doctor.

The repository includes an installable OpenCode reference adapter:

python -m pip install -e "./adapters/retro-opencode"
retro extensions doctor
retro list --host opencode
retro import opencode --latest

OpenCode support is fixture-backed and intentionally conservative. See adapters/retro-opencode/COMPATIBILITY.md before using it with a new OpenCode release.

Provider authors should import only from retro.sdk, declare RETRO_EXTENSION_API = "1", return SessionDescriptor values from discover(), and write captures only through the Layout supplied to the adapter constructor. See docs/extensions.md and specs/source_extension_api_spec.md.

Storage layout

Artifacts land in the configured per-user archive. The platform default on macOS is ~/Library/Application Support/retro/rollout-memory:

rollout-memory/
  raw/<host>/<session-id>/                   # immutable copies of source jsonl + metadata + sidecars
    transcript.jsonl        (claude)
    rollout.jsonl           (codex)
    session.jsonl|json       (vscode-copilot) # verbatim VS Code chat persistence
    session.snapshot.json    (vscode-copilot) # reconstructed JSONL mutation state
    transcript.jsonl         (vscode-copilot) # direct Copilot event log, when available
    events.jsonl             (vscode-copilot) # Copilot CLI / Agent Host rollout
    import_meta.json        (claude)         # source path, project_slug, claude_home, …
    import_meta.json        (vscode-copilot) # workspace, models, sources, capture fingerprint
    thread.json             (codex)          # cwd, model_provider, source_kind, spawn_edges, …
    sidecars/               # Claude todos/tasks or Copilot edits/resources/session-store rows

  normalized/<host>/<session-id>.events.jsonl  # one normalized event per line

  catalog/
    sessions.jsonl                             # derived identity, context, project, tags, exclusion
    state.json                                # derived policy/project invalidation digests
    annotations.jsonl                          # canonical append-only user edits

  rendered/<host>/<session-id>.md             # human-readable markdown view

  mined/<method>/<host>/<session-id>.json     # mined candidates (structured)
  mined/<method>/<host>/<session-id>.prompt.md  # paste-ready prompt block

  signals/
    definitions.json                          # the registered signal set + its digest
    sessions.jsonl                            # one compact line per evaluated session

  memories/
    items.jsonl                               # canonical appended memory records
    events.jsonl                              # memory lifecycle / utility log
    index.sqlite                              # derived SQLite + FTS index, rebuildable

  benchmarks/<benchmark-id>/                  # immutable generated task version
    manifest.json                             # cutoff, snapshot, method, checksums
    tasks/prompts/<level>.jsonl                # isolated learner input for one prompt level
    private/ground-truth.jsonl                # evaluator-only truth + provenance
    private/historical-localization-predictions.jsonl
    knowledge/eligible-event-refs.jsonl        # pre-cutoff evidence boundary

  benchmarks/<name>/task-scorer/               # Git-backed implementation taskset
    selections.json                            # accepted sources and explicit rejections
    sources/<source-id>/                       # immutable rollout/base/outcome evidence
    tasks/<task-id>/public/                    # prompt, base bundle, environment
    tasks/<task-id>/private/                   # scorer, oracle, provenance, validation
    builds/<build-id>/                         # resumable builder stage records
    evals/<eval-id>/                           # attempts and source-normalized report

  benchmark-runs/<benchmark-id>/<run-id>/      # immutable evaluation result
    run-manifest.json
    predictions.jsonl
    results.json
    report.md
    private/task-details.jsonl
    private/runner/                            # optional GhostLab traces and policy

raw/ is treated as immutable: re-importing the same session refuses unless --force is passed.

Capture

Discovery merges each host's default roots and honors comma-separated overrides.

Host	Default roots	Env override
Claude Code	`~/.claude/projects/` and `~/.config/claude/projects/`	`CLAUDE_CONFIG_DIR`
Codex	`~/.codex` (uses `state_5.sqlite.threads.rollout_path`)	`CODEX_HOME`
VS Code Copilot	Platform VS Code/Insiders `User` directories, `~/.vscode-server*/data/User`, and `~/.copilot/session-state`	`VSCODE_COPILOT_USER_DIRS`, `COPILOT_HOME`, `COPILOT_SESSION_STATE_DIRS`

Codex roots are auto-classified:

sqlite_home: has state_5.sqlite → discovery via SQLite (preserves spawn-edge graph).
sessions_dir: has a sessions/ subdir but no SQLite → scan *.jsonl.
jsonl_dir: neither → treat the directory as a flat JSONL bag (e.g. codex exec --json output).

Commands

# Discover what's available on this machine
retro list                           # all hosts, with retention warning for old Claude logs
retro list --host claude
retro list --host codex
retro list --host copilot
retro list --limit 50

# Import a single session
retro import claude --session-id <session-id>
retro import codex  --thread-id  <thread-id>
retro import copilot --session-id <session-id>
retro import claude --latest         # most-recent
retro import codex  --latest
retro import copilot --latest

# Import every discoverable session
retro import all                     # all supported hosts
retro import claude --all
retro import codex  --all
retro import copilot --all
retro import all --limit-per-host 20

# Force overwrite an existing capture
retro import claude --latest --force

# Skip the markdown re-render step
retro import codex --latest --no-render

# Rebuild normalized events (and transcripts) from the archived raw copies after a
# normalizer upgrade; only sessions whose normalizer version changed are rebuilt
retro import renormalize --dry-run
retro import renormalize [--host codex] [--since 2026-10-01] [--no-render]

# Re-render markdown from an already-imported normalized stream
retro render claude <session-id>
retro render codex  <thread-id>
retro render copilot <session-id>

# Inspect what got captured
retro show claude <session-id>
retro show codex  <thread-id>
retro show copilot <session-id>

Multi-root example

# Pull in Claude logs from two locations:
CLAUDE_CONFIG_DIR="$HOME/.claude,$HOME/.config/claude,/backup/claude-archive" retro import claude --all

# Codex saved via `codex exec --json` (no SQLite present):
CODEX_HOME="$HOME/.codex,$HOME/codex-exec-logs" retro import codex --all

# VS Code profiles or remote User roots outside the platform defaults:
VSCODE_COPILOT_USER_DIRS="$HOME/custom-code/User,$HOME/.vscode-server/data/User" retro import copilot --all

# Copilot CLI / Agent Host state outside ~/.copilot/session-state:
COPILOT_SESSION_STATE_DIRS="/archive/copilot/session-state" retro import copilot --all

Capture coverage and retention

When the oldest discoverable Claude transcript is more than 25 days old, retro list prints a yellow warning. Claude Code retains logs for ~30 days by default — capture older sessions before they age out, or raise cleanupPeriodDays in Claude settings.

VS Code Copilot discovery reads both legacy full-session .json files and current append-only .jsonl mutation logs under workspaceStorage/*/chatSessions/. When present, it also captures Copilot's direct event transcript, editing-session snapshots, tool-result resources, workspace metadata, and the matching rows from github.copilot-chat/session-store.db. Empty and non-Copilot chat sessions are excluded.

Copilot CLI and VS Code Agent Host sessions are captured from ~/.copilot/session-state/<session-id>/events.jsonl, including active sessions. The matching rows from ~/.copilot/session-store.db provide titles, workspace metadata, model usage, cache/reasoning tokens, and Copilot request units. Active sessions are point-in-time snapshots; rerun retro import copilot --all after more activity to append the newer source state.

retro can only archive rollout content that exists on the local machine. Account or cloud sessions that have never been downloaded, or whose local event logs were already removed, cannot be reconstructed from VS Code's small telemetry/index entries alone.

Session catalog: ignoring noise and consolidating projects

The catalog derives one record per archived session without re-importing it. It identifies the agent surface, interactive/headless state, subagent parent, cwd, Git remote, project, models, event counts, markers, rule tags, and final excluded decision. Inspect it with:

retro sessions list --included --project github.com/sajjadgg/retro
retro sessions list --excluded --tag scratch --json
retro sessions show codex/<session-id>
retro sessions explain codex/<session-id>

Ignore a test or mimic session while it is running by putting a directive in its title or a user-authored message:

#retro:ignore
#retro:keep
#retro:tag=evaluation
#retro:project=github.com/example/repo

Directives in assistant output, quote lines, inline code, and fenced code do not count. #retro:keep removes an automatic ignore; it does not override a manual ignore. For directory-wide policy, create .retroignore in the working directory or an ancestor. An empty file means ignore; otherwise put one tag per line. Canonical edits are append-only:

retro sessions tag codex/<session-id> ignore
retro sessions untag codex/<session-id> ignore
retro sessions set-project codex/<session-id> github.com/example/repo
retro sessions set-project codex/<session-id> --clear

Built-in rules tag temporary-directory, empty, automated, and subagent sessions. Temporary roots include standard Unix/macOS temp locations and $TMPDIR; a session is empty when it lacks either a user prompt or all assistant/tool activity. Only tags listed in exclude_tags exclude; the default is ["ignore"], so automated alone remains reviewable. Add or dry-run a rule:

retro rules add '{"id":"eval","match":{"prompt":"(?i)^below is a math problem"},"tags":["ignore"]}'
retro rules test --json
retro rules disable builtin.empty

Project identity prefers the current repository remote, falls back to archived Git metadata, and normalizes HTTPS, SSH, and Copilot owner/name forms to one lowercase host/owner/repo. It follows Git worktree commondir files and resolves symlinked cwd prefixes. Temp directories without Git evidence collapse to scratch:temp; Copilot session-state/<id>/files sandboxes without Git evidence collapse to scratch:copilot-session-state. A Git remote/root wins over these synthetic projects. Persist exceptional merges and display names:

retro projects list
retro projects merge path:/old/clone github.com/example/repo
retro projects rename github.com/example/repo "Example"

Excluded sessions remain in raw, normalized, and catalog evidence but do not feed mining, memory reindexing, or task-scorer selection. Use retro mine ... --include-excluded only for an explicit diagnostic override. GhostLab's originator/UUID filter prevents newly marked synthetic sessions from being captured at all; catalog rules handle historical and unmarked test traffic. See specs/rollout_catalog_spec.md.

Signals

Signals are evaluators that observe a captured rollout and emit a comparable value (number / boolean / category). They answer "how did this rollout go, by some objective measure?", and they are derived backwards from the decisions the archive exists to support: which agent/model/client to use for which work, where time and money go, where agents struggle, which steering habits to change, whether a session succeeded, what was risky, and whether the data is trustworthy.

Each session is scanned once into a SessionProfile of facts; every signal is then a pure read over that profile. Signals are organized by category:

Category	Question answered	Examples
`usage`	What did it consume?	`usage.cost_usd`, `usage.tokens_total`, `usage.active_s`, `usage.credits`
`activity`	What did the agent do?	`activity.user_turns`, `activity.tool_calls`, `activity.file_edits`
`outcome`	Did the work land?	`outcome.status`, `outcome.validated`, `outcome.committed`
`friction`	Where did it get stuck?	`friction.tool_error_rate`, `friction.user_corrections`, `friction.longest_error_streak`
`efficiency`	What did it cost per unit of work?	`efficiency.cost_per_edit`, `efficiency.cache_read_share`
`collaboration`	How did the human steer?	`collaboration.first_prompt_chars`, `collaboration.reply_latency_s`
`risk`	What could hurt?	`risk.secret_exposure`, `risk.destructive_commands`
`quality`	Can the numbers be trusted?	`quality.capture_score`, `quality.cost_basis`, `quality.usage_source`
`work`	What kind of work was it?	`work.kind`, `work.primary_language`, `work.validation_commands`

Every signal carries id, title, category, kind, unit, polarity, method, version, description and applicability, plus per-session evidence (event_ids) and confidence. Methods: exact (direct reading), heuristic (rules over facts), regex (text patterns), llm (reserved, unused).

See specs/rollout_signals_spec.md for every algorithm, threshold, known bias and the hand-label validation numbers.

Commands

# Inventory of registered signals
retro signal list
retro signal list --category outcome --describe

# Update signals/sessions.jsonl (incremental: only changed sessions are recomputed)
retro signal run
retro signal run --force

# Restrict scope (keeps the rest of the store intact)
retro signal run --host claude
retro signal run --host codex --session-id <id>

# Inspect one session, or aggregate across all of them
retro signal show claude <session-id>
retro signal summary
retro signal summary --signal outcome.status,usage.cost_usd --json

Output

rollout-memory/signals/definitions.json — the registered signal set and a digest of it ({"schema": 2, "version": "…", "signals": [...]}).
rollout-memory/signals/sessions.jsonl — one line per evaluated session: {"key", "fingerprint", "version", "values", "evidence", "confidence"}.

A session is recomputed only when its catalog fingerprint changes or the signal set changes; a signal that does not apply is omitted rather than stored as a zero. On the reference archive the store is ~2 MB for 1,651 sessions and a no-change update takes well under a second.

Adding a new signal

from retro.signals import SessionProfile, SignalValue, register

@register(
    "friction.my_signal",
    title="My signal",
    category="friction",
    kind="count",
    unit="count",
    polarity="lower_better",
    description="What it measures and how.",
)
def _my_signal(profile: SessionProfile):
    return SignalValue(len(profile.error_events), tuple(profile.error_events[:5]))

Register it in src/retro/signals/definitions.py; retro signal list/run/ show and the dashboard pick it up automatically. Bump a signal's version whenever its algorithm changes — that invalidates the stored values.

Mining

Mining turns captured rollouts into prompt-time memory candidates — paste-ready blocks for the next session. Multiple methods are registered; each produces the same MemoryCandidate shape so they compose. Filters run after methods to prune unsafe / redundant candidates.

Registered methods

Method	What it produces	Source paper
`reme_refine_poc`	Success patterns, failure triggers, tool lessons, risk rules.	ReMe
`skill_pro`	Reusable skills with explicit `activation` / `steps` / `termination` / `verification` blocks.	Skill-Pro
`memp_procedural`	Whole-session procedural memory: `goal` / `preconditions` / `steps` / `warnings` / `outcome`.	Memp
`codex_headless`	LLM-backed extraction through `codex exec --json`; saves the headless Codex JSONL capture.	local Codex CLI

Registered filters

Filter	What it does
`risk_aware`	Drops `risk=high` and low-confidence candidates, dedupes near-duplicates, caps at 8 items.

The default methods are deterministic and local. codex_headless is opt-in and calls codex exec --json on a compact, redacted rollout summary. It writes the Codex headless stream to rollout-memory/headless/codex_headless/<host>/<session-id>.codex.jsonl.

Commands

# Inventory
retro methods

# Single method, single session (default method is reme_refine_poc)
retro mine codex <thread-id>
retro mine claude <session-id> --method skill_pro
retro mine codex  <thread-id>  --method memp_procedural
retro mine codex  <thread-id>  --method codex_headless

# Run every registered method on one session
retro mine codex <thread-id> --method all

# Apply one or more filters after mining
retro mine codex <thread-id> --method skill_pro --filter risk_aware
retro mine codex <thread-id> --method all       --filter risk_aware

# Bulk mine
retro mine '*' '*' --method all --filter risk_aware
retro mine codex  '*' --method skill_pro
retro mine claude '*' --method memp_procedural --all

# Explicit diagnostic override for a catalog-excluded session
retro mine codex <thread-id> --include-excluded

Output

For each (method, host, session_id):

rollout-memory/mined/<method>/<host>/<id>.json — full structured result (candidates, structured payloads, evidence refs, notes, filters applied).
rollout-memory/mined/<method>/<host>/<id>.prompt.md — paste-ready memory block tagged with <retro method="…">, sorted by priority and confidence.

Example skill_pro output

Skill: investigate and patch a bug
  Activation: gi ahead and fix the issues found here
  Steps:
    1. TaskCreate (Scan both ~/.claude/projects/ and ~/.config/claude/projects/…)
    2. read file (file_path=~/…/retro/importers/claude.py)
    3. edit file (file_path=~/…/retro/importers/claude.py)
    …
  Termination: after writing ~/…/Mem/specs/ccusage_comparison_spec.md
  Verification: (no explicit verification observed)

Adding a new method

# src/retro/mining/methods/my_method.py
from ..base import MiningContext, MiningResult, MemoryCandidate, register_method, memory_id

@register_method("my_method", description="…")
def mine_my_method(ctx: MiningContext) -> MiningResult:
    candidates = [
        MemoryCandidate(
            id=memory_id(ctx.session_id, "my_method", 1),
            method="my_method",
            kind="procedure",
            text="…",
            when_to_use="…",
            evidence_refs=[ev.event_id for ev in ctx.events[:3]],
            confidence=0.6,
            priority=4,
        )
    ]
    return MiningResult(
        session_id=ctx.session_id, host=ctx.host, method="my_method",
        task_summary=ctx.task_summary(), candidates=candidates,
    )

Import it in mining/methods/__init__.py. retro methods and --method pick it up.

Time-consistent benchmarks

retro benchmark implements the construction and evaluation protocol from A Time-Consistent Benchmark for Repository-Level Software Engineering Evaluation, adapted to private rollout data. The paper creates tasks from pull requests merged after a cutoff. Retro uses a post-cutoff user goal episode as the task and the episode's file-edit events as hidden modified-file ground truth.

The core validity boundary is unchanged:

repository snapshot and augmentation knowledge are fixed at or before T0;
tasks come only from (T0, T1];
generated prompts never expose the hidden changed-file names;
baseline and augmented predictions are compared only when task, model, prompt level, snapshot, and metric match.

The paper defines but does not publish exact templates for its four prompt levels. Retro therefore uses a deterministic, local, leakage-filtered reconstruction:

Level	Retro contract
`minimal`	High-level action sentence only
`concise`	Minimal task plus affected component
`contextual`	Problem/use context and component, without inline implementation details; default
`guided`	Richer source context plus change-shape and validation guidance

Each level is written to a separate task file. A minimal-prompt run therefore cannot read the contextual or guided variant, post-cutoff event references, expected file count, or modified-file truth.

Build and evaluate

Both --cutoff and --end must be timezone-aware. Benchmark versions and run IDs are immutable; choose a new ID rather than overwriting an earlier result.

# Build tasks from all matching captured hosts for one local repository.
retro benchmark build cortex-20260601-v1 \
  --project /path/to/repository \
  --cutoff 2026-06-01T00:00:00Z \
  --end 2026-08-01T00:00:00Z \
  --host '*'

# Inventory benchmark versions.
retro benchmark list

# Score a model or agent run.
retro benchmark evaluate cortex-20260601-v1 predictions.jsonl \
  --run-id baseline-contextual-v1

# Score the generated diagnostic derived from the captured agent's pre-edit
# inspection events. This is not a generated-prompt baseline.
retro benchmark evaluate cortex-20260601-v1 \
  "/path/to/rollout-memory/benchmarks/cortex-20260601-v1/private/historical-localization-predictions.jsonl" \
  --run-id historical-observed-v1

Each prediction JSONL row has this contract:

{"task_id":"tcl_...","condition":"baseline","model":"model-id","prompt_level":"contextual","predicted_files":["src/example.py","tests/test_example.py"]}

Use repository-relative paths. By default, every (condition, model, prompt_level) group must predict all tasks. Pass --allow-partial only for explicitly diagnostic runs. If both baseline and augmented groups are present, Retro reports the matched per-task mean F1 delta and win/tie/loss counts.

Task metrics are exact file-set precision, recall, and F1. The paper's score is the unweighted mean task-level F1; Retro also reports micro metrics, exact-match rate, zero-precision rate, perfect-recall rate, and zero-recall rate. This is a localization benchmark, not evidence that a patch is correct or that tests pass.

Run tasks safely with GhostLab and OpenShell

The optional sandbox runner requires Python 3.10 or newer, GhostLab, and a connected NVIDIA OpenShell gateway:

python3.11 -m venv .venv
.venv/bin/pip install -e ".[sandbox]"
openshell status

Run one fresh OpenShell sandbox per benchmark task:

export COPILOT_GITHUB_TOKEN=...

retro benchmark run cortex-20260601-v1 \
  --model mai-code-1.1-flash \
  --prompt-level contextual \
  --run-id mai-ghostlab-contextual-v1 \
  --workers 2

If a Copilot-capable GitHub token is already stored by Git's credential helper, --use-git-credential explicitly requests it without writing the value to an artifact. Environment variables are otherwise never inherited implicitly.

The runner:

materializes the pinned commit using git archive, so the sandbox receives no .git history, dirty working tree, hidden verifier, or benchmark files;
marks the uploaded snapshot read-only and disables Copilot shell, write, URL, memory, custom-instruction, remote-control, and built-in MCP capabilities;
uses a provider-only OpenShell egress policy and forwards only the selected credential variable;
creates and destroys an independent sandbox for every task;
rejects unexpected tools, model errors, malformed predictions, and incomplete task sets rather than producing success-shaped output;
stores raw task traces and OpenShell logs under the run's mode-0700 private/runner/ directory and checksums them in the immutable run manifest.

--use-git-credential is opt-in because it grants the isolated model process access to the retrieved token for the duration of the run. The value is removed from Retro's process environment afterward and never included in run metadata.

Build implementation tasks with hidden scorers

retro benchmark taskset is separate from the file-localization benchmark above. It turns a Git-backed rollout with an exact clean base and durable accepted outcome into a single-message implementation task, a hidden executable scorer, and numeric [0, 1] results.

Capture repository state from the coding host's lifecycle hook before the first agent action and after the final action:

retro capture start --host codex --session-id <session-id> --cwd /path/to/repo
retro capture end --host codex --session-id <session-id> --cwd /path/to/repo

Install those two calls as session hooks once instead of running them per session (see Session capture hooks):

retro capture hooks install

The capture records HEAD, its tree, porcelain-v2 status, and recursive submodule status. Existing files are never overwritten. By default, historical rollouts are accepted only when equivalent exact base evidence exists in the rollout.

Construct and run a taskset through Ghostlab's public artifact and scorer commands:

retro benchmark taskset select \
  --name personal-git-v1 \
  --host codex \
  --session-file sessions.txt \
  --environment-config /outside/repo/retro-environment.json

retro benchmark taskset bundle \
  --name personal-git-v1 \
  --selected-only

retro benchmark taskset build \
  --name personal-git-v1 \
  --ghostlab-bin /path/to/ghostlab \
  --task-definer-agent agents/task-definer.json \
  --scorer-builder-agent agents/scorer-builder.json \
  --scorer-auditor-agent agents/scorer-auditor.json

retro benchmark taskset run \
  --name personal-git-v1 \
  --agent candidate-agents/codex.json \
  --seeds 0,1,2 \
  --ghostlab-bin /path/to/ghostlab

retro benchmark taskset report \
  --name personal-git-v1 \
  --eval latest

Historical sessions without exact clean-start proof have a separate, opt-in lane:

retro benchmark taskset select \
  --name reconstructed-v1 \
  --session-file sessions.txt \
  --allow-reconstruction \
  --environment-config /outside/repo/retro-environment.json

Reconstruction first tries the parent of the rollout's first commit, then a recorded rollout HEAD, then the last commit at or before the session start on the branch recorded in session metadata. It never reads reflogs. Retro expands that anchor into an isolated, Git-free scratch snapshot, replays normalized apply_patch, edit, write, and create operations, and compares every touched path with the durable outcome. A source is accepted only when every path has matching pre-edit evidence, every replayed post-state matches the outcome, no shell or otherwise unverifiable edit remains, and reconstruction confidence is at least 0.8 (full verification currently yields 1.0). Failures remain visible as RECONSTRUCTION_PRE_MISMATCH, RECONSTRUCTION_POST_MISMATCH, or RECONSTRUCTION_UNVERIFIABLE_EDIT. The selection and source-bundle manifests record the anchor method and per-file verification.

Every stage is content-addressed and resumable. The evaluated agent receives only the prompt, immutable base repository, and public environment contract. taskset run overrides the agent's sandbox image with that contract's pinned image and executes its argument-array setup command before the agent turn. The outcome, rollout, reference solution, scorer, and provenance remain private. Scorers reward observable behavior rather than patch similarity. Reports macro-average tasks within each rollout source and then sources within the benchmark, so sources that yield multiple tasks do not receive extra weight. Agent failure, harness failure, scorer failure, and a correctly observed task failure remain distinct statuses.

Commit-mined sources and outcome-informed sampling

Most historical rollouts never captured clean-start proof, so they are rejected with NO_EXACT_BASE_SHA. A commit does not have that problem: its first parent is the pre-change state. --sources commits (or both) mines non-merge commits from the repositories Retro already saw in your catalog, or from the repeatable --commit-repo:

retro benchmark taskset select \
  --name personal-git-v1 \
  --sources both \
  --commit-window-days 180 \
  --strategy stratified --seed 0 --limit 40 --max-per-repo 10 \
  --environment-config /outside/repo/retro-environment.json

Mining is read-only: Retro runs git rev-list, git show, git diff and git cat-file and never checks anything out in your working tree. A commit is eligible when it changes a non-test source file, is not a merge, revert, or a docs/lockfile/vendored-only change, stays inside --commit-max-files (default 50) and --commit-max-lines (default 2000), and was not reverted inside the stability horizon. The bundle records whether the commit touched tests. When a captured session ran in the same repository, overlaps or precedes the commit by at most --commit-link-window-hours (default 6) and edited files the commit changed, Retro links it, keeps the rollout in the bundle, and uses your real first prompt as prompt provenance; otherwise the commit message is the provenance, marked commit_message, for the TaskDefiner to back-translate.

Selection is sampled rather than sorted. The default --strategy stratified walks (repository, work kind, diff size) strata round-robin with a seeded order that puts informative outcomes first: sources you corrected or that failed and were fixed rank above clean surviving successes, which rank above unlabeled ones. --seed reproduces an ordering exactly, --max-per-repo stops one repository from dominating (default 25% of --limit when limiting), and --strategy lexicographic restores the original ordering. Every selected source carries outcome_labels — commit survival, pushback proxies, signal outcome, each with evidence references — and the command prints yield by source kind, outcome label and rejection code.

The environment resolver prioritizes an explicit external config, then a project Dockerfile/devcontainer declaration, validated CI commands, and an explicit RepoLaunch executable. It validates fresh base and outcome checkouts twice and records only digest-pinned images. Validation and agent/scorer runs have network disabled. Dependency builds also default to no network; an allowlist is accepted only when paired with a named, externally egress-filtered container network.

Curated benchmark releases and training episodes

Freeze the published candidate pool before comparing agents or exporting training data:

# Default: exact-provenance, deterministic, evaluation-only release.
retro benchmark taskset curate --name personal-git-v1 --suite release-v1

retro benchmark taskset run --name personal-git-v1 --suite release-v1 \
  --agent candidate-agents/codex.json --seeds 0,1,2 --eval heldout-v1
retro benchmark taskset run --name personal-git-v1 --suite release-v1 \
  --agent candidate-agents/custom-script.json --seeds 0,1,2 --eval heldout-v1

curate --policy /outside/repo/curation.json can explicitly create train/validation/test splits, group related task families, exclude candidates with reasons and assign training-sampling priorities. Repository grouping is the default; insufficient independent groups are rejected rather than split randomly by task. Anchored historical reconstructions require an explicit allow_reconstructed policy and are never labelled exact replay. Moved or deleted worktrees can be reconnected with select --repo-root /current/checkout; the original Git evidence is still required and raw captures are not changed.

For a suite with a populated training split:

retro benchmark taskset export --name personal-git-v1 --suite training-v1 \
  --split train --out /path/to/train-export --include-rewards
retro benchmark taskset reset --name personal-git-v1 --suite training-v1 \
  --task-id <id> --out /path/to/episode-1
retro benchmark taskset reward --name personal-git-v1 --suite training-v1 \
  --task-id <id> --candidate /path/to/candidate-state.tar.zst \
  --out /path/to/reward-1 --ghostlab-bin /path/to/ghostlab

The actor gets only public prompt/environment and an exact Git-free repository tree. Optional hidden verifiers are exported separately under private/ for the trusted harness, never the actor. Container images are digest references, not bundled layers. Reset does not run setup or agent code on the host. Terminal rewards reuse the benchmark scorer: observed task failure is numeric zero; infrastructure failure is null, persisted with an error and a nonzero CLI exit. This is an environment/export interface, not an included RL optimizer or a Gymnasium action loop.

Curated runs bind the suite hash, split, task IDs, scorer/base hashes and seeds. Missing attempts reduce coverage rather than disappear. Reports include source-normalized scored rewards, valid coverage, per-task seed spread, conditional pass@k and source-bootstrap intervals; incomplete coverage is not a finished leaderboard result. The dashboard's Tasks view opens with cards for the current verified tasks: the goal, isolated repository environment and what the hidden grader requires. Click a task for its full instructions, reset/work/grade contract and success criteria; exact versions, source evidence and validation are expandable. Creation history & older versions retains candidate yield/rejections and prior releases. Warnings explain which saved records cannot be compared and why; they do not imply that the current tasks failed. Evals contains frozen task datasets, not model scores. Experiments owns job setup, agent results and per-trial grading evidence; results remain scoped to each evaluation cohort.

See the release design and research rationale for the exact policy schema, numerical gates, split algorithm and limitations.

Memory backend

The memory backend turns mined and authored memories into a local SQLite index. Flat files remain the source of truth; index.sqlite is a derived cache that can be deleted and rebuilt.

The index currently supports:

SQLite schema bootstrap with FTS5 keyword recall.
Reindexing from rollout-memory/memories/items.jsonl and existing mined artifacts.
[[wiki-link]] edge extraction and one-hop linked-memory expansion.
Authored markdown import with simple frontmatter.
Keyword retrieval with scope/repo filtering and value-aware reranking.
Utility updates through memories/events.jsonl.
Prompt-time context weaving.
Dashboard counts for indexed memory, status/scope/kind, top utility, and lifecycle events.

Commands

# Create rollout-memory/memories/ and an empty index
retro memory init

# Rebuild index.sqlite from files and mined artifacts
retro memory reindex

# Inspect index counts and dangling wiki-links
retro memory doctor

# Import hand-authored markdown memories
retro memory import-authored ~/path/to/memory-notes

# Retrieve ranked memories; include candidates by default while promotion matures
retro memory retrieve --query "pytest retrieval" --cwd /path/to/repo
retro memory retrieve --query "pytest retrieval" --cwd /path/to/repo --accepted-only

# Emit a compact prompt block
retro memory weave --query "pytest retrieval" --cwd /path/to/repo

# Update observed utility after a memory is used
retro memory update-utility --memory-id <id> --reward 0.8 --session-id <session-id>

Authored markdown

Authored memories are normal markdown files. Frontmatter is optional; missing fields get conservative defaults.

---
id: pytest-policy
kind: tool_lesson
scope: global
status: accepted
risk: low
when_to_use: Use when editing tests.
---
Run pytest after changing retrieval. Link to [[debugging-policy]] when relevant.

Accepted memories that contain prompt-injection markers, credential-looking strings, or invisible control characters are downgraded to needs_review on write.

Optional embeddings are still intentionally out of the core path for now; keyword retrieval works with no native dependencies.

Dashboard

A static, offline dashboard over the session catalog and the signal store. It opens straight from disk without a server, bundler or network. An opt-in localhost server adds experiment controls and private grader inspection; exported dashboards stay read-only.

Page	Answers
Overview	What happened recently across all agents and projects, versus the previous period: sessions, active hours, estimated cost, outcome mix, work-kind mix, notable movers, streak
Projects	Which projects cost time and money; one row per consolidated project (worktrees, clones and symlinked paths collapse into one), with the per-agent split, success rate, tool error rate and last activity
Sessions	Every captured session, filterable by text, agent, client, project, tag, outcome, work kind, model, interactive/automated and date; sortable and virtualized for tens of thousands of rows, with toggles for excluded sessions and subagent roll-up
Session	One session: header, tags and why each was applied, inline markers, signals grouped by category with evidence event ids, the subagent tree, and links to the transcript, normalized events and raw capture in the archive
Signals	Every signal in `signals/definitions.json` with its distribution, compared across agents, clients, projects, models or work kinds with a minimum-sample guard
Tasks	Task/environment cards, full instructions, declared grader input/output contract, source evidence and expandable validation/audit; no agent results
Evals	Current verified frozen task datasets, splits, exact versions and their tasks; older versions remain available separately
Experiments	Frozen job setup, candidate models/scripts, seeds, explicit starts, cohort-bound results and every trial's grade, failure evidence and cleanup record
Data quality	Why sessions were excluded (by tag and rule, with examples), interactive vs automated share, unknown-event rate per host, missing or stale signals, and the latest sync run with its failures

The build reads only published files — catalog, signal and sync contracts plus benchmark taskset/release/evaluation metadata — never the normalized event stream. It does not embed hidden scorer source, accepted patches or candidate code. Rendered transcripts are linked in place and never copied. Private grader source, judge prompts and actual saved input/output JSON are available only through authenticated, on-demand localhost inspection, never static exports. Deterministic graders have code and input artifacts, not an LLM prompt.

Warning panels have Dismiss and Show dismissed warning buttons. Dismissal is browser-local and applies to the exact warning set; new warnings reappear. Dismissal does not repair old records or permit otherwise invalid comparisons.

Session rows are shipped as a compact fact table in data/sessions-*.js shards; the browser does the slicing and aggregation. Per-session detail lives in hashed buckets that are rebuilt only when a session's catalog fingerprint changes. index.html is a two-kilobyte shell.

Preview

Session — one rollout with its provenance, first prompt and every signal grouped by category:

Tasks — frozen task environments mined from real sessions and how each hidden grader decides success:

Experiments — saved agent runs bound to exact dataset versions, with verification status and explained warnings:

Data quality — what the archive contains, why sessions were excluded and capture health per host:

Build

# Build and publish a new generation (reuses unchanged detail shards)
retro dashboard build

# Rebuild every shard
retro dashboard build --mode full

# Build from a non-default archive / into a non-default output directory
retro dashboard build --root /path/to/rollout-memory --out /path/to/output

# Write straight into --out without the generation swap
retro dashboard build --no-publish

The archive root and dashboard output come from the per-user config; override either for one command with --root and --out. Then open the configured dashboard/index.html directly from disk. Each build stages into a temporary directory and swaps the current symlink atomically, keeping at most two generations.

retro dashboard view     # the same data in the terminal

Experiments and local controls

# Register trusted candidate configurations, then open the printed localhost URL.
# This starts no agent/model jobs.
retro dashboard serve \
  --agent /path/to/candidate-a.json \
  --agent /path/to/candidate-b.json

# CLI alternative: freeze a plan, then explicitly start it.
retro experiment create --id comparison-1 --title "Candidate comparison" \
  --description "Which candidate preserves the requested behavior?" \
  --dataset pilot/personal-v1/test \
  --agent /path/to/candidate-a.json --agent /path/to/candidate-b.json \
  --seeds 0,1,2
retro experiment run comparison-1
retro experiment show comparison-1 --json
retro dashboard build

# Optional finished-job export cleanup; task bundles and grade evidence stay intact.
retro experiment cleanup comparison-1 --prune-artifacts

Repeat --dataset to include multiple frozen datasets. Each dataset has its own cohort; its agents share the exact tasks, split and seeds. Setup pins the suite hash, agent configuration hash and referenced assets. Changing those inputs after planning fails explicitly rather than using a different setup.

The server binds only 127.0.0.1, checks request origin/Host and a per-process control token, and accepts only registered agent IDs. UI-created jobs default to at most 100 requested trials and 3600-second timeout overrides; configure the trial cap with --max-attempts. Saving/viewing a plan never calls a model. Starting it is a separate explicit action and may incur charges. Ghostlab's configured sandbox backend must be available. The server stays open until an active job has finished and cleaned up.

Jobs live in rollout-memory/experiments/<id>/setup.json and job.json, separate from frozen task datasets. Every candidate evaluation prompt carries #retro:ignore and experiment tags; the default catalog also recognizes proven historical Ghostlab provenance and its recorded descendants. Experiment traces remain inspectable but do not feed normal learning/mining/analysis workflows. Explicit inspection APIs and intentional catalog overrides remain available.

Local duplicate workspaces/runtime inputs and known scorer staging copies are cleaned after a trial. Saved grades, attestations, input JSON and compact evidence remain; bulky exported candidates can be pruned explicitly after a job finishes. Ghostlab owns sandbox teardown. Retro never globally prunes Docker, OpenShell, task bundles or captured raw data. Automatic archive TTL is not implemented; Docker execution alone does not remove exported files on disk.

Cost computation

LiteLLM pricing snapshot bundled at src/retro/pricing/litellm-pricing.json is the authoritative source for per-model rates (USD per token). The hardcoded DEFAULT_RATES table is the fallback for any model not in the snapshot.
Per-model attribution: each token delta is tagged with the active model (turn_context.model for Codex, per-message model for Claude). Multi-model sessions produce per-model rows.
Copilot usage: VS Code request snapshots provide prompt/completion totals; Copilot CLI and Agent Host sessions add per-model input, output, cache read/write, reasoning tokens, and Copilot request units from the local session database.
Codex token deltas prefer the explicit info.last_token_usage per-turn delta when present, falling back to cumulative subtraction for older Codex builds.
Cached-input handling follows each source format: Codex totals include cached input; Copilot Agent Host totals are split into fresh input, cache reads, and cache writes during normalization; Anthropic input is already non-cached.
Cost source labeling: each session records whether its cost was embedded (provider-supplied) or calculated (token math).

Numbers are always estimates, never billing truth.

Pricing snapshot

The dashboard pulls rates from the bundled src/retro/pricing/litellm-pricing.json — a curated subset of LiteLLM's model_prices_and_context_window.json. Refresh it with:

python dashboard/pricing/refresh.py

The refresh script:

Reads the existing snapshot's model list (top-level keys).
Downloads LiteLLM's upstream JSON.
Replaces pricing fields for known models from upstream; leaves the local set otherwise.
Updates _meta.snapshot_taken.

To add a model: add an empty entry under its name in the snapshot, then re-run refresh.py.

Release and publishing

CI lives in .github/workflows/ci.yml. It runs on pushes and pull requests to main, installs core plus the OpenCode reference adapter across Python 3.9-3.13, compiles the packages, smoke-tests extension diagnostics, runs both test suites, builds both wheel/sdist pairs, and validates them with twine check.

Publishing lives in .github/workflows/publish.yml. It runs when a GitHub release is published, builds the package, publishes it to PyPI with trusted publishing, and attaches the wheel/sdist to the GitHub release.

The core publishing workflow publishes retro-ai only. The retro-opencode reference package remains repository-local until it has its own release and trusted-publisher configuration.

One-time PyPI setup:

Create or claim the PyPI project retro-ai.
Add a trusted publisher for repository sajjadGG/retro.
Use workflow .github/workflows/publish.yml.
Use environment pypi.

Release flow:

# bump version in pyproject.toml first
git tag vX.Y.Z
git push origin vX.Y.Z

Then publish a GitHub release for that tag. The release event triggers the PyPI publish job.

Environment

Variable	Effect
`CLAUDE_CONFIG_DIR`	Override Claude data roots (comma-separated). Wins over the default `~/.claude` + `~/.config/claude`.
`CODEX_HOME`	Override Codex roots (comma-separated). Each root is classified into `sqlite_home`/`sessions_dir`/`jsonl_dir`.
`RETRO_COST_MODE`	Default `--mode` for the dashboard builder. One of `auto`, `calculate`, `display`.
`RETRO_ARTIFACT_ROOT`	Default `--artifact-root` for the dashboard builder (the `rollout-memory/` directory to read).

Notes and guarantees

Local-only. Nothing is uploaded.
Raw is immutable. Re-imports refuse to clobber raw/ unless --force is passed.
Capture gaps are reported, not hidden. Unknown event types become unknown normalized events with the original payload preserved; counts surface in retro show, the unknown_event_count signal, and the dashboard.
Everything is evidence-linked. Signal readings and mined memories carry the event_ids they were derived from, so the dashboard can trace any claim back to the source rollout.
Composable. Capture, signals, mining, and the dashboard each consume the layer below from disk. Re-run any stage independently.
Designed beyond ccusage on purpose. ccusage reports tokens and costs. This project captures the whole rollout, evaluates it with signals, and turns it into reusable behavioral memory. See specs/ccusage_comparison_spec.md for the full comparison.

Project layout

src/retro/
  schema.py              # NormalizedEvent + read/write helpers
  storage.py             # artifact layout
  cli.py                 # Typer CLI (`retro`)
  renderer.py            # Markdown view
  memory_store.py        # SQLite memory index (FTS5, wiki-links, weave)
  llm.py                 # headless `codex exec` helper
  dashboard/
    inputs.py            # readers for catalog/, signals/, sync/runs/
    facts.py             # fact table, signal matrix, quality aggregates
    metrics.py           # shared aggregation helpers
    shards.py            # data/*.js shard writers + detail bucket cache
    build.py             # build orchestration
    publish.py           # atomic generation publication
    terminal.py          # interactive terminal view (`retro dashboard view`)
    assets/              # index.html, app.css, core.js, views.js, app.js
  pricing/
    litellm-pricing.json # bundled LiteLLM rates snapshot
  importers/
    base.py              # ImportResult
    claude.py            # Claude Code transcript reader (multi-root + env override)
    codex.py             # Codex SQLite + JSONL reader (multi-root)
  signals/
    profile.py           # one scan per session -> SessionProfile of facts
    usage.py, pricing.py # per-host token accounting + model rates
    commands.py, patterns.py, classify.py   # command/text understanding, outcome + work kind
    registry.py, definitions.py             # signal metadata + the catalogue
    store.py, sources.py, summary.py        # incremental store, inputs, aggregation
    analyze.py           # cross-session command/tool report (`retro analyze`)
  mining/
    base.py              # MemoryCandidate, MiningResult, registries
    render.py            # prompt-block markdown rendering + write helpers
    methods/
      reme_refine.py     # success patterns + failure triggers + risk rules
      skill_pro.py       # reusable skill blocks
      memp_procedural.py # whole-session procedural memory
      codex_headless.py  # LLM-backed extraction via `codex exec --json`
    filters/
      risk_aware.py      # prune high-risk / low-confidence / near-duplicates

dashboard/
  pricing/refresh.py     # refresh the bundled snapshot from upstream LiteLLM
  README.md

specs/                    # design docs (see top of this README)
rollout-memory/          # captured artifacts (gitignored)

Name		Name	Last commit message	Last commit date
.github/workflows		.github/workflows
adapters/retro-opencode		adapters/retro-opencode
assets		assets
dashboard		dashboard
docs		docs
specs		specs
src/retro		src/retro
tests		tests
.gitignore		.gitignore
AGENTS.md		AGENTS.md
CHANGELOG.md		CHANGELOG.md
CONTRIBUTORS.md		CONTRIBUTORS.md
LICENSE		LICENSE
README.md		README.md
REVIEW.md		REVIEW.md
pyproject.toml		pyproject.toml

Latest commit

History

Folders and files

Repository files navigation

retro

What it does

Table of contents

Install

Global setup and periodic capture

Archive migration

Session capture hooks

Architecture at a glance

Source extensions

Storage layout

Capture

Commands

Multi-root example

Capture coverage and retention

Session catalog: ignoring noise and consolidating projects

Signals

Commands

Output

Adding a new signal

Mining

Registered methods

Registered filters

Commands

Output

Example skill_pro output

Adding a new method

Time-consistent benchmarks

Build and evaluate

Run tasks safely with GhostLab and OpenShell

Build implementation tasks with hidden scorers

Commit-mined sources and outcome-informed sampling

Curated benchmark releases and training episodes

Memory backend

Commands

Authored markdown

Dashboard

Preview

Build

Experiments and local controls

Cost computation

Pricing snapshot

Release and publishing

Environment

Notes and guarantees

Project layout

About

Topics

Resources

Contributing

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages