reproit-proof

Evidence for what Repro It has actually done, structured so you can check it instead of trusting it. The fastest way in:

./repro setup            # once: installs the pinned Playwright Chromium
./repro vert-route-57

Your machine clones VERT at two pinned commits, builds both from their lockfiles, and drives a real browser at each. On the affected commit, opening /about renders the home page, three runs out of three. On the fixed commit it renders the About page, three out of three, and the legal behavior still works. ./repro alone lists the other cases; ./repro doctor tells you what your machine is missing.

Layout

• cases/: executed bugs. Real app, pinned affected and fixed commits, three clean runs each way, a control, a minimized trigger, digest-verified evidence, and a repro.json recipe.
• studies/: OSS-100. 1,196 public issues from 100 repos classified against a frozen capability rubric. Modeled judgment, zero execution.
• oracles/: per-detector records. Exact predicate, implementation commits, planted red/green fixtures, clean corpora, known false-positive fixes.
• reviews/: every executed campaign result that did not become a case, each with one classification and the exact missing evidence.
• production/: SDK-to-Cloud-to-local captures. Empty; its README says why.
• site/: static HTML generated from the records above, committed deliberately as the publish artifact so every regeneration is a reviewable diff. Open site/index.html.
• harness/: vendored browser probe with pinned Playwright, so reproduction needs nothing outside this repo.

The rules the validator enforces

Numbers from different directories are never added together. Concretely:

• 723 (or 725 reconciled) of 1,196 is modeled detectability: an agent read issue text against a rubric. Nothing was run. The defensible floor from the same data is 245 (high-confidence only).
• 3 is the executed count: bugs proven on affected-versus-fixed builds.
• 0 is the production count: no real SDK capture has gone through Cloud to a verified local replay yet.
• An implemented oracle proves its mechanism on planted fixtures, not that any public issue it was inspired by would reproduce.
• ABSTAIN, incomplete runs, tool failures, and owned demo fixtures live in reviews/, outside every total. The Product Hunt demo is in there, labeled as the owned fixture it is.

scripts/validate.py recomputes every published total from the raw records, verifies every SHA-256 against both the copy and (when present) the original source file, rejects run identities that drift, and refuses repro recipes whose commits differ from their case record. If someone edits a verdict or a number by hand, validation fails.

Post-study capability changes never edit the frozen OSS-100 records. They go through studies/oss-100-detectability/reclassifications.json, pinned to CLI commits, issue by issue. One reclassification is recorded as disputed (Insomnia #5167) rather than resolved silently.

Commands

./repro                  # list cases; doctor / setup / <case-id>
python3 scripts/validate.py
python3 -m unittest discover -s tests -p 'test_*.py'
python3 scripts/generate_site.py

Everything is Python standard library plus the vendored harness. Reproduction additionally needs git, Node, and the case's package manager (Bun for VERT, pnpm for Slidev).

Cases

• vert-route-57: opening /about directly rendered the home page. One-action trigger.
• slidev-monaco-2297: pressing Space inside the Monaco editor advanced the presentation. Two-action trigger.

Both are validated-capability: controlled field evaluations. Neither claims automatic discovery or production capture; a case that did would need status production-to-local, which requires the complete evidence chain and which no case currently has.

What does not belong here

Raw campaign scratch, application checkouts, credentials, and unreviewed evidence. Large public artifacts belong in content-addressed storage with their digest kept in the case manifest.