[tobyvee]

Closes ENG-016 (Wave 4), implementing PRD-001. Stacked on #11 (ENG-014) — base it on main once #11 merges. At the architecture/system-design stage the senior technical roles independently rate candidate implementation directions; a chance-corrected inter-rater agreement coefficient gates whether the aggregate winner is adopted or sent to a tie-breaker. Phase 1 ships advisory (records the round + agreement score, doesn't block) behind CONSENSUS_ENABLED.

What's added

• core (pure, heavily unit-tested)
  • Agreement coefficients: Krippendorff's alpha (primary — nominal/ordinal/interval over a units×raters matrix, missing-data tolerant), Fleiss' kappa (single-pick) and Kendall's W (ranked). Degenerate cases (no variation / unanimity) map to 1, never NaN (the explicit risk in the ticket).
  • decide() — the aggregate winner (vote tally / Borda count) gated on coefficient ≥ threshold with a strict no-tie check. The coefficient is a confidence signal feeding the rule, not a hard inference (PRD's honest caveat for 3 raters).
  • zod schemas: ConsensusCandidate / Rating / ConsensusConfig / ConsensusRound / ConsensusOutcome.
• agents — proposeDirections (2–4 genuinely distinct candidates) and rateDirections (one rater, independent, structured output constrained to the candidate ids — ENG-009).
• server — durable directionConsensus workflow: generate candidates → rate the three roles in parallel (barrier) → decide → branch. Tie-break = human escalation via the architecture_decision gate (signal + approveArchitecture client fn + POST /api/epics/:id/approve-architecture, mirroring approveRoadmap). Budget-enforced per rater (invariant Wave 2 — repo provisioning + repo-aware coding agents #3), audited (invariant Wave 1 — API auth, structured outputs, sandbox env scrub #2), and emitted to the decision log (ENG-014), traceable to the originating request (invariant Wave 0 — project backlog, CI, and Temporal workflow tests #1). No-op when <2 viable candidates exist or the runtime/budget is unavailable. Rubric/mode/threshold/tie-breaker are per-unit config (invariant fix(worker): connect to TEMPORAL_ADDRESS so the worker reaches Temporal in Docker #6).

Acceptance criteria

• core schemas + ConsensusConfig for round/rating/outcome (zod).
• Pure, unit-tested agreement fn — Krippendorff primary + Fleiss'/Light's + Kendall's W; perfect/none/partial/degenerate covered.
• Pure aggregation/decision fn (winner + consensusReached vs threshold).
• Rater agent emits structured ratings for each role, independently; candidate generation yields 2–4 distinct directions.
• Durable consensus step runs the parallel raters, computes agreement, branches on the threshold; budget-enforced + audited.
• Tie-break default = human escalation via the architecture_decision gate; resolution recorded with the deciding identity.
• Round/ratings/outcome linked into the decision log, traceable to the originating request.
• Config-driven; audited no-op/skipped path when the runtime is unavailable.
• Workflow tests cover consensus-reached and tie-break→gate (plus advisory + no-op).

Phasing

Phase 1 here (advisory, behind a flag; both single-pick and ranked supported in decide). Phase 2 (promote to a real gate via CONSENSUS_ADVISORY=false) and Phase 3 (agent casting-vote / dashboard view) are wired-for but out of scope.

Tests

pnpm typecheck · lint · test (144) · build all green.

🤖 Generated with Claude Code