Implement publication discovery, normalization, and deduplication #27
Copy link
Copy link
Closed
Description
Discovery readiness is verified in PR #35; merged and deployed. Activation and real weekly operation remain in #28. Updated September 10, 2026.
- Implement pagination, incremental lookback/reconciliation, caching, bounded retries/request budget, normalization, duplicate grouping, and separate candidate storage.
- Preserve editorial corrections, decisions, alternate observations, rejections, and deferrals; never publish or delete automatically.
- Configure anchored discovery identities for all nine current members, including Rongwu; retain mixed/split-profile warnings.
- Complete inclusive year-level membership screening under Verify the lab roster and publication attribution rules #22.
- Test repeat imports, versions, provider failures, partial results, and durable decisions with fixtures.
- Complete an isolated live full/incremental/repeat pilot: 402 unique works, identical repeat queue, all 390 prior decisions/targets/receipts preserved, and no approved content changed.
- Verify anonymous provider access within the 100-request budget. No paid plan or secret was needed.
The 19 additional pilot observations (14 canonical items) screened out because Tim was absent; they were not published or copied over the production queue. Coverage is not exhaustive. Production remains at 66 papers and 390 observations / 316 canonical candidates (42 accepted, 264 pending, ten rejected), plus one accepted manual supplement. Both scheduling gates stay off.
Evidence: pilot report, policy.
Deployment verified: run 34545126097, commit 542656f. The live homepage, all nine current members, representative routes/downloads, redirects, and desktop/phone rendering passed.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Activity