Skip to content

Implement publication discovery, normalization, and deduplication #27

Closed

Description

@advaitmb

Discovery readiness is verified in PR #35; merged and deployed. Activation and real weekly operation remain in #28. Updated September 10, 2026.

  • Implement pagination, incremental lookback/reconciliation, caching, bounded retries/request budget, normalization, duplicate grouping, and separate candidate storage.
  • Preserve editorial corrections, decisions, alternate observations, rejections, and deferrals; never publish or delete automatically.
  • Configure anchored discovery identities for all nine current members, including Rongwu; retain mixed/split-profile warnings.
  • Complete inclusive year-level membership screening under Verify the lab roster and publication attribution rules #22.
  • Test repeat imports, versions, provider failures, partial results, and durable decisions with fixtures.
  • Complete an isolated live full/incremental/repeat pilot: 402 unique works, identical repeat queue, all 390 prior decisions/targets/receipts preserved, and no approved content changed.
  • Verify anonymous provider access within the 100-request budget. No paid plan or secret was needed.

The 19 additional pilot observations (14 canonical items) screened out because Tim was absent; they were not published or copied over the production queue. Coverage is not exhaustive. Production remains at 66 papers and 390 observations / 316 canonical candidates (42 accepted, 264 pending, ten rejected), plus one accepted manual supplement. Both scheduling gates stay off.

Evidence: pilot report, policy.

Deployment verified: run 34545126097, commit 542656f. The live homepage, all nine current members, representative routes/downloads, redirects, and desktop/phone rendering passed.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions