Skip to content

user sync: a failed store is reported as created and redirects are still written #224

Open

Description

@LukasGold

Problem

In the user sync from #174, a failed store_entity call is reported as a success, and User: redirects are still written for the items that were not stored.

  1. _store_users adds each title to report.created or report.updated while it builds the entities, before store_entity runs:
    for resolved in resolution.resolved:
    proposed = resolved.change.proposed
    try:
    if resolved.action == "create":
    entities.append(build_user(proposed))
    report.created.append(proposed.full_page_title)
    elif resolved.action == "update":
    entities.append(
    apply_update(
    resolved.change.existing, proposed, resolved.apply_fields
    )
    )
    report.updated.append(proposed.full_page_title)
  2. When store_entity raises, the exception is caught and recorded only as report.failed["store"]. The titles stay in created and updated:
    if entities:
    try:
    osw.store_entity(
    OSW.StoreEntityParam(
    entities=entities,
    overwrite=True,
    remove_empty=remove_empty,
    edit_comment="user-sync",
    )
    )
    except Exception as exc: # pragma: no cover - network failure path
    report.failed["store"] = str(exc)
  3. run_user_sync then calls _create_redirects for every resolved user, whether its item was stored or not:
    _store_users(osw, resolution, report, remove_empty=not config.prune)
    if config.create_redirects:
    _create_redirects(osw, resolution, report)

Result: report.summary() counts items as created or updated that are not on the wiki, and User:<username> redirects point to pages that may not exist. Only _verify can detect the missing items afterwards.

The same applies to organizations: report.organizations is set only on success, but users that link to an organization are still stored when the organization store failed:

def _store_organizations(osw: Any, org_map, report: SyncReport) -> None:
if not org_map:
return
entities = [build_organization(org) for org in org_map.values()]
try:
osw.store_entity(
OSW.StoreEntityParam(entities=entities, edit_comment="user-sync: orgs")
)
report.organizations = [org.full_page_title for org in org_map.values()]
except Exception as exc: # pragma: no cover - network failure path
report.failed["organizations"] = str(exc)

Expected

  • A title is counted as created or updated only after store_entity stored it.
  • store_entity raises OSW.StoreEntityPartialError on a partial failure. Its stored attribute lists the stored titles and its failed attribute maps each failed title to its exception, so the sync can count the stored titles and record each failed title in report.failed.
  • Redirects are written only for users whose item was stored or already existed.

Tests

The only test in tests/tools/test_user_sync_sync.py that runs the sync is the dry-run test. No test runs an accepted sync, a failed store, the redirects after a failed store, or a sync with prune=True.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions