[raederan]

Summary

Adds osw.tools.user_sync: a maintenance tool that creates and updates OSW User items from an OSL instance's MediaWiki accounts, enriching ORCID users from the public ORCID API. Idempotent, with an interactive preview and confirm before any write.

• Enumerates MediaWiki accounts; excludes bots, privileged groups, reserved system accounts and, by default, blocked accounts (--include-blocked keeps them).
• Core identity is always written (username, name, orcid, email); website and organization are opt-in (--with-websites / --with-organizations); employment_contract_status is never written.
• Finds existing items by username across all User items (paging through the results with the offset the wiki reports), and updates an item whose deterministic page already exists instead of creating it again.
• Placeholder names (MediaWiki users without name data) are written only for new items and never proposed for an existing item.
• --prune removes employment_contract_status from the stored jsondata of existing items, and overwrites website or organization with an empty list when their opt-in flag is not given, in a second edit of the page's jsondata after the store. The removal of employment_contract_status is decided from the stored jsondata, so a second --prune run proposes no removal for an item whose key is already gone.
• --dry-run, --auto-apply, --orcid-only / --mw-only.
• Creates User:<name> redirects; warns (never fails) on users with no ORCID email.
• Unit tests with mocked MediaWiki and ORCID, examples/user_sync.py, and a docs page listed in the existing "Tools" nav group next to CLI and MCP.

Changes after review

• Merged stage into the branch. With oold 2.3.0, entity.__iris__ returns a new dict on each read, so the earlier in-place edits had no effect; relations are now set by attribute assignment, and --prune clears stored values through the jsondata edit.
• A User built or loaded without employment_contract_status serializes the model default; the sync now unsets it before every store, so a normal run never writes it.
• docs/tools/index.md now says the Tools section also covers standalone scripts; the second "Tools" nav group is removed.
• New tests for the store, redirect, verify and prune paths, including one that runs the overwrite merge from src/osw/core.py on a built User.
• A failed store is no longer counted as created, and redirects are written only for stored or pre-existing items (user sync: a failed store is reported as created and redirects are still written #224).
• --prune clears a key by its field type in the User model: a list or set field gets [], a single-valued field (employment_contract_status) is deleted, also when it holds an empty value such as []. Before, every key got [].

Limitations

• MediaWiki's API exposes no real name or email for other users, so MediaWiki-native (non-ORCID) users get username-based placeholder names and no email.
• After --prune, a loaded User shows the model's default employment_contract_status again, because the model re-applies its default when the key is missing; the sync never writes it back.
• The prune edit sends no base revision, so an edit by someone else between the store and the prune edit can be lost.
• Unlinking organizations leaves the Organization items orphaned (not deleted).
• Live test on a test wiki: the --prune dry run proposed removing websites, organizations and employment_contract_status from a test user, the applied run reported updated=1 pruned=1 failed=0, and a second dry run showed the user as unchanged. The wiki's edit form was not opened on the pruned item.

[github-actions]

Release preview

Merging this PR would release v2.10.0 (current: v2.9.1).

Changelog preview (truncated)

## v2.10.0 (2026-11-12)

### Bug Fixes

- Follow properties and items when collecting category templates
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`55e500e`](https://github.com/OpenSemanticLab/osw-python/commit/55e500efb896f0a3d6bde98409b3a6f43a652bdc))

- Keep the stored jsonschema when store_entity regenerates it
  ([`9b76a8f`](https://github.com/OpenSemanticLab/osw-python/commit/9b76a8fbeb813b34c4881b54d329e9e837b08f95))

- Select category templates from the schema allOf like the wiki
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`9f606f4`](https://github.com/OpenSemanticLab/osw-python/commit/9f606f48fefd640cca48e6718f1a92eca118da6b))

- **user-sync**: Align with stage and the tools docs
  ([`bc6b0d6`](https://github.com/OpenSemanticLab/osw-python/commit/bc6b0d642f5697ca8048b4be05fd26c9ad2162f6))

- **user-sync**: Count and redirect only stored items
  ([#224](https://github.com/OpenSemanticLab/osw-python/pull/224),
  [`0b8dc97`](https://github.com/OpenSemanticLab/osw-python/commit/0b8dc974e972e8adaa7a2bb03bd3ae4396589f40))

- **user-sync**: Prune a single-valued key stored with an empty value
  ([`ebda514`](https://github.com/OpenSemanticLab/osw-python/commit/ebda514c731718ab20a084b93288491b3b944627))

- **user-sync**: Prune by field type and decide from stored jsondata
  ([`9b36e4e`](https://github.com/OpenSemanticLab/osw-python/commit/9b36e4ef8417b8847ba304535a95c0eedaf628b9))

### Continuous Integration

- Run Main on pushes to stage and Integration on PRs into stage
  ([`1b55f02`](https://github.com/OpenSemanticLab/osw-python/commit/1b55f02c90cae646d690f83d6208c01f0e26e62b))

### Documentation

- State which schema references the template walk follows
  ([#220](https://github.com/OpenSemanticLab/osw-python/pull/220),
  [`799bc07`](https://github.com/OpenSemanticLab/osw-python/commit/799bc07952aac7e6ab579bc127c5cb3000710491))

### Testing

- Drop the FileController type check, it is no longer a model
  ([`1eddbe8`](https://github.com/OpenSemanticLab/osw-python/commit/1eddbe867559e14806ad72eb076172cd41c7767e))

- Set subclass_of for oold 2 in the jsonschema tests
  ([`bd4ffdb`](https://github.com/OpenSemanticLab/osw-python/commit/bd4ffdb3e950f43f4eea0151d19d07f621eb14e0))

_{Preview via python-semantic-release and conventional commits.}

[simontaurus]

user properties may be available via https://www.mediawiki.org/wiki/API:Userinfo

[raederan]

user properties may be available via https://www.mediawiki.org/wiki/API:Userinfo

only works for logged in users, not even admins can read real name or email
name mapping can only work if username is the realname on MW side created accounts for user items.
email must be provided by each user themselves

[LukasGold]

Fix for #224 pushed as 0b8dc97 (fix(user-sync): count and redirect only stored items).

Changes in src/osw/tools/user_sync/sync.py:

• _store_users counts a title as created or updated only after store_entity stored it. Before, titles were counted while the entities were built.
• On OSW.StoreEntityPartialError, only the titles in stored count, and each failed title is recorded as report.failed["store:<title>"]. Same for _store_organizations. Any other exception still sets report.failed["store"] and counts nothing.
• _create_redirects skips a create whose item was not stored. update and skip keep their redirect, because resolve_plan assigns them only to users with an existing item.
• New links to an organization that failed to store are dropped before the user is stored. A link the existing item already has is kept. A re-run proposes the dropped links again; if the item has other organizations, the re-run shows a conflict instead of a gap fill.
• _prune_users skips updates that were not stored.

Tests: new tests/tools/test_user_sync_store_failures.py covers a partial store, a failed store, redirects after a failed create, a failed and a partially failed organization store, a kept existing organization link, prune after a partial store, and an accepted successful sync.

Not handled: an existing item that is deleted between the load and the store still gets its redirect. This was already the case before the change.

[LukasGold]

Note for anyone with a local checkout of user-item-upd: the branch moved from e97daa4 to 0b8dc97 (the fix for #224, see #174 (comment)). Run git pull before you commit or push more work on this branch, so the next push does not fail as a non-fast-forward.

[LukasGold]

Live test of --prune on a test wiki (stacktest), run from a local merge of stage into this branch.

Result: website and organization are cleared. employment_contract_status is stored as [], but the item still reads as Active, both in osw-python and in SMW.

--prune writes values that do not match the schema

clear_jsondata_keys() in src/osw/tools/user_sync/prune.py sets every pruned key to [], independent of its schema type. The User schema (Category:OSWd9aa0bca9b0040d8af6f5c091bf9eec7, defined in world.opensemantic.base) defines the field as a single string:

"employment_contract_status": {
  "type": "string",
  "format": "autocomplete",
  "range": "Category:OSW781ad17c7eef4161ade5a7b690aca6b5",
  "default": "Item:OSW62770459451644f3841c502df8a5cb1d"
}

The jsondata written by the prune must be valid against the schema of the category. [] is valid for website and organization (arrays), but not for employment_contract_status.

The default applies when the value is empty

An empty value does not clear the status. Both osw-python and the wiki replace it with the default Item:OSW62770459451644f3841c502df8a5cb1d (Active).

osw-python:

• load_entity() runs remove_empty() on the stored jsondata (src/osw/core.py, LoadEntityParam.remove_empty defaults to True). remove_empty() (src/osw/utils/wiki.py) deletes keys whose value is "", [], {} or an empty set.
• The model is then built without the key, and the packaged model fills it with its default_factory, which returns Active (opensemantic/base/_model.py, field employment_contract_status).
• store_entity() serializes with exclude_none=True and also runs remove_empty(). It keeps an empty value only when the caller set it explicitly (_explicitly_set_empty()). So None, "" and [] all become an absent key, and an absent key loads as Active.
• Live result: after the prune, the raw jsondata slot holds [], and load_entity() returns Item:OSW62770459451644f3841c502df8a5cb1d for the same page. A second --prune --dry-run therefore proposes -employment_contract_status again.

Wiki (SMW):

• The User header template sets the property with a wikitext parameter default (slot_header_template.wikitext:36):
  {{#set: HasEmploymentContractStatus={{{employment_contract_status|Item:OSW62770459451644f3841c502df8a5cb1d}}} }}
• Before it renders the template, MwJson removes every table value from the jsondata, so [] is removed and the parameter is undefined (MwJson.slot_main.lua:513-516).
• The template default then sets Active. A missing key gives the same result.
• This is a template parameter default, not the JSON Schema default, but the value is the same.
• Live result: after the prune, an ask query on the page returns HasEmploymentContractStatus = Item:OSW62770459451644f3841c502df8a5cb1d, while HasWebsite and HasOrganization are empty.
• Limit: I read the template and the Lua module in the page packages. I could not compare them with the versions installed on stacktest, because the wiki was unreachable afterwards. The live result matches the code.

Consequence

With the current schema and templates, an empty or absent employment_contract_status means Active, in osw-python and in the wiki. --prune cannot express "not employed" by clearing the field.

A schema-valid alternative is to write one of the other values of the range, Item:OSWacf5f4db53264d29ae8744d03796be0b (Inactive) or Item:OSWa09571ab688e4395aa1a845234e3093b (Unknown). Which value is correct for a pruned user is a design decision for this PR.

[LukasGold]

@raederan please view the changes that I made based on AI assisted review

[raederan]

@LukasGold is fine for me

[LukasGold]

In-depth review with code changes performed on this PR, good to merge