Skip to content

fix(judge): let overlapping saves to one judgement take turns - #268

Merged
harenome merged 1 commit into
ThinkFlowLab:mainfrom
harenome:fix/judge-save-race
Nov 20, 2026
Merged

harenome merged 1 commit into
ThinkFlowLab:mainfrom
harenome:fix/judge-save-race

Conversation

@harenome

Copy link
Copy Markdown
Collaborator

Closes #247

What

store.save read a trial's judgement file, merged the new answers into it, and wrote it back through <file>.tmp. Two saves in that window lost one answer, and one of them raised FileNotFoundError when the other renamed the shared temporary file away. Two batches over the same trials at once (Judge all clicked again, or in a second tab) could do this.

  • A save holds a lock per judgement file for its read, merge and write. Saves to different files still run in parallel.
  • Each save writes through its own temporary file (tempfile.mkstemp in the same folder) and removes it if the write fails.
  • The hygiene readings merge inside the same lock. Before, batch.ask_and_store loaded the record and merged them before it called save, outside any lock. save returns the record as stored, and ask_and_store returns the same record as before.

Touches

backend (nullius/judge/store.py, nullius/judge/batch.py) · tests

Verified

  • New app/tests/test_judge_store_race.py. Two saves read the file before either writes, and both answers must be kept. A second test checks that saves to two trials do not wait for each other. On the base, the first test fails: (["FileNotFoundError(2, 'No such file or directory')"], ['q2']) != ([], ['q1', 'q2']).
  • A stress script with 8 threads x 30 rounds, each thread saving its own question to one trial at the same moment: 0 raised saves, 0 lost answers, 0 files that JSON cannot read, and the earlier version kept every time.
  • In a rootless podman python:3.12 container with Node: python3 app/tests/parallel.py: 3111 tests in 160 modules, OK. pip install -r app/requirements.txt, then python3 app/tests/parallel.py app/ui_tests: 704 tests in 58 modules, OK.

Risk

  • The lock is per process. Two Nullius processes on one workspace can still overlap, as before.
  • A judgement file is now written with owner-only permissions (mkstemp's 0600) where it was 0644 under the usual umask. Workspaces live in the user data directory.
  • Existing stores read as they are. Nothing on disk changes shape.

store.save read the trial's file, merged the new answers and wrote it back
through a shared `<file>.tmp`. Two saves in that window lost one answer,
and one of them raised FileNotFoundError when the other renamed the shared
temporary file away.

A save now holds a lock per judgement file for its read, merge and write,
and writes through a temporary file of its own. The hygiene readings merge
inside the same lock instead of in a separate load in ask_and_store.
Saves to different files still run in parallel.
@harenome
harenome merged commit 6f8975b into ThinkFlowLab:main Nov 20, 2026
2 checks passed
@harenome
harenome deleted the fix/judge-save-race branch November 20, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Judge: two overlapping saves to one trial's judgement lose an answer

1 participant