Read the JSON sidecar for videos generated before InvokeAI 7

[lstein]

Follow-up to #400. That PR read the generation record InvokeAI 7 embeds inside an MP4. Releases before 7 kept it in a JSON sidecar under outputs/videos/sidecars/ instead, mirroring the video's own subfolder — and InvokeAI 7 still writes one when its embedding remux fails, so this is a live fallback and not only a legacy path.

extract_video_metadata now reads the MP4's keyed metadata first and the sidecar second, which is InvokeAI's own order and means a v7 video costs no sidecar lookup at all.

Finding the sidecar

PhotoMapAI indexes absolute paths and never learns where an outputs directory begins, so the videos root has to be guessed: each ancestor of the video is tried as the root, nearest first, mirroring the video's relative path under sidecars/. A candidate must be JSON, hold an invokeai_metadata key, and contain something that passes looks_like_invoke_metadata — the same test the drawer routes on — so a sidecars directory belonging to another tool cannot be mistaken for InvokeAI's.

The walk stops at one ancestor. That is measured, not guessed: 661 of 661 sidecars in a real install resolve at depth 1 (the general / intermediate category), and depth 0 covers a videos root with no subfolder. Going deeper bought nothing and reached outside the album the user configured — at depth 3, a path at the filesystem root.

Measured against a real install

668 videos, 662 sidecars, rather than only against the synthetic fixture:

• 132 sidecars carry a record; 530 carry a null one — a workflow but no parameters. "Exists but has nothing for us" is the ordinary case, and the docs now say so, because it sets the expectation that only about a fifth of older videos will ever show a panel.
• 3 videos in that checkout turned out to have the record embedded; I chased that discrepancy specifically to rule out the ancestor walk producing false positives.
• 0.2 ms per video, including the MP4 walk.
• All 132 recovered records parse, are recognised as video generations, and render with no undeclared fields — which also validates Read and display InvokeAI 7's video metadata #400's v5 video profile against real pre-7 data.

The review pass

The second commit is six defects an adversarial fresh-context review found, each reproduced before fixing. Two are worth a reviewer's attention:

RecursionError escaped the reader and deleted videos from the index. json.loads raises it on deeply nested JSON; it is a RuntimeError, so neither except OSError nor except ValueError caught it, and ~120 KB of brackets is three orders of magnitude under the size cap. The damage was not a lost record: _load_video catches it, returns None, and the file is recorded as bad and left out of the album — exactly what the extractor's docstring promises cannot happen.

The key gate was theatre. Removing it entirely left all 25 tests passing, and it accepted {"invokeai_metadata": {"anything": "at all"}} as a generation record. Hence the looks_like_invoke_metadata requirement above, and a test that actually fails without it.

The rest: the depth cut above (its old test imported the constant it was checking, so it passed at 1 and at 9), symlink resolution, a size cap now matching mp4_metadata.MAX_TAG_BYTES, and a log line naming the wrong stage.

I mutation-tested the fixes rather than trusting green tests, which caught one of my own: reverting .resolve() left everything passing, because cutting the depth had already removed the .. case the test claimed to pin. It is now a symlink test that genuinely fails without the fix. Reverting each of the other five guards fails a test.

Docs

The third commit corrects guidance shipped in #400, which said an update "only re-reads files whose modification time has changed" and so implied Update Index might pick this up. It never will — _get_new_and_missing_images is a set difference on paths, and modification_times is never consulted to decide re-processing. An existing album needs a full rebuild, verified on a real album: 363 indexed videos, 4 carrying a record, 89 recovered only after rebuilding. Now that #402 has landed, the note points at its Rebuild Index button rather than telling people to delete the index file by hand.

Testing

1126 backend + 892 frontend tests pass; ruff, eslint and prettier clean. Verified in the running app against a real pre-7 collection.

🤖 Generated with Claude Code