Skip to content

Demo Auditor: move every demo into one folder, recycling the extra copies (#872) - #874

Open
ccoventry wants to merge 2 commits into
fix/auditor-table-stylefrom
feat/consolidate-demos-872
Open

ccoventry wants to merge 2 commits into
fix/auditor-table-stylefrom
feat/consolidate-demos-872

Conversation

@ccoventry

Copy link
Copy Markdown
Owner

Release note: New: The Demo Auditor's Duplicates tab can move every demo it found into one folder, recycling the extra copies on the way, with Undo.

Closes #872

Stacked on #871 (fix/auditor-table-style). CI doesn't run on a PR into a feature branch (ci.yml only runs for PRs into dev/main); it runs once #871 merges and GitHub retargets this to dev. Local results are below.

What it does

A Consolidate… button beside the duplicates results opens a small panel:

  • Destination from the native folder picker.
  • Layout: all in one folder, or keep each demo's subfolders below the audited folder it was found in.
  • Also move demos in game folders (off by default): demos inside an install's dod folder (the audit's InUse::GameFolder) stay where they are, and so do copies there.
  • Preview line before anything moves, e.g. "Move 1,240 demos (180 GB) into F:\Demos, recycle 3,400 copies, rename 12." It's planned in Rust (consolidate_plan), and the run plans again from the same request, so the preview is never trusted as the plan.
  • Move Demos asks first, then moves each group's kept copy (the Keep picks, per row or Keep copies already in a folder…) and every demo with no copies, and sends the other copies to the Recycle Bin (native::sys::recycle_bin). Different demos with one name get _2, _3. A move never replaces a file.
  • Between drives: copy then delete, with byte progress (throttled ~33 ms), Cancel between chunks (the partial copy is removed), a free-space check before it starts, the modified time kept, and a size check before the original goes.
  • Undo Last Consolidate: each batch logs its moves under demo_consolidate_logs (like demo_rename_logs), saved every 25 moves so a crash part-way still leaves an Undo.
  • Queue: queued demos in the open project follow their files (project_demos.js relocateMovedDemos, the same relocateDemo the Locate flow uses, take index too), on the run and on Undo.
  • Analyzer cache: it's keyed on the demo's path, not its content hash, so each move carries the demo's entry with it (move_analyzer_cache_entry, as Rename Demos does).

Backend: native/src/demo_consolidate.rs (plan, run, undo) and studio/src-tauri/src/consolidate_manager.rs (async commands with spawn_blocking, one job at a time). The audit result now also lists the demos with no copies (singles) and the folders it walked (roots). Frontend: audit_consolidate.js (pure) and audit_consolidate_ui.js (createAuditConsolidate factory), with strings in strings.js.

Decisions for the user

Where the issue left a choice open, I picked the simpler, safer option:

  1. Demos already inside the destination stay where they are, in both layouts (flat doesn't pull them up out of its subfolders). They keep their names, and moved demos number around them.
  2. Keep subfolders is relative to the deepest audited folder the demo was found in. With All drives that's the drive root, so C:\Users\x\Demos\a.dem goes to <dest>\Users\x\Demos\a.dem. A demo under no audited folder goes straight into the destination.
  3. Game folders, unticked: a group's kept copy in a dod folder stays put, but its copies elsewhere are still recycled. Copies inside a dod folder are never recycled unless ticked.
  4. Undo puts moves back only. Recycled copies stay in the Recycle Bin (the confirm says so). Empty source folders aren't deleted, so Undo always has somewhere to put demos back. A cancelled Undo keeps the moves it didn't reach in the log, so the next Undo finishes them; a move that fails to go back is reported and dropped from the log.
  5. No free-space check on Undo (it puts files back where they came from).
  6. Other project files: no automatic update. The confirm warns that they'll ask you to locate a moved demo; the existing file-key search finds it when the destination is near the project or its scan folders.
  7. After a run, the results list clears and says to run the audit again, since the paths it lists have moved.
  8. No headroom on the free-space check: the copies only need to fit in what's free.

Tests

  • Rust: 13 tests in demo_consolidate covering the plan (clash names _2/_3 against each other and against files on disk, flat vs kept subfolders under the deepest root, demos already there, which files move or get recycled, the game-folder rule both ways, free space for cross-drive bytes only, no destination), plus a real move, log and Undo, a copy that keeps mtime and refuses to overwrite, a cancelled copy leaving nothing, a cancelled Undo keeping its log. One more in audit_manager for singles_of.
  • Vitest: audit_consolidate.test.js (request building, preview and no-space text, progress lines, singles after a delete).
  • Playwright: duplicates.spec.js adds three tests: preview, confirm, progress, queue update and Undo; Cancel; no space.

Local results (Windows; not run in the game or the app):

  • cargo test --workspace --locked (debug off): all pass
  • rustup run 1.98.1 cargo fmt --all --check: clean
  • rustup run 1.98.1 cargo clippy --workspace --all-targets --locked -- -D warnings: clean
  • npm run test:unit: 595 passed
  • Playwright duplicates, demo-rename, split-maps: 28 passed

Not yet tried on real drives: a 100+ GB move between drives, or the Recycle Bin step on real copies.

🤖 Generated with Claude Code

The audit now also returns the demos without copies and the folders it
walked. Consolidate plans in native::demo_consolidate (clash names with
_2/_3, flat or kept subfolders, game-folder demos stay unless ticked,
free space for copies between drives), then moves each group's kept copy
and every single, recycles the other copies, logs the moves for Undo,
and moves each demo's analyzer cache entry with it. Moves between drives
copy with byte progress and Cancel, keep the modified time, check the
size, then delete the original. The open project's queue follows moved
demos (project_demos.js relocateMovedDemos).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoventry

Copy link
Copy Markdown
Owner Author

Needs the latest fix/auditor-table-style merged in: #871 gained several commits today (one folder list, Split Maps rework, keep-by-folder rename, footer per tab), so this branch now conflicts with its base. Left for after 0.2.0; resolve when picking it up.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ccoventry

Copy link
Copy Markdown
Owner Author

Caught up with #871: merged fix/auditor-table-style (aa372ed) in, head now c9f99200.

The only conflict was auditor_pane.js's imports (Consolidate's two modules beside #871's auditor_footer.js); everything else merged on its own. #871's single folder list, Keep radios and "Keep copies already in a folder…" were already in this branch's base, so Consolidate reads the same Keep picks as before, and the Duplicates footer now goes through setAuditorFooter. The 8 decisions are unchanged.

Local results (stacked PR, so no CI until #871 merges):

  • cargo test -p native --locked: 749 passed, 10 ignored. One run had hlae_ffmpeg::a_file_that_is_not_a_program_is_not_ffmpeg fail on its scratch file write; it passed alone and in a full rerun. That module isn't touched here.
  • cargo test -p desktop-studio --locked: 159 passed
  • rustup run 1.98.1 cargo fmt --all --check: clean
  • rustup run 1.98.1 cargo clippy -p native -p desktop-studio --all-targets --locked -- -D warnings: clean
  • npm run test:unit: 595 passed
  • Playwright duplicates, split-maps, demo-rename: 33 passed

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consolidate all demos into one folder from the Demo Auditor

1 participant