Skip to content

Keep metadata responsive during OverlayFS copy-up - #20

Merged
reiase merged 2 commits into
mainfrom
feature/light_meta
Oct 10, 2026
Merged

reiase merged 2 commits into
mainfrom
feature/light_meta

Conversation

@reiase

@reiase reiase commented Oct 10, 2026

Copy link
Copy Markdown
Collaborator

Large copy-up operations currently hold host OverlayFS state while copying bytes, so unrelated metadata requests wait for the whole copy. Prepare regular-file content outside the state lock, then revalidate and publish through the existing ordered handlers. A bounded mutation worker lets short metadata callbacks proceed while preserving hardlinks, held descriptors, cache invalidation and shutdown draining. VM filesystem dispatch also offloads lone large reads/directory work and bounds admission by the negotiated virtqueue size.

Add contract/regression tests and a reproducible macOS engineering A/B harness, report and CSV under B-FS-ENG. Pin the pre-change baseline in reproduction commands; preserve original frozen measurement evidence locally rather than rewriting receipts after formatting. Raw artifacts remain ignored.

The retained 30-pair-per-condition, 1 GiB copy cohort found:

  • First stat median: 802.86 → 0.68 ms without a journal; 790.32 → 1.65 ms with compact strict journaling.
  • The 64-path metadata batch during copying improved by 97.49% / 95.81%.
  • Writable-open median changes were not detected, but tail latency remains uncertain.
  • Compact strict-journal idle metadata regressed by 5.92% (+1.90 ms per 64-path batch); the report retains this regression and all slow samples.

Validation:

  • just fmt and complete just lint passed, using CI's actionlint 1.7.7 from a temporary directory.
  • just test pvisor-overlay-core pvisor-overlayfs pvisor-vm -- --no-fail-fast: 431 passed, 7 skipped.
  • Benchmark Python tests: 4 passed; its main script also passes Ruff formatting/lint outside the default recipe paths.
  • Independent retained-cohort audit rechecked all 132 launches and 66 journals; raw report and derived CSV SHA-256 values stayed unchanged.
  • Earlier validation passed Linux cross-compilation, affected consumer builds, rustdoc with warnings denied, and a real macOS FSKit copy-up smoke test.

Limits: native Linux FUSE runtime and cold-disk performance were not tested. Baseline hashing may still wait on the journal mutex; large publication/ordinary I/O remain serialized, and draining does not cancel backing I/O.

@reiase
reiase merged commit f109dd2 into main Oct 10, 2026
15 checks passed
@reiase
reiase deleted the feature/light_meta branch October 10, 2026 11:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant