Repository navigation
boot: use embedded Lua cache in place, faster registry restore - #892
Conversation
…ay dependency declarations
|
Fixed the Windows CI failure in bd22713. The failing TestMergeLuaCacheMovesVerifiedEntriesAndReplacesCorruptDerivedData used os.Stat before rename; Windows resolves its identity lazily from the now-removed source path in os.SameFile. The test now captures identity via File.Stat on an open handle, closes that handle before the move, and keeps the identity/source-removal/content assertions. No production behavior changed. Linux race tests pass for cmd/app, Lua code/cache/lint, registry, and touched boot/supervisor packages; relevant go vet and formatting/diff checks pass. Fresh full CI is running, including the native Windows execution of this test. Rodrigo's review requested. No merge or release. |
|
Final head bd22713 is green: all 13 GitHub checks pass, including the full native Windows test job and the previously failing cache-rename identity test. Rodrigo approved this exact head. Local race/vet/lint checks passed; the rename test also passed 30 repetitions and registry race suites 5 repetitions. A temporary, unpublished integration tree combining runtime #887/#889/#890/#891/#892 merged without manual conflict resolution and passed the combined targeted race suites, make test-application, and make lint (0 issues). That was a local compatibility check only: no GitHub PR has been merged and no runtime or module has been released. |
First launch after a binary change extracts, verifies and installs every embedded Lua cache entry, while registry transitions copy the entire working map after each accepted operation. This reads the binary's authenticated Lua cache from memory and updates only transaction-owned registry state.
Changes:
TestOverlayValidationRefreshesAfterDependencyPatternRegistrationloads a durable entry referencinglive:admissionthroughdata.late, registers that path later, then proves the forbidden durable-to-overlay dependency rejects admission immediately.ApplyOperationstill returns isolated map snapshots, and other builders retain the existing fallback. Tests cover rejected operations, kind changes, branch isolation, listener refusal, compensating effects and rollback.Exported Go boundaries are intentional:
app.BootLoggerand itsBootLogger() *zap.Loggermethod let standalone native hosts supply their structured diagnostic logger before runtime logging/event wiring exists.(*topology.DepIndex).VisitDependencieslets the registry validate candidate overlays against committed declarations across the package boundary without re-decoding unchanged entries.(*topology.StateBuilder).ApplyOperationToPrivateStatelets the runner opt into mutation of its unshared transaction map across the package boundary; callers must retain separate rollback snapshots and publish only committed state.code.Config.EmbeddedCache,cache.Reader, andcache.NewLayeredStoreconnect the authenticated read-only seed to the existing Lua manager and writable cache.lua.cache.embeddedaccepts a typed native-host override, not YAML authorization.Measurements (before = origin/main
728b75942, after =0cac2f848; Go 1.27.0, Linux/amd64, Ryzen 7950X3D, GOMAXPROCS=32; medians of three runs, three iterations per run):BenchmarkLuaSeedDirectory)BenchmarkLuaSeed906)BenchmarkBootTransition1000)BenchmarkPrivateBootStateOperations, snapshots → private)The first three comparisons use identical benchmark inputs on baseline/final worktrees. The last isolates old snapshot semantics and the new private-map path in one binary. Benchmarks use
go test -run '^$' -bench NAME -benchtime=3x -count=3; real-cache adoption also setsWIPPY_LUA_SEED_BENCH_DIRto the verified snapshot root.The real Bee snapshot contains 1,657 verified compile/typecheck cache entries in a 5,852,111-byte compressed archive; one corrupt retained metadata entry was excluded before making the identical benchmark input. Archive construction and state cleanup are excluded from adoption timing. The synthetic archive has 906 entries with 1 KiB proto payloads. The real transaction-runner benchmark restores 1,000 entries without component listeners; history IO and Lua compilation are excluded. The host was busy, so timings are diagnostics rather than an end-to-end Bee startup claim. Immutable bytes stay resident for the runtime invocation, bounded by the existing archive-size limit.
Validation passed:
make lint(0 issues).make test-application(fts5/sqlite_vec/treesitter/sqlite_preupdate_hook).go testandgo vetoncmd/app,runtime/lua/code,runtime/lua/code/cache,boot/components/runtime/lua,system/registry/...,system/supervisor, andboot/components/system.go test -race ./system/registry/... ./cmd/app, plus race on the Lua cache/manager and touched boot/supervisor packages.The four local boot-runtime commits are preserved on a fresh
perf/boot-startupbranch, followed by the direct embedded-cache change and verification/benchmarks. No merge requested.