Skip to content

feat(vr): add VR support - #27

Draft
dim-ghub wants to merge 109 commits into
mainfrom
vr
Draft

dim-ghub wants to merge 109 commits into
mainfrom
vr

Conversation

@dim-ghub

Copy link
Copy Markdown
Collaborator

Adds VR support.

komaruworld and others added 30 commits September 12, 2026 21:45
Co-authored-by: HttpAnimations <shawndouglasyt@gmail.com>
)

* vr: capture the microphone for voice chat on Build 2998

On the vr branch Roblox 2.736.1408 joins voice chat and plays other
players, but never opens a microphone: its FmodAudioDevice capture
methods have no working host path, and main's native capture (537828a
"Fix vc") is pinned to 2.738 RVAs.

Port main's NativeInputCapture and interpose the FmodAudioDevice capture
slots for Build 2998. 2.738 inserted two device-list slots (7 and 11),
so the capture methods sit at {4, 11..15} here instead of {4, 13..17}.
The input-device slots {8, 9, 10, 16}, checked against main's input RVAs
for this image, anchor that layout before anything is patched, and each
capture slot must be a distinct method next to the input-device code.

The voice sink is called through its own vtable after checking that the
vtable and its PCM entry point lie in libroblox, and the retained
shared_ptr is released through the libc++ control-block ABI, so no
further Build 2998 RVAs are needed. If any check fails, capture stays
off and the output routes still work.

MOCKTAIL_NATIVE_INPUT_CAPTURE=0 turns the capture interposition off, and
audio.input_device: disabled reports no microphone to Roblox.

* Requested fix at PR 181
Also fix openxr shit
SteamVR (vrclient.so) binds its Vulkan renderer to the lifetime of the
XrInstance. When Roblox destroys the LuaApp Vulkan device and creates a
second VkInstance/VkDevice for UGC experiences, reusing the original
XrInstance causes SteamVR to dereference freed device handles in xrEndFrame.

Tear down and recreate the XrInstance whenever a new guest Vulkan instance
is initialized, resetting session state, spaces, and controller actions.
# Conflicts:
#	stubs/libandroid_stub.cc
…ark mode

vr.frame_rate_limit / MOCKTAIL_VR_FRAME_RATE_LIMIT replaces
graphics.frame_rate_limit while VR is active and takes the same values.
It defaults to unlimited so xrWaitFrame paces Roblox at the headset
refresh rate instead of Roblox's own 60 FPS cap.

MOCKTAIL_VR_RENDER_SCALE scales the runtime's recommended eye extent.
SteamVR's recommendation for a Quest 2 at 90 Hz (2244x2352 per eye) missed
the 11.1 ms budget and halved the rate to 45 FPS.

MOCKTAIL_VR_UNPACED=1 stops guest frames from blocking in xrWaitFrame:
the open XR frame spans several guest frames and is submitted once per
display period, for measuring maximum render rate.
When Roblox asks for a keyboard (showKeyboard), a world-locked panel
opens below eye level as an OpenXR quad layer. Controllers point at keys
and the trigger types; text and key presses go through the same platform
events as the desktop keyboard, so the focused TextBox edits identically.

The layout is US QWERTY without the function row or Ctrl/Alt, with one-shot
Shift, Caps Lock, Tab, Backspace auto-repeat, Enter and Close. A trigger
aimed at the panel is withheld from Roblox so a key press is not also a
click. A text field above the keys mirrors the focused TextBox, whose
desktop overlay never reaches the eye images. MOCKTAIL_VR_KEYBOARD=0 turns
it off.
Mark the 3092 profile experimental. It still runs when selected with
--roblox-lib, but it no longer becomes the preferred update target for
every x86_64 user, so the shipped payload metadata (2998) is again the
preferred profile.

Select the 2998 entry by Build ID in the partial-input profile test instead
of relying on manifest order, and drop the shipped 3092 entry before the
test re-identifies 2998 as 3092.

Align the example config's VR build wording with the bootstrap template.
VR now stays off at runtime in every build until --vr or vr.enabled: true,
so a normal launch on a VR-capable build never tries to reach an OpenXR
runtime and no longer exits when none is running.

MOCKTAIL_ENABLE_VR defaults to AUTO: OpenXR support is built when the
third_party/OpenXR-SDK submodule is present and skipped with a notice
otherwise, so source packages that do not fetch submodules (Nix, AUR) keep
building. ON still requires the submodule. mocktail-vr-probe is only built
and installed in VR builds.

CI gains an x86_64 job that builds and tests with MOCKTAIL_ENABLE_VR=OFF.
The config docs and README describe the new defaults.
An explicit --vr on a Roblox build without VR support now exits with the
build's version and Build ID, the builds VR supports, and how to select one
(--roblox-lib). When VR is only enabled in the config, the launch continues
without VR and logs the same details. The bridge's own Install error names
both as well. The verified build contracts carry their version labels and
move to namespace scope so they can be listed.
Queue and device-idle locks in the Vulkan adapter are now taken only after
an armed OpenXR backend calls mocktail_vulkan_enable_queue_sharing(). The
guest synchronizes its own queue use; only the runtime and the eye copies
add a second user. Without VR, and in builds without it, submits and
presents take no locks.

FFlagDebugEnableVREmulator is only written when VR is active, so a user's
own fflags.json value is no longer overridden on every launch.

Config loading is back after the single-instance lock and the support
bundle guard. A non-VR build still rejects --vr before creating any data
or state directories, from the command line alone; a vr.enabled config in
such a build is rejected once the config is loaded.
- Every frame error, including the swapchain-image and copy-fence
  timeouts, now tears the session down recoverably instead of ending XR
  output for the rest of the run.
- Session recovery backs off (1, 2, 4, 8 s) and recreates the XrInstance
  after an instance-loss event, an instance/runtime failure, or three
  failed attempts, so VR returns after SteamVR or WiVRn restarts instead
  of retrying forever.
- While no XR session runs (headset idle, session not READY, recovering)
  presents are paced at 60 Hz outside the backend lock instead of running
  unthrottled.
- A PrepareHostPresent failure that consumed nothing (copy resources
  unavailable, failed fence reset or submit) presents normally instead of
  failing the guest's vkQueuePresentKHR; only device loss is propagated.
- A runtime/guest GPU mismatch falls back to the plain host device and
  keeps the game running without headset output, naming both GPUs.
- A failed EGL attach on the GLES path no longer makes window init fail;
  the game runs without headset output and the bind is not retried.
- Once the guest eye format is known, the eye swapchains are recreated
  once in the format that keeps its bytes: the eye format, else its
  UNORM/SRGB sibling (raw copy), else the other channel order with the same
  sRGB-ness (a blit that only swizzles). Mixed order-and-gamma cases no
  longer go through a gamma-converting blit; if only that remains, it is
  logged.
- The desktop mirror segment is created with O_EXCL after removing our own
  stale segment (a crashed run's leftover), and must be private to this
  user before eye images are written to it.
- A new frame_mutex_ is held for the whole XR frame cycle and by every path
  that can destroy the session (taken before mutex_). That lets the cycle
  release mutex_ while it blocks in xrWaitFrame, so guest render-thread
  hooks no longer wait out the runtime's pacing. Swapchain-image and fence
  waits stay under mutex_; they return promptly unless the GPU or
  compositor stalls, and they run while guest eye bindings are copied.
…reads

- MOCKTAIL_VR_HAND_POSE_SOURCE=grip now rotates hand records from the grip
  pose. base_ori started as the identity quaternion, so the norm check
  never fell back to it and hand rotation was dropped.
- Hand records cancel the engine's stored hand pitch (read from
  hand_pitch_storage_rva by the pointer hook, 20 until read) instead of a
  hard-coded 20, so they agree with the pointer ray.
- Shutdown never rewrites the aim-pointer prologue while guest threads may
  be executing it; the jump stays and only forwards once the bridge is gone,
  and a later activation reuses it.
- Guest hooks count themselves in flight before loading the bridge pointer;
  Shutdown unpublishes the bridge and waits for in-flight calls to drain,
  so no hook keeps a pointer to a destroyed bridge.
- Roblox's original state and eye getters are kept for the process
  lifetime. A hook reached with no bridge, including after a failed vtable
  restore, forwards to them instead of returning an unfilled state buffer.
  Restoring the vtable no longer depends on the pointer hook.
- The activation log names the vtable slots in the right order.
- Remove the MOCKTAIL_VR_TEST_SESSION_LOSS_AFTER fault injection; nothing
  used it.
- The scripted head pose and the eye evidence dumper only work in builds
  configured with -DMOCKTAIL_VR_DIAGNOSTICS=ON (default OFF); release
  builds ignore their environment variables.
- Remove FindFmodInputCaptureProfile and HasExpectedFmodInputCaptureVtable
  (only their own tests called them) and the never-defined
  PrepareCaptureLocked declaration; the audio bridge files match upstream
  again.
- docs/VR.md lists every MOCKTAIL_VR_* variable with its default, linked
  from the README.
dim-ghub and others added 30 commits October 21, 2026 22:57
# Conflicts:
#	include/runtime/roblox_desktop_app_policy.h
#	src/runtime/roblox_desktop_app_policy.cc
#	tests/roblox_desktop_app_policy_test.cc
The origin/main merge (d8f6718) resolved bionic_vulkan_loader_adapter.cc
by taking main's file wholesale, discarding the vr branch's OpenXR hooks,
queue locking, and desktop-mirror barrier ordering.

Re-merge the file keeping both sides. Barriers are normalized for the VR
desktop mirror first and then routed through ETC2 emulation, and the VR
image-view destroy note runs inside the ETC2 vkDestroyImageView wrapper.
Roblox no longer lets these x86_64 builds join games. Saved profiles for
them are refused too. The Python deriver now handles 2.746.
Pins the 2.746.1266 GitHub source.
# Conflicts:
#	config/roblox_compatibility.json
#	src/compat/host_abi_profile.cc
#	tests/build_profile_test.cc
#	tests/host_abi_profile_test.cc
Log Roblox's saved graphics quality at startup.
# Conflicts:
#	tests/fixtures/host_abi_regression/20836.json
* main loop: message pump off by default and paced in unthrottled mode

* assets: open reads nothing until the data is asked for

* texture overrides: off unless the folder has pngs

* etc2: stream uploads out of write-combined staging memory
# Conflicts:
#	CHANGELOG.md
#	src/legacy/legacy_runtime.cc
#	src/ui/mocktail_settings.cc
#	src/window/window_pointer_capture_owner.h
# Conflicts:
#	README.md
#	config/mocktail.example.yaml
#	config/roblox_compatibility.json
#	include/mocktail/audio/roblox_output_device_bridge.h
#	scripts/find_roblox_vr_offsets.py
#	src/audio/roblox_output_device_bridge.cc
#	src/audio/roblox_output_device_bridge_internal.h
#	src/compat/host_abi_profile.cc
#	src/graphics/bionic_vulkan_loader_adapter.cc
#	src/graphics/texture_override.cc
#	src/graphics/vulkan_etc2_emulation.cc
#	src/legacy/legacy_runtime.cc
#	src/libc_shim/vulkan_etc1_sky_transcoder.cc
#	src/runtime/performance_policy.cc
#	src/runtime/runtime_config.cc
#	src/runtime/runtime_config_bootstrap.cc
#	src/runtime/runtime_config_file.cc
#	src/vr/openxr_backend.cc
#	src/vr/roblox_vr_device_bridge.cc
#	src/vr/roblox_vr_device_bridge_internal.h
#	stubs/libandroid_stub.cc
#	tests/build_profile_test.cc
#	tests/vr/roblox_vr_device_bridge_test.cc

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants