Skip to content

feat(player): negotiate LumeEngine's audio width up front and export its diagnostics - #220

Closed
bilipp wants to merge 4 commits into
mainfrom
feature/207-dolby-vision-atmos
Closed

bilipp wants to merge 4 commits into
mainfrom
feature/207-dolby-vision-atmos

Conversation

@bilipp

@bilipp bilipp commented Sep 7, 2026 •

Copy link
Copy Markdown
Owner

Summary

App side of #207, round 2a. It pairs with bilipp/LumeEngine#33, which carries the picture fixes:

  • Dolby Vision profile 5 / 20 / 10.0 no longer render pink/green; the engine now reshapes the IPT base layer to HDR10 on the GPU.
  • CPU-decoded HDR is no longer washed out.

This PR makes sure LumeEngine gets the right audio output width, and that its diagnostics reach the exported report.

Audio: negotiate the width before the lane is built

Before this PR, LumeEngine read AVAudioSession.outputNumberOfChannels when it built its audio lane. That lost an ordering race: LumeEngineEngineView starts the session from .onAppear, but FullScreenPlayerView only activated and widened the route from a later .task. A session opens exactly one URL with no rebuild-in-place, so a width resolved too early stays for the whole stream. The result: a 5.1 track could reach a surround receiver as stereo.

  • New PlaybackAudioRoute owns playback audio-session setup. activateForPlayback() is idempotent and is called from LumeEngineCoordinator.makeConfiguration. Whichever caller runs first negotiates and the other reuses the result, so ordering can no longer clamp the lane to stereo.
  • It activates the session before reading channel counts. An inactive session reports 2 even on a route that can carry more.
  • negotiatedWidth is min(granted, maximum), and nil at or below stereo. It never passes maximumOutputNumberOfChannels through: that is what the route could carry, not what was granted, and over-shooting fails AVSampleBufferAudioRenderer outright (silence, wedged pipeline).
  • macOS asks CoreAudio for the default output device's width, instead of the engine's flat 2-channel fallback.
  • MultiViewScreen keeps a deliberately stereo session via activateStereo(). Only one tile is audible, so a surround layout for a muted grid would be wrong.

Diagnostics

  • The exported diagnostic report now includes engine.lume/diagnostics, so the engine's once-per-stream readouts survive into a report a user sends:

    • stream-open (DV profile, object audio)
    • video-format
    • dv-reshape
    • audio-width

    The raw engine.lume/ffmpeg bridge stays excluded, because some streams log to it per frame.

Docs

  • AGENTS.md: Dolby word marks must never appear in user-facing UI, String Catalogs or store copy; use technical labels (HDR10, DV P8.1, E-AC-3 JOC, …). It also records the host-stated audio-width contract.

Refs #207. This PR does not close it: the technical-label UI badges (round 2b) are still to come.

What was measured

  • DV P8.1 on Apple TV 4K: already correct on the VideoToolbox path (all three colour keys present, picture clean).
  • DV P20 (DVprofile20.mp4): tinted on every path, Apple's own decoder included. Fixed in LumeEngine#33 and verified on macOS; still needs a run on Apple TV / iPhone.
  • Audio width on hardware: not validated yet. The Apple TV run used AirPlay (2 ch), where stereo is correct. An HDMI run should show widthSource=configured with route=HDMIOutput(…).

Type of change

  • feat — new feature

Platforms verified

  • iOS / iPadOS
  • macOS
  • tvOS
  • visionOS

visionOS is not verified. No simulator runtime is installed, and main does not build for it independently of this PR: PictureInPictureBridge is gated #if os(iOS) || os(macOS) || os(tvOS) in LumeEngine, while LumeEngineCoordinator references it ungated.

Screenshots / recordings

None; no UI changes.

Checklist

  • Issue linked (feat: Dolby Vision and Dolby Atmos support in the internal engines #207).
  • Conventional Commits.
  • Pre-commit hooks pass (SwiftFormat, SwiftLint --strict, String Catalog normalization).
  • LumeTests green (iPhone 17 Pro / iOS 26.4, -parallel-testing-enabled NO). LumeUITests not run.
  • Builds for the iOS and tvOS Simulators against the current LumeEngine#33 branch.
  • Localization: n/a, no user-facing text.
  • Tests: PlaybackAudioRouteTests (stereo route ⇒ nil; granted 6 / maximum 8 ⇒ 6, never 8; ungranted preference ⇒ nil; clamping) and DebugLogExporterTests.
  • No credentials, stream URLs or other content.

LumeEngine decodes audio to PCM and downmixes to a single output width.
With nothing stated by the host it read AVAudioSession.outputNumberOfChannels
at audio-lane-build time, which lost an ordering race: LumeEngineEngineView
starts the session from .onAppear while FullScreenPlayerView only activated
and widened the route from its own later .task. A session opens exactly one
URL with no rebuild-in-place, so a width resolved too early is the width for
the whole stream — a 5.1 track could reach a surround receiver as stereo.

- New PlaybackAudioRoute owns playback audio-session setup. Its
  activateForPlayback() is idempotent and is called from
  LumeEngineCoordinator.makeConfiguration, so whichever of the two callers
  runs first negotiates and the other reuses the result. Ordering can no
  longer produce a stereo-clamped lane.

- It activates the session *before* reading channel counts. An inactive
  session reports the pre-activation default of 2 even on a route that can
  carry more, so the old order asked for a widening that never happened.

- negotiatedWidth is min(granted, maximum) and nil at or below stereo. It
  never passes maximumOutputNumberOfChannels through: that is capability,
  not the granted width, and over-shooting fails the renderer outright
  (silence, wedged pipeline) rather than merely downmixing.

- MultiViewScreen keeps its deliberately non-widening session via
  activateStereo() — only one tile is audible, so negotiating a surround
  layout for a muted grid would be wrong.

- The exported diagnostic report now includes engine.lume/diagnostics, so
  the once-per-stream colour and audio-width readouts survive into a report
  a user sends. The raw engine.lume/ffmpeg bridge stays excluded: it logs at
  AV_LOG_WARNING and some streams hit it per frame.

- AGENTS.md records that Dolby word marks must never appear in user-facing
  UI or store copy; technical labels only.

Refs #207
…ion-atmos

# Conflicts:
#	Lume/Views/Player/FullScreenPlayerView.swift
#	Lume/Views/Player/LumeEngineCoordinator.swift
@bilipp bilipp changed the title feat(player): state the negotiated audio output width to LumeEngine feat(player): negotiate LumeEngine's audio width up front and export its diagnostics Sep 25, 2026
…layer

DV profiles 5, 20 and 10.0 start fine on KSPlayer, VLCKit and AVPlayer but
render pink/green, so the failure-driven fallback never fires. KSPlayer now
reports the stream's DV record at readyToPlay; when the base layer is IPT the
host swaps that stream to LumeEngine (the only engine that applies the RPU),
resuming where it was. Always on, no setting. AirPlay keeps precedence, and if
LumeEngine fails the stream returns to the user's engine and is not rerouted.
@bilipp bilipp closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant