Skip to content

Add Audio V2 support to Inspector v2#18480

Merged
ryantrem merged 16 commits into
BabylonJS:masterfrom
ryantrem:inspector-v2/audio-v2
May 22, 2026
Merged

Add Audio V2 support to Inspector v2#18480
ryantrem merged 16 commits into
BabylonJS:masterfrom
ryantrem:inspector-v2/audio-v2

Conversation

@ryantrem
Copy link
Copy Markdown
Member

@ryantrem ryantrem commented May 19, 2026

🤖 This PR was created by the create-pr skill.

Adds Audio V2 support to Inspector v2 (scene explorer + property panes + scene gizmo), plus the underlying engine observables and spatial-audio APIs the inspector needs.

Highlights

Core – AudioV2

  • New OnAudioEngineV2CreatedObservable and AudioEngineV2.Instances so consumers can track engine lifetime without polling.
  • AudioEngineV2 now implements IDisposable and exposes:
    • onNodeAddedObservable / onNodeRemovedObservable for top-level audio nodes (sounds, sound sources, buses).
    • onDisposeObservable that fires after the engine is removed from Instances.
  • Spatial-audio APIs now expose the currently attached scene node:
    • AbstractSpatialAudio.attachedNode
    • _SpatialAudio.attachedNode (internal)
    • _SpatialAudioSubNode.attachedNode (internal)
    • _SpatialAudioAttacherComponent.sceneNode (internal)

Core – Gizmos

  • New SpatialAudioGizmo (and ISpatialAudioGizmo interface) that visualizes the position/orientation of a v2 spatial sound source on the utility layer and fires an onClickedObservable when clicked.

Inspector v2

  • New Audio V2 scene explorer service that renders engines, main buses, audio buses, sounds, and sound sources as a hierarchical tree driven by the new engine observables, with per-node visibility/mute commands.
  • New Audio V2 property pane with editors for engine state (pause/resume/unlock, master volume, listener), buses, sounds (play/pause/stop, volume, loop, pitch, etc.) and sound sources, including a clickable "Output Bus" link.
  • New spatial audio property section wired into the gizmo service so spatial sound sources can be visualized and selected from the viewport.
  • GizmoService gained getSpatialAudioGizmo(soundSource, scene) (ref-counted, mirroring camera/light gizmo lifecycle).
  • InterceptProperty fix: getters/setters now forward this correctly, so intercepted prototype properties behave the same as the originals when read/written from subclasses.
  • Default sections metadata: added the audio section ordering for the scene explorer.
  • Test app updated to consume core/pure and to spin up a small spatial v2 sound demo so the new inspector surfaces are exercised end-to-end.

Notes

  • No public breaking changes — only additions (and one internal getter on _SpatialAudioAttacherComponent).
  • scripts/treeshaking/side-effects-manifest.json updated to reflect the new gizmo entry point.
image image image

ryantrem and others added 8 commits May 18, 2026 14:25
@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

Please make sure to label your PR with "bug", "new feature" or "breaking change" label(s).
To prevent this PR from going to the changelog marked it with the "skip changelog" label.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

Snapshot stored with reference name:
refs/pull/18480/merge

Test environment:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18480/merge/index.html

To test a playground add it to the URL, for example:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/refs/pull/18480/merge/index.html#WGZLGJ#4600

Links to test your changes to core in the published versions of the Babylon tools (does not contain changes you made to the tools themselves):

https://playground.babylonjs.com/?snapshot=refs/pull/18480/merge
https://sandbox.babylonjs.com/?snapshot=refs/pull/18480/merge
https://gui.babylonjs.com/?snapshot=refs/pull/18480/merge
https://nme.babylonjs.com/?snapshot=refs/pull/18480/merge

To test the snapshot in the playground with a playground ID add it after the snapshot query string:

https://playground.babylonjs.com/?snapshot=refs/pull/18480/merge#BCU1XR#0

If you made changes to the sandbox or playground in this PR, additional comments will be generated soon containing links to the dev versions of those tools.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

Building or testing the sandbox has failed.

If the tests failed, results can be found here:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18480/merge/testResults/

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

You have made possible changes to the playground.
You can test the snapshot here:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/

The snapshot playground with the CDN snapshot (only when available):

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/?snapshot=refs/pull/18480/merge

Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 19, 2026

🟢 Memory Leak Test Results

13 passed, 0 leaked out of 13 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (13)
Scenario Package
Core Feature Stack @babylonjs/core
Core Rendering Materials Shadows Stack @babylonjs/core
Core Textures Render Targets PostProcess Stack @babylonjs/core
GUI Fullscreen UI Controls @babylonjs/gui
GUI Mesh ADT Controls @babylonjs/gui
Loaders Boombox Import @babylonjs/loaders
Loaders OBJ Direct Load @babylonjs/loaders
Loaders STL Direct Load @babylonjs/loaders
Materials Library Stack @babylonjs/materials
Serializers glTF Export @babylonjs/serializers
Serializers GLB Export @babylonjs/serializers
PostProcesses Digital Rain Stack @babylonjs/post-processes
Procedural Textures Stack @babylonjs/procedural-textures

- Defer OnAudioEngineV2CreatedObservable notification to CreateAudioEngineAsync
  so observers don't see a partially constructed subclass (fixes a critical
  initialization-ordering hazard for _WebAudioEngine).
- Expose useBoundingBox + attachmentType down the spatial-audio API chain and
  honor them in SpatialAudioGizmo so the gizmo pose matches Position-only,
  Rotation-only, and bounding-box attachments.
- Surface routable nodes with outBus === null (e.g. microphone sources, manually
  disconnected nodes) directly under the engine in the Audio V2 scene explorer
  so they don't disappear from the tree.
- Wire SpatialAudioGizmo.onClickedObservable to ISelectionService so clicking
  the gizmo in the viewport selects the underlying sound source.
- useSoundState now also listens to AbstractSound.onEndedObservable and
  intercepts the engine's _onSoundPlaybackStateChanged so natural end / async
  Starting -> Started / FailedToStart transitions all refresh the UI.
- Add Listener section to the Audio V2 engine property pane with a link to the
  scene node the listener is attached to. (Skip a dedicated Unlock button —
  AudioEngineV2.unlockAsync is a thin wrapper around resumeAsync so Resume
  already covers the autoplay-blocked path.)
- Refactor getSpatialAudioGizmo to route shared cleanup through a closure-
  captured lambda instead of overriding gizmo.dispose, avoiding Function.bind
  / Function.call (both prohibited by the repo's performance/style rules).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@ryantrem ryantrem marked this pull request as ready for review May 19, 2026 21:37
Copilot AI review requested due to automatic review settings May 19, 2026 21:37
@ryantrem
Copy link
Copy Markdown
Member Author

Self code review completed by the code-review skill. Review fixes: c9491cce06

  • CriticalOnAudioEngineV2CreatedObservable no longer fires from the base AudioEngineV2 constructor (it would have notified observers before subclass state like _audioContext/listener was initialized). It now fires from CreateAudioEngineAsync after the engine is fully constructed and initialized, with a base-constructor comment documenting why.
  • Critical — Spatial-audio gizmo onClickedObservable is now wired into ISelectionService, so clicking the gizmo in the viewport selects the underlying sound source (mirroring camera/light gizmo selection behavior).
  • Routable nodes with outBus === null (e.g. microphone sources, manually disconnected nodes) now appear directly under the engine in the Audio V2 scene explorer instead of disappearing.
  • SpatialAudioGizmo now honors useBoundingBox, SpatialAudioAttachmentType.Position, and SpatialAudioAttachmentType.Rotation, so the gizmo pose matches Position-only / Rotation-only / bounding-box attachments. This required exposing useBoundingBox + attachmentType getters down the spatial-audio API chain.
  • useSoundState now also subscribes to AbstractSound.onEndedObservable and intercepts AudioEngineV2._onSoundPlaybackStateChanged, so async transitions (Starting → Started, FailedToStart, natural end) all refresh the UI.
  • Added a Listener section in the engine property pane with a link to the scene node the listener is attached to. (Verified that engine.unlockAsync() is implemented as return this.resumeAsync(), so a dedicated Unlock button would be redundant with Resume — skipped.)
  • Refactored getSpatialAudioGizmo to route shared cleanup through a closure-captured lambda rather than overriding gizmo.dispose. No more Function.bind / Function.call (both prohibited by the repo's performance/style rules).

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

🟢 Memory Leak Test Results

13 passed, 0 leaked out of 13 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (13)
Scenario Package
Core Feature Stack @babylonjs/core
Core Rendering Materials Shadows Stack @babylonjs/core
Core Textures Render Targets PostProcess Stack @babylonjs/core
GUI Fullscreen UI Controls @babylonjs/gui
GUI Mesh ADT Controls @babylonjs/gui
Loaders Boombox Import @babylonjs/loaders
Loaders OBJ Direct Load @babylonjs/loaders
Loaders STL Direct Load @babylonjs/loaders
Materials Library Stack @babylonjs/materials
Serializers glTF Export @babylonjs/serializers
Serializers GLB Export @babylonjs/serializers
PostProcesses Digital Rain Stack @babylonjs/post-processes
Procedural Textures Stack @babylonjs/procedural-textures

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

You have made possible changes to the playground.
You can test the snapshot here:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/

The snapshot playground with the CDN snapshot (only when available):

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/?snapshot=refs/pull/18480/merge

Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

Building or testing the sandbox has failed.

If the tests failed, results can be found here:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18480/merge/testResults/

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

🟢 Memory Leak Test Results

13 passed, 0 leaked out of 13 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (13)
Scenario Package
Core Feature Stack @babylonjs/core
Core Rendering Materials Shadows Stack @babylonjs/core
Core Textures Render Targets PostProcess Stack @babylonjs/core
GUI Fullscreen UI Controls @babylonjs/gui
GUI Mesh ADT Controls @babylonjs/gui
Loaders Boombox Import @babylonjs/loaders
Loaders OBJ Direct Load @babylonjs/loaders
Loaders STL Direct Load @babylonjs/loaders
Materials Library Stack @babylonjs/materials
Serializers glTF Export @babylonjs/serializers
Serializers GLB Export @babylonjs/serializers
PostProcesses Digital Rain Stack @babylonjs/post-processes
Procedural Textures Stack @babylonjs/procedural-textures

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 20, 2026

Building or testing the sandbox has failed.

If the tests failed, results can be found here:
https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18480/merge/testResults/

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

You have changed file(s) that made possible changes to the sandbox.
You can test the sandbox snapshot here:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/SANDBOX/refs/pull/18480/merge/

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

You have made possible changes to the playground.
You can test the snapshot here:

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/

The snapshot playground with the CDN snapshot (only when available):

https://snapshots-cvgtc2eugrd3cgfd.z01.azurefd.net/PLAYGROUND/refs/pull/18480/merge/?snapshot=refs/pull/18480/merge

Note that neither Babylon scenes nor textures are uploaded to the snapshot directory, so some playgrounds won't work correctly.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

⚡ Performance Test Results

🟢 All performance tests passed — no regressions detected.

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

🟢 Memory Leak Test Results

13 passed, 0 leaked out of 13 scenarios

🟢 All memory leak tests passed — no leaks detected.

Passed Scenarios (13)
Scenario Package
Core Feature Stack @babylonjs/core
Core Rendering Materials Shadows Stack @babylonjs/core
Core Textures Render Targets PostProcess Stack @babylonjs/core
GUI Fullscreen UI Controls @babylonjs/gui
GUI Mesh ADT Controls @babylonjs/gui
Loaders Boombox Import @babylonjs/loaders
Loaders OBJ Direct Load @babylonjs/loaders
Loaders STL Direct Load @babylonjs/loaders
Materials Library Stack @babylonjs/materials
Serializers glTF Export @babylonjs/serializers
Serializers GLB Export @babylonjs/serializers
PostProcesses Digital Rain Stack @babylonjs/post-processes
Procedural Textures Stack @babylonjs/procedural-textures

@bjsplat
Copy link
Copy Markdown
Collaborator

bjsplat commented May 21, 2026

@ryantrem ryantrem requested a review from RaananW May 21, 2026 22:25
@ryantrem ryantrem merged commit 0454863 into BabylonJS:master May 22, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants