PositionalAudio setMediaStreamSource with WebRTC Question (not hearing any sound)

Hi,

I’m having the exact same problem as @cat
Firefox works, Chrome does not.
I tried all possible scenarios I could think of, including

createElement('audio') and
new Audio() and assigning srcObj

But the above example works for me in Chrome. Can’t you just use the same apporach?

The above example works for me, but the same approach does not work with an incoming MediaStream from a WebRTC Transport. I wonder if this has to do with the preload tag used in the example. For the record, the incoming source audio does work with the HTML audio element.

Do you know any free WebRTC streams that can be used in a live example?

Hmm… Let me look into making a minimal setup. My source is available, but definitely not a minimal setup: https://github.com/AidanNelson/YORB2020

Okay, I managed to create a minimal example using SimplePeer. This example creates two peers in the same page (WARNING: wear headphones to avoid feedback!), initiates a WebRTC PeerConnection between them, pipes the incoming MediaStream into an audio object, then creates a positionalAudio object from it (arrow keys moves the camera for positional effect)

https://jsfiddle.net/3ztf5cs6/6/

As above, this works in Firefox (with the audio element volume set to 0), but not in Chrome, as far as I can tell…

Any thoughts?

Before I dive into your code: Can you please verify if this demo works on your computer?

https://jsfiddle.net/sjvxf4rb/3/

It uses a media stream obtained via MediaDevices.getUserMedia().

This demo works for me in latest Chrome and Firefox.

Oh wait, you have already tested this approach earlier^^.

Anyway, if you use the same code and it does not work with your WebRTC stream, than please file an issue here:

https://bugs.chromium.org/p/chromium/issues/list

Looks like a browser issue.

:grimacing: okay

Thanks again for looking into this! It is super helpful to know I’m not making some silly mistake.

I think there may be a simple way to dynamically adjust volume of HTML audio elements to make some move towards positional audio that works cross-browser. Have you heard of such an effort? Would love not to rebuild the wheel, if possible.

ha, apparently we are working on two similar ideas!

We ended up implementing a workaround for chrome that only implements volume (no panning) based on the distance to each participant.
Works well enough.
It’s basically just

let volume = Math.Min(1, 1 / myPosition.distanceTo(participant.position))
1 Like

Good to know! Thanks for the tip. By the way, it appears that the Mozilla Hubs team is dealing with a different set of (potentially related) Chrome audio system issues:

And they have been developing an avatar audio control system, which may be helpful for anyone on this thread:

2 Likes

I just came across this post (and answer) on the babylonjs forum, where they seem to have got it to work:

I’m not familiar enough with babylonjs to know what magic happens here behind the scene, but then maybe it’s also doable for threeJS ?

I figured out what both me and @cat were doing wrong here…
instead of
this.positionalAudio.setMediaElementSource(this.audioElement);
we need to use
this.positionalAudio.setMediaStreamSource(this.audioElement.srcObject);
to make it work in chrome…

Did that really work @lukasIO? Added on the .srcObject made no difference for me - still no positional audio.