Hello,
Is there a way to use a stream of audio (such as an internet radio / a soundfile-loop stream on a server) as the source of a PositionalAudio instance?
Thnx,
J
You can stream an audio if you create a native HTML5 Audio element and then set it via THREE.Audio.setMediaElementSource(). This approach is used in the following official example:
https://threejs.org/examples/webaudio_visualizer
Doing this is recommended for stuff like background music since itās not necessary to download the whole audio buffer at once via AudioLoader.
Would this approach work for you?
1 Like
This looks good yes!
I will try to see if I can find a streaming audio example and replace the audio-file used here with the stream.
This is probably not the correct forum to ask this but, I am looking for a way to loop and stream an audio-file on a server and pipe it to all clients in real-time (so everybody visiting the website would hear the same playback). Then I would use that as a source for thee.js.
Do you think this is possible? Is there a good node-js forum to ask for this? I asked Quora and Stackoverflow, but didnāt get a straight response, Ideally I would love to find a forum like this one with a lot of activity but for node-js 
if my question is irrelevant, I understand, and it can be ignored 
Thank you for the answer!
If you donāt get a response at stackoverflow, there could be also an issue with your question. So maybe you have to write it differently in order to get more feedback. In general, stackoverflow should be a good space for such topics.
If you can get it work with the pure Web Audio API, we will probably find a solution with three.js, too. Iām not sure this is helpful but AudioContext.createMediaStreamSource() could be interesting for you. MediaStreamAudioSourceNode is not yet supported by three.js. But it could be part of your solution if HTML5 Audio is no option for some reasons.
Thnx! I will look into it.
FYI: With the next release R109, THREE.Audio provides a new method .setMediaStreamSource()for including Web RTC media streams into the Web Audio pipeline. Could be useful to you
2 Likes
Hi @Mugen87 !
Here I am again 2 years later! My project is now live and can be viewed here on this website:
It is working great on desktop, but on mobile it seems to not really workā¦ The page refreshes at a certain point by itself. My intuition tells me this must be a RAM overload issue.
On the website I am presenting a contemporary electronic music audio piece with 14 10-minute-long mono sound files played back on 14 different Positional Audio instances inside three.js.
I have two questions.
-
I am using the setMediaElementSource method. Does this mean that the audio is streamed? the buffer is loaded in chunks as opposed to loading the entire soundfile? This is of course what I want to accomplish. I can imagine that the RAM would get overflown pretty fast if it needs to load that many audiofiles on page-load.
-
in the example you gave https://threejs.org/examples/webaudio_visualizer I see in the code that it is checking the user-agent for mobile and uses the AudioLoader if on mobile. why is this the case?
Thank you for your help
That is correct. The purpose of this method is to avoid downloading the entire audio buffer at once.
Ricardo added this code as an iOS workaround. However, I donāt why it was added and if the latest iOS/Webkit still requires it.
Ok,
Do you have any idea why my website doe snot work on mobile?
Since I am using the streaming method for positional audio does it matter how long the audio files are? or how many positonal audio instances there are in the scene? Are there any rules or limitations concerning this?
Thank you
Does the website work if you add no sound at all?
That should not be important.
Iām not aware of any defined limitations (e.g. max number of panner nodes).
Yes it works perfectly without the sound.
I am using SimpleFPControls to move the camera around ( GitHub - jsantell/three-simple-fp-controls: Simple, boring, first person controls for three.js ). When on desktop I pass a positionalAudio source the camera sometimes hangs and in the next frame I am suddenly positioned somewhere else, as if something goes wrong with the controls. This is especially noticeable on slower computers and I think this must be the reason it does not work at all on mobile.
Once I āpassedā all the PositionalAudio objects with my camera, everything seems āloadedā and it goes smoothly (on my macbook pro 2020)
I am also using DeviceOrientationControls to use the gyroscope on mobile, so maybe this could also be a cause for the issue
I am using these classes to play the sound
export default class Sound {
constructor(url, durationCallback) {
this.duration = 0
this.mediaElement = new Audio(url);
this.mediaElement.loop = true;
this.mediaElement.preload = 'auto';
this.mediaElement.crossOrigin = 'anonymous';
var self = this
this.mediaElement.addEventListener("loadedmetadata", function(_event) {
var dur = self.mediaElement.duration;
self.duration = dur
durationCallback(dur)
});
}
play() {
this.mediaElement.play()
}
setTime(time) {
let newTime = time / 1000
if (!newTime.isNan && newTime != undefined && isFinite(newTime)) {
this.mediaElement.currentTime = newTime
}
}
}
export default class Speaker {
constructor(scene, listener, soundURL, imageURL, position, durationCallback) {
this.object = new THREE.Object3D()
this.object.position.set( position.x, position.y, position.z )
scene.add(this.object)
this.spriteDimensions = 40
this.sound = null
this.setupSoundSource(soundURL, durationCallback, listener)
this.setupSpeakerSprite(imageURL)
this.setupSpeakerAvatarIcon(imageURL)
this.object.layers.set(1)
}
setupSoundSource(soundURL, durationCallback, listener) {
this.sound = new Sound(soundURL, durationCallback)
var soundSource = new THREE.PositionalAudio(listener)
soundSource.setRefDistance(5)
soundSource.setRolloffFactor(1) //0.2
soundSource.setMediaElementSource(this.sound.mediaElement)
this.object.add(soundSource)
}
setupSpeakerSprite(imageURL) {
this.sprite = new Sprite(imageURL,
{
x: 0,
y: 0,
z: 0
},
{ x: 6,
y: 6,
z: 6
})
let color = new THREE.Color(1, 1, 1)
this.sprite.setColor(color)
this.object.add(this.sprite.mesh)
}
scale(value) {
let scaled = this.spriteDimensions * value
this.speakerAvatar.scale.set( scaled, scaled, scaled )
}
play() {
this.sound.play()
}
setTime(time) {
this.sound.setTime(time)
}
getTime() {
this.sound.currentTime()
}
dispose() {
this.speakerAvatar.geometry.dispose()
this.speakerAvatar.material.dispose()
this.object.remove(this.speakerAvatar)
this.sprite.geometry.dispose()
this.sprite.material.dispose()
this.object.remove(this.sprite)
}
}
Hi, so I joined this forum because of this exact same issue. I cannot get the positionalAudio() or Audio() based sound to work if I use setmediaStreamSource() with an HTML5 element
I am using the positionalAudio() and Audio() for some positional and ambient sound respectively. I could give you the code example but my implementation is the same as @uvfauna (if not as elegant). if I use the AudioLoader() to load the sound they both positionalAudio() and Audio() work fine but that would cause memory issues that I would rather avoid. Any advice?
Hello @mudebbela.
For me I could fix the crashing of my website by reducing the amount of speakers.
But yeah. The audio still crashes after a whileā¦ like a crackling sound that eventually completely silences all audio output.
Unfortunately it is simply a known bug on iOS and there seems nothing we can do about it (Iāve spent enough time researching this).
The only thing we can do is pressure the webkit team to fix this goddamn bug!
Ah yes, I found my people. Ive been trying everything to get positional audio to work on mobile to no avail. Thanks for the link.
we are in this together my friend