Connect Audio from Three.JS to MediaRecorder

Question “forwarded” from here: https://stackoverflow.com/questions/57358937/how-to-connect-audio-to-mediastream-from-three-js-audiocontext

Any idea how to connect audio to a MediaRecorder canvas stream from Three.JS? Please let me know if I can provider any more code samples other than the ones in the the question. Help is greatly appreciated.

I’m afraid your use case is not clear to me. Do you want to include the audio of a media stream into the three.js audio graph? There is currently no way to do this but you can try to enhance the THREE.Audio prototype like so:

THREE.Audio.prototype.setMediaStreamSource = function ( mediaStream ) {

    this.hasPlaybackControl = false;
    this.sourceType = 'mediaStreamNode';
    this.source = this.context.createMediaStreamSource( mediaStream );
    this.connect();

    return this;

};

@Mugen87 Other way around actually. I am looking to include the audio from Three.JS into the mediarecorder. Does that make sense?

Are you talking about this API? https://developer.mozilla.org/en-US/docs/Web/API/MediaRecorder

That’s the one. The video works well, but the audio is lacking entirely. Other SO answers haven’t really helped unfortunately.

For developers in the FuTuRe:

const context: AudioContext = ThreeAudioContext.getContext();
const destination = context.createMediaStreamDestination();
this.audioListener.getInput().connect(destination);
this.backgroundGainNode.connect(destination);
stream.addTrack(destination.stream.getAudioTracks()[0]);

In short, to connect audio to the MediaRecorder, call createMediaStreamDestination and connect the gainNode (volume node) to the newly created destination. Then, add the track to the stream.

Couple of hiccups I had:

  • All gain nodes you connect need to be under the same audio context.
  • If you want to connect audio to the stream, even though it may not be audible, you need to create an independent GainNode.

Of course, I say GainNode with the understanding you might be using a different type of AudioNode, but I assume 95% of people just want audio to play without any alteration besides volume.

3 Likes