Hi,
I’m having trouble playing videos inside a WebXR session on Apple Vision OS simulator. For details please see here: three.js - Video playback (HTML5 VideoElement) Apple Vision Pro Simulator in VR mode not playing - Stack Overflow
any help is highly appreciated.
Probably related to: Video texture no longer updating after entering WebXR mode
Experiencing the same issue here. Did you get a response from Apple or someone else?
I can make a 3D environment in the ThreeJS scene, configure xr controllers, and, when I enter VR mode on the Apple Vision Pro Simulator, I can still navigate around the scene. To add more details: with a video player in the scene, playback works fine while in the 2D window mode; but as soon as I enter VR mode the window freezes on the last rendered frame (it does not go all dark). If I rotate or navigate around, in VR mode, the rendered image is still frozen, but if I exit VR mode, the navigation applied while in VR has been executed and rendering (and playback) starts working again.
Yep, you are describing what I’m experiencing as well. Unfortunately I didn’t find any workaround or solution for the issue so far.
Hi. This is still happening for me, even on device. Is there any workaround to solve this?
I don’t have a VR to program. However, it is reasonable for these devices to detect a scene using webGL. Pause other tabs and Media elements that are not part of the 3D scene. the solution would be to receive the video streaming and convert it to the texture outside the html media element
send me one device i make a workarround 
I’ve discovered this problem, video immediately goes to paused when launching to XR. But when I exit, I get no video playback. In Vision 1.0. In Vision 1.1 there is no webgl rendering at all.
Test video here
I figured out a work around and fried my brain doing so of course as Vision 1.1 simulator has a broken web inspector. force play upon xr session. visionOS detected as Ipad or Macintel
onSessionStart() {
//force play for visionOS
if (this.isIpad) {
// this.video.play();
const handlePaused = () => {
this.video.removeEventListener("pause", handlePaused);
this.video.play();
};
this.video.addEventListener("pause", handlePaused);
}
}
I tried this but am still seeing the video pausing, even though I can see that console.log(video.paused) is false after calling this.video.play(). Do you have a full example you could share? Did you play() the video element when creating it? Thanks for any help!
You just do it in the callback of the sessionstart event. I might make a bug ticket about this one. I already have one regards to web inspect being broken and webxr flags not being enabled. These is even a new bug where it refuses to clear cache, the simulator shouldnt be caching at all.
This “transient-pointer” inputsource updates targetRaySpace like the grip controller so can be raycasted like the grip controller. But each pinch is a connect, selectstart, selectend, disconnect event. So hard to replicate hovering and toggling on objects.
I was forced to build a websocket proxy logger due to this bug to keep working on things. https://downloads-electroteque-org.s3.amazonaws.com/avision-logger.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIA3B5OOLKVZKQG4NXQ/20240401/us-east-1/s3/aws4_request&X-Amz-Date=20240401T001523Z&X-Amz-Expires=604796&X-Amz-SignedHeaders=host&X-Amz-Signature=59fb1392a2c20f42efe92e428961b0a5e89005c0615886aeea02ebf9e2ee9c8d
Hmm, still not working for me. I put together a repro using the three.js webxr_vr_video example (ie. three.js examples) here: threejs-webxr-vision-pro-video/index.html at 74df9a372423385c07f169a73321d0faa2db6f9d · jparismorgan/threejs-webxr-vision-pro-video · GitHub -
@danrossi would be very curious what I’m not doing that you are. Thank you for any help!
Update: I see there is also an issue on GitHub about this, will move the discussion there: Video playback stops when entering immersive mode in Vision Pro · Issue #5469 · aframevr/aframe · GitHub
I run all my tests under https to be able to launch webxr from my server on another machine. So try that. I’m trying to get my VR picture in picture hack working for Safari then will release my visionOS changes including transient-pointer controls support.
I’ve not seen any video pause issue but might report it. I think its a webkit bug.
http-server -p 8084 -S -C /etc/letsencrypt/live/rtc.electroteque.org/cert.pem -K /etc/letsencrypt/live/rtc.electroteque.org/privkey.pem ./demo
I’ve been working on visionOS non stop for the past 3 weeks. They cost me a week already with fixing all the work arounds and another solid day trying to differentiate visionOS from an Ipad.
Ive been testing XR without problems and now its decided to pause the video again since an OS update requiring more work arounds. It’s literally a feature not a bug lol. It should be reported to webkit though. They do these things intentionally like not enabling webxr by default.
I do however have transient-pointer inputsource controls working now, it took a week or so tinkering with it. so I have a VR controlbar to toggle play. one pinch to toggle to show the controlbar and another pinch for selections. it basically works the same as the grip controller raycaster and select events but you dont add grip controls and you do selections and raycasting on selectend events and on move events. releasing pinch disconnects the controls.
Incredibly unproductive and insane working with anything Apple, it takes up too much time to support implementing hacks and work arounds. The visionOS showing up as Ipad has to be next level next to Iphone still refusing to support html fullscreen requiring a hack to show webgl canvas as native fullscreen. I figured out a way to determine between the two as visionOS shouldnt be given mobile device orientation controls.
It required checking for visionOS instead of iOS as I had to change feature detection to not give visionOS orientation controls. So it looks like this now. I can get video force playing back.
onSessionStart(ev, event) {
//force playback for visionOS
if (this.isVisionOS) {
const handlePaused = () => {
this.video.removeEventListener("pause", handlePaused);
this.video.play();
};
this.video.addEventListener("pause", handlePaused);
}
}
feature detection found here
visionOS 1.1 has a webinspect bug I reported. Completely broken and sent live broken. Cant clear caches requiring to rename files to get the updates. It made it very hard to do stuff. To get console logging to even see what is going on with the video requires this websocket proxy script I built and include into the page.
https://downloads-electroteque-org.s3.amazonaws.com/avision-logger.zip
UPDATE:
I’ve finally set it live to show video playback working going to XR. the transient-pointer controls should work also by pinching to toggle the control bar for further playback control. It took me weeks to get to this point.
my pip feature and the iphone fullscreen hack is now working again on Safari. As long as visionOS detection doesnt break using the code I figured out above it should keep working. They may do something intentionally to try and break that as that is what they do, sabotage.
absolutely soul destroying trying to work with anything Apple, and said features dont always keep working. Requires extensive testing on all device options now as each work different. Iphone has its own fullscreen hack function to show canvas as native fullscreen, visionOS cant be detected as Ipad and mobile device to turn off orientation controls when going fullscreen, Ipad supports html fullscreen and pip needs testing there etc.
Hi.
Is there any further advice for this?. We have tried the approach mentioned above and this is still happening. We are forcing the video to play on user command, but getting permissions issue or simply the video doesn’t play.
try my demo page and you can see it working, including transient-pointer input source controller support. Clearly its a bug but havent had time to report it to webkit. Perhaps report it ?
Thank you for your answer.
In our case, we don’t need to do the detection as we are only deploying to vision OS. I am able to force the video to play. However, the texture is still not presenting the video. I’m also trying to force ‘needsupdate’ in the render loop without results. Do you have any suggestion for that?