In general raycasting against a skinned mesh is expensive – you need to inspect each triangle. I’m not sure if the BVH acceleration helps with that here or not. The best way to make that faster is to simplify the mesh, gltf-transform can do that. If you’re not happy with the appearance of the simplified mesh, you can raycast against a simpler mesh and display a more complex one.
Thank you for the answers, I am confused because other models that are complex work well, only this frog is creating these issues.
BVH acceleration helps a ton, on the other models that I tried with BVH acceleration I have stable 60 FPS without it the frame rate drops at 16, this is why I was wondering if if there is something wrong with the model and BVH acceleration is not applied.
I will look into gltf-transform.
Thank you.
Can the raycasting move to a web worker, will this help?
The demo appears to be raycasting over 40,000+ triangles every frame, and that’s too much work. I would recommend reducing the number of triangles or reducing the number of raycasts, preferably both. Not really the fault of the model. In general BVH acceleration is a very good thing, but when the model is animated then you have to update many BVH cells every frame too, perhaps it can’t help as much in this case.
While you could dedicate a whole Web Worker to constantly scanning these triangles, it would be much better to reduce the amount of work your code is doing. 
Unfortunately, this is the only way the project works, basically, I have some hotspots and they have to hide if the raycaster is intersecting the model before the marker (3d point on the model) and show otherwise, I will try the web worker as well, please check out this example Start if you have any suggestions how I could improve this please let me know.
The markers will be programmed when clicked to start some model animations and other functions but I am stuck at this part showing/hiding the markers it is driving me crazy if the model is not low poly or it has too many animations the framerate is too low… and the more markers I add the performance worsen because more raycaster intersection are called.
raycast html annotations are a waste of resources. and it’s not necessary. annotation can also be made without expense and without raycast by using blending, which punches a hole through the canvas and picks up the html underneath. this allows html to be part of the scene, it can be in front and behind geometry. see
here’s a simplified example. two meshes, a gray canvas. the white plane is a hole on the canvas, the white is from the dom’s body background color. if an html object is now brought into the exact position of the hole-shape you have scene-layered html.
other than that i would raycast against a simplified hull of the model, but with animations it will just look glitchy i suppose.
I need the markers to literally hide when the model is in front of the marker, your solution will not work for my project the client will not accept this… I have to make ti work somehow to hide and show the markers with precision based on the model…
I also have to add HTML content in the markers, video, formatted CSS text, images etc…
A faster alternative to what you’re doing now would be to raycast just once for each marker, storing the normal vector of the surface at the intersection. Then show the marker when that normal vector is within some range (maybe 75–80º) of facing the viewer.
There are some cases this doesn’t handle (suppose the marker is behind a wall, but still facing the camera) but it’s dramatically faster and good enough for most situations when orbiting an object.
You can see the effect in model-viewer:
1 Like
In general raycasting against a skinned mesh is expensive – you need to inspect each triangle. I’m not sure if the BVH acceleration helps with that here or not.
Yeah mesh BVH does not support skinned geometry, bone transformations, or morph targets. You have to “bake” the geometry into a static mesh and refit the bvh to before raycasting when the geometry has changed but that, of course, can also be slow when done too frequently. There’s an example showing that using StaticGeometryGenerator here. In this case you could only refit the bvh and raycast when the mesh is not animating to avoid frequently performing the slow operations.
It would be possible to make a custom raycast function for SkinnedMesh that automatically rebakes the geometry and refits the bvh as-needed but I leave that up to a PR for someone else to implement if they’re interested.
Reading the GPU depth buffer or occlusion queries may also be better solutions for animated geometry in this case.
I need the markers to literally hide when the model is in front of the marker, your solution will not work for my project the client will not accept this… I have to make ti work somehow to hide and show the markers with precision based on the model…
You should communicate the performance limitations to your client and find solutions that are achievable if you’re having issues making the tooltips work exactly as requested.
1 Like
My 15-year old test-PC (core2 duo 4gbRam + gtx1650) runs at solid 60fps no problem while ray casting that frog, so the problem is not ray casting per se (video captured at 30fps to avoid affecting the frame rate):
1 Like
The issue appears when more intersetcsObject are called, with one working fine but adding more is starting to slug…
I need a check for every marker, the more markers I add the worse the performance .
any chance you can share a minimal live editable version of your code setup to let people debug how you’re going about raycasting? you really only need one raycaster.intersectsObject definition in a labels length loop… something like this…
labels.forEach(label=>{
raycaster.set(camera, (dirTo)label);
intersects = raycaster.intersectObject(label);
intersects ? showLabel(label) : hideLabel(label)
})
const showLabel(label){
if (label.isShown) return;
label.isShown = true
label.visible - true
}
const hideLabel(label){
if (!label.isShown) return;
label.isShown = false
label.visible = false
}
That is exactly how I do it but in a request animation frame since the position of the model changes when it is rotated/zoom, here is the method
hideAndShowMarkers(){
this.markersAR.forEach((marker, index) =>{
let marker3DPosition = marker.marker3dPosition.clone();
// Set the raycaster and check the intersection.
this.markersRaycaster.setFromCamera(marker.markerScreenPosition, this.activeCamera);
const intersects = this.markersRaycaster.intersectObject(this.model);
if(intersects.length === 0){
if(this.allowToShowMarkers){
marker.show(true);
}
}else{
const intersectionDistance = intersects[0].distance + 0.0000001;
const markerDistance = marker3DPosition.distanceTo(this.activeCamera.position);
if(intersectionDistance >= markerDistance){
if(this.allowToShowMarkers){
marker.show(true);
}
}else{
marker.hide(true);
}
}
});
}
Here is the project https://webdesign-flash.ro/Easy%203D%20Model%20Viewer.zip it is a vite project the function can be found in the FWDEMVModelManager.js
I tried the Worker no luck…
I think your solution is a good one but I have no idea how to implement it, can I see the code of that viewer?
Try to optimize everything.
- Intersect only the markers, (not the high-poly objects)
- intersect only those markers that face towards the user.
- make one flat low-poly surface for each marker, parent it to the detailed marker, hide it from the user,and use that for raycasting.
- with these alone, you’ll end up with a fraction of the polygons.
If your app still lags, learn the techniques that allow for optimized ray-traciing. - as a last resort, get rid of the HTML elements and replace them with an all-3D approach for max performance.
I have checked their examples with the hotspots and it is exactly what I want, they also made it so that the hotspot is moving with the animation which is cool, is there a way to see the source code of the project?
Thank you guys, I hope I will figure this out…
Guys can you help me understand how to get the open-source code for the editor and model viewer I literally don’t understand how to unpack and see the source, I have tried all day …
Thank you!
I’ve been trying all day to get access to the source code of the mode-viewer and I literally don’t understand how can I put this together, what you are suggestion works in the model viewer even if in some cases is how you said for example if a marker is behind the frog leg it is still visible but is not a big deal.
Can you please explain to me how this is done maybe show me an example.
I apologize for asking this but as I said I am in the process of learning and it is overwhelming…
Thank you!