How to create Sketchfab like annotations with Three.js?

karanganesan · February 7, 2020, 1:03pm

Check this example

I have created the hotspot/annotation but I can’t find a logic to detect a marker if it goes behind the camera’s field of view

So I have tried a couple of methods already

Based on distance between hotspot and camera didn’t work
Angle between normal of hotspot and camera didn’t work
Using raycasters is computationally heavy and hard to work with

the scene just has a GLTF and hotspots and camera

So I made render of GLTF to 1000
and hotspot to 1100
and set the depthTest of marker to false so that marker is rendered ontop of model

@thrax from Three JS Slack tried something here with dot product but still it didn’t solve my usecase https://glitch.com/edit/#!/three-hotspot

Regards,
Karan.

Popov72 · February 7, 2020, 3:24pm

What you can do:

enable a depth rendering of the scene
create a shader material to apply to the hotspot:
- enable alpha blending
- set depth test to false to always render the hotspot / disable depth write
- in the shader, compare the current depth with the one in the zbuffer (thanks to the depth rendering): set the alpha value of the final color depending on the test (for eg, 1.0 if current depth < z in zbuffer, else 0.3)

Here’s an example. It’s not done in threejs, but it should be easy to port the code.

If the depth in the depth rendering is stored as a linear depth (I don’t know how Threejs does it), here’s a modified sample.

donmccurdy · February 11, 2020, 2:31am

Evergreen article: https://manu.ninja/webgl-three-js-annotations

looeee · February 8, 2020, 2:14am

Nice article! Thanks for sharing it.

This is a very cool and simple test and looks like it should work well for any objects that are fairly regular in shape.

const meshDistance = camera.position.distanceTo(mesh.position);
const spriteDistance = camera.position.distanceTo(sprite.position);
spriteBehindObject = spriteDistance > meshDistance;

sprite.material.opacity = spriteBehindObject ? 0.25 : 1;
annotation.style.opacity = spriteBehindObject ? 0.25 : 1;

I guess if your meshes are highly irregular it might not be good enough though. Would have to test this out in practice to see if that’s the case.

EDIT: and they are definitely doing something much fancier in Sketchfab:

pailhead · February 8, 2020, 3:43am

what effect is being discussed here? When it goes outside of fov, it just disappears on my machine.

makc3d · February 8, 2020, 4:33am

@Popov72 that sounds like terribly overcomplicating things. Here is how I would do it: https://jsfiddle.net/25fjLrnd/

looeee · February 8, 2020, 5:13am

It’s the marker with the number 2 in my screenshot above. The question is how to make it transparent but still visible when occluded by the model.

Popov72 · February 8, 2020, 12:33pm

@makc3d Try to put a texture on the green cube. The hotspot should not be transparent if visible, only if it is occluded.

makc3d · February 8, 2020, 2:09pm

not sure what does the texture change here, but https://jsfiddle.net/x5321dby/

all it does is simply draws the slightly transparent copy of a cube (marker) on top of everything. when not occluded, it draws on top of itself but since it is identical, the result does not change.

looeee · February 8, 2020, 3:08pm

@makc That nearly works, but you want the marker to have opacity 1 when in front of the object and 0.3 when behind the object.

It’s not obvious when you use a cube but if you switch to a sprite you’ll see that doesn’t work as a marker.

looeee · June 16, 2020, 2:51am

Here’s an adaption of @makc3d’s idea. Two sprites, one fully opaque and one transparent with depth test disabled.

@karanganesan is this good enough for your use case?

makc3d · February 8, 2020, 3:38pm

@looeee IIRC sprites system was attached to 3js by a duct tape so I am not surprised that something does not work in that department. so, again, I would just use some hack like this https://jsfiddle.net/x8L1nwfh/ but this is now 3rd time, not making any more fiddles here

Popov72 · February 8, 2020, 10:06pm

@makc3d Sorry, I did not see you used two sprites/cubes in your code, one with a standard material and the other with transparency/depth test=false!

It’s indeed a lot easier than my method.

karanganesan · February 11, 2020, 2:28am

Thanks a lot everyone for the help I will look into them

karanganesan · February 11, 2020, 2:30am

Yea I think so

I didn’t use sprites earlier
I just used Circle Geometry earlier

Will try out this sprite and let you know

thanks

looeee · February 11, 2020, 2:42am

Sprites will make it easier to have the marker facing towards the camera, which is probably what you want.
Then you can use a transparent PNG to the make the sprite a circle - an abnoxious smiley in my example (sorry about that, it was just the first round transparent image I found on imgur ), but to create markers with numbers in them you could either have a number of pre-made PNG files, or use canvas2D to generate the images on the fly.

karanganesan · February 11, 2020, 5:54am

Cool

Thanks for the suggestions

ben · May 28, 2020, 5:18pm

No demo, interesting article

donmccurdy · May 28, 2020, 6:26pm

Another useful option, built with three.js:

https://modelviewer.dev/examples/annotations.html

ben · May 28, 2020, 8:38pm

@prisoner849 holy smokes you gotta see this: https://modelviewer.dev/examples/tester.html check add/remove hotspot bottom left then click anywhere in the scene to add the hotspot - thanks @donmccurdy for sharing, absolutely awesome sauce!