Hello, I’ve created a simple AR application where the user is able to project a simple object in the scene, at the place where the “Reticle” is been shown. So the user opens the application and he needs to find the surface, after he finds the surface, the reticle is shown in the surface. But if he doesn’t find the surface the reticle is not shown.
So, now I would like to show to the user an “instruction” before the surface is detected. How can I do that?
The workflow would be. The user opens the application, and while he doesn’t track the surface the app should show him an instruction telling him “Move the Phone slowly to detect a surface”. After the surface is detected, the instruction should disappear, and the reticle should appear in the surface detected. If the surface is lost, the instruction should appear again. Until the time that the user is able to detect the surface and project the object on it.
My main goal now is: show him this instruction telling him to track the surface until the app detects it. So he will know that he needs to find a surface before projecting the object on it.
Where in the documentation can I find it? Or in which sample can I find it?
Can you share a live example of what you have so far? I’m thinking somewhere in your code you’re adding and removing the ‘reticle’ to and from your scene in some conditional statement, it should be as simple as finding this code line and using the section that’s removing or setting the reticle to visible = false, this would be the place to add your prompt
Yes, sorry the late reply. Was working on some other things.
But yes, for sure, Here it goes.
This is my whole application.
async function activateXR() {
// Add a canvas element and initialize a WebGL context that is compatible with WebXR.
const canvas = document.createElement("canvas");
document.body.appendChild(canvas);
const gl = canvas.getContext("webgl", {xrCompatible: true});
// To be continued in upcoming steps.
const scene = new THREE.Scene();
const directionalLight = new THREE.DirectionalLight(0xffffff, 0.3);
directionalLight.position.set(10, 15, 10);
scene.add(directionalLight);
const ambient = new THREE.HemisphereLight(0xffffff, 0xbbbbff, 0.3);
scene.add(ambient);
const light = new THREE.DirectionalLight();
light.position.set( 0.2, 1, 1);
scene.add(light);
// Set up the WebGLRenderer, which handles rendering to the session's base layer.
const renderer = new THREE.WebGLRenderer({
alpha: true,
preserveDrawingBuffer: true,
canvas: canvas,
context: gl
});
renderer.autoClear = false;
// The API directly updates the camera matrices.
// Disable matrix auto updates so three.js doesn't attempt
// to handle the matrices independently.
const camera = new THREE.PerspectiveCamera();
camera.matrixAutoUpdate = false;
// Initialize a WebXR session using "immersive-ar".
const session = await navigator.xr.requestSession("immersive-ar", {requiredFeatures: ['hit-test']});
session.updateRenderState({
baseLayer: new XRWebGLLayer(session, gl)
});
// A 'local' reference space has a native origin that is located
// near the viewer's position at the time the session was created.
const referenceSpace = await session.requestReferenceSpace('local');
// Create another XRReferenceSpace that has the viewer as the origin.
const viewerSpace = await session.requestReferenceSpace('viewer');
// Perform hit testing using the viewer as origin.
const hitTestSource = await session.requestHitTestSource({ space: viewerSpace });
const loader = new THREE.GLTFLoader();
let reticle;
loader.load("https://immersive-web.github.io/webxr-samples/media/gltf/reticle/reticle.gltf", function(gltf) {
reticle = gltf.scene;
reticle.visible = false;
scene.add(reticle);
})
let flower;
loader.load('Mouthwash.glb', function(gltf) {
flower = gltf.scene;
});
session.addEventListener("select", (event) => {
if (flower) {
const clone = flower.clone();
clone.position.copy(reticle.position);
scene.add(clone);
}
});
// Create a render loop that allows us to draw on the AR view.
const onXRFrame = (time, frame) => {
// Queue up the next draw request.
session.requestAnimationFrame(onXRFrame);
// Bind the graphics framebuffer to the baseLayer's framebuffer
gl.bindFramebuffer(gl.FRAMEBUFFER, session.renderState.baseLayer.framebuffer)
// Retrieve the pose of the device.
// XRFrame.getViewerPose can return null while the session attempts to establish tracking.
const pose = frame.getViewerPose(referenceSpace);
if (pose) {
// In mobile AR, we only have one view.
const view = pose.views[0];
const viewport = session.renderState.baseLayer.getViewport(view);
renderer.setSize(viewport.width, viewport.height)
// Use the view's transform matrix and projection matrix to configure the THREE.camera.
camera.matrix.fromArray(view.transform.matrix)
camera.projectionMatrix.fromArray(view.projectionMatrix);
camera.updateMatrixWorld(true);
const hitTestResults = frame.getHitTestResults(hitTestSource);
if (hitTestResults.length > 0 && reticle) {
const hitPose = hitTestResults[0].getPose(referenceSpace);
reticle.visible = true;
reticle.position.set(hitPose.transform.position.x, hitPose.transform.position.y, hitPose.transform.position.z)
reticle.updateMatrixWorld(true);
}
// Render the scene with THREE.WebGLRenderer.
renderer.render(scene, camera)
}
}
session.requestAnimationFrame(onXRFrame);
}
So, I have a .gif that show to the user how he should do with the phone to detect the surface (floor).
And the gif must appear fixed in the screen
OK, in your onXRFrame loop, you find the above lines of code where reticle.visible is set to true
when the conditional if statements return true, you could modify this if statement slightly in conjunction with an else statement after it to alternate the visibility of both the reticle and your instructions overlay. Does this make sense?
You could add it as an image element on the dom in front of your three js canvas no? Or you need it specifically as a texture inside the three js scene graph?
EDIT: @cebolitos have a look at this example to get an idea of what i mean, it’s not what you need but shows html element overlays when the couch is added to the environment
Yes. This is the most important part of my question actually. How to place the image (my gif) on top of the screen when the reticle is not visible (when there is no surface detected).
What comands should I use? Or what should I be looking for?? The three.js documentation? The Dom documentation?
Hey @Lawrence3DPK, could you help me man ? How could I place a html element on top of my AR screen?
I cannot understand how to do that, considering that since the app is running it is not a page anymore, but on the “camera”. So how can I show a html element on top of the camera view ?
Yes if you look at the example I shared in the last post, you will see in the html I have created the overlay (outlined in orange) in the html and set it to display none in css and the z-index should be above three js canvas…
Where can I see the code of the example that you shared in the last post?
I’m really mixing up the things. Because I have the html page where I have the button to Start the AR. So should I create another html page with the html elements that will go in the z-index above three js canvas?
No you don’t need another html page, you just need to add your overlay to the html, add your image in the html and set z index to above the button and three js canvas and you will see the image right?
i remember this problem now, it was a tricky one to figure out, the answer is in the couch example i shared but hidden, essentially you need to feed the ar button the element you want to use as the overlay first, i did this by modifying three’s ar button here
so that the html elements i want to include are rendered as part of the overlay the xr session uses…
