Hello there,
I am working on an application implying three.js, AR.js and THREEx; it’s a restaurant menu.
I got two problems and two questions.
1)The first problem is that the raycaster seems to intersect objects that are not actually appearing on screen, their properties (like opacity, visible or transparent) not being changed based on marker-detection and rendering.
Basically, I am adding all of the “markerRoots” in the scene while loading the models and it seems that they are all at the same distance (as checked in the console), and given that the raycaster.intersectsObjects(array) returns an array with the items ordered by distance, I suppose that is one of the possible problem causes.
On the other hand, I can intersect the elements even when the screen is empty and my phone’s camera is facing the table.
Here are some videos of the glitches to aid my description: https://we.tl/t-nXbSeIcD6c .
In one of the videos, I can open the second scene (that is meant to open when clicking the “select” button) while randomly touching my empty screen. In the other one, I open the second scene for the meal A while selecting the meal B.
Here are some significant related parts of the code:
const arToolkitSource = new THREEx.ArToolkitSource({
sourceType: "webcam",
//parameter names erroneously inversed in native implementation
sourceWidth: HEIGHT,
sourceHeight: WIDTH,
// resolution displayed for the source
displayWidth: WIDTH,
displayHeight: HEIGHT,
});
const arToolkitContext = new THREEx.ArToolkitContext({
// debug: true,
cameraParametersUrl: "camera_para.dat", //necessary parameters
detectionMode: "color_and_matrix", //improved accuracy
//4x4 matrix code with Bose-Ray-Chaudhuri error correction
matrixCodeType: "4x4_BCH_13_9_3",
maxDetectionRate: 60, //attempted in the camera image (FPS)
// detection resolution for the source
canvasWidth: HEIGHT,
canvasHeight: WIDTH,
});
const camera = new THREE.PerspectiveCamera(75, WIDTH / HEIGHT, 0.1, 1000);
camera.rotation.set(0, 0, 0);
camera.position.set(0, 0, 25);
camera.lookAt(0, 0, 0);
const directionalLight = new THREE.DirectionalLight(0xffffff, 5);
directionalLight.position.set(0, 0, 10);
directionalLight.castShadow = true;
scene.add(directionalLight);
const renderer = new THREE.WebGLRenderer({
antialias: true,
alpha: true,
});
renderer.setPixelRatio(window.devicePixelRatio);
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.shadowMap.enabled = true;
renderer.shadowMap.type = THREE.PCFSoftShadowMap;
renderer.shadowMap.debug = true;
renderer.sortObjects = true;
renderer.gammaOutput = true;
const canvas = renderer.domElement;
canvas.willReadFrequently = true;
document.body.appendChild(canvas);
//LOADING Meal
public loadMeal = async (): Promise<Model> => {
this.markerRoot = new THREE.Group();
this.markerRoot.userData.name = this.name;
scene.add(this.markerRoot);
try {
new THREEX.ArMarkerControls(arToolkitContext, this.markerRoot, {
type: "pattern",
patternUrl: this.pattern,
});
} catch (err) {
alert(err);
}
return new Promise((resolve, reject) =>
gltfLoader.load(
this.source,
(gltf: any) => {
const model = gltf.scene;
model.userData = { name: this.name, type: "meal-model" };
model.scale.set(...this.modelScale);
model.position.set(0, 0, 0);
model.rotateX(-Math.PI / 2);
model.castShadow = true;
this.modelObject = model;
this.markerRoot!.add(model);
const button = this.createTextButton();
button.userData = { id: "select-meal", target: this.name };
this.button = button;
this.markerRoot!.add(button);
model.traverse((child) => {
if (child.isMesh) child.castShadow = true;
});
camera.updateProjectionMatrix();
render();
resolve(this);
},
undefined,
function (error: any) {
reject(error);
}
)
);
};
function getRootParent(object) {
let child = object;
while (child.parent !== null) {
child = child.parent;
if (child.parent.isGroup) return child.parent;
}
return null;
}
function scaleModel(model, targetScale, duration) {
new TWEEN.Tween(model.scale)
.to(targetScale, duration)
.easing(TWEEN.Easing.Quadratic.InOut)
.onUpdate(() => {})
.start();
}
const raycaster = new THREE.Raycaster();
raycaster.params.Points.threshold = 1000;
const pointer = new THREE.Vector2();
function getIntersects(point, objects) {
pointer.set(
(point.clientX / window.innerWidth) * 2 - 1,
-((point.clientY / window.innerHeight) * 2) + 1
);
raycaster.setFromCamera(pointer, camera);
return raycaster.intersectObjects(objects, true);
}
function onTouchMealStart(event) {
if (event.touches.length === 1) {
touchStart = event.touches[0];
previousTouchX = touchStart.clientX;
previousTouchY = touchStart.clientY;
intersectsModels = getIntersects(touchStart, modelClones);
intersectsButtons = getIntersects(touchStart, buttonObj);
if (
intersectsButtons.length > 0 &&
globalThis.currentScene.userData.name == "primary-scene"
) {
const touchedButton = intersectsButtons[0].object.parent;
const model = globalThis.models.find(
(m) => m.name === touchedButton.userData.target
);
modelClones = selectMeal(selectedScene, model);
} else if (
intersectsModels.length > 0 &&
globalThis.currentScene.userData.name === "secondary-scene"
) {
touched3DModel = getRootParent(intersectsModels[0].object);
let touchedArrayModel = modelClones.find(
(m) => m.userData.name === touched3DModel.userData.name
);
initialScale = new THREE.Vector3(...touchedArrayModel.userData.scale);
increment = initialScale.x / 3;
const newScale = initialScale.clone().addScalar(increment);
scaleModel(touched3DModel, newScale, 2000);
}
}
}
2)The second problem is that I cannot cast shadows for the GLB models. I looked over other questions opened on this topic, but it seems I have applied everything mentioned and still not getting any results. Maybe I am still missing something.
I have two scenes, and I am switching between them by rendering globalThis.currentScene in the render function and modifying that variable. In the second one, that has a MeshPhongMaterial plane serving as a background, I managed to cast shadows for the Texts, but not for the models, even if I traversed them and applied castShadow=true on each child mesh.
const light = new THREE.DirectionalLight(0xffffff, 5);
light.castShadow = true;
light.shadow.mapSize.width = 1024;
light.shadow.mapSize.height = 1024;
light.shadow.camera.top = 2;
light.shadow.camera.bottom = -2;
light.shadow.camera.left = -2;
light.shadow.camera.right = 2;
light.shadow.camera.near = 9;
light.shadow.camera.far = 15;
light.shadow.bias = -0.02;
light.position.set(-1, 0, 10);
light.target.position.set(0, 0, 0);
scene.add(light);
const shadowReceiver = new THREE.Mesh(
new THREE.PlaneGeometry(50, 50),
new THREE.MeshPhongMaterial({ color: 0xa3258b })
);
shadowReceiver.receiveShadow = true;
shadowReceiver.position.set(0, 0, -0.5);
scene.add(shadowReceiver);
Here I am cloning the model because if I use the same instance, it does not appear in the main scene afterwards. (I suppose the same object may exist in a single scene)
model = modelClones.find(
(m) => m.userData.name === modelInstance.modelObject.userData.name
);
if (!model) {
model = modelInstance.modelObject.clone();
const scale = modelInstance.modelScale.map((s) => s / 2);
model.scale.set(...scale);
model.userData.scale = scale;
model.position.set(0, 0, 0);
// model.lookAt(camera.position);
model.castShadow = true;
model.traverse((child) => {
if (child.isMesh) {
child.castShadow = true; // Enable shadow casting for mesh materials
}
});
modelClones.push(model);
populatePanels(modelInstance);
Ok and for the questions:
-
May I cast shadows while using the camera? I mean, if I have the markers printed on paper, can I cast the shadow of the model on the surface somehow?
I have seen this example in which something similar happens, even though I cannot see the shadow while scanning the marker on my screen.
AR-Examples/shadow.html at master · stemkoski/AR-Examples · GitHub
Hello, world! -
Currently if I rotate my phone from portrait to landscape my models are also rotating. Can I prevent that somehow? (Similar to how it works in the example above: if you rotate the phone, the model stays still)
I would greatly appreciate any help! Thank you in advance!