Is there a way to apply HTML on top of a geometry/mesh

Hi,

I’m pretty new to ThreeJS so sorry for maybe formulating something wrong. I’m currently working on my own website and wanted to have a menu on the main page where all my sub pages are rendered as curved plane geometries and can be rotated on scroll (rotated clockwise at an anchor).
Currently it looks like this:


I’m using r3f and this to render the curved planes. The <Html/> component provided by react-drei is not applying the the <Html/> onto the geometry and i think the CSS3DRenderer is also not helping here.
Currently my code looks like this:

import React, {useEffect} from 'react';
import * as THREE from 'three';
import { Html } from "@react-three/drei";
import Blog from "../screens/Blog";

/**
 * props can contain following variables:
 * initialPos - THREE.Vector3
 * initialRot - THREE.Vector3, in rad for x,y,z
 * width - double, width of the geometry
 * height - double, height of the geometry
 * widthSegments - double, widthSegments of the geometry
 * heightSegments - double, heightSegments of the geometry
 * bend - double, bend of the geometry
 */
export const CurvedPlane = React.forwardRef((props,ref) => {
    useEffect(() => {
        ref.current.position.set(props.initialPos.x, props.initialPos.y, props.initialPos.z);
        ref.current.rotateX(props.initialRot.x);
        ref.current.rotateY(props.initialRot.y);
        ref.current.rotateZ(props.initialRot.z);
    }, [])

    return (
      <group ref={ref} dispose={null}>
        <mesh geometry={planeCurve(new THREE.PlaneGeometry(props.width,props.height,props.widthSegments,props.heightSegment),props.bend)}>
          <Html transform>
            <div className="wrapper">
              <Blog/>
            </div>
          </Html>
        </mesh>
      </group>
    )
});

function planeCurve(planeGeometry, z){
  let p = planeGeometry.parameters;
  let hw = p.width * 0.5;
  
  //Three points are being defined, left edge, right edge and z value for the plane.
  //These three points cut the sphere surface with radius r.
  let a = new THREE.Vector2(-hw,0); 
  let b = new THREE.Vector2(0,z);
  let c = new THREE.Vector2(hw,0);
  
  let ab = new THREE.Vector2().subVectors(a,b);
  let bc = new THREE.Vector2().subVectors(b,c);
  let ac = new THREE.Vector2().subVectors(a,c);
  
  //Formula for finidng a circle which contains points a,b and c.
  let r = (ab.length() * bc.length() * ac.length()) / (2 * Math.abs(ab.cross(ac)));

  //Center of circle is being calculated. The center is exactly at z -r, because the radius of the circle is r.
  //and b is is with (0,z) on the outer edge of the circle.
  let center = new THREE.Vector2(0, z - r);
  //Generates a vector from the center to the point at the bottom left. 
  let baseV = new THREE.Vector2().subVectors(a, center);
  let baseAngle = baseV.angle() - (Math.PI * 0.5);
  let arc = baseAngle * 2; // <- This can maybe indicate some reflection!
  
  let uv = planeGeometry.attributes.uv;
  let pos = planeGeometry.attributes.position;
  let mainV = new THREE.Vector2();
  for(let i = 0; i < uv.count; i++){
    let uvRatio = 1 - uv.getX(i);
    let y = pos.getY(i);
    mainV.copy(c).rotateAround(center, (arc * uvRatio));
    pos.setXYZ(i, mainV.x, y, -mainV.y);
  }
  
  pos.needsUpdate = true;
  return planeGeometry;
}

This code leads to outputs like these:


As you can see the transform is applied, but the html is not attached (and also the scales are not correct) on the surface of the curved plane.

Would appreciate any ideas of how this can be solved!

html is on top of the canvas, it gets transformed into place, but it has no further relation, it can’t possibly bend around an object. you can attach dreis <Html> component to a mesh and position it, so that it follows along, a good example is this: Mixing HTML and WebGL w/ occlusion - CodeSandbox but that approach is always limited to html being something that’s running in its own context.

you have other options, drei has components for <Text> (sdf using troika) and <Text3D> using plain text geometry. you can also project textures using decals. but curving real html around a mesh … i haven’t seen anything like that yet — i would probably just make the UI in threejs in that case, using shapes and such.

i haven’t seen anything like that yet — i would probably just make the UI in threejs in that case, using shapes and such.

Could you please elaborate what you mean here? As far as i undestand you, there is no known way to make HTML take the form of some surface?
I guess for now i will just build something similat to this (the helix view):
https://threejs.org/examples/css3d_periodictable.html
Using some combination of <Html/> and PlaneGeometry should be able to do this, but am a bit disappointed that there is no easy solution for curving html :confused:

well, how would you curve just plain old regular html, a div, spans, etc? i found this on stackoverflow but doesn’t seem to have an answer javascript - Is it possible to make curved shape in 3d perspective div? - Stack Overflow you can scale, translate, rotate and skew html, but not project it onto a curve, not to mention a complex shape. even if the css filters that they mention worked, they are dead slow.

Could you please elaborate what you mean here?

you don’t have to use html, you can make UI in threejs with meshes, text and shapes. the ui here for instance Github 2049 has been made like that, the article explains how https://formidable.com/blog/2021/future-ui/

other than that, crazy 3d arranged ui is often silly, is it really worth it when it wont be accessible or easy to use and if most people will reject it outright. sometimes it’s better to keep things simple, and you don’t have to break your head. :smile:

1 Like

ps there was a new thing in drei called <RenderTexture> which allows you to project a real scene onto any shape. again it’s not html, but this is probably the closest thing to what you want. https://twitter.com/0xca0a/status/1521842820012818432

1 Like

Okay <RenderTexture/> looks damn awesome! Thanks for the tips, will take them definitely into account!