This is some nice work! I’m glad to see someone taking on this challenge in the context of pure Three.js.
I’ve implemented something very similar to fulfill the exact same needs, but specifically within our own scene management framework Troika. It works very well, but I haven’t publicized it to the community because (1) it requires buying into using Troika as a framework for the whole scene, which isn’t something I’m comfortable pushing, and (2) it’s almost entirely undocumented, which is my own failing. But, I’ll point you to its source and describe some of the differences, in case you want to steal any of its ideas. 
Here’s the source code for reference: troika/packages/troika-3d-ui/src at master · protectwise/troika · GitHub
And a couple examples using it:
https://troika-examples.netlify.app/#globeConnections
https://troika-examples.netlify.app/#ui
This does use Yoga, as others have suggested. This gives it full flexbox capability, including sizing that depends on text length. Yoga is delivered as ASM-compiled code which makes it run fast, though I do still run the entire layout process in a web worker to avoid frame drops which is very important in VR.
The main downside of Yoga is that it’s big. Like >350K big. I’m hoping that improves with Emscripten compiler improvements but can’t count on it. I haven’t found any smaller solutions that have anywhere near its capability.
For building the UI structure I’ve chosen an abstraction where you can “decorate” any component as a FlexNode. I provide builtin decorated FlexNode components for 2D panel blocks, but you can also wrap any 3D object this way to let it participate in the overall flexbox layout. So for example you could have a globe of the earth that flexes within the layout and is sized to fit in its resulting box, but isn’t constrained to a 2D plane.
For common flat 2D panels, it supports backgrounds and borders with solid colors or image textures via the material. It also supports border-radius for each corner. Currently it does this by modifying the fragment shader, which works well enough but has the downside of requiring the material to be transparent and strictly controlling the renderOrder to get proper antialiasing. I’ve got ideas to improve that in the future.
I’ll echo others and suggest using the polygonOffset material properties for the coplanar layers to avoid z-fighting without visible z shifts. Sometimes a z shift can be nice in VR as an extra affordance for buttons and the like, but often you don’t want that. Your point about letting users supply their own material instance is a good one, and I get around that by inheriting from their material via Object.create(baseMaterial) so I can tweak individual properties without affecting their instance. That may be something you could try too.
For the text, I use our own SDF text renderer troika-3d-text – out of all these pieces I’d suggest you take a look at possibly using that for your library. It’s got a standalone Three.js class you can use independently of the Troika framework, and it has some significant advantages over any other SDF text solutions I’ve seen, such as being able to use web font files directly and using standard Three.js materials. It’s fast and folks like Mozilla MR are using it successfully, and I’m actively improving it, so give it a look.
Something yours does that mine doesn’t is mixing text styles within a single block. That’s a great feature and something I’ll definitely consider adding.
Again, this is great work! I hope something in here is helpful to your effort, but if not, keep up the good job!
