Hey, I’ve been working on a projection mapping library, for real-time 3D projection mappings, looking for feedback and testers.
The main use case is to match your Three.js camera to the physical projector’s real-world position and optics using ProjectorCamera, so the virtual scene aligns with the physical surface, which can then be fine-tuned with warping. It accepts any THREE.WebGLRenderTarget or THREE.Texture, so it works with 3D scenes, canvas textures, videos, or any other source. See the examples for 2D content usage.
Current Features:
- Perspective & Grid-Warping (bicubic/bilinear vertex shader)
- Feathered Polygon Masking
- Image adjustments (contrast, gamma, …) & procedural testcard
- Multi-window Mode via BroadcastChannel
ProjectorCamera for hardware optics matching- Auto-save to Local-Storage
Links:
Still in early alpha the API will change. Looking for feedback on use cases, missing features and potential contributors welcome.
8 Likes
This is amazing!! I have been searching for a project like this for ages, even considered jumping to develop it, really interested in getting my hands wet with it.
In the meantime I will be testing with basic use-cases, but let me know if you want me to try more complex stuff (benchmarks maybe?), and feel free to reach out for specific issues/challenges you may be considering.
Keep up the good work!!
1 Like
Thanks for the kind words, Antonio 
Use-case feeedback (API, UX, stability) would be the most helpful right now, If you run into any problems I’d love to hear about it.
A code review of the multi-window (BroadcastChannel) and auto-save (localStorage) features would also very welcome, as these are the areas im least confident in.
Next up: bezier masks
1 Like
When I first read the title I thought of something that I needed in the past, and ended up writing it myself (there was no AI then). It was: the input is the screen 2D coordinates of 4 vertices of a sheet of paper; output is 3D coordinates and orientation of perspective camera, so that a virtual sheet is seen in the same way on the screen. But maybe this is not what your lib does, as it feels more like 3D billboards → see this.
Do you have the option to prescan the original physical surface and get the color of various sections, so when projecting an image to use correct color correction for each pixel? This would make it easier to project on buildings’ facades that are not plain color.
And some observation: When I drag vertex A upwards, at some point the polygon mask behaves unexpectedly:
2 Likes
Hey Pavel, thanks for your feedback.
I’m interested in your project is the code available online, it sounds like the inverse what im doing. The Example with the billboard you posted is something, is something I’m currently looking into with Perspective Reprojection (see. Blender Forum Perspective Projection Mapping).
The repositories that helped me the most were Omnidome (Bicubic Grid Warp) and p5Mapper (Perspective Distortion).
Color correction is supported (Gamma, Contrast, Hue), RGB Channel Wise Curves (like in Photoshop) would probably a good idea. The prescan method you mentioned sounds very interesting, something worth adding in the future, do you have any more information or a paper how to implement it?
On the Polygonmask behavior i will look into it.
