The best way to implement draggable picture buttons in a GUI?

Hello. I’m pretty new to three.js (and to javascript in general).
I have a question about the best way I should implement buttons.

Here is some background on my project. I’m making an educational game that teaches kids how to do coding. It uses 3D graphics for a 3D character that navigates through a 3D puzzle.

I would like to add some 2D buttons with icon images on them.
Here is an example of something like what I’m trying to do:

I would like to have users be able to drag and rearrange the buttons on the bottom, and the buttons will have icons (and no text). Everything is done with javascript to render the interface, so I can’t just have something like html <input type=“button” value=“Click me” onclick=“msg()”>

One problem is the main models are 3D, but I want the interface to be 2D. I could make multiple views (like in the this demo program https://threejs.org/examples/#webgl_multiple_views) but that doesn’t seem ideal to have one view just for rendering a 2D interface.

I could just draw 3D boxes and keep those boxes always neatly parallel to the camera’s view (making them appear as 2D buttons) but that seems like the wrong way to accomplish the goal (and a headache to implement).

I am wondering if there is some library out there that has done all this for me. I would love to be able to input a .png image file, set the draw coordinates relative to the window, and let some library do all the heavy lifting with all of the mouse over and dragging events. The GUI I have seen (dat.gui.min.js) seems like it is more lightweight and not designed to be customized with dragging objects and buttons with icons.

I’ve seen PIXI.js which seems promising, but I’m not sure how that meshes with the rest of the 3D parts of the program. It seems like PIXI.js does something like “var app = new PIXI.Application” and you have to do everything within a PIXI.Application object, so it seems like you choose to do everything in PIXI.js or nothing in it.

I would very much appreciate some suggestions about how I could implement a GUI like the one I’ve described. What is the best way to do something like that?

Thanks in advance.

Hi!

Just out of curiousity, why was it not an option to create UI with html elements?

You might want to use sprites rendered with an orthographic camera like demonstrated in the following example:

https://threejs.org/examples/webgl_sprites.html

Sprites support raycasting so it should be easy to make them interactive.

https://threejs.org/examples/webgl_raycast_sprite.html

Seems, you’re looking for something like that:

Hi @prisoner849,

I would like to have the buttons superimposed over the 3D rendered scene, and have the buttons made from image files, and have the elements able to be dragged around the whole window (and reordered in the list to the bottom). Do you suggest html elements would work well for this purpose? Do you know of any demos of similar things implemented in another program?

You might want to use sprites rendered with an orthographic camera like demonstrated in the following example:
https://threejs.org/examples/webgl_sprites.html
Sprites support raycasting so it should be easy to make them interactive.
https://threejs.org/examples/webgl_raycast_sprite.html

That was one of the things I considered but can you have the elements rendered from an orthographic camera superimposed over the window with the 3D elements? It was my understanding that they would have to be seperate, like in https://threejs.org/examples/#webgl_multiple_views

The shared demo does exactly this. It performs two renderings with two cameras.

prisoner849
Seems, you’re looking for something like that: Draggable Elements That Push Others Out Of Way | CSS-Tricks

That is the sortability I am looking for. I don’t know if html elements can be superimposed over a 3D rendered window though.

If I could render 2D pictures (as if from an orthographic camera) on top of a 3D camera, I could implement these sorting things myself and that would give me complete control over the aesthetic and how the interface works.

I guess I will look carefully at that demo to see how they do that. And I guess the best way for me to implement my interface is to write the button elements myself based on those 2D rendered sprites.

Is that how interfaces with buttons are typically done? Like for example, http://www.playkeepout.com
That game has a 3D rendered world with images (used as buttons) superimposed.

Thanks for the help.

Many official examples have overlay of div with info, that is atop of the renderer.
Why wouldn’t it work with elements of buttons or images?

prisoner849, I don’t recall ever seeing div elements overlaid on top of 3D cameras. I didn’t know you could do that. I’ll look for examples. If you know of some, please share a link.

So I guess it is possible to do what I’m wanting to do with div elements overlaid, or with an orthrographic camera rendered sprites overlaid.
I’m still not sure which would be better. It seems like I could get a lot of work done for me with div elements (for example, the sortable example posted). But I might come to a stumbling block somewhere, and it seems like I could always implement whatever I wanted if I were to make the interface with sprites.

Which of these two paths seems like the best solution? How would you do something like this?

Years ago, I’ve made this a kind of game :slight_smile:

Jag4k - Just a game for kids (jag4k = just anogher game for kids :smile: )

UI is made with div elements, atop of a renderer. Made with r71, but enough to show the concept :slight_smile:

1 Like

After looking at your demo, I think html elements might be the way to go over sprites with an orthographic camera. While I will have less overall control, I think everything I need is already written for me so I won’t have to reinvent the wheel.

Btw, your sample game totally reminds me of one of my 4 year olds favorite shows : team Umizoomi. As the 2D shapes turn to 3D I imagine Gio saying “super shapes!”

Thanks for the help. Now I just have to figure out some stuff like that sortable tiles as was previously linked.

I’ve never heard about it :slight_smile:

It’s so cute!

:truck: :clock130: :1234: :rosette::small_red_triangle::white_small_square:

You should upgrade it to a newer version.

1 Like

@hofk
And there is the concept of this game, made with circles of div for 2D view :smile:
http://west77.ru/content/r71/jag.html