Not sure if it will help here, but there is a color coding approach to object selection.
Basically, you change all mesh materials to have a flat color and then render the scene to a texture. Each object’s color serves as an ID. Since you know mouse position, you can get the color from the texture and know what object you are hovering above. Then you can change its properties like color etc.
With a bit of the camera frustum manipulation you only need 1x1 pixel texture, to render the pixel under the cursor.
Here is an example