Well, i went with invisible objects and raycast against them
I am having issues that raycast only works on a part of the object, it depends on rotation i think where it works
for example, the black part of the orange/yellow cube in the front only works, the one in the background does not work and it is the same instancedmesh
i know that there are probably many useless updates happening now, i was just trying everything that i could find and see if something solved my issue