Raycast issues when used for point in object detection


I’m working on a CSG library and I am trying to use Raycast to determine if a triangle is inside or outside of the target object (mesh) by checking if the triangle’s mid point is inside or not.

For the most part raycasting works fine, but, I started running into precision issues that are conflicting:

  1. Triangle mid point on the surface sometimes being detected as inside and sometimes as outside depends on which surface the point is (which side of a cube) and which direction is being used. ← This one is ok-ish, since I know if the triangle is coplanar or not, I can do additional tests by adding an epsilon to the point and retest.
  2. Triangle mid point is outside of the object (not on surface/boundary) and is detected as inside when it is supposed to be outside (noticed when the target object is a Cylinder with radius bottom or top of 0).
    a. Example - three.js raycast test 2 - JSFiddle - Code Playground
    b. Another example, this time I rounded the point’s x,y,z down to integers ((-1.732050895690918, -1, -3) to (-2, -1, -3)) and the raycasting result is still inside instead of outside - three.js raycast test 3 - JSFiddle - Code Playground
  3. Triangle mid point is inside of the object (not on surface/boundary) and is detected as outside when it is supposed to be inside (noticed when the target object is a Icosahedron with radius 5 and detail 10).
    Example - three.js raycast test 1 - JSFiddle - Code Playground
  4. With #⁠2 and #⁠3 conflicting, how can I guarantee the inside/outside classification will be correct?

In the examples there is a small yellow sphere to show where the point is, it is not part of the raycast flow.

I tried searching online and in the forums for clues as to why I get the conflicting issues but no luck.
Any help will be appreciated!

Thank you.