When using WebXR, layer 1 and 2 are reserved for XR rendering. When assigning objects just to layer 1, they are only rendered for the left eye. Layer 2 is assigned to the right eye. Objects on the default layer 0 are rendered in both views.
However, when using other layers the visibility and raycasting control should work as usual. Can you give it a try with layer 3?