Help implementing a glass pane with colored sides

Hi everyone, I’ve been experimenting with the meshPhsyicalMaterial to implement something along the lines of this:

However, when creating a flat box with meshPhysicalMaterial and meshBasicMaterial plain color sides, this is the best I can come up with.

As you can see, the edges of colored sides seen through the material are very low quality. I’ve seen glass implementations of really high quality that still run well in the browser. Is it possible that I am working with an edge case here that just doesn’t suit meshPhysicalMaterial too well? Or are there any parameters that come to mind that would lead to a better result?
Any help is highly appreciated.