This is a limitation of using a texture atlas like you’re using here. For WebGL1 I’m not sure if there’s a great solution outside of adding gutters between the tiles as you’ve suggested or writing a shader with custom mipmap sampling code.
For WebGL2 you can use a DataTexture2DArray which should allow you to sample one of the tiles based on an index variable and prevent bleeding between the tiles. You’ll still have to write a custom shader for that, though.