Ha, this is a fun topic. In my game engine, I use a whole lot of tricks to work with that. I recommend that you use CSM instead of the standard shadow map. Why? Because the whole point of the CSM is to build the shadow camera frustum relative to the view frustum.
As for me, I use something very similar to a CSM, technically it’s a CSM, but it has only 1 cascade, which makes it hard to call it CMS. It does exactly what @Mugen87 mentioned, and what CSM does - it auto-magically updates shadow camera. I also found that three.js’s frustum culling is painfully slow for scenes with 1000s of objects, but that’s a whole different story not really applicable here.