Help solve a crime

Yes. This is what is done in my viper demo (on static geometry): Projecting a photo onto geometry to facilitate photo manipulation In that case, it depends on knowing (being able to guess) the geometry, motion and camera projection beforehand, though. Some of the necessary information can be obtained and applied during a crime investigation, such as camera specifications, position, rotation, building geometry, hopefully knowledge of the car model (ideally a 3D model of the car). In addition to this, street lighting, weather/sunlight (sun direction) at the time and place of the recording etc. will be useful for the analysis.

You are touching onto computer vision and specifically photogrammetry, which is outside the expected expertise of three.js forum members, as @Mugen87 says. And I honestly don’t expect Danish police experts to be able to apply state of the art methods (or develop the methods needed). I suspect they will say “that’s CSI magic, and not possible in real life”. To some extent they are right. You cannot just click “enhance image” to get more detail, but a video of a car contains multiple noisy and aliased (and compressed!) measurements of a rather tidy process. I guess it may be possible to, e.g., extract car plate numbers from videos, even when each frame does not reveal the number sufficiently well. (Two examples I found now are linked below.)

Now I have watched the video. The car is at large distance, and viewed through multiple windows, if I see right. To compensate properly for the refraction in the windows, it may be necessary to make recordings of controlled geometry and texture (e.g. a plate with a chequerboard-pattern) through the same windows. You could thus be able to generate refraction maps for the windows (or for the combination of windows?), and then “invert” the refraction of the original video.

Links:

  1. https://www.researchgate.net/publication/3428025_Superresolution_of_License_Plates_in_Real_Traffic_Videos

  2. https://articles.forensicfocus.com/2019/06/10/how-to-read-a-moving-low-quality-license-plate-using-amped-fives-perspective-stabilization-and-perspective-super-resolution/

PS: I named camera specification among necessary input, but if you can get to the location of the video recording and make detailed measurements of the hallway, you will probably be able to derive camera position, orientation, aspect ratio and FOV even without access to the camera. Likewise, knowing the suspect car model may help deriving refraction maps for the windows, without having to measure them using a reference. But it probably is a lot harder, though, not least because the motion estimation then is coupled to the estimation of refraction.

PPS: Even though I can confirm that it is possible to project an image from a virtual camera onto geometry in a scene, that is not necessarily what you are interested in doing. Collecting good information about the scene and camera will be useful for trying to reproduce the video by simulating (realistic) motion of the same type of car (probably a Hyundai i30 hatchback, according to the Danish police) with different license plate numbers, window refraction maps, outdoors light settings etc., and optimizing all these variables for video reconstruction. (I don’t know if this is really viable.)

1 Like