Finalizing displaced vertices using displacement map

I am trying to finalize the position of the vertices that are displaced by the displacement map. When I finalize the position , there are holes in the geometry. I am doing this by displacing the vertices along the(normals) y axis of the model. I am not using a normalmap I am not doing this inside the shader since I want the final values of the vertex positions to be saved, but the result has holes in it.
I am doing this by displacing the vertices along the(normals) y axis of the model.

How can I fix this ?

It takes a few seconds for the processing to take place :

Here is a js fiddle : Edit fiddle - JSFiddle - Code Playground
but the url for displacement map is not working properly
This related issue does not tell a fix : displacementMap does not update normals · Issue #19677 · mrdoob/three.js · GitHub

@Mugen87 any hints on this ?