A dashboard i have been building over the last 6 months. Values are provided for ~800 countries or regions on a daily basis, complete history of values available feeding diagrams and animations. Implemented on top of an empty ArcGIS Javascript API scene, lighting / geometry / shadows are all threejs for flexibility and performance reasons.
This project aims to provide an alternative to the many dashboards we have seen by offering a more visual approach that should invite the user to explore the data and to gain a better understanding of what has been going on over the last months.
It looks great! But performance-wise, it’s running really slow on my mid-range laptop. Are you rendering each shape as its own individual Mesh? It seems like the huge number of drawcalls is having an impact on rendering speed.
an attempt to describe the structure of the scene:
─┬─ ROOT (GROUP)
├─┬─ 1057 : GRATICULE (GROUP)
│ └────────── N : MESH - triangles forming the globe (25623 faces total)
└─┬─ 802 : GRAPHIC (GROUP)
├─┬──────── 1 : GROUP
│ └────────────── 1 : MESH - vertical parts of each shape (120566 faces total)
├─┬──────── 1 : GROUP
│ └────────────── 1 : MESH - horizontal parts of each shape (74148 faces total)
└────────── N : PATH - top outline of each shape (60283 line segments total)
so its ~200000 polygons and ~60000 path segments. 2 lights, one casting shadows, shadow map size between 512 and 1024. all geometries are the BufferGeometry versions. a lot of objects are transparent.
no explicit draw commands are issued. rather i call this.renderer.render(this.scene, this.camera.getCamera());
i would really appreciate any hints on where to start analyzing / optimizing. there is so much work in this project that i would really like to add that extra effort to get the most of it.
i would really appreciate any hints on where to start analyzing / optimizing. there is so much work in this project that i would really like to add that extra effort to get the most of it.
I would take advantage of the performance tab in chrome or the three.js plugin. From the performance tab it looks like a CPU bound issue and using a breakpoint I can see that there are 12959 objects in the scene hierarchy and 4590 meshes to render.
It looks great, though! The transparent effect on the geometry is really nice.
So when you perform render(scene, camera), WebGL has to draw each Mesh to the canvas one at a time. This takes a tiny fraction of a second, but when you have thousands of meshes, your framerate will start taking a hit. And when you have realtime shadows, you’re drawing thousands of meshes x2 on each frame, since it has to render the shadowmap first, then the final scene.
See this example for a comparison: https://threejs.org/examples/webgl_instancing_performance When you select Instanced, all monkeys are rendered in a single drawcall (via InstancedBufferGeometry), and when you select Naive, each monkey is rendered sequentially, since each one is its own mesh. You’ll start seeing a framerate drop when the count is high enough.
You might want to render all countries and states at once by merging all your geometries into one, and then changing their colors with Material.vertexColors. The change in height, however, would be a different problem. Maybe custom shaders?
The divisions within countries seems a bit weird. For example, England, Italy, and Germany are split up into provinces/counties, but then Ireland is just split into south/east/west (which are not real regions), while France, Scotland, Whales, Northern Ireland, and all Eastern European countries are not divided at all. That’s just Europe, similar odd divisions seem to occur on other continents too. Is this something to do with the data set you are using?
I’m also trying to make sense of the coloring. It seems like dark blue is relative to the total worldwide number of cases, or perhaps the country with current highest cases? It makes it a little hard to track for one country. Again, focussing on Ireland, when I zoom over the country, it’s hard to see the height map since I’m directly above, then if I select total cases and play the timeline, it appears like the country is getting a lighter shade of blue as number of cases increasing, when I would expect it to get darker. I’m not sure what a good solution is here, but this seems a little hard to interpret, currently. Maybe make an option to have coloring relative to the country, or add another shade so that the highest overall is a red tinge while per country is blue? Than as the cases within a country get higher it will continue to increase in color even while other countries take over.
If i get it right, instanced mode would require objects sharing the same geometry initially and therefore each country on the map would have to be a distinct instance. Maybe i have to do some more reading.
I installed the three.js plugin which helps a lot in understanding some parameters and a few things could be improved:
Merge the entire globe geometry from many meshes down into a single mesh
Whenever a country was composed of several part, it had n-meshes for the vertical parts, now its one country - one mesh
Same applies for outlines - one country - one outline
There were many materials in the scene, i have created a lookup table of materials, and actively reuse materials now
In the “worst” case view of the globe there were
~ 3000 draw calls
~ 4600 geometries
Now its down to (which still appears a lot, but still is good double digit percentage improvement):
~ 2000 draw calls
~ 3100 geometries
For next steps am thinking about 2 drawing “modes” that i may create poc’s for:
“static”, when you are not dragging the time slider. geometries are stationary with respect to each other. since now i have discrete colors i could throw geometries into “color buckets” and then end up with some 50 geometries in total which should bring draw calls down significantly while navigating around the globe.
“dynamic”, when you are dragging the time slider. geometries would then either have to be rearranged to “color buckets” dynamically (if it can be done fast enough) or there could be behavior as it currently is, each country being its own distinct object.
i agree, divisions are strange. when i started with this project there was data from hopkins university, which, at that time, would resolve some countries at regional level for US, Australis, China and Canada. Hopkins has since then come a long way regarding the data they provide. The CSV for January, 23, 2020 has 47 lines (!), today’s data has almost 4000 lines and holds substantially more data per line than it had at the beginning.
Italy started to publish their data at regional level to Github early, February 24, 2020. At the same times my colleagues at SynerGIS were putting up a dashboard for Austria. In the beginning, my focus was European / Regional, having the “NUTS 2” statistical divisions in mind. This is one the reasons why i.e. Ireland is split to regions that may seem awkward. The same applies to i.e. Switzerland or Germany, while other countries, like Austria have statistical divisions that are more aligned with the commonly known regions. As things progressed in South America, Russia, … i tried to integrate that data as well. Then India, Pakistan, …
Today there is a total of 42 data sources that are integrated into the map. Unfortunately this makes for an unhomogeneous quality in several places. Currently i am looking a Czech Republic and Poland to be next for integration, but its hard to find historic data for Poland. Czech Republic has quality data 7 days old and daily data, but nothing in between, so i am trying to find a solution for that.
I will try to explain Symbology, Colors, … in another post.
“Color Intensity” (actually hue and value) are determined by the 7-day average of newly confirmed cases. More color means more cases over the last 7 days (per 100000 population to keep things comparable). A light gamma function is applied to make small changes in the lower number range more visible.
“Height” is determined by the total case on a linear scale.
Lombardia, Italy (as of 23.09.20) is displayed with a high symbol, but almost no color, since the situation appears to be under control there.
Czech Republic (as of 23.09.20) does not have a high symbol, but is dark colored, since only recently numbers began to rise.
Roraima, Brazil is high and dark, due to an ongoing bad situation.
Urugay is flat and has no color, which is good.
Turkmenistan is flat and has no color, because no numbers are reported from there (there actually should be a separate color for that…)
