How To Store Reusable Assets On User Local Machine
While creating a preloader for WhyDungeons, I wanted to mimic what Jagex did with RuneScape 2 - storing most valuable assets locally for faster load times and smaller bandwidth usage. Opening the game for the first time took significantly more time than any consecutive run (mind it was 2004, internet wasn’t powerful.)
Based on Can I user GLTFLoader to load file from the local file system of the browser, floppy_disk Off-line first , Is there anyway I can cache models in the user’s browser cache? and a few others on SO - nobody seems to have given a definitive example of how to store and load assets from local hard drive.
So after going through all the answers and MDNs, here’s the complete solution on how to safely store images / models / audio / textures locally with browser JS (not tested on node / workers.)
Limits & Compatibility
(see sources at the very bottom, if you care about credibility :’) )
General Support table: here
How much can I store?
“The maximum browser storage space is dynamic — it is based on your hard drive size. The global limit is calculated as 50% of free disk space. In Firefox, an internal browser tool called the Quota Manager keeps track of how much disk space each origin is using up, and deletes data if necessary.” ²
Mind the 50% (for Chrome up to 80%³) drive limit is for the entire browser. If your data gets stale, user stops visiting etc - your data will be removed from any persistent storage and needs to be re-downloaded.
Assume you can safely store up to 500MB-1GB of data locally, unless you target IE users (during testing I memory-leaked ~5GB and it didn’t even ask for permission on Chrome. )
You can tell exactly using:
What happens if user does a hard-reload and clears cache?
Nothing - IndexedDB is independent and will not be cleared by a hard-reload.
Saved assets can be cleared using devtools, quota limits, or expiration.
What happens if my app exceeds local storage limits?
Nothing - you will just be unable to cache and proceed as usual.
Can I store only JSON / text data?
No⁴. IndexedDB does not care about data format - see all supported types here. (LocalForage uses WebSQL as a fallback - assume IndexedDB as a current standard, WebSQL is no longer supported³.)
Can I store cross-origin data?
No⁵. Data saved must be same origin and not “dirty” (ie. loaded via
<img> instead of XHR.)
Code Explanation (& How To Implement Yourself)
(Quick note - code above is not yet 100% compatible with Three.js, otherwise I’d already be fighting with @mrdoob about merging revamped
FileLoader to the main repo :’) IndexedDB is asynchronous, while ImageLoader.load requires a synchronous value returned. Also I didn’t test it enough to promise it will always work. )
To enable local cache, add the following changes:
1. Allow to override default Cache behaviour
You can either inject LocalForage directly into Cache object (Cache is shared globally, so it’s enough to add it once.) Or do as is done here and add a method that allows to override default Cache behaviour and add LocalForage support (adjust other methods of Cache to work with override too - complete diff.)
2. Add asynchronous Cache to core loaders
Three core loaders can take advantage of caching -
ImageBitmapLoader. To allow them to use async Cache, take the entire body of the
load methods and place them in
Cache.get callback (see here for example.)
Change all 3 loaders.
3. Enable Cache
Remember to enable Cache. You can now enjoy your locally stored assets
Speed & Performance Comparison
First, if loading speed is all you care about and you don’t serve 100s of assets - it’s best not to bother with caching. Internet is fast these days, and load times will not change much (~1s difference with asset stored locally.)
If you (1) serve a lot of models and textures or (2) want to offload your server / CDN - do bother with cache. After you enable local storage for assets, users will simply stop sending requests for these assets:
(Out of ~40MB of assets, only 1.5MB is fetched from the server - only scripts and fonts.)
When To Avoid
If your assets change often (and are not versioned nicely), once the user downloads the model they will stop receiving any updates of it.
Stale data is a common problem with caching, so be sure to either invalidate old assets, when necessary, or use versioned URLs - otherwise what your users see may become significantly different over time from what you see.