Firefox and IndexedDB from a Privacy perspective
The Firefox web browser like any other modern web browsers uses IndexedDB to store persistent data that is associated with the browsing profile.
A report on German computer magazine Heise suggests that Firefox's handling of the storage may impact user privacy on the Internet.
Websites may store IndexedDB data when a user connects to the site (and allows JavaScript execution). The process itself happens in the background; there is no user interaction or prompt.
While Firefox users have several tools at their disposal to manage the data, it has two deficiencies when it comes to the handling of IndexedDB data.
First, that the clearing of browsing data does not touch the data, and second, that Firefox users have little control when it comes to allowing or denying sites the right to save data in the first place.
Update: Mozilla plans to correct the issue that clearing the browsing data does not clear IndexedDB data with the release of Firefox 56.
Firefox and IndexedDB
Firefox users have two main options currently when it comes to IndexedDB data. They may use Page Info to clear the storage, or the Firefox Developer Tools.
A right-click on any web page and the selection of Page Info opens the configuration window. It highlights if the domain has saved data to the local system, and how much.
The clear storage button works, but it will only clear the data for that particular site. The options to set the process to "always ask" or "block" don't work properly however, and are reset automatically when Firefox is restarted.
The about:preferences#privacy setting "Tell you when a website asks to store data for offline use" does not work either" when it comes to this type of storage.
Page Info's permissions page has little use when it comes to managing local data, as it lists data only for the active domain.
The Firefox Developer Tools improve this slightly; the data that is stored in the database is listed by the browser's Developer Tools, but again only for the selected domain.
Press F12 to open the Developer Tools, and select Storage when the interface opens. If you don't see storage, click on settings and enable storage there first. You can delete entries individually there, or all at once.
The best option right now to find out which sites use the offline storage is the following one:
- Type about:support in the Firefox address bar.
- Click on the "open folder" link to open the Firefox profile folder on the local system.
- Go to storage\default\
You can delete some or all of the folders there to clear the storage.
Firefox has an option to disable IndexedDB completely. Doing so may cause incompatibility issues with some websites.
- Load about:config?filter=dom.indexedDB.enabled in the browser's address bar.
- Double-click on the name dom.indexedDB.enabled to toggle its value.
A value of true means that IndexedDB is enabled, a value of false that it is turned off.
Heise notes that the issue was first reported eight years ago to Mozilla.
Firefox 57 will improve the manageability of site data. It features a new Site Data entry under about:preferences#privacy which you may use to clear all data, and to manage data from sites that used the feature in the past.
This improves the management of persistent storage in Firefox, but it does not address the issue that site data is not deleted when the Firefox browsing history is deleted, nor that the permission system seems broken when it comes to persistent data.
Thanks a lot, this article is extremely useful !
I can confirm that as of Friday 9.22.17 that Nightly v58 has the ability to delete the indexedDB website folders. I had to go to: Options/Privacy/History/Clear history when Nightly closes – check box/Settings…/Offline Website Data – check box. I’ve had those boxes checked forever and they finally do what they are supposed to. Coming soon to a browser near you! ;)
This issue is finally fixed for FF57+ – see https://bugzilla.mozilla.org/show_bug.cgi?id=1333050.
And this one at the same time – pushed to 56+ even!
https://bugzilla.mozilla.org/show_bug.cgi?id=1047098
Another option to avoid the problem is to using private browsing. I just tested it by deleting all the storage folders as per Martin’s third screenshot and then went to the same sites in a private window. The storage/default folder remained empty afterwards. Doing the same thing again in normal mode had them all reappear again.
Self-Destructing Cookies clears local storage along with cookies when a site’s tabs are closed: https://addons.mozilla.org/en-US/firefox/addon/self-destructing-cookies/?src=ss It also has the option to clear the cache on a timed basis. I have that set to one minute.
Sadly though it won’t make it to a WebExtension and according to the Add-ons/Options page it’s incompatible with FF55+.
By the way Martin, there’s no option to clear local storage as shown in your first screenshot on Firefox ESR.
I just looked at the storage / default folder in my FF profile and regretably note that there’s quite a few sites listed there which I will duly dispose of in the minute.
Clarification. For a minute there I lost sight of the fact that indexedDB website folders are removed by CCleaner when deleting cookies, for most browsers, not for Chrome Dev or Waterfox which are not cleaned by CCleaner.
unknown territory for me till now, so thank you so much for this article and all the comments. checked out the permanent folder as well and discovered chrome in both FF and waterfox. anybody got an explanation? ( I deleted both. hopefully I did not cause trouble. fingers crossed )
I also checked the temporary file and found facebook stored in waterfox.
“In a web browser, the chrome includes the URL field, the browser toolbars, the browser buttons, the tabs, scrollbars, and status fields.
On a website, the chrome includes navigation bars, footers, logos, branding, the search box, and so forth.” – Jakob Nielsen
One of many tech quotes I’ve saved from different people. ;)
Oh, I didn’t know websites had a notion of chrome too
Firefox innards use IndexedDB. You don’t need to delete it, but it’s okay if you do, it will be restored next time.
Firefox chrome, Dev Tools and about:home use it. (Google Chrome’s name come from that, not the other way around, the chrome is a part of Firefox)
thank you. good to know
One more good reason to remove Firefox from the list of SW to be used. With greetings to the company that stresses “Just trust me about privacy!”.
Confirmation bias
I had a quick peak in the storage/default folder. The most concerning entry was https+++riot.im^privateBrowsingId=1. Does Firefox store data from private browsing mode sessions like this?
I always wanted to disable indexedDB, but the problem is if you disable it GMail and some other sites don’t work properly. or maybe it’s just me…
and what do you recommend, using another browser?
>Firefox has an option to disable IndexedDB completely. Doing so may cause incompatibility issues with some websites.
I’m pretty sure, long time ago, one of popular java script libraries introduced broken feature detection function, that crashes/throws exception when IndexedDB was disabled. Because of this, lot of pages were broken, even if they not used IndexedDB.
If you have a WebExtension that needs IndexedDB, it doesn’t work at all since Firefox blocks IndexedDB usage in private browsing mode even for WebExtensions (background page).
As rude as it sounds I’m glad I’m not the only one that has been fighting this. After I don’t know how long of cleaning the contents in the ‘storage’ folder manually I finally added it to the “Advanced/Custom Files and Folders” in CCleaner. Make sure when adding the location that you include files AND subfolders.
https://s26.postimg.org/h7wc0m7d5/CCleaner_cleaning_storage_default_folder.png
I too have done this. Disabling IndexedDB breaks some websites, so the next best thing is to periodically clean them out. Not ideal but better than leaving it there untouched.
I personally have had indexedDB disabled for years (same in the ghacks user.js since it started). Unfortunately, it now needs to be enabled for uBlock Origin and uMatrix (web extensions), and it also causes problems with firefox internals, namely startup caches (look in your console). Stylus also uses indexedDB, as will many extensions. So recently it was made inactive in the user.js – see https://github.com/ghacksuserjs/ghacks-user.js/issues/226 – which highlights the pros/cons and also mentions how you can allow IDB but block permissions to the folder (but see next point re moz-extension, you would want to allow read/write/create to those)
Long term, It would be nice if moz-extension and internals were exempted. Meanwhile, after over a month of having indexedDB enabled, I have yet to see a single IDB entry (note: I have service workers disabled and this seems to block a lot of it, but not all – depends on the website – eg youtube will not create IDB if dom.serviceWorkers.enabled = false: see https://github.com/ghacksuserjs/ghacks-user.js/issues/234 )
Note: So … be careful what you remove in that default folder eg: moz-extension+++`id-string` > **idb** > .sqlite file & folder with various numbered items
@Martin : if i am logged in, none of my comments get posted. It’s been days
>long term, It would be nice if moz-extension and internals were exempted.
Cant remember the bug report id, but maybe they’ll whitelist moz-extension, yes.
FYI, IndexedDB is totally disabled in Private Browsing. Its breaking many Webextensions.
@Pants
In the ghacks-user.js I noticed “dom.storageManager.enabled” and “browser.storageManager.enabled” are both recommended as false which is the default for FF v55 but in Nightly the default for both is true. Anyway, when “browser.storageManager.enabled” is set to true then “Site Data” becomes visible in “Options/Advanced/Network” in FF v55 and at “Options/Privacy & Security” for Nightly. You probably know this already and I’m playing catch up. I just thought it was weird that “Site Data” wasn’t included in the “OfflineWeb Content and User Data”.
Try bostonglobe dot com. 48KB of data in its dedicated storage subfolder.
@Pants
Thank You for the input! I was going to ask for some input before adding the storage/default folder to CCleaner and decided to live dangerously. Like an idiot I had forgotten about the moz-extension folders in Nightly. In FF and Waterfox I don’t have any moz-extension folders in the storage/default folder. I’m still using uBO Legacy v1.13.8 and I’m still using Stylish for the UI mod-ability. My 4 webextensions don’t put a folder in storage/default. Nightly is a different story. it has 10 webextensions and 4 moz-extension folders, I can only figure out what 2 of them are, uBO and Stylus. I wish the about:support extension ID matched the moz-ext folder number. Anyway, deleting the uBO folder will cause the subscription lists to need an update, My Filters and My Rules survive. Stylus was basically wiped of all styles. Whoops! At least I had a backup! Boom Shaka Laka! LMAO
Even though I have:
dom.caches.enabled=false, dom.serviceWorkers.enabled=false, I still get folders from youtube, weather .com and a couple others. I changed the inclusion rules in CCleaner to storage\default\https+++www.youtube.com\ (included files, subfolders and folder itself) for now and will keep an eye on what else is added to the folder. SMH
Testing 123
Disabling indexedDB is not an good option, some extensions und site functions are depending on that. As said it get fixed anyway in the next final Mozilla release. There is (after 8 years) no reason to change this, just wait till it gets now more attention and Mozilla will fix it.
Try again please :)
Well well, a certain messenger (I use an app for it anyway) had around 30 MB there, and now my Firefox starts way faster. Thanks!
I second the question about doing the same in Chrome, Opera, Vivaldi etc.
Edit: it was in /home/(my name)/.config/google-chrome/Default
For Windows it should be: C:\Users\{UserName}\AppData\Local\Google\Chrome\User Data\Default\IndexedDB
That’s one of the reasons I always like to refresh my profile (or even create a new one) after each major release.
The Firefox add-on “eCleaner (Forget Button)” will wipe indexedDB databases, and it’s compatible with Firefox 57.
https://add0n.com/ecleaner.html
How does this work in Chrome and IE, ie deleting the indexedDB data.?
In incognito mode it gets deleted. About the other browsers, they only have partial support, e.g. Chrome, it works a bit different here. IE only has the indexedDB since v11 if I’m not wrong and it gets cleaned too in incognito mode.
It basically only affects Firefox.
The “Permissions” section of “Page Info” allows camera/microphone use to be blocked. That’s good! However, if the OS settings have the camera/microphone turned on, and Firefox has the settings turned off, which takes precedence? Is Firefox linked to the OS settings?
Firefox has less to do with the settings the OS uses, which means the OS only controls Windows Store related settings. All external software aren’t controlled by these settings.
This is misleading. Turning off the camera/microphone on OS level will make Firefox unable to detect your camera/microphone at all. OS level is greater than software level. Also, he asked about OS not ‘Windows’.
8 years old bug, which gets fixed with FF 58.
Hello CHEF-KOCH, Do you happen to know the Firefox Bug number? (Curious about all the dirty details.) Many thanks!
Fortunately it has never been much abused. Previously any use of IndexedDB required a prompt, so it couldn’t go unnoticed, and it basically never happened. Nowadays a prompt occurs when a site asks for more than 50Mb, and it happens a little.
There are several points that limit the issue at hand:
– IndexedDB cannot be used from a third-party website, only first party. That protects against tracking across the web
– IndexedDB requires JavaScript to read or write anything
– IndexedDB is seldom used. Most usages are actually by Firefox’s JavaScript engine which stores a compiled asm.js file in IDB for sites who use asm.js. This is not data, it’s kind of similar to caching images and CSS. Facebook does that, probably several of Martin’s screenshot too.
For all these reasons, I didn’t even bother to disable IndexedDB even though privacy is my first concern. I check and clear my profiles folders every now and then, rarely though. I’m still happy that it gets fixed in Firefox 58.
> It’s interesting to know. Sounds like that means service workers don’t require a prompt unlike Web notifications, all the more reason to disable them. :/
Its a scary world out there. Speaking of prompts, there is also a ticket somewhere about OWD not prompting when it is meant to (and not sure about the 50mb applying anymore – and u can change this size in prefs) – so IDK if that will be covered in the whole new about:permissions & storage api etc – there are around 20 tickets to do with IDB I’ve been watching for the last 2 weeks – most closed with duplicate, but still some shady sh*t going on with the whole thing – including cookies + IDB (not about 3rd party, which I knew about – it says this in the ghacks user.js)
under 2701 with the network.cookie.cookieBehavior setting
> * [NOTE] This also controls access to 3rd party Web Storage, IndexedDB, Cache API and Service Worker Cache
Next up: OA’s and mass confusions and permissions – FPI, PB, ContainerIDs – it sure is gonna be a bumpy ride, but fixes and overhauls for the better
According to you down the thread it’s even fixed in Firefox 56, yay!
IDB becomes a complete non-issue for me in five days then. And in Firefox 57, WebExtensions should even be able to control it like they can regular cookies, still according to a post down the thread.
Woooah, sorry, I did double check and it still appears that I got it wrong.
IndexedDB USED to only be accessible to first party, but not since Firefox 43 (https://bugzilla.mozilla.org/show_bug.cgi?id=1147821). Now it obeys the third-party cookies preference, which makes more sense but does make IDB more interesting from a tracking point of view in Firefox versions under 58. (According to Chef Koch)
However as you tested with the vanilla profile, it doesn’t seem to be used by anything but first party sites. So technically, nothing changed since Firefox 42. For some reason that are not as clear any more, IDB doesn’t seem to be much used as a tracking means…
> Factors like service workers disabled block 4 of those right out of the block.
It’s interesting to know. Sounds like that means service workers don’t require a prompt unlike Web notifications, all the more reason to disable them. :/
> Example from several years ago is FB
Ah yeah offline sharing of data, terrible shit, can’t do anything against it besides not giving data to first parties, something that can be very difficult depending on your social circles…
> just checked what FPI covers, and they don’t apply origin attributes to IDB.
Correction – FPI does cover this (I just didn’t see it worded in the relevant bugzilla titles). If you enable FPI and visit a site that creates IDB entries, you will see they have ^firstPartyDomain in them. So why would this be done if IDB is first party only? Just asking :)
> list of sites
That’s just a bunch of sites I tested in a nilla profile. Factors like service workers disabled block 4 of those right out of the block. Things like XSS and other factors (JS) will play a part etc – think uBo/uMatrix default deny or hard-mode etc. Personally, as I said, I never get anything from anyone in my main tweaked FF (although its only been about 4 weeks).
> Yes, but you have to activate JavaScript
Am talking worst case scenarios here :)
> tracking aggregators
companies that share data – so not client side, not even server side – but evil human side. Too much of this going on. Example from several years ago is FB and bank/credit card data – so not even website tracking, but spending habits. But what I meant was website tracking data. We’re all f**ked :) Well, most of you .. I’ll be fine xD (don’t have any money anyway)
I disabled service workers long ago for privacy reasons. (Might reconsider. Are they behind a prompt like WebNotifications ? Can’t remember)
> It only protects against 3RD PARTY, not 1st party. Also, IDB allows spawning of supercookies
Yes, but you have to activate JavaScript for IDB to be readable or writeable on a given first party site. In that case you are uniquely fingerprinted anyway, IDB or not, and supercookies can be restored from there. They don’t really need to though, the data can be kept server side and associated with the fingerprint. Is the value brought by IndexedDB thanks to the difficulty there is (till FF 58) to clear it THAT significant considering the conditions that must already be met for IDB to be usable (JavaScript ON + First party only) ?
To each their own, but as for me I kept the feature activated and I’ve been waiting for the day where it can be turned on/off per site and cleared all at once, like other types of storage. I keep an eye on how frequent the use of this feature is, and it’s extremely rare with JavaScript off by default. Facebook is the one I have to clear, but there again its tracking interest seems dubious for them considering the more powerful tools they have at their disposal (e.g. track user keyboard and mouse patterns, meaning user fingerprint, not just device fingerprint).
> tracking aggregators
As first party you mean ? What’s that ?
> (if you want to test things):
Thanks for the list. I went to all those sites, enabled JavaScript and cookies, and it appears that only two of them store any IndexedDB for me. I didn’t log into anything though.
First was The Guardian. All it did was store one bit in a database named “test”, maybe it adds more if you are logged in or something. I know a couple news sites do that, surprisingly very few though, in my experience, considering how terribly they behave (loading dozens of third party crap that load further crap in turn). I understand it as a nice hint that IDB isn’t too useful for tracking.
Second was WhatsApp, which uses WebAssembly and therefore has a legitimate use to IDB. It also does store informations, but you kind of expect it with a web application so it’s a special case IMO. I wonder what’s the fingerprinting potential of WebAssembly.
I don’t have a Twitter account but I do visit the site regularly and haven’t noticed IDB for maybe a few years. At some point they were using ASM.js I think, so the code compiled by Firefox was stored in Twitter’s IDB.
It surprised me that you have IndexedDB data on YouTube though. Maybe service workers make use of IDB more frequently ? That sounds like the appropriate way for them to store data at least.
> IndexedDB cannot be used from a third-party website, only first party.
I didn’t know that, thanks – just checked what FPI covers, and they don’t apply origin attributes to IDB. Makes sense now.
>That protects against tracking across the web
It only protects against 3RD PARTY, not 1st party (and tracking aggregators). Also, IDB allows spawning of supercookies – an example in one of the numerous bugzillas is weather,com from memory.
Some sites that use IDB (if you want to test things):
https://www.youtube.com/watch?v=3tmd-ClpJxA <– TayTay .. u know u want to :)
http://www.huffingtonpost.com/section/politics
https://www.theguardian.com/international
https://www.washingtonpost.com/
https://twitter.com/i/moments
pinterest.com << redirects to localised domain
https://www.dropbox.com/
https://web.whatsapp.com/
PS: also service workers are evil sons of bitches – see youtube + service workers