Firefox Extensions to use different storage type in Firefox 66

Martin Brinkmann
Feb 16, 2019
Firefox
|
23

Firefox 66 is the upcoming stable version of the web browser by Mozilla. The version is currently in the Beta channel and will be moved to Stable on March 19, 2019 according to the release schedule.

Firefox 66 incorporates several extensions related changes. One of them changes how extensions use local storage. Extensions in Firefox use JSON files for that currently: starting with Firefox 66, extensions will use IndexedDB instead.

Good news is that the migration happens automatically; developers don't need to change extension code and storage used by extensions already installed in pre-Firefox 66 versions will be migrated to the new storage format when the update happens as well.

extension storage firefox migrated

Note: It is recommended that you back up Firefox before you make the change as it is not clear right now whether it will be possible to downgrade the browser after the upgrade. If there is any chance for a downgrade, back up Firefox. Mozilla does not delete the old storage files but adds .migrated to the filenames. You find them in the browser-extension-date folder of the Firefox profile.

The change to IndexedDB is beneficial to performance and memory usage according to Mozilla.

This results in a significant performance improvement for many extensions, while simultaneously reducing the amount of memory that Firefox uses.

Extensions that store "small changes to large structures" benefit highly from the change; many ad-blockers do so according to Mozilla.

Mozilla's Luca Greco created a short video that demonstrates how beneficial the change is in regards to performance.

The video is silent, unfortunately, and it is a bit difficult to understand what is going on. Focus on the startup performance of the Firefox web browser. You will notice that the browser loads the test site a lot faster in Firefox 66 (with an ad-blocker installed) than previously.

Whether you will see an improvement depends largely on the installed extensions. If they do use local storage and make lots of smaller changes, you may see noticeable improvements.

Firefox 66 comes with another memory improvement related to extensions. Extensions that load objects from storage to memory use less memory than before thanks to an improvement implementation.

Firefox users who run Beta or Nightly versions of the web browser should see the benefits of the new implementation already.

Now You: What is your general take on Firefox performance?

Summary
Firefox Extensions to use different storage type in Firefox 66
Article Name
Firefox Extensions to use different storage type in Firefox 66
Description
Firefox 66 moves the storage type that extensions use from JSON files to IndexedDB databases to improve performance and memory use.
Author
Publisher
Ghacks Technology News
Logo
Advertisement

Tutorials & Tips


Previous Post: «
Next Post: «

Comments

  1. John Fenderson said on February 19, 2019 at 8:39 pm
    Reply

    “What is your general take on Firefox performance?”

    It’s fine. But I’m probably the wrong person to be answering this, as I thought it was fine before the improvements. That said, comparing the current Firefox to the one prior to Quantum, I don’t see a huge speed improvement. It’s a bit faster, but not enough to make any real difference.

  2. Thaumiel said on February 19, 2019 at 12:51 pm
    Reply

    Absolutely disgusting. Why am I still using it is because there’s no decent alternative, but once again they continue with their asinine choices for the sake of performance. I’d like to see the differences with a decent adblock such as ublock origin and not dumpster fires like ABP.

    I have a few extensions that use the extensions local storage to do their job and communicate with external programs, guess I’ll have to dump them all or finally take the ungoogled-chromium pill. Can’t wait for them to join manifest v3 together with google, too..

  3. FF 52 ESR User 4 Life said on February 18, 2019 at 7:36 pm
    Reply

    dudes, most newer versions of firefox close bugs that don’t touch me in any way.

  4. Jozsef said on February 18, 2019 at 12:09 pm
    Reply

    What a sad mess. They really want to kill Firefox with bad ideas justified by faulty logic and ignorance. Not much farther to go now as the percentage of their market share is single digit.

  5. supergirl said on February 18, 2019 at 3:24 am
    Reply

    WoW! I guess its back to an ESR for me….

    I was just updated to v65 & Im locked out of some preferences with this warning:

    “Your Organization has disabled the ability to change some preferences.”

    Im sole owner & user of this computer there is…. NO… ‘Organization.’

    Its on both my computers.

    Either the MX repo guys made a mistake or Mozilla chose this for me.

    Uncool…

  6. asd said on February 17, 2019 at 9:37 pm
    Reply

    I hope clearing “site preferences” & “offline website data” will not mean clearing extension data in the new version! that would be major privacy headache to deal with…

  7. Robert Ab said on February 17, 2019 at 7:15 am
    Reply

    And here we have another problem – caused by introduction of IndexedDB databases:
    https://www.reddit.com/r/firefox/comments/ar78wr/firefox_extensions_to_use_different_storage_type/egmdkul

    comment 1:

    > I really dislike the change.
    >
    > **The main problem is that user has lost any control over extension’s data.**
    >
    > * It’s no longer readable. It’s hidden inside ugly sqlite file and unreadable even with SQLit viewer
    > * **It no longer can be imported into new profile. Not unless the extension itself provides such opportunity.**
    > * The only option to access stored data is debugging extension – and from end user perspective this is not an acceptable solution
    >
    > While performance improvements are nice, it’s totally not worth it. I miss the days when extensions data could be accessed from about:config.

    comment 2:

    > Fully agreed. The first is bad enough but there are some solutions like accessing indexedDB via developer tools in browser. The second is just truly awful and will make restoring/backing up my firefox profile 10x more complicated. The internal UUID used changes with each firefox install so the extension settings can’t simply be copied between installs.
    >
    > Even chrome with it’s opaque storage format for extension settings is far better in this regard as it uses a static extension id for the extension storage folder.

    https://bugzilla.mozilla.org/show_bug.cgi?id=1429838#c6

    1. Tom Hawack said on February 17, 2019 at 9:15 pm
      Reply

      @Robert Ab,

      “The main problem is that user has lost any control over extension’s data”.

      Indeed several file formats are hard to read without a proper editor and hard to find sometimes within that editor.

      This may help some of us :

      What file formats do we have with Firefox? json, sqlite and mozlz4

      1.1- Firefox’s about:support includes Extension IDs
      1.2- Firefox’s about:debugging includes Extension IDs and Internal UUIDs

      With that you already have the relationship between esoteric names and extensions.

      2- Viewing/editing Firefox Databases :

      .json files :
      JSON Editor Online at https://jsoneditoronline.org/

      .sqlite files :
      Sqlite Browser Online at https://sqliteonline.com/

      mozlz4 files (.lz4, .jsonlz4, .jsonlz4-xxxx, .baklz4 etc) :
      mozlz4-edit Firefox extension at https://addons.mozilla.org/en-US/firefox/addon/mozlz4-edit/

      Editing is sometimes possible, for instance one’s storage-sync.sqlite with aboove mentioned ‘Sqlite Browser Online’ and cleaning left-overs, but editable or not peeking in those files is often most interesting. Always backup!

      So, we do have a possible control over our profiles’ content but it does require a minimum of work and in a way it is understandable given that excessively easy access to a browser’s undergrounds may lead to the worst if manipulated by users who don’t know the relationship between a match and dynamite : better to hide either of the two.

    2. Tony said on February 17, 2019 at 6:25 pm
      Reply

      That’s REALLY bad. That’s going to make useful backups impossible.

    3. Anonymous said on February 17, 2019 at 6:02 pm
      Reply

      “> **The main problem is that user has lost any control over extension’s data.**”
      “> * It’s no longer readable. It’s hidden inside ugly sqlite file and unreadable even with SQLit viewer”

      They should also compress all profile data with some obscure compression algorithm (especially the “strategic” search engines file). Oh, partially done already, thanks mozlz4. What’s next, encrypting everything ?

  8. Nodeity said on February 16, 2019 at 11:57 pm
    Reply

    How will this affect Apps like Mozbackup et al?

  9. SocialMediaGrandpa said on February 16, 2019 at 5:19 pm
    Reply

    That’s great, now just bring back all the extensions you savagely murdered and we’ll be good.
    with regards,
    slighty bitter in seattle

  10. Anonymous said on February 16, 2019 at 4:40 pm
    Reply

    I feel like it’s important to note that that’s not vanilla Adblock Plus in that video. The user configured it to have almost 50MB of lists to read at startup, which is why it takes such a long time when reading from a JSON file.

    Normal ABP/uBlock Origin/etc wouldn’t cause the browser to lag that much.

  11. Matt A. Tobin said on February 16, 2019 at 2:15 pm
    Reply

    As a matter of Mozilla history.. Profile structural changes are NOT EVER backwards compatible. Users have a history of arbitrarily switching between versions of Mozilla-ish applications but do not take into account that profile migration routines such as this storage change is totally one way.. Old to new..

    In a lot of cases, the older files are never cleaned up so it presents the appearance of working just fine.. However, data loss happens between the two formats. It is even more disastrous when the filename is the same but the schema has changed be it sqlite or json or indexdb.. Total or partial corruption can result.

    As you note, people using Mozilla-based, Mozilla-style, Mozilla-ish, and Mozilla proper software should backup their profile before ANY change in version and should almost NEVER downgrade without using a profile matching the target downgrade version.

    This of course has been made tons and tons worse by automatic upgrades with little to no notification until after the fact if all.

    Back up your data, kids!

  12. John C. said on February 16, 2019 at 12:02 pm
    Reply

    Firefox is certainly getting faster, but I sure wish they’d reconsider allowing end users to manually check for updates. Also, I wish that the tab bar location was optional. I hate having it at the top of the program window, want to be able to put it right above website content where (IMO) it belongs.

    1. Thorky said on February 16, 2019 at 5:59 pm
      Reply

      There is a workaround for the tab bar location:

      From Firefox 65, unfortunately, it is considerably more extensive to keep the tab bar below the address bar. So far (including the still current FX 64) the entry has been sufficient:

      #TabsToolbar {-moz-box-ordinal-group: 2; } # nav-bar {border-top-width: 0px! important; }

      in the userChrome.css in the chrome directory of the profile (C: \ Users \ [USERNAME] \ AppData \ Roaming \ Mozilla \ Firefox \ Profiles \ [PROFILENAME] \ chrome \ userChrome.css).

      In the meantime Github Aris-t2 has shown a solution that keeps the lower tab available. This requires the following two files:

      – github.com/Aris-t2/CustomCSSforFx/blob/master/classic/css/tabs/tabs_below_navigation_toolbar_fx65_v2.css
      – github.com/Aris-t2/CustomCSSforFx/blob/master/classic/css/tabs/tabs_below_navigation_toolbar.css

      Once Firefox 65 has been officially released and installed, this approach:
      1. Close the Firefox!
      2. If not already available, create the folder chrome in the Firefox profile (path: see above).
      3. Copy the tabs_below_navigation_toolbar_fx65_v2.css file to the Chrome folder.
      3.1 If no userChrome.css exists yet, simply rename the tabs_below_navigation_toolbar_fx65_v2.css in userChrome.css.
      3.2 If a userChrome.css already exists, open the tabs_below_navigation_toolbar_fx65_v2.css in the Notepad editor, mark the entire content (Ctrl A) in it and copy it into the main memory. Now open the userChrome.css in the editor, insert the contents of the main memory and save the userChrome.css.
      4. Copy the tabs_below_navigation_toolbar.css into the chrome-folder, too. The @import command of userChrome.css also loads the necessary information from this file into Firefox.
      5. Start Firefox and see if it worked. It worked for me right away!

      Since it can happen that Mozilla makes further changes, it is advisable to look at Aris-t2 for new versions from time to time, as soon as the tab bar is moved again.

  13. Tony said on February 16, 2019 at 11:08 am
    Reply

    My feelings on this are mixed: Better performance and less memory is good. Files that are no longer easily readable is bad.

  14. Thorky said on February 16, 2019 at 10:56 am
    Reply

    Will this have an effect to SpeedyFox?
    https://www.crystalidea.com/speedyfox

  15. flash said on February 16, 2019 at 10:36 am
    Reply

    Ha, this is quite funny. I remember reading a Mozilla design document on the topic of storage methods ( https://wiki.mozilla.org/Performance/Avoid_SQLite_In_Your_Next_Firefox_Feature ) that lists a million and one reasons why sqlite is a bad storage format and comes to the following conclusion: “JSON files or log files will show better I/O patterns almost every time”.

    This was written back in 2014, so I’m unsure what has changed since to suddenly make sqlite suitable and better. I don’t think it was sqlite development, because much of the criticism was about the concept of using a relational database for storage.

    1. Tony said on February 17, 2019 at 8:02 pm
      Reply

      I would like to hear Mozilla’s response to this.

      1. ShintoPlasm said on February 18, 2019 at 1:31 pm
        Reply

        Mozilla’s response: this change improves privacy, security and users’ safety, and we know it does because our telemetry tells us so. Also because we are open and privacy-focused and not-for-profit.

    2. Weilan said on February 17, 2019 at 9:59 am
      Reply

      They are desperately trying to copy everything Chrome does in order to stay relevant. Their decisions make no sense if you disregard that fact.

    3. Anonymous said on February 16, 2019 at 6:02 pm
      Reply

      Dont worry flash, in one year from now Mozilla will go back to JSON format again and will state SQLite is no longer as fast as JSON.

      I’ve lost count of how much mozilla changed the user profile folder structure and storage.

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

We love comments and welcome thoughtful and civilized discussion. Rudeness and personal attacks will not be tolerated. Please stay on-topic.
Please note that your comment may not appear immediately after you post it.