Mozilla creates Shield study rules to avoid another Mr.Robot disaster - gHacks Tech News

Mozilla creates Shield study rules to avoid another Mr.Robot disaster

Mozilla created a set of guiding principles for Shield studies after it launched an analysis of the Looking Glass Shield study which went wrong on several levels.

Looking Glass was released as a system add-on to Firefox which meant that users saw the add-on appear in the browser's add-on manager without them initiating the installation.

This appearance out of thin air was arguably the biggest complaint that users had and something that got them alarmed because it shared the characteristics of malware. The fact that the initial description and add-on name did not reveal anything about the add-on's purpose added to the confusion as well.

The description of the add-on read "MY REALITY IS JUST DIFFERENT THAN YOURS" and "PUG Experience Group" was listed as the creator. Nothing linked the add-on to Mozilla in Firefox's add-on manager.

looking glass firefox

Mozilla announced shortly after the study blew up in the company's face that it had pulled the study, uploaded the add-on to Mozilla's Add-on repository, and started an investigation to "better understand how and why this happened", and how the company "could do better" in the future.

Shield studies are used to A-B test things in the Firefox web browser. The studies are used to test small and big changes to Firefox, from simple icon or color changes to new features, to find out whether the majority of users that have been selected for the study like the changes, or not.

Studies help Mozilla make better decisions in regards to future changes and features in the Firefox browser.

This platform helps us make decisions on new product features, evaluate whether or not a technology update is stable, and generally helps us make sure that we can make good decisions in a responsible way.

Looking Glass did pass the privacy review as it did not collect any data. The fact that it did not should have been a red flag, as it is impossible to evaluate a feature without collecting a bare minimum of data.

Mozilla create a set of principles for Shield studies so that something like Looking Glass won't happen again.

  1. All Shield studies must answer specific questions.
  2. Shield studies will always respect user privacy.
  3. All Shield studies adhere to the "scientific method for answering complex questions".
  4. All Shield studies require a Product Hypothesis Doc which outlines the research question the study is trying to answer.
  5. All Shield studies must be named accurately.

Looking Glass would fail in all but the second.

Closing Words

It is definitely a good thing that Mozilla created a set of guiding principles for Shield Studies. I would have preferred if the organization revealed a bit more about Looking Glass itself: was Mozilla paid for the promotion and how did not anyone object to the study by pointing out the, rather obvious, issues it had?

Related articles

Summary
Mozilla creates Shield study rules to avoid another Mr.Robot disaster
Article Name
Mozilla creates Shield study rules to avoid another Mr.Robot disaster
Description
Mozilla created a set of guiding principles for Shield studies after it launched an analysis of the Looking Glass Shield study which went wrong on several levels.
Author
Publisher
Ghacks Technology News
Logo




  • We need your help

    Advertising revenue is falling fast across the Internet, and independently-run sites like Ghacks are hit hardest by it. The advertising model in its current form is coming to an end, and we have to find other ways to continue operating this site.

    We are committed to keeping our content free and independent, which means no paywalls, no sponsored posts, no annoying ad formats (video ads) or subscription fees.

    If you like our content, and would like to help, please consider making a contribution:

    Comments

    1. Tom Hawack said on January 31, 2018 at 4:50 pm
      Reply

      No system add-ons here, the [Mozilla Firefox install folder]\browser\features folder is emptied on every Firefox install, systematically. No experience, no study, I’m a browser user, not a tester.

      1. Paat said on January 31, 2018 at 5:00 pm
        Reply

        You’re a very naughty browser user, Mozilla doesn’t like that. I can hear them frowning in disapproval. Don’t tell them what features you want added to Firefox, because they’ll mark them as WONTFIX just to spite you.

        1. Tom Hawack said on January 31, 2018 at 5:48 pm
          Reply

          I humbly beg their pardon as I finish coffee and sandwich.
          Users’ reactions concerning system add-ons must make some devs frown more than once :=)

      2. Sockpuppet said on January 31, 2018 at 5:23 pm
        Reply

        Some of the features in that folder are useful though, such as the Web compat one which fixes website incompatibilities.

        People may enjoy Screenshots and Form autofill, or maybe Activity Stream. I disable the last two in about:config. The only system add-on I delete is Follow on search since I haven’t verified in the source code that it respects the main telemetry switch and I’m not aware of an about:config pref that disables it either.

      3. Stephan said on February 1, 2018 at 12:31 pm
        Reply

        I have prior experience writing helpers which use the Linux inotify interface to act on things like “write-mode file handle closed” or “temporary file renamed to its final name”, so I’ll probably automate that.

        Some nice little tool to filter out the ones I don’t need and don’t want as the updater pulls them in. It’ll make a nice complement to pinning the about:config keys and translating my Classic Theme Restorer tweaks into userChrome.css rules.

    2. Anonymous said on January 31, 2018 at 5:09 pm
      Reply

      I think this Looking Glass thing was not a study, but a game. It was advertising Firefox to Mr. Robot fans, not the other way around, as the add-on was completely inactive unless the user did something particular as prompted by one of Mr. Robot’s episodes.

      As such, if anything it’s Mozilla who paid Mr. Robot’s I.P. owner rather than the other way around.

      1. Bruno said on January 31, 2018 at 8:57 pm
        Reply

        What? Advertising Firefox in Firefox? No, it’s not the first time Mozilla tried to “diversify their income sources”. Think of the first Pocket integration or Hello sponsored by O2.

    3. Dean said on January 31, 2018 at 5:10 pm
      Reply

      I think this Looking Glass thing was not a study, but a game that just happened to be distributed through the Shield Study mechanism, which was not made for that purpose. It was advertising Firefox to Mr. Robot fans, not the other way around, as the add-on was completely inactive unless the user did something particular as prompted by one of Mr. Robot’s episodes.

      As such, if anything it’s Mozilla who should have paid Mr. Robot’s I.P. owner rather than the other way around.

      @People: Untick all checkboxes at about:preferences#privacy-reports and you’re opted out of these.

      To protect from bugs where the main prefs would fail to be respected, you could additionally set these prefs in about:config.

      app.shield.optoutstudies.enabled = false
      extensions.shield-recipe-client.api_url = “” (empty string)
      extensions.shield-recipe-client.enabled = false

      1. Tom Hawack said on January 31, 2018 at 5:44 pm
        Reply

        Firefox 58.0.1 : I do have the last two settings in about:config (already emptied/disabled) but not app.shield.optoutstudies.enabled : maybe a hidden one…

        1. Pants said on January 31, 2018 at 6:55 pm
          Reply
        2. Tom Hawack said on January 31, 2018 at 7:29 pm
          Reply

          Lines 37/41 acknowledged, thanks Pants :=)

        3. Dean said on January 31, 2018 at 7:42 pm
          Reply

          As shown by Pants for the app.shield one, these are second line prefs, meaning that if the main pref(s) they are under is disabled, it doesn’t matter what they’re set to.

          It’s not always a given that they should be disabled but in that case, since a lot of work is currently ongoing regarding how telemetry is organized, I’d rather be protected from bugs where the main pref is erroneously bypassed. If second line (more granular) prefs are disabled as well such a bug doesn’t matter :)

    4. John Fenderson said on January 31, 2018 at 5:27 pm
      Reply

      They had a fairly strict policy and set of rules before. Those rule were apparently ignored. Does making a new set of rules decrease the odds that they’ll be ignored again?

      1. Dean said on January 31, 2018 at 7:54 pm
        Reply

        Same as when a recurrent type of security vulnerability is discovered: It prompts involved people into adopting better development practises leading to less mistakes, and if it’s bad enough it prompts them into implementing an architecture that doesn’t just rely on people following guidelines.

        1. John Fenderson said on January 31, 2018 at 9:00 pm
          Reply

          I hope you’re right. The Mr. Robot thing didn’t exactly look like a mistake, but more of an intentional subversion of the shield study program.

    5. ZZ said on January 31, 2018 at 6:39 pm
      Reply

      won’t install; can’t trust them.

      1. Anonymous said on January 31, 2018 at 11:33 pm
        Reply

        It’s pretty easy to check that such a small add-on doesn’t leak data. If you don’t trust them you shouldn’t use Firefox or any of its forks, and probably you shouldn’t use Chromium and any of its forks either.

    6. Tree said on February 1, 2018 at 4:36 pm
      Reply

      They are unethical. I deleted Mozilla Firefox from my system when they deleted Brendan Eich from their payroll.

    Leave a Reply