RoboHornet: independent browser benchmark

Martin Brinkmann
Sep 25, 2012
Firefox, Google Chrome, Internet Explorer, Opera
|
15

The majority of web browser benchmarks are created by companies and organizations that develop browsers. Mozilla has its Kraken benchmark, Google its Octane benchmark, which many web developers believe are not as impartial as they should be. To make matters worse, third party benchmarks such as the SunSpider JavaScript benchmark are only benchmarking a browser's JavaScript performance and not other important browser technologies and features that matter.

The new independent browser benchmark RoboHornet tries to resolve all those issues. Committee members include developers and representatives of companies such as Google, Mozilla, Microsoft or Facebook. That's not the only difference to current browser benchmarks. Unlike others, it takes changes in technology use into account. The weighting of the benchmark changes over time because of this, with the consequence that you can't necessarily compare past benchmark results to recent results.

The benchmark runs a series of test that take minutes to complete. You can customize the test if you want by unchecking certain parts of the test before you click on the run button to start it. The benchmark spawns a popup window which all web browsers will block by default. You should see a notification on the screen that provides you with an option to allow popups on the site. Opera users need to change the popup behavior in the settings before the test runs through without issues.

The test result shows how the browser compares to stable browsers on average hardware. If the score is above 100, it is faster than average browsers, a score below 100 on the other hand indicates that it is slower than the average.

robohornet browser benchmark

How do Firefox 18 Nightly, Internet Explorer 9, Google Chrome 23 Dev and Opera 12.02 on Windows 7, and Firefox 18 Nightly, Internet Explorer 10, Google Chrome 23 Dev and Opera 12.02 on Windows 8 fare in the test? Lets find out:

Windows 7

  1. Firefox 18 Nightly: 145.61
  2. Google Chrome 23 Dev: 142.93
  3. Internet Explorer 9: 128.01
  4. Opera 12.0.2: 122.01

Windows 8

  1. Internet Explorer 10: 221.25
  2. Firefox 18 Nightly: 157.57
  3. Opera 12.0.2: 151.84
  4. Google Chrome 23 Dev: 141.51

robohornet internet explorer 10

Internet Explorer 10's performance may come as a surprise to many. What's even more puzzling is Chrome's performance in the test. Especially the performance on Windows 8, where it is performing worst of the four tested browsers seems to contradict the public perception that the browser is the fastest when it comes to benchmarks. It took Chrome considerable longer to complete the SVG resize part of the benchmark, which may explain why its score is what it is right now. It would be interesting to see if this is related to Chrome Dev, my computer, or Chrome in general.

Have you run the benchmark yet? If you did, which browser came up first?

Advertisement

Tutorials & Tips


Previous Post: «
Next Post: «

Comments

  1. ilev said on September 26, 2012 at 7:10 am
    Reply

    Windows 8 browsers results doesn’t surprise at all. It is known, as Google, Mozilla, VLC… and others , stated, that Microsoft is blocking third-party access to some Windows 8/RT
    APIs. The DOJ has started an investigation into Microsoft’s back-to-its-anti-compatitive-behavior, as does the EU.

    1. netsailor said on October 18, 2012 at 4:29 pm
      Reply

      so, your point is than the other browsers are more slower because they can´t use IE code, is like say my Prius is slower than your gto because i dont have your motor, that funny :)

      1. taigaak said on November 4, 2012 at 11:55 am
        Reply

        Do you even know what APIs are?

  2. Pacman said on September 25, 2012 at 11:19 pm
    Reply

    @KoalaBear
    ” It’s led by Google, it’s not from Google. It’s a team of multi-vendor developers / people.”
    At this point its basically just Google though. The “multi-vendor” part is basically political posturing to make people feel good right now. You can look through the commit history of the project. No one else has actually contributed (or at least, no one had yesterday when it was revealed).

    1. KoalaBear said on September 25, 2012 at 11:20 pm
      Reply

      Ok. Probably it’s only the picture of the outside they tried to make better. Then it’s still funny that IE 10 is the fastest. :)

  3. KoalaBear said on September 25, 2012 at 10:24 pm
    Reply

    @ilev
    It’s led by Google, it’s not from Google. It’s a team of multi-vendor developers / people.

    Who said it was representable for real-life websites? It’s representable for “Web 2.0” websites. So more like webapplications than ‘websites’.

    And I don’t like Internet Explorer, and will never do. Chrome looks more polished and feels better. Also fully integrated with the rest of my online life.

  4. Firefox fanboy said on September 25, 2012 at 10:13 pm
    Reply

    Firefox rocks xD ^^

  5. ilev said on September 25, 2012 at 9:41 pm
    Reply

    Microsoft doesn’t like the new Google’s tool (yes, it is Google’s benchmark tool) and
    even build a site of its own.

    Internet Explorer 10 May Score Well with RoboHornet, but It’s Even Better for Web Browsing in the Real World :

    Yesterday Google released its latest micro-benchmark, RoboHornet, in which Internet Explorer 10 scores rather well. While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn’t all that representative of the performance users might encounter on real-world sites. Like all micro-benchmarks, RoboHornet is a lab test that only focuses on specific aspects of browser performance….

    http://windowsteamblog.com/ie/b/ie/archive/2012/09/25/internet-explorer-10-may-score-well-with-robohornet-but-it-s-even-better-for-web-browsing-in-the-real-world.aspx

  6. virtualguy said on September 25, 2012 at 8:56 pm
    Reply

    The compelling reason would be to run the benchmark. In the time it took to write that worthless response, you could have enabled pop-ups. The benchmark can run in the background. Don’t bother responding. I know how busy you are. ;-)

    I had to disable the SVG resize test in both Firefox (32-bit) and Waterfox (64-bit) to get the benchmark to complete. All else being equal, FF = 130, Wf = 125. I guess the 64-bit advantage is only apparent in a video-centric benchmark.

  7. Morely Dotes said on September 25, 2012 at 6:41 pm
    Reply

    Since the benchmark relies on pop-ups, I don’t have time to run it. I keep pop-ups blocked and see no compelling reason to change that.

  8. Zlip said on September 25, 2012 at 3:27 pm
    Reply

    Its not independent benchmark, it is backed by Google Chrome team.
    http://paulirish.com/2012/a-browser-benchmark-that-has-your-back-robohornet/

    So I think you should probably change the title of article.

    1. Martin Brinkmann said on September 25, 2012 at 3:37 pm
      Reply

      As far as I understand, the panel votes what comes in and what does not?

  9. KoalaBear said on September 25, 2012 at 1:19 pm
    Reply

    I think you are right on the SVG resize problem in Chrome. I’ve runned this test yesterday evening, but it seems to get stuck on the SVG resize part.

    Will run it again if they fixed this issue, at least looks like an issue to me.

    1. Martin Brinkmann said on September 25, 2012 at 1:31 pm
      Reply

      It looked to me as if the test ran multiple times in the Chrome browser.

Leave a Reply

Check the box to consent to your data being stored in line with the guidelines set out in our privacy policy

We love comments and welcome thoughtful and civilized discussion. Rudeness and personal attacks will not be tolerated. Please stay on-topic.
Please note that your comment may not appear immediately after you post it.