RoboHornet: independent browser benchmark
The majority of web browser benchmarks are created by companies and organizations that develop browsers. Mozilla has its Kraken benchmark, Google its Octane benchmark, which many web developers believe are not as impartial as they should be. To make matters worse, third party benchmarks such as the SunSpider JavaScript benchmark are only benchmarking a browser's JavaScript performance and not other important browser technologies and features that matter.
The new independent browser benchmark RoboHornet tries to resolve all those issues. Committee members include developers and representatives of companies such as Google, Mozilla, Microsoft or Facebook. That's not the only difference to current browser benchmarks. Unlike others, it takes changes in technology use into account. The weighting of the benchmark changes over time because of this, with the consequence that you can't necessarily compare past benchmark results to recent results.
The benchmark runs a series of test that take minutes to complete. You can customize the test if you want by unchecking certain parts of the test before you click on the run button to start it. The benchmark spawns a popup window which all web browsers will block by default. You should see a notification on the screen that provides you with an option to allow popups on the site. Opera users need to change the popup behavior in the settings before the test runs through without issues.
The test result shows how the browser compares to stable browsers on average hardware. If the score is above 100, it is faster than average browsers, a score below 100 on the other hand indicates that it is slower than the average.
How do Firefox 18 Nightly, Internet Explorer 9, Google Chrome 23 Dev and Opera 12.02 on Windows 7, and Firefox 18 Nightly, Internet Explorer 10, Google Chrome 23 Dev and Opera 12.02 on Windows 8 fare in the test? Lets find out:
Windows 7
- Firefox 18 Nightly: 145.61
- Google Chrome 23 Dev: 142.93
- Internet Explorer 9: 128.01
- Opera 12.0.2: 122.01
Windows 8
- Internet Explorer 10: 221.25
- Firefox 18 Nightly: 157.57
- Opera 12.0.2: 151.84
- Google Chrome 23 Dev: 141.51
Internet Explorer 10's performance may come as a surprise to many. What's even more puzzling is Chrome's performance in the test. Especially the performance on Windows 8, where it is performing worst of the four tested browsers seems to contradict the public perception that the browser is the fastest when it comes to benchmarks. It took Chrome considerable longer to complete the SVG resize part of the benchmark, which may explain why its score is what it is right now. It would be interesting to see if this is related to Chrome Dev, my computer, or Chrome in general.
Have you run the benchmark yet? If you did, which browser came up first?
Advertisement
Windows 8 browsers results doesn’t surprise at all. It is known, as Google, Mozilla, VLC… and others , stated, that Microsoft is blocking third-party access to some Windows 8/RT
APIs. The DOJ has started an investigation into Microsoft’s back-to-its-anti-compatitive-behavior, as does the EU.
so, your point is than the other browsers are more slower because they can´t use IE code, is like say my Prius is slower than your gto because i dont have your motor, that funny :)
Do you even know what APIs are?
@KoalaBear
” It’s led by Google, it’s not from Google. It’s a team of multi-vendor developers / people.”
At this point its basically just Google though. The “multi-vendor” part is basically political posturing to make people feel good right now. You can look through the commit history of the project. No one else has actually contributed (or at least, no one had yesterday when it was revealed).
Ok. Probably it’s only the picture of the outside they tried to make better. Then it’s still funny that IE 10 is the fastest. :)
@ilev
It’s led by Google, it’s not from Google. It’s a team of multi-vendor developers / people.
Who said it was representable for real-life websites? It’s representable for “Web 2.0” websites. So more like webapplications than ‘websites’.
And I don’t like Internet Explorer, and will never do. Chrome looks more polished and feels better. Also fully integrated with the rest of my online life.
Firefox rocks xD ^^
Microsoft doesn’t like the new Google’s tool (yes, it is Google’s benchmark tool) and
even build a site of its own.
Internet Explorer 10 May Score Well with RoboHornet, but It’s Even Better for Web Browsing in the Real World :
Yesterday Google released its latest micro-benchmark, RoboHornet, in which Internet Explorer 10 scores rather well. While we appreciate the gesture, members of our engineering team took a look at the benchmark and found that RoboHornet isn’t all that representative of the performance users might encounter on real-world sites. Like all micro-benchmarks, RoboHornet is a lab test that only focuses on specific aspects of browser performance….
http://windowsteamblog.com/ie/b/ie/archive/2012/09/25/internet-explorer-10-may-score-well-with-robohornet-but-it-s-even-better-for-web-browsing-in-the-real-world.aspx
The compelling reason would be to run the benchmark. In the time it took to write that worthless response, you could have enabled pop-ups. The benchmark can run in the background. Don’t bother responding. I know how busy you are. ;-)
I had to disable the SVG resize test in both Firefox (32-bit) and Waterfox (64-bit) to get the benchmark to complete. All else being equal, FF = 130, Wf = 125. I guess the 64-bit advantage is only apparent in a video-centric benchmark.
Since the benchmark relies on pop-ups, I don’t have time to run it. I keep pop-ups blocked and see no compelling reason to change that.
Its not independent benchmark, it is backed by Google Chrome team.
http://paulirish.com/2012/a-browser-benchmark-that-has-your-back-robohornet/
So I think you should probably change the title of article.
As far as I understand, the panel votes what comes in and what does not?
I think you are right on the SVG resize problem in Chrome. I’ve runned this test yesterday evening, but it seems to get stuck on the SVG resize part.
Will run it again if they fixed this issue, at least looks like an issue to me.
It looked to me as if the test ran multiple times in the Chrome browser.