Microsoft: AV-Test study that Bing serves 5x more malware is inaccurate
The German IT-Security institute AV-Test published the results of a test study earlier this month that analyzed search engine malware delivery. The company used a sample size of more than 40 million websites delivered as search engine results over the course of an 18 month period. One of the conclusions of the study was that Bing delivered five times as many websites containing malware as Google did while Russian-based Yandex delivered ten times as many as Google Search.
Many websites and news outlet published the data without analysis of their own and word made the round that searching on Bing was less secure than searching on Google.
Microsoft's response to the study paints a different picture. The company noted in a blog post published April 19 that the conclusions drawn from the study are wrong. How this can be? AV-Test used a Bing API to retrieve Bing's search results for any given query that the institute analyzed during the test.Microsoft notes in the blog post that it does not remove malicious sites from its Bing search engine, but rather warns users about them while they are on the site. Results are not suppressed or removed from the index, and since API requests do not include the warnings, the researchers came to the conclusion that Bing delivered more malware than Google.
The conclusion itself is not wrong, as Bing is indeed keeping malicious sites in its index, but searchers are still warned on the results pages when malicious sites have been detected by Microsoft. In addition, links to sites are disabled by default.
The reason why malicious sites are not removed from the index right away according to Microsoft is because the majority of these sites are hacked sites that will eventually return to a clean state. Microsoft warns customers but does not remove results for "completeness and educational reasons".
Completeness refers to the perception of an incomplete search engine. If you search for something and the results get suppressed, you may perceive a search engine as incomplete and maybe even not suitable for you and your searches.Â Educational on the other hand refers to the warning messages that Bing displays. It informs the searcher that a particular result should not be accessed at that point in time, which not only keeps users secure but also circumvents the problem that users might use a different search engine if results were suppressed (and thus find and click on a result with malicious contents).
David Felstead, Bing's Senior Development Lead, notes that about 1 in 2500 results pages on Bing have a result with a warning on it, and that the warning is displayed in about 1 in a 10000 searches (a user needs to click on a malicious link for the warning to appear).Advertisement