Microsoft changed update testing significantly in recent time. Previously, it relied much on teams of testers and "real" hardware tests, but that shifted to test automation, tests on Windows Insider computer systems, and machine learning.
Machine learning helped improve the Windows 10 update experience according to Microsoft. A new blog post on the company's Tech Community website reveals details about the use of machine learning in regards to the creation and release of updates for Windows.
The long article is quite technical in nature but it might be enough to read the opening paragraph to understand how machine learning is used by Microsoft when it comes to updates for Windows.
Machine learning helps us detect potential issues more quickly and helps us decide the best time to update each PC once a new version of Windows is available.
In short, it is used by Microsoft to evaluate updates and to help with roll outs of feature updates. The article focuses on the use of machine learning to assist in the rollout of feature updates for Windows 10.
Microsoft started to use machine learning in broad scale when it released the April 2018 Update for Windows 10. Machine learning was used to determine the release quality by monitoring six "core areas of PC health" including PC reliability.
The number of areas increased to 35 when Microsoft released the May 2019 Update in 2019 and Microsoft plans to extend the coverage further for future updates.
Microsoft notes that PCs that are selected by the company's Machine Learning algorithms "have a significantly better update experience". PCs selected by Machine Learning have "fewer than half the number of system-initiated uninstalls, half the number of kernel mode crashes, and five times fewer post-update driver issues".
The largest portion of the article describes how Microsoft designed and build a Machine Learning model to support Windows 10 updates.
Microsoft uses a dynamically trained model that is trained on the most recent PCs and that is capable of differentiating between good and poor experiences.
Every Windows 10 release starts with the push to Windows Insiders and other early adopters. The experience is actively monitored by Microsoft using diagnostic data and other signals, e.g. feedback, reports on social media.
Machine Learning is used at this stage to identify potential issues to protect certain PC configurations and setups from receiving the update at that point in time, and to predict and nominate PCs that will likely have a good update experience.
The process is repeated daily and the model learns from the signals that it receives from recently updated PCs. Fixes and improvements that Microsoft makes over time are taken into account by the model as well.
Microsoft notes that Machine Learning helps the company identify safeguard holds. It relied solely on "laborious lab tests, feedback, support calls, and other channels" in the past to detect compatibility issues. These are still used according to Microsoft but Machine Learning enables the company to better discover issues that may disrupt the update experience.
The use of Machine Learning will increase in the future; Microsoft hopes to improve automation further and reduce the time it takes to catch bugs from hours to a few seconds.
Machine Learning is not a catch-all solution that ensures a perfect upgrade experience for all devices all the time. Recent updates -- cumulative and feature updates -- have shown that there will always be known issues; some of these may have been avoided if Microsoft would still maintain a large team of testers.
Most Windows customers don't object to the use of Machine Learning, probably, but some may have the opinion that Microsoft is relying too much on Machine Learning and diagnostic data.
One interesting question to ask would be whether there would be more or less issues if the testing teams would still be used by Microsoft.
Now You: What is your take on this?Advertisement
Ghacks is a technology news blog that was founded in 2005 by Martin Brinkmann. It has since then become one of the most popular tech news sites on the Internet with five authors and regular contributions from freelance writers.