Microsoft's data-focused approach to Windows Updates requires an update
Microsoft uses a data-focused approach to determine if updates are ready for wider distribution and it appears that the company is very content with the approach.
Issues of recent updates suggest that Microsoft may need to rethink its strategy when it comes to Windows Updates.
Microsoft started a new series on the official Windows Experience blog some time ago in which company representatives lift the veil in regards to the company's strive for quality.
In "Data, insights and listening to improve the customer experience", information is provided on how Microsoft uses data to determine the quality of updates.
The fundamental question that Microsoft asks for each release is "Is this Windows Update ready for customers?". Updates go through different stages during development:
- Automatic and manual testing of updates.
- Evaluation of quality based on diagnostic data and feedback from Microsoft engineers.
- Distribution to Windows Insiders and further evaluation of updates.
- Distribution to the general Windows population.
The metrics that Microsoft gathers and monitors need to be equal to or better than the metrics of the previous update.
By the time we are ready to ship to our customer base, our metrics must be, at a minimum, at or above the quality levels for the previous release, the idea being that every update should make the Windows 10 experience better.
Is the approach sufficient?
Data plays an important role when it comes to the release of updates. Microsoft, and any other company for that matter, may use it to make sure that certain features behave as intended. It is certainly possible to catch bugs by just looking at data but if you look at recent updates, you will notice that things were not as smooth as Microsoft's data focused appraoch suggested.
Microsoft had to pause the Windows 10 version 1809 feature update because it caused a lot of issues. To name just a few:
- A Data loss issue
- Another data loss issue
- No audio issue and bluescreens
- Font display issues
- Mapped network drives issues
All of these issues were not detected during tests conducted by Microsoft, by diagnostic data, and by feedback that Microsoft engineers and participants of Microsoft's Insider program provided.
A game performance related bug was not detected in the most recent update for Windows 10 version 1809. Microsoft had to update the support article to add the performance affecting bug to the list of known issues of the update.
Microsoft engineers and participants of the Insider program may not provide a sufficient sample size to provide data for all major use cases. Gaming might be such a case. It seems unlikely that Microsoft engineers spend a lot of time playing games on their devices. Even if they would, they could never test new versions of Windows on tens of thousands of games that are available for PC. The bulk of Insider participants may not be interested in games as well. Gaming is just one area where Microsoft's approach falls short.
It is certainly unrealistic to expect Microsoft to catch all issues in all updates before release. The sheer number of hardware and software configurations makes that an impossible task.
But major issues, like game performance in popular games, should not hit the general population.
That's one reason why it is a good idea to install updates a while after release and not as early as possible: you never know what is going to happen.
Now You: Let me know what you think about all of this, and how Microsoft could improve update quality.Advertisement