Microsoft reveals how it validates updates before release
Microsoft promised in November 2018 that it would make an effort to improve update transparency and quality. The company went through a streak of bad updates that caused all kinds of issues on Windows devices.
Tip: check out our overview of the latest Windows updates and services packs here.
Microsoft, for example, pulled the October 2018 Update for Windows 10 after reports of data loss and other issues emerged. These issues did not affect all users or even the majority, but it was reason enough to pause the update for about six weeks to sort things out.
Michael Fortin, Windows Corporate Vice President, published "Windows monthly security and quality updates overview" on the Windows Experience blog on December 10, 2018 as part of the Windows approach to quality series.
Fortin explains that Microsoft is updating up to 1000 devices per second during peak time, and that the company's infrastructure and systems support a rapid distribution of security updates and other updates to hundreds of millions of devices.
Update quality is critical according to Fortin considering the "important of the security and other fixes" that Microsoft releases each regularly at scale.
Every day we build and package the latest fixes, and our engineers test and validate the fixes through a combination of [..] activities.
He lists the following activities in the article:
- Pre-release Validation Program: validates updates before they are made available to "in-market customers". Microsoft aims to catch issues with updates as early as possible.
- Depth Test Pass: automated and manual tests that analyze code change areas to ensure that issues have been fixed and that fixes don't introduce new issues.
- Monthly Test Pass: runs updates on tens of thousands of "diverse devices" to "ensure application and hardware compatibility.
- Windows Insider Program: non-security updates delivered to the Release Preview Ring to get feedback and collect diagnostic data.
- Security Update Validation Program: invitation-only program for "large commercial customers and ISVs" to validate security fixes and identify issues early on.
- Cross-product compatibility tests with other Microsoft teams, e.g. Azure, Office, and SQL Server.
- Live Site Validation Testing: validation that releases are available on Windows Update and successfully downloaded and install on devices running Windows.
- Customer support monitoring.
- Social media and forum monitoring.
Closing Words
Microsoft did not catch the October 2018 Update issues even with all these systems and programs in place. While it is certainly an impossible task to catch all bugs prior to release, it is certainly possible to reduce the number of critical issues but that would require adjustments to the pre-release testing process.
Now You: How was your updating experience this year? (via Deskmodder)
When are all these new improved goodness filled updates supposed to start?
Woody still has the latest ones at Defcon 2:
“Patch reliability is unclear. Unless you have an immediate, pressing need to install a specific patch, don’t do it.”
https://www.askwoody.com/
I bet Fortin is suddenly and mysteriously reassigned soon.
Attention passengers, this is your captain speaking.
Most of you may have noticed that the plane’s engines have failed. There was a typical MS blue screen.
We contacted MS. They said not to panic. A fix will be released in the next 48 hours.
Thank you for your patience with MS Air.
You should update the headline:
“Microsoft reveals it actually validates updates before release!”
This has been discussed on Reddit numerous times. Microsoft has skipped manual testing and on-site testing for numerous updates because it couldn’t keep up with it’s new schedule. In typical Microsoft brilliance instead of expanding their QA team they actually got rid of it altogether in favor of “automated tests”! Microsoft is more concerned about pushing it’s deceptive “Windows as a Service” agenda than about it’s customers.
I was on 1703 earlier in the year and a few months ago messed up by reactivating updates and clicking check for updates. A hour later, I was on 1803 which works but forced me to spend a lot of time removing and turning off all the extra junkware and reverted settings.
That one “update” was it for this year. I have an update from last month which seems important downloaded for offline installation but I’m afraid…
So where in that list does comprehensive in-house beta testing appear? It’s not a line item in that list. Perhaps it’s part of the “Pre-release Validation Program”, but they don’t actually say what is done in that.
I’ve worked on many products that approach the complexity (and diversity of customer machines) of Windows, and what Mr. Foritn says about the importance of automated testing is absolutely true. Without that, it’s basically impossible to adequately test such projects.
However, no amount of automated testing can replace comprehensive in-house testing by human beings that specialize in testing. If you skimp on that, your quality will always suffer.
Apparently MS has a new approach to updates …
https://www.howtogeek.com/398226/now-windows-10-has-c-b-and-d-updates.-what-is-microsoft-smoking/
Not a Windoze user myself but I help look after a few machines for family. The disasters with updates and the lack of freedom to choose when to install them has made me use third party tools to block them. I’ll only do them periodically now and only when I know there are no issues.
Windoze as a “service” sucks big time.
Friends stories….
We continue for bingo. Forced update today on the Windows server and … to take two clients for ass. SuWe continue for bingo. Forced update today on the Windows server and … to take two clients for ass. Suddenly the HyperV service has passed to a better life. The data can be obtained by mounting the unit, but since the service does not want to start, I can not start the machine and I have to reinstall everything. And on top of two clients who just did not want replication. : facepalm
PS: and a week ago an update was loaded on printers link from RDP.ddenly the HyperV service has passed to a better life. The data can be obtained by mounting the unit, but since the service does not want to start, I can not start the machine and I have to reinstall everything. And on top of two clients who just did not want replication. : facepalm
PS: and a week ago an update was loaded on printers link from RDP.
My main reason for fleeing 10 and returning to 7 was the little respect they showed for the users when applying the patches. When I want it. And if you have something to save or some process half, you fuck. And if the update breaks something or erases something, you fuck too. I understand your concern and the need for everyone to have the system up to date on patches, but not like that. Display security options on the one hand and news on the other, which you can. To burst a PC of a user so that it has the 3D Paint among other things dispensable is a tease.
If we add to this that the patches are causing serious problems with more and more frequency, I do not know what to expect to rethink the situation. The fame that is winning Windows 10 I do not remember neither in the times of the Vista or the ME.
Meanwhile, the genius of my company that decided to migrate user computers to Windows 10 is still buried in incidents and complaints. I hope it touches me in a very long time.
@juan: I thought taking clients out for ass had more or less ended as a sales tactic by the mid-1960s. Oh, wait. Never mind. ;-)