Microsoft: update quality is better than ever
The past six weeks have not been pleasant for Microsoft. The company released the October 2018 Update in early October and had to pull it -- Microsoft called it pause -- to deal with data loss bugs and other issues that plagued the millions of users that installed the update in the four days it was available.
It took Microsoft until November 13, 2018 to hit start again and release the new feature update for Windows 10; this time, the speed in which the update is distributed appears to be even slower than before, likely to make sure that no other stopper bugs are found.
Michael Fortin, Corporate Vice President, Windows, published a text on the Windows Experience Blog, Windows 10 Quality approach for a complex ecosystem, that deals with update quality and updating in general.
The October 2018 Update fiasco is dealt with in the first paragraph only; the update had a "small but serious issues" and was the first feature update that Microsoft had to pause.
Data shows, according to Fortin, that update quality is better than ever. The improvement is visualized in a graph that highlights the customer incident rate over time in hundreds per million devices.
The rate went down from an all-time high in 2015 but Microsoft made changes to the graph to increase the effect it has on people that just glance at it.
The first issue is that the graph begins at 0.4k and not at 0k in the Y-Axis. If you normalize it you'd see little progress near the end of the chart.
The metric that Microsoft uses is "hundreds per million devices" suggesting that one has to multiply the values with 100 to get the right number of incidents. If that is the case, that 0.4k figure would actually be 40k per one million devices instead or 4% of the entire population.
Lastly, while the graph highlights improvements made to the update quality of Windows 10 updates, it gives no indication whether the quality was better or worse in previous versions of Windows.
Fortin throws in some big numbers to highlight how diverse the Windows landscape is.
With Windows 10 alone we work to deliver quality to over 700 million monthly active Windows 10 devices, over 35 million application titles with greater than 175 million application versions, and 16 million unique hardware/driver combinations. In addition, the ecosystem delivers new drivers, firmware, application updates and/or non-security updates daily.
One has to wonder whether that approach -- we are doing better than ever -- is the right approach after the release of a feature update that had to be paused for six weeks due to data loss issues and other issues.
Microsoft promises to be more transparent about the updating process and quality testing, but it made the same promise previously and it did not help address the underlying issue. In other words: the perception of quality that a, seemingly, growing number of users and administrators have is at odds with Microsoft's data driven approach to quality.
Microsoft needs to rethink the rapid update process and re-hire quality testers that it let go to show users, administrators and organizations that it is serious about improving the quality of the operating system.
I have doubts that things would get better otherwise.
Now You: What is your take on this?
LOL, I needed a good laugh first thing this morning!
thank u – my experience has been, still happening, and i have no doubt will be happening is THE defining element of Windows10 in my experience beeing the incessant need to mouse click (fill in the blank) multiple times to get a response … open a doc? click click click – open an article? click click click? – open edge? click click click open a photo? click click click etc. etc. etc. the W should be for weak!
God Bless Microsoft, and God Bless AMERICA !!!
Mandatory OS updating is bad enough, but forcing firmware and driver updates is thoroughly unacceptable, given the many problems they can create for the end user. Until updating is made optional like it was in the past and M$ discontinues the privacy-raping telemetry, I have no interest in Windows 10. Period.
Forced firmware updates are possibly there to force you to buy new hardware every few months or years when it bricks.
Firmware updates should be always users choice at his/her own risk.
OK Microsoft. Now how about a graph to show the quantity of issues with Windows 7 updates prior to 2015 – you know, when you had a proper QA department?
Also, it’s not necessarily the number of issues, it’s the severity of them. A release with 10 minor bugs isn’t too bad, but a release with two major ones will cause far more disruption – despite being quantified by yourselves as “5 times better quality”.
Lastly, in 2015, the “Windows Insider” thing was a lot more novel – a lot more home users/enthusiasts were testing with it and reporting issues. Now though, a lot more people are sick of being beta testers for year after year and I’d bet that the Insider program is now more restricted to business testers and fanboys. Hence, are the issues just not being reported as much?
Home users are still beta testers, except this time they supposedly get “final” releases of the bigger updates, which obviously are nowhere near being final.
Using the least knowledgeable to be alpha/beta testers is recipe for disaster. Home users do not know how to file a bug report nor do they realize they should be. Plus the MS error messages are notorious arcane even to experts. So I would expect bugs to be under reported and often reported incorrectly when they are.
What a joke.
Disgraceful, and a complete joke. I’m not even going to elaborate on the joke that this is massaging of the facts represents.
Just buy it…..all this scale and complexity can â€œjust workâ€!!!
Windows 10 is a fantastic operating system that is objectively superior to previous versions. These days the world is connected and changing at an incredible rate and frequent updates are necessary whether you like it or not.
Windows 10 is a poo sandwich that might smell and even taste better than previous poo-sandwiches but it’s still a poo sandwich when you come right down to it, now a poo-sandwich with sparkling wee-wee sauce thanks to the Privacy-slurping features. These days the world is connected and changing at an incredible rate, yes, and updates do need to happen but what we’re saying is that MS is making a right royal ham-fist of them, and needs to do MUCH better if people are going to remain on the platform and not migrate to something cleaner and less infuriating to deal with on a daily basis.
Insultingly blatant PR exercises like the unintentional comedy above is not a company owning their mistakes but one that thinks we’re children it can hand-wave at, and that attitude permeates every facet of Windows 10. I’ve been using Windows since 1993 and Windows 10 in 2018 is where I finally ended my association with the entirely abusive, profit-over-people focused Microsoft and switched to Linux, and good riddance!
@Lunsbox: “frequent updates are necessary whether you like it or not.”
I dispute that assertion. Can you support it?
If frequent updates are necessary, how about make them to work like other operating systems, like MacOS, Android, Linux? Make them fast and not requiring restarts all the time? Sorry my friend, I still have to use Windows because of my work, but I am not a Windows fanboy, so I can’t see anything fantastic about this mess.
My take is that, after 20 years of Windows use, all those M$ screw ups have made me seriously consider eventually dropping Windows (when 8.1 extended support stops) and move over to Linux or Mac. I am sick & tired of M$ sticking its middle finger up at us. I have no confidence in the company anymore, and I believe that M$ is out to shed all its home user customers in favour of the more lucrative business market, preferably with WaaS.
Go Mac, where changes are fairly gradual, well-tested prior to rollout, and major updates are only once a year. Plus the added bonus of less Privacy intrusion.
@ShintoPlasm: I am tempted to switch to Mac, for sure, but there is 1 big issue that scares me:
the Macbook Pro is virtually irreparable because the important components are glued or rivetted or whatever, and therefore cannot be removed if they break. For the kind of money you have to fork out for such a machine you would not expect that.
And a minor issue is the “butterfly” mechanism under the keys of the keyboard. Yes, Apple kind of fixed the dust/crumb issue with the silicon membrane underneath, but is that a viable, long-term solution?
@Klaas Vaak: That’s pretty unfair, Klaas. Microsoft is a company with long international experience and a track record of successful localization. It’s *two* fingers in Britain and a *whole forearm* in France. ;-)
“Windows 10 is a fantastic operating system that is objectively superior to previous versions. These days the world is connected and changing at an incredible rate and frequent updates are necessary whether you like it or not.”
Are you being ironically sarcastic ?
If not, are you:
1. A Microsoft employee ?
2. A Microsoft fanboy ?
3. A Troll ?
4. A delusional idiot.
Gary D gets an imaginary upvote for actually commenting what I was thinking.
I seriously believe Ghacks, along with Woody Leonhard, et all…is actively monitored by M$, and the company inserts “plants” where needed….which is pretty much everywhere these days….
If that is true……then, it’s kind of pathetic really…
An arrogant fan boy troll. Tesla lives by these condescending know it all bubble boys.
The only way to make W10 great again is stop this upgrading/updating madness and turn back immediately to a good LTSB version for all people (in the W7 way, more or less). Normal W10 versions are not good for productivity, nor for enterprises or for serious work, because you can’t, read me well MS’s team, you can’t stop for days your life after doing some kind of silly upgrading/updating twice per year/month, considering also that you don’t take care about your insider’s advices. Update quality? Just please don’t make me laugh no more.
@ John G
Normally, Volume Licensing(= minimum 5 Win 10 Ent licenses) is required to obtain the LTSB edition. Seems, now people can buy just ONE Win 10 Ent LTSB license …
https://community.spiceworks.com/topic/2167558-explicit-instructions-for-purchasing-a-windows-10-ltsc-license (Oct 9, 2018)
Very nice and useful information link, thanks. Anyway, I didn’t know that there are by now 21 ways to get a LTSB version, it’s an unbelievable fact itself. Windows 10 is just so fragmented that I can’t understand how they are able to support all the versions at time while releasing two major ISO upgrades per year. Also it’s not easy and cheap to get LTSB for single computer, and probably that’s the cause to get stuck at W7 or W8, or even older OS (I was glad to saw yesterday an office computer working with no problem at all with Windows XP inside, and the owner said me that it has been working fine for the last nine years). Again, thanks for the info.
@ John G
For those who can’t get away from Windows 10, especially after Jan 2020 = the EOL for Win 7, eg gamers and businesses, US$300 for a Win 10 Ent LTSB 2019(based on version 1809) license may be worth it, ie for 10 years of stable use until 2029 = US$30 per year.
In comparison, Win 10 Pro Retail costs US$199 per license, likely for only a few years of unstable use.
P S – The latest Win 10 Ent LTSB 2019 is not supported by M$ if run on “old” silicon/CPUs like Intel Skylake and Haswell, ie must run computers with new silicon/CPUs like Intel Kabylake or Coffee Lake.
___ M$ has used and will likely use silicon or processor-blocking updates to make obsolete 4 to 5 year old Win 10 Home & Pro computers.
Ironically, the LTSB version isn’t even worth it for private consumers in the long run. Because all programs are updated with adjustments for the newest Windows 10 versions, while ignoring older builds. Personally I had noticed serious flickering issues with Firefox on Windows 10 1511 when the option to color the task- and titlebar was enabled. Cyberduck also refused to install on that version; requiring a newer Windows 10 version. So to me that was the very best proof how sadly even a LTSB version won’t make sense. Unless you are lucky enough to use programs that will never be affected by problems that occur due to developers ignoring older Windows 10 builds.
“Update quality is better than ever” – Wondering what the poor lads have to say who lost important data. Or the people whose computer went ballistic after an update. And I am sure there are many, many others who don’t even realize their PC’s have been screwed up by MS updates.
The only thing for me that is getting better than ever is my experience with a Mac. Still, I am using my Windows PC but slowly but steadily changing everything over to Mac. Small learning curve and it simply works. Not a Mac fanboy, I think they are quite expensive and I don’t care for the close button on the left side. But good for my peace of mind. So, good bye, Microcrap.
Someone confused that graph – must be the Customer Installation Rate.
Keep tapdancing, Microsoft. A large (but not majority) of people I personally know have experienced problems with so many updates that they perceive it’s most of them (although I don’t think that perception is accurate).
A huge part — most of — of Microsoft’s problem is straight-up process and quality control. They bought into the fad that QA people aren’t really so necessary because Agile. This idea is fundamentally wrong.
Another often overlooked aspect of this is related to rapid release. Let’s say (exaggerating and rounding numbers for illustration) that you produce software, and 1% of your releases include a major malfunction of some sort. That’s actually not too bad. But if you’re releasing 100 times per year, that 1% failure means that every one of your customers will encounter a major malfunction every year.
Your customers will perceive that your software is unreliable. The numbers can be interpreted otherwise, but the reality is that customer experience is the only thing that actually counts — and the customer experience is that your software quality sucks, which means that no matter what the statistics say, your software quality sucks.
Rapid release means that software quality must be, technically, orders of magnitude higher than releasing ever year or two. Microsoft isn’t the only company to get tripped up by this.
Is this an out of season April fools joke?
From the Fortin link, fourth paragraph:
“…and added a fundamentally new capability to our approach to quality: the use of data and feedback to better understand and intensely focus on the experiences our customers were having with our products…”
Are they serious? This approach was fundamentally new eons ago. Maybe MS truly believes people are stupid but publishing garbage like this so the millions of employees of companies who really do understand product development and quality can read Microsoft presenting themselves as naive and honestly, dumb, is very foolish.
That graph is plain silly; again, 40,000 ppm unhappiness is dreadful, if that’s even what MS is saying. Maybe they’re tracking through their Net Promoter Scores (LOL!) the number of users with 100 unhappiness events; that seems to represent reality much better!
When users are wary their devices will be harmed or trashed by MS’s junkware, that in itself is a far greater problem than any weird metric MS could contrive. Afraid of buying a car because it may spontaneously crash regardless of what the driver is doing? I doubt it!
This won’t quit as long as MS has a more or less captive market.
Woody’s not impressed with the re-re-re-release of 1809, gives it Defcon 2, don’t download unless you absolutely have to:
Very nice analysis, Martin. I particularly appreciate you pointing out the deplorable scale of the Y axis. The whole implication that time began in 2015 is pretty funny and I remember when Windows service pack betas were invariably bulletproof until sometime before the release of Vista when everything changed completely. At least they deserve credit for consistent progress in a downhill direction since then!
Win 10 is so far into the range of unacceptable that it’s perversely amusing to watch the continuing disaster unfold. I do not believe that management there will change the present policies because their egos and contempt for their customers dictate the chosen course while the profits, in their minds at least, seem to indicate that all is actually well.
Paging all the “Another good reason to move to Linux!” posters lol
Waaay ahead of you. ;)
Win10 is incoherent piece of crap, full of bugs, with securely incorporated data mining spyware.
Keep in mind m$ employs doublespeak through its propaganda outlets, hence less is more.
I think MS has gotten hilarious amounts of shit for W10, most of which underserved IMO….but yeah this statement is made in really poor judgment. They have seriously shit the bed with multiple updates ~in a row now, not the time to say “update quality is better than ever”.
@Shadess: “most of which underserved IMO”
I assume you mean “undeserved” here. What criticism has Microsoft received about Windows 10 that was undeserved, though? I honestly can’t think of any (outside of the usual lunatic fringe, anyhow).
Mostly about the privacy of it all, it really isn’t different for any Apple or Android thing people use. Microsoft just made the “mistake” of kinda showing it to people, instead of burying it deep, deep in the last settings page nobody ever goes to on Android or whatever, that then links to a page with long boring text wall of legalese that says the same things about privacy Microsoft says.
Then I’d also question some of the fanboyism about XP/W7 and even W8 that some people think are somehow much better, but mostly the privacy thing.
M$ will soon announce that she will “copy” Apple’s closed hardware ecosystem, ie new versions of Win 10 will only run on the high-end M$-Surface devices and a few selected high-end OEM computers, in order to give the highest quality in forced auto-updates/upgrades to her highly-valued Win 10 users.
M$ CEO: “Low-end computers are for low class Linux lackeys”.
Netmarketshare numbers for Windows desktop OS:
Sep 2016 = 90.45%
Oct 2016 = 89.97%
Oct 2017 = 88.00%
Oct 2018 = 87.27%
Since the cessation of the 1-year free upgrade(from Win 7/8.1 to Win 10) promotion in Aug 2016, Windows has lost more than 3% in world marketshare = about 60 million desktop/laptop computer users left Windows = these ex-Windows users just up and leave = not feed-backed or recorded by M$ or the numbers were just ignored by M$.
The above numbers likely prove that Win 10 patching has gone down the drain.
Causes: the laying off of about 120 Professional Windows testers in April 2014 to be replaced by Windows Insiders alpha-testers, Win 10 Home beta-testers and forced Telemetry & Data collection; rapid twice per year upgrades; forced auto-updates/upgrades.
“The bigger the lie, the more it will be believed.” … Joseph Goebbels
I’ve been a linux user for years now but I still have to look after a few windoze pcs and I don’t care what they say about the quality of their updates — I have had several issues to deal with. NO MORE. I will be blocking updates with a third party tool and I will just update *after* it looks like the beta testing bugs have been squashed.
Yeah, it’s “difficult” for Microsoft. It’s difficult for Google, too. Why is is that Google can upgrade Chrome OS and Android seamlessly, and Microsoft can’t do the same with Windows ? Why is it that Microsoft once used to be able to do it, and nowadays a Windows update is a virus by another name ?
This blatant attempt at lying by numbers reminds me of so many lies put out by governments, eager to “prove” that what we can see with our own naked eyes in fact does not exist.
But that’s all the rage now. When the consumer / citizen protests, explain him he’s wrong, instead of correcting your own mistakes.
@Clairvaux: an excellent qualification.
@Clairvaux, agreed 100%.
Check out this post on AskWoody for more believable graphs:
Actually, I was wrong. “We’re doing better than ever” (said in the face of major failure) might be fashionable again, but this extraordinary piece of denial by Microsoft is eerily reminiscent of another iconic, official lie.
In 1930, Stalin wrote an article in the Pravda, stating that his collectivisation program of agriculture had made his fellow comrades “dizzy with success” — while it had been an unmitigated disaster, with peasants deported and killed, millions dying of hunger and production plummeting.
Now Stalin, despite the monster he was, might have done a tad better than Satya Nadella in that case. The “dizzy with success” phrase, which has since acquired legendary status, was used to announce a pause in that horrendous destruction of Russian agriculture. The official line was that the program had been so successful that the brave communists overdid it just a bit, and now was the time to “consolidate success”.
Regarding Microsoft, however, there’s no sign that this staggering piece of corporate-speak is a means of backtracking from the horrendous program of Windows destruction set in movement from the top.
My take on the quality of Windows updates since spring of 2015 is that it is *essential* to maintain a set of restorable images (or a couple of bootable clones) of the system drive when it was in a known-good state. If you’re as averse to having to reinstall and reconfigure everything from scratch as I am, the odds of an update borking your system are just too high. Fifteen or twenty years ago, the biggest risk came from bad mechanical hard drives. (Remember IBM “Deathstar” drives and Western Digital’s infamous “click of death”?) Now it’s bad Windows updates.
MRW i read the title : https://www.youtube.com/watch?v=ObpcGNCU944
It really is insulting the constant mockery of Microsoft, the graphic demonstrates the despotism of this company that laughs day after day of the users.
I strongly recommend everyone to consider changing the system, linux for example is free, respectful of your privacy, you have your real control of your pc, equivalent programs and simply fantastic.
Update is something nice here are second gentlemen! and no need to restart!
PS it’s time to change the slap in the face of Microsoft’s insulting arrogance, they’re on the way to the slow but sure collapse!