back to article Memo to Microsoft: Windows 10 is broken, and the fixes can't wait

Windows isn't working – and Microsoft urgently needs to change how it develops the platform, and jettison three filthy practices it has acquired in recent years. In 2014 Microsoft decided it could do a better job if it discarded a lot of software testers. This bright new dawn was lauded at the time by Peter Bright at Ars …

Page:

        1. Anonymous Coward
          Anonymous Coward

          Re: Please Please

          "Telemetry aside, I have fellow astronomers complaining that W10 will happily start an update halfway through an imaging session, wrecking data. Getting scope, camera, guide system, computer all working nicely is quite a hassle, and given the rarity of good, clear nights, the last thing you want when you have got everything working is for the OS to throw a spanner in the works. Call me old-fashioned, but I always thought two key roles of an OS were keeping programs running smoothly, and keeping data safe. Quite clearly, farming out testing to well-meaning enthusiasts is no replacement for professional software testing."

          W10 is a "fuck you, dear user, I'm updating and rebooting NOW" OS, like MS has always advertised.

          Not trying to defend them, but it's been clear for now years !

          So how are you surprised it's screwing up your work, astronomy or otherwise ? There are countless examples of this ! I've once worked as a volunteer for a cross country race where the same happened !

          By the time it finished to update, the race was over !

          Just move to another OS ...

  1. Franco Silver badge

    The fact that the September releases have been switched to a 30 month support lifecycle shows the pressure MS are under from the very people they shouldn't be pissing off. Enterprise is their core market, and flinging out these crappy releases as essentially public betas does not inspire confidence.

    1. Dave K Silver badge

      "Enterprise is their core market"

      Quite right, which is why there is one additional round of testing that the article didn't go into - end users. Once the update has been through the Insider program and is considered ready for release, it's important to note that it is only considered ready to release to home/small business users. Enterprises on the Current Branch for Business (as it used to be known) get the updates a bit later once the update has been unleashed on home users and any final major bugs have been called out and patched.

      Hence, these testing issues aren't a major concern for enterprise - they know that they'll only get the update once millions of other people have received it without major issue. It is however a big problem for home and small business users that are now being treated as a final round of testers for enterprise. It also means that instead of these bugs being caught and quietly patched during internal testing, they're now showing up on end-user systems and are causing much more noise and stink.

      Ultimately, I fully agree with the article. MS's development approach to Windows 10 is broken and changes need to be made if end users are to see Windows 10 as anything other than constantly flaky and beta software.

      1. rmason Silver badge

        Not my experience

        @Dave K

        Manage an enterprise environment that includes windows and windows 10.

        The 'feature updates' hit our WSUS server around the same time as the plebs get them. Within days normally.

        It's up to us to not approve them until the dust settles. Then we have a test group (we limit the laptop models we buy to one brand and 3-4 models for this reason), then we deploy.

        The update was there though. This particular feature update was available on the same day as the first articles about user issues appeared. The day before the data deletion thing became widely know.

        In fact we have a group of five test laptops *with* this current badgered feature update installed on them! something tells me they will have issues updating when the "fixed" version finally lands.

        This issue aside our fleet of laptops never struggle with updates *apart* from the feature update. they don't exhibit any issues, the update simply gets perma-stuck at "downloading, 0%" and they need manual intervention. this is probably 1 in 5 laptops. Which is a pain.

        Basically we catch them as and when we are interacting with that device for any other issue. It has become habit to check windows update status when doing *anything* to a user laptop.

      2. Anonymous Coward
        Anonymous Coward

        Hence, these testing issues aren't a major concern for enterprise - they know that they'll only get the update once millions of other people have received it without major issue.

        Maybe, but those home users are typically doing very different things. The kinds of features that enterprises are using are not being tested at all in this model, and it shows.

        MS did have a go a while back at trying to get an Insiders for Business thing going, but as far as I know there's been very little take-up of that (not surprisingly). Also, the whole insider build approach assumes you're ok with regular in-place upgrades from one build to another, and this process in itself is not well suited to enterprise use.

        1. Danny 14 Silver badge

          our wsus goes along the line of pick a base edition to start with (we started with 1604) then decide on a feature update later (1709 for us). Wait for the bugs to be fixed 3 months later then start testing. wait a year and start the process again. Technically 1809 will be our next feature update: ill probably wait till next summer for the bugs to be fixed.

        2. Tom 35 Silver badge

          Yes home users are great for testing anything domain related, large LANs, SharePoint integration...

      3. JohnFen Silver badge

        "which is why there is one additional round of testing that the article didn't go into - end users"

        Except that kind of testing is inadequate, no matter how many millions are in on it, for a number of reasons such as that the testing covers the use cases of home users, not corporate users (and even then, by definition, only the "happy path" will be heavily "tested"), there is no adequate reporting of problems found, and so forth.

        This notion that you can make up for a lack of quality by increasing quantity is fallacious.

  2. Anonymous Coward
    Anonymous Coward

    It's-a rubbish!

    I had a relatively minor problem (compared to some!) of an update breaking my graphics driver, seemingly disabling what I had and leaving me with some sort of generic driver with a choice of only three resolutions, none of which matched the native resolution of the monitor. I wish I could bill MS for the time I spent trying to fix the issue, but finally resorted to rolling back and, thankfully, everything worked OK again, even to the extent of the desktop icons being back in their rightful places. All my Win 10 machines (I have several computers around the house - some running a DIFFERENT operating system, I'll have you know!) are pro. version, so I have now disabled updates, using the group policy editor and disabling the service, which some folk state can be also over-ridden by MS. True, or not? We shall see but I am NOT updating again until there is a firm testing policy re-instated to get over the present debacle. (Definition: "a sudden and ignominious failure; a fiasco") !!!!!!!!!

    1. Wibble

      Re: It's-a rubbish!

      > I wish I could bill MS for the time I spent

      That's the root of this crappy attitude to testing: it's an overhead which can be offloaded to unpaid "enthusiasts". Unpaid means there's no contract, so no explicit commitment to quality, which is the raison d'etre of professional testers.

      Staggering naivety on the part of the senior management at MS.

      Of course, by extension, this naive culture means that *all* MS software will be unreliable, even the sacred cloudy software.

      If only MS ended up footing the bill for this short-sightedness. When's the class action suit happening?

      1. nematoad Silver badge

        Re: It's-a rubbish!

        "Staggering naivety on the part of the senior management at MS."

        I think you are being too generous. In my mind this is nothing to do with naivety but is a cynical "How can we boost our bonuses?" play. MS is not really concerned with their users, all they seem to worry about is how Wall Street reacts. If the money keeps rolling in why should the top brass give a damn about quality and testing? In their minds that is only an expense not something that should be central to all they do.

        As the old saying goes"Follow the money."

      2. JohnFen Silver badge

        Re: It's-a rubbish!

        "Staggering naivety on the part of the senior management at MS."

        This is part of the rotten core of the rapid release philosophy -- it's OK to release broken shit because you'll fix it in the next release.

    2. Tom 35 Silver badge

      Re: It's-a rubbish!

      "which some folk state can be also over-ridden by MS. True, or not?"

      I have a computer with the service disabled, and a script that will kill it if it tries to start. And yes I have seen that script pop up a few times, most often when you plug in a new device even a USB stick, or a mouse.

  3. JohnFen Silver badge

    QA Prestige

    "Raise the prestige of testers in the company."

    Yes, and not just at Microsoft, but in the entire industry. QA professionals are just that -- professionals -- and are as critical to the success of a project as devs are. For far too long, I've seen QA treated as something "less than". They aren't, and treating them as such just degrades the performance of QA, which hurts everybody.

    1. Warm Braw Silver badge

      Re: QA Prestige

      Probably worth adding that an increasing number of senior management types are getting anxious about the potential liability for data breaches and other damage resulting from a failure to immediately apply the latest updates to all of their systems, so IT departments are finding it increasingly difficult to argue they need to hold off until they've checked compatibility or back out changes that are found to have operational consequences. This really means the vendors need more QA, not less - though Microsoft is far from the only culprit in this respect.

      1. frank ly Silver badge

        Re: QA Prestige

        Everybody hates testers. Developers hate them because they find faults in the developer's baby and managers hate them because they slow down acceptance and release by finding faults. The really 'bad' ones find flaws in the test methodology and make formal reports about it. (I know, I was a software and system tester for many years.)

        1. JohnFen Silver badge

          Re: QA Prestige

          "Developers hate them because they find faults"

          I'm a developer and I love them precisely because they do this. When testers find mistakes I've made, we all look better to our customers. When customers find those mistakes, we all look worse.

          Testers make me look better, and I will forever love them for it.

          1. MatthewSt

            Re: QA Prestige

            Too true. Only shoddy developers hate testers (and shoddy managers for that matter, if that's not a redundant phrase!)

            1. Anonymous Coward
              Anonymous Coward

              Re: QA Prestige

              I hate testers, but i think it's more the shoddy testers my place hires than testers in general.

              The highest bar was set during the first end to end test of a system, weeks/months into the testing when it's mostly signed off a working as expected, and the end of the process flagged up that they didn't receive the file they expected.

              investigations begin, the testers are adamant that it's passed all functional testing so far, we question that they've checked the output files are being passed on to the next stage, and are the right format...

              and no, they haven't checked that at all, but there were no errors generated so it 'passed'. Several months of 'successful' tests all actually failed, because the process fell over before it could generate an exception report that said it hadn't worked. Not a single test case had actually completed but the testers were relying on a part of the code that didn't work, to tell them it wasn't working, instead of testing if it was actually working by looking at the output.

              i frequently ask questions about basic test theory and coverage and get completely blank looks, even when i explain the concepts in case they know it by a different name (testing boundaries etc).

              We don't hire testers at all from what i can tell, just people to write meaningless, impenetrable, test reports that give the illusion of it having been thoroughly tested if anyone asks. One case when i asked about testing not actually touching 95% of the code, i got the simple reply, well you signed off the test plan (high level, practically a statement "we'll test it")

              1. Claptrap314 Bronze badge

                Re: QA Prestige

                Yeah, you might want to start looking for another job before your company goes belly up.

        2. veti Silver badge

          Re: QA Prestige

          More than that, because testing is only as good as the documentation that comes with it. Before you can test, you need - something like an actual spec.

          That rules it out for about 30% of all companies right away.

          And the spec needs to be reasonably clear, complete, accurate and up to date. That probably strikes out another 50%.

          Without that documentation, QA is always getting the mushroom treatment.

          Alternatively of course you could just write comprehensive user documentation. (Then tie it to the leg of a passing carrier pig.)

          1. Adelio

            Re: QA Prestige

            I would suggerst it is probably more like 60% to 90% have poor documentation

      2. This post has been deleted by its author

    2. a_yank_lurker Silver badge

      Re: QA Prestige

      @JohnFen - A good developer will recognize the need for peer review and professional testing of their code. They catch bugs and make the developer check for and fix issues. Is it annoying at times, yes, because as developer I would rather be writing new code than fixing my or someone else's mistake. In fact I had a situation at work were the tester found a problem that required me to investigate the issue. It is not relevant where the issue was just that it was caught before the code was released.

      Slurp is failing with Bloat because they want to be 'hip' and 'sexy' while OSes are staid and boring. But without an OS how does MS propose for us to use our computers; otherwise they are very expensive paperweights. There is a lot of unglamorous software users depend on everyday and there is money to made if you got your act together. Destroying Bloat destroys Slurp by destroying customer trust. I have permanently banished Bloat 7 to a box that is not connected to the Internet; non of the Bloatware requires a connection. I have no intention of buying any Slurp product as they as shown themselves to be untrustworthy. Chromebook, Mac, Linux box I would consider.

      1. Anonymous Coward
        Anonymous Coward

        Re: QA Prestige

        Oh yeah, Linux. Just had the 'pleasure' of setting up a Lubuntu system. Default document editor flickers like crazy, prints half size, automatic security updates are broken and still won't sync with Google Drive.

        I've heard that Apple systems tend to work fairly well when Apple isn't blocking third-party repairs but from my limited experience, I reckon that's more show off Apple users than anything else. P.S. don't try to configure Macs on an enterprise network. Don't Even Think About It.

        1. Fungus Bob Silver badge

          Re: QA Prestige

          "Just had the 'pleasure' of setting up a Lubuntu system..."

          Were you installing it on a TRS80?

          1. Michael Habel Silver badge

            Re: QA Prestige

            No it was a ZX-400 of the Timex make.

        2. Updraft102 Silver badge

          Re: QA Prestige

          Oh yeah, Linux. Just had the 'pleasure' of setting up a Lubuntu system. Default document editor flickers like crazy, prints half size, automatic security updates are broken and still won't sync with Google Drive.

          Oh yeah, cars. Just had the 'pleasure' of driving a 1980 Trabant. Headlights flicker like crazy, smokes like a chimney, windshield wipers are broken, and it's noisy.

          Yep, guess all cars must be bad then.

        3. Anonymous Coward
          Anonymous Coward

          Re: QA Prestige

          > Oh yeah, Linux. Just had the 'pleasure' of setting up a Lubuntu system. Default document editor flickers like crazy, prints half size, automatic security updates are broken and still won't sync with Google Drive.

          I call Bullshit and you are TheVogon.

        4. ShamballaJones

          Re: QA Prestige

          To be frank, the problems you describe are more indicative of a lack of the requisite knowledge on your part than of shortcomings in the OS.

      2. JohnFen Silver badge

        Re: QA Prestige

        "Is it annoying at times, yes"

        It is -- but when I am annoyed with problems found, I am not annoyed at the tester -- I'm annoyed that we failed to catch the problem before the tester began work.

        "as developer I would rather be writing new code than fixing my or someone else's mistake."

        Over the decades, I've learned that there are many different temperaments in developers. Some devs (such as yourself) love writing new code, designing new approaches to problem-solving, and so forth. Others love the puzzle of digging into malfunctions and resolving them. A great software house has both of these types (and let them each focus on the sort of task they love).

        Personally, I'm a third type: I love producing high-quality software, and don't really prefer either writing new code or fixing broken code. It's all good. Although there is a special sense of accomplishment involved in producing new code that is well-engineered enough that it frustrates the testers in their goal of finding what's broken!

      3. LDS Silver badge
        Facepalm

        "Chromebook, Mac, Linux box I would consider"

        You would consider a Chromebook while calling MS "slurp"? ROTFL!!!!! Where do you believe the idea of all the slurping in Windows 10 comes from? The worst thing Nadella did was turning MS into a a Google wannabe. Not surprisingly, all Google software is in a perpetual "beta" state....

      4. Doctor Syntax Silver badge

        Re: QA Prestige

        as developer I would rather be writing new good code

        FTFY

  4. JohnFen Silver badge

    Peter Bright

    "This bright new dawn was lauded at the time by Peter Bright at Ars Technica in a piece titled "How Microsoft dragged its development practices into the 21st century"."

    This would be the same Peter Bright who recently wrote that bizarre article talking about how rapid release isn't really the problem with Windows.

    I think that it's safe to say that his judgement is questionable.

    1. Destroy All Monsters Silver badge
      Flame

      Re: Peter Bright

      It's not like most of the stuff the IT press puts out is faddish crap meant to fill space.

      On the level of Krugman op-eds in the NYT.

    2. Dan 55 Silver badge

      Re: Peter Bright

      File on the same pile as Paul Thurrott.

    3. Rob D. Bronze badge
      Thumb Down

      Re: Peter Bright

      Also safe to say that the Bright article referenced was about Agile generally not so much about testing, didn't make any reference to 'crowd sourced' anything, and doesn't contain the quote somewhat sloppily implied as being attributed to Ars (but is in the ZDNet article, which in context casts the point about laying off testers as a bad thing not as an agreed good point).

      As it happens the Bright article referenced was of its time four years ago i.e. too much jumping on the Agile bandwagon and not enough demonstration of real world experience to appreciate the nuances (so yes, it was/remains a terrible article).

      But the reference in this Ostrowski article to the ancient history in Ars/ZDNet was almost completely inappropriate, and the attributions/quotes were a misrepresentation of the Bright/Foley content anyway. It all came across as petty sniping between hacks without adding any substance to the undeniable issue of the mess Microsoft are making of Windows updates (and patching) with this switch to forced, frequent updates (and using the customer base for testing).

      1. Dan 55 Silver badge

        Re: Peter Bright

        Not sure where the misattribution is, you just need to look for the words "Foley wrote" and "Bright wrote" in the article.

        If you wanted a smoking gun "MS is doing everything right" cheerleading quote, you could try this:

        The integration of testing and QA into part of the regular development process, instead of the old test and stabilization phase, means that the code quality is always decent and always shippable.

        That's why they've just released the operating system equivalent of a car crash.

        The layoffs were even flagged up in that article:

        one victim group appears to have been the dedicated programmatic testers in the Operating Systems Group (OSG), as OSG is following Bing's lead and moving to a combined engineering approach. Prior to these cuts, Testing/QA staff was in some parts of the company outnumbering developers by about two to one. Afterward, the ratio was closer to one to one

        But then absolutely no thought was made to what this meant about the quality of the operating system and, in the context of the whole article, it doesn't really matter because Agile.

        The entire gist was Agile is great, MS can release faster and more often, Devs can somehow do better code if they test it by themselves, QA can somehow test better when they don't have to do automated testing but just "real world" testing (i.e. click and hope for the worst), and the party poopers holding everything up aren't needed as much any more because Agile. Under that methodology, testers got fired because they're less important to the process and we know they got fired because MS announced it.

        Whilst it might work for Bing where they can do fixes in production, it's useless for an operating system where every upgrade means a one-time installation process that irrevocably changes users' data. I bet you could have counted the number of devs who unit tested running just one update scenario with the latest build on the fingers of one hand and QA didn't go through all the possible update scenario combinations because they're only doing "real world" testing which misses a load of stuff because the real world is much bigger than Redmond.

        1. tiggity Silver badge

          Re: Peter Bright

          Devs should produce their own test software - and this should run as part of "build / deploy" process

          However, these tests should not be the ONLY automated tests, QA should be involved in adding some.

          If Dev A has misinterpreted / omitted something in the brief then Dev A tests will omit them too and so although the code passes Dev A tests it does not match the brief and should fail proper tests.... Which is why you need QA to add their own independent test suites - some automated. some manual (we have not reached the stage where all tests can be automated yet)

        2. Updraft102 Silver badge

          Re: Peter Bright

          Devs can somehow do better code if they test it by themselves,

          That one always seemed too ridiculous to be taken seriously.

          If the code generated by x hours of programming require y hours of testing, how is reducing a programmer's hours of x and making him do y for part of his work week instead supposed to help save money? If he's programming a third as much because two thirds of his time is now testing, and so are all the other programmers, then they need three times the programmers they used to have to achieve the same output. It's still the same amount of labor per line of code if that testing is actually being done. The only difference will be that they replaced the relatively cheap testers with programmers for testing duties, and that doesn't save money.

          The entire argument about "test your own code" seems to forget that time is a scarce resource. For the people who actually believe it, of course. It isn't the programmers themselves doing the testing.

          1. Claptrap314 Bronze badge

            Devs MUST test

            1) If you are aware of the costs yourself, you are much more likely to avoid mistakes in the first place.

            2) If you write the test before you write the code, you are MUCH more likely to write proper code.

            3) If you write the test before you write the code, you are MUCH less likely to write code with needless functionality (which will break needlessly and inopportunely.)

            Sorry if you haven't learned these things yet, but untested == broken. Far more time is lost sending code back & forth than would be taken by having the devs do TDD.

            The test team should be made up of senior devs whose minds are sufficiently warped to think about the nasty things that should not happen but do.

            1. jglathe

              Re: Devs MUST test

              You need to have an age rating on TDD, though. No programmer with under 10 years of experience should even consider it.

            2. Dan 55 Silver badge

              Re: Devs MUST test

              Nobody's arguing devs shouldn't test their own code or that code should be bounced back and forth between the dev and QA. People are arguing against that devs taking on many of the responsibilities of QA and QA being stripped to the bare minimum. If that happens you might as well test it in production.

          2. Doctor Syntax Silver badge

            Re: Peter Bright

            "The only difference will be that they replaced the relatively cheap testers"

            Pay peanuts and....

    4. J27

      Re: Peter Bright

      The same can easily be said about this article. The Register is easily the most anti-Microsoft major tech site.

      1. Pascal Monett Silver badge

        El Reg is against sloppy software. It's not El Reg's fault that Microsoft is now in that category, nor is it El Reg's fault that all other "major" tech sites are brown-nosers.

        Remember the "Biting the hand . . ." line ? That's what El Reg is for, and Microsoft deserves to be bitten.

        Hard.

        1. Michael Habel Silver badge

          I hope the Vultures don't get an upset tummy after bitting on this carcus.

          RIP Icon is MIA.

      2. Piro

        Re: Peter Bright

        I'd say they stand against broken software, regardless of who's peddling it.

      3. Franco Silver badge

        Re: Peter Bright

        "The Register is easily the most anti-Microsoft major tech site."

        Clearly you have never read The Inquirer.

        El Reg is (IMO) one of the less biased sites, pretty much every major company who deserves a public slapping (which is most of them at one time or another) gets one.

  5. usbac

    "It's entirely possible that the absurd breakneck pace of change we're seeing masks a complete breakdown in Microsoft's ability to produce reliable software," wrote Woody Leonard. "All I know for sure is that Windows is on a vicious downward spiral."

    Two comments:

    1) Did Microsoft ever really produce reliable software? If they did, I don't seem to remember it. And, I've worked in IT for 27+ years.

    2) The "vicious downward spiral" started about 20 years ago.

    I don't think any of these things are a revelation?

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019