back to article Deploying software every day is... actually... OK – what devs tell their real-life friends

“High-performing organisations” which have employed agile and devops methodologies are decisively pulling away from their fuddy-duddy peers in the number of software deployments they can manage. But while the idea of deploying software 200 times more frequently than low performing companies might fill some developers with …

  1. Anonymous Coward
    Anonymous Coward

    Military Platform - back end

    Takes years to get new software installed.

    They take their time "to eliminate the possibility" of bugs making it onto the fleet.

    But the fleet software is still laced with bugs, in spite of their caution.

    Then of course it takes years to get the simplest bugs fixed.

    It's a monumentally stupid process. Being slow and cautious adds vast amounts of negative value.

    So yes, let's pick up the pace. Doing so would indisputably make the end result better, faster, cheaper.

    1. Rich 11

      Re: Military Platform - back end

      Or they're just crap at design, testing and debugging. If they don't fix those problems first, picking up the pace isn't going to help much.

      Indisputably

      Not a good choice of words.

      1. Anonymous Coward
        Anonymous Coward

        Re: Military Platform - back end

        "Not a good choice of words."

        Actually it's the precisely correct word.

        Realize that I'm reporting this as an involved party.

        There's no question at all that they're doing it wrong. None whatsoever.

        Their approach delivers the negatives and none of the positives.

    2. Anonymous Coward
      Anonymous Coward

      Re: Military Platform - back end

      "So yes, let's pick up the pace."

      Agreed. The military is only in charge of the nuclear weapons, high-performance jets, large floating cities (aka - aircraft carriers), nuclear subs, a large fleet of satellites, and numerous other tanks, boats, choppers, artillery pieces, etc. Oh, and their people like to get paid regularly. They should be able to roll out at least two updates a day to all of that stuff once they drop the old-school mindset that the stuff is critically important or needs to be tended to cautiously. Who do they think they are, anyway?

      1. Anonymous Coward
        Anonymous Coward

        Re: Military Platform - back end

        Two updates a year would be about ten times the rate that they're achieving.

        Disagree if you wish, but then you'd be wrong.

        Their process is indisputably daft.

        All negatives captured, failed on the advantages.

  2. BoldMan

    Multiple deploys a day? I'd hate to be a custom of one of those companies!

    Fixing bugs is important but so to a great many people is stability. How many of those deploys are required to fix a problem caused by a previous deploy earlier in the day because it wasn't adequately tested? Oh testing, that something someone else does, devops are too cool for PROPER testing...

    Less haste, more speed.

    1. Wild Bill

      Where does it say that the work goes out without testing? Just because there are several deployments per day doesn't mean the work was done on that same day. The work for each deployment will have been done a few days earlier, gone through manual testing and then gone through integrated testing before being released

      1. John 104

        Testing

        @Wild Bil

        In my experience in a DevOps shop, there was less and less testing as the methodology "matured." If it worked on the dev's workstation, it must be good. Put it in production and suddenly there are all sorts of issues because the dev had x version of .Net installed, but production could only run y version for one reason or another, or the data set on the workstation was small, but the production set was 10 times larger, or more.

        To me, this is the far swing of the pendulum. As stated above, customers value stability. Yes, they want all the new features, but bring the system down too often and they will look elsewhere for their needs...

        1. herman

          Re: Testing

          Now try to do seismic data processing that can run for 18 months on a system that gets restarted twice a day.

    2. Joe Montana

      Bugs..

      When you're doing frequent deployments, each change is relatively small and easy to test... Not only that, but no matter how much internal testing you do there's no substitute for actual user testing - bugs will always be found once the users get their hands on.

      But if you're doing small changes and regular deployments, those users will find the bugs while the developers are still very familiar with the recent changes which makes fixing things much easier.

      If you infrequently deploy massive changes, then each change will quickly result in a large number of bugs, the developers may not have touched the affected areas for a long time, are likely to get overwhelmed by a sudden flood of bug reports and if the fixes take a long time to be pushed down users might get used to working around bugs instead of reporting them and having them fixed.

      1. BoldMan

        Re: Bugs..

        "But if you're doing small changes and regular deployments, those users will find the bugs while the developers are still very familiar with the recent changes which makes fixing things much easier."

        But its NOT THE JOB OF USERS TO FIND BUGS!!! That is the job for proper QA which all this devops bollocks totally misses the point of. If you are making multiple deployments a day, there is no way this can be adequately tested!

    3. a_yank_lurker

      @ Boldman - Your suspicion of the frequent installs covering bug fixes is probably accurate. What is needed is to allow devs enough time to write quality code and have adequate testing.

  3. Electron Shepherd

    Practive What You Preach?

    So, the report, produced by Puppet, says that low performers are those achieving a deployment between once a month and once every six months.

    This is from a company that has released two versions of Puppet Enterprise since the start of the year.

    If releasing every day is such a good idea for production software, why don't they do it?

    1. Anonymous Coward
      Anonymous Coward

      Re: If releasing every day is such a good idea for production software, why don't they do it?

      DevOps seems to be all about telling other people how to do their jobs.

    2. Joe Montana

      Re: Practive What You Preach?

      Because puppet "enterprise", the version catering to users who don't want frequent updates.

      You have the option of tracking the puppet github:

      https://github.com/puppetlabs

      which seems to be updated very frequently

  4. Wupspups

    "One possible explanation is that low performers ignore critical rework so they can push new features, but it’s highly likely this is at the expense of racking up technical debt.”

    Or maybe they are actually sorting the bugs out and doing proper testing and not introducing new bugs.

  5. scrubber
    FAIL

    I can vouch for this...

    'They found that those “high performing companies” typically spent much less time fixing mistakes'

    As I explain to customers why they shouldn't care about the bugs they raised because: look, shiny new features...

  6. Anonymous Coward
    Anonymous Coward

    Redefining failures as success

    Multiple deployments a day? Why is this necessary unless previous deployments had significant issues? How is this more efficient especially when taking into account knock on effects on the users of a constantly shifting software environment. What sort of level of thought is going onto what features make sense, how they inetract, work flow, usability ?

    This sort of frenetic pace is the mark of failure and mass organisational delusion.

    1. Wensleydale Cheese

      Re: Redefining failures as success

      "Multiple deployments a day?"

      It sounds like what I know as "Continuous fire-fighting" i.e. putting out lots of small fires rather than addressing the cause.

      It can be satisfying because it keeps you busy, busy, busy, but ultimately, how productive is it?

      1. herman

        Re: Redefining failures as success

        Ayup, there is a huge difference between being busy and doing work. Devops is all about being busy.

    2. Anonymous Coward
      Anonymous Coward

      Re: Redefining failures as success

      It's not so much redefining failure as success it's more a case of counting differently: instead of one release that fixes 20 bugs, it's now 20 releases that fix one bug.

      1. Joe Montana

        Re: Redefining failures as success

        And 20 opportunties to make sure that a fix for 1 bug doesn't introduce any new ones...

        1. BoldMan

          Re: Redefining failures as success

          And 20 opportunties to make sure that a fix for 1 bug introduces 20 new ones...

          FTFY

    3. Mellipop

      Re: Redefining failures as success

      No, it's not failure. That's the point of the study. Companies that have a devops culture are successful.

      I would guess the luddites here have never known the advantages of rapid customer feedback cycles you get with devops.

      You probably think it's your God given right to pronounce on the quality of your delivery ignoring the high probability of a user breaking it in a way your defined process control never imagined.

      Welcome to the real world of real engagement with the customer and real responsiveness. It's not bollocks.

  7. asdf

    I'm that guy

    tl;dr, made fapping motion after first paragraph, disappointed enough to waste valuable work time (hahaha) to post complaint. Is it Friday yet?

  8. Anonymous Coward
    Anonymous Coward

    More DevOps Bollocks

    Meanwhile, the whole point of computers is scale - do it once, properly, and the machine will do all the rest. Here in the real world, bollocks to ongoing deployments; I sit quietly at home, writing stuff in a leisurely fashion which somehow is currently processing billions of records every day in a goodly chunk of the world's major enterprises (sometimes a slightly scary thought). I guess DevOps is the current "Silver Bullet" (pace Fred Brookes).

  9. Anonymous Coward
    Anonymous Coward

    Could have confirmed this years ago. Low rate workers and a few high rate workers always get praise and rehired. It's the rest that struggle and don't realise it's not the quality of their work that matters that get a raw deal.

  10. BitterExScientist

    High performing companies perform better than bad ones

    Another "If we sort companies by performance, we find that the best ones perform far better than the worst ones using the the same metric ( eg power law distribution)" tautology based article?

    How about making a nontrivial predictive statistical inference and showing other possibly confounding factors... For example, firmware, vs systems vs web are entirely different beasts.

  11. Mike Lewis

    Daily deployments?

    I can see a problem doing daily deployments with safety-critical software that must be certified every time it is released. Mission critical could be a problem too. At Telstra's Australian EFTPOS network, we were not allowed to deploy any new software at all in December and January, our busiest time of the year.

  12. Oengus

    Can't be bothered with bugs

    Everyone wants to do "new feature" work and no one wants to be a maintenance coder.

    We have some packages that are updated weekly. The developers can't be bothered looking at reported bugs in the system. You raise a bug report, no one gets assigned to look at it. 6 months later they ask if you are still having the problem (usually at Pub'O'Clock on a Friday) and when you don't get back to them over the weekend they automatically close the ticket.

  13. Anonymous Coward
    Anonymous Coward

    Request (sorry, 'story'): DevOps flag on articles?

    Please, please, please, El Reg, can you start clearly marking all DevOps puff pieces with DevOps in bold at the start of the heading? that way anyone who actually knows the slightest thing about development in the real world knows that they can ignore that article because it's just another press release from Puppet rewritten as if it was a Reg article.

    Puppet are obviously practicing wheat they preach though, seeing as how they bang out a new 'release' every time they change a bit of punctuation (and The Reg dutifully copies it out and puts it up as 'news').

    Thank you, carry on.

  14. Roland6 Silver badge

    Win2003 and other legacy OS's

    There was no indication whether those organisations running a 13-year-old operating system were high, medium or low performers.

    Clearly, given the basis of the report and it's criteria for low, medium and high performance, they will be among the low performers since they won't be pushing updates to these systems unless MS are still providing support for Win2000 for those willing to pay...

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like