back to article Continuous integration platforms are broken – here's what needs fixing

About half of 3,880 participants in the 2016 "State of Agile" industry survey reported using continuous integration. Now new research from Oregon State University and University of Illinois suggests that the platforms still leave a lot to be desired. The survey of 523 developers from more than 30 countries found that their top …

  1. Anonymous Coward
    Anonymous Coward

    Wouldn't it be better all round if developers stopped checking in buggy and crappy code in the first place. Still, that would involve teaching the current batch of college leavers that coding an application doesn't involve cutting and pasting snippets found on Stack Overflow.

    1. Anonymous Coward
      Anonymous Coward

      ...which is exactly why you have a well defined CI pipeline that prevents buggy code from being committed.

      No one "checks in" anything any more.

      1. Anonymous Coward
        Anonymous Coward

        re: No one "checks in" anything any more.

        They do, but it's gated so it doesn't break the build or any tests.

        Ideally you get someone who knows what they're doing to write the tests for whatever the student new kid is writing, then he can't commit until the tests pass.

        1. Jonathan 27

          Re: re: No one "checks in" anything any more.

          Not breaking the build is kind of a low bar for software quality and unit tests can only really test for things the developer can envision, and most problems with things I've written come from situations I'd never have thought of. This is absolute bare minimum.

          I'm not saying continuous integration is bad, just that's not fool proof. Even after code reviews and gated check-ins you can still have problems.

        2. Anonymous Coward
          Anonymous Coward

          Re: re: No one "checks in" anything any more.

          "They do"

          That was more a jab at the archaic language. "Checking in" is a relic of the bad old days of global locks and expensive branches. We commit and we merge and we patch, we don't check in.

        3. Anonymous Coward
          Anonymous Coward

          Re: re: No one "checks in" anything any more.

          Ideally, says it all.

    2. Lysenko

      If you walk calmly down the street and get where you're going, no-one pays any attention.

      If you walk down the street, fall on your ass and then perform a Bruce Lee/Kung Fu ("Ninja!") backflip to correct the problem then people say "Wow! That guy is Agile!"

      Peer recognition is therefore predicated on first falling on your ass.

      1. Anonymous Coward
        Anonymous Coward

        Funny, but true.

        Agile just lets shoddy thinking overrule tried and tested techniques developed over decades.

        Can someone quote a firm that does it well?

  2. Anonymous Coward
    Anonymous Coward

    What? The snake oil not working?

    Perhaps a new brand of it is needed?

    Here cometh the new and improved

    CI Snake Oil.

    Everything you learned before is useless. This 'Brave New World' is where is it at. All those not on board are luddites.

    Sceptic? What do you think eh?

    1. Anonymous Coward
      Anonymous Coward

      Re: What? The snake oil not working?

      Original Chinese Snake Oil contains 20 percent eicosapentoaenoic acid which has strong analgetic and anti-inflammatory properties. It's Nurofen!!!

    2. Anonymous Coward
      Anonymous Coward

      Re: What? The snake oil not working?

      This time it will be different!

    3. AdamWill

      CI is fine

      Well...I'm as sceptical as anyone about the cults of Agile and DevOps. (Both have fairly reasonable ideas at their core, and the people who came up with Agile were mostly perfectly sensible software developers who just wanted to do some software development, but of course they got taken over by a bunch of charlatans and turned into ridiculous management cults). But CI (properly understood) is really a pretty simple and sensible idea that doesn't *need* to be tied to either of the cults. It's really just this:

      0. Have some tests.

      1. Don't take any code that doesn't pass them.

      ...that's it. That's the whole thing. And it's really not a bad idea - especially since adopting CI forces projects which hadn't done 0 yet to do it, which is a very *very* good thing.

      Your code really *should* have a good test suite that's easy to run. And once you've got that, it's really not difficult at all to set up some hooks to run it whenever someone wants to merge code, and reject it if the tests fail. It's simple to understand, it's not that difficult to set up (especially in ecosystems where there's a kinda standard way of doing it, like github+travis), and it really *does* help.

      It's not a panacea, of course not, and anyone who tells you it is should be laughed at. It doesn't replace, you know, planning, or integration testing, or lots of other things you should still be doing. But it sure does help.

      (Sadly there does seem to be a nascent Cult of CI developing, which at least from where I'm sitting mainly seems to be based around completely misunderstanding what CI actually is and applying the term to all sorts of other stuff.)

  3. Anonymous Coward
    Thumb Up

    Here's what I'd like...

    1. The build server or whatever should be a detail I don't really need to know about. The running instances of developer IDEs should elect one of themselves to run a master build somewhere. This means all developers have to use the same IDE. I'm fine with that. If no-ones developing, then the master build can go idle once the current build is done.

    2. Build errors from the master build should look as much like local build errors as possible, except more serious, indicating who last pushed, and illustrating how the pushed code differs from local code. This could get intrusive I realise, so needs some clever UI (I.e. not designed by me). I don't want separate emails and stuff; it should all be in the IDE.

    3. One or more developers could elect to fix the build, and this would be highlighted in everyone else's IDEs. This step would be optional; sometimes many people need to fix things and it's easier for them to get on and fix things.

    4. I would like to elect one environment to be the 'run lastest build' environment. The master build system would magically deploy to this environment from time to time, bouncing the environment as needed. I might wish to elect multiple environments for this: one to be updated on every successful build, one to be updated every week, one to be updated only when I say so.

    Successful build means all tests pass too of course.

    1. Jonathan 27

      Re: Here's what I'd like...

      Visual Studio + TFS supports almost all of your points there.

      1. The build server is still a separate thing, but it uses the exact same compiler and generates the same error messages as a local build.

      2. As above, they're the same messages and are delivered to whoever pushed to the build server, which can reject the build (gated build).

      3. You can send selfsets to other developers, I think that might cover this.

      4. You can set TFS to deploy to a set environment after every check-in, or at intervals and you can do it multiple times so that covers this entirely.

      Requiring tests to pass also an option.

      Anyway, my point is that Microsoft's tooling already supports 99% of what you want to do here. I'm really surprised this isn't common across the board. What are you using? Subversion? CVS? TI Pocket Calculators?

      1. SVV

        Re: Here's what I'd like...

        Using Subversion and Jenkins provides much the same as this for the Java world. Jenkins runs the build and tests continually, sucking in new code from Subversion at configurable intervals.

        For a team of 5 or more developers, this is an optimal way of working, as any nasty conflicts caused by non co-operating checkins get caught (and solved) quickly and without fuss (none of the devs roaming the offices complaining "who broke this?" withe the corresonding waste of time)

        Of course it's never going to catch every problem - your quality assurance is only as good as the tests you write - but the whole team found this to be really genuinely hellpful on a fairly large project and it certainly enabled us to get it finished more efficiently.

        Negative points? Well, it wasn't straightforward to set up and configure, and some better integration linking back into developer IDEs to make it really seamless would of course be much nicer, but from my experience, I wouldn't describe it as "broken". Suboptimal but genuinely useful is more like it.

        1. Anonymous Coward
          Anonymous Coward

          Re: Here's what I'd like...

          Using Subversion and Jenkins provides much the same as this for the Java world.

          But I want it so tightly integrated with the IDE that I don't need to know the thing's called Jenkins. I know there's many tool sets out there that 'kind of' do what I want, but 'kind of' isn't good enough any more, and seemingly apart from Microsoft, there's nothing yet with the deep integration I want.

          Finally, I'd just like to send a brief message to whoever came up with Maven: Please stop coding for good.

      2. Anonymous Coward
        Unhappy

        Re: Here's what I'd like...

        I'm really surprised this isn't common across the board.

        Be surprised. It isn't.

  4. Buzzword

    Stability

    I just wish we had some stability. Ever tried writing for the browser in 2017? The toolchain keeps changing. We had jQuery, then React, then Angular. Building it required Grunt, then Gulp, SystemJS then Webpack. Modules went from AMD to UMD to CommonJS to something else. Package management moved from Bower to NPM to Yarn and back to NPM again.

    Examples and quickstart projects which compiled perfectly six months ago no longer work because somewhere a package was upgraded. Answers on StackOverflow are irrelevant because they applied to version 1.3.5, and you're using version 2.7.1 now. Bugs on Github are left open, with not even a WONTFIX comment from the developers.

    Not so long ago I remember a world where major tools were upgraded every 2-3 years. Windows XP was standard for over a decade. VB6 even longer. We had years to learn the quirks of IE6's "quirks mode". Nowadays if your Github project isn't on a six-week release cycle, it's abandonware.

    I'd take a sabbatical if I could, but my skills would be worthless by the time I return.

    All of the above problems are magnified on a CI system, because half the tools don't log to stderr or set the error code correctly.

    Edit: this guy nailed it: https://hackernoon.com/how-it-feels-to-learn-javascript-in-2016-d3a717dd577f

    1. andy 103

      Re: Stability

      I work as a web app developer so can sympathise. But to me, a lot of the problems we see are not to do with things like jquery vs React, etc. They're to do with the fact everyone is so focused on "automation", version control, CI, or whatever other bullshit buzzword they've read about. And whilst focusing on these they have dropped "old fashioned" core principles like planning and testing (previously known as "common sense" and "what you should be doing anyway").

      You want to build a web app for a browser in 2017? It's not that hard. Write a detailed plan of what you're going to do. If you get some tosser saying it has to be done in a "agile" way, stay away from them, because they've no idea what it really takes to deliver software successfully. As you go along, throughly test things - yes, manually *shock horror*, with human input and interaction. Once you get to the end and have completed your final fixes/tests, it's unlikely to "break" unless either the code is randomly changed without testing (which it shouldn't be) or a *new* browser that wasn't around for your testing at the time comes along. In the latter case it may never cause an issue and should be a relatively minor one if you've planned and coded the application well.

      Even now I bet someone will downvote this or say, oooh why would you use manual/human processes when there is *insert new tech in this space*. Well, it's because some of us know how to do things properly, which is why I'm able to take some time to write this at lunch, rather than piss around configuring a CI tool.

      1. Anonymous Coward
        Facepalm

        Re: Stability

        andy 103: 'If you get some tosser saying it has to be done in a "agile" way, stay away from them, because they've no idea what it really takes to deliver software successfully .. Even now I bet someone will downvote this or say' ..

        Going on the downvotes you got here, there's at least two self identified tossers on this forum :)

      2. Adam Connelly

        Re: Stability

        I'm not going to downvote you, but I do disagree. Personally I think you should be properly planning and testing as you say AND using automation. They're not mutually exclusive.

        Configuring a CI tool really isn't rocket science. If you're a software developer, and you understand the concept of taking the source, converting it to a working product, and ideally running some form of tests against it then you understand how to configure a CI tool. There's nothing magic about it - it's just a case of taking what you already do manually and getting the CI tool to do the same thing.

        There's a ton of different reasons for using CI, ranging from getting immediate feedback about problems when working in a team through to ensuring consistency when creating builds. If you deploy directly from your development machine rather than a CI system, for example, you're asking for trouble. I'm not saying you'll definitely have issues, but there's at least the chance that you make a mistake and accidentally leave in debugging statements or work in progress that shouldn't be there. Obviously you shouldn't do this, and you should 100% test everything you do, along with every possible thing it can affect, but people aren't perfect and make mistakes. If you use a CI system these kind of issues pretty much go away.

        So yes, you're completely correct that people should properly plan their work, and then make sure that they test it properly by hand. But that doesn't also prevent you from using automated testing and CI systems. Both are part and parcel with professional software development to my mind.

        I'm able to take the time to reply to this over lunch because I spent the time 5+ years ago configuring a CI tool and rarely have to worry about it.

    2. Jonathan 27

      Re: Stability

      What I love is how npm package updates ALWAYS break something. Half the time the developer of the package has made so many breaking changes you have to totally re-write the code that calls that package. I wish I could say we're writing wrappers around every 3rd-party plugin now, but we're not...

      This is annoying for front-end JS, but if you're using Node.JS it's horrible. Your entire application can require days of reworking when you update a single package, which then causes all of it's dependencies to update. And even simple Node applications have 50 - 100 packages because of dependencies.

  5. malfeasance

    CI is just software development in disguise

    CI accrues technical debt just like any software development exercise; you made a choice to do things one way, it turns out less optimal than you wanted, so you either gaffer tape it or you rewrite...

    We have a situation where we're building our application + modules as rpms; base.rpm + opt-a.rpm + opt-b.rpm and so on. So you can just install all the stuffs with our custom yum repos via puppet etc.

    It's working quite well with their initial use case; and it's a hell of a lot better than what they had before (i.e. no CI at all)...

    However, what the CI people haven't quite grasped is that opt-a.rpm might have conflicting dependencies with opt-b.rpm; it's entirely possible that if we need to install base + opt-a + opt-b, it might not work because we're in still in package dependency hell (and thus was it ever like this, with DLLs, jar files etc).

    The pragmatist in me wanted the CI to happen, what we have is better than what went before; but now they have got their toes wet, we all need to revisit the solution and sort out the technical debt.

    Fat chance of that and so the story goes on...

  6. Anonymous Coward
    Anonymous Coward

    Yet another Puppet press release

    Typed up as 'news'.

    Is there some constitutional imperative that every time Puppet put out some guff about DevOps, someone from the Reg has to write it up as if it was news?

  7. Anonymous Coward
    Anonymous Coward

    No Excuses

    Cem Kaner hired a course designer, wrote up, video taped, and gave away for free all the testing education a developer could ever need.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like