Teams that "can be fed with two pizzas"
What, I have to work on my own?
Dilbert might mock the mythical man month, Fred Brooks’ argument that “adding manpower to a late software project makes it later,” but most enterprises still think they can hit their deadlines by hiring more people, feeding ever larger teams, rather than by embracing DevOps-friendly practices that favor small teams and high …
In a previous job the brown stuff hit the fan and one of the teams was in working throught the weekend. When we got in on Monday morning there was a stack of empty pizza boxes the height of the coffee machine (6 ft?) including some of the mega-24 inch jobs! Not the healthiest of lifestyles...
...is a recent buzzword invented by PowerPoint Ninjas, Conference Organisers and Consultants to describe a methodology for siphoning money from PHB training budgets.
More specifically, it is a fusion of:
* Continuous Delivery (A concept borrowed from Sewage Engineering)
* Agile (Borrowed from Gibbons)
* Talking (Borrowed from Homo Erectus)
* Selling Magic Tools (Borrowed from Elves and Dwarves)
* Beta Test in Production (Borrowed from early Egyptian Pyramids)
* Unit Testing (Borrowed from Smalltalk circa 1998)
...in isolation none of these strategies can shift tickets for an undersubscribed conference or bag £1000 a day consultancy fees, but combined with "thought leading"
obfuscators educators there is hope they might address pressing business needs to reduce training and tooling budgets (by giving away money) and reduce customer complaints (by making failure the new standard).
Whilst it's easy to mock, there are plenty of big 'experienced' IT companies who still cling to the separation of techops and developers, to the point where neither side actually understands what the other side has done to break something.
There are more than a few admins who take pride in very deep, but silo-ed knowledge that allows them to tune a server farm down to the bare metal, but doesn't allow them to predict that the next deploy is going to hammer the network into the ground.
Similarly, there are plenty of developers who 'fire and forget', assuming that the magic servers will cope with whatever lunacy they've come up with.
If the sum total of your big and expensive deploy is half a dozen webapps on a handful of load balanced servers, that might not matter so much. Let's be honest, its surprising what scale of business that sort of platform is capable of supporting.
On the other hand, if you're using disparate integrated systems, want to have a testing environment that vaguely resembles your production environment and are keen to buy into the continuous deployment process, then having people who are able to cross the line and get their hands dirty on both code and metal is probably a good thing. Give 'em a name so we can identify them as not being a generic undertrained IT guy.
"there are plenty of big 'experienced' IT companies who still cling to the separation of techops and developers"
Once you have more than one application (should I call it an 'app'? Only if I go and wash my mouth out with soap (SOAP?)), the chances of having someone who can keep abreast of multiple applications, changing hardware, network and traffic needs &c is heading to nil.
Hmmm. The thing is, Ops don't really get "down to the metal" either. How do you know your server farm running the new code isn't going to pop your breakers or overload HVAC capacity?
Real "down to the metal" is knowing what diameter copper you need in a 64A power lead ...and that's a Facilities job. Wouldn't it be great if they were on the same page too?
Yes!! I have it!! Ladies and Gentlemen, I give you:
...I'm off to organize a conference !!
Why not call also other essential stakeholders (can I still use "stakeholders"?) like pizza or chinese/indian delivery restaurants, people that clean the office, Human Resources Humans, Company Lawyers, Middle Managers, Facilitators/Motivators/Intermediators?
Just call it PizChiIndCleanHRHParasitesMoreParasitesOhLookMoreParasitesFacOps.
> [...] there is hope they might address pressing business needs [...]
What surprises me is that people keep falling for it time and time again. Remember Agile™, Extreme Programming®, and bollocks like that?
In the meanwhile, it seems that the only ones coming up with actual products are start-ups, which then end up getting bought by Big Corp.
"Devops" is an inanity that says "move fast and break things" is a plan for a large organisation with integrated systems. It's central tenet is "if you say the same rubbish over and over and call it a strategy, pointy haired managers will buy it". It's a risible attempt to sell services for non-existent problems with solutions powered by hopium and makeyuppyum.
The last time I heard this much drivel, someone was selling UML and Rational Rose...
The company I work for has supped deeply of the DevOps nipple. Now instead of one group triaging problems, we have six, with five of them touchy feeling "making sure we're all on the same page" idiots.
I don't know from what I can work out Dev Ops seems to boil down to people actually doing their fecking job.
I sort of feel like when I have seen this much buzz about it and no one still really knows what it is, then;
a) its probably something thats been given a cool marketing name (serioulsy one of those marketing people it's going to change his name by deed poll to max power one day and the rest will probably orgasm to death at how cool it will be for clients to hire someone called that).
b) the management are going to spend a fortune hiring someone to come in and spout a load expensive stuff that at best they have been told already but ignored because it was being delivered by their own staff at a cheaper cost, or at worst is complete bollocks.
We have a dev ops team that were given a project by some colleagues who had previously written some basic file conversion apps. They were now "doing the right thing" by company policy. 18 month and £200,000 later,, Devs Ops had delivered nothing, so my colleagues cobbled something together in a week or so which worked. They freely admit that they are not programmers and it may not be the best structured or supportable code. The reason they could do it so quickly was that they know so much about what they wanted to achieve that they did not have to specify and specify every minor detail and the project to go back and forth.
At home I tend to do most of my own home developments (DIY) for a similar reason. If I try to specify every detail of how I want something done, it tends to scare builders off. I suspect they also don't like being told how to do something - human nature and all that. I end up just getting specific trades (mostly plastering) in for stuff I really cannot do.
But I can see the point in the article that getting things done 'properly' can often take ages and cost a lot when the quick and dirty may get results quicker.
A "DevOps" team that produces nothing in a month (let alone 18months) is not one. In fact, what your business types did by cobbling something together was probably more DevOps than your eponymous non-delivering team.
IMHO, DevOps embraces "Quick & Dirty" but uses established and, to the *appropriate* level, formalized processes to ensure "Not too dirty" with the proviso that the said processes meet "But still quick"
Organisations who think they can get on the DevOps bandwagon just by using that name are like mine --- we can do "Big Data" because we redefined it as RDBMS over a certain size. Of course it's no more "Big Data" than fly, and I think that's what's happened with your DevOps team.
El Reg keeps pushing it and I am thinking that this is more-or-less paid-for (sponsored) content or that they are 'partnering' with somebody and we'll find out why soon, or they are positioning themselves for some new incarnation, or all of the above. Also wonder if some of our departed reporters didn't fit in with this Next Big Thing that appears to be slouching towards Bethlehem.
Well the rest of El Reg serves up some useful and interesting stuff, so it's worth sifting through the odd bit of DevOps bollocks.
Let's face it, the Reg audience probably has a higher-than-average number of adblockers per head and the money has to come from somewhere.
And with a bit of luck, someone will come up with a tool that scans for the word "DevOps" and changes the link colour, or something, to save time.
Well luring us in with pizza was pretty underhand. To be fair DevOps was mentioned in the first paragraph, so not too much time was wasted. Personally I'd prefer it if the word was in the title or subtitle so I get a choice on whether it's worth clicking or not. If time allows, I'd probably click through for the comments anyway (there's some quality snark under these articles); but when you're trawling for actual news and time is tight; it can be quite irritating.
and everywhere the DevOps sorry Snake Oil sales droid started to rejoice. Then the fell down on their knees and prayed to the great mythical god they worship so vheremently.
on the otherhand, those of us who a long in the tooth say, sorry, seen it all before. Didn't work last time or the time before that. Then we get on with delivering what is needed.
It is as if nothing was delivered, ever, before DevOps came along.
Never in the history of software and systems development has so little been delivered by so many disciples of the latest gobbledegook.
Dont understand the heat, perhaps because the latest buzzword bingo has not reached my corner of the IT jungle. Maybe 2 month holiday insulated me also. From what I can glean in nouns and verbs, DevOps is the way pre-1998 IT worked in many Oz Federal gov departments. The business owners and users specified their requirements to the infrastructure people who spoke to the developer who advised the sysadmins and DBAs what was required and it happened. Issues arising like regular OS, firmware and application patching was discussed by the technical staff with the business owner who used to be called a user representative. This resulted in mutually acceptable schedules decided within two short meetings.
Users got what they needed, tech staff got their jobs done and developers were not an enemy of the sysadmins. Did I mention no formal QA, ITIL or process managers were involved ? Lots of testing at dev, sysadmin and user end done before application rolled out. Rare for any issues to break production. So rare I can't think of any. (alright, I'm old) Some of those old apps are still running fundamental roles within these departments because nothing does the job better despite the efforts of outsourcerers.
In my own direct experience projects came in on time and budget mostly.
So is DevOps when working another back to the Future event ?
When I'm having trouble keeping up with the workload, management offers me extra workers (ie someone spare in the office).
Trouble is the work I'm behind with is the work that would take extra work to get the staff trained (and authorised to do while not directly - as in me looking over their shoulder - supervised) to accomplish.
Of course I could train them in a quiet period, but by the time they were needed, they'd have forgotten how to do it.
The project our small team finished a couple of months back had a 4 months lag from programmers starting to 1st useful work. Wish I'd known that in my 1st 4 months, would have worried less about being fired. Every time the client extended the work there was very little point hiring anyone new.
Not necessarily the job security I like though!
"Of course I could train them in a quiet period, but by the time they were needed, they'd have forgotten how to do it."
Reminds me of the time when a County Council bought a magnificent new expensive snow-clearing machine after a winter where it would have been useful. By the next time it was needed the employee trained to drive it had left....
"the time when a County Council bought a magnificent new expensive snow-clearing machine after a winter where it would have been useful"
I remember when Cheshire County Council bought its first snow-blower, to much protest from the Ratepayers Associations etc (yes it was that long ago).
Then a couple of years later there was a bit more snow than expected in parts of the UK that hadn't taken precautions. The revenue from renting out the snowblower to those authorities more than covered the cost.
Share and enjoy.
"Developers need time and space (and pizza) to develop good systems, not meetings, time-sheets, progress documentation and bureaucracy!"
Whereas managers, especially at times of crisis (real or otherwise), are told by Upstairs to deliver "meetings, time-sheets, progress documentation and bureaucracy!"
Hmmm, what could possibly go right.
Best team I ever worked in, w best manager, understood this and understood (based on experience) that the team *would* deliver what they had committed and that any distractions from Upstairs were exactly that - distractions. You can get a lot in the right circumstances from a small number of competent people.
Worst places I ever worked, as things got worse and deadlines got closer, wanted shit like daily reports. Not least because they had lots of barely-competent people (including managers). But how do reports help fix that?
This is a tricky one. What you're trying to do is defend your budget while appearing to save money, announce wholesale change without admitting that everything you did in the past was wrong, and change nothing that will break something. That way you might be able to claim success because something happened, but the ship didn't sink. And you have to sell this to people with the attention span and peer pressure anxiety of a six year old, but without the tech savvy.
Welcome to the jungle, baby.
That's relatively easy as Supermarkets proved years ago.
Every BOGOF ("Buy One Get One Free") amounts to "We've Been Charging You Double for Months, Sucker!!"
No-one ever seems to pick them up on the obvious admission of past bad practice and (even more remarkably) they can go back to price gouging afterwards without any negative press.
 Yes, I know they are often loss leading and cross subsidizing. The point is they don't even have to bother to say so.
I'm a contract developer in a bank. Breaking down the silos in here would require bunker-buster bombs. We can't even get a single dev server and use an old PC hidden under a desk with the login of someone who's left the bank. No one at all knows how all the silos fit together and the tech lags by at least a decade (Win XP for devs, IE8 for all). DevOps? Not a chance.
It's true that as a team gets bigger the quality drops. Also adding people to speed delivery after project started, slows it.
It depends on project size as to when any coding should start. A larger project needs a small design team, which might spend weeks or months not writing code, but designing APIs / module communication etc before detail implementation with full team starts. Then you can't change anything unless it's wrong spec, in which case you have to scrape schedule and reset to design phase.
Small teams do work better, just as small meetings do, but that has nothing to do with DevOps or its principles.
Have a small team do the design, and especially the inter-component interface design, and distribute that to a number of smal teams each of which owns a component. Have the testing also divided into small teams, each of which tests some sub-assembly. You'll get far better results than you would with a single large team trying to do everything, since in my experience such a team will keep "improving" the interfaces, inconsistently. Whether those teams use waterfall, DevOps, or any other methodology is largely irrelevant.
Of course, it takes good management to get those teams working together constructively. All too often "DevOps" is just an excuse for "nobody wants to manage this properly so we'll just let the smart guys do their own thing and cross our fingers". Certainly, with that approach "small teams" makes it easier to spot the fuckups early, but that's no reason to equate "small teams" to "DevOps"
For some reason I keep thinking of the famous paper about there being no silver bullet.
Some "DevOps" practices and tech are useful just as some "big data" tech and practices proved useful from the last bandwagon. Generally, pragmatism tends to be more successful than dogmatism - yes you can't have a baby in a month by getting 9 women pregnant, neither can you deliver a large software project with one developer on a tiny timeframe
"When you want to quote Dilbert, why link to twitter rather than say dilbert.com?"
The recently-returned Matt Asay wrote the article, and the link is to one of his twits, for some reason.
Or as you note, readers could use the Interweb, it's good for disintermediation, and go direct to
"The recently-returned Matt Asay wrote the article, and the link is to one of his twits, for some reason."
Makes sense, using El Reg to improve some sort of influence metric of his twitter account which can then be translated in to more writing gigs. We've all got to make a living I suppose.
F.P. Brooks' The Mythical Man Month should still be compulsory reading even if some of the references are a bit dated (microfiche?).
To make that even simpler, there is (obviously) the Wikipedia article
Or go direct to an apparently legitimate free download
"Or go direct to an apparently legitimate free download"
That's the 1st edition. There was a second edition that celebrated the 20th anniversary of publication. THAT was published in 1995. The original is over 40 years old and we're lumbered with manglements who still don't get it.
I worked in a Software Support Group.
We were a mixture of developers and operators (mainframe) and we supported the OS, the applications, and could also operate the machines. We had our own development systems for testing new software releases, new OS releases and utilities we put together to make things run smoother.
This must have been DevOps as we were a mixture of developers and operators ;-)
Agile used to be "OMG marketing just shipped the demonstrator!"
I think all the Agile and DevOps hype is basically a delivery demonstrator - look, we hacked a small system together over the weekend and it works. Great. Let's scale it and ship it.
Most experienced people know that small teams where responsibilities from requirements capture through to operation are all shared can work very well. The same people also know that the concept just does not scale beyond a certain point.
A spider is more agile than a mouse is more agile than an elephant.
Ability to divert and refocus also matches the size.
Response to an abrupt change can similarly be demonstrated by a simple drop test from a 20 foot tower.
Somewhere there is the ideal team size and composition for the given task.
One size (or strategy) does not fit all.
RE the continuous deployment of DevOps articles here on the reg of late... Just look at the click adds on the side.
Continuous Lifecycle London. Sponsored by none other that The Reg. and Heise Developer.
Deep Devops In The Heart Of London
Like it was said elsewhere, gotta keep the lights on somehow. Of course, if the trend continues, the quality will continue to drop and so will the viewership...
P.S. The lure of pizza was a cheap way to hide a Dev Ops article. Like another poster, I read a few lines into it and went straight to the commentard section.
DevOps seems to be how most games are produced nowadays aka F@ck It, Ship It. We'll let the users find the bugs in production.
We've got the push on for it where I work (outsourcer) and the focus is on getting something out even if it's not working properly just so a milestone is met (and progress payments made.) Some projects are still using waterfall but just getting rid of the documentation part and saying it's Agile/DevOps (even though the server builds don't mean the automation aspect of the methodology.)
On one system I was assisting with we found a security flaw but they didn't want to fix the issue because the customer's CEO was launching their new app to the media and they didn't want to tell them it wasn't ready. We were told it was Agile/DevOps and would be fixed later - it was, after a malware outbreak occurred which had taken advantage of the flaw. Lots of finger pointing for that one...
We've known for decades that "management" (and it's bitch "marketing") and "engineering" are two completely different thingies. The culture of the management track absolutely, categorically, and pathologically refuses to listen to the technical track.
This newly-named "DevOps" continues the trend.
I, for one, seriously hopes this continues ... Cleaning up after various CEO/CFO incompetent technical disaster sessions are helping to fund my "retirement", without dipping into my savings :-)
Biting the hand that feeds IT © 1998–2019