back to article Report: Prez Obama kicks Healthcare.gov contractor to curb for web disaster

The Obama administration has reportedly patted CGI Federal on the back, forced it to pick up its various toys, and led it away from the fail-filled Healthcare.gov site it created and subsequently tried to repair. It's not usually this easy to get out of procurement contracts, but per the Washington Post, which made the claims …

COMMENTS

This topic is closed for new posts.
  1. Captain Save-a-ho
    Coat

    That's low

    How desperate must things have been for Verizon to see the US government ditch then for HP. Oh the horror!

    1. Anonymous Coward
      Anonymous Coward

      Re: That's low

      I'm sure the NSA was standing there and saying, see, that is what you get for not handing everything over.

  2. Anonymous Coward
    Anonymous Coward

    Accenture, where projects go to die. Has anything ever existed after it was run by Accenture?

    1. Philip Lewis

      Hmmm - quite a bit actually. Including some projects with which I had something to do.

      Deutche Börse springs to mind.

  3. Rob 5

    This is pretty much CGI Federal's MO - trouser the money, fuck up the project, then walk away to repeat the process with the next contract. Just ask the Canadians.

    1. Anonymous Coward
      Anonymous Coward

      I don't get it

      Why do governments around the world keep giving business to the same group of usual suspects. It's not like these companies don't have a fine track record of tendering low and then fucking things up.

      1. ecofeco Silver badge
        Paris Hilton

        Re: I don't get it

        Patronage.

        What's not to understand. The well connected live in a totally different world than you and I and while they call loudly for reward of merit, they do not apply this standard to their "peers", ESPECIALLY with all things government related.

        You see, in capitalistic countries, government exists to benefit business and not people and only a dirty, dirty commie sees anything wrong with this.

        Obvious example ---------------------------------------------->>

      2. Matt Bryant Silver badge
        Boffin

        Re: AC Re: I don't get it

        "Why do governments around the world keep giving business to the same group of usual suspects...." Well, they gutted out their own IT departments (or paid them so poorly they ran to work for the consulting companies). These "cost savings" mean they are now just about totally reliant on external parties doing their projects for them. And when it's a national scale project they all have checklists that say companies tendering have to have completed a state-size project, have enough size to employ enough consultants and coders, and are liquid enough to stay the course, all of which keeps it in the hands of the same big boys. Having seen it in the government sector I now see the same thing in just about every segment of the commercial market too.

      3. Peter Simpson 1
        Holmes

        Re: I don't get it

        Because this small group of huge companies does two things extremely well:

        - identifies, bids on and wins government contracts

        - contributes to politicians' election campaigns

        You'll notice that neither of the above involves producing a usable product.

      4. Tom 13

        Re: I don't get it

        Well for one thing, you get points awarded for having previously worked on similar projects. There isn't any mention of you success at pulling them off.

        Next up you have to spread around the cash for the project to buy enough votes in the House and Senate. No, no, I don't mean the lobbyist payola, I mean the projects in their district/state that prove to their voters that they are good at bringing home the bacon. Why Kansas should have a critical roll in submarine building is a bit beyond me, but it does. The North Dakota angle I get (big deep cold lake, not so much with the waves, natural test bed), but it is pushing it a bit.

        And of course the stuff the guys above me already mentioned.

        1. Anonymous Coward
          Anonymous Coward

          Re: I don't get it

          What kind of "roll" are we talking about here?

      5. Anonymous Coward
        Anonymous Coward

        Re: I don't get it

        Because these fuck-ups know how to write campaign contribution checks. Our "leaders" we elect don't have to deal with the outcome when the project goes down in flames. It just means toss more money at it and thus more campaign contribution checks.

  4. Vociferous

    What's the problem?

    Could someone explain exactly what the problem is? Yeah, hundreds of thousands of users, but for that money they should be able to rent capacity to deal with an onslaught like that -- so it must be the site itself which is buggy?

    1. Anonymous Coward
      Anonymous Coward

      Re: What's the problem?

      Probably a lack of clear requirements, lots of bad design and then worse code written to that bad design by the cheapest coders they could find. Heaps of global state, thread locking, lack of resource pooling, poor link...all leading to a system that simply cannot scale.

      Should have nailed down the broad brush-strokes, then release the minimum functionality early. You can't "big bang" this stuff.

    2. tom dial Silver badge

      Re: What's the problem?

      Fuzzy requirements

      delivered very late (first quarter of 2013!)

      resulting in late delivery of the integrated system,

      practically no system testing,

      and lackluster (at best) project management.

      It appears to me to be a basic design in which there are a lot of network links, some probably with a fair amount of latency, where failure or stall of any link that has significant traffic could constipate the entire system fairly quickly.

      Coupled with a lot of traffic the first few days (some probably from customers clicking and reclicking in frustration over the terrible response time) that which could be expected to happen, did.

      Total enrollment by the 24 December 2012 deadline was about 1.1 million (Washington Post, 12/20/2012), with 975,000 of them in December. I make that about 1 - 1.5 completions per second, which may not be too shabby given that the process apparently is fairly complex and probably took about 10 minutes each, so considerable state had to be maintained for 500 - 1,000 sessions. In addition, there apparently was a fair amount of work from customers who were just shopping and didn't complete the process. It certainly could have been better, but also could have been worse, and was in October/November.

    3. Tom 13

      Re: What's the problem?

      Lots of them. But here's one that caught my attention:

      You have to provide all your data before they let you start browsing for plans.

      Think about that. Think about how just about any retail site you have ever been to works.

      ...

      Yeah, that's bad.

      Oh, and I think they changed the plan for this particular aspect of the site design about half way through the already too short development time. With no* plans for integration testing. It was all just going to magically work.

      *OK, technically they allowed 2 weeks. I'm not a professional programmer, hell, I'm not even an assistant project manager. But I KNOW that in multi-system, multi-Department programs like this one, two weeks is equal to zero for all practical purposes.

  5. Anonymous Coward
    Anonymous Coward

    Accenture?

    They may want to speak candidly to the UK gov first.

  6. Herby

    Sorry Michele, your buddies...

    ...had to go home. That's how life is.

    Note to self: Don't become friends with politicos to get work. It will never work out well.

    1. Dave Stevens

      Re: The problem?

      I'm guessing subcontracting most of the work oversea.

      Bad project management for sure. Probably bad testing.

      A lawsuit might shed some light.

      1. Tom 13

        Re: A lawsuit might shed some light.

        No, no it won't. That would just cost the taxpayers even more.

        I wish it would. I really do. But it won't.

    2. Brad Ackerman
      Pint

      Re: Sorry Michele, your buddies...

      Getting work based on friend$hip with politicians works just great. You get twice as much money as a private-sector organization would pay for similar work, then come the overruns. By the time you've finished, the government has paid ten times and you've delivered maybe half. As long as you don't do anything that causes your facility security clearance to be pulled (such as inviting a delegation from the Russian embassy over to the data center for a weed-fueled orgy), you can keep lowballing bids and billing the government $150/hr for people who aren't even worth minimum wage until the cows come home.

      Oh, wait... maybe you were thinking about how it doesn't work at all for the taxpayer? Nobody signing the cheques gives a flying frak about them; they're just wallets on legs.

  7. Anonymous Coward
    Anonymous Coward

    Makes for good PR but

    it isn't a good thing for any project to switch contractors in mid stream. Even if the incumbent sucks.

    A lot of knowledge will be lost and have to be re-learned by Accenture. But they don't mind, they'll just bill the government for all the on-the-job learning that will be required to learn the things the current group already knows. The administration doesn't mind, because if something is broken, making it look like you're taking concrete action to fix the problem counts more than actually fixing the problem.

  8. dssf

    Hocks de-hammed

    Hehehe, looks like Obama boiled the hocks off these hams, or the ham off these hocks.

    At the risk of quoting Scripture: All of this has happened before, and ALL of this WILL happen again.

    And:

    " "I see the universe. I see the patterns.""A part of me swims in the stream but in truth, I'm standing on the shore. The current never takes me downstream.""

    And, this little gem:

    " I have eyes. I can see. God has taken your hand and purged you of the questions, the doubt. Your journey can finally begin, but there isn't much time. The others, the ones I left behind, they need your help. But not as much as you need theirs. "

    If only gov entities had a Leoben to vet their courses of action before severe damage unfolds...

    Oh, wait: some OTHER politician would be the corollary to "I see you've got this THING for rivers and streams.... I say we oblige you..." hehehe

  9. Destroy All Monsters Silver badge
    Trollface

    Politically charged soviet-style super-centralized project fails; film at 11.

    huge budgets, tons of consultants, a fragmented procurement strategy

    Plus unrealistic goals due to political pressurizing.

    All of this to fulfill democrats' fantasms about Prussian-Style Social-Democratic Centralized Healthcare For Everyone that no-one wants and no-wants to pay for.

    After this, surely the next stop is federally mandated single-payer healthcare.

    1. asdf

      Re: Politically charged soviet-style super-centralized project fails; film at 11.

      Well if GOP didn't run un-electable candidates like Walter Mondale Romney then perhaps this wouldn't be going down. The way to change policy is to actually win national elections (GOP lost popular vote in 5 of the last 6 presidential elections). The best an almost regional party can do winning carefully carved out little hilly billy districts is gunk up the works at the expense of making said party look like the do nothing party. Passing bills that go nowhere is still doing nothing no matter who is blamed for it.

      1. Vociferous

        Re: Politically charged soviet-style super-centralized project fails; film at 11.

        > The way to change policy is to actually win national elections

        The GOP disagrees. They have found that in their core issues (stopping abortion, destroying environmental protection, lowering taxes, guns everywhere, god everywhere) it is far more effective to focus on the state level.

  10. Stephen Channell
    Unhappy

    lazy journalism: B+ for reading WSJ, D- for NHS Reference

    It's lazy journalism to compare failed projects... Like you were a skunk-fueled adolescent Lamenting "what's the point of anything". If you look past the buzz-words Obamacare and NPfIT could not be more different.

    Obamacare didn't have a single project management authority; first thing NPfIT did was to hire a master project management contractor.

    Obamacare website does little not than a GP survey to BUPA website; NfPIT was about the difficult stuff of healthcare.

    Obamacare failed because it didn't do proper testing; NPfIT failed because it required clinicians (that couldn't agree) to test & signoff.

    Oracle Sedona is technically a better comparison, but few have read about that billion dollar disaster outside a small IT community.

    Lets all celibrate HP getting the gig, those Oscilloscope they used to make prove they know health! (or maybe that's just plucking random words)

    1. Anonymous Coward
      Anonymous Coward

      Re: lazy journalism: B+ for reading WSJ, D- for NHS Reference

      "Lets all celibrate HP getting the gig, those Oscilloscope they used to make prove they know health! (or maybe that's just plucking random words)"

      Wrong company. That one is called Flagilant, or something similar

    2. phil dude
      Pint

      Re: lazy journalism: B+ for reading WSJ, D- for NHS Reference

      as someone who actually managed to signup (on the phone) "Obamacare" hasn't failed, but the crap website did.

      It is all very well them kicking the contractor to the kerb, but just as with the banks, is anyone going to jail?

      I guess the B ship crashed nearby...

      P.

      1. Tom 13

        Re: "Obamacare" hasn't failed

        With 5 times as many people booted off the insurance rolls as have signed up for it?

        Yes, yes it has. And that's before we even get to it's stated purpose.

      2. phil dude

        Re: lazy journalism: B+ for reading WSJ, D- for NHS Reference

        so I got a thumb down for the healthcare comment or the oblique HHTG ref?

        We should be told...

        P.

  11. Tony Green

    English translation needed

    I have no idea what "kicks Healthcare.gov contractor to curb" means.

    This seems to have been written either by an illiterate or perhaps someone who only speaks 'Merkin English (which amounts to the same thing).

    Checks URL bar... yes, thregister.co.uk

    1. Lord Elpuss Silver badge

      Re: English translation needed

      Curb = to inhibit or restrain

      Kerb = raised stone edging to road

      1. Number6

        Re: English translation needed

        Actually, curb has a longer history, the kerb variant seems to have appeared later in proper English to refer to the line of stones along the edge of a road. The implication behind both is that they're restraining the edge of the road (curbing/kerbing it).

    2. asdf

      Re: English translation needed

      > 'written either by an illiterate or perhaps someone who only speaks 'Merkin English (which amounts to the same thing).

      I hope you aren't saying all Americans because honestly a fair number of the best English writers by almost any measure for at least the last 150 years have came from the States.

      1. Anonymous Coward
        Headmaster

        Re: English translation needed

        "A fair number of English writers by almost any measure for at least the last 150 years have came from the states."

        "Have come from the States."

        I apologise for the PGN behaviour, it is built in.

        1. asdf

          Re: English translation needed

          I see the irony but may I be the first to admit I am certainly not one of them. I have my Merkin tendencies but after spending almost three years in continental Europe (non military which is important because Stuttgart is America east) I at least unlike many of my countrymen understand more than one geographical and cultural viewpoint. In the US due to geography and economics it is rather non-trivial to go outside the country safely for more than a vacation and learn about other countries (lets be honest Canada is safe but it isn't a lot different than the upper midwest US).. In Europe you are almost forced to unless you want to say in a geographical area the size of a single one of our states (Even Germany is smaller than Montana for example).

          1. asdf

            Re: English translation needed

            Oops meant to say I am not one because my writing sucks not because I am not a Merkin type. I was bred to be one (pure white trash) and revert often but did somehow make it through college (lax standards for the win).

  12. gary27
    Facepalm

    Obviously doomed from beginning - Mark Logic XML db

    Such a shame Obamacare - great strategy spoiled by poor implementation - insurance is a major failing of capitalism as currently structured - dominated by large corps run by managers with no skin in the game - because the main metric used for bonuses etc is very short term, such as quarterly profit.

    Result is most of them cheat and use any excuse to avoid paying out, i believe therefore - even though I'm a capitalist business owner myself - that insurance and similarly long term savings pensions institutions - who also lie and cheat - need to be run by governments or very heavily regulated.

    I wish mr Obama had been given some sensible advise - trying the run this system on an xml database was a monumentally idiotic decision - hierarchical databases are no better than file systems or key-value stores such as mongo.

    A properly designed highly normalise relational database is the only sensible or practical method currently available which can ensure data integrity - also rdbms performance now also fast - since io handicap now resolved by ssd.

    1. FutureShock999

      Re: Obviously doomed from beginning - Mark Logic XML db

      Oh...but don't you know, XML databases are one of those "new Web technologies" that simply must be applied to all problems to prove that the development staff are up to date and hip. The next alternative was probably Hadoop, so they could call it a "Big Data" solution.

      You are entirely correct - the only proper solution to such an important application, at an enterprise level of transaction processing, was a well-designed and well-normalised relational database with huge parallel abilities. Oracle (despite the cost) would have been an obvious choice, but other RDBMS packages would be nearly as capable - in some cases, such as HP's NonStop SQL, perhaps more so and even more suited.

      We sometimes forget that Web-technologies evolved to do things that prior technologies didn't do very well - like enable massive social interactions, or end-user customisation and extendability. But that doesn't mean that they are the appropriate choice to replace legacy technology for what it was specifically DESIGNED to do, such as massive transaction processing in a well-delineated environment.

    2. Philip Lewis
      Headmaster

      Re: Obviously doomed from beginning - Mark Logic XML db

      advise = verb

      advice = noun

      license = verb

      licence = noun

      practise = verb

      practice = noun

      do you see a pattern here?

      Whatever happened to the English language?

    3. Anonymous Coward
      Anonymous Coward

      Re: Obviously doomed from beginning - Mark Logic XML db

      Logic behind choosing XML was apparently driven by the need to integrate with hundreds of different insurance providers. XML is de-facto English used in machine-to-machine data exchange, especially in health industry. Yet it is true that software industry does not have many XML database vendors today. Nearly all options availabe are some legacy code based XML databases such as Software AG Tamino, Oracle's outdated XML or IBM's DB2 based XML database. Even Microsoft's Biztalk Server relies on SQL Server, once advertized as an ultimate XML database. The problem is that none of those products have been properly engineered, mostly relying on ages old code still using legacy tabular or hierarchical databases and only presenting in marketing as XML databases. One can say that legacy DB vendors made attempt to protect their multi-billion SQL market with waporware promises about XML databases rather than actually innovating and delivering fast and scalable alternatives to SQL. MarkLogic was perhaps the best choice in the Obamacare web site planners opinion as the vendor claims now to be selling a native XML database. Unfortunately MarkLogic technology is also more than a decade old software and was never designed for use as a generic transactional database. MarkLogic did great business in selling application solutions in industry verticals such as content management, libraries, media. It was not sold as an XML database when they have started. It was not sold as XML database when they have built profitable business in verticals. Why they have changed their established mojo, nobody really knows. Undisputably it is a good technology, when used properly.

  13. AJames

    Looks to me like CGI Federal did a great job

    CGI Federal have fulfilled their most important purpose - to take the blame for the screwup, leaving the government managers ultimately in charge of delivering healthcare.gov to hide in the shadows. I bet they all moved on to other posts up the chain before it hit the fan, and their replacements are able to say it's not their fault.

    1. ecofeco Silver badge

      Re: Looks to me like CGI Federal did a great job

      That too.

  14. Anonymous Coward
    Anonymous Coward

    Why Accenture?

    They were incorporated in Bermuda and then moved to Ireland. So the governments of the world view these companies as tax dodgers but yet they are willing to spend money with them.

  15. W. Anderson

    Replacing one doofus with potentially a worse one.

    The disaster that was the initial development process of the healthcare.gov website portal is sure to repeat with the Obama Administration replacing CGI with Accenture, a consulting firm that has an almost equally dismal record for inefficient and costly technology consulting projects, and more disturbing - as a very close business ally of Microsoft- to the detriment of all competiors, will push to "replace" any of the Open, Internet Standards based and indisputably proven Free/Open Source Software (FOSS) with Microsoft Windows and Azure technologies, which then guarantees a nightmare in poor reliability, limited scalability and calamitous security issues for the future ACA Web portal.

    The Federal Government are potentially replacing poor project preparation, inadequate quality control and rushed delivery from CGI with the world's record champs for enterprise class technology that in reliability, scalability, performance security and Return on Investment (ROI) could not be worse than in Microsoft. The factual records should bear this out.

    1. phil dude
      Linux

      Re: Replacing one doofus with potentially a worse one.

      I agree, we all now have to pay to *fix* it too!!

      But I actually signed up to "healthcare.gov" but on the phone, and that seemed to have worked.

      What a lot of people have missed in the FUD about a crap website, is that they changed the law. You can now buy health insurance that cannot legally deny you for pre-existing conditions. There is also a maximum "out of pocket" expense if you file this on your income tax.

      I am still wondering have you can spend $300*10^6 on something so crap...?

      P.

  16. Tom 13

    Let me fix this for you

    By getting rid of the contractor, as reported, US officials hope to stop the rot on an IT project that has all the traits...

    By getting rid of the contractor as reported, US officials hope to remove the spotlight highlight all the government mandated mistakes on an IP project that has all the traits...

    I'm not saying the contractor is blameless, just that even if they were, they still couldn't have succeeded given the mistakes made before they even got their money grubbing fingers on it.

This topic is closed for new posts.

Other stories you might like