back to article Stricken 2e2 threatens data centres: Your money or your lights

The collapse of UK IT contractor 2e2 descended into farce tonight as its largest data centre customers were told to each pay £40,000 just to keep the lights on. Clients of debt-crippled 2e2 were told to cough up the cash to keep systems running until they transition to another provider, sources close to the situation have told …

COMMENTS

This topic is closed for new posts.

Page:

  1. JimmyPage Silver badge
    Facepalm

    Sounds fair to me ...

    although I can forsee a slew of downvotes ...

    1. Destroy All Monsters Silver badge
      Mushroom

      Re: Sounds fair to me ...

      Yeah well, that STRONGLY depends on the SLA.

      Does it stipulate that in case of wind-down operations, customers may have to pay to defray any running costs plus VAT, may have to wait weeks to exfil their data from servers that belong to HP and to which they have no physical access and generally be up shit creek without a paddle unless they have a second cloud on retainer? (An even then they will probably want to nuke their first cloud by wiping disks etc. but that is another matter still)

      Can anyone verify?

    2. Mike007

      Re: Sounds fair to me ...

      Call it a "stupid tax" and it sounds fine to me. If keeping your servers up is that critical that you'll throw £40,000 for a possible extra week or so (nothing stopping them taking the cash and shutting it off anyway), why don't you have a backup you can switch to?

      1. Captain Scarlet Silver badge

        Re: Sounds fair to me ...

        "why don't you have a backup you can switch to"

        Maybe thats with 2e2 as well

  2. Anonymous Coward
    Anonymous Coward

    well that sounds like

    Hey, you have some nice data there, it would be a pity if anything happened to it.

    I guess companies that outsource such operations will have to think hard about where the data is and how to get it back in the future.

    1. Fatman
      FAIL

      Re: well that sounds like

      This is a 'poster child' for one of the many reasons why you do not outsource mission critical functions. It is also another horror story in my book of horror stories that gets used anytime a wet-behind-the-ears MBA type starts spouting "cloud" at WROK PALCE.

      I keep telling those 20-something who head into the CEO's office trying top sell her on the "cloud" to do their homework. Most of the time, they haven't.

  3. Adze
    Alert

    Heavy clouds...

    ...usually precede rainfall. Anyone else tempted to move their data to someone elses servers and services today?

  4. Anonymous Coward
    Anonymous Coward

    Seems to indicate that the ~400 remaining staff now officially have no hope.

    :-(

  5. Anonymous Coward
    Anonymous Coward

    The point is ...

    if you build your business on a single point of failure, with no contingency or business continuity compensating controls, then you are the author of your own misfortune.

    1. Anonymous Coward
      Anonymous Coward

      Re: The point is ...

      Exactly right. I was telling a customer of 2e2 they were at considerable risk some months ago. Not because of 2e2 specifically, or anything I knew about 2e2's situation, but because they had no, zero, zilch continuity/contigency plans that took account of a 2e2 technical, financial or corporate failure. They must be in complete panic-mode right now, this could conceivably bring their business down as well.

    2. Fatman

      Re: The point is ...then you are the author of your own misfortune.

      Try telling that to the wet-behind-the-ears MBA types that can not grok what could possibly go wrong when your cloud connection dies?

      They like to think in big-vision terms, and often they are not able to see the underlying problems because they lack the ability to appreciate nuts-and-bolts issues.

      1. asdf
        Trollface

        Re: The point is ...then you are the author of your own misfortune.

        >Try telling that to the wet-behind-the-ears MBA types

        And that is why other than useless crappy social network companies (that flop post IPO) the upper level people making the decisions are not in their 20s.

  6. Anonymous Coward
    Anonymous Coward

    Hey! We're Having a Bankruptcy Party

    Join Us.

    Bring Your Own Money.

    1. Destroy All Monsters Silver badge
      Trollface

      Re: Hey! We're Having a Bankruptcy Party

      Is this Enron?

      1. Anonymous Coward
        Anonymous Coward

        Re: Hey! We're Having a Bankruptcy Party

        It must be Blackberry!

  7. nsld
    Facepalm

    So thats

    £10K to run the data centre for a week and £950K to keep the administrators steering the ship!

    Seems odd they cannot get someone to take over that aspect of the business as it clearly has customers.

  8. Anonymous Coward
    Anonymous Coward

    What did I say...

    Well I did say yesterday that goodwill alone wont keep the data centres running.....

    £1m to keep it running for just over a week, no wonder 2e2 made no money

    1. Anonymous Coward
      Anonymous Coward

      Re: What did I say...

      Probably find a large proportion of this is for the fees of the administrators...

      1. Fatman

        Re: What did I say...fees of the administrators...

        administrator fees > 90% (maybe??)

  9. The Alpha Klutz
    Devil

    That's the problem with clouds

    They have a tendency to just disappear in a shower of rain. I bet their customers are all really happy with the "savings" they've made....

  10. Anonymous Coward
    Anonymous Coward

    You want what?

    Oh Mr Customer, you want what? The admin password to access your virtual servers? That will be £500K thanks

  11. Duffaboy

    the money is for

    the Overpaid contractors.

  12. The Godfather
    FAIL

    you're havin a laff.......

    From an insolvency perspective, this will have to go into a file of its own; not only in terms of how does one handle an MSP/hosting or services business Administration but also ineptitude in the lead up to Administration and beyond.

    Rule is, if you outsource, you'd better be comfortable with your chosen partner and have solid terms and conditions backed by back-up alternative actions to by-pass disruption and avoid being blackmailed.

    Don't ever recall this outfit handling any other IT insolvency.

  13. Anonymous Coward
    Anonymous Coward

    Err... isnt it getting to the point where its time someone gives the public a list of who these affected customers are? considering it appears they're either about to have their budget blown to bits to the amounts of whatever the 2e2 administrators demand, or lose their data centre.

    Seems the fallout from this is about to get a whole lot bigger.

    1. Destroy All Monsters Silver badge
      Devil

      Whirr..... WHIRRR!!!

      That will probably drive the spinwheel in Brussels another couple of rounds.

      Ready for the Cloud Directive!

  14. Ex-BenhamBoy
    WTF?

    What a state

    Seems to me FTI Insulting are in a worse state that 2e2 were!

    With no money left and saying they'll not be liable for any costs in any way why not take the money, turn the lights off and draw a line under it all? Probably the biggest Administration cock up ever!

  15. kain preacher

    Savings

    I'm sure they can afford with all the savings they made from outsourcing.

  16. Anonymous Coward
    Anonymous Coward

    What happens if

    What happens if half the big customers refuse to pay, I am guessing the £900k+ gets paid by the rest of the customers?

    Funny that Channel 3 consulting who stitched up several NHS trusts with 2e2's services is still bragging about the fact on the front page of their website.

    Perhaps Sussex Partnership are wishing they had spent more than three weeks on the due diligence of a £37m contract.

  17. Steven Roper

    I love it!

    Every time something like this happens, as with the Megaupload fiasco last year, it demonstrates in huge 500 point Arial Black exactly why relying on The Cloud is a really, really bad idea. And the sooner everyone realises this and decides to keep their own files on their own servers the better it will be for the whole industry and the public as well.

    This business of giving control of my data to others has irked me from day one, and anything that hammers this fact home to businesses and consumers everywhere is always welcome in my book. One day, people will get the message: you give control of your data to other companies only at your extreme peril.

    1. Destroy All Monsters Silver badge
      Trollface

      Re: I love it!

      So is it like outsourcing making lasagne?

    2. Dr Who

      Re: I love it!

      Once again I have to say that this really just isn't the case. Why should a data centre be any less trust worthy than a company employee. There are countless examples of pissed off sysadmins destroying data and backups held on internal company machines.

      If you want stuff in house recruit your sysadmins very very carefully. If you're going to outsource, pick you outsourcing suppliers very very carefully. It's as simple as that. There is nothing inherently more risk about outsourcing.

      1. Peter Gathercole Silver badge

        Re: I love it! @Dr Who

        That is why you have a reliable DR set of procedures that take periodic copies of the data out-of-easy reach, and (importantly) that you control the process and the media.

        E2E may have the best DR procedures around, but if they 'own' the media that the backups sit on, they are just as unavailable to the end user as the servers that they back up. This is why it is more dangerous.

        I wonder how many outsourcing contracts contain clauses that immediatly revert the ownership of the backup media and documentation for the backup process to the end customer in case of insolvency of the outsourcer. Sounds like a good clause to me.

        In the case of trusting your administrators, you should spread the responsibility to more that one administrator to make sure that it is performing properly, but this will allow for the situation that if someone trashes all the data on your live systems, you just invoke DR.

        Of course, if someone is really malicious, and is allowed to screw with the systems for more than your longest off-site backup is viable, then it is still possible to destroy all the data, but you hope that there is no conspiricy and that such behaviour would be spotted. But you could say the same about the financial director, the pruchasing officer, or even the caretaker. It's not just IT people who could be holding a grudge.

        1. Anonymous Coward
          Anonymous Coward

          Re: I love it! @Dr Who

          I think some are missing the point, if you have outsourced your entire IT support operation then you don't have either the hardware or staff to keep backups.

          The NHS and other public sector organisations using 2E2 need to wake up and ensure in future they carry out full due diligence, don't assume because they are on a government framework that they are sound financially because the GPS did a credit check on 2E2 in December and there were no issues reported.

          They also ensure suppliers have insurance in place to mitigate against this.

          1. Peter Gathercole Silver badge

            Re: I love it! @AC 10:57

            I quote from my original comment.

            "and (importantly) that you control the process and the media."

            I think you just made my point over again.

            My response to "Dr Who" was actually about rogue administrators in an un-outsourced IT department being as big a risk as an outsourced operation going bad. But I tried to make it a little relevent to the original story as well.

            1. Anonymous Coward
              Anonymous Coward

              Re: I love it! @AC 10:57

              If you outsource you lose control over the process and the media. If, somehow, you don't you lose control you might as well DIY.

      2. Test Man
        Stop

        Re: I love it!

        "If you're going to outsource, pick you outsourcing suppliers very very carefully. It's as simple as that. There is nothing inherently more risk about outsourcing."

        Well, there is. SImply put, putting your data in the hands of a company that you have no control over is a lot more risky than if it was in your own hands.

    3. Fatman

      Re: I love it!

      I upvoted you, because you made sense.

      How in the world could someone who is supposed to be educated (possessing MBA degree) could not possibly foresee the havoc the crash of an outsourcing provider, especially one providing mission critical functions, can be anything but a disaster for a company.

      I do now understand what motivates such deranged individuals - executive bonuses for incompetent management manglement.

    4. Joll3295
      Meh

      Re: I love it!

      I disagree, and see this as possibly an over-simplified reaction (for the record, no I'm not with a hosting provider). Businesses are progressively realising that what they are strong at is running their business, whether it's providing products or services, not necessarily running IT organisations.

      That's not to say that this use case is suitable for everyone, but the ability to scale elastically on-demand, pay as you use, and leverage broad infrastructure together with a number of diverse and specific skillsets without the millstone of regular and heavy capital investment/maintenence, can provide businesses with improved agility, or simply allow them to focus on what they do best.

      1. Anonymous Coward
        Facepalm

        Just Consultant Speak

        Businesses are progressively realising that what they are strong at is running their business,

        "Running a business" includes everything from paying the electricity bill to keeping the works canteen going. IT comes in there somewhere along the way. What managers focus on is management and if they are not broad-minded and intelligent enough to properly delegate things they might not understand themselves to people who can do the job, they should not be managers, let alone directors and CEOs.

      2. PhilBuk
        Headmaster

        Re: I love it!

        @Joll3295

        Now write something without quoting wholesale from your MBA course notes.

        Phil.

  18. Anonymous Coward
    Anonymous Coward

    Ransom bu outsourcing

    So 2e2 dont even own the assets they are hosting their clients services on!

    IT Directors of the world - you have been warned. Mind you thats assuming that you genuinely have the strategic interests of your business at heart...and not just the desire to make your bonus at any cost and sod off elsewhere while watching the flames grow.

    1. Anonymous Coward
      Anonymous Coward

      Re: Ransom bu outsourcing

      No, the article, as many other on El Reg has been, is massively over-simplified.

      The HP kit is only "some" of what is in the DCs.

  19. Anonymous Coward
    Anonymous Coward

    The perils of business continuity...

    Outsourcing isn't necessarily a bad thing for all businesses. Cloud infrastructure (hate that phrase, personally - should be much more akin to utility based computing or some such phrase in my personal opinion) is also not necessarily a bad thing.

    But as others have pointed out, failing to perform due diligence (surely the amount of turnover, profit posted and debt were readily available to these customers? And surely it would be clear that it was a house of cards built on a bed of shifting sand?) AND failing to put contingency in place is business-suicidal.

    I remember doing some troubleshooting in one of the datacentres of one of the companies that 2e2 bought some years ago. The staff at the datacentre were very pleased with all the alternatives they had provided - alternative electricity supplies from multiple energy suppliers, a standby generator and UPS's, alternative points of access to the internet via multiple exit points within the datacentre but still only a single data centre for an awful lot of the customers.

    I've lost count of how many times I've had the conversation about simple things such as cloud backup. Great. But, for example, only if getting that data back down a fairly narrow bandwidth internet pipe isn't a concern. Oh and what happens if that backup company goes under - how do you / can you get your data back?

    Like many projects, this comes down to customers offloading their responsibilities for what they perceive as benefits - usually in cash terms - without sitting down and talking about it, and more importantly, thinking about it - 'it' primarily being worst-case scenarios.

    I'm curious about one thing - it's been a while since I had any kind of involvement in offsiting to datacentres but close to a million pounds to run for one week? Seriously?

  20. Anonymous Coward
    Anonymous Coward

    What a farce and hopefully a wakeup call to those people that think outsourcing everything to a single supplier without any contingency is an awesome idea because it saves cash. At least our hosting provider has insurance in place to protect us against this kind of failure happening to their business, its something we require from all our major suppliers and I am amazed its not for others!

  21. Anonymous Coward
    Anonymous Coward

    this is extorsion

    or blackmail, whatever the term.

    1. Test Man
      FAIL

      Re: this is extorsion

      Extortion.

  22. Anonymous Coward
    Anonymous Coward

    ****ed

    As an employee of a large company who has a significant amount of kit in their DC, I was privy to a conversation yesterday where our head of IT finance came in and said in no uncertain terms that part of the business was ****ed.

    I am absolutely certain the legal wheels have already started turning - although I'm not sure I see the point.

    Nevertheless it's not as bad as was made out - thanks to the fact that we've just employed our DR strategy for that site.

    Would have been nice to have a little more notice though!

  23. The Jase
    Trollface

    LOL

    that is all

  24. Anonymous Coward
    Anonymous Coward

    We have capacity

    Want somewhere to move to? www.acscloud.com

    We can handle the Cisco Hosted Unified Comms, as well as IaaS and other requirements.

  25. Anonymous Coward
    FAIL

    I hope the DataCentre staff walk before they are pushed!

    Meanwhile the stupid banksters keep on outsourcing to India! ROFL Hahahaha!

Page:

This topic is closed for new posts.

Other stories you might like