back to article Azure lost some virtual machine backups for eleven hours

Microsoft's Azure cloud has had a nasty hiccup that saw it unable to find virtual machines recently added to its backup service. As Microsoft's status page records, from “04:40 to 17:36 UTC 17 Feb 2016, customers using Azure Backup Service might be unable to discover newly added IaaS Virtual Machines within their old Backup …

  1. Destroy All Monsters Silver badge
    Windows

    Permutation Datacentre: Very Virtual Servers on Other People's Hardware

    As ever, the problem isn't the technology: it's the wetware in front of the keyboard that causes the problems.

    The problem is still the technology. This is the first attempt ever in history to do anything like this and there will be surprises. Especially as the infrastructure code has lots of really bad design decisions and random cruft stemming from the idea of "a machine on every desktop" approach (especially in the case of Microsoft). Unecessary complexity, code written with an "it's okay, ship it" approach (especially in the case of Microsoft) and the fact that the backup is not physically in a vault in your cellar will cause some sleepless nights.

    1. Mark 85

      Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

      the fact that the backup is not physically in a vault in your cellar will cause some sleepless nights.

      Only "some" sleepless nights? Let's face it, the more data a company puts into the "cloud", the less accessible it will become if the cloudy company goes bust, decides to not have a cloud, or some major disaster. If it were me, I'd sleep better knowing that there's tapes in a vault under my control.

      1. Locky

        Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

        If it were me, I'd sleep better knowing that there's tapes under my bed

        There, fixed that for you

        1. Afernie
          Facepalm

          Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

          "If it were me, I'd sleep better knowing that there's tapes under my bed

          There, fixed that for you"

          Your house (and your bed) is fire-proof AND patrolled by security guards? Could I please book secure media storage space under your magic bed forthwith?

          1. Locky

            Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

            You mean yours isn't?

    2. allthecoolshortnamesweretaken

      Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

      The problem isn't the technology per se - it's the implementation of said technology.

    3. boatsman

      Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

      sorry. it is not technology.

      code is written by people. even if you make it HFDCT (c), we will keep seeing failures.

      cuz it is us, not the machine. the machine does what we *actually* told it to do.

    4. CrazyOldCatMan Silver badge

      Re: Permutation Datacentre: Very Virtual Servers on Other People's Hardware

      > This is the first attempt ever in history to do anything like this

      It could be argued that this is exactly the sort of thing that mainframes (and mainframe users) have been doing for a long time..

  2. Anonymous Coward
    Gates Horns

    99.99% Availibility.

    We can do IT better than your in-house monkeys.

    Course we can.

    We're THE EXPERTS.

    $ubscribe now

    1. hplasm
      Holmes

      Re: 99.99% Availibility.

      Send money to: Clown Computing Inc. Now!!! Save $.

    2. Afernie

      Re: 99.99% Availibility.

      "We can do IT better than your in-house monkeys."

      If we assume in-house IT is somehow better, how honest are we being regarding the comparative professionalism of what we've all seen in the way of primate behaviour in IT departments across the land? Present company excepted, of course; we never make mistakes.

      Famously, your last backup is only as good as your last restore. How many departments actually schedule meaningful restore tests on a regular basis, rather than bunging the tapes in the safe, and how is that better or worse than cloud?

      How many sites have changed organically over years, with servers added but the UPS capacity remaining the same and runtime reduced because, budgets?

      How many in-house services possess single-points of failure, and no real redundancy or DR/BC plans because, budgets?

      Are Cloud services better than in-house? I'd say much the same, but with a different set of monkeys. At least there tend to be more of them.

    3. SVV

      Re: 99.99% Availibility.

      Can we therefore conclude that an infinite number of monkeys bashing away at laptops for an infinite amount of time, will one day inevitably develop a 100% reliable cloud service?

      1. Afernie

        Re: 99.99% Availibility.

        "Can we therefore conclude that an infinite number of monkeys bashing away at laptops for an infinite amount of time, will one day inevitably develop a 100% reliable cloud service?"

        When the infinite monkey theorem was tested. the Macaques in question produced 5 pages consisting mostly of the letter 'S' (perhaps they were trying to write the Bard's name?) then smashed the computer (presumably in frustration) before (perhaps predictably) defecating and urinating on the wreckage.

        They've got some way to go for 100% reliable cloud.

      2. Doctor Syntax Silver badge

        Re: 99.99% Availibility.

        "an infinite number of monkeys bashing away at laptops for an infinite amount of time, will one day inevitably develop a 100% reliable cloud service?"

        s/reliable cloud service/downtime/

  3. Mystic Megabyte

    ??

    “A recent upgrade updated a property with a wrong value resulting in discovery failures for existing backup vaults.”

    NSA_scan_finished = FALSE

    1. Anonymous Coward
      Anonymous Coward

      Re: ??

      We sent router hardware to Russia for our new MPLS network. It spent a month with Russian Customs for "national security reasons". I think when we say 'customs' we can reasonably assume we really mean "FSB".

      Cloud or on-premises makes no difference. If a state actor wants your data badly enough they will get it. Anonymous Coward, because I don't like the taste of Polonium 210.

  4. Mad Mike

    What did people expect.

    As usual, this is all about liabiltiy. Mistakes happen to everyone, whether a cloud provider or an end users IT department. However, the question is liability.

    If you're doing your own IT and something goes wrong, it ultimately is your (as a company) fault. It might be you haven't employed the right people, trained them correctly, haven't got the right processes etc.etc. However, liability rests with you.

    However, if you put it in the cloud or anywhere other than with you, liability rests with someone else. Do they employ the right people? Do they train them correctly? Have they got the right processes? You have no control over this and in an attempt to drive up profits, you can rest assured these will all drop in quality. And the liability is normally a few service credits when something goes wrong!!

    So, anyone considering cloud should really understand that they are effectively giving up control of their systems and more importantly the risk factors around them. In return, the cloud company gives you a liability clause that pretty much negates all liability. They give you a rubbish service, you simply get a few months more of a rubbish service for free. Wonderful.

    CEOs (and other decision makers) should really consider these decisions in the context of their own jobs. Why shouldn't we cloud the function of CEO? Simply feed the right information into someone elses 'cloud' and back comes a decision. If it proves to be wrong, you can get the next 3 decisions for free. It's the same thing. Do you think CEOs think that's a good idea? If it isn't, why is IT cloud a good idea, especially for mission critical systems?

    1. Doctor Syntax Silver badge

      Re: What did people expect.

      "However, if you put it in the cloud or anywhere other than with you, liability rests with someone else."

      No, you made the decision to do that. The liability still rests with you.

      1. Afernie

        Re: What did people expect.

        ""However, if you put it in the cloud or anywhere other than with you, liability rests with someone else."

        No, you made the decision to do that. The liability still rests with you."

        I consider myself to marginally be a Cloud advocate. That said, this is right on the money. Wherever you store the data, it's still your responsibility.

        * Make sure your backup works

        * Make sure you have a DR/BC plan.

        * Make sure the terms your Cloud Provider offers are not onerous in ways that may damage your business.

        And many other caveats. Ultimately Cloud Providers are simply glorified Web hosters - you need to understand what you are getting into, and for heaven's sake - read. the. small. print.

  5. boatsman

    there is no patch tuesday for carbon entities

    hence, this will keep happening.

    just like it always did. we moved the issue, we did not solve it. and we never will.

  6. StaudN
    Flame

    All the hardware bods...

    Hehe, nice to see the herd of in-house hardware guys looking around at each other for comfort (read "fear induced glee").

    One thing is certain: Cloud makes for serious consolidation of the demand for your specialised skillset and if your (relatively) home-brew self-built setup is as awesomely awesome as the cloud or not [it's not] makes little difference to the long-term argument... once the top 3 cloud providers settle into a comfortable groove and as developers improve at the scale-out vs scale-up mindset, you guys are toast.

  7. nilfs2
    FAIL

    Microsoft fails, nothing new here

    They have had a failed OS since it's conception and people still use it, now they have a failed Cloud service since it's conception based on their failed OS and people still use it, what do you expect? you do the math

    shit + shit * shit = ?

    1. Afernie
      FAIL

      Re: Microsoft fails, nothing new here

      Microsoft has a huge collection of corporate character flaws. Still, that "failed OS" is so unsuccessful that it powers the bulk of global business IT, with no real evidence that that will change any time soon. No amount of downvotes or wishful thinking will make this less factual.

      On a separate note, how is the view from mom's basement?

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like