back to article Act of God damaged data on Google cloud disks

Google has admitted that some customers running Persistent Disks in its europe-west1-b region have been forced to recover data from snapshots after a combination of lightning and old storage kit was is to blame. The outage hit last Friday and left some users unable to connect Persistent Disks – a disk that exists independently …

  1. Version 1.0 Silver badge
    Thumb Up

    Surprised?

    That's not bad actually - try that at home and you'd have lost a lot more data. Disks die, backups fail, and life goes on mostly.

    1. Shane McCarrick

      Good result

      Having worked in data recovery- thats a remarkable achievement and a definite feather in Google's bow. Well done to them. 4 lightening strikes in quick succession- was a tad unlucky- there are days I wish I could conjure up such an electrical storm..........

  2. x 7

    "4 lightening strikes in quick succession"

    is the data centre on top of a mountain? hard to believe otherwise

    1. This post has been deleted by its author

      1. Anonymous Coward
        Anonymous Coward

        perhaps big metal building right under the storm

        that should do it...

        not necessarily a design flaw as any above ground location is suceptable to the local weather...

    2. Paul Crawford Silver badge

      Most likely on the pylons that feed it. Typically they are the highest thing around even on plains and if you have a big storm you can get several bolts all in a few km region. Even though it is more common to hit the pylon than cable, you are still talking about serious levels of induced surges and possible disconnected by circuit breakers as a result.

    3. TeeCee Gold badge

      It's probably in Central Europe.

      There's a reason why the house I had in CZ was covered in effing great lightning conductors, as were all the others in that area.

      Power outages due to electrical cables, substations and such being repeatedly struck[1] were all too common.

      [1] Or, on one memorable occasion, struck once very, very hard. I saw the bolt and it was like the finger of god.

      1. Christoph Silver badge
        Trollface

        "It's probably in Central Europe"

        Transylvania perhaps? Maybe Google's latest experimental project went wrong and the lightning didn't get piped to the hidden lab concealed in the data centre.

        1. durandal
          Boffin

          Transylvania perhaps?

          Or the first bolt was plenty, and the other three got dumped into the side-project by a quick thinking Igor

    4. Anonymous Coward
      Anonymous Coward

      "is the data centre on top of a mountain? hard to believe otherwise"

      In his autobiography "Just a simple seaman" - Andrew Parks describes a thunderstorm that hit Frome Grammar School in England in the 1950s. As it started raining two groups of boys were instructed to carry some gymnastics equipment from a field back to the gymnasium. As the second group approached the school there was lightning strike on the building gable - dislodging some masonry. Moments later the building was struck again. The third lightning strike was in the field - directly into the first group of boys on their third trip. One boy was killed.

  3. John Robson Silver badge

    0.000....1%

    of the data in this data centre, or of their global data store?

    Looks likely to be the data centre, but I generally assume that stats are fudged...

    1. Simon Sharwood, Reg APAC Editor (Written by Reg staff)

      Re: 0.000....1%

      This data centre.

      1. MacGyver

        Re: 0.000....1%

        Hopefully that wasn't the only copy Martha and Stanley's wedding pictures.

        Just 0.00001 of a Petabyte is a still someone's data, well, was someone's data.

  4. Anonymous Coward
    Anonymous Coward

    "Act of God"

    maybe She doesn't like her email being read?

    Actually, where in the quotes from Google do they blame anyone else?

  5. x 7

    Google IS God (or at least belieives so), so they only have themselves to blame

    1. Anonymous Coward
      Anonymous Coward

      OMG - I didn't know Google had a lightning gun...

      Now I REALLY need a tinfoil hat, and one with an earth lead too!

      1. DropBear Silver badge
        Trollface

        Re: OMG - I didn't know Google had a lightning gun...

        You better make sure the connection to earth is robust, and for a mobile electrode there's no substitute to mass for that. May I suggest one of our lovely new ball-and-chain models (now with surge-resistant anti-static shackles)...? We even have an extra-long-chain version to ensure safe ground dissipation as far from you as possible!

  6. Anonymous Coward
    Anonymous Coward

    That's one way to clear your browsing history

    1000 Kudos point to God.

  7. Daggerchild Silver badge

    "Ulysseus, you dare defy the Gods?!"

    I wonder what Google's T's&C's have in them about data loss caused by angry deities, or if they have insurers frowning at the increasing number of times Gods have tried to destroy Google. (Thort for the day: What's in the search/browser history of a God?)

    "Incident: Disruption to Google's eastern datacentre caused by dimensional incursion of daemonic army. Minor disruption while HR conducted interviews, and dealt with objections from Illuminati and Templar Union reps."

  8. sisk

    They did better than us. Last time our power grid took four bolts in quick succession some of our users lost most of a day's worth of data. Though given the difference between our budget and theirs they danged well SHOULD fair better than we do.

  9. Cloud 9

    For 'forks' sake - I bet the engineers 'sheet' themselves.

    Enough of the lightning puns ...

    Sales of multi site replicated storage services have just gone up up UP.

  10. Peter Simpson 1
    Mushroom

    Mark Twain

    "Thunder is good. Thunder is impressive. But it's lightning that does the work."

  11. Anonymous Coward
    Anonymous Coward

    Skynet

    I hear that someone reported some strange guy crouching in a circular dent in the pavement shortly after the bolts hit, mumbling about a tactical time weapon and <NO CARRIER>

  12. Maria Ashot

    My Google failed on Wednesday morning

    The storm was on Friday. "Problems persisted through the weekend." All of my system was working precisely and smoothly through late on Tuesday night, around midnight. (I am in Belgium at the moment.) On Wednesday morning I woke up to a fried browser. Nothing worked. Seems very strange to have it take so long for the data loss to trickle down into my system... Most bizarre.

  13. herman Silver badge

    So the new system will defy god? Thor will have to upgrade his hammer then.

  14. kchris

    Yes, shit happens. I think backing up your critical data to an independent secure storage is the only solution that protects you against data loss. I think Spinbackup looks like a good one.

  15. Kelli

    Europe-West1-B

    Lazy journalism, a simple google search shows you that europe-west1-b is in Belgium. You could add that in the article instead of copy-pasting.

    https://cloud.google.com/compute/docs/zones?hl=en

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2020