back to article New York tech firms form 'bucket brigade' to fuel flagging servers

Three technology firms have joined forces to avoid any data center downtime in the aftereffects of Hurricane Sandy amid the continuing power outages crippling Lower Manhattan. Employees of Peer 1 Hosting, blog host Squarespace, and Fog Creek Software have formed a 'bucket brigade', lugging diesel up to the backup generators …

COMMENTS

This topic is closed for new posts.
  1. The Axe

    Fukishama

    So they've put the generators away from harm, but not the pumps that supply the fuel. A bit like Fukishama.

    1. Blain Hamon
      Boffin

      Re: Fukishama

      There's this slight problem with putting pumps up high, called gravity. It's the same reason you can't suck water up 10m with a straw or why a barometer will have empty space up top despite the vacuum. Simply put, there's only so high atmospheric pressure will push the diesel despite how good the pump is sucking, a problem not faced by a pump that's pushing. So you have to have the pump as low as the tanks, or else they won't work beyond the third floor, if you're lucky.

      1. Yet Another Anonymous coward Silver badge

        Re: Fukishama

        How do oil wells work then?

        You can pressurize the tanks and have the pumps wherever you like providing the pressure to push the fuel up. You can also have hydraulic pumps where the motor and hydraulic pump part are high up and the drive pump is in the sump.

        The reason not so suck diesel (and other fuels) with a vacuum pump is that you would evaporate off all the volatiles, leaving you with tar in the tank and a motor full up fumes.,

        1. Infidellic_

          Re: Fukishama

          Offshore oil wells work off the natural pressure of the well at first (from the compression that in fact made the oil) so no pumping/sucking at all. Later in the well's life when the pressure is insufficient to naturally come to the well head and rig they actively pump water down into the well to increase pressure and push the "last few bits" (usually actually quite a lot) up.

        2. Vic

          Re: Fukishama

          > How do oil wells work then?

          That depends on the well.

          Some are under sufficient pressure that the oil will flow to the surface without pumping

          If that is not sufficient, a pump is sent down the shaft, driven from the surface.

          > You can pressurize the tanks

          No-one sane pressurises the tanks.

          > providing the pressure to push the fuel up

          Now do some calculations to see how much pressure you'd need to move diesel up the requisite number of floors. Fancy that in a diesel tank?

          > The reason not so suck diesel (and other fuels) with a vacuum pump is that you would evaporate

          This is not true. The OP was correct - the maximum you can get out of a lift pump is determined by ambient air pressure, and that will only move diesel around 12m if everything is perfect. And nothing is ever perfect...

          Vic.

          1. I ain't Spartacus Gold badge

            Re: Fukishama

            Little bit of bollocks getting talked here. Actually enormous amounts. From memory water can only be sucked up 7m, at normal atmospheric pressure, so you could only have the pump 7m above the tank, that's 2 floors. I don't know the specific gravity of diesel. It floats on water, so it's going to go a few meters higher. Vic says 12m, and his post makes sense, so I'll go with him.

            Also, using suction lift on a backup generator is risky. Your whole system basically relies on the footer valve, which is usually a spring loaded check valve / non-return valve at the bottom of the pump inlet pipe. If that spring doesn't close properly, then diesel will leak back into the tank, to be replaced by air, and the pump will lose prime, and won't work. And then your back-up system will fail. I don't sell pumps on suction lift, if I can persuade the customer not to, because of the hassle they always seem to cause. Also, isn't diesel always a bit dirty? I do water, not other stuff, so this isn't my area. But vital valve with spring closure and liquid with bits in = bad combination and leaky valve.

            You DON'T pressurise your diesel tank. What if it sprung a leak? Erk! Anyway how would you fill it? How would you have an overflow pipe? Not a good idea.

            Any well deeper than say 5m will have a submersible pump lowered down it on a chain, and that will pump the water up to the top. The controls and inverter (if variable speed) will live at the top of the well, but the pump and pump motor at the bottom.

            Normal pumps in a basement won't be water-proof, it's very difficult to waterproof motors that are air-cooled.

            Stick submersible pumps in your diesel tank? Well I don't deal with fuel, but you're sticking a big old electrical thing in a tank of fire-risk. I know diesel doesn't burn that easily, but even so. Also submersible pumps dump their heat into the surrounding liquid. No probs when the tank is full, or being topped up from a pump, but as the tank gets empty, and non tanker truck arrives, you're warming up a decreasing supply of diesel in a large container with an electrical item. I don't know the rules for doing this, it's not my area. I suspect there are lots of them though.

            1. Tom 13

              Re: you're warming up a decreasing supply of diesel

              in a large container with an electrical item.

              I smell a Mythbusters episode!

            2. I ain't Spartacus Gold badge

              Re: Fukishama

              Can I just add by the way:

              Tank = vessel holding liquid under pressure

              Cistern = vessel holding liquid at atmospheric pressure (i.e. with vent / no lid).

              Everyone calls both a tank anyway. Even in the industry. Because confusion is fun...

      2. Anonymous Coward
        Anonymous Coward

        Re: Fukishama

        Obviously you've not installed much less used any kind of pump. In the land of Fukushima, we have oil pumps for kerosene on, you guessed it, the floor it's consumed on. Water pumps for wells are installed, not at the bottom of the well, but you've guessed it again, at top of the well or even higher. Gosh, that is farking amazing! Pumps! Whoda thunk...

        1. Ole Juul

          Re: Fukishama

          Water pumps for wells are installed, not at the bottom of the well, but you've guessed it again, at top of the well or even higher. Gosh, that is farking amazing! Pumps! Whoda thunk...

          You just made that up. I operate my own water system, as does everybody else where I live. So yes, I have installed pumps - many times. A shallow well has the pump at the top. A deep well has the pump at the bottom. As mentioned above, air pressure is limited and quite low so you would not get any water out of a deep well if you had the pump at the top.

      3. Vic

        Re: Fukishama

        > So you have to have the pump as low as the tanks

        Not *quite* as low as the tanks; you can suck the diesel a short distance.

        Whether or not that would be enough to make a difference, I really wouldn't know...

        Vic.

  2. Ivan Headache
    Happy

    Squarespace

    sent out a cicular last night saying things were bad.

    Access tonight is a little slow but it's still there at 2125 GMT.

  3. bep

    They should use 44 gallon drums

    They'd be lighter!

  4. Dave 32
    Pint

    55 Gallon Drums of Diesel

    They're lugging 55 gallon drums of Diesel fuel up 17 floors by hand (presumably without elevators, since the building doesn't have any electricity)?!? Good &deity!!! Those things will be about 385 pounds (175 Kg)! I salute their efforts.

    Dave

    P. S. I hope they have a bunch of cold, frosty beverages lined up after work like that.

    1. Marvin the Martian
      Paris Hilton

      Won't anyone think of the HSE?

      I'm wondering what health & safety thinks of that, hand-carrying part-emptied drums upstairs through semi-darkness, outside working hours with people that have worked far beyond their normal shifts.

      But then again, I think if it helps some blogs about cats staying online that's worth a few lives.

      1. Anonymous Coward
        Anonymous Coward

        Re: Won't anyone think of the HSE?

        I suspect they dont care and are concerned with Real issues.

  5. asdf
    WTF?

    wow

    I guess some people like their employer enough to do hard manual labor hauling a very dangerous, flammable carcinogenic substance. As for me I ain't that guy. I personally would be taking care of my family and property instead.

    1. asdf

      Re: wow

      Yes diesel is harder to ignite but it does contain more energy per volume that gasoline I believe. Regardless even though its a common substance in our environment if you look at a hazmat sheet on either and you would realize they are some of the more dangerous chemicals most people ever deal with.

      1. Orv Silver badge

        Harder to ignite...

        ...is a bit of an understatement. Once, while using diesel fuel to light a fire, I dropped a match into a puddle of it. It put out the match.

        1. M Gale

          Re: Harder to ignite...

          Diesel can be burned like wax. Dip something porous and somewhat flammable in it (like a rag, paper, whatever), and burn that. In fact I've seen candles that use diesel instead of wax, with the wick on a floating thing atop a pool of the stuff. Cheaper than lamp oil, I guess.

        2. Ole Juul

          Re: Harder to ignite...

          Thanks for putting us straight. I can't believe the earlier comments making statements about diesel that are completely wrong. In fact I wonder why somebody who obviously doesn't work with the stuff would venture a comment. Embarrassing, really.

          1. asdf
            Trollface

            Re: Harder to ignite...

            TBH I wouldn't haul water bottles forget drums of it for an employer. Went to college to avoid having to do that.

        3. Anonymous Coward
          Headmaster

          Re: Harder to ignite...

          I have designed furnace burners that run on diesel - and there are two ways to do it.

          One is to spray a heavy fog of diesel onto a big pile of lit paper and rag inside a furnace and that lights up the spray, until the furnace gets hot enough for the fuel to ignite by temperature alone.

          The second way is to preheat the diesel prior to spraying it, just like the kerosene based blow torches - and let me tell you something - it has to be REALLY REALLY REALLY fucking hot to autoignite on contact with air as a spray.

          (guessing a bit here - I have the tables stashed away)

          I think it's natural flash point is about 250*C or 280*C.

          Something like that.

        4. Tom 13

          Re: Harder to ignite...

          You CAN do the same thing with gasoline. In both cases it is the fumes which ignite, not the liquid. It is admittedly much more difficult to do with a bucket of gas since it has a lower temperature for volatility, but it is possible. And yes, this would fall under the category of "Don't intentionally try this at home."

    2. slooth
      Pint

      Re: wow

      Still, I take my hat off to those guys who are trying to provide a service to their customers despite the odds. They all deserve a BIG cold frosty

    3. Destroy All Monsters Silver badge
      WTF?

      Re: wow

      Well, as long as you don't breath it in and keep it off your skin, you should be good. It's not classified as a carcinogen either (though diesel burn products are):

      PDF

      "In a multi-site, case-control study, there was evidence for an increased risk of prostate cancer and squamous cell carcinoma of the lung [36] but this effect could not be attributed to any particular chemical. The International Agency for Research on Cancer (IARC) have evaluated diesel fuels as being “not classifiable as to their carcinogenicity to humans (Group 3)” [9]: there is “inadequate evidence” to classify diesel as a human carcinogen and “limited evidence” for the carcinogenicity of diesel to experimental animals.

      What is amazing is that generators are on the upper floors. I seriously doubt that is a good idea. What if a fuel line breaks?

      1. Dave 32
        Happy

        Re: wow

        It's not too uncommon for high-rise buildings to have service floors periodically. That allows water pumps and tanks to be installed. Otherwise, an incredible pressure would be needed to pump water to the top of some of those buildings, pressures high enough that it would burst common pipes. And, for that matter, turning on a faucet at a lower level would produce a stream of water so intense that it could cut things (seriously hard things, like granite and steel). Thus, to avoid problems like this, it's common to put holding tanks and helper pumps on service floors of high-rise buildings.

        Additionally, the water storage tanks aid in fire fighting by providing a reservoir of water for sprinklers and fire hoses. This source of water even works without electrical power, which may be compromised in the event of a disaster.

        Furthermore, the water in the storage tanks can be used as a dynamic mass damper to reduce the tendency for the building to resonate when excited by earthquakes, or even by the wind.

        The service floors also make a convenient place to put elevator motors for the banked elevators which service the floors below the service floor.

        The service floors may also be convenient places to put electric distribution/switching panels, and/or transformers. So, it would be somewhat logical to put backup generators in the same space.

        This concludes this architecture lesson; we now return you to the normal commenting.

        Dave

        P.S. Mine's the one with a copy of "The Tower and the Bridge" by Donald P. Billington in the pocket. I attended a lecture by Professor Billington once; it was quite interesting.

      2. Anonymous Coward
        Anonymous Coward

        Re: wow

        A big salute to these folks. Just hope that their company recognizes and remembers their efforts.

  6. Gene Cash Silver badge

    Fog Creek s/w

    Isn't that Joel Spolsky of "Joel on Software"?

    1. Anonymous Coward
      Anonymous Coward

      Re: Fog Creek s/w

      That's the one. Plus CEO of StackOverflow (et al).

  7. Fingerless Pyro
    Facepalm

    Suck versus Blow

    To everyone who yammers on aghast about the pumps being in the basement, and how it's a horrible design: there's a very simple reason why they're there. It's called basic physics. If you are trying to get fluid from a low level to a high level (and please God don't try to suggest the diesel tanks should be anywhere but the basement) you have two ways to do it: suck or blow. If you want to suck it up (i.e., have the pumps up high) you can only raise it up until the head pressure equals the vapor pressure of the fluid. For example, if you're trying to suck mercury up a tube, you can only raise it 760mm (assuming you're at sea level). After that, you simply have a vacuum over the top of the liquid (actually mercury vapor, but that's pretty close to a good vacuum). You simply can't suck it higher than that. Water, (and diesel), being less dense than mercury, can get sucked up higher, but you can only go so high before again, you only get vapor, not fluid.

    The other approach is to blow. Put the pump inlet down at the lower level, and then you can send it as high as the outlet pressure of the pump can push it. Given that pumps can easily hit hundreds of PSI (and thousands in the case of hydraulics), you can push a liquid far, far further than you can pull it.

    So for the love of all things holy, please quit saying how horrible a design it was to put the pumps in the basement. It's the only place they can really do their job.

    However, feel absolutely free to question why the pumps weren't waterproof (or submerged in the diesel tanks), along with watertight electrical feeds to them. Now THAT is a horrible design.

    1. Infidellic_

      Re: Suck versus Blow

      So why not pump air into the tank in the basement to push the diesel up to the roof where the generators and pumps could be. Surely this is a pushing approach but using pumped air rather than atmospheric pressure?

      If you're worried about air in the fuel lines then have another temporary tank on the roof to allow any air bubbles to vent before being used

      1. Dave 32
        Unhappy

        Re: Suck versus Blow

        So, let me get this right, you want to pump an oxidizer (air) into a tank full of flammable fuel? Note that compressing air heats it up. For that matter, that's exactly how a Diesel engine works, by compressing air to about 10 atmospheres of pressure, where it becomes hot enough to ignite the Diesel fuel.

        Dave

        1. Anonymous Coward
          Anonymous Coward

          Re: Suck versus Blow

          Physics and energy levels are ALL wrong.

    2. Lars Silver badge
      Pint

      Re: Suck versus Blow

      "why the pumps weren't waterproof (or submerged in the diesel tanks), along with watertight electrical feeds to them",

      I suppose the problem is no electricity right now.

      Bye the way, sucking pumps that suck higher have to pipes one bigger and one smaller and what you get is the difference between the size of the pipes. The bigger pipe brings it up and the smaller lets the rest back down. You can suck higher with this system but hardly this high.

      1. Trevor Marron

        Re: Suck versus Blow

        ""why the pumps weren't waterproof (or submerged in the diesel tanks), along with watertight electrical feeds to them",

        I suppose the problem is no electricity right now."

        As long as you had enough fuel to get the generators running then they could power the pumps too.

  8. Anonymous Coward
    Anonymous Coward

    Your company doesn't run on diesel...

    it runs on people. I can see Fog Creek knows that, and the other two. Why is it that nearly every other company fails to 'know' that? In the past I've waded across a flooded bayou to show up and help keep a data center going. (Forgot about the snakes) But can I even imagine doing that now? You can replace me - I can replace my job - I *can't* replace *me*.

  9. Anonymous Coward
    Childcatcher

    Don't say "your company runs on people"

    The way business execs and the economy are these days, you'll probably find some brightly lit datacenter with curiously few employees around, and the generators are being run on a biodiesel fuel that the CEO calls "oilent green"

  10. ZZLEE
    Flame

    Helo

    fly it in to to roof.

    chopper it in to the roof

    when the winds die down

    1. Aldous

      Re: Helo

      with a small civilian helicopter (and you would not be able to land a heavy military one) you would be lucky to get 4-5 drums in (using previous commentards 175kg figure) not to mention the danger involved and the sheer price of doing it that way.

  11. William Higinbotham
    Pint

    Use the cloud

    Why do they not have redundant Cloud? Fast, reliable, and not affected by disaster, war, political discontent. Though in this case, it will still be hard to not have your cloud become part of the hurricane. Maybe we should think of having a more inland alternate backup Wall Street. The military has multiple redundant headquarters. How about the Poconos? Any site suggestions?

    1. Tank boy
      Trollface

      Re: Use the cloud

      The cloud ain't going to carry buckets of fuel up the stairs, no matter how hard you pray it can.

      1. jonathanb Silver badge

        Re: Use the cloud

        The idea is that you have a backup set of servers with exactly the same data etc on them somewhere else that wasn't affected by the hurricane. Maybe the West coast, or Australia or something like that. Then it doesn't matter that your New York servers are down due to lack of power.

        1. Corinne
          FAIL

          Re: Use the cloud

          jonathanb, you've clearly never costed disaster recovery options. Fully mirrored systems don't cost QUITE as much as the original system, but you still have hardware, software, networking & maintenance costs.Totally out of the question for smaller organisations.

    2. richardcox13
      Boffin

      Re: Use the cloud

      Some web sites have done that.

      StackExchange.com (including StackOverflow.com) are running out of their backup DC in Oregon, some of Fog Creek's applications are now in AWS.

      Of course this isn't something you can just magically do, throw some servers into another DC and it will work, you need both application changes and administrative processes (and monitoring) that will allow the fail over. (Eg. StackExchange recently – in a rather timely manner – tested their fail over, found some issues and fixed them.) Whether for a given web site it is worth the costs for the level of risks is a business choice.

  12. Chris 228

    So sad

    At least 68 people are dead and likely more. Lives and property destroyed forever. The damage pictures are surreal.

    God Speed people...

  13. Anonymous Coward
    Anonymous Coward

    Buckets?!?

    I cannot believe they are not using fuel-canisters/jerry-cans/containers/whatever-you-call-them-things.

  14. Donald Becker

    Curiously, only a few people above demonstrated knowledge of basic physics.

    You can draw water only about 20 feet up into a pump. In theory a little under 10 meters, but practical pumps can't draw a hard vacuum and easily cavitate. Because diesel is a little less dense than water, you can draw 15% further, perhaps 4 fathoms, but that's a minor effect. We'll call that two office floors, not three.

    Most water wells are deeper than this and must use submersible pumps to push up rather than draw from the top. That's why the well casing is so large: so that a large-ish pump can be lowered down (plus to trap an occasional curious child).

    The old fashioned hand pumps have a rod extending to a lift disk near the bottom of the well.

    Pressurizing the basement tanks would allow pumping from the top. Although at that point you can go slightly further and just use the air compressor. But now you have a highly pressurized tank of fuel. It would have to be immensely strong for safety, and any tiny leak is a disaster. Far worse, if you use regular air the partial pressure of oxygen becomes a major problem. Even cold, the fuel will spontaneously decompose, ignite and explode.

    The pictures make it clear that they are transferring the oil from the 55 gallon barrel to 5 gallon buckets. A pair of those is a reasonably effective load for a man to carry. Even in excellent shape, you won't get in many 17 floor trips before you call it a day.

    1. Anonymous Coward
      Anonymous Coward

      "Curiously, only a few people above demonstrated knowledge of basic physics."

      Not particularly curious, I suspect the only people that know basic physics are 40+.

      Welcome to the dawn of modern education, now how to do bold text in Excel

      1. Tom 13

        Re: only people that know basic physics are 40+.

        and even then, if it isn't your day job, you likely have forgotten much of it. I forgot about the atmospheric pressure angle on the pump even though I do know how a barometer works and that mercury was originally chosen specifically because of its high specific gravity.

        And I thank all those who did remember for their explanations.

        As this story proves yet again, its awfully difficult to make something foolproof, because fools are so damned ingenious. Even the non-sentient ones.

  15. Barry Mahon
    Facepalm

    I suspect the problem is partly because the building is OLD. If you look closely at the street view of the building there is a sign in stone saying "international telephone building" probably a former Ma Bell facility. So, not strong enough for stuff on the roof?

    The real question is no redundency? amazing lack of risk assessment, lower Manhatten is below sea level isn't it?

    1. Richard Gadsden
      Boffin

      The generator is on the roof

      But you can't have fuel tanks above a certain size that high because of the fire risk

  16. Anonymous Coward
    Anonymous Coward

    Joining forces?

    Helping each other out? That sounds a bit socialist to me. Whatever happened to the good old winner takes all it's-a-jungle-out-there American way? The place is going to the dogs.

    1. Destroy All Monsters Silver badge
      Holmes

      Re: Joining forces?

      That kind of drama place only exists in Progressive's hindbrains, which is why they are hellbent on forcing everyone to help each other all the time.

  17. Arrrggghh-otron

    I wonder how many hand cranked pumps* and lengths of pipe it would take to stage a drum on every stair landing and man each pump hand cranking fuel up to the next landing. Gotta be better than lugging the drums up the steps.

    *Idle curiosity, I know they said that pumps are in short supply.

  18. Anonymous Coward
    Anonymous Coward

    Looking after those guys...

    Speaking from the inside prone of those companies, we are incredibly proud and slightly shell shocked at the lengths these guys are going to to keep things running. I know a hotel room has been arranged locally so they can rotate off and get some rest and more guys are being flown in from other locations to help.

    They are bei and will be looked after, the reason they're doing it probably has something to do with the fact that the company looks after it's staff superbly normally (a few cold ones waiting? We do that every Friday for our guys, so for this we might manage something a bit better).

  19. Snowy Silver badge

    Why not suck it up 2 floors and blow it the rest of the way?

    1. richardcox13

      I would expect Peer1 (who run the DC) will be looking at that in the future.

      The fuel tank may well be stuck in the basement by local planning (zoning) regulations, but maybe putting the pumps in the ground or first floor (1st or 2nd for USAians), maybe with a submergeable booster next to the tank (to get things moving).

    2. David Pollard

      Why not suck it up 2 floors ....

      a) Because of the difficulty of priming it and/or keeping it primed. It's not easy to make a self-priming pump because it would have to be able to generate a near vacuum at the inlet and also pump liquid out under pressure. It's quite difficult to do both efficiently. Alternatively, if the pump requires priming, there will have to be a non-return valve near the tank end. As mentioned in one of the posts above, such arrangements tend to be unreliable.

      b) Because it's more convenient to have the tank and pump more or less adjacent and in a part of the building that isn't going to be used.

  20. Anonymous Coward
    Facepalm

    Fido can wait.

    Ask the hosted sites if they want to stay up with people killing themselves or shutdown for a while and get put on a list of sites prepared to "Put others first".

    I don't know how much fuel and run time can be saved by each service/server going down but I'd hate to think I killed an important service sooner because I was precious about my train spotting or pet dogs blog.

  21. Anonymous Coward
    FAIL

    Only 17 Floors......

    Ummmm 17 floors is actually NOT that high....

    Like the specific gravity of diesel is about 860g / L, so assuming 4 meters per story, that is about say 76 meters in height.

    I can't be fucked working it out now....

    But I was thinking that having a manual pump, and an empty 55 galleon drum every say 3 or 4th floor, and pumping the fuel up in short stages - and then just moving the pump, up 3 or 4 flights of stairs, every time a drum is filled - that is a lot more energy efficient than people dragging small amounts of fuel AND themselves up all the stairs as well.

    Anyway, what is just so fucking important about keeping one set of servers online in a shit hole of a disaster?

    Like manning soup kitchens and helping heaps of other people clean their shit infested flooded houses and business's out, cleaning up the streets, getting super markets hosed out, helping to clean out and gather up all the trash etc..

    Fuck the servers.

    1. FutureShock999

      SLAs

      Almost without doubt, those hosting companies have Service Level Agreements that stipulate how much percentage of uptime they must maintain to their customers - it won't be 100%, but it will be fairly high (99.5%+). And if they fail to maintain that, then those same SLA contracts stipulate that they will be legally liable for damages to their customers, and will need to pay compensation. For an extended outage like this, that actually has the potential to be hugely damaging to those companies, if not potentially put smaller ones out of business.

      Hence why those guys that have jobs there are killing themselves lugging diesel up 17 flights of stairs - because they would like to maintain those jobs by preventing the company from going tits up. Which, when this storm ends, will be very, very important, because the economic damage of this storm could lead to some set-back to the recovery, putting jobs in shorter supply. (Of course, it could be a very good time to be in the building trades for a while...). I hope the execs give them all a nice few stock options for the work they are doing saving the companies involved...

      1. Mr Anonymous

        Re: SLAs

        SLA, compensation is usually limited to the charges made in any one period and if it's some special insurance backed contract, probably not covered by 'acts of god'.

        1. Tom 13

          Re: probably not covered by 'acts of god'.

          They almost certainly do.

          But regardless, the PR value of them doing this is enormous.

    2. Anonymous Coward
      Happy

      Re: Only 17 Floors......

      All you stupid shits that have voted me down - basically because you ARE stupid shits - that have never been in a situation like this...

      In Bumfuck Nowhere Australia - our whole town got wiped out in huge floods about 19 months ago.....

      Everyone got out alive, but everything on the ground and in all the houses was wiped out.

      After the floods, just take the entire contents of all your homes and business's - pile them out on the street and the council sent around HUGE mining vehicles - the earth moving front end loaders and mining trucks and carted all the floor coverings, furnishings, cars, everything...

      There is so much cleaning up to do - and things are just wiped out for fucking ages.....

      And there are a lot of people who lose everything......

      In a disaster area, just switch the servers off for a month and go hose peoples houses out or something.

      Fucking morons.

  22. Mr Anonymous

    It's simple, do you design for a one in one hundred year event. Usually No, if hundreds or thousands of lives at risk, yes. Deaths due to Gawker being down... 0.

  23. Anonymous Coward
    Anonymous Coward

    Err...

    Who runs a hosting service from Manhattan? And who runs a hosting service from a sky-scraper? A nice anonymous warehouse type building up state where the real estate prices are far smaller and you can put in drainage would seem a lot more sensible.

    Also a failover site as well....

    Also, also, carrying buckets of fuel up the stairs of a skyscraper which has partially at least lost power, during a time of national disaster. What are they thinking? There are more important things than keeping a few web sites up, such as the personal safety of their employees.

    Heroic, yes, foolish, definitely.

    1. Tom 13

      Re: Err...

      I'd take carrying the diesel over mucking out the flood waters any day of the week. A well defined an manageable set of risks over ones that are completely unknown. What exactly got into all that salt water as it flowed over the railroad tracks lifting off the tankers and other cars, through the chemical manufacturing plants in New Jersey and on and on.

  24. Anonymous Coward
    Anonymous Coward

    Does nobody here know what a bucket brigade is?

    If anybody does, I've missed it, sorry.

    For those who think this involves people physically carrying buckets up and down flights of stairs - it needn't.

    You get a brigade of people - enough people to form a line of adjacent people between where the buckets are filled and where the buckets are emptied.

    The people DO NOT MOVE.

    The bucketfiller person at his/her end fills an empty bucket and passes it to the adjacent person who passes it to their adacent person. Repeat until full bucket reaches bucket-emptying end. Then empty buckets are passed back the other way.

    The people DO NOT MOVE. Only the buckets move.

    Have we got it yet?

This topic is closed for new posts.