back to article BOFH: Do I smell burning toes, I mean burning toast?

BOFH logo – telephone with devil's horns There's trouble in the state of Mission Control. It has come to the attention of the Director that the PFY and I occupy a slice of priceless real estate complete with six full-height pivoting windows opening out to one of the few views which doesn't have a vast expanse of another …

Page:

  1. Boris the Cockroach Silver badge
    Coffee/keyboard

    !!!!!!

    "And… he’s about to find out why we don’t put server rooms in the basement?"

    "Yep," the PFY says, as he turns the key on the basement deluge control…

    You lot owe me another keyboard.... and a pot of coffee, and a new chair because I've fallen off this one laughing and several 1000 pounds to cover the lost earning because I hurt my back too

    1. Anonymous Coward
      Anonymous Coward

      Re: !!!!!!

      scarily i know when / where this basement idea happened, not servers flooded - just the back up generators got flooded during massive thunder storm where mains elec failed

      strangely the gennys never kicked in for a MAJOR data centre, still the good news was that it was the cheapest place to house them so money was saved until . . .

      1. Gordon 10 Silver badge

        Re: !!!!!!

        Yes we are looking at you Vodafone. Twice in different DC's

        https://duckduckgo.com/?q=vodafone+flood

        1. Mr Sceptical
          Facepalm

          Re: !!!!!!

          It was C&W in our case - back in 2006 they had a datacentre in Leeds fed from two power stations being used to house our new Exchange & File server farm.

          Mains failed one weekend and when the UPS tried to kick in it blew, took down not only all the client machines (ours included), but they were running the whole control room off the same UPS system! Can anyone say Single Point of Failure. They only told us Monday morning when the complaints about lack of emails flooded in. Whoops, sorry, didn't think you needed to know.

          That was a short while after we'd had a project planning meeting with them when one of their reps stood up to talk and immediately knocked a cup of coffee into my mate's new laptop. You couldn't make it up...

      2. LondonGull

        Re: !!!!!!

        I worked somewhere that thought about that scenario during a major rebuild - generators on roof; tick.

        Then the 36in water main burst and flooded --- the basement transformers

        1. Gerhard Mack

          Re: !!!!!!

          "I worked somewhere that thought about that scenario during a major rebuild - generators on roof; tick."

          Had my servers in a place like that. Then the power went out on the coldest night of the year and unfortunately the diesel in the fuel line was frozen....

      3. Dr Dan Holdsworth Silver badge
        Holmes

        Re: !!!!!!

        Reminds me of the several incidents that happened to a major UK University I could, but won't name.

        First off, why you don't cheap out and put only VoIP phones in the datacentre control, when the networking kit isn't on secure power: power goes down, phones go down, management have no way to harangue the operators but the operators have their own mobile phones and can quite easily pass over terse instructions to the management.

        Secondly, datacentres are air conditioned, and need aircon on all the time. So, when the power goes off, and the UPS generators kick in, that is not the time to wonder why everything in the datacentre is a bit less noisy than usual and seems also to be getting rapidly warmer. Cue very rapid machine shutdowns all round.

        Thirdly, when you decide to turn the now rather obsolete datacentre into one absolutely gigantic office, it is unwise to assume that all the various odd machines that used to be in there will all migrate to the new, pay-for-space datacentres you've hired. No, they end up under various academics' desks, in comms closets and otherwise scattered around the place in silly places, and worst of all you don't know where they all are, so cannot apply blanket security policies without random roastings from, for example, the Professor of Cruel and Unusual Geography.

        1. Doctor Syntax Silver badge

          Re: !!!!!!

          "cheap out and put only VoIP phones in the datacentre control, when the networking kit isn't on secure power: power goes down, phones go down, management have no way to harangue the operators"

          That sounds like a win.

      4. The Count
        Facepalm

        Re: !!!!!!

        Just ask the folks in NYC when the server rooms in the basements of many a building in lower Manhattan flooded back in 2012 during "Super Storm" Sandy.

        It was a mess.

      5. Anonymous Coward
        Anonymous Coward

        Re: !!!!!!

        In 2007 our local authority emergency response centre had moved to a nice brand new building.... in the cheap ex-industrial land next to the river downstream from the city centre. Guess what happened next in 2007?

      6. Anonymous Coward
        Anonymous Coward

        Re: !!!!!!

        I was introduced to this concept in the early 90s when a charity client had had their basement file servers flooded. Later the second floor sanctity of the machine room was unquestioned. (Basement / Ground was regarded as optimal previously due to the desire not to build stuff in place with the limits on elevators.

        Similarly even in the early 80s mainframes provided heating to the rest of the building passively as was explained in my milk round tour of the MU5 / MU6 (circa 1980).

    2. DuchessofDukeStreet

      Re: !!!!!!

      "You lot owe me another keyboard.... and a pot of coffee, and a new chair because I've fallen off this one laughing and several 1000 pounds to cover the lost earning because I hurt my back too"

      You might need to start a class action.

      1. hplasm Silver badge
        Happy

        Re: !!!!!!

        "You might need to start a class action.

        I'll send the target sign to hang around your neck... avoid the basement.

    3. 2+2=5 Silver badge
      Mushroom

      Re: !!!!!!

      > You lot owe me another keyboard.... and a pot of coffee, and a new chair because I've fallen off this one laughing and several 1000 pounds to cover the lost earning because I hurt my back too

      No he doesn't - you knew the risks when you started reading...

    4. tfewster Silver badge
      Thumb Up

      Re: !!!!!!

      I lost control at "I could pretty much guarantee it"

  2. Toltec

    the basement deluge control…

    Why am I visualising massive water tanks filled with sharks?

    1. Jason 24

      Re: the basement deluge control…

      And frikkin' lasers!!

      1. magickmark

        Re: the basement deluge control…

        And sharks with lasers!!!

    2. DropBear Silver badge

      Re: the basement deluge control…

      "Why am I visualising massive water tanks filled with sharks?"

      Because you're probably thinking of the Chernobyl bubbler pools. ...wait, did you mean "sharks with lasers"? Not radioactive ones? Oh, my bad...

      1. fajensen Silver badge
        Pint

        Re: the basement deluge control…

        Great Link! I shall find some way to use the term "Void Coefficient of Reactivity" in my next financial report on my project (I have a bet on whether anyone, machine or human, ever read these things).

  3. Alister Silver badge

    A vintage bit of BOFH

    Thanks Simon.

  4. Doctor Syntax Silver badge

    Nice false ending in the middle.

  5. Sixtysix
    Pint

    Glug...

    ...glug glug

  6. JimC Silver badge

    'No decisions have been made yet',

    > "Whenever someone has to point out that

    >'No decisions have been made yet',

    > it pretty much means that those decisions

    > HAVE been made," I note.

    How very true. It means "the final rubberstamp is hovering over the paper"

    1. TRT Silver badge

      Re: 'No decisions have been made yet',

      I've always taken that to mean that the final solution has been arrived at without any actual decisions having been made because that would mean someone would have had to think in order to make the decision.

    2. Fr. Ted Crilly

      Re: 'No decisions have been made yet',

      Very much like 'its not been set in stone yet'....

      1. Mark 85 Silver badge

        Re: 'No decisions have been made yet',

        Very much like 'its not been set in stone yet'....

        Translation: The concrete is still wet.

    3. My other car is an IAV Stryker
      Flame

      Re: 'No decisions have been made yet',

      I know the opposite: "The plans are all set; I just need funding." -- MEANS: "I have funding to sit and twiddle my thumbs but will NEVER see a DIME to improve the kit in here."

      BUT if funding just HAPPENS to come available, they quickly succumb to feature-creep:

      1. Start the upgrade project.

      2. Get halfway done.

      3. Add one more bit of kit off-plan to add features.

      4. Realize they need to re-do half the previous work to make space / ensure compatibility / etc.

      5. Repeat every time progress hits 1/(2^n).

      Result: a never-ending upgrade "project".

      At work, this is ongoing in a vehicle lab involving the chassis dynamometer control system, which includes new visual and infrared cameras, displays for visiting vehicle control systems (laptops), and the like.

      Even modern churches have this issue: First was a new audio desk (i.e. digital mixer/fader board) and video projectors. Then new projectors. Then they switched from VGA to HDMI-over-Ethernet due to resolution issues. Then stage renovations with new audio junction boxes, including a new keyboard and direct-wiring in the Hammond organ (with the Leslie speaker unit in parallel). Then even the new boxes HAVE to get wired to a new Ethernet-connected remote audio interface unit that supposedly works seamlessly with the main desk. Soon the Leslie is going to be relocated. (Have fun running THAT cable now that the stage is done!) And us in the "orchestra" just want our monitors to work so we can stay on beat -- gah!!!

  7. Alien8n Silver badge

    Basements

    Reminds me, I think it was Paddington Exchange that took down the whole of the UK's banking systems during the floods a few years ago. Somehow the flood actually caused a fire in the basement exchange. And for some inexplicable reason the exchange was a single point of failure for the entire banking network...

    1. TRT Silver badge

      Re: Basements

      Paddington Exchange? Does that explain the Great Marmalade Sandwich Drought of 2011?

  8. Alan W. Rateliff, II
    Pint

    Good on ya, PFY

    Good to see the young man taking more initiative.

    Yeah, a lot of recognizable decision making code-word in this one. Any time management walks into a room and starts "surveying," "assessing," or anything which implies making a plan one must realize there is no plan there is only do. The decision to move forward on an idea was made before the idea was even fully formulated, inspired by a half-read blog post or magazine article. Half-read and quit before the section on caveats and pitfalls, which is always at the end, anyway, since the person writing in the first place lead with the meat to make the idea seem good or practical, putting all the realistic reasons why it will not work way in the back knowing eyes will have glazed over by then.

    Like those news stories, "Is your cat plotting to kill you? We'll tell you why at 10." The foregone conclusion is in the headline and you should immediately suspect the opposite is true.

    1. earl grey Silver badge
      Unhappy

      Re: Good on ya, PFY

      Is your cat plotting to kill you?

      You know my cat then?

    2. JimC Silver badge

      Re: Good on ya, PFY

      If you really want to feel old, calculate how old the PF"Y" must be now...

      1. Andy Scott

        Re: Good on ya, PFY

        All I'll say to that is older than me

    3. WolfFan Silver badge

      Cats=Evil

      "Is your cat plotting to kill you? We'll tell you why at 10." The foregone conclusion is in the headline and you should immediately suspect the opposite is true.

      You, sir, a a victim of the Great Feline Conspiracy. Of course your cat is trying to kill you. That's what cats do. They rub up under your feet to trip you so that your throat gets in range of their claws even if you manage to avoid breaking your neck in the fall. Cats Are Evil(tm).

      1. Aladdin Sane Silver badge

        Re: Cats=Evil

        I refer you to Betteridge's law of headlines.

      2. Tom 38 Silver badge

        Re: Cats=Evil

        Cats aren't trying to kill you specifically, they're trying to kill everything. Cute cuddly little psychopaths.

        1. Dave Bell

          Re: Cats=Evil

          I would venture that problem cats are the product of problem people.

          1. TRT Silver badge

            Re: problem cats are the product of problem people.

            Did no-one ever explain...?

            You see when a mummy cat and a daddy cat are very much in love with each other...

            1. Steve Knox Silver badge

              Re: problem cats are the product of problem people.

              You see when a mummy cat and a daddy cat are very much in love with each other...

              Is that what started all that mess in Egypt way back when?

              1. TRT Silver badge

                Re: all that mess in Egypt way back when?

                Way, way back, many centuries ago, not long after the Bible began,

                Jacob lived in the land of Canaan, a fine example of a family man.

                Jacob... Jacob and sons, working hard on the farm to earn their keep.

                Jacob... Jacob and sons, spent all of their days in the fields with sheep.

      3. My other car is an IAV Stryker
        Big Brother

        Re: Cats=Evil

        For some folks -- like my wife -- there is no plotting involved. Get a cat close enough and it might kill her due to her severe allergy, which also crosses over to many dog breeds.

        A former neighbor had a cat. If I went to talk to them, I'd have to immediately wash my hands after coming back in the house.

        Cats: So evil, they're unapproachable.

        (So why do our daughters think they're SOOOOO cute? THAT must be part of their plan.)

    4. Primus Secundus Tertius Silver badge

      Re: Good on ya, PFY

      Cats are always plotting. If not to kill you, then to do something else.

    5. phuzz Silver badge
      Stop

      Re: Good on ya, PFY

      You cat is not planning to kill you. If it did that, who would open the cat food?

      Of course, you don't need your legs to use a can-opener...

      1. Blofeld's Cat
        Coat

        Re: Good on ya, PFY

        "You cat is not planning to kill you. If it did that, who would open the cat food?"

        No, that's what they want you to think.

        A Real Cat would simply step over your corpse and acquire another human with appropriate tin-opening skills.

    6. shawnfromnh

      Re: Good on ya, PFY

      "Is your cat plotting to kill you? We'll tell you why at 10." The foregone conclusion is in the headline and you should immediately suspect the opposite is true.

      YEAH, you won't be alive at 10 to see why because.....

      1. TRT Silver badge

        Re: Good on ya, PFY

        The sheep carcass on the back doorstep was just a warning.

  9. Michael H.F. Wilkinson Silver badge

    Epic episode!

    Loved the use of the "Beware of the Leopard" bit. Interesting rhythmic devices that counterpoint the surrealism of the underlying metaphor ..

    I'll get me coat

    Doffs hat to the late, great Douglas Adams

    1. Anonymous Coward
      Anonymous Coward

      Re: Epic episode!

      Used to work with a head of behaviour once who had a sign on the door "beware of the tarantula".

      She wasn't joking, there was one on her desk. Great way to make kids behave.

  10. chivo243 Silver badge
    Alert

    decisions HAVE been made!

    I've suffered through this exact nightmare... Finance officer and Facilities manager sauntering through our serverroom/office taking notes. Now the server room is 1/4 it's original size... and two additional offices have been shoehorned in...

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019