back to article 'The server broke and so did my back on the flight to fix it'

Reg readers are awesome and none more so than JT Smith, who sent us an epic tale of the time he was called out to solve a backup problem. It started with a chap called “Hubswitch”, who was in charge of technology at a remote site that JT oversaw. Hubswitch earned his name because he kept using the word to describe any piece …

  1. Anonymous Coward
    Anonymous Coward

    That is a really idiotic design choice.

    Notwithstanding this engineer, that is the kind of situation you're likely to be called in to look at at 3AM. One keypress from obliteration without so much as a warning message when you're only half awake is seriously not what you want to deal with.

    1. graeme leggett Silver badge

      Re: That is a really idiotic design choice.

      "There were two options on the screen. The first was 'No'. The second with the bold default lines around it was 'Okay'"

      Isn't there a rule that the option that does least 'damage' should be the default?

      1. John Sager
        FAIL

        Re: That is a really idiotic design choice.

        It's obviously Apple following the Microsoft dictum of making stupid choices for the unwary. Though they would cast it as being user-friendly for the complete newbie (new to this? Ok, we'll format the disk & then you can load up OS-X from the DVD you have in your hot little hand).

        Quite why you would want a server to behave this way is beyond me though.

        1. Gene Cash Silver badge

          Re: That is a really idiotic design choice.

          I like to think of it as the BOFH GUI designer going "READ the damn dialog before blindly punching buttons, or take it up the butt, MUHAHAHAHA!"

          I have recently run into the hated practice of double negatives in dialogs in my TomTom GPS:

          "Do you not want to avoid freeways?"

          Yes! No! Wait! What?

          1. gnasher729 Silver badge

            Re: That is a really idiotic design choice.

            My TomTom often says "Do you bla bla bla Toll Road bla bla bla". Once led to a heated discussion with my wife whether to press YES or NO because of double or triple negatives. It would have been so easy to have a button "Toll" and a button "No Toll". Especially since reading and thinking about the meaning of a sentence while driving is not the safest user interface.

            1. This post has been deleted by its author

              1. Tom 13

                Re: "Do you want to save? Yes / No / Cancel"

                Seems pretty clear to me and gives you a third option which is sometimes preferred.

                The TomTom one is annoying because they chose the wrong default. Here in the US, if it includes a toll road, you probably want to take it as opposed to losing an hour of your life to traffic on a four hour trip.

                What's worse is that it doesn't necessarily reverse routes for planning. This past weekend I was using one to get to a friend's house. For the first part of the journey it was okay. It took me up I270 to the PA Turnpike, across to Philly and north to my friend's house. For the return trip it wanted me to go south on I95 then back up I270 to get home. Because of this I spent an hour in hyperspace trying to get to the toll road.

              2. J.G.Harston Silver badge

                Re: That is a really idiotic design choice.

                Exactly. When you press a button you are issuing a command, the button should must be labeled with the command it initiates. That's such basic basic user interface design it predates feudalism.

          2. Anonymous Coward
            Anonymous Coward

            Re: That is a really idiotic design choice.

            "...my Tom Tom GPS..."

            The more I learn about Tom Tom GPS, the happier I am that I never bought one.

            Several coworkers own Tom Toms. Reportedly, they periodically need to bring them into the house and connect them to the Interweb in order for the receiver chipset to work properly in regards to boot time. They didn't know any better, not having seen the counterexamples, and so thought that this was normal.

            Crazy.

            Excuse the interruption.

            1. Anonymous Coward
              Anonymous Coward

              Re: That is a really idiotic design choice.

              TomTom's are a POS. Mine now langushes in the cupboard because it flatly refuses to update the maps.

              I even rebuilt an old laptopn with Windows 7 just for connecting to TomTom. All I get is basically 'connect your device to the PC'. It is effing connected. Windows knows this but your crappy software can't understand that.

              The Support forums are full with posts like this. In the end I just gave up.

              I'm buying a new M'Bike at the end of the month. This one comes complete with a Garmin SatNav so I'll have firsthand experience of both popular makes.

              1. phuzz Silver badge
                Meh

                Re: That is a really idiotic design choice.

                I think nowadays the two main makes of SatNav are Apple and Google.

              2. werdsmith Silver badge

                Re: That is a really idiotic design choice.

                I'm buying a new M'Bike at the end of the month. This one comes complete with a Garmin SatNav so I'll have firsthand experience of both popular makes.

                Welcome to the wonderful world of Garmin uncommanded restart when approaching a complex junction.

                1. Anonymous Coward
                  Anonymous Coward

                  Re: That is a really idiotic design choice.

                  If you are watching your SatNav rather than the road ahead when navigating a complex junction then frankly you deserve to become an organ donor.

                  Sorry for being blunt but that is how it is these days.

                  Far too many others drivers are bindly following their SatNavs these days.

                  I was lucky and got away with just a dislocated shoulder last year when some Idiot who was just following their SatNav directed them to turn into a One-Way Street the wrong way and we came to a meeting of machines. Needless to say, the tin can (car) won.

                  Posting Anon because I get enough calls from Ambulance Chasing Scum (lawyers) as it is thank you very much.

                  1. ChrisC Silver badge

                    Re: That is a really idiotic design choice.

                    Who said anything about *watching* the satnav? Or are Garmins unique in the world of satnavs in not having a voice guidance ability?

                    1. TeeCee Gold badge

                      Re: That is a really idiotic design choice.

                      Hmm, I tend to sneak a look at the junction layout picture as, with a complex junction, the voice direction for turns, merges and changes is invariably too early, too late or complete bollocks.

                      Easier to look and see where it expects you to end up and then work it out yourself with the aid of the signage.

                      1. Kevin Johnston

                        Re: That is a really idiotic design choice.

                        I remember using one which treated a protected '1st exit' lane at a roundabout as a separate junction so unless you wanted that exit you were always being told to leave the roundabout one exit early.

                2. Captain Scarlet Silver badge

                  Re: That is a really idiotic design choice.

                  "Garmin uncommanded restart when approaching a complex junction"

                  Yes my current model likes to do this, where as my previous Garmin's were rock solid (i3 and C510).

            2. Luiz Abdala

              Re: That is a really idiotic design choice.

              My Tomtom has this button... wait for the signal before showing the planned route or...

              just show the planned route, even if the GPS was turned off in a different place that he is now and cant get a signal just yet...

              Major facepalm.

              JUST SHOW ME THE GODDAMN ROUTE from whatever place you think you are AND LATER YOU FIGURE WHAT THE FUCK HAPPENED WITH THE SIGNAL.

              But other than that, he works better than my android. No advertising, no lock-ups, no facebook beeps, it is great!

              1. Tom 13

                Re: wait for the signal before showing

                Have you ever told it to wait for the signal?

                I did once, and still sometimes inadvertently fat poke it. It's a truly amazing option. It never seems to find satellite signal, so I invariably wind up canceling, then planning again from last position.

                For even more amusement, I've sometimes left the device on all the way home, and am using the "Plan from last Known Position" option in the house. As I walk to the car it finds satellite data, and starts replanning the route.

              2. Anonymous Coward
                Anonymous Coward

                Re: That is a really idiotic design choice.

                > But other than that, he works better than my android. No advertising, no lock-ups, no facebook beeps, it is great!

                *ahem*OsmAnd*ahem*

          3. elDog

            Re: That is a really idiotic design choice.

            It's just another case of some kid with programming and GUI chops who has never been in a situation like the one in this story.

            I've always felt that someone who is designing/developing a system should spend at least a year using something similar in production, a few months answering customer calls, and a few months in QA.

            Better yet, all initial designs/prototypes for a system should be acknowledged and then thrown away with the team sent back to their caves to make it better.

        2. Anonymous Coward
          Anonymous Coward

          @John Sager "why you would want a server to have this way"

          It probably wasn't a server, just a desktop Mac that was put into the role of a server. Apple hasn't made any servers for quite some time, and when they did AFAIK none ever ran OS X but rather A/UX.

          Not that this is a good default for a desktop machine by any means... And surely if the disk is unbootable you'd still want the OS to scan around on it for a bit and see if it can find any hints it has a filesystem on it before offering to format!

          I find it hard to believe that OS X doesn't do just that, and maybe the drive had suffered one of those controller deaths where the drive's data is left inaccessible for reading or writing, and it shows up with a capacity equal to the size of the cache (and Hubswitch formatted the 8MB cache)

          1. /dev/null
            Boffin

            "Apple hasn't made any servers for quite some time"

            Erm, you don't have to go back to the good ol' days of A/UX ...until fairly recently they did a version of OS X called OS X Server (which is now just an "app" for OS X), and it wasn't so long ago that you could buy a proper rackmount XServe system to run it on.

      2. SteveK

        Re: That is a really idiotic design choice.

        Can't come close to matching the suffering of the story, but on the subject of illogical UI design, was upgrading RAID driver on an elderly Windows server a little while back. Downloaded the latest driver from the manufacturer and ran the installer. Was presented with a standard Windows dialog box: "Previous driver detected, click 'OK' [the default] to uninstall, 'Cancel' to upgrade."

        Err, ok. What about if I now decided that I wanted to do neither and just exit instead? I only have those two options (or kill the process...). And why is the default button the one that uninstalls the driver for the disk array which might be fairly vital.

    2. Anonymous Coward
      Anonymous Coward

      What is, having a spine instead of an exoskeleton?

      In fairness, the ID (Idiotic Design) argument is only taught in US schools, most of the rest of the world thinks we just evolved that way.

      1. sisk

        Re: What is, having a spine instead of an exoskeleton?

        In fairness, the ID (Idiotic Design) argument is only taught in US schools

        ID is not taught in US schools. There's a push by the ID believers that direction, and for a (very) brief time schools in one state had the option to include it in the curriculum (after which the state school board who made that decision were shown the exit en masse during the next election), but only in religious schools is ID taught as anything other than...well, religion.

  2. LaeMing
    Unhappy

    I feel your pain (quite literally)

    My manager one pulled me off important work and sent me to the medical centre, I looked in so much pain despite efforts to soldier on. She even escorted me to the taxi rank to make sure I didn't try to walk there (as is my habit).

    Turned out one of my ribs had dislocated from its socket while sleeping badly the previous night and every slightest movement or twitch of my body activated the muscle that was supposed to keep it in its correct place to try to pull it back there, but the muscle contractions kept missing the socket and making it worse in the second most agonising thing I have ever experienced (kidney stones still win that one!).

    Backs are hell when they don't work to spec.!

    1. Doctor Syntax Silver badge

      Re: I feel your pain (quite literally)

      I don't think any believer in Intelligent Design can ever have had back trouble.

    2. dcluley

      Re: I feel your pain (quite literally) (LaeMing)

      The trouble with backs is that spines were designed as beams for four-footed animals. Walking upright we have turned them into columns and the stresses are completely different. Hence the backaches and other problems.

      1. Doctor Syntax Silver badge

        Re: I feel your pain (quite literally) (LaeMing)

        "The trouble with backs is that spines were designed as beams for four-footed animals. Walking upright we have turned them into columns and the stresses are completely different."

        No. They evolved as beams for four-footed animals. Then they evolved into columns. Evolution is effective at optimising things but it can only achieve local maxima. So what we have is probably the best column that can be adapted from an articulated beam but probably not the best column that could have been evolved directly from a notochord and certainly not the best column that could have been designed. And hence, as you say, the backaches & other problems.

        1. BinkyTheHorse
          Boffin

          Re: I feel your pain (quite literally) (LaeMing)

          "Evolution is effective at optimising things but it can only achieve local maxima."

          Well, someone slept through their Heuristics 201. Evolution is wholly capable of breaking out of local maxima, if not within a species, then within an ecosystem - that's what mutations are for.

          Nobody said it's easy, fast, or guaranteed 'though.

          (the rest of your post is spot on)

    3. I ain't Spartacus Gold badge

      Re: I feel your pain (quite literally)

      Ah, back pain memories... Or should I say, Aaaaarrgghhh!!!!? It's astonishing how painful they can be. Rib injury that led to the whole upper back and neck going into spasm for me. Morphine took a couple of days to touch the pain, and I have no memories at all about the rest of that week.

      One thing it's taught me is never to try to soldier on. Loosening excercises every half hour and maybe NSAIDs or muscle relaxants. If you're in too much pain to do stretches, then it's time for the doctor.

      I've learnt to dread that slow progressive stiffening and tightening, with increasing levels of pain, as the muscles start to spasm. And it's a right old bugger to stop major back spasms when they're taken hold. Plus hurting like someone digging knives into your brain. But I've found it mostly can be controlled early on in the process.

      Now if I'd got my domestic robot and jet pack, like Tomorrow's World promised, there'd be a lot less strain on my back. Can I sue the BBC?

  3. Anonymous Coward
    Thumb Up

    Respec'.

  4. Efros

    The trouble with backs

    If your boss has never suffered from back pain then you are a whining skiver.

    1. Sgt_Oddball

      Re: The trouble with backs

      Back I don't mind so much. My neck and previously broken ribs are what does it for me.

      Having to keep your head bolt upright gets very annoying after a while but the ribs like to remind you their broken every now and again for no good reason (and usually only solved so I can breathe again my hitting my chest. Looking like I'm beating myself up)

    2. Cpt Blue Bear

      Re: The trouble with backs

      I had one of those once.

      Right up to the afternoon he slipped on the stairs* and landed on his coccyx on the edge of a tread. The poor bastard never walked quite the same and watching him get up from a meeting room chair after half an hour was traumatic. The following few months were interesting as by mid-afternoon he was usually out of his tree on codeine.

      The upside was you only had to start doing lower back stretches to get sent for a break and anything up to three days sick leave (the maximum he could authorise) got waived through as long as it was back related.

      * I swear, officer, I was no where near at the time.

  5. Roq D. Kasba

    'Lost lift'

    I doubt you did, relative to the air around it your plane carried on without any drama - it was the body of air having a wriggle that caused you the sensation.

    Sorry to be a dick in mentioning it, but we're all engineers here and engineering terms have very specific meanings in our world ;-)

    1. Anonymous Coward
      Anonymous Coward

      Re: 'Lost lift'

      An aircraft crossing a windshear boundary can suddenly find its airspeed drop. This can cause hopefully-temporary loss of lift, until the pilot puts the nose down to regain speed. These can happen at altitude, for example over Greenland for one famous incident, where a large aircraft 'lost lift' and several people were injured.

      'air having a wiggle', in the case down, would be a downdraft. Downdrafts are typically found close to the ground, under thunderstorms. Usually on approach into an airport in the southern USA it seems.

      So it's not clear that your correction in necessarily correct.

    2. I. Aproveofitspendingonspecificprojects

      Re: 'Lost lift'

      Would the engineers working for the airline have mentioned anything about Cb when they sold him the ticket?

      Sounds like a nice get rich slow scheme for a lawyer. Also the dickheads at Pimple should be asked to pay for selling a dick a dipswitch.

  6. streaky

    Hubswitch..

    Hubswitch earned his name because he kept using the word to describe any piece of kit he thought might be the problem

    We sure it wasn't because he was one of those people who couldn't tell the difference between them, which not all that many years ago was shockingly common amongst allegedly qualified network engs?

    1. Phil O'Sophical Silver badge

      Re: Hubswitch..

      The previous generation always blamed problems on "the picture valve" :)

      1. eJ2095

        Re: Hubswitch..

        Oh god yes i rem this....

        Back in the early telewest days where people where advised by pcworld to buy a switch (Not me i knew better)

        Then i get a call to go and see why its not working with the cable modem... sure it was a early Surfboard 4100. (Thats the modem)

  7. Mr Dogshit

    "Apple server"

    Now there's an oxymoron. Funny how they don't make them any more.

    1. streaky

      Re: "Apple server"

      The thing I always remember about Apple servers was it's the thing they could actually do competently in a world where Linux wasn't the big deal it is today. Weird how times change.

    2. MJI Silver badge

      Re: "Apple server"

      For years I never realised you could buy a server without Netware!

      And I have come across some shocking PToP systems.

  8. Will Godfrey Silver badge
    Thumb Up

    Back Problems

    Always a great joke... until you suffer from one. Boy, do the jokers clam up fast then!

    In this case "Beyond the call of duty" comes to mind.

    1. Alan W. Rateliff, II
      Paris Hilton

      Re: Back Problems

      A number of years ago I was involved in a very bad vehicle collision in which the otherwise pleasant driver of an SUV traveling 40mph (give or take) tried to park said SUV, without applying the brakes, in the trunk of my fully-stopped compact sedan. Not many months afterward I called up my father and told him that I get it now, then apologize for all those times I was younger when I became aggravated because whatever the family was doing was brought to an abrupt end because "your father's back is hurting him."

      As I accumulate years I find myself empathizing more with old farts and poking less fun at them.

  9. Chemist

    so he scoffed “a huge handful of ibuprofen and acetaminophen”

    Just got to point out that an overdose of acetaminophen ( paracetamol) is VERY dangerous and will often prove fatal. I hope everyone knows this but ...

    1. This post has been deleted by its author

      1. phuzz Silver badge

        Re: graeme@the-leggetts.org.uk

        Not to mention that paracetamol is no more effective than a placebo for back pain.

        1. Will Godfrey Silver badge

          Re: graeme@the-leggetts.org.uk

          That's a rather strange study. Paracetamol is only intended for short term pain relief. It says so on very packet I've seen, so why are they using it as a long term recovery strategy?

          Indeed, they make no mention at all of pain relief.

          1. Chemist

            Re: graeme@the-leggetts.org.uk

            "Paracetamol is only intended for short term pain relief. It says so on very packet I've seen"

            No it's often used for chronic pain like that of osteoarthritis. OTC bottles of all sorts of drugs are required to tell users that they should only use them for short period.

            Paracetamol is relatively safe at normal usage levels but recent studies have suggested that chronic use at higher than recommended levels whilst not acutely toxic can in time lead to liver damage.

            1. John 62

              Re: graeme@the-leggetts.org.uk

              Last I heard about paracetamol and pain relief (last year, radio 4) is that paracetamol on its own is crap at pain relief, but it can increase the effectiveness of other analgesics.

        2. streaky

          Re: graeme@the-leggetts.org.uk

          Not to mention that paracetamol is no more effective than a placebo for back pain.

          Works for me. Science. Seriously though it depends what exactly the back pain is.

          Plus also it takes a fairly massive dose of it to do any sort of damage, more than a mentally stable person would actually take.

          1. NumptyScrub

            Re: graeme@the-leggetts.org.uk

            Plus also it takes a fairly massive dose of it to do any sort of damage, more than a mentally stable person would actually take.

            People suffering from severe pain are not renowned for their acute mental faculties and diligent decision making processes. In fact in my experience, otherwise intelligent people suffering from severe pain will often gobble anything that remotely looks like it will help, and keep doing so until it feels like it is helping.

            Or in other words, people in serious pain are not necessarily classifiable as "mentally stable" as per your definition :(

            1. Tom 13

              Re: otherwise intelligent people suffering from severe pain

              I'll second that. I've only ever had one kidney stone, but as I was attempting to get an X-ray at the SECOND facility and they were again trying to pull the "You need to schedule that at least 3 days in advance" crap, I believe I told the receptionist that if I weren't already in such pain I'd likely kill for not paying attention to my doctors orders. Or something to that effect. That attracted the attention of a manager who quickly took over for the receptionist and got me through as quickly as they could.

            2. streaky

              Re: graeme@the-leggetts.org.uk

              People suffering from severe pain are not renowned for their acute mental faculties and diligent decision making processes

              I've always managed, if you're taking that many paracetamol you're gonna notice it and realise you have a problem that can't be resolved by self-medicating, *especially* if you're still in pain. They're not addictive, so there isn't much excuse.

          2. Chemist

            Re: graeme@the-leggetts.org.uk

            "Plus also it takes a fairly massive dose of it to do any sort of damage,"

            ~20 tablets over a few hours or less will do it esp. with a lower weight person or one with an already damaged liver. As the original article mentioned "a huge handful of ibuprofen and acetaminophen” it also appears to depend on the size of hand. I'm sure, in fact, that it was a colourful phrase rather than a measure but with some common drugs it really is too easy sometimes to treat them as relatively harmless

          3. Alan Brown Silver badge

            Re: graeme@the-leggetts.org.uk

            "Plus also it takes a fairly massive dose of it to do any sort of damage"

            The LD50 of paracetamol is 200% of the effective dose - where the effective dose for the average 60-90kg human is roughly 1000mg 4 times in a day. This is one of the closest margins between effective and Uh Oh for any painkiller and the closest for any OTC one.

            As a pharmacist and doctor both pointed out to me, paracetamol is one of those analgesics where taking less than the effective code means you may as well not bother (I get stinking migraines which only respond to a cocktail of analgesics and only if caught early. if not, you lose a day with blinding headaches and throwing up or worse)

    2. SolidSquid

      Re: so he scoffed “a huge handful of ibuprofen and acetaminophen”

      I'd thought it was difficult to kill yourself with paracetamol, but that it did large amounts of damage to your digestive tract. A quick google tells me I'm wrong on that, but that it *is* difficult to kill yourself quickly with it. It's your liver that's destroyed by it, it takes 3-5 days for you to die and is incredibly unpleasant since your liver is slowly dying

    3. Loyal Commenter Silver badge

      Re: so he scoffed “a huge handful of ibuprofen and acetaminophen”

      I wouldn't advise an overdose of ibuprofen either, unless you like vomiting blood.

      1. Alan Brown Silver badge

        Re: so he scoffed “a huge handful of ibuprofen and acetaminophen”

        Never take any NSAID without food. I like the lining of my stomach where it is, thanks.

    4. sisk

      Re: so he scoffed “a huge handful of ibuprofen and acetaminophen”

      Just got to point out that an overdose of acetaminophen ( paracetamol) is VERY dangerous and will often prove fatal.

      That's no joke. Acetaminophen is one of those things that the more you learn about it the more you wonder why the hell it's legal. It's dangerous stuff.

      1. Anonymous Coward
        Anonymous Coward

        Re: so he scoffed “a huge handful of ibuprofen and acetaminophen”

        Actually what I think is "Why is it not sold with the antidote already in it?".

        Methionine is the stuff that's needed, in some countries it is compulsory to supply paracetamol with the appropriate percentage included.

  10. Stuart Castle Silver badge

    "The system came up and said it couldn't find a boot disk, but hey, there's a hard drive here would you like to format it?"

    Hmmm, I may be misreading this here, but I have several years experience with Mac, both on OS and OSX, and both Client and Server side. I've never once had a Mac offer to format the boot disk.In fact OSX will prevent you formatting the boot disk, at least from the GUI.

    That's not to say you can't do some monumentally stupid stuff with it (mucking up the network connection while connected remotely is a favourite of mine, and something I've also managed in Windows).

    1. Anonymous Coward
      Anonymous Coward

      Yes, that part of the story seems a bit off. I've seen drives fail where the controller goes leaving it impossible to read/write from the disk, and it shows up with a capacity of however big the cache is. You can read/write from the cache, and since the cache would appear to be all 0s when it powers up, I could see the OS helpfully offering to format it. I think that's what may have happened here.

      I even managed to bring such a borked drive back to life once by swapping the electronics out from another identical drive (same model) I read a lot of warnings about why that won't/can't work, but I had nothing to lose since the most recent backup (this was my home PC) was a few weeks old and it allowed getting back all the data fully intact.

      1. Sgt_Oddball

        I always wondered why eBay seems to do good business on dead hard drives.

      2. Anonymous Coward
        Anonymous Coward

        "I even managed to bring such a borked drive back to life once by swapping the electronics out from another identical drive [...]"

        One of the youngsters had a dead drive with some of his newly composed songs on it. Unfortunately he hadn't bothered using the USB drive he had been given for back-ups.

        I had several other drives of that model - and one had the same rev numbers on the controller board components. Had high hopes of a "miracle" - but to no avail.

        Nice to know that strategy can pay off sometimes though.

        1. This post has been deleted by its author

          1. Anonymous Coward
            Anonymous Coward

            It would have been early 2000s when I did this, and yes after I got the data off it I trashed the thing.

        2. sisk

          I had several other drives of that model - and one had the same rev numbers on the controller board components. Had high hopes of a "miracle" - but to no avail

          As I understand it your odds of successfully performing that particular miracle increase substantially with access to a laboratory clean room. I probably wouldn't even attempt it without that.

          1. Anonymous Coward
            Anonymous Coward

            Why would a clean room matter? We're talking about removing the circuit board, not the platters. Try it on a dead drive, you'll see nothing involving a clean room, or even a "has been vacuumed and dusted in the past three months" room is involved. You just have to be careful you don't zap the electronics with static electricity, so probably using a $3 wrist strap might be advisable (can't remember if I used one or not, but I know I have one around here somewhere I haven't used in forever)

      3. Naselus

        "Yes, that part of the story seems a bit off"

        The whole story sounds a tad fishy to me, tbh.

        An Apple server on OS/X in a production environment running critical file storage? A CEO deciding to take his chief techy on a tour of a new site during a major server outage (after the techy has flown in specifically to look at that outage), rather than demanding that the problem be fixed immediately? No backups taken for 5 months? 3 slipped disks from a little air turbulence?

        Yeah, having some trouble believing any of it, tbh.

        1. anothercynic Silver badge

          The CEO didn't think that...

          ... Someone else did.

          If you read the article *carefully*, you will notice that the CEO was rather discomfited to see him at the plant instead of at the office where the problem was.

          But hey... Details, details!

        2. Anonymous Coward
          Anonymous Coward

          > 3 slipped disks from a little air turbulence?

          Severe turbulence can and has caused major injures, amongst which broken limbs and spinal injuries are predominant.

          With that said, I'm amazed that someone would be so stupid and irresponsible to put a work task ahead of his own health and well-being (exception made of a limited number of professions).

  11. Stuart Moore
    WTF?

    CEO...

    "That plan was derailed by a request to visit a new plant because the CEO would be there."

    ... wait, what? Why on earth did someone think that was a useful use of your time?

    I'm not calling you a liar, but I am trying to understand the logic that goes "We need you to go out and fix this server ASAP.... but first go on a site visit you're blatantly not needed for since you happen to be passing"

    1. I. Aproveofitspendingonspecificprojects

      Re: CEO...

      Sounds like the fool left in charge once the idiot left was also due for a sacking. At least you got to meet the fool who hired and continued paying monkeys.

    2. Steve Davies 3 Silver badge

      Re: CEO...

      IMHO, this is not that uncommon.

      You get sent to god knows where on the planet to fix the mess that has somehow happend and you get 'summoned' to the local head honcho's office for a briefing as to why it is so important that you get the site operational ASAP whch is just what you would rather be doing than sitting in his office (it is usually a man who needs to satisfy his self importance...) drinking frankly awful tea/coffee.

      Yep. Been there done that. Thank god for modern network connectivity. Saves the visit to the CEO's office.

      1. Alan W. Rateliff, II
        Paris Hilton

        Re: CEO...

        I am happy to say that on more than one occasion I have terminated such a discussion or eliminated such a meeting request by explaining up-front that I would love nothing more than to spend time, hours perhaps, sitting around and chit-chatting, using terminology which requires even more time to explain, speculating as to the myriad causes of a problem I have yet to identify, and fielding questions to which I have yet to discover an answer. Particularly being that I am paid for every minute; but perhaps the money would be better spent allowing me to get the job done as quickly as possible, then delivering my analysis in a rather brief, 15 minutes at most, meeting once all is identified, fixed, and prevention recommendation established.

        Fortunately, once the crisis is over, management has decided to give themselves the remainder of the day off for doing a good job in concluding the crisis at hand and these meetings rarely ever happen. Instead, I can whip up a detailed report and email it off to whomever is ultimately responsible in the first place.

        1. chivo243 Silver badge

          Re: CEO...

          Up Vote! And also when you're hub deep in a ferris wheel, trying to fix the disaster, and you're called out of the trenches to be asked "how long before my people can work again...." Ask me again at the end of the meeting, I will add the length of the pointless meeting on to the estimated time of remediation... these are the times I wish I was a contractor.

          And add 45 seconds for each time the phone rings....

          I have to wonder if these guys would pull the surgeon working on a loved one out of the operating room to have a chat and see how things are going? Really?!

  12. Mike Dunderdale

    Dead hard drive PCD recovery.

    Once recovered a shop's database off one such drive by replacing board with ebay one - of course no backup made (an oversight by the owner that was soon rectified..)

  13. Anonymous Coward
    Anonymous Coward

    One of the worst backaches I ever had was due to taking a rack-mount Compaq UPS on site for installation, and finding that the local IT contact (on whom I had been relying on for additional muscle) wasn't there, and in fact, the only person on site was the wee lass in reception.

    They did (fortunately) have a trolly, but manhandling 50+ kg of lead acid batteries from the car to the trolley and from the trolley into the bottom of the rack had me laid up for an entire weekend. I seem to recall the sodding thing slid off the supporting rails as I fitted it at one point, and I had to pull it back out of the rack and refit.

  14. Nolveys
    Devil

    And The CEO Was Dead

    “I drove up to the new plant and pulled into the parking lot nearly running over the CEO."

    Am I a bad person for wishing that "nearly" was a comma instead?

    1. LaeMing
      Go

      Re: And The CEO Was Dead

      Not a bad person. Just a BOFH.

  15. Daedalus

    Seriously?

    If you are hurt you go home. No company, no job is worth pain, especially as the suits don't give a toss.

    1. Permidion
      WTF?

      Re: Seriously?

      the sentence "having decided that the last thing he needed was a doctor" really tripped me off.

      knowing how serious back problem can be, any sane person would go the nearest hospital to get a check. I would certainly not jeopardize my health for the rest of my life just to have a sever up and running a bit sooner :/

  16. Sebby

    Interesting story, but I still think the real mistake was putting a dreamer in front of a Mac server. Those things just aren't built for idiots, no matter how much they promise to be (and I'm sure many a sysadmin would wish otherwise). I don't think Apple knows how to build servers that aren't overcomplicated and full of moving bits that break if you breathe a bit too hard on them; you really need a Unix head in front of it. Or better yet, just shred all that complexity and put Unix/Linux on the server and raise the bar, accordingly (and sadly).

  17. Anonymous Coward
    Anonymous Coward

    Hubswitch

    No back-ups for five months? That situation ends with an invite to room containing a bottle of whisky, single glass and a revolver loaded with a single bullet.

    1. Rich 11

      Re: Hubswitch

      Why would anyone want to be so generous with the whisky?

      1. Martin-73 Silver badge

        Re: Hubswitch

        Nobody said GOOD whisky

  18. BernardL

    Meetings with management instead of fixing the problem?

    In the immortal words of Cmdr. Spurgeon/Sturgeon/Fish, "We can teach, or we can do. What's your pleasure?"

  19. Hairy Airey

    Seatbelts

    This is why you always fasten your seatbelt when seated on a plane! It stops unexpected turbulence bashing you in the head with the overhead lockers for a start.

    1. Anonymous Coward
      Anonymous Coward

      Re: Seatbelts

      > This is why you always fasten your seatbelt when seated on a plane! It stops unexpected turbulence bashing you in the head with the overhead lockers for a start.

      That is true, however note that back injures can still occur to correctly attached passengers and crew, not to mention the overhead bins coming open.

  20. macjules

    Eh?

    “After your assistant left I decided I really needed to know how the server was setup so I redid it. I haven't had time to setup the backups again. So we really need to get this drive working. Do you think you can help me with that?”

    Ok, couple of things wrong there. First of all, how come it was left that the idiot could redo the server? Secondly, and purely from a server admin POV, how come the idiot was allowed to play with the servers at all? Surely logic dictates that an amateur should be allowed to sit in his own little world in his own little basement office and play with old machines until he knows how they work?

    1. sisk

      Re: Eh?

      Surely logic dictates that an amateur should be allowed to sit in his own little world in his own little basement office and play with old machines until he knows how they work?

      Possibly that's what he was supposed to be doing while he was playing with the server. Hence why he was about to be shown the door.

    2. Alan Brown Silver badge

      Re: Eh?

      "First of all, how come it was left that the idiot could redo the server? "

      This kind of demand frequently comes from manglement.

      Along with "no, we won't pull outside contractors in to do this as you won't know how it goes together if they do it" - sometimes adding 3-6 months to a project.

  21. John Klos
    IT Angle

    Must've really been an idiot

    If you click the "initialize" button in OS X when you're asked, it doesn't actually do anything to any disks - it simply launches Disk Utility. Hubswitch would've had to have actually done some things in Disk Utility to format the disk. Pitiful!

    1. chivo243 Silver badge

      Re: Must've really been an idiot

      he could read, if disk utility came up, then there the tabs:

      First Aid, Erase, Partition and so on

      First aid didn't work... next tab.... huh? better call support...

  22. Matt Bryant Silver badge
    Angel

    Oh dear.

    Now I do feel (slightly) guilty that JT's story seems to prove the theory of Yin and Yang in that my tale has no suffering, quite the opposite! In short, whilst working on a worldwide project, I was asked at very short notice to fly to the States to fix a problem I had already told them how to fix. Their "expert" CIO didn't believe in my simple fix. (Un)fortunately, it being the height of the season, the only seat available at short notice was first class on Virgin, limo at both ends! To make matters more "terrible", the flights and hotel booking were non-refundable. I arrived after a very pleasant flight to find the fix had been implemented successfully an hour before my takeoff by a local sysadmin, but the "expert" local CIO had forgotten to inform the UK office. I was therefore "stuck" with being paid for a week of traipsing round the sights in Washington DC, complete with five-star hotel room (yup, all that was available at short notice at the height of the season), all expenses paid, before actually receiving an apology for the "inconvenience" when I returned to the UK. Sorry JT, it seems you got the Yin for my Yang!

    1. Paul 129
      Devil

      Re: Oh dear.

      Downvote for being "The LUCKY Bastard!"

      1. Matt Bryant Silver badge
        Happy

        Re: Paul 129 Re: Oh dear.

        "Downvote for being "The LUCKY Bastard"!" Understandable. I forgot to mention the irony of how the company manglement was going through one of its regular cost-cutting spasms at the time - "no unnecessary travel, bla bla bla".

  23. J.G.Harston Silver badge

    Using a noun instead of a verb - that should have got him fired on day one.

  24. Medixstiff

    All this talk of SatNav's

    Makes me wonder, am I the only one that likes a good old fashioned UBD and some grey matter excercising.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like