back to article Error pop-up? Don't worry, let's just get this migration done... BTW it's my day off tomorrow

Welcome back to On Call, where you get to take a breather and enjoy tales of tech support adversity from your peers. And taking a breather is exactly what this week's Reg reader, "Adrian", needed to do at the end of this particular incident. About two decades ago, Adrian was the network administrator for a medium-sized …

  1. Oliver Mayes

    So the only administrator for a business critical system just took the day off during a major infrastructure change?

    That sounds like terrible planning from the company to me.

    1. Version 1.0 Silver badge

      took the day off

      That's quite normal, the PHB probably thought that everything would go fine and the staff would fix any issues - no need to miss a nice day for golf. Turned out that way too - the PHB probably got a nice bonus at the end of the year for the successful migration too. Welcome to modern business management.

      1. Anonymous Coward
        Anonymous Coward

        Re: took the day off

        unless something goes wrong that involves upper manglement attention to fix, they never appreciate the complexity and difficulty in achieving things.

        I've done a several multisite (10+) WAN migrations in different jobs, i remember one was global and had the full attention of the business leaders as it forced the multi billion $ business to go manual on its 24/7 business operations whilst we cut over, only a few hours of disruption of which downtime was less than 5 minutes but we had the boards full attention, blessing and thanks for completing such a complex task on time with minimal downtime and disruption.

        Another one was a bit smaller (still multi billion £ turnover 4k+ uk staff), no one in the business cared about the potential for disaster and after a year of planning to have the whole weekend from close of business ~ 7pm Friday to 6am Monday to do the work the organisation decided 4 days before the work started that they needed the network back on the Saturday afternoon. Started ~ 8pm Friday finished ~ midnight and caused no issues but found tons of odd stuff to work on later (stupid async routing across stupidly implemented dual links across stupid other stuff) but we had next to none business testing only what i could test and what those that needed the live service on the Saturday worked on. No one reported any issues though as no one noticed any issues, barely got a thanks that time.

        You never get thanks for things people don't appreciate.

        1. Anonymous Coward
          Anonymous Coward

          Re: took the day off

          If it works nobody cares and nobody pats you on the back.

          If it doesn't work everyone cares and they are all ready to kick your ****

          This is why I left IT Support. Doesn't matter why things don't work, if you are the nearest IT professional you get it in the neck regardless. I can't think of any other department where that happens.

          1. Anonymous Custard Silver badge
            Headmaster

            Re: took the day off

            One of the great maxims of life:

            If it goes right no-one remembers.

            If it goes wrong no-one forgets.

            1. steviebuk Silver badge

              Re: took the day off

              Although everyone remembers Y2K because nothing happened. So everyone not in IT thinks it was all a ruse by IT techs to get paids loads. Ignoring the fact nothing happened because most issues had been fixed before hand.

              1. Alan Brown Silver badge

                Re: took the day off

                "Although everyone remembers Y2K because nothing happened."

                Several things happened in the leadup to it.

                Most of the world's alarm systems were discovered to be set to malfunction on 9/9/99 (reported here at the time) as this was used as a "test date"

                A lot of NTP-using switches and routers around the world went titsup earlier in February 1999 when the number of seconds since Jan 1 1970 overflowed a signed long integer - this knocked out virtually the entire Chinese Internet until the cause was realised (the routers were rebooting, and would then reboot again when they tried to NTP sync. The fix was to disable NTP)

                1/4 of New Zealand's phone system went completely titsup for 12 hours due to memory corruption whilst loading in Y2k fixes and corrupted backups meant that it took the best part of 3 months to replay all the database changes from a known-good backup once dialtone was restored.

                Various other things happened, but these were all stuff I experienced personally and they were all reported on this website at the time.

                1. I Am Spartacus

                  My Y2K nightmare happned ...

                  My Y2K nightmare happened on 1/1/2001. It seems the billing engine on the so called "intelligent network" device decided that 1/1/1 was a test day and started tossing billing records in the bin. Somewhere, some people has a great January as we had no data to bill them against.

              2. Chris King Silver badge

                Re: took the day off

                "So everyone not in IT thinks it was all a ruse by IT techs to get paids loads".

                I didn't get paid an extra bean. They even chained up the door to the computer centre just in case the heating control system set fire to everything, that's how confident they were. (I did point out that this was the same type of heating control system that covered student halls, which were still occupied by a few overseas students over Christmas)

                Just after midnight, I quickly checked the most critical systems, said "Meh, my stuff's fine, everything else can go whistle" and went back to partying like it wasn't 1999 any more.

                Some sites turned EVERYTHING off, just in case. Some of them then discovered that leaving stuff powered down but still plugged in won't necessarily save it from a nearby lightning strike. Just when you think you've covered all possibilities in your disaster recovery plan, Mother Nature says "Hold my beer".

              3. Anonymous Coward
                Anonymous Coward

                Re: took the day off

                On the systems I use we still have a license server which is not Y2K compliant.

                It thinks today is 24 December 19118.

                1. jfm

                  Re: took the day off

                  Written in Perl by any chance?

                  1. Gerhard Mack

                    Re: took the day off

                    The maintainers for the date function etc hacked in some weird Y2K compliance by allowing the 2 digit field to overflow to 3 digits. It makes for some amusing output, but it mostly worked.

                2. TeeCee Gold badge

                  Re: took the day off

                  That'll be cyear from the ctime library with a hard-coded century tacked on. Somewhere there'll be a mouldering set of code comments to the effect that this needs reviewing before 2000.

                  I remember that, come Y2K, the fact that cyear is the number of years since 1900 turned out to be one of those things that a lot of people didn't know and just assumed it was the two-digit year...oops.

                  Most high-profile one I found on the glorious day was the US Naval Observatory's clock on the web merrily showing the year as 100...

            2. Sgt_Oddball Silver badge

              Re: took the day off

              The greatest thanks in It is silence, the best reward is your work going unnoticed.

              1. jake Silver badge

                Re: took the day off

                As I tell prospective new clients "It's my job to ensure we see as little of each other as possible".

                Related: I implemented a four hour minimum for on-site visits in (roughly) 1990, a couple years after I went solo. Double on weekends/holidays. A few clients balked at the new rate ... I simply told 'em "Don't call me unless you actually need me".

                A new issue soon arose: Convincing 'em to pay 4 hours for a one minute visit. The old TV repairman's maxim applied, "I'm not charging you for thumping your TV with a screwdriver. I'm charging you for knowing where and how hard to thump your TV, and for showing up to do it". The explanation seems to have worked ... although about nine months ago, a child CEO wondered why I'd need to thump a TV with a screwdriver.

                IT is many things, but it's rarely boring. "May you live in interesting times" may not be an actual old Chinese curse, but it's applicable anyway.

                1. Sir Runcible Spoon Silver badge
                  Trollface

                  Re: took the day off

                  "IT is many things, but it's rarely boring."

                  Since you've been around the block a bit I'm finding this statement to be a bit difficult to swallow, mainly because the number of fuckwits *in charge* of IT teams are boringly predictable in their never ending quest to do the wrong thing and ignore all advice to the contrary because they apparently can't read an email explanation that contains more than one concept.

                  Bah, humbug!

          2. bobajob12

            Re: took the day off

            Payroll.

            Nursing.

            Plumbing.

            It's not just IT. People forget that infrastructure requires work to keep working, whether that infrastructure is your body, your water, or your IT.

            1. mr-slappy

              Re: took the day off

              And midwives. (As the husband of a former nurse and midwife, from my experience many women go into labour during the night. And that stuff just can't wait...)

              1. Anonymous Coward
                Anonymous Coward

                Re: took the day off

                My uncle was an Ob. If it was saturday, and a full moon, he already knew to be around his phone (beeper age) and sleep on the TV couch.

                He had statistics, it was 4x the odds of delivering babies during a full moon, on saturday night, if both were combined.

          3. N2 Silver badge

            Re: took the day off

            "If it works nobody cares and nobody pats you on the back.

            If it doesn't work everyone cares and they are all ready to kick your ****"

            And nobody admits they fiddled relentlessly.

        2. A.P. Veening

          Re: took the day off

          "Another one was a bit smaller (still multi billion £ turnover 4k+ uk staff), no one in the business cared about the potential for disaster and after a year of planning to have the whole weekend from close of business ~ 7pm Friday to 6am Monday to do the work the organisation decided 4 days before the work started that they needed the network back on the Saturday afternoon."

          I would have cancelled the whole migration about five minutes after being informed about that decision and rescheduled for a couple of months later, again a complete weekend.

      2. Aristotles slow and dimwitted horse Silver badge

        Re: took the day off

        Sorry, no. It's not normal at all. As the OP suggests it is actually incredibly bad planning. If either a PM or TM had been that lax on my watch, he would be out of a job for allowing such a basic oversight.

        On Operational day #1 after a major migration or implementation you should be in a Hypercare phase or similar, whatever you want to call it - where ALL key support and admin staff for critical applications are available on a rapidized reponse basis to quickly investigate all issues that arise that may prevent business operations continuing.

        To allow the sole admin or support contact for (what sounds like) a mission critical business application to be OoO on the first day after a major migration or implementation is a schoolboy resource planning error that should have been caught by your pre go-live checklist or G/NG meeting, and then captured as an issue to be impact assessed and agreed with stakeholders at the final Y/N before the migration started.

        It just sounds like shit planning, as the OP suggests.

        1. UberMunchkin

          Re: took the day off

          Of course the real problem here is that there is only one person in the business able to support/administer the software.

          We have a similar issue where I work, I am the only Dynamics Admin for our company and also the only Dynamics developer. So when I take time off there is no support for the product, I don't take work related calls on my days off so if something goes wrong it has to wait until after I get back to work.

          It's nowhere near ideal but also not my fault. The board have, time and again, refused to hire a junior Dynamics person that I can train up.

          Also with projects like this where the date get often gets shifted you can't expect people to constantly cancel and re-book things that they have planned and possibly already paid for just because the delivery date has slipped again.

          1. ArrZarr Silver badge
            Unhappy

            Re: took the day off

            I have considered sending in a story to On Call but the details would be too recognisable to my boss who also reads El Reg.

            Suffice to say we have had one person on 1st level support for 9 months, including on call.

            1. Jamie Jones Silver badge

              Re: took the day off

              ArrZarr? I know who you are!

              YOU'RE FIRED!

              1. Sir Runcible Spoon Silver badge
                Coat

                Re: took the day off

                So, you now have zero people on 1st line support? Sounds like you did them a favour ;)

    2. Anonymous Coward
      Anonymous Coward

      This was common practice, usually the management team of a project would take a few weeks off around the migration date. They justified it because they booked early for after the implementation date and then couldn't change when that date slipped.

      If the implementation went smoothly, they came back to take credit for their good planning, if it didn't they came back to rescue the company from others incompetence.

    3. Diogenes

      When I worked for a large qango, someone noticed a pattern that the Director of IT would be on leave when the results of the salary review came out, so an enterprising db admin wrote a dbtrigger that would notify him, and thus everybody else when the review was about to emerge , sorry director was taking leave (always after 5pm on a Friday), so instead of doing a POETS, we would all stay back

  2. Herby Silver badge

    Reminds me of...

    Pay no attention to the man behind the curtain.

    Of course this is kinda like "on error resume next".

    1. Zippy´s Sausage Factory

      Re: Reminds me of...

      Ah, "On Error Resume Next". That takes me back. Not necessarily in a good way, though, it has to be said...

      1. Anonymous Coward
        Anonymous Coward

        Re: Reminds me of...

        Back?! Gawd you're a lucky one - I still see that one (or its derivatives) on a daily basis!

        1. error 13

          Re: Reminds me of...

          Case Elsevier

          Message box "something else gonna wrong"

    2. Nick Ryan Silver badge

      Re: Reminds me of...

      I used to go pretty much stratospheric at developers any time I saw anything such as "on error resume next". Making stupid assumptions about anything always causes problems somewhere, this used to be one of the worst...

  3. chivo243 Silver badge

    RTFS much?

    Seriously? He didn't read the error message? Didn't press the application guy for more info? He got off easily... What a knob.

    1. Terje

      Re: RTFS much?

      He asked the guy responsible for the application about the error and got told it was nothing to worry about. Unless you have a particular reason to not trust that person then I can't see that he did ANYTHING wrong, you got to remember that error / warning messages can quite often be rather unintelligible unless you know the software rather well. I do hope the application guy got a nice welcome when he got back to work though...

      1. Anonymous Coward
        Anonymous Coward

        Re: RTFS much?

        "Unless you have a particular reason to not trust that person"

        I normally consider breathing sufficient cause to distrust....

        1. ArrZarr Silver badge

          Re: RTFS much?

          Wow, your life must be fun.

          Ask the question, keep the paper trail. Best of both worlds. Anything further is opening the doorway to insanity.

        2. chivo243 Silver badge
          Pint

          Re: RTFS much?

          @AC

          The voice of reason, have one on me!

          +1 as even I distrust myself at times, and ask for clarification, usually when a major freakin migration is about to happen.

  4. Will Godfrey Silver badge
    Unhappy

    Time to move

    If I'd had to deal with such a situation I would have started to make plans as quickly as possible.

  5. mr_souter_Working

    been there - done that

    had all these support calls at one time or another - and this is the tip of the iceberg

    app no longer works - license expired

    Site no longer works - Certificate expired

    Internet not working - didn't pay the bill for 6 months

    1. Doctor Syntax Silver badge

      Re: been there - done that

      "Internet not working - didn't pay the bill for 6 months"

      I had an ISP for my home internet who didn't get paid for a few months because my card had expired. I eventually got a final and only warning. Was it too difficult for them to have emailed a warning the first time payment failed? Sometimes these companies can't look after their own interests as well as they might.

      At one time we used SunAccount. They put a warning on every screen at login about a month before the licence was due for renewal. It played havoc with our screen-scraping program we used to keep account details in sync with the ordering system.

      1. Ryan 7

        Re: been there - done that

        It played havoc with... what, sorry? I must have mis-heard you.

        1. techulture

          Re: been there - done that

          Robotic process automation (RPA), to rephrase it.

          1. Darkk

            Re: been there - done that

            "Macros"

    2. Chris King Silver badge

      Re: been there - done that

      UK web site suddenly selling little blue pills in Canadian Dollar values - domain renewal allowed to lapse. Seen that happen all too often.

  6. Chris Hills

    My story

    I was once asked to go live on an email migration project the DAY before I finished my job, with no handover in place (~1000 users). I hope that worked out for the best.

  7. disgustedoftunbridgewells Silver badge

    At a small firm I worked for, it seemed that the best way to get something to expire or need to unexpectedly buy more capacity in something was to have the boss go away on a business trip, taking the only company card with him.

    1. Anonymous Coward
      Anonymous Coward

      In our case, his PA took care of that. She photocopied the card, both sides, and kept a copy in a special folder.

      1. phuzz Silver badge

        I tend to disdain anyone who has a PA, but PA's themselves are very useful people to be friends with.

        1. big_D Silver badge

          Yes, my father drummed it into me when I was growing up.

          When you go somewhere new, the most important people on site are:

          1. Security / Receptionist

          2. The secretaries / PAs to the top level management

          3. Site services

          and the everyone else

          These days, I'd add BOFH high up on the list, but I'm biased, having graduated from the BOFH School back in the 80s.

          1. Anonymous Coward
            Anonymous Coward

            Big D, I'd add the Admin staff to your list. PAs might be good for getting you an 'in' with their pet manager, but some of them have almost the same percentage of a clue as their manager!

            Don't forget who orders the stationery, puts the holiday bookings on the system, can normally find an answer to ANY problem you have (even the ones that don't have anything to do with their job!), logs the faults when the printer/kettle/toilet stops working properly...

            Anon because I've just done something that will cause my local admin team some grief and I want a chance to get the cakes in BEFORE they find out!

            1. Sir Runcible Spoon Silver badge
              Thumb Up

              I can tell you have experience of messing up, this is an approved method of mitigating the fallout, reducing the punishment level down to the level of 'good natured piss taking'.

              I approve.

        2. Anonymous Custard Silver badge
          Headmaster

          It's one of the first things you learn at uni when you go into post-grad education (and for the advanced player even undergrad education).

          Forget professors and lecturers - if you get on the good side of secretaries, security, storesmen and technicians (including syshacks) you can get absolutely anything done and in an amazingly short time.

          The best lesson I ever learned in education, one that's served me very well in my real life pretending to be gainfully employed (well I pretend to work and they pretend to pay me, so it balances!).

          Editted to add - I see I wasn't the only one to learn that lesson. Nice one big_D!

          1. GlenP Silver badge

            Forget professors and lecturers - if you get on the good side of secretaries, security, storesmen and technicians (including syshacks) you can get absolutely anything done and in an amazingly short time.

            Being on the right side of a technician obtained us the superuser account for a Unix box to which (for no apparent reason) we only had access during limited hours. The box was standalone in a small lab and only we were using it so it was incredibly frustrating to be kicked out of our opsys project* work for a few hours in the middle of the day. From superuser a quick su would get us into our group accounts and we could carry on working.

            *I wonder how many undergrads these days are expected to write a multi-tasking operating system?

            1. MonkeyCee Silver badge

              CS grads

              "I wonder how many undergrads these days are expected to write a multi-tasking operating system?"

              If you're doing a computer science degree, it should be a third year project. At least my buddies who did them had to that.

          2. Chris King Silver badge
            Headmaster

            "Forget professors and lecturers - if you get on the good side of secretaries, security, storesmen and technicians (including syshacks) you can get absolutely anything done and in an amazingly short time".

            Oh hell yeah, and I bet this earned the Prof a "Laser Stare of Doom" for a long time afterwards !

            (Icon, for a Prof reflecting on bad life choices)

            1. Joe W
              Headmaster

              Secretaries at universities...

              Same here. Our department assistant had a secretary. While he was a nice guy and good with all admin stuff he was responsible for his knowledge paled in comparison to hers. She even called me when I was close to missing some exam deadlines. Knew every student I believe, and saved the bacon of quite a few. They don't make them like this any more :(

              1. bemused obsever

                Re: Secretaries at universities...

                Amen Brother Joe! Our university department secretary was a gem at taking care of (often pissy) students. Many times I would have quit out of frustration if not for her; I owe her my PhD. .

                I made up a walnut plaque inscribe "Surrogate Mother of the Year Award" that had scissors cutting red tape. All the other grad students jumped at the chance to chip in to buy a cake and throw a little party for her.

                Here's to you Jean Sanford. May you be treated as well as you treated us students.

          3. Anonymous Coward
            Anonymous Coward

            When I was at school (age 13) I was picked on by one nasty shit of a PE teacher. I remember telling my friend, who's dad (who happened to be the school janitor) overheard and said he would sort it out, which he did.

            I never did learn what it was the janitor had seen the PE teacher doing that gave him the necessary leverage, but the PE teacher was practically human towards me from that day onwards.

        3. Zippy´s Sausage Factory

          I tend to disdain anyone who has a PA, but PA's themselves are very useful people to be friends with.

          All to often it ends up being the PA that does the actual work that the manager gets paid for, which makes me wonder if the existence of the PA isn't just an example of the Peter Principle in action...

        4. Antron Argaiv Silver badge
          Thumb Up

          That's why the ladies at the front desk and the office manageress, the IT drones and their boss get a little something from me every Christmas.

          Nothing significant, but *every* Christmas. I have never missed one. Because when "stuff" happens, these are the folks who are a big factor in getting whatever it is, "un-stuffed"

          Also...Sysadmin Appreciation Day.

          // Take care of the people who take care of you.

        5. Anonymous South African Coward Silver badge

          Our PA drives me crazy with IT requests for help, but I jump - as I can ask for a favour or two when I have such need. (No rumpypumpy but work-related favours).

  8. TonyJ Silver badge

    Trial version of Windows

    Many years ago I got called out to a site in, as I recall, Hammersmith in London.

    Quite a long drag for me as a Midlands lad.

    One of my colleagues had built the customer a shiny new Citrix farm. With an Evaluation copy of Windows NT4 TSE

    Because he erroneously believed that you could simply add a license.

    Which back then, you couldn't.

    And it wasn't as graceful as the reboot every 24 hours. Oh no...this was a BSOD with a license violation error every 24 hours.

    Cue a rebuild of the environment.

    At least it was documented...lol of course it wasn't.

    Unfortunately the guy who did the original build was a few days into a two week holiday abroad so he ducked that cluster-stuff nicely.

    1. Waseem Alkurdi

      Re: Trial version of Windows

      Because he erroneously believed that you could simply add a license.

      Which back then, you couldn't.

      As if you could today?

      On the 90-day Enterprise Evaluation SKU, you could simply never add a license. The most you could do is extend the trial another 90 days for two times (total of 90 * 3 = 270 days). Pas mal, as every Windows installation *needs* a reinstall after that, with the performance degradation and all that.

      But add a license to go to official Enterprise? No way. It's a different SKU.

      ABSOLUTELY NO IDEA why is that.

      1. Antron Argaiv Silver badge
        Thumb Up

        Re: Trial version of Windows

        ABSOLUTELY NO IDEA why is that.

        Because, at this stage of their corporate life, Microsoft's priority is profits over performance.

        // it's been that way since XP or thereabouts.

        // love the new comments page layout

        1. Adrian Midgley 1

          Re: Trial version of Windows

          Since Q-DOS?

      2. J. Cook Silver badge

        Re: Trial version of Windows

        The way I understand it, you are not supposed to be running a production environment on an evaluation license. However, real life frequently doesn't work like that. :)

        SQL server licensing, however, lets one swap from express to standard to enterprise by re-running setup and performing an edition upgrade, which basically flips some bits around and kicks the database engine to activate the new features.

      3. TonyJ Silver badge

        Re: Trial version of Windows

        Yeah you can do this today.

        DISM is your friend for this.

    2. Sir Runcible Spoon Silver badge
      Happy

      Re: Trial version of Windows

      "so he ducked that cluster-stuff"

      It's ok, you can say cluster-fuck, we won't get upset.

  9. PM from Hell

    Data Center Migrations and application errors

    Some of the commentators on this thread have obviously little or no experience of large scale migration projects. When you are migrating several hundred application services there are always errors present in the log files. The apps are often written in technologies you know little or nothing about so you have to rely on the application admins to be honest and knowledgeable about whether reported errors are significant. I always used to ask if the errors had been present during operation a month ago. I soon got used to the answer 'we never check the log files unless there is an outage' These days admins are often responsible for many and with automated event handling many admins no longer know what 'normal' looks like. I this case I place no blame on the poor chump running the migration but lots on the PHB who let any application admins have leave during this period and on the app admin himself for not investigating it further.

    1. bobajob12

      Re: Data Center Migrations and application errors

      Yes! It seems that a commonly missed factor in migration planning is "what does the system look like now" - right down to inspecting log files for what "normal" really looks like. The gold standard is that a good system "runs clean" - you can explain EVERY entry in the logs, and there's nothing there you can't explain. Got a tiny little assert logged? Run it down. If you can't do that, then you had better know well in advance that these things are "normal" and that you shouldn't freak out after a migration when you see them again. It just cuts down on stress for everyone.

      1. Sir Runcible Spoon Silver badge

        Re: Data Center Migrations and application errors

        I'm no sysadmin, but even I know that when troubleshooting something new and you find something likely-looking in the logs that you need to go back a bit to see if the error/issue was pre-existing.

  10. Baldrickk Silver badge
    Happy

    let's just get this migration done... BTW it's my day off tomorrow

    Our Devops guy finished for Christmas yesterday.

    We persuaded him to put off a server migration to the new year

    1. jake Silver badge

      Re: let's just get this migration done... BTW it's my day off tomorrow

      "Our Devops guy finished for Christmas yesterday."

      And the rest of you are partying ... where?

    2. Trixr

      Re: let's just get this migration done... BTW it's my day off tomorrow

      Our security guys decided we need to close one hole in our cloud accounts, after some of them had been already compromised (after years of us lowly admins whinging about changing non-expiring passwords on generic accounts etc, and other such basic account hygiene).

      Cut to me running a script on 6000+ cloud accounts on FRIDAY AFTERNOON - most of the office, including techs, has already disappeared on leave - to enable MFA. I'm amazed I didn't get any phone calls over the weekend (fingers still crossed).

  11. Snarf Junky

    Not sure who is more dull...

    The person that thought that story was worth telling or the one that decided to publish it.

    Wake me up when it's time to go to the pub.

    1. Alien8n Silver badge

      Re: Not sure who is more dull...

      Or, you know, you could send one in yourself.

    2. gotes

      Re: Not sure who is more dull...

      Well the comments on these stories are usually worth a read, even if the main article itself may be a little uninspiring.

      Your comment excepted, of course.

    3. VanguardG

      Re: Not sure who is more dull...

      "Everyone's stories are dull and boring. Except mine. Share my own stories? Oh no, I'm far too busy for that. I'll just leave condescending comments, because....me.

      1. Sir Runcible Spoon Silver badge
        Coat

        Re: Not sure who is more dull...

        Next time, please use an endquote; my OCD just triggered and I was all calm before.

        1. grod

          Re: Not sure who is more dull...

          Here, take this one and stick it wherever you want - "

          1. Sir Runcible Spoon Silver badge
            Headmaster

            Re: Not sure who is more dull...

            aarrrrgggghhhhh....you fiend!

            1. jake Silver badge
              Pint

              Re: Not sure who is more dull...

              Good fiends are hard to find.

  12. Andrew Moore Silver badge

    And the opposite...

    Just finishing work for the week/year here and I have just been told of a major server upgrade that needs to be done* and if that was "something you can do during your Christmas Holidays"

    * I can guarantee you that manglement have known about this for months but have only dropped it into my lap today.

    1. short a sandwich

      Re: And the opposite...

      That will be a no then. Failure to plan on your part does not constitute an emergency on mine.

      1. Anonymous Coward
        Anonymous Coward

        Re: And the opposite...

        That'd be on overtime rates, plus reimbursement of any costs incurred for cancellation of plans already made. Preferably with time off in lieu to cover the missed holiday, seeing as you're asking me to bail you out at short notice. Too expensive? Oh, then no can do, I'll be on holiday...

      2. Nunyabiznes

        Re: And the opposite...

        @short

        I was laid off once for telling a manager that. I could prove that he held the project just to screw over a couple of us and our scheduled vacation but it didn't matter.

        Ended up getting a better job, so no harm no foul. I wouldn't mind having a little anonymous wall to wall counseling with said manager though.

    2. Alan Brown Silver badge

      Re: And the opposite...

      > if that was "something you can do during your Christmas Holidays"

      Sure, but as I'm on holiday, it will have to be rebooked - at your expense - and because of the short notice I expect penalty rates.

      "* I can guarantee you that manglement have known about this for months but have only dropped it into my lap today."

      When I was tecknishunning it was common for customers to put up with a comms fault all week and then drop it in our lap as an urgent fault last thing on Friday - cue lots of overtime and having to drive to a site amongst weekend traffic.

      1. Anonymous Coward
        Anonymous Coward

        Re: And the opposite...

        I have taken to starting and finishing early on Friday's (without telling people I don't trust).

        That way, when they drop something in my lap at 5pm that would take 2 hours to sort it doesn't 'officially' get seen until Monday morning.

        I say 'officially' because I would actually know at 5.01pm and just spend the rest of the weekend basking in the glow of a bastardly deed well done :)

  13. Danger Mouth

    Project Planning

    I've had holidays booked in months where nothing was planned to go live you can pretty much guarantee the dates will slip, slide and meander to the dates I've booked off. The only way to plan a holiday and guarantee not to be in a go live where I work is to book it for the original planned release date.

    1. A.P. Veening

      Re: Project Planning

      "The only way to plan a holiday and guarantee not to be in a go live where I work is to book it for the original planned release date."

      The very first time you really try that, Murphy kicks in and it will be the first time in company history an original planned release date is made. Just to top that, there will be a minor but crucial error in the released version which only shows up on production.

    2. MonkeyCee Silver badge

      Re: Project Planning

      Even better, I booked my xmas leave in June, got told I was being silly "we're not in that week, it all goes to the Oz desk". Still booked it, still got approved. Suggested my team do it too.

      October rolls around, turns out Oz is shitty about taking our calls (no surprise, they have nfi about our clients) so manglement tried to ask us to stay, then tried to revoke our leave, then tried to enforce our contracts.

      Turns out HR fucked up the contracts (I'd pointed this out to them several times) so when they tried to holds us to their terms, we went work to rule.

      We were an afterhours team. 1800-0200 in the week, 0700 - 1900 weekends. Our contracts said we worked M-F between 0700 and 1800.

      So after accepting they couldn't force us under contract, manglement announced that, bugger rules and promises, we would be covering xmas.

      So half the team (including myself) quit.

      Guess what happened. Yup, now there wasn't enough team to cover xmas, Oz would have to do it.

      But they did lose their supervisor and two most skilled techs. So well played....

      Incidentally, I did work there again, albeit at 3x the rate. New manager, who was quite happy to see me, since he knew bloody well why the previous manger had gotten the boot.

  14. Ima Ballsy
    Devil

    Along the same vein ...

    Time 1991, had just been laid off due to restructuring (ie Retail Company had to find budget money to bring EDS in)

    8 of us let go out of 30 or so.

    The remaining crew were had gotten out of being 'on call" because "we are female and it's a bad part of town" or "We are working on the new systems and we shoudln't have to work on the old programs" ..etc..etc.blah blah blah. Guess who remained on the on-call list- yup the 8 of us that got laid off.

    Anyway, the layoffs occurred at 4PM on a Friday.

    On following Tuesday morning at 2 AM got a call from operations "We got an error on xyz job, you need to come in and fix it"

    Me: no can do, been laid off.

    Them: Well I guess we can call Joe then.

    ME: Nope, he got laid off too.

    Them: What should we do?

    Me: Call the bosses' boss and tell them no one is on call anymore ...

    Looking back I should have done the standard SOP :

    "HEY, restore all the backups from 5PM yesterday and I will be in to fix things", and then

    rolled over and went back asleep and when they called again, apologize - " oh I must have dozed off' and kept the ruse up till the remaining crew

    showed up in the morning - but I didn't.

    Live and learn I guess ....

    1. Anonymous Custard Silver badge
      Trollface

      Re: Along the same vein ...

      Or just offer a quote for your freelance professional support services.

      That being of course roughly 5-10x what they originally paid you to do the role before they canned you...

      1. J. Cook Silver badge

        Re: Along the same vein ...

        Yep. I've pulled that line at least once in my career. it's a dick move for sure, but being let go is kind of one to begin with.

    2. Ozumo

      Re: Along the same vein ...

      Hope you submitted an invoice for answering the call!

  15. Anonymous Coward
    Anonymous Coward

    That reminds me

    Reminds of the time I was working with a colleague to help out with a problem in a software package. Going through what he had done before step by step an error message reading "XYZ failed - click OK to recklessly proceed anyway" popped up. "Ah" said I, "I think we should look into that" as my colleague reached to click "OK". "Oh, don't worry, that's not the problem" said said colleauge with absolute assurance. What is gobsmacking about this is that he knew (that is the reason I was there) that I was the main author of the software!

  16. ma1010 Silver badge
    Facepalm

    It definitely happens

    Shortly after I started work for a medium-sized company as their "IT guy," our web site and email went down. If you guessed it was the company domain name had expired, you're right.

    Although somewhat a lame excuse, I really hadn't been there long enough to start looking into all the infrastructure details[1], and my predecessor had helpfully set all the contact information on the domain to his own company email account, which he deleted before he left, so nobody got any warnings. At least it was an easy fix. I reset the contact info to go to both myself and my boss, the controller.

    [1] It took me several days just to clean up my new work computer. It was good hardware, but my predecessor had a porn collection to rival the PFY's, categorized and all (including "BESTIALITY"). He also had a browser hijacker and other crap on there. I couldn't just wipe and re-install the computer because they didn't have the install CDs for all the software(!) It took me a couple of days just to get that computer sorted so I could start looking into the many other issues.

    1. Tom 7 Silver badge

      Re: It definitely happens

      Never ever use personal email addresses for things like that. Use an administrative name that is really a distribution list and have some paperwork somewhere in big crayon that ensures several people should know about it all whom can be attached to the arse kicker should it fail.

      1. katrinab Silver badge

        Re: It definitely happens

        Never use an email address on the domain to which the registration relates to.

        1. Sir Runcible Spoon Silver badge

          Re: It definitely happens

          and other top tips, such as don't saw the branch you are standing on :)

      2. Donn Bly

        Re: It definitely happens

        Never ever use personal email addresses for things like that. Use an administrative name that is really a distribution list.

        I have come across more and more vendors and services, such as SSL Certificates, that no longer allow that. You have to resort to using a email address that LOOKS like a personal address even though it is a distribution list.

  17. David Gillies

    Some things are forgivable

    But bringing down a mission-critical resource because you've forgotten to renew the license is not one of them (q.v. the recent O2/Ericsson outage; I would hope multiple P45s ensued).

    1. Smoking Man

      Re: Some things are forgivable

      Quite often it's not that somebody forgot to renew a license or certificate, it's just that nobody from management wanted to pay for the new lic/cert.

      But hey, it's still your fault, you didn't send the request/reminder often enough.

      1. Anonymous Coward
        Anonymous Coward

        Re: Some things are forgivable

        There's also, it's not your fault because you're going through a reseller that you only have because the business bypassed IT, you paid said reseller for licence renewals, but said reseller screwed up the licences.

  18. VanguardG

    One has to wonder...did this application owner have some incredibly gigantic payout clause if he were called in during his time off? Maybe he wanted to be called in to both heroically fix the license issue and get a huge payout for the incovenience

  19. N2 Silver badge

    Sounds like Hansa financials

    If it wasnt, it had a very simialr trait, of: 'if in doubt switch to single user mode'

    Yuk.

  20. Nematode

    With the coming of browser checks for https certificates, and warnings when they are not right / expired / chain broke worrying users who don't understand these things, why do people still insist on migrating their sites to https when they have no data traffic worth encrypting? Don't they realise they are creating themselves huge maintenance headaches? Er, that'll be a "no"

  21. Nematode

    Re. being laid off and still being needed to Fix Stuff, the old joke is

    "There was an engineer who had an exceptional gift for fixing all things. After serving his company loyally for over 30 years, he happily retired.

    Some time later the company contacted him regarding a seemingly impossible problem they were having with one of their multi-million dollar machines. They had tried everything and everyone else to get the machine fixed, but to no avail. In desperation, they called on the retired engineer who had solved so many of their problems in the past. The engineer reluctantly took the challenge.

    He spent a day studying the issue. At the end of the day, he marked a small "x" in chalk on a particular component of the machine and proudly stated, "That's where your problem is". The part was replaced and the machine worked perfectly again. The company received a bill for £50,000 from the engineer for his service. They thought this was steep and demanded an itemized accounting of his charges.

    The engineer responded with the following account:

    Chalk: £1

    Knowing where to put it: £49,999

  22. Duffaboy

    Typical User

    No end of times I get this.

    User " An error popped up".

    Me "Ok, What did it say"

    User " I can't remember"

    Me "What were you doing at the time"

    User "I can't Remember"

    Me "When did it last happen"

    User "Months ago" (even though you are there on an unrelated matter.

    or

    User " I have a message here what do I do"

    Me "Ok what is it saying"

    User "Well its just asking me to click OK"

    Me "Anything Else"

    User "No, Just ok"

    Me "Well you don't have much choice really than to just click OK."

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019