My Worst...
I was working on some temp contract work for a company as a friend is the IT director. He asked me in as the IT team had mostly left because they were either clueless, or got upset at doing anything other than playing games over the LAN all day when he took over. They were about halfway through a system refresh that had gone BADLY wrong due to lack of organisation, and had resulted in his predecessor being fired for missing the budget and timescale by over a year! They dealt with remote offices for the most part (3-6 machines in each location, all linking up through a nicely VPN'd Watchguard setup).
One of the jobs he handed me was a hardware refresh on part of the company that was still running NT4 machines (in 2005!). This part of the company was a recent purchase, so hadn't had any "interference" as yet beyond their servers being moved, to the main location, and the VPN's being put in. I rolled up to the first job of the day with a new set of desktop machines, ready imaged (I figure a day spent imaging, and a couple of hours at each job is a Good Idea), and just needed unpacking again, their node names changing, adding to the domain and making sure the ICA client worked ok. This is the point where this part of the company showed why they hadn't been upgraded yet.
"Dead easy this job" is running though my head as I pull up outside. Check the job sheet, 3 machines to slap on the desks, turn on, add-in to the domain and bugger off (no training needed as they had already had that done). No such luck. I ripped the machines off the desks, plugged the new ones in and booted them up. First one I switched on I come back to, go to add it to the domain and find that the domain password isn't working. Odd, it was supposed to be the same as the main domain (Servers had already been upgraded to 2k3 at the other end by this unit's IT support). So I call my mate, and he passes me through to the Service Desk admin, he says all the logins should be as it says on the worksheets they gave me. I gave him the password, he double checked it, but then pointed out that the support is by a 3rd party company for the most part in that business unit, and I would have to check they haven't changed things (terminating a contract would have been more expensive than letting it expire).
He rings me back about 30 minutes later, and the tone of voice on it's own isn't good news. I'm told that the 3rd party company had revoked all the domain admin accounts, and would only allow their approved engineers access to the root accounts. He's not noticed this has happened because they didn't have a need to access anything on the servers or desktops because they were waiting for the upgrade before bringing the systems in-house properly, and that needed to wait until the service contract had expired. Now I like the guy, he's one of the 'old' team, and he's been promoted as he was one of the minority with clue. He's also been warned I have a bad temper when I either don't get paid, or I'm stopped from working. He's also rung the 3rd party, and he's had no joy, and prefers not to "bother" the IT director on a Weekend. I have no such qualms and he knows that, he also knows I was utterly obliterated with the Director the night before, so he'd prefer me to ring him. I do, lets just say he wasn't best pleased at what's happened. "I'll sort this shower of Sh*t out" was one of the nicer things.
At this point I'm actually annoyed. I can ONLY do this job on a Sunday as they worked 6 days, and I can ONLY get in to the offices because the unit managers have kindly agreed to turn up and let me in. The poor woman was looking like she wanted to be elsewhere so I explained the problem to her. She responded with a comment along the lines of "Yeah, we've had this type of thing last time". Turned out that the 3rd party company had given them trouble before, even things like password resets had an SLA of 7 days. She'd been with the company for years, and they had never enjoyed ringing the IT desk because they got utterly fobbed off every single time. While she's telling me this my brain is getting The Rage, I'm wondering what type of company would work like this, and my mobile fires off. JOY! It's someone from the 3rd party company, turns out it's the Helpdesk manager. No they won't give me the passwords, no they won't add the machines to the domain unless it's an image they have created, no they won't send an engineer out on a Sunday (ever), and they won't do it without charging for each machine to be added to the domain. I'm protesting, and he put the phone down on me. I lost it, totally.
Back on to the mate, and he's pissed now as well. He's trying to enjoy his Sunday lunch at this point (by this time I should have been on the 2nd or 3rd place). "Stay there", is all I get when I explain to the problems, and he hangs up on me. Bit later phone goes again. "Head Office, now" is all I get from him, and the phone goes down again. I head up the office which is a good hour away, and see my friend's car I the carpark, NOT a good sign as he _Doesn't_Work_Weekends_, beep myself in and go looking for him. Not in the office, hmm, head down to the IT desk, and the 2 guys there are looking sheepish. Before I speak they point me at the server room door. I go in and the IT Director, and Helpdesk manager are already pulling cables, neither look happy. I'm sat down at a desk with the server images, and told to extract the essential data, and reinstall. Ended up doing the desktops for the entire week after that.
Turned out the refusal to allow access to their own systems was reason to terminate the contract, after the 3rd party guy had spoken to me he'd ranted at his own customer he'd told them it wasn't allowed, but my friend had noticed while madly reading the contract there was a term about always allowing access. The guy refused, and when told he'd need to supply all passwords or they had terminated the contract he'd responded with another term in the contact that said "within 28 days, and you can F**k Off if you think it'll be any sooner". Now my friend is pretty clued up(did the proper PFY route out of school), and figured he could slam the migration through fast in a couple of days if he had the manpower, and as they had already got all the hardware it wouldn't be a major issue to slap a bit of overtime out to folks so it could be done.
I did have the misfortune a couple of years later of doing some contracting for the 3rd party I had to deal with that day. Lets just say "disorganised", being sent out to the same job 2 or 3 times because they hadn't packed up the van with the right kit, or hadn't given me the right info, or insisted on sending an engineer out as well as me doing a desktop refresh. I was thankfully paid by the hour, and they were supplying the transport, but a little demon at the back of my mind was screaming at me to warn these poor suckers what type of service they could expect!
Anon as I'm not mentioning names (I've promised not to), and my friend just filled in a couple of bits I missed out!