IT support, as we know, is that job function in the technical ecosystem that takes the flack for any problem affecting a user. These can range from the straightforward if annoying forgotten password requests and slightly cryptic ‘my-laptop-isn't-working-anymore’ complaints, through to the more serious ‘accidentally deleted’ …
A story against myself
I'm not an IT support person, but my knowledge of the subject is a bit above average and I can solve most of my own problems if I have the access rights to do so.
At my desk I run my laptop with an LCD as the primary display and the laptop screen as secondary - so far so good!
On one occassion I was setting up to deliver some in-house training using my laptop and a datashow - datashow still turned off at this point. All my laptop would do when booting was stop at a blue screen - it would boot properly in safe mode, but in normal mode, no go. After several attempts I gave up and we did the training the old fashioned way, using the handouts and a whiteboard - all more or less happy.
Back at my desk I plugged the laptop back in to everything and gave it one more go at booting before logging a support call - important note to self - even if a projector is turned off, the laptop may still see it as a connected primary display and display all the useful stuff such as login dialogues on it.
Anonymous because some of my colleagues read El Reg, and I need to maintain at least a minimal level of credibility.
I worked at a prominent university in my first job and used to get many scatty professors caught doing silly things.
One day a female professor came in with a keyboard saying she thinks it's broken.
"Why?" I asked.
"Well I vomited on it this morning and after that it didn't seem to work very well, so I though I'd clean it with cooking oil..."
And sure enough it appeared to be covered in bits of puke and was very greasy. I'm not sure I'd like to see the inside of her house...
Question of the week
I used to work in Support for a software publisher.
Best call I ever got was from an excellent customer who never called with the usual dumb mistakes. He had a server farm. The servers were named after characters from "Rocky and Bullwinkle", e.g. Boris, Natasha, Dudley, Nell, etc. He was installing a new server and could think of only one character whose name he hadn't used yet.
Thus, the only reason for his call was to ask, "What was the name of Dudley Doright's horse?" I told him right away and he sent in an excellent customer satisfaction survey.
As a User, I Admit When I'm Wrong.
If I open a ticket, then find the solution; whether it is a workaround/fix, or PEBCAK, I make a point of letting the folks on the other end know, especially if it was a goof on my part.
After all, it's the polite thing to do.
It has never been received badly (well, except for one guy at the Adobe Photoshop Beta Desk, who just blacklisted me because I was...strident; but I don't really blame him).
Once upon a time there were (may even still be) a number of Advanced Gas Cooled reactor power stations run on somewhat less than advanced Ferranti Argus computers (think genuine ferrite "core" storage (4k x 24bit words) and a master console - aka The Wurlitzer - with a large number of toggle switches and winken-blinken lights.
Well, the disk storage was well passed its best - 1.92MB on a 1m diameter platter and enough angular momentum to make anyone dizzy - so we replaced it with error correcting solid state memory. We were quite good at doing such things and VERY VERY CAREFUL ("Large cloud of radioactive fallout now passing over X due to cockup by Messrs X, Y, Z" is not the sort of headline you want to wake up to...)
Anyway, we tested it thoroughly (which included shift work enshrouded in bin bags staring at a high speed 'scope to track down the last 2ns glitch) and it was perfect.
Until it got to site - at which point the "ultra reliable Argus systems", which hitherto had only crashed about once every two weeks (always safely switching to the backup I hasten to add) began to crash every 24 hours. And when everything fails more often the likelihood of two things going wrong at once increases accordingly. Nightmares of "core dumps" of CO2 from emergency reactor shut-downs all round.
Fortunately, we had a good relationship with the (as was) CEGB: they called and said "Your fault", we said "Tisn't!", they said "We know... but you have to prove it" - but it took us weeks and lots of really high tech (for those days) equipment.
Turned out that all we had actually done was amplify an existing problem - an illegal bus state (caused by an illegal instruction in the original safety-approved code) that our scrupulously programmed system refused to accept, but which the old Burrough's disks were perfectly happy to just "ignore". The bug in the source code was then fixed.
After which, everything worked absolutely fine...
But - have you ever had to jump start a computer? Really - the old Argus had a 12V lead-acid accumulator in the CPU rack, and if it was dead it wouldn't boot....
Is it plugged in? Did you check the battery...?
AC... just in case
I got this:
'Can you press ctrl-alt-del for me please?'
'No? Why not?'
'I've only got one hand.'
In my PFY days I was a tech for Earthlink (an ISP). One day about a week into my time with live customers I get a call in the windows cue from a customer not able to connect to the internet. In total the call lasted just over an hour, the bulk of it due to said customer not understanding the concept of a right mouse click. After beating my head against the desktop repeatedly I could hear typing on the other end of the phone. I asked clueless (l)user what he was doing he said he was "right clicking on the desktop but it's still not doing anything". I then inquired "and how are you doing this". His response has given me a laugh to this day "well I have the mouse thingy on my desktop and I'm typing click like you told me to do". Yes folks he was indeed attempting to write the word "click" on his desktop instead of using the right mouse button. And yes I did indeed have to then spend a full five minutes explaining to him the concept of left and right mouse buttons /face palm.
Not but two years ago I was contracting with a major southern california city. I was out in the field doing migrations when I get a call from the IT manager of the agency who's office I was working for that day. Said manager had been out to the office I was at a few days earlier attempting to hook up a multi function printer for one of the agency big wigs. Seems he was having no luck at all getting the printer to show up on the network despite being out there four hours. I looked at the printer and proceed to laugh uncontrollably. The IT manager when he set up the printer used the telephone cable (still plugged into a clearly marked phone port in the wall) into the Ethernet jack of the printer. Fortunately he had a good sense of humor about it otherwise my subsequent call giving him a hard time about it might not have ended so well. Just goes to show no matter how experienced we are, we all have out clueless user moments.
Pair cos.... well should be obvious shouldn't it :-).
Pick up on some of the "true stories" above lol
I had the director for commnications call (note i said DIRECTOR OF COMMNICATIONS) about the fact that she had not received email all day, I connected to her machine (very painful over a 4k vpn link to china) waited for her screen to appear and promptly clicked on the received colom header. (to change the sorting to date not sender)
after all that she did not even say thank you
Funny modem tricks
Couple of funny modem tricks:
1) Had a customer (a small deli) that kept killing modems. Went to replace their (internal) modem, and found that their PC was on a small desk right next to one of their walk in refrigerators. Didn't think anything of it until I accidentally brushed my hand against the fridge. Seems that they had a bit of a grounding issue.
2) Had a customer of a small dial up ISP constantly complaining about connection speeds. This was the kind of guy who would take out a stopwatch and bitch if it took an extra 5 seconds to complete a connection. After weeks of trying to determine why his 56K connection suddenly dropped to about 26K, he eventually wondered aloud if it had anything to do with accidentally catching his phone line with his "weed whacker".
Printer not working, how to promote project...
As a favour, years ago, I labelled some old machines in an office with their status (which had hard drives potentially containing sensitive data, etc). A few days later, an irate phone call revealed the label printer hadn't worked since I used it. Needless to say, once I finished using it, I shut down the PC and the printer - apparently for the first time since it had been installed, since the regular user had no idea it needed to be switched on to function.
Last year, working on organising a conference, some of my colleagues had an idea of how to promote it. Since they had plenty of time on their hands (I'd already automated all the paperwork except for actually depositing the cheques in the bank!) they spent a week or two browsing the web, gathering email addresses to send the conference information to. Sadly, they went ahead with this plan - and worse still, actually got away with it, using a throwaway Gmail account. I had really hoped Google's own abuse precautions would cut them off at the knees, but no...
Regular Client phones me up - "Having problems with wireless". He then proceeds to tell me that he is at a customer site, in a room set aside for use of wireless. "It worked yesterday."
"Where is the wireless access point?", I ask him.
"On the other side of the room", replies my client.
"Is there a fat bloke sitting in front of it?", I enquire.
"Yes, there is. Why?", as the client tries to stiffle his laughter.
"Move around the room until you can see the aerial", I tell my client.
I then here more laughter from the client... as sure enough wireless starts working again when it doesn't have to pass through that fat bloke.
And another thing
During the course of a long career some things stick in the mind, and occasionally elsewhere.
"Hello Robbie, my program's not working." ... "The screen's on, but I can't read it." ... "What big box is that?". Someone had liberated the under-desk system unit .
"Hello Robbie, my program's not working." ... "Yes, I've put the floppy disk in" ... "I've tried several copies" ... "The last one was a bit tight". Six floppies in one disk drive.
"Hello Robbie, my printer's not working." ... "It was OK just a minute ago" ... "BANG" ... 3 inch paper clip embedded in ceiling tile just missing left eyeball. Don't mess with daisywheel printers.
A User Writes ...
I work for a Large UK University. We used to have a great academic help desk: two people who knew almost everything, knew who to ask about everyone else and who could also establish quickly - and remember - the technical competence of users, and tailor heir advice to suit.
Then everything changed and we academics got the same help desk as the management types. Shortly afterwards, I had to call with a question. This was the question:
"I want to print something from the VAX (dates me, eh) on the academics' printer in building XXX. Can you remind me of the queue name, please?"
And this was the first answer, and indeed the only answer
"Have you tried rebooting Windows?"
boot on the other foot
I am our office IT person, but since it is a tiny little office of a dozen people I also have a 'real' job, the IT is simply because I am the least bad.
I have to deal with all sorts of 'it doesn't work properly' type rubbish, most of which disappeared when we started buying decent hardware, and windows XP (we actually ran an office for about 4 years on Windows ME - the only operating system ever named after a debilitating disease.
However the finest piece of stupidity is actually when we bought a brand new, 3k Dell Poweredge 2700 - the finest machine we had ever bought (more power in a cheap laptop now, but it's still running as a file server.). We bought it sans OS, as the only operating systems it supported were Win2K or win2003 (I think), and we were quite happy with a peer to peer winxp network. So retail copy of XP, and big hulking machine.
I got the raid drivers installed ok, but after 40 minutes of wondering were the hell the CDrom was (including seeing if it was inside for security reasons), I gave in and phoned dell support - who took 15 seconds to point out where it was.
I'd never seen a half height drive before.
I'm retired now, but some of my colleagues' spreadsheets brought tears to my eyes. My fingers itched to rewrite them properly.
Only one initially impressed me as the minus figures showed in red. "****" me" I thought. "She's used conditional formatting." Took a closer look, and found she hadn't. She'd gone through the spreadsheet and manually formatted the cells with negative numbers to display in red.
My report did't print!!!
My personal experience was Gabbie, an expeditor for a home building company here in the states.
We had added a custom screen to the sales order screens for adding miscellaneous items to go out to a job site (extra lumber, brackets, etc.). At least three times a week she would call saying that her miscellanious item report didn't print. That had never happened even during development and testing. I came to find out that when she had printed the report was during the printing of work orders to go out to the shop floor. They got sent to the printer as individual print jobs, all 150 - 200 of them. Her report was in the middle of the stack. The days she didn't call they either had no extra items to send or she ran it before or after the work order print.
She went on maternity leave for two months. Two months of no calls. The person subbing for her would go look through the work orders. The day Gabbie came back I got the call "My report didn't print".
Static and Floppies
Back in the late 90s I provided IT support to a manufacturing company which had a "main frame". The head of IT was proud and defensive of her AS-400 and never referred to it as anything other than THE MAIN FRAME. My job was to introduce PCs and PC servers into the facility to bring the company into "main-stream".
I provided her some Windows PC set-up info on a floppy disk and watched as she walked away, slowly sliding the diskette into the curved back pocket of her nicely filled denims. With a sigh, I walked away to continue setting up the new company Windows server.
10 minutes later she is in the server room telling me that the floppy disk was full of garbage data. The BOFH horns came out and I stated that it was obvious she was wearing silk underwear. She blushed beet red and stammered" How could you know that?"
I pointed out that I had seen her put the disk in her back pocket and that cotton jeans against silk panties would result in a high static charge as the two materials rubbed against each other when she walked. This static charge would destroy the disk data.
I took the disk, reformatted it and reloaded the data file. I handed it back and reminded her that this was a PC (windows) disk and to use her new desktop PC not the AS400 terminal loader. "Oh, and do not put it in a back pocket or a shirt pocket if you are wearing a silk teddie." She turned beet red again and exited quickly holding the disk out-stretched between thumb and fore-finger.
(I knew all along that she had garbaged the disk on the AS-400, but what the Hell!!)
10 Minutes later an officious email from the Director of IT was informing all facility employees that carrying unprotected PC disks while wearing silk and cotton was prohibited!
Some days its like clubbing baby seals!
Over the years I've received many interesting support calls. A couple that still keep my chucking even now include the instance one of the office administrators was given a new mouse. All seemed fine until we got a call complaining that every time she adjusted the dial on the bottom of the mouse to make it faster, the ball would fall out.
More recently had a remote user freaked out that her laptop had been hacked by someone. As she was typing out a new word document she had managed to turn on the microphone and voice recognition software. Although not very accurate, the microphone was picking up all conversations in the office, including the call to me and entering it into her document. I'm not sure she ever accepted our solution and remained very suspicious of her laptop.
Despite 8 years on a Helldesk the funniest support call I had was as a user.
I was on a 2 year stint as an analyst for IT trends in the company (by that stage I was burned out with support so needed soemthing different). And was using an NT desktop with the BSOD screensaver (which simulates the screen everyone sees in NT eventually quite well).
Anyway..went to lunch one day, came back and found one of the junior local support techs about to inset a build disk into my PC. He'd noticed the BSOD and thought to kick off a rebuild while I wans't around (which I thought was pretty good of him) - watchign his face when the "BSOD" cleared after I pressed ENTER was priceless!
1) I was doing support for a timeshare company with resorts in the Canaries and mainland Spain. While I was in the resort in the Canaries once I got a call from the mainland saying none of the workstations could see the server on the network. This was an old 10base2 network. I asked if any computers had been moved and was told "No", and then I got the head receptionist to go round checking all the T-connectors and terminators like I had taught him to do, and when he said he had done so, there was still no server.
I logged in remotely from where I was and couldn't do anything from the remote access workstation, nor could our hardware support guys back in UK. After 2 days of trying to sort out the problem, it was decided I should fly to the mainland to sort the problem, they had guests checking in on the Saturday and Sunday, I was flying on the Thursday so they were even getting ready to manually check people in using the book they had had before the computerised booking system.
I arrived on the Thursday evening and promptly went round checking the network cable, I managed to get the network working again by terminating it outside the locked accounts office, which was where I decided the problem was. Early Friday morning I went into the accounts office which was run by a couple of sisters. The younger one was in, and I found the source of the problem, the older sister's computer was missing and the cable was lying there with no T-connector or terminator. When I asked the younger sister where the computer was, I was told the older one had taken it home so she could do some work from home as she was 5 months pregnant at the time. So much for no computers being moved and the cables being checked.
2) Another time, the same company had sales offices in Eastern Europe which we ran on laptops. I got a call one day from the office manager in one office saying her laptop wobbled and wouldn't sit flat on the desk. I thought maybe one of the rubber feet had come unstuck from the bottom, so I got her to turn the machine over and check. No, all the feet were there and in the right places.
She told me she thought it was the 'big round thing' sticking out of the bottom of the machine. This had me very confused as I had an identical machine and mine sat fine. So I got her to describe what she could see, and we finally decided where I had a flat panel with slots covering the processor fan, she had a big round fan sticking out of the base of the machine. I asked where that had come from, and it transpired the processor fan had been playing up so one of the local staff had a look at it, he couldn't get a low profile fan to replace the faulty one, so he had gone out and bought the smallest fan he could find and then cut a hole in the base of the computer so he could fit it. Not only did his modification not work properly because now the machine was sitting on the fan itself and so even less air was getting through, but only a week before the phone call, the company had paid to extend the warranty on this particular machine by 2 years.
3) Back to our 2 sisters in the accounts office in mainland Spain. I got a call from the younger one saying that when her computer was switched on there was a buzzing on the phoneline, and if she tried to print from Word across the network to the deskjet attached to her sister's machine, the printer physically switched itself off. Both of these had me really stumped but I was due out to the resort a couple of weeks later so I left them until I got there.
When I arrived, all was as she said, the buzzing on the phone and the printer physically switching itself off. Knowing what the electricity was like in Spain I thought maybe we had an earthing problem, so I got maintenance to bring up their little plug which lit up in different ways depending on what the fault was in the sockets. The maintenance guy plugged the plug into the power socket and it lit up like a christmas tree, we both looked at the plug, then at each other and then backed out of the office very fast. It transpired the local electric company had been doing some work recently and somehow had managed to wire across two phases. We had an entire block containing 16 apartments, the offices and reception all putting out 415V through the normally 220V sockets, and they had been like that for nearly 4 weeks.
I had a client with an error message on their screen. Standard protocol they read me the message, I tell them to say OK to the error. They say OK, then I ask them what they see and they read the error again. Confused as this is usually a one shot error, I again tell them to say OK, and tell me what they have on the screen. When they again state the same error I fired up the remote control software to take a look.
I see the standard error on the screen with the bright candy like OK button and tell them to say ok. They again, say OK, but I notice their mouse cursor never moved.
I then stated slowly, "Click OK"
remote support is a nightmare
i am a developer and so mostly support testing systems rather than problems 'out in the wild' but occasionally i get called in when people are at a loss. where i work, the helpdesk is purely there to weed out the timewasters, any real problems go on to a proper support team (they can check code, release small updates, etc, although still 90% of what they get are timewasters that slipped through), but when they get stumped, i'm called on.
remote support is near impossible, due to the users propensity for lying* and the users general retardedness when it comes to computers. (just mail me the file on your c: drive in the logs folder called xxx.log is akin to asking them to perform brain surgery with a meat cleaver after a few pints of vodka) not forgetting the works of genius, who report problems 2-3 weeks after they had them, without mentioning it.
generally, my experience of remote troubleshooting, is 80-90% getting the user to admit what they really did before the problem occurred, and the rest is firing up a testing system and recreating it myself locally to solve.
*although i do enjoy pulling transaction logs and responding to the reports with exactly what they really did, pretty much keystroke for keystroke**, and why they shouldn't have been doing it, usually some attempt to bypass restrictions they know are placed on the system.
**not a keylogger, but a financial system with very good auditing capabilities, as people trying to bypass restrictions generally means some sort of attempt at fraud whether they realise it or not :)
Just a little typo I'm sure
Figuratively, I read a zillion electronic help requests every week. Some are so grammatically tortured that they are incomprehensible. On rare occasions they are so well written any reasonably intelligent person could understand the problem as well as what actions had already been taken, whether they were an IT guy or not. And, of course, there are some which make no sense at all. For instance, my all-time fav arrived just a few months ago. The entire problem description, which had to do with a dev tool, was only two sentences long. The second sentence read, quote: "It is changing "<" to "<" and ">" to ">"."
Ultimate for me.....
Getting a call from an irate business user who couldn't dial in to anything from his workstation. Everyone else could. LAN was up, no problem. Ok thinks I, "Humour me, could you plug a telephone into the socket your using to dial out please ? I think it might be dead"....... I got called every idiot in christendom and in the end sent a field engineer round. He did the same diags I did, then called me....."Humour me, could you plug a telephone into the socket your using to dial out please ? I think it might be dead"....... Now this guy (lovely chap, good field engineer, but never afraid to admit to himself if he went out of his depth), did just that. Guess what ? One patch lead later everything is fine. Oh yes, everything was on the support log too, including my initial stab at the eventual solution. I suppose I could have said something about patchbays, but given the customer........ Best not.
When I was working on the service desk at one of the local banks one of the desktop engineers came down to show us a laptop wrapped in a plastic bag that was dripping muddy water. The reason?
The bank had issued all of it's Rural Lending Managers with new laptops and cell-phones (this was back in the late 90's) with the idea being that they could do the full applications etc while at their customers farms. They did not provide these guys (mostly ex-farmers/farm managers themselves) with any training on using laptops.
The manager whose laptop it had been had been unable to work out how to open the laptop and turn it on - he was a bit of a technophobe. He kept it in his car and filled out the old paper forms, so was able to work that way. One day, while driving back down a fairly rough metalled road his car got stuck in a large pothole. His solution? Wedge the laptop behind the wheel of the car to get more traction!
When he was called to explain what had happened he said "It was the only use I got from the f*cking thing, please don't send a replacement"
At a company recently aquired by HP...
I was subcontracted to provide onsite support after L1 offsite (different country, different First Language) couldn't resolve the problem...
1) Received server for rack mounting and final install from city "A", the project manager was in city "B". The team that built the server had told the PM the server had a DRAC card but upon receipt, no card in evidence. Empty slot. It's not an IT problem, it's a physical lack-of-existence problem. By a server build team.
2) User having weird problems. Sent to see him by L1, turns out L1 had asked him to change the attributes on his entire profile to +R +H. WHY??? (PS, try it, it's hilarious!)
3) Different job, ask user for their username, the reply was "smellybum99" (Seriously!) Um, no, not your password, your username please.
new year's day call
I work for a local newspaper and on my first year in the job, received a call on January 1st.
At that point I was used to late night calls, weekend calls, etc.., since this a newspaper, the bulk of the work is done by night, every day.
But some of the older journalist arrived at work very early. Even on January 1st.
I waked at about 8 AM with the soothing ring of the phone.
I answered groggily with something that may have sounded like "what the f*** do you want at this hour!" (remember, it was Jan 1st)
It was the senior editor of the international section, an elderly and respected guy (oops).
He was having some problems with the cable reception. I don't mean cable TV, but old cable news reception.
I asked him to reset the cable reception PC, something that he knew how to do, because the dam thing stalled frequently.
And then he told me, in a very calm voice: "yes, but there is a lot of smoke coming out of the room".
This woke me up in one second.
I dressed as fast as I could and drive there.
When I arrived, the fire was already extinguished, but it had burned a whole cable rack, used to interconnect the reception PCs with the newspaper network.
I removed everything I could from that office, in the hope that the damage was not so severe.
Well... one of the machines digested a LOT of smoke and was dead.
The others were coughing, but recoverable.
After a couple of hours of trying to reach the boss, finally answered his phone:
- Boss, I believe you should come here
- (some words muttered)
- Well... you see... a fire in the cable reception room...
Minutes later he was there.
In that time, some of the maintenance guys have (supposedly) replaced several of the CAT-5 cables, some of the cables connecting the cable receptors with the PCs, etc.
Tried to get everything to run...
And it worked!
We just could not believe our luck.
Then I turned to check the cables: they were not replaced, just welded together and in two cases, twisted together and isolated with scotch tape! (I swear to God I am not making this up!)
But the things were working, so no one would accept anything else.
Later, we discovered the cause of the fire: one of the maintenance guys used the room to take naps (apparently). So he rigged an electric fan to cool the room to his preferred temperature. For some strange reason, he removed a thermostat from the fans electric motor.Then, he left the fan working since Dec. 30.
Predictably, the thing kept working until it started to burn, fell from the fan (burning), and damaged everything surrounding it.
That was some hangover!
I worked third line helldesk for a large distributor. Typically we supported consultants, resellers and solutions providers -- people who should have at least some degree of IT literacy.
A consultant for a large provider logged a call that his Netware 3.12 and 4.11 servers connected through NIAS (Novell's remote access solution of the day) running over ISDN.
This guy was one of those who was very quick to get loudly offended if I asked him anything basic so I spent weeks trying to fix his problem and got nowhere.
Finally I risked his ire and started from basics. About the fourth question went along these lines:
Me: Can you confirm the protocol settings you are using on both ends please?
Him: I have already been through that, they're fine. It is all set up correctly. I have been doing this for years.
Me: I appreciate that, but please humour me.
Him: *mutter, mumble, waste of time, what a stupid question*
Me: I am sorry, it must be a bad line, I didn't catch that
Him (eventually): I am using TCP/IP address 10.10.10.10 and IPX address 123456
Me: OK, what IP address are you using at the other side?
Me: You have only given me one IP address, IPX will work with just the network defined at both ends, but IP needs unique addresses, so I need to know both IP addresses.
Him: You are not paying attention, listen carefully: the netware 3 server is using IPX and the Netware 4 server is using TCP/IP.
Me: <long pause>: the 3 server is just using IPX?
Him: *in a "are you fucking stupid" tone of voice: yes
Me: no IP on it?
Him: *same tone of voice*: No.
Me and the 4 server is just IP through NIAS, with no IPX?
I did talk him through getting it working but he still refused to believe that IPX couldn't talk directly to IP, even when adding IPX to the 4 server's NIAS interface fixed the problem.
Disk Space Issues
I was called into the customer services office one afternoon to be told to get in my car and head to a particular customer site as there was a problem with their UNIX system. As there was no-one technical on-site I would be able to find the fault and relay details back over the phone. This was all before the internet etc.
I arrived to find the server completely dead. and the console complaining about the OS They had rebooted it that morning in-line with their normal operating procedure to clear hung processes.
Whilst discussing this with the techies in the office a vague conversation about the costs of a new disk had been mentioned a couple of months ago as they were short on space. They had not purchased one due to cost.
Upon further conversation with the customer it appeared they solved the problem by deleting some big files they had found. It turned out that they had deleted the UNIX kernel and settings. As they had re-booted the kernel and settings were no longer in memory and thus when they had tried to re-boot the system could not find the OS.
Took a lot of work and an engineer to recover the environment as the configuration was not on any of their tapes!
Ghosts in the systems
Some of our secretaries called saying that their nice new systems were occasionally bleeping and typing in random characters. The users claimed we had IT poltergheists. After checking for an epidemic of stuck keys and viruses, and discussing the price of exorcism we found a simpler cause.
All the new systems came with barcode scanners and the stands for these were almost always unused. The scanners were all trying to read barcodes off the woodgrain . The bleeps were when they actually succeeded which they then inserted into whatever programme was running.
Perhaps others have come accross the problem of mouse pointers suddenly shooting accross the screen? After again ruling out ghosts we removed the mouse mats that had large white areas.
Does being the Help Desk for most of your family count?
One day my aunt calls. A brand new computer. I just under 3 hours, she has found a place for every cable. Everything except the speakers is plugged in. She can't find the right place for the cable from the speakers. I tell her to look at the back of the computer, for a row of small holes, with little symbols for mic, line in and so on. No holes she tells me. No small holes what so ever, she assures me.
I then ask what brand the computer is. Fujitsu. I remember my Moms computer is the same brand, and even if its not the same exact model, i gave her a call. It was her sister after all.
I explained why I called (and gave her a bit of a fuss, for having giving her sister my telephone number).
"Mom, I need you to go to your computer, and find the wire from your speakers, and tell me where they are connected to the computer."
Moms puts down the phone, and goes to the computer. Returns a few minutes later, and says "it goes to the power strip".
"Mom, there should be another, thinner, wire going from the speaker. Could you tell me where that ends?". Another few minutes, and I get the answer "it goes to the other speaker".
"OK Mom, there should be a THIRD and last wire going from the speaker. Could you please...."
This time it take a little longer, before my Mom returns to the phone, and answers "it goes to the phone outlet on the wall".
What? Deep patient breaths in my end of the phone.
"Mom, that makes no sense what so ever. Why would the speakers be connected to the phone line?".
Her logic answer: "Well I don't know, but I can hear when the computer tries to connect to the internet".
I explain to her, that those are sounds coming from her modem, not her speakers.
But knowing my Mom, I had a bright moment: "Mom, have you bundled all the wires in some sort of way?".
Turns out, she had bought some sort of cable organizer "to make cleaning easier" and just pulled the wire from the speaker going into the cable organizer and decided that the wire that moved "the most" in the other end, was the other end of the speaker wire, and that just happened to be the phone line, ergo....
When I got her to look a last time, all she could tell me was that the speakers 'bright green plug' was going into the back of the computer.
I called my aunt back and asked her, if the plug on the speaker cable was bright green. Yes it was. Could she find ANYTHING on the back of the computer with the same colour?
Yes, there was in fact a series of small 'knobs' in different colours on the back of the computer. And yes, now that I asked, the "knobs" DID have small holes in them, precisely the size of the metal part of the plug from the speaker cable.
Problem solved. And it only took forever.
A heart, 'cause I love my Mom none the less.
Turning the tables for a change...
I'm a developer who's also worked as a TME in a previous life, so I can sympathise with IT support. One time, working for a large corporate (who shall remain nameless), the intranet went down. I checked with my local colleagues - yep, definitely not my PC, everyone is affected. I checked with a colleague in another country - same thing her end. So the intranet world-wide for a major company had apparently gone down. I expected alarm bells would be ringing somewhere and that IT support staff would likely be panicing and chasing through patch-panel cables etc., so I left it 30 minutes. But half an hour later, still no intranet. Right, time to phone IT support....
"Ah, hello sir", said an Indian voice (starting to get a sinking feeling in my chest at this point already).
"Yes, I'd like to report that the intranet is down.", I replied.
"So, which office application are you having trouble with?", continued the Indian voice.
"Uh... I'm not. I'm calling to report that the intranet is down, world-wide!", I said in a direct and clear voice.
"So, are you saying that you are having problems with Word?", he continued.
"No, the intranet is down! This is quite important. It's affecting staff world-wide.", I repeated, now really starting to loose hope.
"Ok, thank you sir.", he replied followed by a pause.
Brilliant I thought, it's finally got through to him and he understands.
"So what exactly is the problem you are having with Word?"
... so let's not be too damning about the kernel load known as "users".
This is definitely true because it's on your screen
I once took a call from a user who was having terrible trouble getting on the wireless network with his MacBook. "Wait there", I told him, and went along to see what he was doing. When I arrived he asked, "Are you here to fix the network?" I took his MacBook, smashed it over his head and kicked him down the stairs. Hilariously, the slightly dented MacBook then connected to the network. I laughed and laughed, then went back to the comms room to close the ticket and call an ambulance. You wouldn't believe what happened next. A squirrel had somehow gotten into the comms room and eaten through a three phase power line, taking the entire campus offline for the day - including the ticketing system! So although I resolved the MacBook problem quickly and efficiently, I still ended up out of SLA. Talk about awkward users!
My faux pas...
Ok, here's a little story about when I messed up... but really, what I did should not have been possible! I was working in a development lab for a major worldwide telecom player. This lab was an engineering lab and was supposed to be isolated from the corporate network. I was working on a CompactPCI board which was known by it's three letter acronym name, which began with the letter C. Well, I punched in the details in the boot-loader part / initial flash setup bit... very similar to RedBoot and all the others. You know the stuff, Board Name, Board IP address, IP address of DNS server, TFTP protocol, etc. etc. Well, try as I may, I couldn't get the darn thing to connect. 15 minutes past. Then three scary looking IT support staff stormed into the lab. One screamed at me "Turn that board off!" I didn't know what was wrong and started to try to ask why. But he cut me off and screamed "Now! Turn it off now!". I obeyed, because it was obvious he was extremely unhappy. What had happened, is that I had in a moment of brain fade, reversed the fields for the board's IP address and DNS server IP address. I think you can see where this is leading... yes, because C (the first letter of the name of my board) comes before D in the alphabet (which is the first letter of DNS), my board had suddenly become the intranet DNS server for the entire organisation world-wide! Ho hum.... well, at least they helped me get connection to my target board.
I have to chuckle thinking back to that day. It should so not have been possible! I was in a lab that was supposed to be isolated!
Later that year, we had no email world-wide for 3 days due to a server replication problem ... rumour has it that our CEO at the time phoned up Bill Gates and after their conversation, Microsoft put a team on it which finally got the problem sorted.
misc restaurant support stories...
I used to support POS systems and telecom for a major restaurant chain---I personally supported around 70 restaurants---I drove all over hell and back and was on call till 1am most nights...
My favorite: One of the units complained that the main computer would no longer communicate with any of the POS terminals. Upon opening up the case, I found a family of mice had been living inside and had peed and defecated on the motherboard until finally this nastiness ate through a trace on the circuit board going to the PCI riser card in the machine. YUCK! The machine had perhaps a half pound of mouse shit in it! The machine was still happily running though--those old NCR servers were tough.
I remember spending about 8 hours rebuilding one of these in a far away unit, so tired I was almost seeing double. I finally had it running and it was propped on a chair with the cover off as I tested it. I'd just powered it off to put it back together when a random employee wandered by and spilled perhaps a gallon of soapy water all over the innards. I was too frazzled to do anything but laugh. The tone of my laughter scared them a little I think...
Also made a frenzied dash to a unit to discover the system wouldn't power up because they'd plugged the UPS into itself. A perpetual motion machine? Not quite.
Had a lightning strike at a strip mall that one was located in... I parked next to the rest of the contractor's vans, as every store in the mall had something fried. The CPU, modem, credit card terminal, phone system, and electronic safe all were DOA. Was able to get all running but the safe :)
Favorite quote: Employee looking at the display of a credit card terminal "Why does it say it's dilating?" (mental image of the machine in stirrups, ready to give birth)
Wrong kind of mouse...
Working support in a media monitoring company in Sydney, we got a call from a user in the Canberra office. "We've noticed that we have mice in the building. Can you help?".
Er, no, we're IT, not Rentakill.
"But <forgotten_name_of_manager> said to call IT"
Still can't help. Call a pest controller.
Turns out they'd told the manager that there were problems with mice, she heard it as computer mice, so said call IT.
A friend pointed out that it was lucky we didn't receive any calls about Komodo Dragons...
Got a call out a to a warehouse, where there was a report of a loud buzzing coming from the network cupboard, which was pretty much inacessable after the warehouse's layout was changed to accomodate taller racking. Not being able to see into the back of the cupboard i assumed the buzzing was a dieing fan in the switch or router... I unrack the router all apears fine, try to re rack it wouldnt go in easily, push a bit harder figuring the rail was bent next thing i know i know im sliding down the ladder trying to escape the swarm of pissed off wasps that had seen fit to build a nest in the cupboard, very glad im not alergic to the buggers...
stories from the trenches
A developer now, but once I was part of a server support team. What memories!
1) Replacing a server's hard drive at a small rural site. All the staff are out and about but the site was unlocked (hopefully because they were expecting me, not because they were careless). Do the replacement without trouble. Need to call HQ so someone can keep an eye on the disk rebuild. None of the land lines work. Mobile phone has no signal. In the end I commandeer one of the PCs and send Novell's popup messages to any and all of my colleagues. Eventually got one to remote into the site's server and we had a conversation using the console.
2) Similar to 1, except the site was locked. Not wanting the several-hour drive to be in vain, a workmate and I proceeded to dismantle a window and essentially break in to do the maintenance work. Thankfully we managed put the window back without any damage, though it wasn't exactly any more secure than it started.
3) Some angry user at another remote site complained that the site's server was too noisy and wanted it gone immediately. When we said that we couldn't just remove the server, he filed a Healthy and Safety type complaint (which, as I'm sure many of you can imagine, causes all sorts of unrelated people to be suddenly interested in the affair -- in other words a right mess). Anyway, when we were trying to get the server to slow down its fans someone had a bright idea. We powered the server down and rung up and asked, "Sorry to bother you sir, but we've made a small change to the server, is it still too noisy?" ... "Yes! Grumble, grumble, when are you going to fix it? Urgent!" Suffice to say that put an end to the validity of his complaint. We think he was complaining about the air con noise all along...
4) When testing networked printers en mass during an upgrade, I would send a test page of my own design -- a single smiley face icon in Word blown up to fill the entire page. Ring up a nearby user and ask them to confirm what's been printed. Years later, it was a game to spot these smiley faces after users took them and used them as cubicle decoration.
5) The old grey engineers who knew the OS backwards told me this tale of yore. They would install a camera in the server room. Then they would send new guys down to check out some "faulty" server. Using their knowledge for evil, they would have the server prompt the poor sap "This server uses palm recognition. Please place your hand on the screen." The camera would capture their bewilderment and eventual obedience. Replace "hand" with "face" and occasionally "voice" for different flavours.
Best wishes to all the bods out there in support. I've been there so I know how you should be treated. "Urgent" is a dirty word.
Government office called about a computer that was working fine but "couldn't start windows". After a couple questions and answers that made no sense I popped around to have a look since it was just down the street.
Several people were standing around the switched-off computer staring intently at a monitor controls menu. The computer apparently was switched on and off each day using the monitor switch. I pressed the button on the front of the computer and left.
"My computer is posessed by the Devil"
A few years back I had a frantic phone call from a lady . Her computer had become "possessed by the Devil" and could I come round right away, preferably with a priest. When I got there (without a priest), she demonstrated the problem. It was running Red Hat (version 8, I think) and the machine was in a darkened room with only the screen for illumination. About 10 minutes after switching it on, the screen suddenly darkened and then slowly lit up again with a display of flames. Slowly a large evil-looking head arose from the flames, with a sinister grin (it was also smoking a pipe, but I didn't notice that till later).
It was, of course, the "xflame" screen saver, but it gave me quite a shock, particularly in that darkened room. I can well understand an inexperienced and superstitious user assuming the worst. The exorcism consisted of deleting xflame from the list of screensavers. I decided not to worry her with stories about the daemons and zombies that habitually inhabit Linux/Unix boxes. After that, I removed xflame on all the linux boxes I commissioned, to prevent further callouts.
On a different note, a few weeks ago my daughter called me to say that "the internet was broken" and her mouse had "gone retarded". It eventually turned out she had let her pet rabbit loose in the room and it had acquired a taste for PVC. The ethernet cable was bitten clean through and the mouse cable had a series of tooth marks in it. I made her pay for the new mouse.
Xflames, of course
caller: I can't log on.
me: let me take a look. Let's start with your user name . . .
calller: blah blah
me: Hmm, looks OK. Not locked out, etc, etc. Do you know the PC ID, or IP address, or . . .
caller: no. And I can't log on to find out.
me: I'm stuck. You should be fine.
caller: blah blah blah (friendly, but I'm still in trouble).
me: Sorry. I'm really stuck. I'll have to visit your branch and take a look.
<the next day>
The office is a mess. Papers everywhere. Filling cabinets knocked over and contents on the floor. It turns out they had been robbed, and the PC stolen (not the keyboard, or screen, which still switched on).
All I can say is...
I used to work for Packard Bell.
Administrating, Maintaining, & Troubleshooting a Windows 95 network of over 5,000 PC's, 12 Servers, 4 "R&D" stations, and MILES of 10bT cabling...
I mean literal *miles* - the complex was an old Military Armoury - ten buildings, each two American Football fields *wide* & 1/4 mile long.
Networked together for controlling the R&D, testing, maintenance, production, & shipment thereof, Packard Bell personal computers.
I'd still be going through therapy if it weren't for my ability to suppress the memories of that hell hole & the suffering working there caused.
If I never see a Packard Bell machine running Windows 95a again, I will be a VERY happy geek.
The caller was my mum!
I was supposed to be on 'third level support' for a brand new SAP deployement. Suffice to say, it was complicated and expensive, and we were expecting a few calls.
The grand pooh-bar of IT was on the floor, just to add moral support. A few senior managers who wanted to be seen to be helping were there too. It was the first morning of 'real live users'.
At my desk. Everyone trying to look professional in front of management. My phone rings. I answer professionally. It's my mum, but I don't let on. She's trying to print a photo at home, but it isn't working.
I walk through everything. Have you tried this, have you tried that. What do you see now. How about this. Is the light on? is the cable in? Does your printer actually work? Has it ever worked? try agian. Is there an error message/ read it to me, etc, etc.
Over an hour later, she's printed OK.
There have only been one or two other calls, and other people have taken them. It hots up a bit later, but the managers have gone by then.
Grand Pooh Bah of IT holds a meeting at the end of the week to discuss the project - a few issues, but mostly OK. Everyone is relieved.
A big shock: I get a 'recognition award' and a big pat on the back. And a Myers voucher for $100, I think.
It turns out he was most impressed with the way I handled what was obviously a 'very difficult' user on the morning of go-live. He was listening to me for an hour or so, and I solved the callers problem. He was very happy. That's real 'customer service, etc, etc.
I never told anyone it was my mum.
Blonde user syndrome.
Many years ago, back in the halcyon days of windows 95/98 we had a customer phone up explaining that her computer would turn on and then suddenly stop, we dug further to discover the machine was hanging at POST.
Asking her what the last thing she did was, nothing she replied, just turned it on but then proceeded to ramble on about how she would need a plumber to fix this...
A Plumber? i asked (by now i'd activated the speaker phone so the other chaps could hear this)..
yes she said a plumber.. We were now of course all thinking she had had a leak or a flood and obviously some thing was water damaged, like the keyboard or something..
Can i just ask why you think you need a plumber to fix your PC i enquired??
Well, she said, the last thing right at the bottom of the screen is L2 Cache, 32 pipeline burst...
I shit you not....
Paris because she knows about dripping pipes...
Ok, yes, second post, but just one word: Secretaries.
Lovely laides but completely clueless when it comes to applications and computers. Still, usually good for a cup of coffee and always so pleasant and greatful.
I provide for a few local small businesses and they can always ring me for support during the day so I guess it counts as remote support.
I took a call last year from a user not renowned for her grasp of technology (alright, she's thick as two short planks and is known as Thick L****** to most of the employees of this company and plenty of their customers too.) which went along the lines of;
User: My computer just crashed.
Me: OK were there any error messages or did it just stop responding?
User: Yeah there was an error message but it's gone now.
Me: OK, what did it say
User: Some computery shit I didn't understand.
Me: OK so did you write it down?
User: No, I just reset my PC, I thought you would know?
Paris. Totally Paris.
Yes, I too once had the "my computer is on fire" phone call.
me: Is it really on fire? I mean, really? Actually 'burning', with real flames? You're not joking, right?
caller: Yes. It's burning, right now. The flames reach all the way up to the ceiling. . .
me: That sounds very dangerous. Why did you call me - you should call 000 (that's emergency in Oz)
caller: I wasn't sure what to do. I'm new at computers, and you seemed to be helpful once before.
me: It's NOT about the computer. It's about the fact that you have a huge fire on your desk. Call 000 right now.
caller: So it's not because I'm saving my file to A:\ drive? I asked Steve, and he says that we shouldn't save to A:\ drive . . .
me: JUST CALL 000. Please! Right now!
caller: I'm never sure with computers . . . You know, I sometimes save to the wrong drive. Don't get mad at me, it's by accident. It can cause things to go wrong . . . I'm still new with this system, and I don't want to cause any problems . . .
caller: ouch! this phone is getting hot. Can I call you back from another desk.
me: I can hear the sirens in the background. I think that's the fire brigade. Just hang up. I'll call you tomorrow (about a replacement PC). Goodbye.
The next day: A three PC branch office. One PC destryed by fire. Other two PCs destroyed by fire retardant damage from the fire brigade. (Extinguisher was on the wall, about 10 meters away, apparently. The fire brigade bloke grabbed it and put the fire out).
Here's one for the oldies
One of my user departments had gone through the trials and tribulations of converting their office from a wholly paper/index card operation to a computer based system. Their reports were printed on Fan-fold paper.
Comment from their user manager: "This system has made an enormous improvement, but can you arrange for those prints to have the hinges at the side rather than the top."
If you understand the joke in this you must have been in IT forever and should have retired ages ago :-)
Users - Don'tya Just Luv 'Em?
I work abroad and we support users using several languages and have too many ditties to count. However, two stand out......
Due to the vagaries of the local infrastructure, we often have to use a device called a "Stac", which is a voltage convertor and stabiliser. We had one user log a call for us to supply a Stac for use with his electric pencil sharpener.
On another occasion, one of my local colleagues fell foul of the language barrier and logged a call on behalf of a user who apparently wanted "cretin people" removed from the e-mail address book.
slightly unrelated ...
... but along similar lines. A long time ago, before I got into publishing and IT, I worked in a Hi-Fi equipment shop. A guy had just bought a large and rather expensive Hi-Fi (record deck, radio, tape deck). A little while later he rang to say that it wasn't working and, as he lived locally, could he bring it back. I said OK. A little while later I saw his car pull up outside, and then he got the large Hi-Fi unit out of his car, and struggled into the shop with it, and then went back to get the speakers (which were equally big and heavy). Anyway, I soon saw the cause of the problem – when he'd got home and started to assemble the unit he'd looked at the plugs on the end of the speaker leads and thought they were "for American power outlets" so he'd cut 'em off and fitted 13 amp mains plugs to 'em before plugging the speakers into the mains!