back to article Attempt to clean up tech area has shocking effect on kit

Hello, Friday, El Reg’s old friend. We’ve come to talk with you again… because the vision that has softly crept in must be the latest instalment of On-Call. This week, our reader’s tale of tech support conundrums solved is a real shocker – so without further ado, let's meet “Gerald”. He takes us back a couple of decades to …

Page:

  1. AdamT

    Inadvertent Van de Graff generator

    Similar tale from a couple of decades ago:

    A clean room with ESD conductive flooring, lots of high speed (for the time) opto-electronics components and a very expensive high-speed oscilloscope. On a trolley. With rubber wheels. Bring the 'scope over to investigate circuit board not quite working properly, apply probes, suddenly circuit board is working even less well than it did.

    Eventual realisation that conductive floor + rotating rubber wheels + conductive trolley = Van de Graff generator.

    Solution? Attach a small chain to the trolley frame that drags on the floor....

    Apocryphality rating? Well, this is the story I was told when I asked why all the equipment trolleys had little chains attached to them that dragged along the floor...

    1. Steve K

      Re: Inadvertent Van de Graff generator

      had little chains attached to them

      ...or they were nicked from Tesco's for £1 each

      1. defiler

        Re: Inadvertent Van de Graff generator

        ...or they were nicked from Tesco's for £1 each

        Nicked for £1 means bought for a bargain!

  2. big_D Silver badge

    Electrifying

    I had a problem at a previous company. I work in Germany and the plugs, like the UK, have the Earth connect before positive and negative. The difference being, the Earth prongs in the plug are exposed, which is also good if you need to earth anything / yourself...

    Anyway, I was standing in my office one day, leaning against the window, looking at my whiteboard and thinking. Running under the window sill was trunking with about 9 power sockets. I leaned forward and lost my balance and reached instinctively behind me and my fingers went into the power sockets and I gripped the window sill and the inside of the socket to hold myself.

    BANG!

    I got an electrical jolt up my right arm and down my back. I let go and staggered forward.

    I then limped down the corridor to the techs and told them. They said, nah, impossible, but I was white as a ghost and was in pain, so they came back with me and tested the sockets. All, but one, were fine. The electrician who had wired them up had somehow managed to swap phase and earth on one socket. The one socket that nobody had used since the buildings renovation 10 years earlier. The one socket I managed to stick my finger in when I lost my balance!

    After about an hour, my arm had recovered enough that I could continue working and the techs had re-wired the socket, properly this time!

    1. big_D Silver badge

      Re: Electrifying

      Bah, the earth prongs in the socket!!!

    2. MJI Silver badge

      Re: Electrifying

      Proves again BS1363 FTW

      1. Will Godfrey Silver badge
        Happy

        Re: Electrifying

        .. or as I was once told...

        L = leave empty

        N = no connection

        E = every wire

      2. Alan Brown Silver badge

        Re: Electrifying

        "Proves again BS1363 FTW"

        All that would have proved is that when someone plugged into that outlet you'd now have a live case on whatever devices was on the end of the cable.

        1. Yet Another Anonymous coward Silver badge

          Re: Electrifying

          That's why the world was a better place when you wired your own plugs = evolution won out

      3. Herby

        Re: Electrifying

        "Proves again BS1363 FTW"

        Nah: NEMA 5-15 Even better FTW.

    3. Lee D Silver badge

      Re: Electrifying

      In a school. One of the offices has a fuseboard that kept popping. We had electrical problems all over because we just kept expanding and expanding, but we slowly eliminated all the causes (things like crossed-phases on a two-plug heated canteen trolley, etc.) and got them fixed.

      But one continued to baffle me - when the office woman put her fan on, sometimes the fuse popped. But not immediately. Often some hours after being plugged in. Even when there was nothing else on the circuit. It took months to narrow it down to the fan and I still kept thinking the fan was faulty somehow, but it always checked out and worked fine elsewhere.

      Traced the problem eventually. Someone had re-wired the plug on the extension lead at some point and got it back-to-front and got the brown and blue mixed up - the PCs and printers wired into it didn't care. But the fan somehow did*. I was always amazed that it lasted that long, that such a low-wattage item could take out the whole circuit, and that it would run happily for days at a time without popping.

      Rewired the extension lead properly, and everything has been good since.

      (*maybe because it was a metal guard on the fan if it spun and touched something that was earthed? I don't know, I can't imagine that the fan guards are electrically connected at all, and the earth pins were fine).

      1. Daedalus

        Re: Electrifying

        Fan motors: typically they have heavy duty coils that cause current surges when starting up. Not as bad as vacuum cleaners, some of which will make your lights flicker, but in a delicate situation it's easy to see how the breaker might go off.

        It might even be a shutdown surge: as the coil fields decay, they produce current that normally goes to neutral, the live side and its breaker being disconnected at the fan switch. But here fan neutral was going to live....

    4. Stevie

      Re: Electrifying

      Take zat, Tommy pigdog!

      1. Anonymous Coward
        Anonymous Coward

        Re: Electrifying

        > Take zap, Tommy pigdog!

        FTFY

  3. frank ly

    Typing Cleaners

    Some years ago, a colleague was developing an application as a user interface to a real time controller/monitor for an industrial process. He ran it overnight (connected to a process simulator box) to make sure it was working and stayed working but it kept throwing up errors the next morning.

    Eventually, he figured it out and disconnected the keyboard for his overnight runs. The cleaner was running her duster or cloth over the keyboard.

    1. ChrisC Silver badge

      Re: Typing Cleaners

      I now have my keyboard and mouse hooked into the USB hub on one of the monitors for a similar reason - if I need to leave the PC running whilst I'm away, switching off said monitor then disables keyboard and mouse, so that when I return I'm not faced with random things having happened thanks to one or other of our cats deciding to walk over or sleep on the keyboard and mouse pad...

      ...also ended up having to change the setup of my laptop power button after recently discovering that our latest feline addition a) is *really* interested in walking all over it as I'm trying to get some work done and b) can apply just enough pressure from a well-placed paw to depress the button far enough to activate it.

  4. Steve Cooper

    Flickering lights

    In a previous life I installed a dozen TVs up on the walls around the office displaying various monitoring information running via HDMI over Cat5 adapters from a PC 30 metres away in the comms room. I couldn't work out for ages why the TVs would randomly go blank for a few seconds then back on again until I stayed late one evening and found they worked perfectly when the office was empty. Turned out to be interference from the IR motion detectors that kept the lights on in the office was somehow inducing enough noise into the Cat5 to upset the HDMI signal when people moved! Shielded Cat 6 rather expensively fixed the problem.

    1. Chris 239

      Re: Flickering lights / previous life bah!

      You whippersnapper - making me feel old!

      HDMI is not old enough to count as a previous life - VGA maybe, but Composite Video to really qualify.

  5. chivo243 Silver badge
    Facepalm

    She's gone and tidied up...

    and I can't find anything!!

    The office cleaners regularly hit my desk, and everything is in a different position! Mousepad moved onto the wet spot where they cleaned the first half of the desk, and then stuck to the desk when I arrive the next day...

    The cleaners have no access to our sensitive areas, we prefer to do them ourselves when required.

    1. Steve K

      Re: She's gone and tidied up...

      The cleaners have no access to our sensitive areas, we prefer to do them ourselves when required.

      Ooh matron! Fnar, Fnar! etc.

    2. John Brown (no body) Silver badge

      Re: She's gone and tidied up...

      "and I can't find anything!!"

      Most places, the desks are out of bounds to the cleaners. If your desk is messy or dirty, it's your fault, your responsibility. Especially if the cleaners are a 3rd party contract.

  6. Peter Christy

    It happened to colour TVs, too....

    Back in the late 60s, between leaving college and starting a "proper" job, I worked as a salesman / delivery driver for a local shop that supplied TVs, washing machines, Hi-Fis, etc. One of our customers had just bought one of these new-fangled colour TVs - a dual standard 405/625 model, if memory serves correctly. These early sets were very sensitive to stray magnetic fields - even the Earth's - and had to be carefully aligned by a service engineer in situ.

    Our resident alignment expert went out with it on delivery, carefully de-gaussed the screen and carried out all the usual purity and convergence adjustments, leaving the customer with a crystal clear picture.

    A week later, the customer called to complain his TV had gone screwy. The service engineer went out again, and sure enough, the purity and convergence had all gone to pot. He carefully re-aligned it all, and left the customer with a perfect picture again.

    Exactly a week later, the same problem ensued! By now the customer was getting a little irate (these sets were very expensive!), and the service engineer very puzzled!

    Since the problem always seemed to happen on a Thursday, the service engineer convinced the boss to let him go and sit in the room on Thursday - all day if necessary - to see what was going on.

    He arrived at 9 o'clock sharp, and the TV was fine. It stayed that way until around 1130, when the cleaning lady arrived and proceeded to hoover the room containing the TV, pushing the hoover with its heavy and powerful electric motor under the TV! Needless to say, the picture immediately went bananas, and required a careful de-gauss and re-alignment to restore proper operation!

    The cleaning lady was very apologetic, but in truth it wasn't her fault! No-one had anticipated the effect a powerful vacuum cleaner might have on a CRT!

    Later TVs had much better screening and better built in de-gaussing systems, and of course, modern displays aren't affected by stray magnetic fields. But back then, it was all one big learning curve......!

    --

    Pete

    1. Jos V

      Re: It happened to colour TVs, too....

      Truth that. I usually carried a magnet with me from a decommissioned subwoofer. You could "smear" the blot on the screen back out by expertly waving it over the screen. Magic! Usually tv sets that alligned north-south a long time would get that.

      More fun with people putting plant pots on top of the set and watering all the time too heavily. And heavy plants would sometimes even break the circuit boards underneath, requiring a lot of trace resoldering. Extra points for fixing TV's by vacuuming the interior cob-webs and dustballs out.

      1. Alan Brown Silver badge

        Re: It happened to colour TVs, too....

        "Extra points for fixing TV's by vacuuming the interior cob-webs and dustballs out."

        Ugh. TVs. High voltages. the days when people smoked indoors. A lot. Tar and ash coatings on everything, but especially on the HV components, and arc tracks through them.

        Memories of having to use copious amounts of isopropyl to clean them and the colouration of what trickled out. *Shudder*

        1. Lilolefrostback

          Re: It happened to colour TVs, too....

          Just imagine what that did to their lungs.

      2. Rich 11

        Re: It happened to colour TVs, too....

        Extra points for fixing TV's by vacuuming the interior cob-webs and dustballs out.

        And using methanol to remove the worst of the oily nicotine build-up on any surface which had been exposed to the fumes from a chip pan bubbling away for an hour or more every Saturday before the footie.

        Edit: I'm glad to see I'm not alone in remembering that muck, Alan!

        1. Nunyabiznes

          Re: It happened to colour TVs, too....

          *Raises hand*

          I've fixed a lot of CRTs over the years, and I was grateful when indoor smoking became a thing of the past.

          We used to have to clean overhead cable trays in a certain high security area every year. It took a few years before we got all of the bleah out of the crevices after smoking was banned in the area. Ick.

          Isn't it amazing how much has changed since the 60s (specifically in IT/electronic repair/etc)?

    2. Lilolefrostback

      Re: It happened to colour TVs, too....

      Long time ago (> 20 years), our team was being upgraded from dumb terminals connected to the mainframe to UNIX workstations (HP or Sun - don't recall) with lovely hi-res colour monitors. Two of us ran into problems - the colour monitors went crazy. They tried multiple monitors in both cubes - no joy. After about a day's investigation, it was determined that we were sitting directly over the incoming power mains for the entire building. And, oh, by the way, no one should be sitting in those cubes as the magnetic field was too high to be safe.

      Yay!

      It only took them a couple of months to find us new cubes.

      And still no super powers.

  7. Anonymous Coward
    Anonymous Coward

    Where would you plug a vacuum cleaner in?

    I was investigating some issues with failures on a large Unix server in a major UK bank and was in the computer hall where the said kit was hosted. The issues happened at around the same time each day, so a team convened at the appropriate time in the data hall. Everything on the box checked out ok and much head scratching was going on until the cleaner arrived.

    He wandered into the computer hall with his vacuum cleaner (the floor tiles were carpeted) and we then watched him pull out an extension lead with a 3pin socket on one end and a C13 on the other end. The vacuum plugged into the 3 pin socket, the cleaner then opened the cabinet wher the errant server was, plugged the C14 into a socket on the PDU and started the vacuum cleaner. Exactly on cue the server crashed.

    1. Anonymous Coward
      Anonymous Coward

      Re: Where would you plug a vacuum cleaner in?

      In hospitals, the apocryphal tradition is it ITU!

      "Why do patients always die at the same time of day in this bed?"

      1. Anonymous Coward
        Anonymous Coward

        Re: Where would you plug a vacuum cleaner in?

        The ITU case may be apocryphal but this was in a datacentre between Leeds and Bradford.

        It’s the same place that used different coloured carpet tiles in the computer halls to show you the route to emergency exits and then plonked an IBM ESS ‘Shark’ storage array across one exit route.

        1. Yet Another Anonymous coward Silver badge

          Re: Where would you plug a vacuum cleaner in?

          "plonked an IBM ‘Shark’ storage array across one exit route."

          With laser beams ?

  8. Alien8n

    MOSFETs and clean rooms

    During my engineering days it was decided that anew generation of MOSFETs that the company was designing needed to be made in an entirely clean environment. They had a spare building which they converted into a clean room environment with all the process machinery in there. Including the moulding press for encapsulating the device. In case you aren't aware, moulding presses create a hell of a lot of dust.

    Also, the rest of the MOSFETs, IGBTs, and Diodes were made in the main building, the only precautions taken were to give people anti-static slip on shoes, a lab coat, and all paperwork was kept in an anti-static bag. Most failures actually came from a combination of 2 sources.

    1. Accidental swapping of paperwork (usually as part of the curing process in the ovens as multiple batches of devices were cured together in the same ovens).

    2. Outright and utter stupidity. The usual excuse was "well it was in my area so I assumed it was my next job" when encapsulating devices that had actually been put aside waiting for wires to be attached. Some people really couldn't be arsed to take one look at the batch and make sure the wiring section had actually been signed off as complete.

    It was No 2 that persuaded me to design the system at my next company so that you couldn't start the process on the system unless the previous process had been signed off.

    1. J.G.Harston Silver badge

      Re: MOSFETs and clean rooms

      Part of that is idiots who use *your* intray as *their* pending tray, and then they wonder why their work has vanished. Ditto morons who *store* things in *rubbish* bags. It's in a *RUBBISH* bag, so it got put in the *RUBBISH* bin! WHAT. DID. YOU. EXPECT?????

      1. Nick Ryan Silver badge

        Re: MOSFETs and clean rooms

        Or electronically... storing emails or calendar items in the "Deleted Items" folder and then wondering why they can't find them later because they've been deleted (usually due to a centrally controlled purge).

        I once had to physically demonstrate to one office manager the stupidity of storing things in a rubbish bin by shoving the contents off her desk into the bin under her desk and asking her if she expected them to still be there in the morning after the cleaners had been.

      2. 2Nick3

        Re: MOSFETs and clean rooms

        Funny how "TEMP" sometimes translates as "Keep the most critical data you have right here - it will be safe forever!"

        I once had to admit to an IBM VP that I had indeed reformatted the 3995 optical disc that the DBA had stored the copy of the DB2 config file on just prior to his managing to corrupt the production config file.

        When the VP finally stopped for a breath I added, "The disc was both physically and logically labeled TEMP002A/B." He took 2 more breaths and the tirade was redirected to the DBA.

        1. J.G.Harston Silver badge

          Re: MOSFETs and clean rooms

          I once had to recover a DOS/Win system where some bright spark had set TEMP=C:\DOS

        2. Alan Brown Silver badge

          Re: MOSFETs and clean rooms

          > Funny how "TEMP" sometimes translates as "Keep the most critical data you have right here - it will be safe forever!"

          It still happens - to the point where I send out periodic warnings that files in these areas can and WILL occasionally disappear without warning - and if that happens we _will not_ assist with data recovery.

          1. Terry 6 Silver badge

            Re: MOSFETs and clean rooms

            And being fair.cleaning up the temp files in Windows systems will sometimes mean that a programme suite (particularly Office) will fail to update if you want to make changes; because the msi is demanding a file that it had installed and has only ever existed in the temp folder!!!!!

    2. Alan Brown Silver badge

      Re: MOSFETs and clean rooms

      > Some people really couldn't be arsed

      'Quality Bru^Hitish workmanship' (and quality british 'designs', both of equipment and manufacturing lines - did you ever hear the one about the Triumph motorcycle where they discovered they couldn't install the engine into the frame on the first day of production line operation? - "But it worked on the prototype" - because they _built_ the engine in the frame)

      It's stuff like this that resulted in "Made in Britain" being regarded as a warning label across the commonwealth for a very long time. When you look at the reliability of Range Rovers and other quintessentially 'british' cars, you can see that fine tradition is still being upheld.

  9. Arachnoid
    Joke

    Clean room?

    So you let cleaners into a clean room......... cant be that clean then

    1. Yet Another Anonymous coward Silver badge

      Re: Clean room?

      You can let clean clean room cleaners in to clean a clean room

      They also run retail distribution of bivalve mollusks in littoral area

  10. Anonymous Coward
    Anonymous Coward

    earthing stories ...

    There are so many. I was working at a global company and we were managing appliances all other the globe.

    They were very reliable and apart from 1-2 issues out of 100 IT appliances, we were doing good.

    Except from the 3 appliances in our indian site: their MTBF was 1-2 months max. All 3. Which was a real issue, because shipping a new one there, once the next one was totally fucked (mobo burnt) would cost us too months (logistics in India, you'd love it).

    It took us a while to remotely figure out what was happening. In fact, there were 2 separate electrical lines, and each appliance had one PSU linked to the first and the second PSU to the second.

    After a dozen of such failures, I asked the local dude, to plug both PSUs from one appliance to the same line. 3 months in, this very appliance was still doing allright, while the 2 other had their usual failure rate.

    Explanation: electricity standards are ... different in India (read: shit job is the norm) and the dude that did the 2 lines of the DC, somehow didn't link the 2 lines' earth properly. There was different voltages between the 2 PSU earths, and it regularly killed the mobos.

    Solution was quick: we put all appliances PSUs onto the same line, and never had any failure ever since !

    1. Jos V

      Re: earthing stories ...

      You can also find out if grounding in your setup is done proper when you try to measure the 80V/16hz ringer circuit output (as in telephone ring voltage), by hooking your 1:1 oscilloscope probe to ground and touch the ouput with the probe. It's very effective, and capacitors give of nasty smoke when they blow. Can be expensive too.. (it wasn't me, ex-boss, I swear)

    2. Alan Brown Silver badge

      Re: earthing stories ...

      "Solution was quick: we put all appliances PSUs onto the same line, and never had any failure ever since !"

      Which is fine until either

      1: The phase imbalance burns out the supply transformer

      2: The bad earth finally lifts and you find out that 3phase 240V supplies (nominal to earth) can suddenly rise to 480V and ALL your PSUs burn out more or less simultaneouly and spectacularly.

      Working around a problem of shit power without understanding what you're doing can make the hurting much much worse in the long term. Find the problem and deal with it. Fuckwit electricians can be dealt with by making them hold onto the end of the line whilst you megger it.

      1. Anonymous Coward
        Anonymous Coward

        Re: earthing stories ...

        "Which is fine until either

        1: The phase imbalance burns out the supply transformer

        2: The bad earth finally lifts and you find out that 3phase 240V supplies (nominal to earth) can suddenly rise to 480V and ALL your PSUs burn out more or less simultaneouly and spectacularly.

        Working around a problem of shit power without understanding what you're doing can make the hurting much much worse in the long term. Find the problem and deal with it. Fuckwit electricians can be dealt with by making them hold onto the end of the line whilst you megger it."

        I understand what you're saying. But there was absolutely no way to have the electricity team even investigate this. Your problem, my problem. Clearly, it was NOT their problem, and VERY clearly, it would only be investigated once something bad happens.

        Probably it did at the end :)

        The AC writing the original story

  11. Anonymous Coward
    Anonymous Coward

    Cleaners again...

    Was working in an electronic factory where spare parts were kept in a trolley...which every night, the cleaners moved outside to a carpeted area while they cleaned the floor :-)

  12. Luiz Abdala

    Grounding the work force... permanently.

    A friend of mine told me about his previous job.

    They were about to move their business into a new building... that the company would build from the ground up, to better suit their demands upon expansion. But, before moving, they had to install everything in it, including all the electrical bits. He was responsible for the IT part so he lets know of the grounding demands: about 75 wrist straps in an assembly line, and means to ground them, nothing complex. Well, let's just say the Sparky in charge wasn't exactly AWARE of a few demands for it.... or building codes for that matter....

    ...like not using the same ground for WRIST STRAPS and LIGHTNING RODS. This friend of mine, on the first inspection visit, quickly noticed this weird line across the ground, before the installation of raised floors... hooked up to the rather thick lightning rod lines running outside. About 75 people would've met their demise in the same manner as anecdotal Benjamin Franklin. The first electrical storm on the brand new facility would be the last for the majority of the workforce, no matter how thick those grounding lines could be. Stranger yet, nobody else noticed it.

    Would you use a wrist strap hooked to a lightning rod? I guess not.

    1. Aladdin Sane

      Re: Grounding the work force... permanently.

      Did the sparky sign his work as J. Jones?

      1. Stevie

        Re: Grounding the work force... permanently.

        J. Jones? More like B.S. Johnson if you ask me.

        I knew Jim Jones. Apart from the fact he smoked twenty Capstan Full Strength a day and spat copiously every three seconds he was a magnificent chap. Leader of St John Backsides' school orchestra's string section and under his leadership the string section nearly always beat the brass section (headed by an equally competitive Fred Cole) to the end of whatever we were playing. They say our rendition of The Valiant Knight brought tears to the eyes of all who heard it on parent's night.

        Ah the music, the bulging eyes and sweat-soaked brow of Mr Cooper as he madly tried to keep up with his conducting, the protesting squeaks from the clarinets, flutes and bassoon lollygagging in the rear as usual.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like