...it wasn't the first time the tenant had trouble slotting a 2.5" in somewhere.
Welcome to another Friday (!) and therefore to another edition of On-Call, The Register's column in which we let readers vent about jobs gone bad. This week, meet “Ian”, who once worked in a data-centre-for-hire, doing all the stuff that tenants needed done. Of course he's had some mirth-making moments along the way. “In my …
Did you mean STFU?
Also, it must have been a pretty crappy server chassis if the idiot was able to fit the drives into the caddies the wrong way around; most caddies have a securing mechanism that locks into the screw holes of the drive (either regular screws or pins from a quick-release mechanism) and the holes in the drive are asymmetrical front-to-back so you can't put the disk in the wrong way around.
Maybe the "server expert" drilled his own holes for his "special" drives?
Ah, Urban Dictionary....
Who could forget the Dirty Sanchez?
Seriously, it's pretty fucking awesome at obscure slang, but, now that the vast majority of terms are in, the point system gaming seems to drive people towards coining terms for ever more inventive and unlikely sex acts. The poor DS is looking almost mainstream nowadays...
Maybe the "server expert" drilled his own holes for his "special" drives?
I've seen enough people try and force things into obviously wrong slots on motherboards and such like, to think that some people are just too stupid/ single minded / determined to think this is tricky what happens if I try it the other way round?
Quite willing to belive someone like that would actually use a hammer to force a companent into a space that is obviously not designed for it.
A guy I know worked as level 2 server support for Dell in the mid-90s. He got a call from a customer with a misbehaving server. Back then, most VGA adapters had a forty-pin feature connector. Customer hooked an IDE hard drive up to the feature connector to "give the graphics card more video memory".
A guy I know worked as level 2 server support for Dell in the mid-90s. He got a call from a customer with a misbehaving server. Back then, most VGA adapters had a forty-pin feature connector. Customer hooked an IDE hard drive up to the feature connector to "give the graphics card more video memory"
Hmm, I remember the DB9 Token-Ring plug would, when plugged into the DB9 CGI video card, spectacularly crash the network.
Not a hammer, but will publicly admit* to using a hacksaw an 8-bit ISA card that had part of the board hanging down so it wouldn't fit into the only free 16bit slot in the PC. Much to the amusement of the office.
Should point out that a) had a spare card in case it didn't work and b) after closely examining the PCB and deciding that the area I was cutting into was for an optional parallel port that wasn't installed.
* but still AC
ISA cards were so much less fiddly, though. Much more space to get your tools in there. And when I had to stick 6 HDDs in a unit with only four IDE device capability on the mobo, very useful that was too. Most cards for disk controllers didn't have switchable addresses, and those that did were eye wateringly expensive.
So you take an IDE controller from the old, old, old days when IDE controllers weren't integrated with the mobo. Grind out a gap in two of the address lines, I forget which ones now, then cross wire them. This swaps the address of the card's controller into an unused segment of the memory map. Bit of a tweak in the PCI address settings in Windows 95 which expected things to be in very defined places and Bob's your uncle.
PCI-e comes in a variety of lane lengths, once it's beyond 1x physical the rest of the connections are just data. Server boards tend to have PCI-e slots that are less than 16x physical. It's useful to use graphics cards in server boards sometimes, they're practically all 16x, and 1x graphics cards are ludicrously expensive.
Enter.. the hacksaw. Carefully saw off any part of the connector that's longer than the slot. Alternatively use a scalpal at the end of the PCI-e slot to cut it open. People have done both, and it works.
I looked at the feasibility, likelihood of ruining a board, ran away screaming and bought a slot adapter cable. It does mean the cards sit higher than they should, and that it's more difficult to get them into a case, but at least no hardware is ruined.
I once saw a VGA cable that the user had managed to plug in upside-down. Yes, they even got the set screws to thread and tighten. Needless to say the shell was a big mangled and the pins were all bent. But they got it to mount to the port on the PC, and that's all that counts (until you actually want it to work).
I had a client who built a "high spec" gaming system, in a micro ATX box. finding that the honking great graphics card with its three fans and an aluminium heat sink that Scotty could have used to keep the warp core cool was getting fouled on the cables of the hard drive he resolved to move the hard drive forward in the case. He achieved this by taking the hard drive out to his shed, sticking it in a vise, and then drilling four carefully measured, and centre -punched holes via his drill press, straight into the disk chamber. He brought it to me when it mysteriously refused to boot.
I work in schools.
Often get called / sent to a struggling school as a favour between my employers and their sister schools.
Went to one tiny independent (i.e. private) school that were having "a lot of IT issues with their contractors", and I was asked to tell them if they were being conned or not.
They had no permanent IT staff, so were paying through the nose for support from a "specialist" firm of contractors for educational IT for absolutely EVERYTHING to do with IT.
In the space of one morning, I found:
- The "wireless network" was one 802.11b WAP sitting out in the open in the IT suite (not even screwed to a wall), which only had WEP on, and was plugged into a network point right next to the computers (no security, no port isolation, no VLAN, get the WEP key and you're online on the school network unhindered - and this was in 2015!)
- The "server" (singular!) was a desktop PC with a RAID card in it, sitting in an office behind someone's desk.
- The "network" was 2 or 3 small unmanaged switches placed in random locations and usually out in the open and not even in a cabinet.
- The "IT Suite" was ten Dells stuck in a room with no settings, security or anything. Just joined to the domain, off you go.
- The "Internet connection" was an old ADSL line that barely kept 1Mbps, but also was swamped with Windows Updates and all kinds of stuff because there was no management of traffic whatsoever.
But the best bit:
The "server" had previously gone offline, because they ran out of space. Obviously, it had held EVERYTHING from finance to pupil work to staff profiles. They paid the consultants for the upgrade. A guy came in. This is how he "upgraded" the storage on that lone server:
- Pull hotswap drive out of RAID5 during the school day, let it degrade.
- Put in larger blank drive.
- Sit and watch it resync for 8 hours.
- Charge a day's labour at extortionate rates.
- Come back next day, do same to next drive.
And then at the end, he would resize the RAID array to fill up the disks and presumably charge to watch that progress bar too.
Except... well, it didn't quite go to plan. Obviously after several days of constant RAID resync, one of the old drives fell over. No hot spare. RAID corrupt. Dead. Gone. And, the guy found out at that point, no backup. Seriously, he hadn't even checked if there was a backup, or made one, before embarking on the RAID upgrade of the ONLY SERVER that held everything. Total data loss.
Cue massive arguments with the school where they:
- Paid for his time.
- Paid for the drives.
- Paid for data recovery on the remaining array drives.
- Accepted blame for "not having a recent backup beforehand" (the school was basically run by two non-IT people, and these contractors were supposed to do EVERYTHING for them, including supply, support and manage backups!).
- Paid for them - by the hour - to restore the only backup they *did* have from a few weeks prior.
I threw a damn fit when I was then brought in to report on the state of their IT, and my report was extremely damning while being nothing more than factually accurate. These guys were screwing over the school royally, overcharging for said privilege and still not accepting liability for any of their screw-ups, or providing anything near an IT service.
I wrote up a report for their bursar, sent it to them, and then they hand-wrung for a few months or so and I basically gave up on them ever doing anything about it. As far as I know, they're still with the same company and people, and paying through the nose for them.
It's a back scratching free-for-all.
Set up an independent school, make yourself governor, employ your friends and family as teachers, and outsource the rest to the ones who look after you the most. Not the school, you, personally.
That caper can run and run. You just have to make sure you don't employ anyone with a soul, who might consider the money being sidetracked to elevate the lives of the greedy, should actually be squandered on educating the little cherubs.
many businesses simply have no clue how to maintain office computers. I had a lady through a friend call me because they know I do software development pleading with me they lost their entire MS office contact base and records on their windows 7 machine died .
I told her I couldn't do anything .. I didn't want to get involved.
was called to fix a PDP11 that was totally dead. Well, power on, fans running and a few lights here and there, but none of the blinking that indicated a happily running CPU. Now PDPs were the PCs of their time, with a lot of installations getting interfaces added or removed as needed, by third parties or the customer themselves, with even homebuilt, wirewrapped prototype cards being used. Because of this one of the first questions when troubleshooting invariably was "Did you reconfigure anything recently?".
In this case, the answer was a resounding "No", so said colleague opened up the machine and started testing. After a short while he turned to the customer, asking innocently "Did you hear a loud bang or clunk when you switched the system on?" "Uh, no." "Well, there must have been. All the bus continuity cards have popped out of their slots, bounced against the card cage lid, and gotten jammed back in turned around 180 degrees".
A PDP11 with the old Q Bus? We had those. One time we bought some hardware for one of them, and the vendor had their people install it. Those idiots didn't understand the idea that you cannot leave any gaps in the Q Bus slots (all slots from the first to last device had to be filled), so they couldn't get it to work. Even with me explaining it repeatedly.
Unibus, where you need to fit bus grant cards in all empty slots, with the cards often being flip chips, roughly half a linguine square, instead of full-size two-row cards. Because those didn't have any guiding markings or cutouts and only went into a single row you could put them in the wrong way just like that. For Q-bus you only needed grant cards in any unused slot between the processor and the last card on the bus. Those cards occupied both rows, with a ridge between the rows making it impossible to put them in wrong.
I used to service robotic tape libraries and frequently got called out when the customer would put the tapes into the library upside down.... In the early 2000's that was ok because you'd get a good call out rates after hours.
Eventually you'd feel soory for the idiots, so I printed a photo and attached it to the library door. That went ok for a while - but because nobody wants to do the tape juggling - the staff soon left to take on more responsibilty (usually that meant backup admin - i.e. Moving from mopping the floor to the fries)
So the new tape juggler arrived in the job - and soon the call outs started again. Turns out that the new guy put on the barcodes upside down - and the other new guy therefore put the tapes upside down in the robot.
I could've started billing these idiots, but they had a nice cafeteria and subsidised lunch....
I know several people who could probably fail at putting 2 bricks of lego together. These are the sort of people who should never be in a function where they are of any consequence but unfortunately always end up being "Manager of" X where they will do most damage to productivity due to their particular brand of incompetency.
Worse yet are power users, that is to say people with a little bit of knowledge but not nearly enough wit to realise how little they know.
Many years ago I took a support call whilst working for a particular dodgy ISP. Their main site name was $FOO, but their infrastructure was still named $BAR because nobody could be bothered to change the domain names.
So, when this power luser sees his machine connected to $FOO internet regularly making connections on port 53 to machines in the $BAR domain on his newly-installed firewall, he panics and uses this firewall to block these connections, thinking it to be a hack. Then, as he put it, this terrible hacker must have done something truly appalling to his machine since the Internet went ever so slow.
As slow, in fact, as a machine trying to connect each time to its primary DNS, getting blocked and timing out to hit the secondary DNS server...
These are the sort of people who should never be in a function where they are of any consequence but unfortunately always end up being "Manager of" X where they will do most damage to productivity due to their particular brand of incompetency.
Reminds me of a passage from The Silver Chair by CS Lewis :
When the police arrived and found no lion, no broken wall, and no convicts, and the Head behaving like a lunatic, there was an inquiry into the whole thing. And in the inquiry all sorts of things about Experiment House came out, and about ten people got expelled. After that, the Head's friends saw that the Head was no use as a Head, so they got her made an Inspector to interfere with other Heads. And when they found she wasn't much good even at that, they got her into Parliament where she lived happily ever after.
If you're out in the sticks and need to get a PC fixed, you can usually find some half-baked "expert" preying on the locals, and charging them over-the-odds for shoddy repairs.
"Cowboy Keith" was something special though. I could have made a career out of cleaning up the mayhem he left in his wake, but then I would have had to charge danger money to make some of his "repairs" safe...
One machine had a RAM upgrade, with four completely different sticks of RAM - FPM, SDRAM, EDO and who knows what the hell the fourth stick was, but "compatible" it certainly wasn't. He also claimed to have fitted a "brand new" CD drive in the machine, but I found it was caked with dust.
Another machine got a new no-name PSU which promptly set fire to its own cables (I've never seen PSU cables that thin before or since) and he'd used super glue on a CPU heatsink, to hide the fact that he'd overclocked a slower processor and charged the customer for something faster.
There were several other "experts" I encountered in that job, including the local "computer shop" that complained that I wouldn't send business their way. Maybe if they hadn't stiffed me on a personal order and caused me so much data recovery work with their "generic" (spelled C-R-A-P-P-Y) floppies, I might have been a little more charitable.
Then again, wailing that I was threatening their business and demanding my dismissal hardened my stance somewhat.
Years ago, I was trying to reflash the BIOS on my home PC - IIRC, to enable it to handle disks bigger than 1GB - and the process failed halfway through, leaving me with a large brick.
I knew what type of chip it was, and the code was on the motherboard manufacturer's website. Went to my local Cowboy Keith shop, and asked if they could run me up a new BIOS chip.
They looked at me as though I had two heads and was speaking Martian. Had no idea what BIOS was.
In the end, a new motherboard was little more expensive than a new BIOS chip. And Cowboy Keith is still in business.
Not related but this reminds me of a large computer seller in Coventry that tried to fob me off with a faulty CDRW when these were SCSI based (mid 90s). They 'claimed' the reason I was having write errors was because of the soot being made from the burning process.
In my networking days as "Senior RegionaI Engineer" I got called in to placate a customer who was unhappy with the turnkey computers/network system our firm set up. Walked through the front door and saw a Cat5 cable stapled to the wall, following the steps upstairs - and I mean across tread, up riser, across tread, up riser, etc. That was the high point of the work our MCSE had done there.
I remember we had a PC go down, pulled it apart expecting it to just require a new disk or cables reseating. Took one look inside and it was something we had never seen before; an octopus-like plastic box linked motherboard and everything else together, dozens of IDE and other cables. We decided to call outside support.
An engineer arrived and after half a day poking at it reluctantly admitted he hadn't got a clue, but unfortunately had pulled it all apart and had no more idea on how to put it back together than we did.
The icing on the cake was when support called us to rate their service; did the engineer arrive on time? Was he smartly dressed? Was he polite? Everything leading to a 10 out of 10 score and self-congratulation at a job well done without ever asking if the engineer had actually fixed the problem, done his job. The ear-bashing rant they got from our PM was quite enjoyable for us, probably not so much for them.
In a past life before IT I worked for a company where they had one guy who worked on the shop floor who was supposedly the person to go to if you wanted a new car stereo fitted. Luckily it wasn't my car or I think i would have lynched him:
Installed stereo and then informs the owner that he had to do some rewiring in order to get the stereo to work. Okay, I've installed a few myself and you often need to add extra wiring looms to convert the car specific wiring to the 3rd party stereo connectors. Except he informs the owner that in order to turn on the lights he now needs to open the bonnet and flick a switch in the engine bay (he'd disconnected the wiring from the dashboard to the lights for some reason).
Needless to say, within a week the car developed a terminal electrical fault that left it a burnt out wreck on the owner's driveway.
Worse even than the lighting in my kitchen that was installed by the previous owner (and disconnected by myself the moment I noticed what he'd done). 5A spot lights under the cupboards, wiring going to a connector box directly under the sink, then to a light switch and from there to the 30A cooker main. Never trust an electronics engineer who hasn't been trained in electrics to wire your kitchen.
"Never trust an electronics engineer who hasn't been trained in electrics to wire your kitchen."
Hey! I haven't been trained in domestic electrics but have wielded a soldering iron at the insides of a BBC Micro in my life.... But, you know, stupid is stupid and wiring lights to the cooker supply via a connector under the sink is... Stupid.
Like the Statesman oven my landlord had fitted which has a 2.1kW element running at 220V and the internal wiring, which also has a fan and a light on it, is done using a common neutral using 1mm2 steel multi strand like all the other wires inside the thing?
If you do your maths, you'll see that the oven element on its own draws 9.5A, and 1mm2 wire is rated at about the same. One slight juddering stall of the fan motor and the neutral vaporised leaving a hollow PVC tube.
They sell you this shit.
Lights on cooker outlet:
Unusual perhaps, but in no way dangerous of itself
Yes it is, without a 5A fuse immediately after the outlet. Cooker outlets are fused at 30A or have a 32A MCB and require 6mm2 cable (UK). Even ignoring the connection under the sink it's safe to assume that the cable used to the light switch was probably 1mm2 (because even 2.5mm2 cable would be tight, and 6mm2 simply won't fit), which is good to carry about 11A.
The switch itself will have been at best safe for 10A (some older or cheaper switches are 6A) and would likely not have been able to break the circuit in the event of a short (contacts don't move far enough apart, and at 30A+ may have welded together anyway).
Likewise lamp fittings are usually rated for no more than 10A.
A pure short circuit may well still have blown the fuse / popped the MCB, but the cable and maybe the switch would have been damaged by the energy passing, and probably not damaged in a particularly visible way.
An overcurrent of (say) 40A would probably cause enough heating in the cable to melt the insulation and probably cause anything vaguely flammable nearby to catch fire, well before the fuse decided to pop.
you can start a serious fire with a few hundred mA and a fuse is NOT there to protect the appliance
it is there to protect the cabling... go read the BSI standards.
I have even seen CORECT BSI approved fuses not blow and cables to completely burn.
go check out the washing machine & dryer fires.....
Some harsh down voting.
Depending on total loading a spur protected by a properly rated breaker should be fine off the 30A feed. You obviously don't rely on the breaker for the 30A feed.
Wiring in the sink area is not a crime either as long as it is properly insulated (this example wasn't).
@heyrick : This guy was that stupid. I was trained by my step father who used to be an electrician. So I knew putting the 5A lights on the 30A main was a total no no. There would be no way for the fuse to trip prior to any fire starting if the lights developed a fault.
@The First Dave : See above and yes, an open connector block immediately below the taps. One leaking tap and nothing to stop water pouring into the connector block. He even built his own kitchen lighting for the ceiling, after seeing what he had done with the rest of the kitchen I condemned it as potentially unsafe and replaced it with a standard 3 spot ceiling unit.
OP doesn't mention whether this lighting ran via a fused spur, but if so then provided appropriate fuse installed then no risk. If inappropriate fuse then doesn't really matter if connected to 32A cooker breaker or 16A lighting circuit, it'll still burn the house down.
Common sense is the order of the day in domestic electrical work, not a slavish but rote knowledge of the rules without the understanding of WHY they were written.
So called professional in my house managed to wire an FCI the wrong way round, immediately blowing the fuse, breaking the ring and leaving the appliance reliant on the 32A MCB for protection. Same company handled the under cabinet lighting by twisting the wires together and wrapping in insulation tape.
Place came with a full electrical safety certificate...
Running 5A kit on a 30A fuse is essentially the same as not having a fuse at all, as the 5A kit would be on fire well before it pulled enough current to trip a 30A or 32A breaker.
Can you explain something to me please?
Here in NZ, apparently 10amp mains circuits (like your wall socket your TV etc gets plugged into) are supposed to be on 20a breakers.
In my last shop we had a huge bank of spare 10a breakers, but lots of power points (sockets) were fitted to single breakers (ie we would have 5 points on 1 breaker). I wanted to have them split up, and I wanted them on 10a breakers.
I spoke to a couple of electricians who said it couldn't be done, standards/safety etc. My argument was that a lot of the stuff we plugged in expected the mains supply to be rated for a maximum of 10a, and could catch fire if say a fault meant a PSU was trying to draw 15a. Given that we left equipment on over nights and weekends sometimes (eg a data recovery), even though I had a special "test bench" for those machines (insulated metal box per machine, if it did catch hopefully it would have no chance to spread) I still wanted to be sure that things would fail at 10a.
Any chance you can explain why 20a is a standard? Wiring gagues haven't changed either, what used to be wired to a 10a fuse as standard is now wired to a 20a breaker as standard (or so I am told).
Would love to have someone clearly explain the safety benefits of doubling the current. At 220v, an extra 10a is a hell of a lot of BANG!
I'm not an electrical engineer here, so I could be talking out of my rear, but could it be due to peak load?
For example, I have a 400Watt inverter for running things off a 12V battery. It is rated for a peak load of 800W, the 400W being continuous load.
I would guess that it is that high purely because if it were to blow every time you turned on say, a hairdryer, it would get annoying, fast, and that if it were to short, it would happily try and draw more than 20A causing the breaker to cut power.
I'm not an electrical engineer here, so I could be talking out of my rear, but could it be due to peak load?
That would make sense in some cases - eg at a friends place he and his kids would get home, one would be turning on the jug, one the computer in one bedroom and another the TV in another room, and the breaker that happened to link those 3 points would pop. Work-arounds were 1) move the computer to another socket or 2) one of them do something for a couple of minutes while the breaker cooled down from the start load of the first 2..
That still doesn't cover the concept that 20a is a "mandated minimum", if the electricians I spoke to were getting their standards right. But I have just been doing some "vehicular maintenance" for one of them, apparently "coz I read it on the internet" is a good excuse for putting farking used vegetable oil into his crankcase!1 2. I'd have free electrical repairs for life.. If I was willing to trust him with such high-explosive potential!
1 Not a huge lot of work. Some cleaning, new head gasket, few other gaskets replaced. Cylinders etc check out OK for the "he was more lucky than he could know" factor. Poor thing was seriously cooking. Engine life probably reduced by a significant amount (I'd go as high as saying it'd be lucky to reach 250,000k's) but given how he treats the body (lots of power wash and never any wax) and how he treats other things (clutch makes a good handbrake on a hill y'know, and starting the car in gear is fine - good way to load test your starter motor!), the rest of the car will be lucky to reach 100,000.. I'm sure some of the "dis works fine in ur mota!!!!!11!!!" or "this one weird trick saves you lotsa monies!" youtube vids are from car dealers wanting "repeat trade"..... Yeah sorry mind still in freaked-out-need-to-vent mode..
2 Please note that this particular person isn't a reflection on the intelligence of other electricians. Hell, he isn't a reflection on the intelligence of anything (except maybe purchasers of bulk supplies of orange body paint!)
"Any chance you can explain why 20a is a standard? Wiring gagues haven't changed either, what used to be wired to a 10a fuse as standard is now wired to a 20a breaker as standard (or so I am told)."
I'm not familiar with NZ wiring regs, but choice of wire gauge is related to reaction times of breakers and fuses. If you look at time/current curves for typical fuses you might be surprised at how long they take to blow at just over the rated value. The important thing is that they blow quickly enough in serious over-current conditions to prevent melted insulation or fire.
"Any chance you can explain why 20a is a standard?"
Don't know about NZ (sorry), but in continental Europe fuses can be as small as needed. It's quite common to have 10A fuse for a single Schuko socket or 16A fuse for a set of two sockets. Slow blow, fast blow, whatever you need for your particular application. But wires have to have higher rating than fuses. No skimping on copper.
UK systems with their ring topology are a different kettle of fish. In these it's very important to have right fuses in right places.
If NZ really does have some kind of regulation mandating 20A, you are probably still allowed to use an extension with its own fuse. Perhaps with a ground fault breaker too. Makes every sense for a repair/testing environment.
Don't know about NZ (sorry), but in continental Europe fuses can be as small as needed.
That's how I would like it to be, and how I would expect it as well. Client saying "Look, the power supplies in these things are rated for 10a, if they fault, draw 15a and catch fire they're still below a 20a breaker and still performing above their rated spec, so give me 10a breakers please!" I can't see a problem myself. So if I plug enough devices in that it pops the 10a breaker, then I learn to plug less into a point and also get some exercise resetting the breaker. And yes, I have seen TV's and computer PSU's catch fire, it's not that uncommon. I simply cannot see how a 10a breaker is somehow "more dangerous" than 20a! But that is why the requested fitting was ruled out.
But wires have to have higher rating than fuses. No skimping on copper.
I once caused some surprise in my first week at a factory job, by turning up slightly late and calling out "where the hell are the fire extinguishers". A retro-fitted 'high-stop" light in my mid-80's car had somewhat less copper than it should've. I had a bad run in to work that day where I was on the brake a lot more than normal (had only had the car a little while) and had spent a while at a sloped intersection a few moments before. Suffice to say the insulation started to go just as I was getting in to the car park, all I knew was I could smell burning wire in the cabin. Thankfully nothing serious.
If NZ really does have some kind of regulation mandating 20A, you are probably still allowed to use an extension with its own fuse.
I don't think I've seen an extension with one of those. I have seen multi-boxes (boards with 4 or 8 or whatever sockets) with breakers, but that's not a given. As above, I've had a small electrical near-fire, I use the smallest fuse that will do a reasonable job (allowing for start loads etc). I'd rather replace a fuse than something more expensive.
I had a problem many years ago when a customer logged a support call because their mapping system was printing out maps with odd colour schemes. A site visit revealed that someone had changed the printer cartridges with no regard for matching colour of ink with colour specified in the cartridge slot on the printer.
I used to have a very cheap VPS hosted by a company who have since ceased trading. One day my VPS was acting up, so I raised a support ticket. The reply stated that a disk in the RAID had failed and the host had gone into read-only mode. Not to worry, the disk will soon be replaced and we'll be be back up and running in no time.
The next day the VPS was gone completely. I raised another support ticket and was told that the *wrong disk* had been replaced, losing all data.
Thankfully I take back-ups, but it was still a pain to find another provider and configure the new VPS. The offending provider did offer sincere apologies and month free, but I declined.
the *wrong disk* had been replaced, losing all data.
Ah yes, the "reverse back up from blank drive to full one" problem. Used to happen with removable disk packs. There's a reason those floor-standing drives had a write-protect switch, but that message didn't always get through.
the *wrong disk* had been replaced, losing all data.
Had a local computer shop do this to one of my hard drives. Needless to say I was ready burn the place down. Then they had the gall to insist I still pay for the service.
I was ready kill at that point. I came un-fucking-glued and literally read them the riot act in front of other customers.
No, I did not pay.
No, it was totally unrecoverable. They and I both ran the best data recovery available at the time and it was FUBAR'd but good.
I'll raise you two more ...
We had (past tense) a customer who bought a Mac server from us (yes, that dates it a bit). Their "technical" guy wanted to plug something USB into it - and assumed that the USB connectors were behind those pop out panels on the front. He genuinely couldn't connect having popped out the three drives with the system failing ! Luckily, when I told the RAID controller to add the 2nd disk back in, things seemed to be intact and I left it to rebuild the first disk.
Then we had a customer with a support contract with a large manufacturer. The techie comes out to replace a failed drive, and being unsure which one was the failed one, proceeded to pull each drive in turn ...
This is why you replace one drive in a RAID array after a year, while they are both still working. That way, the drive you just pulled out is a working one with all the right data on it, giving you a second chance. And then when the other one fails, the one from which everything now has to be restored with only one chance to get it right is still a year younger than it.
In my yoof - I worked as an engineer in a local PC hardware shop. We had a bignob sysadmin from a large financial that had regional office near us, call up asking sarcastically if we sold fake crap and tell us that the Adaptec SCSI we sold him was 'manufactured wrong' and wouldn't fit in the system.
So we asked him to bring it back.
So I take the card and test it in the back, seems fine. I tell him this, he says 'I have the server in the car' can we try it. sighing, ok, bring it in.
Lumps the server on the counter, cracks it open and then proceeds to try and plug the PCI SCSI card in upside down, the connector edge pointing skywards.....
I take his hand and physically turn it around and slot the card into the PCI slot.
I quote: "oh! I'm used to ISA slots and the components facing the other way"
I remember that PCI cards always look "backward" compared to ISA cards, but never managed that. There was that flat bit of metal with the bent bit on one end of the card that screwed in the case - backplate? The bent bit went at the top.
No, I am NOT this genius - I'm the one with the hacksaw in the earlier comment
I got a defunct 2U server cheap off a site clearance, planning to break it for parts. It had apparently stopped working in the rack and been taken out to be fixed but no-one could get it to power up so the HDDs were pulled and it was left on the workshop shelf until the annual turf-out of old and broken kit came round.
A couple of minutes under the hood revealed both hot-swap power supplies had been inserted upside down in their docks with sufficient force (quite a lot, really) to bend the PSU mating connector structure inside the server out of the way far enough for the retaining latches at the back of the PSUs to snap into place. For some reason the designers hadn't polarised the PSUs and the docks to prevent this happening in the first place. A minute or so with a rubber mallet bent the mating connector frame back into alignment, I put the PSUs back in the correct way round this time and the server powered up perfectly.
Two tales spring to mind.
Back in the days of 3.5 inch floppies, the metal sleeve had come off the floppy and had lodged in the drive, so the user jammed the floppy back in thinking the sleeve would re-attach, when that didn't work, he took a second floppy and really jammed that on in, thinking that both would then come out, that's when he phoned for help.
At another customer site they had an HP/Compaq EMA12000(?) SAN. For some reason the local IT bod wanted to tidy up the cables at the back, so he pulled them all out and all very neat and tidy re-routed them and went to plug them back in (without taking any notes of which cable plugged into what). Two things struck him as odd, the cables where not always long enough, and the connectors did not seem to match (the old thick SCSI connectors), So with pliers and a hammer he pulled the cable connector to stretch them and hit them home. He was only forced to call for help when after the fourth day the SAN was still down and the business was complaining.
Welllll... the HDD replacing part is just the 35th step. The previous 34 ones are just to get there. And THAT part's even easier than LEGO, nowadays. You just CANNOT force-plug the connectors... Well, you can. I've seen it. Broken bits of plastic and all ("Yeah, some bits fell off... But IT FITTED! This HD IS @$#&*!")...
Aniway, your example is not exactly about a hot-swappable caddy in a server.
Talking about connectors...
I have been building networks for many years, starting with thin wire ethernet and 50 Ohm baluns,
through Cat5, ATM networks with fibre optic cables, through to Infiniband networks with big CX4 connectors, and now high bandwidth Infiniband and Omnipath with SFP+ connectors.
Oh the shame of it... One day I was asked to do a favour in return for beer...
Move a small setup with one Infiniband switch to another room. So we set to, and $NETWORKING_GENIUS (ie me) put everything back together. Switched it on. No blinkenlights.
I ran every diagnostic program I could remember. I rebooted. I power cycled the switch.
Finally $NETWORKING_GENIUS was reduced to calling the supplier of the system.
"Oh" they says "Try putting the connectors in the other way up. they might work that way (you fool)"
Cue much red facedness.
Thin wire Ethernet?
If it's not yellow and 1/2 inch diameter, it's not Ethernet (tm)!
We used to DREAM of connectors, as we hand drilled holes in the cable and carefully inspected them for that one strand of braid wire which would short out the entire network.
And don't get me started on Token Ring...
[Crosses self and spins round three times, throws salt over shoulder]
Customer wanted to fit a CD Burner (circa 1999 & still quite pricey), that had got missed off his order.
"Ship it to me & I'll fit it myself!"
One HP burner shipped out 2 days later.
"This piece of crap doesn't work!"
"Bring it all back in please Sir"
One day & a drive back to us from North Devon, we open it up to find the drive is secured with 1" wood screws not the ones supplied in the box ("I didn't see those\Why didn't you tell me?" "Did you read the enclosed instructions Sir?").
Or as the guy that flipped the 220V PSU switch to 110V "to see what would happen" found out.
Fitted the new PSU & found Windows was totally corrupted.
"I'll reinstall it myself he says"
The HDD is in such a state of shock from having been "Shut down unexpectedly" it won't reinstall.
Another HDD, reinstall Windows & apps & recover his data took about 2 days as he didn't have a backup
My home TV set suddenly started switching off at random times. I figured it was probably heat-related, so took the back off, cleaned it out. Tested it - it worked fine. Sat down to enjoy a program and the darn thing turned off! So pulled the back off again, checked for signs of overheating, re-seated all the connectors. Tested it - it worked fine. Sat down ... and the TV turned off.
It was then I noticed the top of the spare TV remote peeking from between the couch cushions ...
There was a tale, possibly apocryphal, about a System36 or 38 server near Kingsford Smith Airport (SYD) . It would boot, and mysteriously reboot /power down every few seconds. All the blinken lights were working, all parts swapped out, new boxen supplied. ...Until one Engineer was gazing out of the window in thought, noticed a pattern, every time this big mother of a radar dish was pointed at the building - bzzzt.
He shot down to the local supermarket & several rolls of aluminium foil later applied to the window all was well.
He inserted the disks, connected the server and turned it on without even bothering to check that it was actually running? That's the part I don't quite understand to be honest.
But not to worry: I'm sure this guy can always apply for a job with Gitlab as backup controller. I think he fully meets all the requirements.
I recently bought a new satellite decoder for the old folks as the old one didn't let them record from Irish (digital) terrestrial channels. I got rid of the DVD recorder at the same time (which went to the charity shop) since it couldn't decode the terrestrial digital signals. They have a slim Playstation, too (for the grand-kids), so that could handle DVD playback. Or so I thought.
Only, when I went to show them how to watch a DVD, it didn't show up in the PS menu. Cue head-scratching mode. I could hear something spinning, so I went off the the net and started a system update. Came back just to check one more time (cancelling the update) and realised that I hadn't pushed the CD down far enough in the caddy so it wasn't locking into place. After a quick push and click, the new Bridget Jones film got its first (though no doubt it won't be the last) airing.
I'd add doing "rm -rf" on the root directory of a remote server.
Did that when deleting some old auto-generated status emails, ended up running "rm -rf / var/log/mail/" and hit enter before I realized the typo. Had to get the hosting provider to restore an old backup since there was no way I was going to fly 5000 miles to fix it.
I worked on a flash storage array outage where the network admin had turned off the firewall that prevented alerts going out for days ... the array was over committed , and went read-only when it was 110 % full , the ESX 1000 Window VDI's become comatose ; Another storage admin then did numerous power cycles on the array thinking it would magically free space but instead it corrupted data beyond repair .The outage lasted for over a week until they could recover and rebuild 1000 VDI's . ;
Back when I was a skint youth I supplemented my meagre pocket money with doing a few repairs for the local TV shop, which was next door to where my dad worked and meant I could hang about until he finished and get a lift home instead of going to all the extravagance of bus fares.
One rather nice VCR came across my bench, a front-loading one which would accept a tape, start to drop the carriage, make a clicking sound and eject again. Pretty easy, foreign object in the deck, rolled under the tape carriage, unjammed it, cleaned it up, retimed the mechanism which had skipped a couple of teeth causing it to shut down with a cryptic error code instead of the equally cryptic flashing "12:00 12:00 12:00". Pop tape in, whirr click kachunk whirrr, off it goes, press eject, whirr clunk whirr click, out comes the tape. Great. Pop it on the shelf for Mrs Smallchild to pick up the next day.
The next afternoon, in came Mrs Smallchild with her eponymous small child, who promptly began stress-testing all the buttons on any TV within his reach. "There you go, Mrs Smallchild", I breezed, "but you might want to consider putting it in a cabinet or up out of the reach of little hands."
"How very dare you", exclaimed Mrs Smallchild, "Henry knows he's not allowed to touch the telly or the video, and he's as good as gold, he never goes near it when he's watching his videos."
"Well, okay then, but that only leaves the possibility that it is one of the grown-ups in the house that is responsible for feeding the poor tormented machine a piece of toast, half a Milky Way and three or four very chewed Duplo men..."
I still cry a little inside when I think how many tellies I could have got working again with a BU208 and a pattern LOPT, had I dared. But horror stories about my dad's contemporaries blowing themselves to bits with mains-derived EHT had given me a phobia about TVs, so I passed up a bit of easy money for fear of high voltages. It was lost on me that EHT from the line output transformer is inherently energy-limited.
Needless to say, my first proper job in electronics involved gas ignition controls -- complete with live chassis and nasty high voltage.
Fun fact: The ignition spark on a gas boiler contains about as much energy as dropping a penny on your foot from a height of one metre. They say "the dose makes the poison" but in this case, the route of administration also makes a difference .....
This is similar to what I face on a daily basis while providing Server Hardware Warranty Support for an unnamed Server Vendor. Sys Admins call in, always claiming hardware failure. The ones who resist sending Server Diagnostic Logs don't want it discovered that they misconfigured something, causing something to not work. Of course, once we get a tech onsite to replace hardware, it comes to light that the Sys Admin doesn't know how to update Firmware, restore their raid config, or some such Sys Admin duty that they SHOULD be able to perform, if they had the skills, but obviously don't. And don't get me started on the "foreign off shore Sys Admins" who barely speak English and don't have the skills to do their jobs. No one is saving money via outsourcing. It makes those of us in the States have to work three times harder, without making a penny extra for all the extra work we need to do...
I had a cow-orker with a Phd in sociology try to get his computer working by himself. This was in the days before the IBM PC and the machine had an S-100 bus. Turns out that he wanted a "better" graphics card in his fancy machine so he took out the old one and plugged in the new card. And the machine wouldn't boot.
So he took out the card and booted the machine without a display... and then plugged the card in with the power on. Unfortunately the graphics card must have been defective because all the magic smoke that makes electronics work escaped from several of his cards.. including the cpu card.
And then he called my support group for help.
... and are that ignorant to entrust someone who will spend the majority of their time LOOKING FOR THEIR REAL NEXT FULL TIME JOB , instead of taking care of business ... deserves all the shitte they get .
IT cesspool staffing firms have destroyed this business
Many years ago, I was the Ops manager at an offshore software house in Shanghai. We hired a just-out-of-college intern to inventory the systems. He started in the storage room cataloging drives in custom enclosures, bringing a few of them to me so that I could point out where the serial numbers were.
After he was done, he started inventorying the server room - and I started seeing RAID volumes losing disks. After the OS volume on one server went offline crashing it, I ran into the room and saw him with a drive in his hand. He has been pulling drives to read the serial numbers.
I started yelling at him. In the 3 years that I was in Shanghai, the engineers had never heard me lose my temper and they came running from all 3 floors. The intern started crying, so I had to hug him to calm him down. I hope no one took pictures...
Yup I did that to myself and wondered why my system wouldn't boot!
In another rather amusing situation, my brother and I were adding an external SCSI hard drive to his PowerPC Mac. In the process of connecting things up, we created a closed loop system with no connection whatsoever back to the machine. We had the SCSI cable connected on both ends to the hard external drives and wondered why we couldn't see the "new" drive or any of the other drives he had on his desk. We still laugh about this one today.
Someone I know is an indepedant IT contractor who does computer maintainance for various local companies.
One day they discovered one such company had server issues because one of their employers deleted my friends account on the system because they didn't think it was needed anymore - it was the admin account on the system...
Thankfully said friend has some legal hacking tools that allowed them to 'break in' and modify an existing account to give it admin privileges, and then proceeded to remove all dangerous privileges on the account of the employee that fucked things up.
Re: Not a PC
Respectfully mate, actually very dangerous.
Circuit breakers are meant to be inline with their load.
A 5A light shorts, the 30a fuse won't blow before the wires turn into white hot light.
The fuse on any circuit needs to be matched to the load. If the fuse is over rated it will simply not do its job and allow items down circuit to blow.
Eg: my house had the aircon on the light circuit (7.5A) and the lights on the aircon circuit (10A).
Aircon would trip all te time.
If the lights ever went bad, the chance of the wrong fuse doing its job is very low.
Very dangerous situation. I'm a 30 year old, 15m copper cable with too much load and the bite out of my cable from the rat 2 years ago will now cause arcing - didn't before because a fuse would trip before the arcing occurs, not now..
It seems that many people fancy themselves to be electricians. A number of years ago we had some renovations and updates done to the house we had just purchased, and in the process the electrician uncovered what she called some really scary shit.
We had a whole basement, workshop included, not even fused. The previous owner ran the lines for the basement on the "wrong" side of the breaker box, bypassing the circuit breakers. If that wasn't bad enough, we found that he had used lamp zip cord to boot instead of the shielded wire!
Then there was the first floor. The kitchen, parlor and first two side rooms were all wired to a single 5A fuse while there was a single outlet in the kitchen connected to a 20A fuse.
While tracing wires, she came across others not grounded and some left uncapped in the ceiling, and many more ungrounded outlets and an unmounted ceiling fan.
She remarked she was quite amazed the house hadn't burned down before we purchased it!
After doing some research, we found out that the previous owner's son fancied himself as an electrician and had done the work on his dad's house. I hate to think that this idiot is still doing this today!
Biting the hand that feeds IT © 1998–2019