Snakes on a backplane: Server-room cabling horrors

I know one data centre, in Kenya

Where real, actual poisonous snakes can be a problem.

Anon, to protect my customer (a good policer of snakes, in a rough environment).

'The more computers you have, the more cables you need'

To you sir, I say this; 10base2.

More cables doesn't sound so bad now does it.

Worst cabling mess I've ever seen ...

was actually not LAN (yes, LAN was terrible, but a minor problem in this case).

No, it was the bloody SAN FC fibers flying like bloody Christmas decorum all over the DC space !

SAN ! Attached fibers going in everywhere at mid distance betwenn floor and ceiling, attached as a temporary solution, to any pipe, power bar or any mounting, rack etc ...

Reason was my predecessor had rushed under massive mgmt pressure the moving of the DC before any fiber end points were ready.

It was a real joke to move to proper cabling, after this. Some fibers were so damaged it was a wonder the SAN was at all working.

Co-inky dink.

I'm doing a small DC uplift at the moment (just 60 racks). Replacing under-floor cable non-management with lurvly new in rack structured cabling (fibre and copper). Makes a massive difference. New server goes in? Just patch to the sides of the rack, rather than running new cables under the floor.

Once it's all migrated to the new racks, we can rip out all the old crap and there is tonnes of it. I would take pics, but it's a bit of a no-no.

In my current workplace, within the first year, I found:

An entire office ran from underfloor Ethernet modules, which cabled back to a 8-port switch, which was hidden in the radiator covering, powered by a hidden plug under the floor, cabled 124m around the building to a proper switch cabinet 10 meters away.

A main server-room rack cabled in, out, behind, through, between, over and around itself such that it was impossible to insert anything else into the rack (literally could not push thing in because of the criss-crossing cables half-way back), with patch and external cables literally terminated to the inch to get them into their patch panel and no more, with fibres dragging out of the bottom of the rack cabinet and on the floor such that when we went to pull the rack forward to work on it - on the night we decided to recable the entire thing - we very nearly sliced right through 18 fibres connecting the server-room to the rest of the building by trundling over them with the rack they were housed in.

That's not counting the mysterious four-feet gap in patched cables that our access control guys found. Literally, in the middle of the main building, through a huge piece of cable ducting with 100+ cables going through it, a bunch of unlabelled cables had been deliberately sliced IN THE MIDDLE and pulled apart. The other ends, no idea which they are - they're probably still patched in "just in case" somewhere. Why you'd do that, in the middle of the cable run, rather than at one end I have no idea.

If you think that is bad....

...you don't remember the bad-old days of RS232, co-ax and twin-ax terminal cables.

I wish I had taken a picture of it, but when an IBM building was decommissioned and we left it in the early '90s, I had a chance to look around as I had a access to help move the kit I was responsible for.

I found the comms room that contained the 3270 comms controllers for about half of the desks in the building. I kid you not, the 3174s, which were supposed to be floor standing devices, were on shelves stacked three high, with about 2 feet deep of co-ax cable cascading from them to the ground, and then to the patch panels and on to the risers to be distributed around the building. It was far, far worse than the pictures in the article.

Like another poster said, the reason for the cables, some of which were clearly not plugged in, was that they were so laced into the mish-mash that they could not be removed without risking breaking some of them, so they just left them there.

Good cable management is possible, and in the long run will save both time and money. It's a shame that most projects do not have ongoing supportability as one of their deliverables.

The poor geek you send to

"The poor geek you send to figure out why the server has dropped off the network risks tripping over something"

Haven't tripped over anything but in my youth I have got stuck behind a comms cabinet when I tried to get round the back in a tiny room.

I had to get inside a comms cabinet once. The cables were everywhere; front, top, sides and even through the middle. I had to wriggle underneath the port panels and in behind to detangle some leads. Unfortunately, there was a 4-way on the floor (hidden under more cables) and I stepped on it, killing power to all the switches in the room.

Days gone by...

Ah, for the days of Cat-5 when all systems were linked with a single co-axial cable!

Ahah!

So it IS a series of tubes!

Network admins?

But I thought the CEO of CISCO said we won't need network/system admins any more? I guess I missed his solution for self cabling networks.

You know when it's a real mess...

... when you take a power saw to the rack in order to re-cable it.

Which do you trust more, software or cables?

"software-defined networking, which has the potential to help us deal with these nightmares"

And it will deal with the nightmare of cable by replacing some of it with software? I've never had a cable replace itself with an updated version of itself that didn't work the same way, so colour me sceptical...

Once saw a rack that had fallen over face first due to the weight of the rat's nest protruding too far out from the front of it. I think the rat's nest cushioned the fall as everything was still working.

Wading through cables

When I joined a large charity some 13 years ago, the server room patch panels had cables that spilled out all over the floor. When I finally managed to get someone in to tidy it up and replace the network switches - a weekend job, they decided to start on Thursday morning identifying each cable. Friday lunchtime came and went, at which point the cutting began.

In my current job, I am slowly replacing the mess with lovely 20cm cables to join switch to panel. My cabinets are as neat as I can make them.

Manchester Uni

http://www.gamecommunity.co.uk/unimess/1427835037.jpg

http://www.gamecommunity.co.uk/unimess/1427835068.jpg

Unfortunately I was never allowed to take pictures, but imagine this:

Beige cabinet, 4 feet tall, 2 feet deep, 6 feet wide, open ended, and on top of a raised floor.

Now, put 6 of those cabinets together to make a 36 foot long super cabinet.

Now, imagine it STUFFED full of fiber optic cables, so much so that you could barely close the doors (actually 2 of the doors wouldn't close, they used a patch cord to loop to keep it as closed as they could), you couldn't see the back of inside of the cabinet, all of them so tightly packed, there was no slack in any of them, with no rhyme or reason, all of them UN-labeled.

The company (a bank) went looking for some vendor to outsource cleaning up the cabinet, label, re-route, and generally make it look good. Every vendor that walked in, took one look at it (begged to take pictures, but was denied) promptly laughed and walked out. The company literally begged for quotes, all of them said "No. We're afraid even if we write down an astronomical fee, you would take the offer."

Needless to say, it was decided that it would be better to just start over and slowly abandon the fiber cabinet from hell.

Sorry...

Before showing me in to their data centres, it's very common for customers to apologise for the state of them. Almost invariably this means there's an undisposed of cardboard box or two in the corner or maybe a single console cable draped between a rack and a laptop on a table. The one data centre that I did go into that you would think did deserve an apology was one the customer seemed almost proud of. That was the one where to get between the rows of racks you had to step over the foot thick bundle of Ethernet cables on the floor at the same time as ducking under the similar sized bundle dangling about 4 feet off the ground.

The server room test

The number of random cuts to cables before loss occurs.

The fewer the cuts, the tidier the setup.

"Unfortunately, software-defined storage does not act as a comb for untangling knots."

In my cabling days, we used to call disentangling the Cat5 spaghetti before pulling into the rack "combing the old lady's hair"

It's not just datacomms ...

There's a long series of "crazy cabling" pictures over at DarkRoasted Blend

http://www.darkroastedblend.com/2008/03/disturbing-wiring-part-3.html

(parts 1-2 and 4-7 equally worth a look, all linked).

Datacomms, POTS, mains wiring, plumbing. chemicals plants ... entropy wins every time.

re: ALL the above

...and you try telling that to the young people of today - they won't believe you :)

(copyright Messrs Python & Co inc)

Cabling your comms room correctly the first time !

We posted on the same subject a few years ago.. and it still goes on, the countless business we see in such a mess, love to share our imagery with you and your readers

http://www.speedster-it.com/cabling-done-correctly-first-time/

I remember one company I was working for... They Kept hounding me, to spend a weekend there to "re cable" all the network racks and routers to make them all neat...... HOWEVer. While showing me how their help desk worked, any time they had a client pc that dropped off the network or lost connection. Their "resolution" was for the help desk person to go into the patch panel, unplug their network connection from the switch, and plug it into a different port, then have them tell the end user to "reboot" their PC.... Needless to say, I never did re-cable their patch panels. And if they could not understand the reason for that, shame on them!

Cable and Job disaster - nearly!

One system operator nearly lost his job due to such a nightmare - by being thoughtful and diligent. Under the false floor in a particular systems room ran a plethora of RS-232, 10-base-5, r.f. telemetry coax, and proprietary disk interface (yes, disk cabinets), together with untold kilometres of used/unused old cables. An operator was on duty for that room 24/7.

Whenever this one fellow was on night duty there would often be unexplainable major disk failures that would bring one or more of the systems down. Only when HE was on duty, not when some other operator had that lovely quiet pastime with the odd beer or two. Clearly it was thought that this poor fellow was doing something he shouldn't, perhaps purposely to add some excitement to his time.

As it turned out after a long study of his work it WAS due to something he did...

One night accompanied by a colleague, nothing went wrong. Next time he was on his own, damn, down went the disk and a system again. So he was of course called in for a 'disciplinary talk' and to account for his every action during his night shifts: he explained that he was very consciencious, took everything very seriously, why, he even went around the room at intervals to make sure there were no important error indicators showing. And no, he didn't bring beer or suchlike into the room. Following night he demonstrated exactly what he would do right through a shift - under supervision. There wasn't any 'chat' with the supervisor like there had been with his colleague, and after an hour or so he went walkabout around the room to check for any error indicators - and THERE! A disk went offline and brought a system crash! All without doing anything actually "wrong". The following day all was examined around where he walked, and finally the edge of a disk interface cable was found trapped between the corner of a floor tile and its support, as soon as any pressure was applied to that tile it would short out a couple of wires within that cable, bringing the disk down. So he was eventually exhonorated, and kept his job, after spending a month or so under the threat of suspension or worse.

He was very lucky since he already had a reputation for the occasional accident, e.g. you'd avoid the corridoor if he was walking along it carrying a tray full of coffee from the machine!

CCTV

I have seen CCTV installations at casinos that would rank with the worst of this. Nowadays it's common to use baluns and run video over twisted pair, which does a lot to de-bulk the cable, but back in the day you could easily have a bundle of sixty RG-59/U coax cables going into a rack of VCRs. RG-59/U is a "miniature" coax but sixty of them makes for a very stiff, thick bundle. Or a big mess on the floor if no one bundled them up.

That's not to mention the sixty DC power supply outputs needed to power those cameras, each of which with an unlabled fuse. Sometimes I would resort to deliberately blowing the fuse to trace power wiring.

Your old data center was disgusting. I don't have any pics of mine, which I designed in 2000 when the company moved in as I'm now retired. However, I will say mine never looked like that! When we built-out the room, I worked with the cabling company and the builders to ensure the wires were neatly wrapped and bundled and all terminated properly into panels. Any connections from the switches was done via a panel and not directly to any computers. There were two exceptions to this and they were related to our PBX system.

I hate to say it, but I have a feeling your new room will look the same in a few years.

Loved reading through this topic and the links but not one mention of the wankers that installed the tray or pipework between buildings.

Tray with botched corners and cuts with no regard for the installation engineers fingers, none of the cuts were de burred in any sense of the meaning and like razor blades having been cut with an angle grinder.

5" pipe is plenty big enough for 24 cat 5 and 4 fibre cables but a bottle of silicone lubricant and the works van were needed to pull the group through.

Bloody idiots used 90 degree bends instead of two 45's a contractor informed me, I don't know what bandwidth the fibre attained but I suspect very poor.

FYI I'm neat if I have the time/budget to do the job properly!

Try re cabling hospitals, steam pipes not lagged, open drains in places and vermin so when crawling on all fours to reach an old part of the hospital was not nice at all.

Ian.