Eight things people forget when buying infrastructure

Racks come in more than one size

More importantly. Racks come in more than one sharpness.

The biggest and most important rack differentiator in a heterogenous install is the amount of meat and blood sacrificed when adding/removing equipment. If you are dealing with a cloud install it does not matter as you just pile them and remove the dead bodies when they kick the bucket.

You do not regularly unplug, service, add hardware, remove hardware, etc. In a mixed install you have to do it, so you learn to value racks without very sharp edges very quickly.

Number 9:DIY

from the command line.

The more pretty GUI's and layers of abstraction from what is really going on with the underlying OS means that one can employ those who have been taught how to pass an exam to administer ones infrastructure for a nominal fee. Experience is not cheap and one has to cut costs.

This does present a much broader attack surface to those that wish to own your ass though. And when your ass is owned, those cheap administrators one has hired prove to be no use at all.

Make sure the MM does what you think it does!

Some management modules come with everything enabled, some require licenses for full functionality and others are basically crippleware to make you buy licenses when you realise they do nothing.

Also, some of them are a complete pain to install licenses on, so if you need advanced functionality, either pick a brand that includes all the functionality you need without licensing, or get the licenses pre-installed by the vendor.

you lost me at Buy the management module, there is nothing in a management module that can't be done from ssh and the shell. if you want redundancy have two machines in one location then use them to "hit off" against each other, it's not difficult, with bios settings you won't be having problems.

I can put six of these in my 42U rack

I tried this with 4x IBM blade chassis in 2007, the colo provider said it would be fine.. till it started deawing something like 8kw of power, they then very swiftly backtracked and I had to take another rack.. thankfully, it was the next door cab and i just fed the power through..

I love blades, but you pay a ridiculous premium for them... after this we went back to pizza boxes, horrible amount of cabling, but when we investigated the cost difference, it was an extra 40-60%.

Interesting article

Very interesting comment section.

A number of dyed-in-the-wool network admins lurking about, I'd wager.

Good show, chaps ! I certainly learned something here.

Yeah, and don't buy IBM/Lenovo servers

Unless you enjoying kicking dead whales down the beach.

label everything as you go as well, saves us poor on call engineers workimg out whats what at silly oclock.

Oh and that cheap KVM, whats the frecking key presses to change screens???

Diverse Entry

We went to a lot of effort for this.. power and data available on both sides of the building. Building Service "helped" by routing the feeds from both boxes through one entry port "to save some money"... yeah.. the landscaping company decided to replace a tree. Down for 2 days. IT took the flak until someone produced an email with BS demanding control of the entry point for "cost effectiveness".

On the other hand, when we built out, we saved money by not routing all power and data via the ceiling but under a raised floor. The cost savings from IT for cabling paid for the floor for BS..... and there was still money in the budget left over.

Corporates can be a real nightmare of turf wars....

Back to Basics

The writer obviously moves in better circles that I do. From past experience, the following still needs to be pointed out far too often:

1. A cloak room is not a server room, even if you put servers inside.

2. You cannot power 100A worth of equipment from a 16A wall socket. Not even if there are 2 of them.

3. You cannot cool the above by opening a window and puting two fans in front of the computers. A domestic AirCon unit won't do much good either. Also, don't put drippy things above sparky things.

4. Ground lines are not for decoration. They need to be used and tested regularly for safety.

5. DIY plugs cannot be wired any which way. Not even in countries that allow Line and Neutral to be swaped.

6. The circuit breakers at the end of your circuits are part of your installation.

7. You cannot protect a rack of equipment with a UPS from PC World. If you really need this, you're going to have to buy something which is very big, expensive and very very heavy. And the battaries are only good for 3 to 5 years.

8. Buildings have structural limits. You cannot put several tonnes of densely packed metal just anywere. Know your point and rolling loads and then check the building.

9. Electrical fires are nasty. Chemical fires are worse. You need stuff to protect the installation.

10. If you want 24h service, you'll need 24h staffing. A guy that "does computers for us" won't do.

11. A 1 Gbps uplink cannot feed 48 non-blocking 1 Gbps ports.

12. Metered and managed PDUs will save you bacon one day. Buy them.

13. Label all the cables BEFORE installing them. (No, you cannot just "follow the cable" afterwards)

14. My favourite: Don't blow your entire equipment grant on computers. All the stuff avove costs money.

And that's just off the top of mehead.

" there is nothing in a management module that can't be done from ssh and the shell."

You can power on the machine remotely. You can go into the BIOS or EFI if there's anything that needs adjusting. If the system doesn't boot, you can access grub and single user mode. Even if a system's working, if it takes a bit longer than you'd expect to boot to an ssh-able state, it may be informative to watch it boot (although there should be reasonable logs in /var/log too.)

I must admit to having never used a system with an integrated management module, but I can see how they'd be quite useful.

Don't be too fastidious

There is a middle ground between the horrors of a data centre with type 1 token ring cables everywhere, and the most perfectly crimped accurate to the inch cabling.

We wandered into a comms room with a server feeding a few dozen modems - except for the fact they didn't work. Upon closer examination it was found the cables between the modems and phone socket were routed within an inch of their life - it was a thing of beauty. Unfortunately the cables are quite fragile and don't react well to being bent. So, go through and test each modem individually, then try and find new cables (they all look the same, but aren't).

Likewise, what goes in will some day come out. There are some installations that are a joy to work on - it's racked, and undoing the bolts lets the unit slide out as smooth as silk. On others, it's underneath three other heavy systems, and superglued to the bottom..

Only 9 pins on the Canon D socket? 25 pins are much more fun. LA120 and "Secondary Request to Send" anyone?