back to article Stay out of my server room!

We always design our data centres and server rooms with the best of intentions. A nice, enclosed space with excellent ventilation and air conditioning, sufficient power points to ensure circuits are never overloaded, and enough space to get at both the front and back sides of the rack – ideally all cordoned off into its own room …

This post has been deleted by its author

We sort of inherited our server room from the previous occupants so it's a lot bigger than it needs to be. This means when the bottles of champagne and advent calendars are stored in there ready for December there's plenty of cupboard space for them near the door. Which is well away from the racking for the spares and build PCs which are also well away from the server racks and switches.

Best thing is, because of who the previous occupants were, the air con unit in the server room is isolated from the rest of the air conditioning. Bit chilly in there right now, but I'm sure it'll be lovely next summer ;)

5
0

About two years ago, I went to visit one of our branch offices. Is not so far, about an hour driving.

That branch does not a have a server room, just a communication rack with a router, switches and telco equipment, plus a couple of UPS.

It has never been a model of a room, because it is simply a converted broom cupboard.

For that reason, the door is never locked. That way, in the hot days of summer, which easily brings the mercury to 35°C (~95F), the door can be fully opened to allow for some air circulation.

I have asked for an AC for the room, but of course the answer is yet to materialize.

On my visit, I could barely open the door.

When finally got it open, I discovered that the room was full, from floor to ceiling, of old newspapers.

Everywhere: on the rack, around the rack, inside the rack... it was impossible to reach switches, connectors, cables.

I was apoplectic.

There was so many risks... communications failure, overheating, fire...

I talked to the office manager and sure enough, the newspapers were promptly removed.

Only to be replaced with the janitor's stuff.

Now, the room is used as closet.

R

17
0
Silver badge

"It has never been a model of a room, because it is simply a converted broom cupboard.

For that reason, the door is never locked."

Hasp and padlock.

Vent grilles on the door, with a suitable fan attached.

And a policy of "anything not authorised to be in here will immediately go into the dumpster"

1
0
dl1

All that nice warm air blowing out the back of the racks is great for drying motorbike leathers.

14
1
Anonymous Coward

I have a similar use... I go swimming several times a week - there is a pool very near the office.

So the server heat combined with dehumidifying effect of aircon, does a grand job of drying my gear.

3
0
Silver badge

I've been thinking of building a biltong box on the back of one of ours...

2
0
Silver badge

MD gets his hand in his pocket but only after a few tricks from IT

At the place I worked at a long time ago we had our servers in the IT office with no AC and just a small window we could open. Summer temps in there were typically anywhere between 30°C and 40°C.

Asked for AC and of course it got rejected by the MD as an unnecessary expense that simply wasn't required.

The IT manager and I decided enough was enough so we came up with a plan to keep rebooting the Exchange box at least twice a day when we knew the MD was in the building. After a week of the MD getting disruption to his email and us telling him the Exchange box was overheating he soon got his hand in his pocket.

27
0
Silver badge
Pint

PAOFH

Passive Aggressive Operator From Hell

17
0
Silver badge
Gimp

Let's see, there's the server closet in which it rains every spring. The servers in there are placed on little chunks of wood to prevent them from sitting directly in pooling water.

There's the server closet which contains a furnace, water heater and hydronic equipment to heat the entire office. Oddly enough the temperature is somewhat reasonable in the summer. In the winter it's probably pushing 40C. With the massive amount of complex and occasionally sketchy piping I'm just waiting for the day when the server, switch and comms are introduced to a torrent of boiling water.

Then there's the server...thing...which consists of a 12 ft by 3 ft area with a shelf mounted on one side at neck height and another mounted, on the opposite side, at gut height. Getting in there requires some interesting contortions. It's also filthy as dust from the attached shop accumulates in thick layers.

Another one has a little broom specifically to get rid of the vermin feces. I recommend against sitting on the chair in that particular closet.

The best one I've seen was a crawl space full of our old friend vermin feces along with fiberglass insulation and broken glass. The server was placed on some old 2x4s to prevent it from resting directly on the dirt floor. Luckily I was only at that place twice. I wouldn't be surprised if they had relocated their server to the bottom of a literal cesspit by now.

12
0
Anonymous Coward

I can top your rain filled cupboard...

Back in a previous life I worked in telecoms. I went to customer site one day to assist another telecoms engineer (from a large international telephone switch manufacturer) to commission some of our gear.

Upon arrival I was taken down to the telecoms room, which was in a basement room. The room was nice and cool... with a stone tile floor... Curious I thought. Then I noticed all the telephone switch gear was raised off the floor by about a foot on stacks of breezeblocks.

I asked the other engineer what was going on... He replied "You see that man hole cover? Sometimes the drainage gets blocked, and the room gets flooded...".

I looked, and indeed, in the middle of the floor was a very large manhole cover.

I said "Oh perfect! I bet that's nice job to clean up".

"Yes" came the reply "Especially with the business they are in here"...

The customer was a meat processing plant!

(Luckily I never "experienced" a flood there).

13
0

Slaughterhouse Blues

Yup that happened (before my time) at the slaughterhouse I worked at - The PFI (Intern) was sent in to clean it out.

0
0
Silver badge

"Another one has a little broom specifically to get rid of the vermin feces. I recommend against sitting on the chair in that particular closet.

The best one I've seen was a crawl space full of our old friend vermin feces along with fiberglass insulation and broken glass. "

Sorry, can't work on that, Environmental health breach.

0
0
Silver badge

In my last job we had a whole ex-office as a server room, which of course meant that not only were we expected to keep all the IT equipment in there (from spare PSUs to laptop bags), but we also had to share with accounts, who kept boxes and boxes of old paperwork in there. Of course, being bean counters they didn't want to pay for shelves, but the cardboard boxes collapsed if stacked more than three high, so their collection of fire hazards slowly grew to take up half the floor plan.

At least I had one of the few keys to the room. On quiet days when the boss was away I could let myself in and catch up on kip in the warm spot behind the racks, knowing that nobody else could get in :)

2
0
Bronze badge

Fires and floods...

Worked at one national airline that stored old CP listings in a network room. They caught fire, and the fire brigade couldn't get through the mantrap...

Worked at another where there were two datacentre rooms connected by a very thick cable strung between buildings. The monkeys loved the cable. One monsoon season, the roof of the backup centre leaked a little. Water and mainframe do not mix well... One of the plant rooms was so cold everyone condensed on entry and exit during the buildup to monsoon. The UPS was a series of truck engines with their exhausts sticking out the wall. They belched huge fumes of black smoke when they started up, so you'd always walk a little quicker past that section of wall! Happy days :-)

8
0
Anonymous Coward

Never have a window

Lost our nice and big computer room and was made to stick it in our store room.

The old computer room in now a meeting room and because we can't store paper of cardboard nowhere to store brand new equipment.

0
0
Silver badge

In an early job, it was decided that the MicroVAXen would live in a cupboard behind the reception area.

A sparky from HQ was duly sent down to sort out "aircon" for the cupboard.

He fitted one of those little extractor fans people use in toilets, with a thermostatic control.

Unfortunately, he wired it all up so the fan stayed ON until a certain temperature was exceeded.

One unusually warm Easter weekend later, the whole cluster got cooked.

7
0

We had a similar situation. Luckily, I noticed before any kit was installed, and was believed when I pointed out how dumb this was to my PHB.

1
0
Silver badge

Has your precious data centre been turned into storage? Not exactly "mine" but yes, that happened in the past. It was especially nice with all the combustible material stored that in case of fire would render the whole extinguishing system useless. (That wasn't just an ordinary server room but a data centre of one of the 30 FSB G-SIBs).

More recently, and on a much smaller scale, it was the storage room to which a server rack was added. And why installing an aircon when we simply can remove the door and walls of the rack to allow for free air circulation?

1
0
Anonymous Coward

Did you beat back the barbarians at the gates?

no, we failed totally. A quiet phone call to building security later, whenever Chief Financial Barbarian's deputy demanded access to offsite server room (ok, it was space in a colo) the building security would not let them in without a ticket / reference number, which they could only raise if they were on the authorised list (they weren't). The CFO/B went on the list, after being given a piece of paper that said if you let anyone in, from that moment until one of the IT guys has signed saying they didn't break anything, all fault reports go to Finance. :-)

strangely, no-one ever asked ..

7
0
Anonymous Coward

Re: Did you beat back the barbarians at the gates?

And IT couldn't be overridden by the rest of the Executives? Because IT is rarely on the board, who usually CAN override.

0
0

Re: Did you beat back the barbarians at the gates?

No, it's a co-located server, so the physical infrastructure is under the control of the co-lo company (though you may or may not own the actual computing hardware) so access and the like is down to them and their security team. And so out of direct control by your boss.

0
0
Silver badge
Facepalm

The computer room was kindof okay

Some twenty racks, with assorted servers, disk shelves, tape drives and comms stuff, all reasonably orderly. Not at the 'neat' level, but I've seen worse.

The problem was with the aircon. One of the two units had a leak, and they had already used up their quotum of CFC-based coolant so it was out of action, with the other running flat out. It had to: the room was in a 'temporary' wooden single storey building, knocked up in the 1950's or maybe early 1960's (this was 2003, so ...) with a low, tarpaper-covered roof. Temps in the adjoining offices could easily reach 35C on a moderately sunny day. To increase aircon performance they had installed a pair of garden sprinklers underneath the heat exchanger, which fortunately happened to be mostly in the shadow for a large part of the day. Around 10 o'clock someone would have to open the tap, and the last person to leave had to turn it off again. On warmer days this required staying overtime, occasionally until 20:00. Even hotter days required the back door to be opened and half a dozen floor-standing fans adding a feeble breeze to the existing airflow.

My suggestion to visit the nearest hardware store and drop a couple of buckets of white paint on the relevant section of roof, or, even better, deploy a few rolls of reflective bubblewrap foil, was dismissed with "this is a temporary building". Which was a) actually the root cause of the problem, and b) had been its status for the past four decades.

Yes, this was a (semi-) government department

12
0
Silver badge

Re: The computer room was kindof okay

Let me guess. The way this government department's budget worked, it didn't pay for water or for electricity, but would have had to pay for the paint or reflective foil?

13
0
Silver badge
Facepalm

Re: The computer room was kindof okay

I suppose so, and there's that OPEX vs CAPEX thing as well.

Mind, they could also have bought one of those lawn sprinkler timer valves which would have paid for itself in one single day, by letting the Tap Controller go home two hours sooner, but see above.

0
0

Too many problems to count. I'm responsible for numerous sites around my Province of Ontario (Canada).

Location 1: Single rack with network gear, a DVR, UPS, and ancillary equipment. The room also houses the demarc and the phone system. By company fiat ONLY IT equipment is to be in this room.

But...

The site has put a drinks cooler in there, propped the door open 24x7, and is now using it to store janitorial supplies and tools.

Location 2: Same as above for the IT gear, plus a small print server.

But...

The site has removed all the venting from the door, resulting in NO air flow, and the night cleaning contractors put their half full mop bucket in there the other night. Hello humidity alarms at 3am...

Location 3: The MDF has a couple of servers, 3 separate network racks, the PA system, the phone system and acts as the demarc for two buildings.

But...

The site has decided to store sales paraphernalia, including sweaters, t-shirts, golf bags, posters, flat packs of floor displays and everything else they can think of in there. That's in addition to all of the PCs that I have recently lifecycled that they are requiring "be stored in a secured area" before I (literally) chuck them in a big wooden crate and the company we contract with comes and takes them away...in 6 months.

There is no longer any such thing as sacrosanct when it comes to IT gear. It used to be considered inviolate, sacred, and feared.

Now it's in the way...until it goes down.

Fun times...

6
0
Silver badge

"By company fiat ONLY IT equipment is to be in this room."

That's your authorisation to transfer the non IT stuff to a dumpster.

0
0
Anonymous Coward

Piggin 'eck

I once walked into my carefully and lovingly crafted server room, to find a pigs leg in a box on the floor. It was a huge Jamón ibérico, which cost around £500. The company had been sent it as a Christmas gift from a supplier. The leg couldn't fit in the fridge, so some bright spark had the idea that the server room was so cold, it could be used as a giant fridge.

The crazy thing was that the company was owned by a jewish family, and most of the people there were jewish friends and family, so the thing never got eaten. It was surreal, the intended recipients couldn't / wouldn't eat it, but because of it's value it wasn't simply given to whoever in the office could have divvied it up and taken it home. It just sat there, in my server room on the floor for weeks until it had to be thrown out.

9
0

Re: Piggin 'eck

haha I just posted something similar, except mine was 12 x 1kg containers of Beef mince! smh

0
0

Your room has been repurposed...

A section of the raised floor is walled off so the IT department can have a decent lab (everything is tested in the lab before going into production. Everything.). Comes time to move in and we find the room full of boxes. It appears that the designated storage room (on a slab, shares office aircon) was deemed too small, so they took over the next largest space. The lab was built in the former storage area. After some heat related equipment failures, they finally relented and installed aircon. And overhead wiring trays and power runs. Everything the new storage room has in fact!

0
0

Re: Your room has been repurposed...

That's great! When they fill up the storage room and need more space, there's an ideal spot with identical environmental conditions waiting to be used.

1
0

I don't run to a server room (when we moved in to the current office I specified a minimum size, fortunately somewhat larger than I really needed as what I got is 50% of that). I do have a server "cupboard" with a 7KVA air conditioner and, in case they're tempted to use it as a fridge, a Chubb safe door! There are only two keys, one in my pocket and one in the HR department who have strict instructions to only use it with my authorisation.

3
0
Anonymous Coward

Server Room, we should be so lucky!

At a Russell Group university our server room has had all the servers removed and is going to be a big open plan office for all the IT staff.

5
0
Anonymous Coward

Worked for a small company as their IT guy a few years ago. They decided to expand from a top floor storage unit to an entire warehouse. I was promised a shiny new server room, with air conditioning and enough space to store all my kit.

I dutifully went down a few weeks before opening as the office was being put together, measured and installed all the cabling, wired in the UPS and set our single server rack up.

All that would be left was a Friday evening shutdown and journey 5 miles down the road.

The boss was conspicuously absent all day. As I arrived I found him trying to unclip all the wiring and reroute it to a stationary cupboard and my racks thrown out because they wouldn't fit in said cupboard.

His wife had decided she wanted to have a job and needed an office for all 8 hours a week she worked. I was very angry. and refused to rewire or help. Quit soon after.

15
0
Silver badge
Joke

> His wife had decided she wanted to have a job and needed an office for all 8 hours a week she worked. I was very angry. and refused to rewire or help. Quit soon after.

You should have calmly offered to let her work on the help desk. :-)

6
0
Silver badge
Childcatcher

Quit soon after.

The boss was conspicuously absent all day. As I arrived I found him trying to unclip all the wiring and reroute it to a stationary cupboard and my racks thrown out because they wouldn't fit in said cupboard.... I ... refused to rewire or help. Quit soon after.

This! This is how it's done! I have had to do this myself. Either the people you work for and with will back you and value the service you provide or they are going to continue to make whatever you do worthless. Much better to find something worthwhile to do than to stick around in a messed up situation that you know is going to stay that way.

11
0
Thumb Up

Well put

"Either the people you work for and with will back you and value the service you provide or they are going to continue to make whatever you do worthless."

You put it into perspective nicely!

I've found myself in the latter situation too often, and wondered how come I got demotivated, stopped producing anything meaningful, and felt generally worthless despite normally being reasonably capable, productive enough to pass general muster, and capable to be cheerfully cynical.

I've always written it off to lack of inspiration in unimaginative environments, but the notion of value seems much more relevant.

4
0
Anonymous Coward

Re: Well put

Trouble is, with the job market as bad is it is, sometimes you're against the wall. You either put up with it or can no longer support oneself because there are no other opening (all the good ones already taken).

1
0
Anonymous Coward

Here's what my company does...

They have a big impressive Server Room full of blinking lights and twirling fans.

But then they run the whole company off of an under-clocked Raspberry Pi.

At least that's what it seems... LOL.

1
1
Silver badge
Gimp

@header pic

Whomever chose that header image for the article has ...... a kink or three to work out.

<appropriate icon?>

3
0

Paper trails are your friends.

I think that it's important to remember that part of our jobs in IT is risk management. This is part of the lingo that most upper management speaks and understands, and so if you can communicate these things in their language you'll have an easier time getting your point across. The other thing is to NEVER have risk management discussions verbally, or if you do be sure to IMMEDIATELY follow up with a memo to everyone involved outlining everything that was discussed. Be fair about it and give everyone's position a respectful treatment (regardless of how stupid it may be), or it will backfire. This sort of thing is just as important as running proper backups. This way when poor compromises are made about physical access (or a myriad of other issues) and it eventually comes back to bite everyone, you have clear documentation of the decision-making process. If done properly these sorts of disasters wind up enhancing your position rather than leaving you as a scapegoat.

It's important to remember that no matter how smart you are technically, your position will always be limited if you can't interface your ideas with other departments. Like most areas of contention, you will be most effective if you are capable of respectfully making the other sides' case better than they can (even if it's dead wrong). Once you can do that, you will be better able to persuade them of your position. Think of it as a hacking challenge or something like that.

19
0
Silver badge

Re: Paper trails are your friends.

Just remember that a paper trail should include real paper for the case of utter disaster. Whenever I was developing rules and procedures, I'd have a folder with all the inputs for the ultimate question :Who fucked up? Why?

3
0
Anonymous Coward

Re: Paper trails are your friends.

So what happens when that room you keep the paper trails in has a FIRE? And given the environment, odds are passing fair any fire there would beat a fire safe.

0
0
Silver badge
Windows

storage of IT equipment.

Functional or not as the case may be.

Most of the IT equipment I deal with lives in proper DC space.

Every once in a while something in a network/comms closet gets thrown my way.

Things I've found in said network/comms closets in my travels:

Bicycles (one closet had four. One of which was in numerous parts, including what looked like having had part of the frame cut using a hacksaw, the metal shavings from said operation still lying on the 'raised floor' tiles in the room)

Old TVs/Display panels (usually stood up long ways against the backs of the cabinets.)

Bundles of fibreglass insulation, pink, yellow and green. In one case, neatly taped into packs of three slabs, and stacked in the space between the back of the racks and the wall. Tightly packed of course.

Cases of empty beer bottles. (in that one, 248 two-fours, stacked against the aircon unit, blocking airflow *into* the unit ....)

dead PCs. from a cleanout of a callcenter as far as i could tell. I made it to 150 before I stopped counting.

Broken toilets (just sitting in the middle of the room, chunks of shattered ceramic)

Unused desks, unused chairs. A very large, sloppily decorated plywood box. As far as I could tell it was a magicians "cut someone in half" box.

An 8 cylinder diesel engine, and I'm guessing the clutch to match it.

7 years worth of tax filing documents for a subsidiary of a company that my company bought. Each year was in a several file folders. The file folders were laid flat in a rack, and stacked on top of one another. It appeared that the computer rack was dedicated file storage. (there's a joke in there somewhere)

4 (4'x4'x4') cardboard boxes full of nothing but those stupid styrofoam peanuts.

Once, an old old OLD freezer. Locked. Rusty. most definitely was NOT empty.

in one case in excess of 80 bottles of photography chemicals, loosely arrayed on shelves in a rack over the PDUs that fed the equipment in the next 2 racks.

I try not to go to weird locations to recover hardware any more. It has left me with various nightmares.

Oh- and the birds nest. There were eggs in it. It was rather dusty so I doubt that the eggs were likely to hatch, but it also made me wonder where the parent(s) were.

I know someone that had the 'farm computer' experience. At least I'm not dealing with *that* level of ........

11
0
Silver badge

Re: storage of IT equipment.

"Once, an old old OLD freezer. Locked. Rusty. most definitely was NOT empty."

My bad. No carpet in the van that day.

15
0

Re: storage of IT equipment.

A friend of mine was tasked with disposing of an old fridge at a very prestigious university frequented by the higher echelons of Parliament. He took one look inside and immediately refused to have anything else to do with it. They had to get a hazardous waste team in for it. Turns out the fridge still contained samples of Anthrax that had been forgotten about.

1
0

Serfver cupboard

A photography company I worked in, early 2000's, went from analogue (E6 line in the basement) to digital, and suddenly we required proper server, storage and back ups. Small outfit, with a very hands on approach. Researched and ordered server, raid, external backups (we were new to this, and on a tightish budget). Installed said on shelf in cool, dry basement in a part of the office (the noise was fine - the music from the studios drowned out the drone of the fans and drives).

Went away one fine easter bankholliday, safe in the knowledge that everything would tick over in my absence. Came back a few days later to find the server and drives no longer visible. It being early morning, the 'togs were not in, and the music off. The boss - something of a tidy fairy - had taken all the ugly computer kit, and placed it in an under the counter cupboard. I located it by a combination of the sounds of the fans screaming, the drives clicking and screaming, and the worksurface temperature of being in the too hot to touch range.

8
0
Anonymous Coward

Re: Serfver cupboard

Did you leave it in-situ, screaming as before, then ask the perpetrator to put their hands on that work surface?

Sometimes it takes a practical lesson to explain why something is wrong.

5
0
Anonymous Coward

Re: Serfver cupboard

And sometimes you find practical lessons are lost on some people: like someone who routinely cooks and is used to hot surfaces.

0
0
Silver badge

The best I've found

was back in the days when electro-mechanical PABXs were the standard - in this specific case, what was known as a BPO* 100-type.

My off-sider and I went into the PABX room at a small business and found the manager's outboard motor leaning against a wall, complete with a partially full tank and a general aroma of petrol. Cue very prompt removal of motor and petrol followed by ventilation of the room.**

We prevented this from happening again by removing the all external and internal covers from the PABX (especially and particularly the 'pulse set') , taking the manager into the room and letting him watch calls being made - with the lights out, so he could see the sparks more easily.

* British Post Office, for all you young'uns.

** This was many years ago - nowdays we'd have to have evacuated half the block and called the Fire Service, Police and $deity knows who else to deal with it, as well as filing sundry H&S reports.

10
0

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Forums

Biting the hand that feeds IT © 1998–2018