An investigation has been launched at Leeds' famous St James' hospital after a server room disastrously overheated, permanently frying a new computer system for storing patient x-rays. St James, known as Jimmy's locally, is run by Leeds Teaching Hospitals NHS Trust. It confirmed that early assessments indicate damage has …
Re: Poor planning/design?
Errrr, I'm not sure you have got to grips with how AC works, but its not "pushing" air in through those lagged copper pipes! Its cooling air that stays in the room by circulating it, no pressure build up! The pipes contain the heat laden refrigerant, literally transporting just the heat out of the room.
Are you one of those people (usually women, sorry for the sexism, it's just an observation!) that says "Oooh it's hot in here, turn on the AC and open a window?"
AC Works best in a sealed environment. Now if you were talking about fans extracting air, then you need a neutral pressure inlet that allows air to flow in at the rate it's pulled out to avoid a negative pressure build up as the fan tries to suck all the air from the room.
While I don't work for the NHS *directly* my employer has a pretty close connection with them and all their policies, and from first hand experience I can honestly say the people making the decisions and running the IT are nothing more than cowboys with little/no solid real world experience in the massive projects they're expected to lead. If they're not technical wannabe's they're doctors/professors/business graduates with little or no experience (anyone can learn a script and repeat it parrot-fashion... Head of IT demonstrated this skill very well in a recent meeting when I fired a few baited questions to test his knowledge. Neither he, nor my boss caught on while two equally knowledge colleagues stifled laughter for several minutes - whiteboard diagrams, buzzwords, corpspeak... you name it, he did it!).
Better still, these cowboys are regularly unavailable for meetings etc as they're away on training courses or arguing with suppliers/consultants in a vain attempt to do things their way. There will be changes to the IT structure here in the next week or so and I can only begin imagine the even bigger influx of disgruntled users who can't access email because their region has to share a 1 or 2Mb ADSL line with anything between 50-100+ user while the The Ranch (ie, head office where all the cowboys are) play with their overspecced & overbudget custom built PCs and 30-40Mb+ lease line. One of the few clued up guys has handed in his notice today for most of the reason above, and there are other people thinking about going the same way.
Makes me sick to think my tax £££ are being wasted on such a joke.
Re: It's simple, nobody was thinking.
"100*30=3,000 watts continuous power use. He bought an air con that removes 1000 watts. I asked him where the other 2,000 watts were supposed to go, got a blank look."
Will a single 60 watt lightbulb in a room with no aircon will eventually be hotter than the surface of the sun? would extracting 3001 watts from a room using 3000 hit absolute zero sometime?
OK 3000 v's 1000 seems obvious and insufficient, and good for fag packet* maths, but don't forget conductivity and heat capacity of the walls/floors/ceilings, solar gain through any windows, airflow, pressure pockets, insulation etc. it's not rocket science but it is engineering science.
I've seen a server room (a real one, that could remove more heat than the servers could ever produce) cook because the aircon cooled pockets of the room and when a server caught fire the airflow was so badly engineered that the smoke detectors only ever had clean, cool air blown over them.
A 3Kw may not be good enough by itself, conversely 1Kw may be fine, for a hobbyist, just ramp it up to 5Kw and you'll probably be OK, probably, without calcuating for all these other factors you really don't know, an extra $300 for a higher 5Kw "overspec'd" aircon won't be an issue, but you can't do the same thing on the enterprise scale. Perhaps somebody made this exact mistake, someone who thought it was simple, was working to a budget, added up the power usage and thought that would be good enough?
*fag packet, for US readers may not mean what you think it does.
Posted Friday 28th September 2007 13:34 GMT
***Errrr, I'm not sure you have got to grips with how AC works, but its not "pushing" air in through those lagged copper pipes! Its cooling air that stays in the room by circulating it, no pressure build up! The pipes contain the heat laden refrigerant, literally transporting just the heat out of the room.***
I do know quite a bit about the way aircon works (it was part of my job for a long time). In many places, they have an "indirect" air conditioning system, where the aircon unit is fitted to an outside wall or on the roof. It takes air from outside the building and cools it. Then, there are conduits taken to where the cold air is needed. Fans push the cold air from the aircon to the required rooms. That is the sort of aircon I'm talking about. This includes virtually all supermarkets, hospitals, and large office blocks. If the room has no venting, you get a pressure buildup, as I described.
***Are you one of those people (usually women, sorry for the sexism, it's just an observation!) that says "Oooh it's hot in here, turn on the AC and open a window?..."***
Erm, no, I'm not one of those people. Having installed many heating and cooling systems in commercial establishments over the last 30 years, I do realise that a "direct" aircon system works best in a sealed environment (as in my car), but "indirect" systems don't work if the room is sealed. You can only push a limited amount of air into a sealed room before the pressure reaches equilibrium, then the airflow stops.
Doing it right is hard (and really expensive.)
Server rooms are becoming really pretty difficult. Without knowing the exact details it is virtually impossible to know what went wrong. And it is harder to get right than many people realise, and vastly more costly too. It is fast coming to the point where to create from scratch an enterprise quality machine room, the price is going to be of a similar order to the cost of the equipment housed.
You can get a single rack of gear that dissipates 32kW. That is extreme, and is supplied with specialist cooling for the rack, but even less energy dense server systems can very quickly turn into power hungry monsters.
There is a huge and costly difference between "comfort" air and "data centre" air. A data centre system must be capable of running 24x7x52, and runs close to capacity all the time. It also provides humidity control. And is n+1 redundant. Trying to cool a computer room with a system designed to cool human beings is a very bad idea. Such systems are simply not designed to do it - with interesting design issues that cause premature failure if they are pressed into service. For instance a scroll compressor is incapable of coping with liquid refrigerant - it shatters the insides if it ingests fluid. So on a very cold morning, such an air-conditioner may fire up as the machine room comes on line, and starts to dissipate more heat. But things are still cool - and there is not enough heat in the machine room to evaporate all the refrigerant before it reaches the compressor. Exit compressor. After all - the designer of the air-con could hardly imagine that anyone would run it when it is freezing outside and only 20 degrees in the room. I know this one. The system in question has had four new compressors fitted so far.
We have just commissioned a new room for our computational clusters. 60kVA of UPS and 45 kW of (redundant) air-con heat rejection. Cost from starting with a bare concrete shell? 225,000 pounds. Now you get to spend money on servers.
Having done my time nursing rooms full of cooking supercomputers with old and cranky air-con failing in the early hours of the morning - and coped with the drastic drop in reliability of disks and systems in general that goes with it, there really is no other way. But finding the money is often insanely hard. In so many areas the money is typed. There is money for toys, but no money for infrastructure. In many businesses the money to lease kit is easy (spread out, tax effective), but money for capital works is impossible (pay now, no tax breaks).
National Programme for IT (NPfIT)!
Who the hell thought that was a good acronym? Sounds like someone trying to spit out a fly. NPIT would have been a) pronounceable b) readable and c) more accurate. I can just imagine the (endless) meetings that decided finally on sticking a stupid lowercase f in there. " Oh God, we can't use NPIT - think of all the jokes people will make" - maybe they should have thought of that *before* thinking up yet another dull name. Not joined-up, but definitely consistent.
As someone who knows a more than little about PACS, it is entertaining to read this thread, but first to put some details in place:
1) Yes, there is a bungling intermediate company - CSC (Computer Sciences Corporation), one of the 4 (together with Accenture, BT & Fujitsu) are overcharging NPfIT/CfH by a factor of about 4 for the English PACS instalations
2) Yes, PACS does need a lot of storage, and it is rarely used after the inital few days, so tape would seem like a good idea, but jukeboxes are hugely expensive and unreliable, so disk has taken over. That said, anyone using SAN (or even worse, that proprietary, expensive mess called CAS) for PACS data must have a corrupt financial reason for doing so - we shouldn't be paying more than £1k/TByte (I'm told the NHS is paying > £20k/TByte for off-site storage!)
3) There are now systems (e.g. MAID from Copan - http://www.copansys.com/) which only power the disks up when needed - avoiding power and cooling costs which really should be considered for this type of application (I don't work for these people - I just like the concept - others may do the same or perhaps even better).
Just an innocent enquiry re "NHS IT"--- does anyone else think they may have put the "H" in the wrong place?
Money, money, money....
So now we know that, when Tony the Blair-witch said that his government poured so many *EXTRA* billions into the NHS, this is what happened to those billions !!
ALL reports to date indicate that the patients are *NOT* getting any benefit from all that extra dosh !! From all I've heard, we'll have got far better value-for-money just buying a large herd of ponies and getting guys to ride from hospital to hospital with saddle-bagsful of X-ray films; and it's environmentally friendly, too !!
Perhaps we should adopt a concept first postulated by Larry Niven (science fiction author), that any NHS administrator who specced a impossible system should be dismantled and his parts offered for free orgen transplant to the needy. Obviously, the brain will be the most valuable part - one previous owner, hardly ever used !!
I work at Jimmy's - insider info
I worked in an academic research section on the site and we have been advised by an insider that the air con didnt fail but that someone working in the room turned it off.
We havent been told if it was an IT related person or someone else who had access or was given access to carry out work.
Lets hope it wasnt the cleaner plugging in her hover!!
- Nokia: Read our Maps, Samsung – we're HERE for the Gear
- Ofcom will not probe lesbian lizard snog in new Dr Who series
- Kaspersky backpedals on 'done nothing wrong, nothing to fear' blather
- Too slow with that iPhone refresh, Apple: Android is GOBBLING up US mobile market
- Episode 9 BOFH: The current value of our IT ASSets? Minus eleventy-seven...