back to article Your servers are underwater? Chill OUT, baby – liquid's cool

Every time we Tweet, procrastinate by watching an online video of a puppy with hiccups, or query a cloud, we spin up a chain reaction of hardware and electrons in some data centre somewhere. This generates heat that must be dissipated. Moore’s Law – the observation that recently celebrated a 50-year milestone and which …

Page:

  1. Voland's right hand Silver badge

    Minor problem

    Capacitors :(

    Most capacitor designs are intended for operation in air at ambient pressure. Dipping capacitors into liquid coolant really screws that up.

    So you either need a very special design where you pull the voltage regulators and power supplies outside the main coolant dip or you have to throw out your mainboard and power supply every 6 months.

    1. TheVogon Silver badge

      Re: Minor problem

      "Most capacitor designs are intended for operation in air at ambient pressure. Dipping capacitors into liquid coolant really screws that up."

      It takes about 10 metres depth of water density coolant to add 1 atmosphere of pressure - so unless you are thinking of James Bond movies style nuclear reactor cooling tanks, that's really not an issue at all...

      However the highest efficiency future datacentre designs generate power directly in the rack - e.g. from gas catalyst fuel cells, etc. - and these systems are already at the correct voltage - so minimal power supply electronics are required...

      http://www.datacenterdynamics.com/critical-environment/microsoft-our-in-rack-data-center-fuel-cell-concept-works/84945.fullarticle

      1. MacroRodent Silver badge

        hard drives?

        What about hard drives? I think they still have a hole with a filter for equalizing air pressure inside the shell, and will certainly not work if full of oil. Of course not an issue if all your storage is solid state.

        1. cray74

          Re: hard drives?

          Puget Systems, which makes mineral-cooled kits for home PCs, has tested the idea of submerging hard drives in mineral oil. As with fish, mineral oil submersion is poor for hard drives:

          https://www.pugetsystems.com/blog/2013/09/09/Can-you-submerge-a-hard-drive-in-mineral-oil-501/

          1. razorfishsl

            Re: hard drives?

            Since most drive heads require a layer of air to float, adding a viscus liquid is really going to mess things up.

            not to mention the torque on the motor of grinding a disk surface against a liquid

        2. Brandon 2

          Re: hard drives?

          I thought the guts were sealed and filled with nitrogen...

          1. AndrueC Silver badge
            Boffin

            Re: hard drives?

            I thought the guts were sealed and filled with nitrogen...

            Nope. They've always been open to the air with filters to trap particles. Back when I was in data recovery the engineers would have drives spinning with their tops off for days in their clean room. We even had one on a stand at a trade show for a few days. The trouble with sealing them is dealing with the changing pressures especially comparing pressures in an aircraft hold with operating pressures nearer sea level.

          2. Gene Cash Silver badge

            Re: hard drives?

            You're probably thinking of the new drives that are filled with helium:

            http://www.theregister.co.uk/2015/03/25/lighterthanair_drives_floating_to_top_of_hgst_heap/

            I'll bet those have no problem running in oil, and I wonder if you'll be able to see the bubbles of helium leaks?

        3. Suricou Raven

          Re: hard drives?

          The He6 has a niche then.

        4. Gazareth

          Re: hard drives?

          Don't think the drives need cooling particularly. Or at least, not to the level that would require liquid cooling.

          1. razorfishsl

            Re: hard drives?

            yes they do.....

            Take a raid 5 array, run a rebuild.... and see just how much cooling they need.

    2. Olius

      Re: Minor problem

      hahaha, I don't think we would be reading an article about this tech if it didn't already work ;-)

      1. Charles Manning

        Please consider the upside

        If you're a BOFH, or training to be one, you can write off your SCUBA training and equipment as a business expense.

    3. Flocke Kroes Silver badge

      Have you tested this theory?

      Oversimplified:

      The capacitance is εA/d where ε depends on the material, A is the area and d is the distance between electrodes. Increasing the pressure will decrease d, decrease A twice and might do something to ε. Unless you have proper figures from the manufacturer, a small decrease in capacitance with pressure is a sensible guess.

      The maximum voltage depends on d and the material. Increasing the pressure reduces d, but also (often) increases the breakdown voltage. Unless you have proper figures from the manufacturer, expecting the maximum voltage to remain constant is a sensible guess.

      The other figures of merit are impedance at working frequency, maximum current, and temperature for 1000 hours of life. (1000 hours is far too short, so over specify the voltage by a factor of two to double the life, and over specify the temperature by 10C to double the life. Repeat until life time = mandatory guaranty.) It is really hard to guess what some extra pressure would do to these values, but you can be confident that the voltage and temperature markings on the capacitor wildly exceed those it will actually experience, and the whole idea of liquid cooling is to be more effective than air cooling.

      How much pressure: every 10m of water get you an extra atmosphere of pressure. I doubt that the tanks are a whole 2m deep, and oil floats on water, so a sensible guess is less than a 20% increase in pressure.

      In digital circuits, the capacitors are there to stop ground bounce. The capacitors were not selected for capacitance, and ±20% is often chosen because they are cheap. The figure that matters is impedance at some frequency. Often two sizes are fitted to cover a wide range of frequencies. The actual impedance has to be 'low enough', and as the components are cheap, adding plenty is often cheaper than experimenting to find out how few you can get away with.

      In power circuits, the important number is maximum current. Selecting for this normally limits you to capacitors that have more capacitance than your circuit requires. The result is usually harmless, especially as modern (or ten year old) switch mode controllers have soft start built in to deal with large capacitive loads.

      The only thing where pressure will make a clear difference is _after_ the circuit has failed. Abusing electrolytic capacitors causes them to create gasses inside the can until the can ruptures. The cans have been scored so they burst before the internal pressure becomes excessive. Increasing the external pressure will delay the rupture, and a liquid environment will do a better job of transmitting a shock wave to the rest of the power supply. This is entirely academic because the capacitor only ruptured because the power supply was already badly broken.

      I can understand manufacturers voiding warranties as a precaution because they have not tested there components at 1.2 atmospheres. When I have looked for data, manufacturers did not know and did not care. The theory does not point to a clear and obvious problem, so the only way anyone is going to know for sure is to dunk a hundred power supplies, run them for two years and count the survivors.

      1. Gideon 1

        Re: Have you tested this theory?

        Capacitors have more of a problem with low pressures, especially types that do not have a hard case, e.g. resin dipped tantalum. The airworthiness directives are a good guide.

      2. Anonymous Coward
        Anonymous Coward

        Re: Have you tested this theory?

        "The cans have been scored so they burst before the internal pressure becomes excessive."

        So that's the reason for the two cuts across the top of a small SM capacitor can?

        A small buck voltage up-converter module failed recently on first use. The circuit was defined as producing up to 35 volts output from a lower input voltage. The output voltage was selected by a potentiometer.

        The output capacitor was only rated at 35 volts. On initial power-up - after a few seconds the SM capacitor exploded. The can didn't burst though. It was propelled into the air intact - like a bullet.

        It seems likely that the extreme potentiometer setting could produce more than 35 volts. Apparently identical modules from other sellers have a 50 volt rated output capacitor - with a 35 volt one on the input side..

      3. TheVogon Silver badge

        Re: Have you tested this theory?

        "Increasing the pressure will decrease d"

        No - no it wont. Capacitors for power supplies are generally solid or liquid filled, so not compressible except at extreme pressures. Merely immersing them in low levels of liquid is going to make no measurable difference

        "have not tested there components"

        their

  2. Potemkine Silver badge
  3. Anonymous Coward
    Anonymous Coward

    Making the servers waterproof in the first place would be better as water is more readily available.

    We can then have datacentres in the Thames - The heat used to draw up water through the Underground to cool it and the resulting heat sent to buildings - the pipes are probably still there from the hydropower thing.

    And better for connectivity than an icelandic glacier!

  4. Kubla Cant Silver badge

    And at the other end of the scale

    I recall reading an article about five years ago, on an overclocking site, where somebody immersed the electronics of his PC in a tank of Mazola or something. Workable but messy.

    1. Dan Wilkie

      Re: And at the other end of the scale

      Wasn't that on Toms Hardware when they were going for their Super-Mega-Ultraclock on an i5 if I recall? Think they were aiming for 5Ghz/Core?

      1. Gordon861

        Re: And at the other end of the scale

        Yes TomsHardware went through a crazy phase of trying to see how far they could overclock stuff.

  5. Joe Drunk

    It must get awfully messy for the poor sod who has to replace a blade/hard drive/memory/power supply in the event of failure.

    1. Anonymous Coward
      Anonymous Coward

      Well I assume at the point it becomes really practical most things will be blades and there'll be an autoloader. Who cares if a drive/memory/power supply/cpu has broke, unload it load in a replacement send the server to a dryer machine then packaging machine, then into the googlecar to drive the day/week/month bad batch to the repair depot.

      Just imagine how quiet one of these data centres would be compared to the data centre of today.

      1. TRT Silver badge

        No swearing as techs slice their fingers open yet again on razor sharp aluminium extrusions because beancounters found a vendor 20% cheaper even if they didn't roll the edges of their stamped out chassis?

        1. Yet Another Anonymous coward Silver badge

          And you would prefer sticking your hands into the chip fryer they are proposing here ?

          1. TRT Silver badge

            I was referring to the techless server centre.

      2. Suricou Raven

        No drying. That sort of coolant doesn't evaporate. You have to let it drip off, and you'll never get the thin slime off completely.

        1. Adam Foxton

          I use oil-cooled electronics daily- it's been common for cooling and waterproofing subsea kit for decades, I'm amazed it's considered such a new idea by you guys- and they clean up nicely. Just remember a non-conductive contact cleaner. Even Limonene or alcohol based ones are good if the equipment won't be powered on for a while!

  6. Anonymous Coward
    Anonymous Coward

    I would have thought the lack of take up was more obvious

    Probably the weight of higher density servers + liquid coolant is beyond a lot of locations floor or indeed building's capability.

    If the cooling liquid is non-conductive and non-corrosive I would not have expected the boards to have a reduced service life in datacenter terms. I trust however that these liquids are non-toxic however...

  7. Cuddles Silver badge

    Heat capacity

    "a dielectric mineral oil blend called ElectroSafe, an electrical insulator it claims to have 1,200 times more heat capacity by volume than air"

    Not really an amazing claim. Petrol has volumetric heat capacity 1,200 times higher than air, so all they're saying is that their mineral oil is, in fact, oil (specifically short-chain alkanes very slightly longer than the mainly octane and heptane in regular petrol). This also highlights why water is still the preferred substance for actually shipping the heat away with all the faff that having an extra heat exchanger requires - its heat capacity is another 4 times higher again.

    1. Loyal Commenter Silver badge

      Re: Heat capacity

      I seriously hope it's not alkane-based, or you're going to be able to toast marshmallows from half a mile away. It's more likely to be silicone based.

      1. Cuddles Silver badge

        Re: Heat capacity

        "I seriously hope it's not alkane-based, or you're going to be able to toast marshmallows from half a mile away. It's more likely to be silicone based."

        Nope, mineral oil is (mainly) alkanes. Quite commonly used not just as coolant, but also as heat transfer fluid in consumer electric radiators and in plenty of other places (see Wiki - http://en.wikipedia.org/wiki/Mineral_oil). You probably wouldn't want to use it as a open bath since the fumes could cause issues, but it's pretty safe since it's heavy enough not to be easily flammable - it's around the same weight as diesel, and you can happily drop lit matches in that all day long without setting anything on fire.

    2. Anonymous Coward
      Anonymous Coward

      Do they want a fire?

      Because that's how you get a fire.

      1. DropBear Silver badge

        Re: Do they want a fire?

        They claim it's non-flammable. Actually, they claim it's safe enough to drink. Not that I'm inclined to test that...

    3. SImon Hobson Silver badge

      Re: Heat capacity

      > This also highlights why water is still the preferred substance for actually shipping the heat away ...

      Indeed, water is a very good medium for thermal transfer - and if kept pure enough is non-conductive too. In fact, some large generators (think 10s of thousands of volts !) have water cooled coils by the technique of making the copper conductors in the coils from hollow bar and pumping water through them.

      Yes, it does have to be very pure !

  8. Marvin O'Gravel Balloon Face

    The future is Iceland.

    OK, not so great for trading data due to latency, but cheap geothermal power, an endless supply of cold air, and the heat generated can be recycled into domestic/industrial heating and/or energy. Plus the tax breaks are pretty good.

    1. Yet Another Anonymous coward Silver badge

      Re: The future is Iceland.

      If you are going to put all the computers in what is effectively a deep fat fryer - then the future is probably Scotland.

  9. CAPS LOCK Silver badge

    It's the seventies again ...

    .... hurrah.

  10. Little Mouse

    "The warm output water can then be used in other parts of the business..."

    If the coolant is oil-based you could dissipate the heat by cooking chips.

    1. Andy The Hat Silver badge

      Re: "The warm output water can then be used in other parts of the business..."

      Warm oil baths are good for the skin, apparently, so why not have a beauty salon on site and charge fat, rich people extortionate fees to luxuriate in blade induced warmth - "an ultra high frequency digital toning treatment using futuristic heat pumposoidal techniques and plasmondo-rich liquid drenching"? Warming the treatment rooms, plus washing and drying laundry using the heat exchangers all saves cash ... got a feeling I can make more money out of de-ugly therapy than the data centre itself!

      How much does it cost to strain fat people out of plasmondo oil?

      BTW 'plasmondo' is my trademark which I invented just now for a super beauty product that does magic stuff if you soak bodies in it and, by sheer coincidence, is an inert liquid cooling product sold under another name at one third the price by 3M ...

  11. druck
    Mushroom

    Liquid metal cooling

    Liquid sodium as used in fast breeder reactors, burns in air, explodes in water - nice.

    1. DNTP

      Re: Liquid metal cooling

      Just make sure there's plenty of high-def security cameras watching that.

      Not to prevent sodium theft, but so when something goes horribly wrong we can see it on youtube.

    2. Solmyr ibn Wali Barad

      Re: Liquid metal cooling

      Now that's really cool. Or hot.

      /coat.jpg/

  12. Anonymous Coward
    Anonymous Coward

    I'm sorry Dave, I can't go into orbit with that

    I know we usually use ancient computer technology in space, but if liquid cooling became the norm, the gap with earthbound computers might well widen some more.

    Just imagine all the problems with the oil floating around your spaceship if you tried that for the main computer on your long interplanetary voyage and had to deactivate the computer...

    1. cray74

      Re: I'm sorry Dave, I can't go into orbit with that

      If your space hardware needed liquid cooling then you could probably use point-cooling with liquid-cooled heat sinks and dry disconnects.

      I believe the science and electronic racks on the ISS already offers water cooling. While the external radiators use ammonia, heat is collected inside the ISS with water-based coolant through low temperature and medium temperature loops. The medium-temperature loop cools avionics.

      http://www.nasa.gov/pdf/473486main_iss_atcs_overview.pdf

      Going a bit off-topic, the figure on page 14 of the above .pdf illustrates the challenges of cooling in space, which I find interesting. The deployed radiator array with human to scale is a 14-kilowatt radiator. Compare that to the 50- to 300kW radiator of your typical automobile.

      1. ScottAS2
        Headmaster

        Re: I'm sorry Dave, I can't go into orbit with that

        Of course, this is because the ISS radiator is actually a radiator; i.e. it mainly dissipates heat by radiating it. The "radiator" in your car - indeed, most things us Earthbound humans call radiators - are actually more properly convectors, since we have this oh-so handy "air" stuff that you can just dump heat into and allow it to float off. Just goes to show how hard space is.

        Trivia of the day: the Discovery One in 2001: A Space Oddessy was originally going to have enormous cooling radiators for the nuclear reactor at the back, but they were dropped because Kubrick didn't want to have to explain why the (otherwise fairly realistic) non-atmospheric spacecraft design had "wings".

  13. John H Woods Silver badge

    Air cooling has some other problems, too ...

    ... zinc whiskers? Or would that also be a problem with these liquid coolants?

    1. cray74

      Re: Air cooling has some other problems, too ...

      Tin whiskers are a problem in vacuum and air, but may be suppressed with non-ecofriendly options like adding lead to your solder or more environmentally conscious conformal coatings like parylene. Or both.

  14. Anonymous Coward
    Anonymous Coward

    Serious question, how high could I clock a Raspberry Pi in this juice? Considering it normally runs at room temperature without a heatsink or fan.

Page:

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019