back to article How'd your servers get that baby-smooth look? Dutch and Brit cool kids dunk Supermicro systems in synthetic oil

Dutch liquid cooling specialist Asperitas and British systems integrator Boston have linked arms on a range of server systems based on "immersion cooling" – with the hardware submerged in giant tubs of synthetic oil, a dielectric compound very similar to Vaseline or baby oil. Four fully contained standalone systems were …

  1. Steve K Silver badge

    Practical question

    I can see the benefits of this, but from curiosity how do you remove the oil coolant from servers/parts when you need to replace hardware?

    Or (since it is dielectric) does it not actually matter (as long as the parts don't slip out of your hands!!)

    1. chris 143

      Re: Practical question

      I don't think it's significantly more magical than lift it out and try not to drip oil all over the floor.

      I've seen demos where they'd more or less taken the top off a 1U/2U server and removed the fans then dunked it in a vat of oil.

      Apparently ssds/helium hdds are happy enough being submerged too. Although I imagine it'd invalidate any warranty

    2. Mark 85 Silver badge

      Re: Practical question

      I shudder to think of the messes and slippery floors doing that. I guess hot swappable is out of the question then?

      1. Gordon 10 Silver badge
        Pint

        Re: Practical question

        I shudder to think of what the lubed up BOFH's and engineers would look like afterwards.

        1. Steve K Silver badge

          Re: Practical question

          They'd have lovely, smooth skin?

    3. Roo
      Windows

      Re: Practical question

      At most datacenters I've worked with entire boxes are swapped out - the broken one taken away for further examination/repair/exchange/disposal. As long as they have nice self-sealing connectors for the plumbing they'll be fine.

      The boxes will be substantially more dense than normal ones though - the oil will be heavier and they appear to be packing a lot more components into the same volume than a vanilla box... Perhaps professional wrestlers could do the job - they should be well practiced in grappling with heavy oily units. :)

  2. Anonymous Coward
    Anonymous Coward

    Cinzano?

    > thanks to their self-contained nature, such systems can be deployed in any kind of building, in any climate – no data centre or dedicated server room required.

    The heat still has to go somewhere. If the oil bath doesn't go through a heat exchanger then the heat is still going to warm the room, just a bit more quietly than if it had fans.

    1. chivo243 Silver badge
      Headmaster

      Re: Cinzano?

      ...and a secondary warm water circuit ejects the heat from the system" Isn't this a heat exchange?

      1. Anonymous Coward
        Anonymous Coward

        Re: Cinzano?

        > ...and a secondary warm water circuit ejects the heat from the system" Isn't this a heat exchange?

        You're right of course, but that doesn't really fit with their 'self-contained' and 'any climate' statements. If they'd just said "needs less intrusive cooling", or something of that ilk I'd be fine with it.

        TBH, I can't see my clients with traditional DCs buying this - if you've already got all the air con why would you then even consider adding more complexity to your cooling arrangements? And if you haven't got a traditional DC why wouldn't you use the cloud?

    2. DougS Silver badge

      Re: Cinzano?

      If you had a lot of these, I would think the best solution would be a direct exchange system, where your HVAC has a "freon" (whatever the current substitute is called) loop piped to each cabinet.

      The bigger problem would appear to be weight, though if you designed your datacenter around this you could dispense with the raised floor and it wouldn't matter if you had 3-4x the typical per sq ft weight.

      1. This post has been deleted by its author

  3. Arthur the cat Silver badge
    Pint

    A shame (Si) chips can't run hotter

    Or you could use vegetable oil instead of mineral and combine your machine room with a chippie.

    1. Steve K Silver badge

      Re: A shame (Si) chips can't run hotter

      Or a water-bath/confit if temperatures aren't high enough for a chippie (sorry - been watching MasterChef)

      1. TRT Silver badge

        Re: A shame (Si) chips can't run hotter

        Sous vide-o output?

    2. GrapeBunch Bronze badge

      Re: A shame (Si) chips can't run hotter

      Or perhaps the temps are about right for decarboxylation.

  4. vir

    Respect Your Elders

    They need a Cray-2-style waterfall heat exchanger somewhere in the system, just to show who got there 30 years ago.

    1. Voyna i Mor Silver badge

      Re: Respect Your Elders

      But not the hideously expensive Fluorinert used as the medium.

      I do sometimes wonder if Cray thought about using purified mineral oil at around 1/1000 of the cost and then thought, no, expensive computer needs expensive cooling system, and if everyone else thinks fluorinert is essential, they won't copy us.

      1. Chz

        Re: Respect Your Elders

        I was going to say that the Fluorinert couldn't be huge next to the total system cost, but then I looked it up and a Cray-2 was "only" $15M. Seems cheap next to today's massive clusters!

        Still, a 750lb barrel of Fluorinet is "just" $35k. Old Seymour didn't believe in doing anything if it wasn't all-out. Bless him. His biography is fascinating.

        https://www.chemworld.com/Novec-FC-770-Fluorinert-Electronic-Liquid-p/novec-fc-770.htm

  5. Ken Moorhouse Silver badge

    Not suitable for Bad Programmers...

    All those Memory Leaks will lead to flooding.

  6. Tezfair

    new spin on old tech?

    I recall this being a thing some 20 years ago

    1. Anonymous Coward
      Anonymous Coward

      Re: new spin on old tech?

      tl;dr

      Google aquarium pc

    2. martinusher Silver badge

      Re: new spin on old tech?

      Back in the early 1980s I worked on a R/F amplifier that was cooled by purified water. The water had to be ion free because the amplifier tube that was being cooled was sitting at around 7.5Kv. It worked well enough except in frosty weather -- the water froze when the prototype wasn't running.

      (I've used a similar setup on load resistors I used for testing high power audio amplifiers. Nothing sophisticated here, I just dumped them in a bucket of water.)

      1. Colintd

        Re: new spin on old tech?

        I've got a dummy generator load which is an oil barrel with water and caustic soda inside. Neutral (earthed at Genny) to barrel, lengthy of live wire stipped and dangled in middle of barrel. Works a treat as long as you have good ventilation (to avoid explosion).

        1. John Brown (no body) Silver badge

          Re: new spin on old tech?

          Those pole mounted transformers have been oil filled for many years too. As you and others have said, it's not new technology.

    3. Anonymous Coward
      Anonymous Coward

      Re: new spin on old tech?

      "I recall this being a thing some 20 years ago"

      me too - seem to recall TomsHardware building a system immersed in oil which they were able to overclock significantly.

    4. Adrian 4 Silver badge

      Re: new spin on old tech?

      These things go in cycles (not just spin cycles..) - in 10 years time when liquid cooling is the norm, someone will be pushing the latest thing in air cooling, no plumbing, no leaks.

    5. Jan 0

      Re: new spin on old tech?

      Nearer to 40 years ago, by Seymour Cray at CDC. I can't find any pictures but I recall the same design appearing in Wim Wender's film "Until the End of the World".

  7. elDog Silver badge

    I don't believe it. Lend me a few top-end units so I can test myself.

    I'll supply the mineral oil (baby oil), canola, corn, sesame, peanut should work also. Besides, think of the aromas!

    Now to come up with a workload to put my demo units through their paces. Perhaps "Answer to the Ultimate Question of Life, The Universe, and Everything". Shirley with these specs the "Anna AIC (CLOUDzone243) is intended for GPU-based workloads like 3D rendering and machine learning – featuring the same 1,056 Xeon cores, but adding a whopping 331,776 CUDA cores across 72 integrated Nvidia cards" should be able to compute 42 in less than the 7.5 million years (probably performed on 8-bit CPUs.

    Anybody have a spare hydro plant to loan me for a few million years?

    1. Anonymous Coward
      Anonymous Coward

      Re: I don't believe it. Lend me a few top-end units so I can test myself.

      Now to come up with a workload to put my demo units through their paces

      Windows 10 with Tetris ought to do it..

  8. Anonymous Coward
    Anonymous Coward

    Is this really comparable to 3M Novec

    Not sure if this is comparable to 3M Novec as this sounds like a traditional liquid cooled system compared to Novecs novel boil off heat transfer method. I imagine the price differential would be similar to the performance differential between the two solutions. Either way its cool to see this tech making its way to our datacenters.

  9. Gnoitall
    Flame

    May be more than just magic smoke

    Jokes about the computer working hot enough to catch fire won't be that funny when the coolant ignites.

  10. Anonymous Coward
    Anonymous Coward

    Snake oil ?

  11. Starace
    Alert

    So they've fixed the problems then?

    Other times this has been tried it usually didn't work too well - poor circulation/localised cooling, reactions between components and the oil and issues with dielectric effects all tended to cause problems.

    Also someone mentioned hard drives - they don't work for long either as either the pressure equalisation valve or a seal will give out and let oil in.

    It's not that immersion cooling doesn't work at all, but you can't do it cheaply and expect a lot of success.

    1. ElReg!comments!Pierre Silver badge

      Re: So they've fixed the problems then?

      poor circulation/localised cooling

      That's why you should use diesel as the coolant.

      https://www.theregister.co.uk/2002/05/11/bofh_and_the_vax_cluster/

  12. circusmole
    Happy

    How about using...

    ...castor based oil that used to go into my old Manx Norton. You then get the benefit of the glorious smell.

    1. Wellyboot Silver badge
      Thumb Up

      Re: How about using...

      Castrol-R - the racing smell.. good memories

    2. Voyna i Mor Silver badge

      Re: How about using...

      IIRC I read that in WW1 castor oil lubricant (used in aircraft engines so crude I just cannot image someone taking off in a plane powered by them) was inevitably taken in by the pilots in the exposed cockpits of the day, causing the runs.

      The cure for the runs was whisky.

      Part of the death rate was due to drunk pilots.

      The thought of a data centre full of IT staff either headed for the loo or on the whisky bottle is a little concerning.

  13. M.V. Lipvig
    Joke

    Pretty cool

    Put your gear in clear glass cabinets, then when the workload gets heavy watch the oil start to boil. Be like working in Frankenstein's IT room.

    1. Jack of Shadows Silver badge

      Re: Pretty cool

      I've some datasets to train the nVidia version here that should quite do the job!

      [My first thought as well.]

  14. Roland6 Silver badge

    However, due to their unusual footprint (600x1200mm), these systems are unlikely to serve as drop-in replacements for a standard 19-inch frame.

    Note to self - they also won't fit in a 21-inch Open Rack standard cabinet footprint...

    1. Blockchain commentard Silver badge

      19" rack is about 600 x 1000mm so remove the rack, slip this in (pun intended) and you only lose 200mm/8 inches of inter-rack corridor space. Can't see a problem really.

    2. Bronek Kozicki Silver badge

      ... and there is also a question of weight.

  15. Asperitas

    Hi all, we noticed a lot of questions around this article.

    It’s impossible to answer all individually, that’s why we’ve planned a webinar on Friday, April 12th, 2019.

    During this webinar Asperitas and Boston will cover all your questions. Register for free at https://eveeno.com/the-future-is-now

  16. HieronymusBloggs Silver badge

    Dielectric

    "The company uses mineral oil, which is much cheaper than proprietary dielectrics"

    My computers use the cheapest dielectric of all (air) for cooling.

  17. G R Goslin

    Prior Art?

    A long time ago, Shortly after the war, in fact, I read an article where the principle was tried in America. It was better than the system proposed here, in that it used a low boiling point fluid, One of the Freons, I believe. The advantage here is that taken away from the housing, the coollant would merely evaporate away, leaving the kit clean and dry. There was too the advantage of a boiling coolant, that the temperature of any part of the assembly could never achieve a temperature in excess of the boiling point, and the coolant didn't have to be pumped around the cooling circuit.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Biting the hand that feeds IT © 1998–2019