back to article Linux literally loses its Lustre – HPC filesystem ditched in new kernel

Linux has literally lost its Lustre – the filesystem favoured by HPC types has vanished in the first release candidate of version 4.18 of the Linux kernel. Linus Torvalds’ announcement of the new release lauds the fact it’s shrunk markedly, much of which can be attributed to the removal of Lustre. “The removal of Lustre may …

  1. Nolveys

    IBM’s IBM Spectrum Scale

    Is that made by IBM?

    1. JacobZ

      Re: IBM’s IBM Spectrum Scale

      No, Tyler Perry

  2. Blockchain commentard

    Is something wrong with Linus? Not a single swear word uttered !!!!!!

  3. anothercynic Silver badge

    Uh-oh...

    ... That'll make the HPC people across the road very happy. NOT.

    1. HPCJohn

      Re: Uh-oh...

      I don't know there... there is a lot more choice these days. The article quite rightly points out that there are more alternatives these days. Fails to mention BeeGFS though which is gainign in popularity. Als CEPH is gaining in the more throughput oriented field, such as particle physics. There was a post of the CEPH mailing list today from a researcher at Lawrence Berkely Lab, who compared to Lustre. SO I think a lot of 'big iron' shops are contemplating or moving to CEPH.

      1. Bronek Kozicki

        Re: Uh-oh...

        Other great filesystems aside, I think that you do not need to have Lustre in the upstream kernel in order to be able to use it - just build your own modules from out-of-tree Lustre sources. Not entirely sure about this and happy to be corrected.

    2. ExpatZ

      Re: Uh-oh...

      Gonna make my job harder, we use Lustre on our big Beowolf/Grid Engine cluster and I am tasked with upgrading it all right now.

      1. DrXym

        Re: Uh-oh...

        "Gonna make my job harder,"

        But if the impl in the kernel is half baked, surely you need to patch it anyway? And if that's the case, what difference does it really make - the patch will change but it will still be a patch.

  4. kbuggenhout

    This is creating a lot of unnecessary confusion

    Imho, and as a person in the field of hpc, lustre is still the most used filesystem in hpc systems worldwide. The mentioning of spectrumscale is an odd one, is spectrumscale is as old as lustre. Has no support in the kernel (doesnt need it) but is horribly expemsive. We are deploying clusters with lustre every week. The dev direction of lustre is to make builds that do not need specific kernels, but can use kernel modules to obtain its functionality.

    Besides that, intel stopped its own offering if the fs, which was a specific patchlevel geared towards enterprise usage. This seemed to be a bit too far fetched even for intel. They still have the largest group of devs for lustre in house, they still offer l3 support and community support. In the field we encounter other filesystems, where 90 % is divided between lustre,BeeGFS and spectrumscale. The other 10% are the newcomers that grew up in the big data world. Scale out nfs is rare, as nfs does not provide support for large scale applications that need special offloaded I/O.

    To call lustre down and out is way too premature. It has existed for most of its life without native kernel support, and with the latest efforts will not need that native support as it will be running by loaded modules.

    This article is accurate to the point that the stubs in linux kernel are gone, but thats all.

    Hdfs has no place in hpc. Unless its a hybrid system that offers best of both worlds, even then there is most likely both.

    Just my 2 c

  5. Gene Cash Silver badge

    If you look at the history of the project in Wikipedia, it looks like a game of 1930s football...

    It started at CMU, went to Sun, Sun got bought by Oracle, Oracle says it's ceasing development, Whamcloud picked it up, Intel bought Whamcloud, then Xyratex gets the IP from Oracle, and it's apparently now being developed by Open SFS.

    Or something like that.

    Whew.

    1. Androgynous Cow Herd

      I wasn't aware that 1930s football featured software development, but the world is a poorer place now that the practice has stopped.

  6. Anonymous Coward
    Linux

    a Thinner Linux Kernel

    Wow, then what would happen if Linux kernel development team threw out a lot of other crap,

    perhaps Linux would win the dieter of the year competition.

    Lots of crap inside was the reason many people stopped compiling their kernel,

    too crappy!!!

    1. Anonymous Coward
      Anonymous Coward

      Re: a Thinner Linux Kernel

      LWN has repeatedly shown that old interfaces are found lacking and replaced by new and shinier interface2 ... that not always age well either. So now we have a lot of deprecated interfaces that cannot be removed in the name of compatibility. There is also stupidity like strfry.

      So why not declare a flag day in 2025 where Linux 5.0 will kill of all deprecated interfaces and clean up it all? Embedded developers will be grateful.

      The alternative is a different platform, perhaps Redox-OS where they are not afraid of removing misfeatures.

    2. Lee D Silver badge

      Re: a Thinner Linux Kernel

      I don't think anybody every compiled such a module by accident.

      Until you're insane enough to use allyesconfig, this stuff never affected compile times.

      Nobody compiles a kernel any more because everyone does it for you nowadays. Getting a pre-fab kernel for your specific hardware used to mean custom compiling. Nowadays, you just pick up any distro and boot the default, unless you have some fancy esoteric tastes, in which case the relevant kernel module gets included for you.

      Last time I compiled Linux was pre-2000, I think.

      1. Bronek Kozicki

        Re: a Thinner Linux Kernel

        I build my own kernel for every minor release - it's pretty easy actually, but not because of my (rather embarrassing level of) knowledge of how to do it, but simply thanks to the distribution making it easy to customise and build own packages.

  7. Camilla Smythe
    Unhappy

    Good!

    Now they have less work to do they can sort out my BlueTooth. I'll have you know these headphones cost me fifteen quid. Grumble grumble.

    1. shedied

      Re: Good!

      Have you been to your distro's support forum, or the distro's IRC channel? There are people there who might be able to help get your BT headset up and running--it's worth a shot.

POST COMMENT House rules

Not a member of The Register? Create a new account here.

  • Enter your comment

  • Add an icon

Anonymous cowards cannot choose their icon

Other stories you might like