back to article Check your machines for malware, Linux developers told

Following a series of embarrassing intrusions that hit the servers used to maintain and distribute the Linux operating system, project elders have advised all developers to check their Linux machines for signs of compromise. Emails sent Friday by Linux kernel lead developers Greg Kroah-Hartman and H Peter Anvin arrived as …

COMMENTS

This topic is closed for new posts.
  1. Sol Rosinberg
    Alert

    Malware issues

    I had some botnet problems arise on my Linux box, so I know how these things go. I still have no idea how they managed to execute commands from the web server and cause the www-data user to run an eggdrop IRC bot along with a menagerie of IRC stuff, but it did, nonetheless. Luckily I run the web server as www-data or it would have rooted me, and that's definitely not good. After updating Wordpress, Apache, PHP, and other things, it seems to have gone away. That, and I blocked the IPs that were constantly scanning me and the providers of the malware in the iptables firewall, which really took a chunk out of the botnet's ability to take over my server. Netstat and tcpdump are your best friends when dealing with this type of problem. Most of the addresses were in China, Russia, and other such places.

    1. Displacement Activity

      So, are you saying that you think kernel.org ran Apache as root?

      1. Captain Scarlet

        No need for root

        I have seen PHP Shells run IRC type items from an infected website after a website with an out of date e107 cms script was breached, you don't always need root just a hole in a setup or a script with known bugs.

  2. Anonymous Coward
    Anonymous Coward

    Intrusions and such are way beyond OS capabilities these days; every modern OS (yes, this includes Windows) has plenty of means to keep your environment secure.

    It would be interesting to know what the majority of involved developers use for the home OS; my bets are on Linux though. So does this mean that Linux has finally become "insecure" because it finally attracted too much "unwanted attention" ? I don't think so; it merely goes to show us that any OS can be setup in an insecure fashion.

    The first requirement to setup a secure environment is to understand the OS fundamentals which you're using, the second is to apply the right security measures and stick to them.

    1. Voland's right hand Silver badge
      Devil

      We have been here before. People just forgot.

      First of all, there ain't such thing as a secure OS.

      Second, in the days before the authors of Back Orifice showed that a windows rootkit is possible Linux was the primary target. I used to run a mid-size academic network in the mid-90es and there was a point where the average time before we got hit by a _NEW_ rootkit variety was down to 48 hours. Sendmail compromises, compromises in basic daemons like ntalk, compromises in bind, etc - you name it. I lost 7 kg spending sleepless nights in front of the keyboard with tcpdump chasing k1dd10tz (it was in the days before snort), rewriting code and patching systems like mad.

      The first automated exploit framework observed in the wild was targeting linux too (I had to deal with the fallout from that one too in my day job).

      These petered out towards 1998-2000 and dropped to nearly nothing after all major distributions picked up key components out of OpenBSD.

      All of this happened versus the backdrop of the rising wave of Windows rootkits so people simply forgot where we started. It however never went away. It was there, it is there.

      1. Sam Liddicott

        sendmail!

        sendmail was the biggest security hole (and one of the first software packages to be commercialized!) - exim and postfix put paid to that.

        I think bind also has a lot to answer for.

      2. Gerhard Mack

        @Voland's right hand

        Mostly true but you give far too much credit to OpenBSD, a lot f things got replaced (sendmail, WUFTPD etc) but even then in the early days OpenBSD's daemons had their fair share of exploits as well.

        Thankfully these days the system software people pretty much have their act together and most Linux root kits are either password guessing or exploiting a web app.

    2. Anonymous Coward
      Anonymous Coward

      "it merely goes to show us that any OS can be setup in an insecure fashion."

      Yeah but if the people writing the kernel can't set up their OS securely, what hope is there for anyone else?

      1. Usually Right or Wrong

        Where there is a motive...

        there will be a compromise, and the motive is not always financial, sometimes it's ego or malice

        I run annual penetration tests and the subsequent remediation programs to fix the weaknesses found. I have done this with various organsiations for many years and not one year have the pen testers failed to gain access, so I am never under any illusion, even applying defence in depth, that I could have systems configured 'securely' and this would prevent a breach.

        the best I can achieve is to make the effort not worth any rewards, but if the motive is ego or malice, not financial, I would probably fail.

      2. Graham Dawson Silver badge

        @AC

        Logical fallacy. They may be writing some part of the kernel, that doesn't mean they know jack about securing it.

      3. Anonymous Coward
        Anonymous Coward

        @AC

        "What hope is there for us?"

        Now, this may sound degrading (not meant to be) but IMO "Knowing how to program doesn't imply knowing how to keep your OS secure" can be applied here.

        This is guessing / speculating on my part but truth be told it also wouldn't surprise me if someone simply picked up on Linux with the assumption "I'm safe because I'm on Linux" and without any specific configuration just went along doing what he wanted.

  3. -tim
    FAIL

    Does anyone learn?

    The first rule of secure computers is don't load software your not going to run. That applies to packages as well. If its not being used, it shouldn't be on the box.

    1. bazza Silver badge
      Thumb Up

      @tim

      It's a mystery to me why anyone would down vote such sage advice. Here's a counterbalancing up vote.

  4. typoo

    Oh, really?

    The servers used to maintain and distribute the Linux operating system get rootkitted and it's no big deal. Un-effing believable.

    1. Anonymous Coward
      Anonymous Coward

      Yes imagine the crowing we'd hear from the freetards if this had happened to the Mac or Windows distribution servers.

      1. sisk

        @AC

        You mean the yawning and mehing you'd hear? Granted there's a vocal minority of Linux users who like to rub it in the faces of Mac and Windows users, but they grow up eventually. Most of us either don't care about virus outbreaks on other platforms or actively help clean them up. You'll not find many Linux geeks past college age who point and sneer at them.

        As for Linux, some of us have realized for some time that we're not immune to malware. I'll be running some security scans myself when I get home from work this evening. I have the software already set up because I know I'm vulnerable to attack.

    2. Tomato42
      Stop

      Cryptographic hashes used by git are distributed and generated on client.

      Cryptographic signatures used for signing packages are created on distributors PCs, not on kernel.org servers.

      We don't know how the signature system is done on Windows so we have to assume the worst: automatic signing with signatures kept on distribution servers. Compromise of such servers would be catastrophic. Kernel.org's, not so much.

      1. TeeCee Gold badge
        FAIL

        "....automatic signing with signatures kept on distribution servers."

        I seriously doubt it. Apart from that being bloody silly, there wouldn't have been such an outpouring of hissy fits when MS enforced signing if it all happened automagically. Also you'd have a bit of trouble explaining why the various code signatures differ, depending on who wrote a particular bit, rather than all being signed by some MS server.

        Still, you just couldn't resist the irrelevant (and ridiculously inaccurate) dig at Windows in a comment section attached to an article on the mass pwnage of Linux, could you?

        1. Tomato42
          Linux

          Like I said, "we don't know". And that answers why we would bash MS if something similar happened to them. It was a hypothetical answer to a hypothetical question.

          Yes, kernel.org has been hacked, it shouldn't have happened but the only damage it caused was a unaccessible site and mirrors. Hardly a disaster.

  5. J 3
    Headmaster

    "He went on to advise developers follow seven steps to see if"

    I'm afraid you mean eight steps, counting from 0 like a good programmer ought to... :-)

  6. AlexS
    Happy

    OK Then...

    Who is going to be the first Windows user to laugh?

    No?

    Didn't think so (superior breeding doesn't dwell on such trivial matters).

    1. bazza Silver badge

      Laugh

      Linux's invulnerability turns out to be an illusion, just like for every other OS. Vociferous proponents now have egg on their faces, and for the moment it's not washing off.

      I'm not sure about superior breeding. Microsoft have dealt with security, bugs, etc. quite well over the past few years. Windows went through security hell, but seems to have emerged stronger from the experience.

      The lack of information on this flaw in Linux is beginning to look very shabby indeed. It's an open source OS, everyone out there should be able to examine the code for the flaw. Looks like the only person who did was the attacker.

      The best information we have seems to be that authorised users on Linux boxes can achieve privilege escalation to get root access, and that there is no way of stopping them doing so. That state of affairs doesn't really recommend Linux to anyone does it?

      1. Destroy All Monsters Silver badge

        "The lack of information on this flaw in Linux is beginning to look very shabby indeed. It's an open source OS, everyone out there should be able to examine the code for the flaw. Looks like the only person who did was the attacker."

        You are implying that there is some new trick going here.

        Might just have been an old trick, judiciously applied.

      2. Santa from Exeter
        FAIL

        Ooh look, it's a rare spotted title

        'The best information we have seems to be that authorised users on Linux boxes can achieve privilege escalation to get root access, and that there is no way of stopping them doing so'

        Written by either a Microsoft shill or a moron.

        either bugger off or learn about user security in Linux before spewing rubbish!

        1. bazza Silver badge

          @Santa from Exeter, @Destroy All Monsters

          @Santa from Exeter

          http://www.theregister.co.uk/2011/08/31/linux_kernel_security_breach/

          Paragraph 3:

          “Intruders gained root access on the server Hera,” kernel.org maintainers wrote in a statement posted to the site's homepage shortly after Hawley's email was leaked. “We believe they may have gained this access via a compromised user credential; how they managed to exploit that to root access is currently unknown and is being investigated.”

          That's from the horses mouth, so to speak. If you don't like what *they're* saying, tough sh*t.

          @Destroy All Monsters

          "You are implying that there is some new trick going here."

          Yes, it's fair to say that I am. But given the length of time it's taken so far to find out what mechanism the exploit used I'd have thought that they would have been able to test for and eliminate the known tricks by now. In contrast, something new could take ages and ages to discover. Presumably the attacker was competent enough to clean up log files to hide their methods.

          If one is responsible for a business critical system running on Linux then one is going to have to at some point consider the likelihood of such an inference being correct. I guess that the lack of reports of mass compromises of Linux servers on the web is encouraging, but it is hardly a guarantee.

          Ok, so the damage done to the Linux source code is nil (the widespread distribution and signing of Linux source code has been well done). But I think that the real problem is the means by which the attack was carried out. I genuinely hope that it turns out to be an oversight of configuration on the part of the sysadmins at kernel.org. But I personally find the cagey nature of how this is being reported less than reassuring. I've never bought into the arguement for non-disclosure until a fix is ready. If that takes a long time then all the users are ignorant of their vulnerability whilst the attacker has a free run. At least give the users a chance to secure their own systems by telling them what's going on. We all hammered Microsoft for such behaviour.

          It's interesting to analyse the motives of the attacker. Money? Not likely from kernel.org I'd have thought. Altering the Linux source code? Unrealistic, maybe, and building in a secret backdoor would seem superfluous given the mastery they'd already have to have over Linux and many other things to achieve that. Maybe a naive and doomed attempt at altering the source code? Could be. Showing off? Who knows. Purely as an attack vector on kernel.org users and similar? Seems to be few pickings to be had from that. Dry run for a later attack against some other Linux website? Not exactly a discrete way to practise.

        2. pitagora
          FAIL

          Santa: the problem is a supposedly an uptodate Linux got rooted and nobody knows how? Doesn't this concern you at all? It's a disaster. This could mean every single Linux machine out there could be vulnerable. Until we know for sure that how we should treat the situation. Personally I don't have/run any Linux servers, but some of my contacts do and they are freaked out!

          1. Tomato42
            Thumb Down

            @pitagora

            There's always one bug more than you think. It's not something we didn't knew already.

            And if your contacts freak out about this they have no idea about computer security. I'm also quite curious what other "impenetrable" OS you're running, surely not Windows, are you?

            Post mortem analysis is far from easy, so just because they still don't know how they gained root, it doesn't mean they used some new vulnerability.

      3. Tom 13

        No, MS have won the PR war like usual.

        I know I'm an amatur when it comes to security stuff. But I use to be able to read the security alerts on the av sites and at MS and make something out of it. These days they are all regurgitated from the same cut and paste recipe book.

    2. TeeCee Gold badge
      Happy

      Laugh? Never.

      Sigh, shake head and mutter "I told you so..."? Quite probably.........

    3. Yag
      Linux

      They should have used a more robust OS!

      Erm... Wait...

      (Okay, done, now let's do something else)

  7. bazza Silver badge

    Guarantees?

    1) At least one of the developers was careless or unlucky enough to get compromised

    2) Does that guarantee that they won't get compromised again?

    3) If just one person doesn't do the checks then the whole thing may start all over again

    4) We still don't know what the compromise mechanism actually was

    5) We have to conclude that the compromise route is still partially open to an attacker

    1. Paul Crawford Silver badge

      @Guarantees?

      We will have to wait until the analysis comes out to find the truth behind this fisaco. However, my suspicion is one of the developer's home PC was rooted, either due to carelessness or from some package in use (or development) that was flawed. Once rooted, the hacker had a 'free pass' in to the kernel development machines, etc, due to that developer's trust level.

      Why has this not happened to MS & Apple in such a spectacular manner?

      Probably because they don't allow anyone outside of their corporate network to access any of the development machines. When you think about it, keeping a globally accessible system safe is SIGNIFICANTLY harder to do.

  8. The obvious
    Terminator

    The real security problem remains the meaty bit.

    KILL ALL HUMANS!!!

  9. Reg T.
    Big Brother

    Not New -

    Several years ago, it was RedHat/Fedora who were cracked - and the lengthy silence from them was deafening.

    The Security Wars are as transparently calculated as is the War on Drugs or the War on Terror.

  10. Sam Liddicott

    TPM

    This is why linux needs TPM.

    Correction; it's why people need TPM on their computers whatever OS they run.

    1. Tom Chiverton 1

      TPM protects software vendors from users, not users from themselves.

    2. Tomato42
      Alien

      Are you implying that just by signing software I get rid of all the bugs in the software?!

  11. Dazed and Confused

    Who said it was a Linux weakness

    If the client end is infected it can steal the credentials for accessing the secure system.

    Imagine a situation where the admin guy's PC gets owned. They use a key logger and find his passphrase for his private key, they find the password for the target system, they find the root password on the target too.

    If they are then able to launch another login session from his client, possibly while he's busy working and they can be connected and do their worse. This does not require any weakness in the target OS.

    How many admins routinely check the number of connections we currently have? Do you know what every single open socket on your PC is doing.

  12. Martin Gregorie
    FAIL

    What's all this trust in chkrootkit?

    The current release of chkrootkit (0.49) is quite old as it was released some time in 2009. The developers aren't answering e-mails, so I'm wondering if its now abandonware.

    I reported the following to them at the beginning of last month to a deafening silence:

    chkrootkit has periodically false alarmed over SuckIT using a sig. pattern against /sbin/init. I also use a behavioral test that says I'm not infected. chkrootkit has periodically done this after kernel updates to Fedora 13 and 14 during the last 12-18 months. Currently it thinks Fedora 15's systemd management package, which replaced the old Sys V init, is infected with SuckIT ever since I first installed it, but again the behavioral test says no.

    If there's a replacement for chkrootkit that's better maintained and has more responsive developers I WANT TO KNOW ABOUT IT.

    1. 437T
      Thumb Up

      Maybe rkhunter?

      It seems to be actively maintained, has a mailing list, and gets a few updates per year with fast updates for new threats.

      1. Martin Gregorie

        Many thanks for the tip: found it in the F15 repository, downloaded and installed.

        Run: says system is clean.

        chkrootkit is now toast.

        1. SnowShell
          Facepalm

          Epic

          Epic so you've installed rootkit hunter, do you know how many false positives that will throw in your face causing you unnecessary concern? I've seen paranoid people do crazy things to their own OS because they believe without a shadow of a doubt what rootkit hunter is spewing at them!

This topic is closed for new posts.

Other stories you might like