Following a series of embarrassing intrusions that hit the servers used to maintain and distribute the Linux operating system, project elders have advised all developers to check their Linux machines for signs of compromise. Emails sent Friday by Linux kernel lead developers Greg Kroah-Hartman and H Peter Anvin arrived as …
I had some botnet problems arise on my Linux box, so I know how these things go. I still have no idea how they managed to execute commands from the web server and cause the www-data user to run an eggdrop IRC bot along with a menagerie of IRC stuff, but it did, nonetheless. Luckily I run the web server as www-data or it would have rooted me, and that's definitely not good. After updating Wordpress, Apache, PHP, and other things, it seems to have gone away. That, and I blocked the IPs that were constantly scanning me and the providers of the malware in the iptables firewall, which really took a chunk out of the botnet's ability to take over my server. Netstat and tcpdump are your best friends when dealing with this type of problem. Most of the addresses were in China, Russia, and other such places.
So, are you saying that you think kernel.org ran Apache as root?
No need for root
I have seen PHP Shells run IRC type items from an infected website after a website with an out of date e107 cms script was breached, you don't always need root just a hole in a setup or a script with known bugs.
Intrusions and such are way beyond OS capabilities these days; every modern OS (yes, this includes Windows) has plenty of means to keep your environment secure.
It would be interesting to know what the majority of involved developers use for the home OS; my bets are on Linux though. So does this mean that Linux has finally become "insecure" because it finally attracted too much "unwanted attention" ? I don't think so; it merely goes to show us that any OS can be setup in an insecure fashion.
The first requirement to setup a secure environment is to understand the OS fundamentals which you're using, the second is to apply the right security measures and stick to them.
We have been here before. People just forgot.
First of all, there ain't such thing as a secure OS.
Second, in the days before the authors of Back Orifice showed that a windows rootkit is possible Linux was the primary target. I used to run a mid-size academic network in the mid-90es and there was a point where the average time before we got hit by a _NEW_ rootkit variety was down to 48 hours. Sendmail compromises, compromises in basic daemons like ntalk, compromises in bind, etc - you name it. I lost 7 kg spending sleepless nights in front of the keyboard with tcpdump chasing k1dd10tz (it was in the days before snort), rewriting code and patching systems like mad.
The first automated exploit framework observed in the wild was targeting linux too (I had to deal with the fallout from that one too in my day job).
These petered out towards 1998-2000 and dropped to nearly nothing after all major distributions picked up key components out of OpenBSD.
All of this happened versus the backdrop of the rising wave of Windows rootkits so people simply forgot where we started. It however never went away. It was there, it is there.
"it merely goes to show us that any OS can be setup in an insecure fashion."
Yeah but if the people writing the kernel can't set up their OS securely, what hope is there for anyone else?
Where there is a motive...
there will be a compromise, and the motive is not always financial, sometimes it's ego or malice
I run annual penetration tests and the subsequent remediation programs to fix the weaknesses found. I have done this with various organsiations for many years and not one year have the pen testers failed to gain access, so I am never under any illusion, even applying defence in depth, that I could have systems configured 'securely' and this would prevent a breach.
the best I can achieve is to make the effort not worth any rewards, but if the motive is ego or malice, not financial, I would probably fail.
Logical fallacy. They may be writing some part of the kernel, that doesn't mean they know jack about securing it.
sendmail was the biggest security hole (and one of the first software packages to be commercialized!) - exim and postfix put paid to that.
I think bind also has a lot to answer for.
@Voland's right hand
Mostly true but you give far too much credit to OpenBSD, a lot f things got replaced (sendmail, WUFTPD etc) but even then in the early days OpenBSD's daemons had their fair share of exploits as well.
Thankfully these days the system software people pretty much have their act together and most Linux root kits are either password guessing or exploiting a web app.
"What hope is there for us?"
Now, this may sound degrading (not meant to be) but IMO "Knowing how to program doesn't imply knowing how to keep your OS secure" can be applied here.
This is guessing / speculating on my part but truth be told it also wouldn't surprise me if someone simply picked up on Linux with the assumption "I'm safe because I'm on Linux" and without any specific configuration just went along doing what he wanted.
Does anyone learn?
The first rule of secure computers is don't load software your not going to run. That applies to packages as well. If its not being used, it shouldn't be on the box.
It's a mystery to me why anyone would down vote such sage advice. Here's a counterbalancing up vote.
The servers used to maintain and distribute the Linux operating system get rootkitted and it's no big deal. Un-effing believable.
Yes imagine the crowing we'd hear from the freetards if this had happened to the Mac or Windows distribution servers.
Cryptographic hashes used by git are distributed and generated on client.
Cryptographic signatures used for signing packages are created on distributors PCs, not on kernel.org servers.
We don't know how the signature system is done on Windows so we have to assume the worst: automatic signing with signatures kept on distribution servers. Compromise of such servers would be catastrophic. Kernel.org's, not so much.
"....automatic signing with signatures kept on distribution servers."
I seriously doubt it. Apart from that being bloody silly, there wouldn't have been such an outpouring of hissy fits when MS enforced signing if it all happened automagically. Also you'd have a bit of trouble explaining why the various code signatures differ, depending on who wrote a particular bit, rather than all being signed by some MS server.
Still, you just couldn't resist the irrelevant (and ridiculously inaccurate) dig at Windows in a comment section attached to an article on the mass pwnage of Linux, could you?
Like I said, "we don't know". And that answers why we would bash MS if something similar happened to them. It was a hypothetical answer to a hypothetical question.
Yes, kernel.org has been hacked, it shouldn't have happened but the only damage it caused was a unaccessible site and mirrors. Hardly a disaster.
You mean the yawning and mehing you'd hear? Granted there's a vocal minority of Linux users who like to rub it in the faces of Mac and Windows users, but they grow up eventually. Most of us either don't care about virus outbreaks on other platforms or actively help clean them up. You'll not find many Linux geeks past college age who point and sneer at them.
As for Linux, some of us have realized for some time that we're not immune to malware. I'll be running some security scans myself when I get home from work this evening. I have the software already set up because I know I'm vulnerable to attack.
"He went on to advise developers follow seven steps to see if"
I'm afraid you mean eight steps, counting from 0 like a good programmer ought to... :-)
Who is going to be the first Windows user to laugh?
Didn't think so (superior breeding doesn't dwell on such trivial matters).
Linux's invulnerability turns out to be an illusion, just like for every other OS. Vociferous proponents now have egg on their faces, and for the moment it's not washing off.
I'm not sure about superior breeding. Microsoft have dealt with security, bugs, etc. quite well over the past few years. Windows went through security hell, but seems to have emerged stronger from the experience.
The lack of information on this flaw in Linux is beginning to look very shabby indeed. It's an open source OS, everyone out there should be able to examine the code for the flaw. Looks like the only person who did was the attacker.
The best information we have seems to be that authorised users on Linux boxes can achieve privilege escalation to get root access, and that there is no way of stopping them doing so. That state of affairs doesn't really recommend Linux to anyone does it?
"The lack of information on this flaw in Linux is beginning to look very shabby indeed. It's an open source OS, everyone out there should be able to examine the code for the flaw. Looks like the only person who did was the attacker."
You are implying that there is some new trick going here.
Might just have been an old trick, judiciously applied.
Ooh look, it's a rare spotted title
'The best information we have seems to be that authorised users on Linux boxes can achieve privilege escalation to get root access, and that there is no way of stopping them doing so'
Written by either a Microsoft shill or a moron.
either bugger off or learn about user security in Linux before spewing rubbish!
Sigh, shake head and mutter "I told you so..."? Quite probably.........
They should have used a more robust OS!
(Okay, done, now let's do something else)
@Santa from Exeter, @Destroy All Monsters
@Santa from Exeter
“Intruders gained root access on the server Hera,” kernel.org maintainers wrote in a statement posted to the site's homepage shortly after Hawley's email was leaked. “We believe they may have gained this access via a compromised user credential; how they managed to exploit that to root access is currently unknown and is being investigated.”
That's from the horses mouth, so to speak. If you don't like what *they're* saying, tough sh*t.
@Destroy All Monsters
"You are implying that there is some new trick going here."
Yes, it's fair to say that I am. But given the length of time it's taken so far to find out what mechanism the exploit used I'd have thought that they would have been able to test for and eliminate the known tricks by now. In contrast, something new could take ages and ages to discover. Presumably the attacker was competent enough to clean up log files to hide their methods.
If one is responsible for a business critical system running on Linux then one is going to have to at some point consider the likelihood of such an inference being correct. I guess that the lack of reports of mass compromises of Linux servers on the web is encouraging, but it is hardly a guarantee.
Ok, so the damage done to the Linux source code is nil (the widespread distribution and signing of Linux source code has been well done). But I think that the real problem is the means by which the attack was carried out. I genuinely hope that it turns out to be an oversight of configuration on the part of the sysadmins at kernel.org. But I personally find the cagey nature of how this is being reported less than reassuring. I've never bought into the arguement for non-disclosure until a fix is ready. If that takes a long time then all the users are ignorant of their vulnerability whilst the attacker has a free run. At least give the users a chance to secure their own systems by telling them what's going on. We all hammered Microsoft for such behaviour.
It's interesting to analyse the motives of the attacker. Money? Not likely from kernel.org I'd have thought. Altering the Linux source code? Unrealistic, maybe, and building in a secret backdoor would seem superfluous given the mastery they'd already have to have over Linux and many other things to achieve that. Maybe a naive and doomed attempt at altering the source code? Could be. Showing off? Who knows. Purely as an attack vector on kernel.org users and similar? Seems to be few pickings to be had from that. Dry run for a later attack against some other Linux website? Not exactly a discrete way to practise.
Santa: the problem is a supposedly an uptodate Linux got rooted and nobody knows how? Doesn't this concern you at all? It's a disaster. This could mean every single Linux machine out there could be vulnerable. Until we know for sure that how we should treat the situation. Personally I don't have/run any Linux servers, but some of my contacts do and they are freaked out!
No, MS have won the PR war like usual.
I know I'm an amatur when it comes to security stuff. But I use to be able to read the security alerts on the av sites and at MS and make something out of it. These days they are all regurgitated from the same cut and paste recipe book.
There's always one bug more than you think. It's not something we didn't knew already.
And if your contacts freak out about this they have no idea about computer security. I'm also quite curious what other "impenetrable" OS you're running, surely not Windows, are you?
Post mortem analysis is far from easy, so just because they still don't know how they gained root, it doesn't mean they used some new vulnerability.
1) At least one of the developers was careless or unlucky enough to get compromised
2) Does that guarantee that they won't get compromised again?
3) If just one person doesn't do the checks then the whole thing may start all over again
4) We still don't know what the compromise mechanism actually was
5) We have to conclude that the compromise route is still partially open to an attacker
We will have to wait until the analysis comes out to find the truth behind this fisaco. However, my suspicion is one of the developer's home PC was rooted, either due to carelessness or from some package in use (or development) that was flawed. Once rooted, the hacker had a 'free pass' in to the kernel development machines, etc, due to that developer's trust level.
Why has this not happened to MS & Apple in such a spectacular manner?
Probably because they don't allow anyone outside of their corporate network to access any of the development machines. When you think about it, keeping a globally accessible system safe is SIGNIFICANTLY harder to do.
The real security problem remains the meaty bit.
KILL ALL HUMANS!!!
Not New -
Several years ago, it was RedHat/Fedora who were cracked - and the lengthy silence from them was deafening.
The Security Wars are as transparently calculated as is the War on Drugs or the War on Terror.
This is why linux needs TPM.
Correction; it's why people need TPM on their computers whatever OS they run.
TPM protects software vendors from users, not users from themselves.
Are you implying that just by signing software I get rid of all the bugs in the software?!
Who said it was a Linux weakness
If the client end is infected it can steal the credentials for accessing the secure system.
Imagine a situation where the admin guy's PC gets owned. They use a key logger and find his passphrase for his private key, they find the password for the target system, they find the root password on the target too.
If they are then able to launch another login session from his client, possibly while he's busy working and they can be connected and do their worse. This does not require any weakness in the target OS.
How many admins routinely check the number of connections we currently have? Do you know what every single open socket on your PC is doing.
What's all this trust in chkrootkit?
The current release of chkrootkit (0.49) is quite old as it was released some time in 2009. The developers aren't answering e-mails, so I'm wondering if its now abandonware.
I reported the following to them at the beginning of last month to a deafening silence:
chkrootkit has periodically false alarmed over SuckIT using a sig. pattern against /sbin/init. I also use a behavioral test that says I'm not infected. chkrootkit has periodically done this after kernel updates to Fedora 13 and 14 during the last 12-18 months. Currently it thinks Fedora 15's systemd management package, which replaced the old Sys V init, is infected with SuckIT ever since I first installed it, but again the behavioral test says no.
If there's a replacement for chkrootkit that's better maintained and has more responsive developers I WANT TO KNOW ABOUT IT.
It seems to be actively maintained, has a mailing list, and gets a few updates per year with fast updates for new threats.
Many thanks for the tip: found it in the F15 repository, downloaded and installed.
Run: says system is clean.
chkrootkit is now toast.
Epic so you've installed rootkit hunter, do you know how many false positives that will throw in your face causing you unnecessary concern? I've seen paranoid people do crazy things to their own OS because they believe without a shadow of a doubt what rootkit hunter is spewing at them!