CERT: Linux servers under 'Phalanx' attack

@vincent: Time for GEOS

Have fun with your command line, mate. I'm heading back to GEOS 128.

Paris, having fun heading back...

What a crummy RK

It wins the getting installed race, but then lets you cd right into it. It needs work. Bah, no one takes any pride any more. It's a damned kernel level rootkit it's supposed to be undetectable at least locally it doesn't even clean up after it self.

'cs'?

"bash: cs: command not found"

sure you didn't mean 'cd'? Simple typo. (apologies if this comment appears below about a thousand all saying the same thing, there was only one comment when I hit 'Post Comment'

Other than that, very helpful to know. *goes to check his boxen*

Key and password?

This takes advantage of the same secure openssh server that can not be configured to require a key and a password. Is Theo slipping?

Heh. Clean up is not prevention.

What a mess. I hope the Linux crowd doesn't have to learn about security from the Windows crowd.

What?

"I'm still absolutely adamant this is a problem system administrators should have handled a long time ago,"

I suppose the problem in question would be keeping the weak SSH keys around, right?

Of course, the problem instead might be people from userland trying to ring up the BOFH or the Boss giving out clue-depleted orders etc., all of which can be handled.

(And why "still absolutely adamant"? Might there be reasons to make him less adamant or un-adamant later on, given a re-assessment of the situation?)

Dead horse

"The CERT advisory makes no mention of the flaw in the Debian random number generator,"

Because it's been fixed,... and the debian ssh packages depend on blacklists that block weak keys from being used for ingoing and outgoing connections, ship with tools to find weak keys etc blah blah blah. You realise a good starting point for finding keys would be to bruteforce boxes than allow password authentication don't you?

@E

...and why hadn't you updated openssl and regenerated all your keys back when this massive vulnerability became public, then?

Paris - even she would've figured that one out.

@ E

You missed option c) for the tin-foil hat wearing brigade: remove hard drive and render it inoperable by means of a small shaped nuclear explosion... wearing said tin foil hat for protection!

Like suckit..

This rootkit looks a lot like one that was released a few years ago called suckit, except phalanx is designed for 2.6.x kernels while suckit was for 2.4.x.

re: E

"The cleanup is, depending on your paranoia level, (a) boot off an install DVD to repair mode and delete the rootkit, or (b) format your disks and reinstall Linux."

Not much different from what is required for a Windows rootkit then, although option (b) is far more preferable for 'doze as it's not easy cleaning a system from a boot disc due to problems with registry dependencies / hooks etc.

Hmm

Windows is a memory hungry bloat-monster

OSX has the EULA of Satan

Linux shown to be insecure

Is there no viable OS in the world?

Acorn MOS come back, all is forgiven!

Amateur

The main problem I have with this is that it still looks like an amateur programmed it. So they've managed kernel-level access, brute-forcing of low-entropy keys, they collect keys and spread between trusted machines - fantastic. But nobody could be bothered to make a primitive attempt to hide it's presence in the filesystem, thus making it trivially detectable (especially if you have anything that monitors filesystems) and (less important) trivially removeable.

Why does it need more than a prescence in something like /tmp in order to initially infect and then it's got kernel-level access, so why does it show up at all? There is all sorts of code available in projects both legitimate and not that will let you hide data inside files that are not visible, will let you pre-pend code to kernel images, will let you modify bootloaders, etc. - if it weren't visible in the "real" filesystem, it could quite easily avoid manual and automatic detection by rootkit hunters etc. and be much more prevelant and dangerous.

Again, viruses built by pluggable modules that just show that the actual author of it (should they be caught) aren't the problem - someone, somewhere wrote a set of modules to do the difficult stuff and *they* are the problem, here, not some 12-year-old that joined a few of them together in "Build-a-virus '98".

Having said that, it was only a matter of time before someone did this properly and the Debian flaw is the perfect opportunity to kick-start such an attack.

This is why I build any SSH keys on a totally seperate machine to the one they are going to be used on (physically and technologically), give them long passphrases and then keep them in as few places as absolutely necessary. If I'm the only one going to use them, then it's onto a USB key with a backup on the machine that created the key should the worse happen. Lose the USB key - you can't get in (except if it's a real emergency, when you can do it from a manually-created copy from the key-generating machine), but you absolutely **know** that someone else may have them and that you need to regenerate and change the keys immediately.

Oh, and install something that will absolutely go mad if someone gets in and does anything - you want to log every login, even if they immediately try to delete the logs, you want to log every significant file change, even if they try to immediately fix the checksums, disable the monitor program etc.

It's interesting, yes, but it probably won't hit critical mass. If it does, it'll only be among home and hobby machines and possibly the odd cheap hosting outfit. The problem is that we now have a warning about what its successor may do - and that could be a lot more nasty...

There are things that can be done

Once you can get access to a system, the whole security requirements changes.

There is no system in existence that will prevent apparantly authorised users from doing some damage on a system, but the degree of damage is what is important. Where Linux benefits is from a strong divide between normal and privileged access. Sure, if your private key AND IT'S PASSPHRASE are compromised, then someone can get access as you to your system. But this is just your non-privileged account, isn't it.

Of course, if lazy admin's directly access root using SSH, or have passphraseless SSH keys, or have sudo rules that allow them to cross the security divide without further confirmation, or store both private and public keys on their boundry systems, or use the same private key throughout their whole environment, then these fools deserve to get their systems compromised.

So here are the rules.

- Use a non-privileged account for initial access any system

- Su or sudo to obtain root access, but use additional authentication steps

- Don't use passphraseless SSH keys, unless you tie down what can be run (see SSH documentation).

- If possible, use hardware based authentication to secure private keys

- Guard your private keys like your life depens on them

- Don't store private keys on systems that do not need them

- Make sure that the permissions on your .ssh directory only allow your ID to see the private keys (I know ssh does some checks, but 0600 is best on files, and 0700 is best on directories)

- Use different keys in different parts of your organisation

- Consider using passwords with SSH (with strong change and strength rules) rather than SSH keys for very critical systems (really)

- Be careful about storing your private keys on shared Windows systems, or systems that have remote users with administrator access (consider portaputty and store the keys on an encrypted USB key).

- If you are really paranoid, regularly change your private and public keys on all your access boxes (please note it is NOT enough just to change the passphrase!)

If you follow these, then even if one of your private keys is stolen, then the amount of damage can be limited. As always, you can run a potentially secure system in a non-secure way. Security is only as strong as the weakest link, and this is often the sysadmins!

Oh, and by the way, if you want to see a file that cannot be seen using a hacked ls, try "echo *", or "find /etc -print". Or maybe use filename-completion in the shell. This is UNIX (or close to it) so there is nearly always more than one way to do something

some results ..

@E

i checked .. at least on latest available versions for Gentoo rkhunter chkrootkit at 5 am EST do not detect that particular rootkit .. at least it aint in the list that they check for.

bit amazing but that's what the program spits out in it's out on terminal.

bunch of rootkits .. but no phalanx.

still the solution is the same : dont need ssh on the public interface running ? shut it down.

btw .. that's an issue known for months and a non issue on a lot of

distros .also .. that aint the first rootkit .. that aint the last...

a lot of noise for much of nothing. :)

FuzzyTheBear

FuzzyTheBear ..

Shouldn't they already patched?

I'm relatively new to Linux, but it seems everytime there is vulnerability reported, Linux already patched the hole.

If the BOFH cannot patch "his" machine, then he is not BOFH.

virus-like?

"There's a viral aspect to this attack. As new SSH keys are stolen, new machines are potentially vulnerable to attack."

Bunk! That's the mark of a worm, not a virus. The biological behavioral analog of this attack is a bacterial infection, which is what defines a worm, not a virus.

Kindly remember that a computer virus is a malware code fragment that emulates a biological virus by embedding itself in an executable to activate it's payload and/or replicate itself, often using some of that executable's code as a means of getting control of machine functions.

The attack here is clearly a worm, because it spreads itself using the classic bacterial model - break in by compromising a well known authentication mechanism, copy in a tool kit, and consume the host system's resources primarily to replicate itself and install backdoors for later use.

Another clear sign that this is a worm, is that the rootkit uses hidden directories to hide in plain sight, rather than burrowing into the bodies of executable files, which is a much more difficult way to hide. File size & checksum scanners have been around for years, and will find that sort of thing if not properly done. Also increasing the difficulty level, most OS binaries on *nix are writable by root, so you have to compromise root before you can hide there.

I don't believe I'm explaining this stuff to Reg writers :-(

/gds

Maybe, just maybe..................

Maybe this RK was written by someone who is a lot smarter than some here suggest. Maybe, just maybe they wrote and released this version of the RK as a precursor to the real RK that does everything just right so that lazy admins and clueless users have one last chance to get it right. Or then again, maybe, just maybe it was released by Bill Gates and his crew to finally show all of the Linux zealots that they are not immune to the RK BS. Maybe, just maybe.................................................................................................

@AC 07:26

If this was exploiting the Debian SSH vuln, then no, it only affects Debian-derived Linux distros (that's Debian, all the Ubuntu variants, and a few also-rans; but *not* Red Hat, SUSE, Mandriva, Gentoo, Slackware, PCLOS, or any non-Linux system).

Re: Hmm

You're obviously not a Monk or you would know that all software sucks. It is just degree of suckage that defines viability, that degree being measured on the Lovelace (Linda, not Ada) scale. I'm shocked - shocked, I tell you! - that nobody has mentioned this.

rootkits

One thing that needs re-enforcing is that unless you have a security hole that allows a non-privileged user across the security divide, it is just NOT POSSIBLE to install a rootkit on a properly run Linux system. Rootkits, by their very nature, need to alter/add code to the kernel, libraries or modules that are used to run the system. And this needs root permissions. This is why it is very important to make sure that you do AS LITTLE AS POSSIBLE as root on a UNIX-like OS.

There are a number of well known ways to try to subvert a user currently logged on as root, but a reasonably savy sysadmin should be able to avoid these (you know, don't browse the internet as root, check that your path does not allow commands from the current directory, make sure that there are no executable script files with world-write, don't read mail as root, keep firm control of the permissions on directories in root's path, all the simple stuff).

If a rootkit cannot be installed, it cannot compromise your system, nor can it get access to SSH keys other than the user it is running as.

Please note that I am not saying Linux is totally secure, there have been, and will be in the future, code and design defects which could allow a system to be compromised. I firmly believe, however, that the open source model allows such things to be identified and fixed much more rapidly than a closed source model. Couple this with an effective notification and patch delivery system, and Linux just is more secure.

Contrast this to Windows, where many people by default use an account with Admin. privileges, or with the security notifications turned off? Asking for trouble as far as I can tell.

But the amazing thing is that the UNIX/Linux security and source model is decades old (I've been using UNIX for 30 years, and the Bell Labs. UNIX V6 and V7 code, and all the BSD code used to be available for inspection and modification to the academic community and others for at least that long). And Microsoft (who, in fact, have been UNIX licensees for at least 24 years - they did the original Xenix port to various architectures, before spinning off the original SCO) just don't seem to be able to learn.

Missing the point

I am sick of this crap. There is a real security problem here that is potentially very very large. And most of what I have read is finger pointing or people suggesting that either "now Linux is as bad as Windows", or it is the fault of bad, lazy people.

Fact of the matter is that this rootkit *is* international, it attacks one of the basic things we've all trusted for the past decade - SSL/SSH - and we all may as well face up to the fact.

So, I'll point a finger at the fules who claim to know the score and who tell us about who's to blame.

I know a *lot* of computer sci people, I know a *lot* of sysadmins, and I know about two or three people who really understand public key crypto when I ask them hard questions. For all you armchair security experts: public key crypto == underpinnings of ssh/ssl.

And, well, if you don't know basic XML syntax, you can go back to school, and good riddance to you.

re: Peter Gathercole, but in re-reading this screed before posting, in fairness only partly:

<rant and="fuck off and die if you don't like it" also="there are some fair points below">

Yes, Windows user as admin is a common thing and a problem. We know that XP makes the first user - if created as part of the install process - an administrator and that is a problem insofar as that user typically is, well, just a user. We also know there is a massive ecosystem (gahhh) devoted to keeping exactly that Windows user safe including liberal use of system modal dialog boxes and idiot-light style pop ups from the system tray. You've gotta be pretty stupid not to notice when your Windows A/V software thinks there is a problem.

But! on Linux there is no such infrastructure. Rootkit hunter apps are not even installed by default AFAICT, and I've used a lot of distros: RH, Fedora, SuSE, Debian, Slackware, Yellowdog.

So - fine. I'm an admin and every box I've built in the past two years was built from a recipe and the recipe included rkhunter and chkrootkit running as cron jobs. I know this because I wrote the recipe. Might I f*ck up and miss a step sometimes? Perhaps, but I go back and check.

You know, I'm careful but I am not perfect and neither is anyone else who raises the "lazy sysadmin" flag. And neither is anyone at all.

Way back in the day, before a lot of you people had outgrown your milk teeth, before kernel modules, it was known and commented on that the monolithic kernel was safe from device driver (read: kernel module) attacks. Don't bother replying with remarks about /dev/kmem: I have not said the monolithic kernel was safe from that. Modules happened anyway, and they had to because else we need distro kernels with support for every possible device - how would you like a 100MB kernel (+/-)? (or, by now in 2008, hundreds of install kernels with limited device support - Slack had maybe two dozen kernels at one point circa the early 90's before modules - I *knew* the hardware in my boxes and still I sometimes picked the wrong install kernel). But I suppose that just makes me a lazy sysadmin, eh?

So, here is a suggestion, and it is not meant as a finger of blame by any means. How about the makers of distros just add things like rkhunter and chkrootkit as standard software, installed by default and updated by default and run periodically via cron, with notifications sent to the "main user".

I'll leave defining the main user to the lazy sysadmin finger pointers - they ought to be able to figure that out, having already defined who's to blame elsewhere.

I do not think that the bulk of Linux users will be any better than the bulk of Windows users at actually ameliorating a detected rootkit.

Windows A/V software has however raised awareness to the point that I, as a sysadmin, do get asked by Windows users if such-and-such a Windows notice or dialog is a problem. I have never been asked by any Linux user about their suspected virus/rootkit/trojan/substitute-your-fave-noun.

People who say it's all up to the admin are either living in a world where the admin *is* actually God or they are dishonest. In academia anybody who can get grant money can build her/his own Linux box, and many do so, and most of them do not ask for support or inform anyone who might have a clue. Outside academia anybody who buys a computer can install Linux and he/she does so if so inclined. And, you know, WTF, why not, it's a free country, right?

How many of you arm chair security experts have, or have considered purchasing, a "Got Root?" t-shirt? How many of you have thought about exactly how very easy "got root" has become? Don't even bother replying - I'll just tell you to download an Ubuntu install image - remember that problem where the first XP user is an administrator? Grunt, grunt, uhhuhuuh, grunt, drool. *Don't* waste my time.

Is it just possible that, at some 5% or 10% market penetration outside the corporate controlled datacentre, Linux is a big enough and so very useful target to the black hats? Or that insofar as it is a multiuser-from-the-network-OS, very easy to install but not so easy to understand, it is far more attractive for black hats than something like Windows? Wake up, a**holes: Linux has grown up. It's time for the users and distros to do so too.

</rant>