Enterprise customers of a widely used McAfee anti-virus product were in a world of hurt on Wednesday after an update caused large swaths of their machines to become completely inoperable. The problem started around 2 pm GMT when McAfee pushed out DAT 5958 to users of VirusScan Enterprise. The virus definition falsely identifies …
How many times does this have to happen
And for those who can't reboot to apply the new file?
How to fix if you can't get update from server
Okay, download the McAfee update DAT from McAfee site, then log onto the machine with the problem and put the file into C:\Program Files\Common Files\McAfee\Engine. Reboot machine. All should be good again :)
If the machines cannot boot, having had svchost removed from their OSes, how would you boot far enough to install that file? If you had 5000 desks, that would be pretty rotten having to do each one manually - this is a pretty bad f*ck-up, it could massively wound McAfee :-(
How about the millions of home users now with NO Net access
So, how are the millions of home users with XP and McAfee that now don't have ANY Net access going to be able to download the updated Dat file? Will McAffee identify them all from their subscription data and post the file to them ???
wound it? need a mercy killing
'it could massively wound McAfee :-('
lets hope it kills the ugly beast; nearly as useless as Norton FFS
Hours of fun
We've been bitten by this. The immediate response of our IT people was to tell everyone to start yanking network cables - fair enough, as it looked like a day-0 worm spreading like wildfire across all our sites.
Ironically, it's only people not at their desks or bloody-minded enough to ignore IT that have survived, because their machines were still on the network for the virus definition rollback. There are hundreds of PCs that are going to require a bit of TLC to fix because they don't boot far enough to be fixed remotely.
Me? I'm a smug Mac/Linux admin.
I would have thought that.....
Best practice would be to be professionally paranoid and quarantine all patches for critical software like O/S and core apps until it has emerged that there are no show stopping gotchas or the patches have been tested. Still wtf do I know.
ahem.. companies using mcafee should be using a Mcafee EPO (e-policy orchestrator) server. with this you can delay mcafee updates being served to the client machines, which is safe practice.. because it's not the first time such a thing has happened..
You missed a bit
"with this" [Any Half Competent BOFH] "can delay..."
Our company uses EPO and still got hit :( .
You have a trade off between the potential of the AV updates to cause problems and the potential of not releasing the updates to allow a new virus to spread through the network.
To test every DAT file quickly enough you pretty much have to have someone dedicated to doing that on a daily basis. It has to be tested on every variation of machine you have, every OS, every OS level, every critical app. We quarantine engine and product updates, but not DAT files, we simply don't have the resources to test them and get them out quickly enough to avoid the potential risks of un-patched machines.
We could of course use the "previous" branch in ePO to update, then we'd have time to delete the DAT's from current if problems are reported. But again the problem then is that if a new virus gets into the network and we don't have the latest DAT's it can cause far worse problems.
On the plus side, at least now I have more fuel to use in my recommendation that it's time to ditch McAfee.
That's great but...
I only got an email from McAfee at 9:30pm last night informing me of the problem, which of course had already affected a lot of our machines.
Not really a solution
(1) what is a "safe" waiting time?
(2) if you delay reports of disasters, isn't the overall population in the same spot?
(To wit: now EPO users less likely to be hit, others more likely; so everybody else installs EPO with same values; so population ends up as initially, just slightly longer infective for viruses due to delayed definition install).
I think you misunderstood me. I was talking about this from the point of view of a sysadmin setting policy for the rolling out of such potentially troublesome patches across an enterprise, rather than, in this case from the point of view of McAfee.
Frakk!! When are those of you using this going to stop paying for this POS?
Yeah it's a Piece of shit !
Doug fully agree with you !
My wife's machine has died tonight of the same ailment. Might be able to get it back but really not hopeful !
What you really have is POS security/antivirus running on POS o/s. True recipe for disaster.
The lesson to be learnt ? Don't use m$ shitware in the first place for mission critical services. That way you don't have to rely on retards like mc-crappy to fuck things up even further for you !
Stop using it?
...as soon as management lets us buy something else. ;-)
It seems like every other month that you are writing about one cock up or another concerning this anti virus software why do people still use it?
an afternoon of fun
yep. the day was going well until about 14.25. then it all went Pete Tong. been a rather interesting
last few hours at work. we took multiple steps to stop windows systems from getting the DAT file without just pulling the internet plug. sort of worked...we estimate just 400 machines need sorting out - better than the c. 8000 it could have been.
McAfee is a virus
Several times over the last couple of years I've had to get the BOFHs at work to fix McAfee inflicted damage on the pc in my office that runs windows. I wouldn't be surprised if there were other times where McAfee screwed up but the BOFHs fixed it before I found out.
False positives forced me to abandon McAfee for Avira years ago. Once identified, the module would go into quarantine with no way to use it except to turn McAfee off completely. McAfee had no mechanism for me to report a false positive, instead telling me to boot a repair disk and scan the system again, fruitlessly. Avira lets you ignore a false positive and continue to use the module, and allows you to submit the module for analysis which, once found to be false, is fixed in a day or two. What a difference!
Not the best
I managed to get mine out of the reboot cycle and back up and working by disabling all McAfee services via Safe Mode and registry editing (Network Polices prevents the Service Manager from doing it).
Some other guys in the office reported svchost.exe was deleted by it (ouch) and were less lucky.
Why won't our sys admins get avast :(.
Looks like tomorrow could be a fun day at work! Lets hope my AV server has some how managed to not download this update and fire it around the network.
Where's my hip flask...
Stand and deliver...
I pity the poor IT dept that has to use that load of rubbish. It's bad enough at home having that ransomware on your machine, with pop-ups appearing all the time saying "pay up or your computer gets it!". Isn't there a more grown-up anti-virus that enterprise users can take advantage of?
That would explain
Why our internet proxy server went the way of Simon then. Should be fun at work tomorrow as all the computers go *foop*.
Think someone at McAfee is getting MSCE soon
...if only just so they can realistically determine which files are critical Windows system files.
After leaving work at 7.30pm with still lots of machines down and critical deadlines approaching, I think I can join in the movement to hang McAfee from the nearest tree. If it was free, or even cheap, it could be forgiven. But its not. Bugger them all to hell.
GOOD JOB MCAFEE
MCAFEE basically sent a virus out to their entire customer base! MORONS! If i had the decision power behind our software selection for antivirus, i would DUMP THIS PROGRAM!
Where is their CHANGE MANAGEMENT process? Where is there IMPLEMENTATION REVIEW process?
I would not be surprised if MCAFEE loses a crap load of customers over this. Their stock is already down .20 cents today. not enough if you ask me. but this is my opinion.
20 cents.. not far enough!
Yeah... .5%... big whoop.. about 50% of my day was crap! Might have to seriously investigate Linux
Um, we had this impact 20+ machines before we worked out what was wrong.
I wrote up some removal instructions here:
Good luck eveyone!
Fix for the 5958 DAT problems
There is an easy fix for these problems once the machine has been 'bricked'
Details are available over on my blog:
O NOES! Is ePolicyOrchestrator...
...poised like the Sword of Damocles over every naughty Windows system file in your network?
Secret of timing is comedy!
Always nice to make InfoSec a little more... sporting, eh?
Good work Fellas! :P
What is the best alternative to McAfee?
Had enough of this shit
"What is the best alternative to McAfee?"
(or a mac before I get really flamed)
Best alternative to McAfee?
How about an Operating System where little things like privilege separation and non-executable files are baked in, rather than crude hacks bolted on from the outside.
And a culture where Source Code is passed around, shared and re-used; as opposed to treated as though it were allergic to daylight, with the consequence that everybody is forced to rewrite common functions from scratch, occasionally missing an awkward edge case.
i find bit defender extremly good
but meh some liek vanilla some like chocolate
Never had a bug come through with bit defender .. yet
mines the one with the anti spam lining
Evolution at Work
Old dinosaur companies that are too slow to respond to years of failure by McAfee are now being removed from the breeding pool. Why do people keep buying this crap? Its not even like its bought and paid for, you have to ante up every year.
Ever since AVG 8 turned my computer into a POS I've been surfing naked. I've never had a virus scanner find an actual virus since the days of the STONED virus that spread on floppy in the early 90's.
RE: Evolution at Work
I know I shouldn't feed the troll, but here goes anyway...
I ain't McAfee's biggest fan - truth be told I ain't a fan of them at all - so I'm not trying to defend them nor any of their competitors, but if you're not using any AV software then how do you know you've not been infected?
I think user education is more important than any software solution, and I do agree with your recommendation of using a firewaa to minimise exposure, but I'd not rely on the firewall and a Flash-free browser alone to ensure I was virus free.
Not that I really care as I don't use Windows on my own PCs anyway, but I do have to use this abomination of an OS in the workplace.
Surfing naked ?
Please, please, please... put some duct tape over your webcam then.
PLease check out Ubuntu
If unprotected your machine will get infected - and it's then used to attack our servers. This is why many sysadmins are really p***** off with MS.
You'll find Ubuntu to be all that's needed.
I ceased to use McAffee in 1994, when it successfully destroyed NATAS. .. only to curl up and die because of an "unknown" virus. That "unknown" was DIR II.
I reverted to MS Antivirus back then (remember CPAV? MS bought them!), and later to Norton. I'm currently using avast!, though I had a brush with ZoneAlarm/Checkpoint... until they also brought upon me a bad false positive. Whoops!
Remind me again
Why people actually choose windows.
Signed a happy solaris "downtime, what's that?" user.
What's all the fuss ?
No problem with OpenSuse here.
No problem with Windows, either, AVG-using freetard that I am.
I am now thinking about all the people I know of that have told me they couldn't be botherd with changing thier anti-virus that came with the machine or who say they can only rely on the 'big boys'.
Well that was an afternoon wasted
At first we thought a virus had hit our Domain controller and pused out to all the boxes. So everyone assumed the best way to avoid it was to update Mcfee..FAIL
I feel sorry for the IT bods, they will be having to manually fix a couple of hundred network PC's over the next couple of days ! :S
Someone please sue them....
I fail to see how they could defend any legal action.
It would appear even the most basic testing should have picked up it canning a windows system file.
Go on someone please take them to court for your costs caused by this update. That way they might actually do their job properly.
Personally I stopped using their software quite a few years ago (having been a fan for quite a few before) as I started having problems with it.
I've used AVG ever since, never had any issues with their software or any infections.
Yay.. what fun. So I've stopped the reboots... and now somehow sound does not work and various programs just gave up. It was crazy to come into the office today and have everyone gone.. guess they just gave up and said hey nice weather... wish I could have done that.. but being the drudgen that I am.. I cannot.
"Bricking" reduces the utility of a computing device to that of a brick. It happens to game consoles and shitty phones that are so locked down that software bugs can render them unusable. But how the hell do you brick an average computer? Okay, maybe this means that you can't boot your primary OS. Does it not still boot from other partitions or devices?
Call me pedantic, but I don't think a device is a brick if you can have it mostly recovered, by yourself, by the end of the day.
When is a brick not a brick
As most electronic devices have a flashable bios of some sort it's likely that most devices, e.g. PSP, etc, could have the chip removed, reflashed good and replaced. Or just replaced.
Hence also not a brick. Depends on the lengths to which you wanna go.
It is a brick until it is not a brick.
I've bricked a system before, not a happy experience. These are not bricked, a quick BIOS change and a Knoppix CD gets you out of most continuous reboot sequences. And allows you to mangle McAfee so it won't start... And allows you to replace svchost.exe... Or whatever else file McAfee decides to eat for lunch that day.
Besides, I thought everyone had shut off that "Reboot on serious error" cruft that Windows XP ships with after the first bad XP patch got pushed out.