back to article Paid secur-o-ware is generally better than free, but not always by a lot

Antivirus tests that assess the effectiveness of security products from the moment users visit infected websites have exposed widely differing performances among the various anti-malware products. The unsponsored tests by Dennis Technology Labs, which were run over a three-month period, revealed that the efficacy of paid-for …

COMMENTS

This topic is closed for new posts.
  1. Alex Walsh

    No Avast! on the home test? Shame.

    1. Ian McNee
      Stop

      Indeed...

      And why no comparison between the free and pay-for versions from companies like AVG, Avast and Avira? Also as other have said the failure to test products with widely-used non-MS browsers is a big gap.

      1. multipharious

        Re: Indeed...

        Failure to test non-Microsoft browsers is a gap? Have you noticed infection vectors lately? Adobe Flash and Oracle Java, with Java taking the clear lead this year.

        1. multipharious

          Re: Indeed...

          This test was not about vulnerability, it was about detection.

          For that they could have used a worst case scenario customer configuration with out of date Java and Flash. The end result is that very few if any of these vendors can protect against custom Zeus-class malware builds with undetectable signatures, which are as easy to generate as a mouseclick. This is where the reputation link scanner comes in. Does it protect against anything? In the end perhaps no, even if you keep WIndows Update on the one step ahead of the spider schedule. Visiting a website with pwnage is one SQL-injection away. And don't forget that if these cats can test it, just about any dildo who wants to make bank with a "crime pack" can too...but you see it is easier to test the custom builds for detection. I won't advertise for the paid service that does testing against all vendors with current defs.

          Oh hell. It cannot be that bad can it? Better go to bed.

      2. RICHTO
        Mushroom

        Re: Indeed...

        Same as with OSs then. Hardly a big surprise....

  2. Miek
    Unhappy

    No Avira on the test either? Avira would have wiped the floor with the other offerings!

  3. koolholio
    Stop

    Was this based upon virus signature or 'advanced' heuristics options?

    MSE actually uses a simplified version of a heuristics engine which I would rate quite highly, its virus signature database does let it down quite dramatically though in the advanced tests that I have performed against it.

    Lets take Mcafee and AVG for instance --- they are mainly, but not completely... virus signature based, so thus, what does one expect with these products?

    AVG added the heuristics feature a few years ago, and Mcafee have no doubt upgraded their modules/databases and heuristics.

    This is where alot of 'bog standard' anti-malware/spyware/adware 'security package' users arent clued up?

  4. Timo
    Unhappy

    Used to think Microsoft Security Essentials was OK

    But not after this article. Well and I just spent all of last week cleaning rootkits and mass-mailing java viruses off of my neighbors machine that was running MSE. They've been pwned for months, and MSE didn't find much. I understand that a rootkit would hide itself while running, but even plugging the disk into another (healthy) machine and scanning from there with MSE failed to discover much of anything. Nor Microsoft's boot-disk scanner, nor their Malicious Software Removal Tool.

    Microsoft is clearly falling behind, not sure why.

    1. koolholio
      Facepalm

      Re: Used to think Microsoft Security Essentials was OK

      rootkits may only come into play, by a signature hole or an exploitable hole that hasnt been patched?

      Running of arbitrary code (to escalate code to root privelleges) comes after the initial exploit?

      The TDSS rootkit wasnt picked up by many, although kaspersky were one of the first to release a 'removal tool'?

    2. Anonymous Coward
      Anonymous Coward

      Used to think Microsoft Security Essentials was OK

      Umm, don't you think it's a tad naive to expect sterling performance from the company that gave you the problem in the first place? Microsoft has been using anti-virus as a band aid to cover up their security model deficiencies since Windows 95.

      Personally, I have been using Kaspersky for years - they are less prone to leave law enforcement backdoors - until I abandoned the Windows platform altogether. I only have a small VM left which gets started every 2 weeks (and then needs a good half hour to catch up with all the patching). .

      1. Davidoff
        FAIL

        from the company that gave you the problem in the first place

        "Umm, don't you think it's a tad naive to expect sterling performance from the company that gave you the problem in the first place?"

        What a stupid statement. MS doesn't create malware, so how can they have given him the problem in the first place?

        "Microsoft has been using anti-virus as a band aid to cover up their security model deficiencies since Windows 95."

        Yeah, right, since only Windows can get malware.

        1. Ian 55

          Re: from the company that gave you the problem in the first place

          They gave him the problem by being incompetent and/or not caring about security from the start.

          As they were primarily concerned with single user systems, it's perhaps understandable that security came somewhere below 'what color toilet paper shall we buy' in the list of Microsoft's priorities, even if it isn't forgivable.

          How many years have most users (outside corporate locked down envirnoments) been effectively forced to have admin rights? Programs written for their bought-in DOS had to run under Windows 1 so it couldn't restrict rights properly, and programs written for Windows 1 had to run under Windows 2 so it couldn't restrict rights properly, and.. repeat until infected. Being judged on program speed and size didn't help.

    3. Elmwoodie
      Go

      Re: Used to think Microsoft Security Essentials was OK

      I dont know who Dennis is. I do know who Virus Bulletin is. They say MS Essentials is catching 100% of "in the wild" malware. They also do more realistic testing with platforms I use and recommend. WinXP SP3 with no additional patches and IE7 and known bad versions of third party software is not a realistic test bed. Dennis did indicate that they MSE did have the lowest (or second lowest) rate of False positives which is important. I think there are some serious issues with Dennis' methodology.

      The reason I uninstall most of these paid packages from any PC i see is the terrible performance impact. And according to every other report I have seen, their performance IS NOT BETTER than MSE.

      http://www.virusbtn.com/vb100/latest_comparative/index

      1. Anonymous Coward
        Anonymous Coward

        Re: Used to think Microsoft Security Essentials was OK

        Using only MSE at home (and obvious precautions re Java and scripting, and avoiding IE) I've had no problems at all. And that's as an AC who enjoys the ample reserves of free grumble on t'web. I reckon I frequently visit sites that are more than likely minging with malware of every description. They certainly have redirects to lord knows where (blocked by my settings), and often curiously long scripts trying to launch vids (again blocked). I'm sure there's every drive by you could name on some of these sites, and more besides. I do recognise that non-grumble web sites are also a risk but I'm just making the point that I'm using MSE on sites that clearly present a higher risk.

        In reponse to the obvious retort "that's because you'd never know and your machine is compromised", I do occaisional runs of third party scanners (including the "perfect" Kaspersky), none of which have picked up anything. No reports from friends of spam, or other indicators of trouble.

        No anti-malware solution is going to be 100%, but I don't buy that nonsense from Dennis Labs that MSE is pants. Looking at the full reports, I wonder how Dennis Labs make their money? Could it be that a certain "premium partner" funds all of this? No, surely not.

        Recommended approach: Patch everything, set anything that can auto-update to do so. Use MSE (or the AV suite of your choice), use any decent non-IE browser 9the lower the market share the better), use No-script or similar, use Ghostery; browser set to not keep history, cookies, or content. Use a non-Flash media player by default.

        May be a few things I've forgotten, but paranoia is the word.

  5. koolholio
    FAIL

    Norton and Kaspersky et all

    Norton for certain does have lesser heuristics detection than other companies, Kaspersky has better detection rates, but I'd say they still are most likely 'vulnerable' to certain variants/packers.

    It might be an idea to look at a fully comprehensive independent test operated by av-comparatives?

    http://www.av-comparatives.org/

    I happen to believe them more than this report. Go compare! hic hic!

  6. TRT Silver badge

    I've been using a combination of Comodo AV and MSE, with a Comodo Dragon browser. It seems to have a much lower overhead on system resources than the corporate approved "McAfee Enterprise".

    The test conditions are also a little rough. Win XP sp3 and IE7? I know, it's a highly infectable combination. MSE does well, IMHO on Win 7 with a non-MS browser.

    1. koolholio
      Boffin

      Im suprised that combination works and hasnt detected each others virus signature databases as containing a virus lol

      IE7 is older than IE9 or IE10, I hope you've updated Flash and Java? else you may aswell not have bothered much?

      A non MS browser will not protect an operating system or its network from other methods of attack? :-/

      1. TRT Silver badge

        Oh, and it's not me running IE7, I was referring to the test bed used in the article. Using a non-MS browser is purely a matter of market saturation anyway - presenting a narrower target to an attacker.

    2. Anonymous Coward
      Anonymous Coward

      Please don't tell me....

      ...your running two AV products, that is a total no no....

      1. TRT Silver badge

        Re: Please don't tell me....

        So conventional wisdom would have use believe, but the old "dual-anti-biotic" approach does have some benefits. I've found no noticeable slowing up with this combination of products. Run Symantec and McAfee together and you're going to have a bad time, but MSE seems to play nice with a second package.

        The way that the virus signature databases work wouldn't give you false positives - they only contain signature viral code fragments and even then those are usually simple checksums.

      2. TRT Silver badge
        1. multipharious

          Re: Please don't tell me....

          This test wasn't about vulnerability, it was about detection.

    3. david 12 Silver badge

      Comodo and MSE

      Microsoft Security Essentials does not do On-Demand scanning if it detects another AV doing On-Demand scanning.

      So (1) It is not twice as good as using only one AV system, and (2) it is not twice as bas as using only on AV system.

  7. El Bertle

    I think the article is slightly misleading in that it does not include the system spec used, and I would expect MS to put a portion of its AV effort into OS patches rather than their add-on product.

    7.1 The targets

    Each product was installed on a clean Windows XP Professional target system. The operating system was updated with Windows XP Service Pack 3 (SP3), although no later patches or updates were applied.

    A selection of legitimate but old software was preinstalled on the target systems. They included out of date versions of Adobe Flash Player and Adobe Reader.

  8. Vitani

    Hmm, time to install that free copy of Kaspersky which Barclays keep insisting I install...

    1. Can't think of anything witty...

      I used to run kaspersky, but i felt that it had a really high overhead on my system, reducing it to a crawl for about 5 minutes everytime that i booted it. I switched to MSSE when the Kaspersky licence ran out and i don't think that i have had any problems since... Certainly, Windows runs a lot faster.

      A tricky trade off...

  9. Anonymous Coward
    Anonymous Coward

    Symantec Endpoint Protection

    Having taken on the administration of SEP within our organisation I find it to be a bug ridden piece of crap:

    1) The reputation based scanning seems to slow some software start up times by over 90%

    2) If their support can't resolve an issue it results in a response to turn the feature off (lowering detection possibilities)

    3) Product testing seems to be completed once it's been rolled out judging by the number of "there is a fix for that in the next release" type of responses

    4) Marking a call as high priority results in a response from them within 2 days if you are lucky

    Interesting article though as I was considering evaluating the MS Forefront Endpoint Protection

    1. david 12 Silver badge

      Re:MS Forefront Endpoint Protection

      The article compared free AV to paid AV, and found that paid AV is better. MS Forefront Endpoint Protection is paid AV.

      Since part of what you get with MS Forefront Endpoint Protection is better reputation-based protection, presumably it falls into the 'paid AV is better' group.

    2. Anonymous Coward
      Anonymous Coward

      Re: Symantec Endpoint Protection

      Have to agree SEP was very buggy, although I havent administered an SEP system for over a year (Outsourced :().

      I still can't understand how it got triple AAA, seen two machines in the past two weeks with scareware completly missed by it even though the scareware is at least 6 months old (FBI warnings or Mi5 warnings).

      1. Anonymous Coward
        Anonymous Coward

        Re: Symantec Endpoint Protection

        I have long abandoned the use of anything that has been near Symantec or Norton.

        I'm not sure what they do to their products or maybe it's me installing it wrong (default install, so I can't see that being an issue), but installing their stuff on a high end system is a bit like driving a Porsche around with the handbrake on full.

    3. Anonymous Coward
      Anonymous Coward

      Re: Symantec Endpoint Protection

      > 1) The reputation based scanning seems to slow some software start up times by over 90%

      Do you mean it nearly doubles the software start up time (to 190% of the original start time) or do you really mean speed is reduced by 90% to 10% of the original so it will take ten times as long to start?

  10. Uncle Slacky Silver badge
    Linux

    Wot no mention of...

    ...Linux yet?

  11. Anonymous Coward
    Pint

    Security isn't a dirty word, but performance is for some of these.

    I find that it's a compromise between performance and security, particularly on lower-end or older machines. There's no way I would install Norton, for example, due to the brutal effects it seems to have on even modern quad-core computers running Windows 7. Slowdown is noticeable. Likewise AVG - pretty good performance up until the latest release.

  12. Rabbit80

    Performance

    This happened today...

    The company I work for processes batch production records for a food manufacturing firm. We return the data in a specific format on DVD contained within a self-extracting RAR file. This file today contained approx 70,000 tiffs and xml files compressed and the total size was approx 2.6Gb.

    Recently, they have switched from McAfee to Kaspersky.

    Using McAfee, the file would open the self extraction window in less than a minute, today, using Kaspersky the file took over 30 mins to open - copying it to HDD first didn't help.

    We use ESET as we find it has an excellent memory footprint and lightning fast scan times. Both Kaspersky and Symantec / Norton cause us massive issues as we typically have 100M+ files on our servers at any given time and they simply kill our servers when running a scan.

    Moral of the story - sometimes scanning speed is more important than detection rates for rarely seen wild viruses.

    1. Anonymous Coward
      Anonymous Coward

      Re: Performance

      I assume you were using the McAfee Enterprise Product?

      If so I know for a fact you can customise every type of scanning including not scanning compressed archives over a certain size (Which is set by default in the ePolicy console). As a home ESET user I do remember seeing the same option where its set not to scan archives over a certain size, I assume therefore Kasperspy is trying to scan the archive .

      1. Rabbit80

        Re: Performance

        Sorry - maybe I wasn't exactly clear. It is our client (the food mfr) that was using McAfee (and has switched to Kaspersky) - and yes, they are both the enterprise products! I didn't know about the default option not to scan over a certain size of archive - and yes, Kaspersky is attempting to scan the file.

        My point is simply this - Due to the settings in Kaspersky, and the slow scan times, I ended up spending over 1 hour (possibly even two hours+) on our customers site longer than I needed to. This was partly due to an error in our data which meant I had to extract everything a second time. This translates into real costs for my company. (We were not charging for my time since we have just had a number of internal processes change which meant the chance of an error with the data was high and is the only reason I was on site in the first place!)

  13. Bucky 2
    Coat

    MacAfee, some years ago....

    I remember some years ago already, I got a new laptop which had MacAfee pre-installed.

    The first thing I did was create a limited account for daily use.

    Surprise. MacAfee started throwing errors. It seemed that MacAfee wouldn't run unless you were running as a user with Administrative access.

    So I uninstalled it and forgot about it.

    I'm sure they've improved their product since then (though I guess they've failed to produced a passing grade in this most recent test), but I still remember my astonishment at the program's reaction to a garden-variety first-tier approach to securing one's computer.

    Thought I'd share....

    Right. I'll get my coat.

    1. sjsmoto
      Thumb Down

      Re: MacAfee, some years ago....

      I stopped using MacAfee after it would trundle for an hour just so it could determine the estimated time it would take to perform the scan.

  14. h3

    The thing about security esseentials (Windows 8 Defender) is it doesn't spam you all the time or waste too much in the way of system resources.

    With Smartscreen on Windows 8 you have to force it to let you install pretty much anything. (Legit or otherwise)

    I don't bother with it.

    The combination of Malwarebytes and Security Essentials likely performs allot better than the spammy annoying ones.

    It means nothing they don't even list which variants they used. So you couldn't attempt to verify the results.

    They say it is not sponsored but they don't provide everything needed to verify the results. (Or even proper data about the methods used).

    Or even names of pieces of malware.

    (Rootkits are basically not a problem for 64 bit Windows even less so with secure boot).

    No false positives is one of the most important features as well.

    If you have ever dealt with Symantec Support you wouldn't use them for anything regardless.

    Most of the malware these days is really poor quality. (Excluding the stuff like stuxnet). It is obvious it is on your system just because of harddrive thrashing.

    Having a daily backup (Moderately recent system image) is good enough.

    MSE + MalwareBytes (Just the free MalwareBytes unless you are really careless - quick manual scan of anything you suspect to be dodgy).

    There is nothing about resource usage or how annoying they are either.

    I could make up a nonsense report like that without actually doing any testing.

    1. RICHTO
      Mushroom

      Root kits certainly are a problem for 64 bit Windows 7. Windows 8 maybe not - If your hardware supports secure boot.

  15. h3

    Even more obvious here :

    http://www.infoworld.com/d/wp/dennis-technology-labs-effectiveness-in-virtual-environments-201714

    Dennis Technology Labs

    Sponsor Symantec

  16. RonWheeler

    Security Essentials doesn't trash your PC

    Most 'modern' AV products are massively overaggressive and resource hogging for a sensible user who a: sits behind a passable NAT router and b: doesn't take click random 'fix my registry fast!' downloaded .exe, / hit pron sites all day / use IE, and so on.

  17. Davidoff
    Mushroom

    This test should be taken with more than a grain of salt!

    Aside from the fact that Dennis Labs is a nobody in the computer security field, people should be very careful to take their test results as facts.

    Dennis Labs says on their website:

    "Dennis Technology Labs started testing for vendors in 2008 and has been conducting tests for magazines, including the UK's leading IT title Computer Shopper, since 2002."

    The 'UK's leading IT title Computer Shopper' is what is usually called 'infomercial' (advertising which should look like independent information), paid for by their advertisement customers. Computer Shopper is published by Dennis Publishing Ltd. If the name sounds familiar: yes, Dennis Publishing Ltd also owns Dennis Labs.

    Aside from the very strange test environment (unpatched XP SP3 system with outdated software) this 'test' is very likely rigged to favour their advertising customers.

    Why this is worth of an El Reg article is beyond me. There are much better and well established independent sources for evaluating computer security products like av-test.org. Their results also contradict Denis Labs' conclusion that any 3rd party product is better than Microsoft Security Essential.

    1. Ian 55

      Re: This test should be taken with more than a grain of salt!

      I wouldn't say it was a strange set up, I'd say it was typical of what's out there. Not everyone spends their life downloading and installing patches.

  18. Peter Murphy
    WTF?

    Call me naive

    I'm reading this sentence:

    Antivirus tests that assess the effectiveness of security products from the moment users visit infected websites have exposed widely differing performances among the various anti-malware products.

    But what if your browser blocks the sites for you? Firefox does that a lot. Do you have to fork out money for the anti-malware product, or would it be a waste?

  19. FrankAlphaXII
    Alert

    I use MSE

    And Ive never had an issue since I started using it, but I also use Clam on both my windows and Linux machines. Defense in depth is something that everyone ought to be practicing. Its just a shame that some AV engines still conflict with one another, they need to sort that out.

    Symantec's programs suck nowadays, which is a shame because they used to be really good like 10 years ago, I'm not permitted to use Kaspersky (Its controlled and co-developed by the Russian Foreign Intelligence Service, or SVR for short, and Eugene Kaspersky was a KGB officer. So as far as current Counterintelligence Policy is concerned, its tantamount to making your security dependent on the Kremlin, and the Military isn't NASA, we're not about to make ourselves reliant on the Russian Foreign Intelligence Service for information security) on any machine that I use to connect to Army Knowledge Online or Intellipedia, so thats out which sucks because its a good product, even if not entirely trustworthy.

    McAfee I simply wont go anywhere near, even though my ISP, my credit union, and the Army will give it to me for free*. So I went for it about a year ago and I had the same issue with it taking an hour and a half to only figure out how long the scan would take, when even Norton 360 didn't take that for a full system scan. You can't exactly say that 360, Norton Internet Security, or even Norton Antivirus are quick either, so that's pretty piss poor on McAfee's part.

    *- Just to clarify, the US Army also has Norton product keys for free for Soldiers, but unfortunately not Symantec branded products, which Ive never quite understood. I figure that buying an enterprise license for Endpoint Security would probably be much cheaper for DA, considering we have 766,984 Soldiers in the Regular Army and Army Reserve. Almost all of them have at least one computer. But then again Acquisitions Corps/Materiel Command functions strangely, and anything involving an Information Systems purchase also has to go through the Defense Information Services Agency, as well as the Central Security Service (the Organization inside the National Security Agency that coordinates the various Service Cryptologic Elements like the Army's Intelligence and Security Command, the Navy's Tenth Fleet/Information Operations Directorate, Marine Corps Intelligence Activity, and the Air Force's Intelligence, Surveillance, and Reconnaissance Agency)

  20. DF118
    Paris Hilton

    Somewhat of a fail in the number of products tested.

    One might, on a hurried reading of this article, be given to believe the results mean certain products are the best money can buy when in fact that may not quite be the case. [/understatement]

  21. ScottishYorkshireMan
    Stop

    Are only those who advertise with Dennis publishing the ones that get reviewed?

    Is there any bias in this test? There are so many excellent anti virus products that aren't even mentioned in this test so I think the grain of salt is probably quite accurate. For a real breakdown of anti virus software I have always worked off the recommendations of the Virus Bulletin Awards or VB100 site at http://www.virusbtn.com/vb100/index

    Far more concise and seems to include all the players, not just those companies with £M's to spend in advertising.

  22. Tony Rogers
    Happy

    Felix Dennis is spending an enormous amount of time planting trees and forests.

    It is not totally surprising to me that his "lab staff" cannot see the wood for the trees !

    I am sure his new AV lab resources will one day match the quality of his magazines ???

  23. Anonymous Coward 15

    I'm not using the reputation-based stuff

    I don't want my browsing data sent to the AV company.

  24. Anonymous Coward
    Anonymous Coward

    Norton is the best AV

    and <insert arms manufacturer here> is the best way of shooting yourself in the head.

  25. Elmwoodie
    FAIL

    This misleading story is based on a report with a methodology issue

    I agree with the Dennis reports conclusion: If you run XP SP3 with no additional patches, IE7,old versions of Flash and Adobe reader and do not care about performance, Microsoft Security Essentials aint your solution.

    This platform was picked " due to the high prevalence of internet threats that rely on this combination." The "logic" behind this statement in the report boggles me: "The prevalence of these threats suggests that there are many systems with this level of patching currently connected to the internet." Wikipedia indicates that the most common IE version on the web for at least the last two years has been IE 8, not IE 7. In 2012 it might be IE9.

    I suggest that the prevalence of these threats is because they are possible and well known, and have nothing to do with the number of potential targets.

    This misleading story is based on a report with a methodology issue.

This topic is closed for new posts.

Other stories you might like