Sign in / up

back to article Security researchers plot revamped anti-virus tests

Security researchers are close to formulating plans to overhaul anti-virus testing amid growing concerns that current tests can be misleading. Anti-virus packages are traditionally tested for their effectiveness in detecting a sample of malware packages known to be in circulation. Products that fail to detect a sample get a …

COMMENTS

House rules Send corrections
This topic is closed for new posts.
  1. Ross

    Full circle

    Wow, looks like we've gone full circle.

    I remember my anti-virus software on my Win 3.1/DOS box detecting viruses with both signatures and heuristics. It was fairly decent at it too. Of course back then anti virus was actually a technical challenge what with viruses deploying stealth techniques like polymorphism, IAT hooking etc rather than an exercise in generating hashes of files.

    0 0
  2. Dr. Vesselin Bontchev

    Good, but not good enough

    The proposal is a step in the right direction - but it is not good enough. If the tests are conducted as the proposal suggests, they will essentially test obsolete (months-old) heuristic scanning capability. This isn't good enough, although it's better than the current situation.

    The proper way to do it is to install an up-to-date version of the AV software and attack it with *live* malware. Don't just let it scan a "dead" collection of samples. Start executing the malware - and then see if *any* part of the package manages to prevent (completely!) the infection of the test machine. It doesn't matter whether this is the scanner, the heuristic analyzer, the behavior blocker, or anything else that stops the malware - it's sufficient if the malware is stopped.

    0 0
  3. Brian Gladstein

    There's the problem with the antivirus industry

    If you ask me - this is what is wrong with the antivirus / endpoint security industry today. Too many people patting themselves on the back for fighting malware, and not attention paid to real-world effectiveness. This article just sent me off on a rant - http://bit9.com/blog/home/tabid/15398/bid/2456/Antivirus-Protecting-Against-Yesterday-s-Malware.aspx

    0 0
  4. Anonymous Coward
    Anonymous Coward

    Up-to-date heuristics/out-of-date signatures

    Is it not possible to mate, for the purposes of testing, an old or even blank/minimal signature file with the latest heuristic engine if you want to test the capacity of the heuristics to detect threats?

    0 0
  5. David Wilkinson
    Thumb Up

    It sounds like a great idea to me.

    They want to test the ability to protect against previously unknown threats.

    The best way to obtain test against unknown threats would bet to travel one week into the future and obtain the latest real world nasties.

    However until they get their time travel machine working, they decided to do the next best thing.

    Today's threats vs AV software that has been frozen in time for a week.

    As far as it being unfair because the AV software doesn't have the latest updates, I wish I lived in a world where AV software became dramatically more effective on a week to week basis. :)

    0 0
  6. Dr. Vesselin Bontchev

    Blank signature file

    Anonymous Coward: Yes, theoretically, it is possible. In practice, however, it is not. First of all, practically no AV vendor will supply you with a "blank signature file". We (F-PROT) used to do it only for our macro malware signatures and nowadays even we don't do it any more. Furthermore, the term "signature" is misleading. Contrary to popular belief, it's not a collection of scan strings for known malware. Nowadays it is a complex database containing whole programs for detecting malware. Often even the scanning engine of the AV product is updated by this database. So, if you use an old database, you're running the risk of using an old (even buggy) AV engine.

    0 0
This topic is closed for new posts.