AV Performance Statistics

Discussion in 'other anti-virus software' started by Blackcat, Jan 18, 2007.

Thread Status:
Not open for further replies.
  1. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,010
    Location:
    Christchurch, UK
    "The statistics provided here are an indication of the ability of the AntiVirus system's ability to deal with near 0-day infections".

    Performance of 29 antivirus engines displayed.
     
  2. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    So Prevx is better at this then Kav and Nod.:rolleyes:

    Dont you just love tests.:rolleyes:

    And this from a Spam Fingerprint Identification Service.
     
  3. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,010
    Location:
    Christchurch, UK
    Just don't shoot the messenger!
     
  4. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    No problem, you are entitled to post what you find and no one can fault you for that my friend.
     
  5. pykko

    pykko Registered Member

    Joined:
    Apr 27, 2005
    Posts:
    2,236
    Location:
    Romania...and walking to heaven
    from NOD32 detection: error occurred while reading archive :eek: :eek: Is this recorded as detection? :doubt:
     
  6. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    Updated Hourly? o_O

    That would mean its different from your ordinary tests, maybe something like Jotti's....and in that case one can't say its reliable unless we get to look at the methodology.
     
  7. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    So basically its like uploading the sample to VirusTotal upon finding a new sample for the first time and seeing which AV already has a signature/heuristic detection for it?

    And CastleCops is involved? o_O

    I do not think this is an effective way to test zero day protection, but maybe the test condition can be further clarified...:doubt:
     
  8. pykko

    pykko Registered Member

    Joined:
    Apr 27, 2005
    Posts:
    2,236
    Location:
    Romania...and walking to heaven
    well, it seems to be updated daily more likely. :D The latest update is from 17 january, or perhaps they didn't receive any new malware since then.

    But how do they check the malware is new ?
     
  9. SSK

    SSK Registered Member

    Joined:
    Nov 28, 2004
    Posts:
    976
    Location:
    Amsterdam
    Interesting, but I can think up the following problems:

    * Detection categories: does every scanner detect the same categories -> I think not!

    * Scanner set-up: is the VirusTotal service giving every scanner the best possible detection parameters?

    again, nice to see a another comparative, but the shortcomings should be noticed / described!

    EDIT: Thanks Blackspear for posting the link! :)
     
  10. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    i wont shoot the messenger but this isnt reliable AT ALL.

    i cant count the amount of times dr.web on VT has said "its clean" even though im testing a file found by dr.web, to see if others find it as a threat, more than not, it is a virus but virustotal says dr.web says its clean.

    dont rate virustotal, either its not set up right or something is just outdated.

    and cmon..... panda is no way #2, on files ive sent there, panda misses more than quite a few.

    with prevx1 up there near the to, beating loads ..... is funny, that hardly finds anything on the VT tests ive tried.
    mcafee near the bottom?... godddd, sooo bad, but was an interesting read i suppose.
     
  11. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Ya it doesnt seem too reliable. I really don't think panda should be number 2 either Im curious to seee if Fortinet really is that strong though on the next 2 tests on av-c.
     
  12. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    Stamford, CT
    The reason Panda, Fortinet, eScan and the like are so high is the way a "detection" is counted, they detect what they call "suspects" or packers/crypters and flag almost anything packed with these.

    Below is one example, it is Sysinternals Autoruns.exe packed with FSG2.0

    http://xs511.xs.to/xs511/07034/ex.01182007.1121.jpg

    FSG 2.0
    http://www.virustotal.com/vt/en/resultadof?7c0125ba8094ad20c28e04c3d9af5189

    UPX 2.0
    http://www.virustotal.com/vt/en/resultadof?1ce469520fff459dc492c8b977497eba

    PeLock 1.06
    http://www.virustotal.com/vt/en/resultadof?2140844648179866d47b17531a24756c

    yoda's Crypter 1.2
    http://www.virustotal.com/vt/en/resultadof?31858b37d9b275b04493b65657544930

    tElock .98
    http://www.virustotal.com/vt/en/resultadof?1980db6cf432431dd99a0a05ba9b74bf
     
    Last edited: Jan 19, 2007
  13. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    701
    The test shows that some of the antivirus programs such as Fortinet, Panda or eSafe have very simplified "heuristics" - they just report anything that is compressed with a runtime packer without further checks.
    This will detect lots of malware - but also cause lots of false positives. Strange though that Sophos isn't in the top ranks - they do a similar detection "Mal/Packer".
     
  14. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
  15. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    A strange thing with this test is the fact that "PECompact" and "PEPatch" are counted as detections for the Kaspersky and AVG engines respectively. The problem is that AFAIK these two engines do not do packer detection as a sort of "heuristic". So what is going on here is a mystery and it would be best to ignore this test as the results are meaningless due to many reasons explained in the posts before this one.
     
  16. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    701
    LOL, if they really count those KAV log lines as detections then the test really is totally useless.

    KAV does some packer/cryptor reporting ("Klone"), but they do lots of tests to verify the detection. Unlike those programs mentioned before.
     
  17. Technodrome

    Technodrome Security Expert

    Joined:
    Feb 13, 2002
    Posts:
    2,140
    Location:
    New York
    Not necessarily. If you see the above screen shot from virustotal.com you’ll see that file was packed with FSG and Panda reports it as clean.


    tD
     
  18. pykko

    pykko Registered Member

    Joined:
    Apr 27, 2005
    Posts:
    2,236
    Location:
    Romania...and walking to heaven
    well, it depends on the packer Technodrome. But Fortinet, Panda has indeed heuristics analysing the envelope rather then the content.
     
  19. Technodrome

    Technodrome Security Expert

    Joined:
    Feb 13, 2002
    Posts:
    2,140
    Location:
    New York
    Which means Panda does not report anything packed as infected. Which means it has working heuristics unlike stated above.Thats all I am saying…. ;)



    tD
     
  20. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    this test is funny, its showing real flaws in AV's...... about what they can and cant do.

    well the test isnt showing that, mainly the AV experts who come on saying this and that about the rating of one product.

    still an interesting read :)
     
  21. Wolfe

    Wolfe Registered Member

    Joined:
    Jan 23, 2005
    Posts:
    160
    Please read back - that's a statement coming from an average Joe.

    Not really. All has been said and pointed out in the meanwhile.
     
  22. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    lol, yes it is a statement from an average joe, that is me :)

    i dont claim to be an expert on such things, just a trusted drweb user like a few other people, yes i do know certain things with pc's but im just another registered user here, no different, just an average joe :D

    AND DARN PROUD OF IT RARRRRR
     
  23. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    Stamford, CT
    I added some more examples to the post here
    I figured 5 examples is pretty good. :)

    This thread just cements the statements about the reliability of the online scanner usage for detection rates.
     
  24. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    yep i see what you mean likuidkewl,

    but i still think VT is not correct and should not be relied upon, numerous times my AV has found something yet on there, it says its clean, which doesnt make sense.
     
  25. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Just a curious, but are you adding many different CLEAN packer/crypter files too in your next false positive test set? ;)

    Best regards,
    Firefighter!
     
Loading...
Thread Status:
Not open for further replies.