Detection Rates KAV / NOD32 / Anivir / Prevx1

Discussion in 'other anti-virus software' started by adam_72, Dec 21, 2006.

Thread Status:
Not open for further replies.
  1. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    Recently I got hold of a harddisk packed with tons of malware (viruses, trojans, dialers, ...) The guy had obviously no AV-software running and operated XP SP0 (!) with not a single hotfix applied ... :blink:

    Thought i could use it for some real life AV testing so I came up with the following procedure :

    1) Image "bad" physical disk onto a vmware virtual disk ! (only once, for all following test runs)
    2) Install and fully update AV on a fresh W2kSP4 virtual machine under VMware 5.5.3
    3) shut down machine and mount "bad" virtual disk as 2nd disk (slave)
    4) do a snaphot to preserve "bad" virtual disks original state
    5) restart machine
    6) conduct a full system scan with manually maximized scan settings (scan all files, enable heuristics, unwanted applications ...)
    7) repeat test (for verification purposes)

    The harddisk had not been in use for at least 2 weeks, so there should be patterns for even the latest piece of malware ! here are my results:

    AntiVir PE 7.0.3.............468 detected items ("medium" heuristic setting)
    NOD32 2.70.16............1081 detected items
    KAV 6.0.1.411..............510 detected items
    KAV Rescue Disk.........1303 detected items (offline scan)
    Prevx1 2.0.4.19.............30 detected items (extensive scan, full verify, permanently online)

    I am aware of the non-scientific character of these tests, but I feel that that two points should be mentioned:

    • I scanned and cleaned the disk online using KAV and 510 items were neutralized. A consecutive scan with rescue disk detected another 793 items which sums up to a total of 1303, matching the original detection rate of Rescue disk ! Could that difference be due to the nature of offline scan ? (no open files, no running processes ..) just to remind you: same scan settings and virus definitions in both test runs
    • The detection rate of Prevx1 ... I repeated the test just an hour ago with ver. 2.0.5.6 all updates applied, now 34 infections were detected. I already did a thorough research at the prevx thread over at castlecops forum but found no mistakes within my test procedure. After deleting those 34 infections, I mounted the disk under NOD just to see if it's maybe some GUI issue or the way it counts infections and more malware than visually reported has been detected, but NOD found well over 1000 "leftovers" [Edit: 1057 items, to be exact] Is it possible that prevx and the virtual machine interfere in a negative way ??
      I know that Prevx1 has good reviews and is considered a powerful tool but I'd be much more comfortable with a plausible explenation for these strange figures

    happy holidays ..... :cool:

    [Post edited several times for clarification purposes]
     
    Last edited: Dec 21, 2006
  2. Rob E

    Rob E Registered Member

    Joined:
    Jan 17, 2005
    Posts:
    16
    Just curious... You comment that AntiVir's heuristics were set to medium, but make no mention of KAV or NOD32's settings. What were they? Out of the box, neither are at their tightest. Would be more interesting (to me at least), if all three AV's were at their highest (tightest) settings, or even... all three set to their out-of-the-box configurations. Interesting nontheless. Thanks for the post.

    Robert
     
  3. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    6) conduct a full system scan with manually maximized scan settings (scan all files, enable heuristics, unwanted applications ...)

    ;)

    I only mentioned Antivir's heuristic settings, since it's the only configurable heuristics which can be configured in a way ... I thought it would be best to leave it at default settings since preset heuristic detection is always a kind of ompromise between good rates and false positives !
     
  4. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    I am not surprised by how well Avira detected compared to the others.:)
     
  5. FRug

    FRug Registered Member

    Joined:
    Feb 7, 2006
    Posts:
    309
    just for the record, a different "item" count does not necessarily mean that more or less viruses have been detected, this all depends on how the AV reports and counts infections. If AV A triggers on an installer and calls it 1 detection, and then saves the time and does not unpack it because it is known malware, while another AV unpacks it and flags each contained file, you get a different file/item count but the same protection. Same goes i.e. for runtime packed files, does it count the file on disk and the extracted/unpacked/restored temp file as two items or one?

    Also, what is an ITEM? A file? A registry key? A system setting? A directory? A tracking cookie? An entry in the host file?

    Keep that in mind for such comparisons, item, file or infection count is not a good base for comparisons.
     
  6. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    That's perfectly true, the numbers can only give you an rough idea of what's going on !

    But AFAIK the term "item" relates to a single malicious file (while the term "malicious" still lacks an exact definition ;) ) ... a quick glance at the scanning logs showed no indications of the AVs complaining about settings, HOSTS entries, reg keys ... of course it might have slipped my eyes !
     
  7. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    That also might be a possible explanation for Prevx1's seemingly horrifying detection rate. I think I remember someone mentioning, that Prevx is no regular AV scanner, but compares files against a common database upon execution !!

    It could well be that it doesn't bother to unpack (runtime) packed files for scanning purposes but verifies them on "real" access, so a lot of malware could stay undetected after the first run, but the system is still well protected !
    I'll try to figure out how to proove this !
     
  8. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    yeah some antivirus companys flag detection numbers differently.

    dr.web scans my machine clean, then panda says i have 22 infections, what? omg no.... 22 tracking cookies, which ive noticed dr.web doesnt bother with, and am glad it doesnt, they can be a pain to scan and find all these infections.
     
  9. JohnnyBravo

    JohnnyBravo Registered Member

    Joined:
    Jan 26, 2006
    Posts:
    82
    Can someone comment the result of the AntiVir :)
    I'm not very immpresed :(
     
  10. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,010
    Location:
    Christchurch, UK
    I would not think of dropping AntiVir as a result of 1 non-professional test ;) ( no disrespect to adam_72)

    Otherwise you will be changing your AV every other day!!!!!!!
     
  11. ggf31416

    ggf31416 Registered Member

    Joined:
    Aug 20, 2006
    Posts:
    314
    Location:
    Uruguay
    For example if a malware creates 20 files, 5 important and 15 useless without the main program,
    Antivirus A report all detected files as a single infection
    Antivirus B detects 5 infected files
    Antivirus C detects 20 infected files
    AntiSpyware D detects 20 infected files + Some other items + 1000 tracking cookies

    The 4 programs give the same protection. The problem is determine which detections are actually dangerous.
     
    Last edited: Dec 21, 2006
  12. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    no offense taken :D

     
  13. Howard Kaikow

    Howard Kaikow Registered Member

    Joined:
    Apr 10, 2005
    Posts:
    2,802

    Counts are not useful as each AV counts the critters it examines ddifferently, not to mention that each AV might not examine the same files.

    An interesting statistic would be how many files were NOT vetted.
     
  14. adam_72

    adam_72 Registered Member

    Joined:
    Nov 12, 2006
    Posts:
    11
    I have to agree .... the problem lies in the nature of infected "real life disks":
    You don't really know the exact quality or quantity of malicious items on the disk, so it's hard to track back how many of them have actually been neutralized in a test run. As long as you don't have a precisely defined set of malware samples (which are reserved to professional institutions for obvious reasons ;)), these private tests seem to be useless ... and can well cause confusion !

    I just realized that and apologize for this "pointless" thread :blink:
     
  15. BlueZannetti

    BlueZannetti Administrator

    Joined:
    Oct 19, 2003
    Posts:
    6,590
    It's never pointless to learn something new!

    This thread does have merit to remind folks - I hope - that, as they switch between various products, having a one product flag files not flagged by an earlier product means very little in isolation. It is not immediate proof that a previously installed option "missed" some key malware and that one is now infected, which does seem to be a frequent inference.

    Blue
     
  16. Howard Kaikow

    Howard Kaikow Registered Member

    Joined:
    Apr 10, 2005
    Posts:
    2,802
    Until recently, I had used only NAV.

    Never did bother to find out what the counts meant.

    Recentky, I used KAV, and the count was quite different, e.g. NAV would claim to have scanned about1.2 million critters, KAV about 2.6 million critters.

    KAV does svan archibes that are allegedly not scanned by NAV.

    But, there's my big butt again, I created a MSFT Word template that had a single code module.

    I do not recall the counts, but NAV clained a much larger number for this single template.

    KAV does allow for the creation of a log so you can see what is scanned.
    For a system wscan th file is huge, so I wrote a program to list only the items of interest.
     
  17. tsilo

    tsilo Registered Member

    Joined:
    Apr 29, 2006
    Posts:
    376
    Once I downloaded 1700 viruses and unpack it, my nod32 catch ALL of tham and 20-30 % of that viruses was unknown, detected heuristically...

    no comments..
     
  18. Howard Kaikow

    Howard Kaikow Registered Member

    Joined:
    Apr 10, 2005
    Posts:
    2,802
    from where did you download those test viri?
     
  19. BlueZannetti

    BlueZannetti Administrator

    Joined:
    Oct 19, 2003
    Posts:
    6,590
    To all,

    This is not a malware trading site. If you wish to be engaged in these types of discussions, take it offline via PM.

    Blue
     
Loading...
Thread Status:
Not open for further replies.