FF-av-test 26-May-2005!

Discussion in 'other anti-virus software' started by Firefighter, Jun 4, 2005.

Thread Status:
Not open for further replies.
  1. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Hi again. I have removed those samples from my testbed which IBK has checked to be removed, although most of them were still detected by av-scanners, especially Backdoor clients, editservers and all other riskware samples except Adware. I have submitted my CRC-checksum list to IBK and there were now only CRC-checksums of each sample. It took several days until I could remove 4 duplicate samples, because my Virweed proggie denied to work properly, but today I managed to do that.

    Special thanks to IBK for his help. Without him I couldn't clean my collection, so it was a job well done.

    My Adware samples are only checked as infected by several av:s, mostly by eScan Free 4.4.7 updated to 6.2.9, McAfee VSE 8.0i with AntiSpy module, NOD32 v2.50.19, DrWeb 4.32.b with beta AntiSpy defs and Ad-Aware SE 6 antispyware. All other samples were checked by Virweed, so that there isn't any duplicate samples and CRC-checksum list was submitted to IBK, who adviced me to remove hundreds of samples.

    Best regards,
    Firefighter!
     

    Attached Files:

    Last edited: Jun 8, 2005
  2. NAMOR

    NAMOR Registered Member

    Joined:
    May 19, 2004
    Posts:
    1,526
    Location:
    Arkham Asylum
    Thanks for your hard work FireFighter. ;)
     
  3. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,010
    Location:
    Christchurch, UK
    Thanks to FireFighter for all his hard work as usual and to IBK for helping FF to "clean" up his collection.

    Together with Likuidkewl's unofficial tests, we now have two sets of member's results to peruse :cool:
     
  4. rothko

    rothko Registered Member

    Joined:
    Jan 12, 2005
    Posts:
    579
    Location:
    UK
    indeed, thanks for the results - always nice to see them.
    obviously as a nod32 user i'm not entirely happy with the results, and based on the above should i ditch my paid AV for one of the free ones reviewed? No, but if i was looking at these results as someone who is looking to make a purchase, i may just overlook NOD32 in favour of a more affordable product with better detection.

    in a nutshell, and i'm sure others will pile in behind me with answers - i cant see how nod32 (latest version too) scored so badly here. i'm just an average user so dont know much about the differences in samples, but i can only assume the samples used in the test are not the ones that pose a serious threat in the real world where i do my surfing, and in which nod32 has always kept me safe.
     
  5. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    In my mind all scanners that have a detection rate 90 % + are very good scanners overall and NOD has a powerfull unpacker engine, so there is no worry about these results. Unfortunately I had tested only a few scanners, but if I will test some ten more scanners too, i'm pretty sure that NOD belongs to the better half of scanners. Also there is always a failure rate to the detection rate against all infections available, as you can see here in my test table calculations. There is a 2 % precision/accuracy level in this test according to this calc and the reliability/confidence level is now 95.889 %.

    Best regards,
    Firefighter!
     

    Attached Files:

    Last edited: Jun 7, 2005
  6. tahoma

    tahoma Registered Member

    Joined:
    May 31, 2003
    Posts:
    228
    bidefender looking a good choice among the lightweights
     
  7. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    Any idea about the heuristic detections of the various AV scanners, Firefighter? Please? :)
     
  8. Trespasser

    Trespasser Registered Member

    Joined:
    Mar 1, 2005
    Posts:
    1,194
    Location:
    Virginia - Appalachian Mtns
    I may have missed something, but why wasn't Kaspersky included in your test group? I must say I'm a bit surprised by NOD's results while BitDefender IS looking better and better as an AV choice.

    Have a good one. :).

    It's a beautiful day here in Virginia.
     
  9. JerryM

    JerryM Registered Member

    Joined:
    Aug 31, 2003
    Posts:
    4,221
    Where are the results displayed? I couldn't find them and could not link to the "attached images."

    EDITED: For some reason the results came up after I closed the thread and then went back to it. Thanks.
    Jerry
     
  10. rdsu

    rdsu Registered Member

    Joined:
    Jun 28, 2003
    Posts:
    4,456
    Thanks for your work, Firefighter ;)
     
  11. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    eScan uses the KAV engine and has hourly updates too; therefore KAV was not tested.

    Funny how I thought that BD will get better some time back - For some reason, whatever I think will happen, really happens!

    Yes, BD is getting better and better, we already saw that at AV-Comparatives. :)
     
  12. quexx88

    quexx88 Registered Member

    Joined:
    Nov 26, 2004
    Posts:
    235
    Location:
    Radnor, Pennsylvania
    Wow! AntiVir is looking impressive, at least by this test...
     
  13. JerryM

    JerryM Registered Member

    Joined:
    Aug 31, 2003
    Posts:
    4,221
    I too get confused. I have the highest for Firefighter's tests. So I am not sure why in the AV Compar. tests NOD blew all others out of the water, but here it does not look all that great. o_O o_O
    Jerry
     
  14. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    AntiVir is quite good even at AV-Comparatives, the only thing that I dont like is the updater (I'm on dialup you see)
     
  15. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    AV-Comparative's latest test was about the heuristic capabilities of the various AVs tested, and you know that NOD has the best heuristics ;)

    NOD does very very well at AV-Comp's On-Demand tests too, so I do find these results slightly surprising.
     
  16. quexx88

    quexx88 Registered Member

    Joined:
    Nov 26, 2004
    Posts:
    235
    Location:
    Radnor, Pennsylvania
    Except in the last full blown on demand test, AntiVir picked up only 76,61% of actual viruses. Here, it grabs 94%. Have there been any significant upgrades to its engine that could place it a single percentage point below McAfee?

    EDIT: For clarity :) I saw your deletion there, Firecat :p
     
  17. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Unfortunately I was able to check only NOD with normal heuristics plus AH and DrWeb against all my samples. The results are quite low but I think that it is mainly because my samples are all zipped and plenty of them are also packed with different self extracting packers. This thing has said by Illukka some weeks ago with NOD too.

    37.2 % -- 1046/2809 NOD with Heuristics and AH

    15.7 % -- 441/2809 DrWeb with heur only

    Best regards,
    Firefighter!
     
    Last edited: Jun 5, 2005
  18. JerryM

    JerryM Registered Member

    Joined:
    Aug 31, 2003
    Posts:
    4,221
    Thanks, Firecat. I have to conclude that there are so many aspects of this that I do not understand that I will just have to go with the top ones, and let it go with that.

    Since I am using e-scan 4.4.7 that you gave so much help so that I could finally install and run it, and BD 8.0 that I am about as well fixed in the AV area as can be.

    I think if I left BD, which I an not considering, I would probably go with NOD. But then again, the overall detection of KAV in both Trojans and Viruses would make that a hard decision.

    BTW I notice that FF has upgraded e scan 4.4.7 to 6.2.9. I wonder if that is a freebie, and if so does it also clean?

    Jerry
     
  19. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    It's a hard work to count detected samples from logfile. That's why I like proggies which are able to delete/rename/move infected archives as my samples are. Special thanks to NOD when it is able to do that now in the new version.

    Not sure if Kaspersky is able to delete infected archives now but it was not able to do that some months ago.

    Best regards,
    Firefighter!
     
  20. tazdevl

    tazdevl Registered Member

    Joined:
    May 17, 2004
    Posts:
    837
    Location:
    AZ, USA
    FF, did you use default settings for NOD32 or BlackSpear's?
     
  21. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    Stamford, CT
    Nice work FF, I know how hard it is to weed out the crap.

    -Please keep in my mind my tests are not very professional at all, and that must be taken into account ;)
     
  22. Stephanos G.

    Stephanos G. Registered Member

    Joined:
    Mar 29, 2005
    Posts:
    720
    Location:
    Cyprus
    just wondering if the sample is satisfactory (about 2809)

    I ask this, as i see that andreas clementi sample is 8259.

    Thanks
     
  23. RejZoR

    RejZoR Registered Member

    Joined:
    May 31, 2004
    Posts:
    6,426
    Firefighter,can you also test Norman? The latest beta version is free,so you can easily test it. I'm really interested in results :)
     
  24. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    @Firefighter: Thanks :)

    @quexx88: I deleted my post because I didnt understand your post and I thought you were referring to NOD32. Once I understood my mistake, I deleted it. :)

    @JerryM: eScan 6.2.9 still does not clean :(. It only says that either you pay about USD 10/month or you buy the commercial edition if you want to clean.
     
  25. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,818
    Location:
    Innsbruck (Austria)
    @FF: I only told you which samples have to be removed for sure, the rest I can not say if they are good or garbage samples without getting the files and checking them. So it is now a bit cleaned, but I do not think that it is now garbage free.
    I think you was a bit to fast to make this test; I told you that after removing the known garbage there are other steps to work on before (e.g. no archives, correct extensions[!!!], etc.).

    [Please do not compare AV-Comparatives tests with FF tests etc., the difference in quality should be clear - I am not saying this with bad intentions, I am helping]

    E.g. some scanners will not detect samples if the files have non-executable extension (like most of the files of FF), so their results will look lower than they are.

    I think in order that ppls see qhat samples were used by Firefighter and how, this list should help: FF Filelist
     
    Last edited: Jun 4, 2005
Loading...
Thread Status:
Not open for further replies.