FF-av-test 26-May-2005!

[Firefighter]

Hi again. I have removed those samples from my testbed which IBK has checked to be removed, although most of them were still detected by av-scanners, especially Backdoor clients, editservers and all other riskware samples except Adware. I have submitted my CRC-checksum list to IBK and there were now only CRC-checksums of each sample. It took several days until I could remove 4 duplicate samples, because my Virweed proggie denied to work properly, but today I managed to do that.

Special thanks to IBK for his help. Without him I couldn't clean my collection, so it was a job well done.

My Adware samples are only checked as infected by several av:s, mostly by eScan Free 4.4.7 updated to 6.2.9, McAfee VSE 8.0i with AntiSpy module, NOD32 v2.50.19, DrWeb 4.32.b with beta AntiSpy defs and Ad-Aware SE 6 antispyware. All other samples were checked by Virweed, so that there isn't any duplicate samples and CRC-checksum list was submitted to IBK, who adviced me to remove hundreds of samples.

Best regards,
Firefighter!

[NAMOR]

Thanks for your hard work FireFighter.

[Blackcat]

Thanks to FireFighter for all his hard work as usual and to IBK for helping FF to "clean" up his collection.

Together with Likuidkewl's unofficial tests, we now have two sets of member's results to peruse

[rothko]

indeed, thanks for the results - always nice to see them.
obviously as a nod32 user i'm not entirely happy with the results, and based on the above should i ditch my paid AV for one of the free ones reviewed? No, but if i was looking at these results as someone who is looking to make a purchase, i may just overlook NOD32 in favour of a more affordable product with better detection.

in a nutshell, and i'm sure others will pile in behind me with answers - i cant see how nod32 (latest version too) scored so badly here. i'm just an average user so dont know much about the differences in samples, but i can only assume the samples used in the test are not the ones that pose a serious threat in the real world where i do my surfing, and in which nod32 has always kept me safe.

[Firefighter]

Originally Posted by rothko

indeed, thanks for the results - always nice to see them.
obviously as a nod32 user i'm not entirely happy with the results, and based on the above should i ditch my paid AV for one of the free ones reviewed? No, but if i was looking at these results as someone who is looking to make a purchase, i may just overlook NOD32 in favour of a more affordable product with better detection.

in a nutshell, and i'm sure others will pile in behind me with answers - i cant see how nod32 (latest version too) scored so badly here. i'm just an average user so dont know much about the differences in samples, but i can only assume the samples used in the test are not the ones that pose a serious threat in the real world where i do my surfing, and in which nod32 has always kept me safe.

In my mind all scanners that have a detection rate 90 % + are very good scanners overall and NOD has a powerfull unpacker engine, so there is no worry about these results. Unfortunately I had tested only a few scanners, but if I will test some ten more scanners too, i'm pretty sure that NOD belongs to the better half of scanners. Also there is always a failure rate to the detection rate against all infections available, as you can see here in my test table calculations. There is a 2 % precision/accuracy level in this test according to this calc and the reliability/confidence level is now 95.889 %.

Best regards,
Firefighter!

[tahoma]

bidefender looking a good choice among the lightweights

[Firecat]

Any idea about the heuristic detections of the various AV scanners, Firefighter? Please?

[Trespasser]

I may have missed something, but why wasn't Kaspersky included in your test group? I must say I'm a bit surprised by NOD's results while BitDefender IS looking better and better as an AV choice.

Have a good one. .

It's a beautiful day here in Virginia.

[JerryM]

Where are the results displayed? I couldn't find them and could not link to the "attached images."

EDITED: For some reason the results came up after I closed the thread and then went back to it. Thanks.
Jerry

[rdsu]

Thanks for your work, Firefighter

[Firecat]

Originally Posted by Trespasser

I may have missed something, but why wasn't Kaspersky included in your test group?

eScan uses the KAV engine and has hourly updates too; therefore KAV was not tested.

Originally Posted by Trespasser

BitDefender IS looking better and better as an AV choice.

Funny how I thought that BD will get better some time back - For some reason, whatever I think will happen, really happens!

Yes, BD is getting better and better, we already saw that at AV-Comparatives.

[quexx88]

Wow! AntiVir is looking impressive, at least by this test...

[JerryM]

I too get confused. I have the highest for Firefighter's tests. So I am not sure why in the AV Compar. tests NOD blew all others out of the water, but here it does not look all that great.
Jerry

[Firecat]

AntiVir is quite good even at AV-Comparatives, the only thing that I dont like is the updater (I'm on dialup you see)

[Firecat]

Originally Posted by JerryM

I too get confused. I have the highest for Firefighter's tests. So I am not sure why in the AV Compar. tests NOD blew all others out of the water, but here it does not look all that great.
Jerry

AV-Comparative's latest test was about the heuristic capabilities of the various AVs tested, and you know that NOD has the best heuristics

NOD does very very well at AV-Comp's On-Demand tests too, so I do find these results slightly surprising.

[quexx88]

Except in the last full blown on demand test, AntiVir picked up only 76,61% of actual viruses. Here, it grabs 94%. Have there been any significant upgrades to its engine that could place it a single percentage point below McAfee?

EDIT: For clarity I saw your deletion there, Firecat

[Firefighter]

Originally Posted by Firecat

Any idea about the heuristic detections of the various AV scanners, Firefighter? Please?

Unfortunately I was able to check only NOD with normal heuristics plus AH and DrWeb against all my samples. The results are quite low but I think that it is mainly because my samples are all zipped and plenty of them are also packed with different self extracting packers. This thing has said by Illukka some weeks ago with NOD too.

37.2 % -- 1046/2809 NOD with Heuristics and AH

15.7 % -- 441/2809 DrWeb with heur only

Best regards,
Firefighter!

[JerryM]

Thanks, Firecat. I have to conclude that there are so many aspects of this that I do not understand that I will just have to go with the top ones, and let it go with that.

Since I am using e-scan 4.4.7 that you gave so much help so that I could finally install and run it, and BD 8.0 that I am about as well fixed in the AV area as can be.

I think if I left BD, which I an not considering, I would probably go with NOD. But then again, the overall detection of KAV in both Trojans and Viruses would make that a hard decision.

BTW I notice that FF has upgraded e scan 4.4.7 to 6.2.9. I wonder if that is a freebie, and if so does it also clean?

Jerry

[Firefighter]

Originally Posted by Trespasser

I may have missed something, but why wasn't Kaspersky included in your test group?

Have a good one. .

It's a beautiful day here in Virginia.

It's a hard work to count detected samples from logfile. That's why I like proggies which are able to delete/rename/move infected archives as my samples are. Special thanks to NOD when it is able to do that now in the new version.

Not sure if Kaspersky is able to delete infected archives now but it was not able to do that some months ago.

Best regards,
Firefighter!

[tazdevl]

FF, did you use default settings for NOD32 or BlackSpear's?

[dan_maran]

Nice work FF, I know how hard it is to weed out the crap.

-Please keep in my mind my tests are not very professional at all, and that must be taken into account

[Stephanos G.]

just wondering if the sample is satisfactory (about 2809)

I ask this, as i see that andreas clementi sample is 8259.

Thanks

[RejZoR]

Firefighter,can you also test Norman? The latest beta version is free,so you can easily test it. I'm really interested in results

[Firecat]

@Firefighter: Thanks

@quexx88: I deleted my post because I didnt understand your post and I thought you were referring to NOD32. Once I understood my mistake, I deleted it.

@JerryM: eScan 6.2.9 still does not clean . It only says that either you pay about USD 10/month or you buy the commercial edition if you want to clean.

[IBK]

@FF: I only told you which samples have to be removed for sure, the rest I can not say if they are good or garbage samples without getting the files and checking them. So it is now a bit cleaned, but I do not think that it is now garbage free.
I think you was a bit to fast to make this test; I told you that after removing the known garbage there are other steps to work on before (e.g. no archives, correct extensions[!!!], etc.).

[Please do not compare AV-Comparatives tests with FF tests etc., the difference in quality should be clear - I am not saying this with bad intentions, I am helping]

E.g. some scanners will not detect samples if the files have non-executable extension (like most of the files of FF), so their results will look lower than they are.

I think in order that ppls see qhat samples were used by Firefighter and how, this list should help: FF Filelist