AV-Comparatives February 2007

interesting report as always. kudos to F-Prot, a great improvement with the new product.

Symantec is pulling well away from McAfee, and in my mind they were always quite similar products - at least they were always next to each other in PC World or you could guarantee that one of them would come loaded on your new pc.

Hard to believe that Symantec can do so well with just 73,132 signatures too! I know they shouldn’t mean much, but it still surprises me.
Actually, if you take the number of signatures they had in the August 2006 test: 72,713 then it seems they only added 419 signatures in those 6 months.... surely not?! (for comparison, KAV added over 12,000)
----not wanting to get into a "signature debate", just thought I'd point it out.

NOD32 slipping to Advanced shocked me at first, but when you see that KAV was less that 1% away from doing the same, it isn’t so bad!

 

Maybe someone who knows more than I can answer this question:

How would the free version of Antivir compare in these tests with the paid version? I know the paid version is able to scan for other types of malware, but how much of a difference would this extra protection make in the AV-Comparatives test?

 

My AntiVir PE Premium license expires March 8. I'll be renewing it! Welcome Avira to the realm of the very few.

 

I think it was already answered: https://www.wilderssecurity.com/showthread.php?t=166519&page=2 Read posts #50, 51, 52

 

Oh...I totally suck. I hate it when people do what I just did (ask a question before reading the whole thread). Thanks for the reply and sorry for being a forum idiot.

 

No problem. To err is human

Welcome to Wilders!

 

With all the discussions of "slippage" in ranking people need to read the report and realize that the dropping of detections for DOS viruses has a lot to do with this. It is near the end of the report.

 

Another great work, IBK and AV-Comparatives Team!

It is nice to see that more AV's are getting more higher results, and this results in a better detection for the users because of the concurrence between the companies...

Glad to see that avast! reached the Advanced level.

Regards

 

Interesting in the Feb results NOD32 this time did not get an Advanced +

 

AVG seems to be getting better and better with each new release and product update/upgrade.

 

It is getting better be be weary as the 96 is for AVG anti-malware not for AVG pro or AVG free which i estimate to be around 90-93%

 

Please understand that IBK uses a very huge test set which also includes thousands of corrupted and non-functional samples. Obviously it's very difficult to test each sample for functionality but this should be taken into account when looking at the results. There are AV vendors who basicly simply do not react to such samples for obvious reasons and even advanced heuristics won't emulate such files. In real life you'd get much different results.

 

Very interesting results.
Still I'm very curious what Trend Micro would have done.

 

I was under the impression IBK has said that he only uses functional samples in his tests.

Londonbeat

 

So am i, and i stil believe all files are functional. Inspector Clouseau hates corrupted samples and if IBK used corrupted samples he'd be flipping out right now. So, I believ they are all functional

 

There are a small number of corrupted samples in IBK's test set, but they do not significantly affect the detection rates of any AV (margin of error is 1-1.5% at best). Taking that into account, I think it can be assumed that Marcos was doing damage control for Eset.

 

Advanced heuristics detected these samples in the past to get advanced +,now they get advanced they blame samples.

 

Frankly, I thought that new GData AVK 2007 (KAV + Avast!) would score below of its 2006 oldest brother (KAV + BitDefender) on this year's tests, but... they still remain on the top!

Call me whatever you want but I'm back to AVK 2007 because nothing comes for free

 

What Marcos is trying to say is that NOD32 and Symantec missed the A+ for just some tousands of files, and that if those thousands of files are considered by them as not worth to detect or corrupted (in such a big set it is very possible that some are corrupted etc.) and would not have been included in the set, they would probably still get A+. So I just suggest to read the report in detail which also tries to explain a bit that those 2 missed A+ for a handful of files. Anyway they miss over 5000 to the next product score, which I do not think that all those samples are still just garbage.

 

Oh is it? Nobody can prevent that trash samples making their way into virus test collections, especially not in very large virus collections. But you should at least try to keep them low as possible.

There are several reasons, but the most important one is getting samples from "unknown" sources. There you have the highest chance that it could be trash. That's also one reason why professional virus testers should never ever exchange viruses with so called "virus collectors".

Doing a proper av test is a VERY COMPLICATED thing. It is not only letting a few scanners running over a folder with malware and then counting the results. That's basically a kindergarten test then. Much more important is the correct maintenance of the testbed, the false pos testset and the speedtestset. If this is not in a proper condition you don't even need to start scanning - you'll get flawed results straight away.

And yes, Andreas collection contains trash. But i don't say it's a bad test for one simple reason: He's willing to learn and to improve his testing ability. And that deserves my respect and that's why we gonna help him. He got invited (and agreed within 3 min) to join the workshop for professional antivirus testing in april. That's a 2 days workshop which explains from scratch how to perform "high-end" accurate av tests including how to determine trash files yourself and how to proper maintain virus collections and test sets. Lot's of other things are covered as well, but this should be the most important parts for him. So basically he's a good guy

 

Thanks. I know AVG Anti-Malware was tested, I use it on one of my computers. Anyway, it's always nice to see av vendors trying to improve their products for customers.

 

According to my calculations NOD32 missed A+ for 1461 samples (497608*97%-481219), not 5000. Anyway I wondering why they complain now rather than before the results were official.

 

5000 compared to e.g. kav i meant.
the results were already corrected by eliminating completly one source which contained more garbage than other things. the rest does not change anymore the results in a way that would affect much the results.

 

I don't see a big conflict between what IBK and Marcos are saying. IBK's explanation made sense and IC summed it up nicely.

No AV test is perfect but in terms of publicly available ones AV-Comparatives is probably the best there is. NOD32 might have slipped from a small bit, but really it is not a big deal. It just missed the top classification. It is still a very good AV.

 

Exactly, which is why I am led to think that Marcos was doing damage control....