Can you trust anti-virus rankings?

Yeap.

 

you're missing one important difference - malicious exploits in the wild are used to attack while benign proof of concept exploits are used to defend (or rather to test that defenses have been properly applied)...

 

Oh well, isn't what everyone tells? This is a business just like any other, there is competition. Nobody, even if he is an expert, will ever come out and say "Hey people, X product is better than mine!". Kaspersky is an AV expert but he is also a businessman. It's natural. He wants to sell... He must say something in favour of HIS product!

AV industry isn't a charity institution. They actually want to make money, they aren't Wilder's enthusiasts.

 

With the upcoming changes to the VB testing method (RAP) I guess some more AVs will drop out because they will have less than stellar ratings. Tests that are easy to pass are good for marketing, tests that are difficult to pass and let your product score low must be wrong - of course.....
We saw this in the past. If this trend does continue eventually there will be no more AV tests because no AV dares to participate.

Of course there are flaws in every test around, after all a test does only reflect a fraction of the malware around, the tested malware is too old or the actual situation differs from a customers PC situation.
And AV companies should stop claiming that it doesn't matter that their on-demand scanner did not detect the malware because the malware would be stopped by their behaviour blocker. They still do sell gateway products, don't they? There is no behaviour blocker installed there to close the detection gap. Of course you need both, and it's good to know that on the "end-point" (finally a new buzz word, hurray!) you will be protected by the behaviour blocker if the gateway scanner fails.

Zombini, I already mentioned this. Pure PoC detection is quite prone to false positives on randomly corrupted files. I had plenty of false positives with my PoC-only detections because of this. To make the detection more reliable, I need also to detect more things, shellcode, hidden executables and other suspicious structures.
Secunia got it easy, they just have a on-demand PoC scanner, or? Try that with on-access with a customer base of nn million users. You wouldn't believe the amount of slightly corrupted files that are in usage that possibly trigger false positives. Even more funny, those customers swear they can work fine with these files while during testing these files with the same applications in our labs, the applications crash at once you try to open the files.

 

yes but, when buying antivirus or security software you should still consider your requirements as a user.

 

So where is the bias behind VB100's tests? They just load a couple thousand samples on a test bed and see if an AV can detect them all. Just like what avg. users do to their computer, then run to the store to buy an AV to cleanup the mess.

Actually, I would consider VB100's tests incomplete, they do not test the firewall or disinfection.

 

that is correct. the test shows only one side of the AV apability

 

The only thing I can say is that I trust my common sense, these AV tests are secondary to me.

Peace.

 

Like me...

 

What changes?

 

the primary source of bias in the VB100 test is that it tests exclusively from the wildlist which itself is biased in a variety of ways - not the least of which being that it only contains viral malware, whereas non-viral malware is quite popular amongst the bad guys these days...

 

Correct as well. Many people buy an AV and expect it to nuke all the malware on their machine. VB and many other tests test detection over disinfection. Often times highly rated AVs fail in disinfection.

http://www.virusbtn.com/news/2008/03_13a.xml