AV tests which include a good AT

AV Comparatives tests have a high confidence factor. I realize that detection rates alone should not be the single criterion for selection of an AV.

I noticed in the latest tests that some AVs do well in the virus categories, but fall down in the area of trojans.
With that in mind, I wonder how some would fare in combination with a good AT in the total malware detection rates.
I suppose that it would be a lot of extra effort, and maybe the different ATs would reflect different results.

I recall that Firefighter did a test with AVs in combination with Ewido, and without Ewido. Some of the AVs, such as KAV did not profit much from the combination, while others did.

In the recent test Bit Defender fell down in the area of trojans. While there are some issues with it and ATs, I suspect that the overall results would be equal to NOD or KAV if it were tested in combination with an AT.

Is such a test practicable from the standpoint of the test groups? Would it be useful or useful enough to do it?

Thanks,
Jerry

 

I will not comment it yet, but maybe in ~1 month

 

GREAT!
I figured it would be dismissed as not a workable test considering the difficulties (at least in my mind) of accomplishing it.

I think if it can be done OK it might make the differences between many of the various AV apps negligible or almost so.

I will be looking forward to the decision.

Thanks,
Jerry

 

I will not make a combination test of AV and AT, but at least I think to be able to show something and explain in about ~1 month maybe .

 

Thanks, and we will be looking forward to it.

Jerry

 

Recently someone posted a link to an AV test that included lots of ATs. I downloaded the pdf report but can't remember where I got it from (and it doesn't have a url in the pdf).

As I can't find the link, I've temporarily uploaded it here:

http://home.freeuk.com/nedslider/12122005en.pdf

If anyone can post a direct link to the test would a moderator please remove my link (or if re-hosting it breeches any guidelines).

I wouldn't necessarily place a huge amount of faith in the study and post it for information only (Ewido shows 78% detection of their backdoor/trojan samples, ~53,000 samples).

However, I too look forward to AV-Comparatives doing something similar or at least including some ATs in their tests against their backdoor/trojan samples.

Ned

 

Firefighter ran some tests on 12 Feb 2005 where he compared AVs with and without ATs. BD 7.2 with Ewido found 96.5% of trojan like malware. BD 7.2 alone only detected 79.6%.

Others fared well without Ewido, and some were improved. I can't bring up the post here on Wilders. Maybe someone else can if interested.
I wish such a test could be conducted again with the current crop of AVs and ATs.

Jerry

 

This one?

(From VirusP's website.)

 

Ah, yes - that looks like it. Thanks

 

Jerry,

I tried running a few tests of my own. I used Kaspersky to run an initial scan as it's recognised as being one of the very best, and then used BitDefender, McAfee, A-squared and ewido on the samples missed by KAV. Each program found samples the others had missed. My sample set of backdoor/trojans is small at about 1500, but includes almost exclusively samples found on infected machines during the last 12 months.

Some samples missed by KAV were identified by every other company on Jotti's site. When submitted to KAV they'd come back clean and KAV would say they were corrupted samples (therefore theoretically harmless).

So what can you conclude from this - well, not a lot! Firstly, running two products is probably better than one, and three product is probably better than two, but you reach a point of deminishing returns. But we all kind of knew that anyway. Secondly, we (well, me anyway!) are not expertly knowledgable enough to be able to properly identify good samples from bad (or false positives) and without a properly screened sample set of significantly relavent size any tests are useless (for example, mine!).

Whilst AV-comparatives may not be absolutely perfect in everyone's opinion (some AV company's maybe), most seem to agree that they do one of the best jobs out there, so I say let them continue doing what they do. But, in the meantime, I think it's safe to conclude that if a particular AV is weak at trojan detection, then a good AT will probably compliment it well. Will it make it as good as a really good AV - who knows?

I too look forward to AV-comparatives maybe widening their testing to include programs such as ewido, and would also be interested to see how a program such as ewido would perform when combined with various AVs

Ned