AV-Comparatives Results - Nov 2007 Retrospective/ProActive Test

An infection might cause losses of time (reimaging, cleaning, reduced productivity) and money (stolen bank account, pay the tech to clean up the mess, reduced productivity, corporate espionage)
A FP might cause losses of time (unbootable systems, crashed mail server, reduced productivity) and money (reduced productivity, deleted documents, deleted mails, etc)
So, in certain situations, having accurate detection is as important as having high detection.
For people whom understand a bit how an AV works, FPs may not be important. You do a bit of research, exclude the flagged file and report it to the AV vendor.

 

Hi bellgamin:

Agreed, I would say that 1 undetected infection is worse than 10 quarantined FP's.

These tests are good for comparing tools on the same basis at the same point in time.

Users of tools producing high #'s of FP's should quarantine ALL "hits" til they know it is "safe" to delete. So maybe the discussion thread here is more useful than we thought!

Did you really fly a Steerman? Where is the jpg ?

 

ok sir, it's your opinion. i hope IBK don't get mad at you

 

A quick question if I might:

Does anyone know why BitDefender v9.x was tested instead of V10 (2008 version)? V10 has been out for several months....

 

v10 was tested. v11 is 2008, and was not available already in august.

 

In practice, how many people understand what a FP is? They even don't know what the meaning of the abbreviation FP is. So how should they understand the difference between a real alert and a FP? They see an alert and hit the "disinfect" or "delete" button. And will think: praise The Almighty Lord, i have an AV that saved me for the bad and ugly....

 

Thanks IBK for your quick post and for straightening out my BD version numbers!

HOWEVER, IBK, if you will look in the HTML-based results, the title for the tested BD product shows (clearly) BitDefender Prof.+ V9.5, even though BD V10 was tested (which one)?

Thanks in advance,

 

Sorry, IBK, I was looking at the November 2006 test results instead of November 2007. It's been a LONG week.....

Sorry again.....

 

Very few. That's why I think that having a low amount of FPs is as important as having a high detecton rate for 95 % of people.

 

Avira hasn't decided any such thing regarding FP's as Stefan Kurtzhals has expressed in previous posts displesure with the Number of FP's the product produces.

 

71 + 20= 91%

So having no signatures NOD will be at the bottom line in that case.

 

Even Avira?

 

If you look at the actual False Positive detections for all vendors, you can see that nothing system critical is detected apart from Fortinet (even with heuristic disabled), and even that is minor. No one is detecting things like winlogon.exe!

Disregarding Fortinet, it seems that all False Positive cases are easily recoverable, and indeed probably unlikely to be encountered in a "real life" situation, although for testing purposes they serve as an example.

We cannot be sure of the consequences of true malware missed, but I'd bet recovering from a missed "real" infection won't be quite so easy.

My opinion is that the award system is flawed. It's weighted too heavily on false positive detections, with little regard to the real protection provided.

 

Correct. That is the reason why i share your opinion.

 

Re: Av-Comparatives Results

Many of us can but not ordinary users.

Believe me false positives are as bad as a missed detection in case of ordinary users.

But we must remember that default install of Avira9 used by avergae users) has medium heuristics that might not cause so many false positives( not sure though)

 

I agree. However I think it might be considered if there is some better option available.

 

By the way, can u give us an idea what will be the detection/ false pisitive score of Avira used on medium9 default) heuristics?

Or may be IBK can tell us!

 

if I would re-run the test, I would be able to tell. But I will not do that. I guess it would be 5k less.
with medium heuristic it would be 11 FP's.

 

As a long-time user of computers and the internet but possessing only a modicum of technical expertise with respect to virus and other form of malware, I definitely agree with Bellgamin's assessment.
While it is possible to impute potential costs to the FP issue, I must conclude that, within reason, detection an removal of infections take priority.

Rob

 

IBK,
What can you say about this?

Thanks and great work as always

 

a) as stated in the report, signature means it occurs also when you disable the heuristic. New Malware.dq occurs also with disabled heuristic.
b) thats what fprot gives as output. i think most of that noname detections are now covered by eldorado (but i am not sure about this, i think i read this somewhere).
c) that's probably.

 

Neither are good results. I think all agree on that point.

It is the frequency of potential exposure/alert that is the great unknown. For example, on those Windows files that McAfee generated a false positive a while ago, see here for example, any McAfee user with that update would see the false positive. So, in some circumstances it will impact all users. Exposure on the malware side is of much lower frequency, so you really can't do a 1:1 comparison.

Are all false positives like this? Of course not. By the net level of false alerts provided is one measure of the likelihood that this eventuality could occur. I realize that it's a low probability event.

Not really, since we're all not exposed to all malware.

It's serious, but it's not a certain eventuality

By that logic, don't read the on-demand test, you'll blanch at some of the numbers.

We're not talking about the applications here, rather an evaluation of them. As for functioning in this era, as the technology becomes more pervasive, it simply has to get easier and more transparent to use, not more complicated. Look at any technology as it captured the market. PC's are a good example. At the dawn of the PC era, your had to be a pretty good hardware jock and it really helped to have actual programming prowess. That's not required anymore. The same will happen with security - how it evolves, I don't know, but I do know that if it's important, some of the details people claim are absolutely necessary to know today will simply be irrelevant in the future.

Blue

 

Thanks. Still a singnificant no of flase positives.

 

Strange, isn't it?

 

Yup, I shore did fly a Stearman. It were an "advanced" aircraft (in its day).