AV-Comparatives Results - Nov 2007 Retrospective/ProActive Test

The latest results have only confirmed again the limitations of all AVs towards the potential of getting infected. I stopped running an AV on my system some time ago, and I don't regret it. All this business of standard, advanced, advanced +, is quite irrelevant IMO. What is it a University/High School test?

The truth of the matter is that any of these AVs ran WITHOUT a sandbox or some kind of HIPS are worthless.

 

tests are just that , it really doesn't measure how well the engine works in the real world in a program and how one uses his/her machine. Unfortunately that's left up to we the users.

 

I have to agree that the best prevention against machine/system compromise, at least from an internet standpoint is thru artificial containment apps like sandboxes AND virtualization which GUARANTEE a significantly better margin for safety then relying on any signature-based AS/AV alone.

The results speak loudly for themselves. How often does one read they been infected by malware if their defenses include say a combo of a sandbox or virtualizer plus HIPS or other behavioral blocker. The odds are just as far less if using a reliable ISR also.

It's high time if in order for AV's to continue to be competitive that they also impliment something similar along those lines because this walking the tightrope between FP's (Heuristics) vs. More Accurate Detections is akin to the same old cat & mouse marathon that still burden's both AV users and the AV Companys alike.

Any thoughts, suggestions, or opposings to the contrary?

 

Make Me

 

True, but at the end of the day this is a test, and some measures must be taken to differentiate between vendors who detect nothing, vendors who have a high score because they have good proactive detection, and vendors who have a high score because they, well, detect everything...

If you think being infected is a hassle, then it follows naturally that you should be concerned about FPs, because products with high FPs inevitably train users to ignore legitimate warnings.

 

To be vigilant means survival here and if FPs cry wolf, so be it, would rather cry wolf and check and be clean than loose data with a deadly virus.

 

True, it's just too bad that too many people who're snug in this knowledge don't realize the fact that many high-FP scanners WILL fail them anyway in the face of a prevalent ITW zero-day threat.

 

As a user of AV and not HIPS I resent that statement

 

That's why I like an AV + HIPS on real-time duty.

 

Using only an antivirus program without a good HIPS beside it is *no* sufficient protection nowadays. But then, HIPS aren't the final solution either. The authors of those highly adapted malware families will soon target the most popular HIPS systems and bypass them aswell. If this is not already happening already. Hm, I have to ask Costin if he is aware of malware explicitely targeting the KAV HIPS.

 

I feel safe enough with just antivirus and some sort of firewall.

 

I have seen a couple targeting KAV PDM, and various malwares that will bypass sandbox types of protections and write files, registry onto the physical drives.

 

Well, that's nothing new, at least.

The easiest way to do that used to be modifying the system date. Now that MP1's been released, I don't know, especially since I haven't been keeping up much thanks to exams and no internet at home.

 

Why? It is an opinion not an indictment against AVs. I also don't think people using only an AV are reckless or doing the wrong thing. My statement is within that margin of security that no AV so far has been able to guarantee: most of them on their own won't give you more than 90% security. The 10% left over (even if it is down to 2-3%) it means thousand of possible infections who could be dramatically reduced by the use of a sandbox (in conjunction with the AV) or a HIPS (I personally don't run any HIPS).

People in the end, they are free to choose as they wish.

 

Havn´t read the whole thread since it´s to long, just a few posts. But those that discards AV solutions on the basis that they are "worthless" or "yesterday security solution", seems to consider themselves as belonging to the main target group exposed for zero-day attacks. I don´t belong to this group, even if I also use other security solutions as a part of my security strategy. Just the fact that your computer is connected to Internet means that you automatically belong to the main target group for malware attacks. If you ever have been exposed for some malware attack in any form (that you are aware of...), how many of these attacks have been by-passed by your AV/AS solutions? Even if I´m using a layered approach and consider myself as "safe" for exposure, I know that when I double-click on that specific .exe file, it could be just my AV that saves the day... Btw, I´m using Avast myself .

/C.

 

Which ones? Can you provide us with names?

 

Re: Av-Comparatives Results

You the nail right on the head i agree 100% i will take the best detecting over alot of fp...

 

Some Chinese trojan horses, You want the names for the malware or the Security applications were bypassed? Unfortunately I don't keep those samples, but because lots of Chinese internet cafes and school computer labs were over reliant on these security application such as Deep freeze/power shadow etc......

 

I just wanted to react concerning these fp...

First the main question is: Can it lead to system failure?
- If the fp is about a system file (Microsoft one, mainly and drivers...) so the antivirus should not be qualified to any ranking as it represents a high threat to system integrity.
- if the fp is about anything else, so the consequences are close to null, going from a file you are forbidden to download, to an installed program which provokes an alert or an error on opening. It is worrying but not so terrible.

So the fp should first be divided in these two parts. Then the second part should be included in the statistics by decrementing the proactive detection rate:
AVIRA
16 out of 2500+ is about 0.64%
So the new rate is around 81-0.64=80%.
BITDEFENDER would be 43 instead of 44...

To give an example, how is it possible to count F-secure AV package or Trend micro AV package as a fp, when you know that the consequences of installing a second AV (you already have an AV, right as you have a fp) are more dangerous than the fp itself?
Other examples are packages. Well, it is disturbing but this fp doesn't occur on install, but on download. These fp will have no consequences to the user other than wait a bit and see the antivirus correcting it...

 

Thats all I have, router with Avira Premium, I have extended thread category checked in Avira, all the threats.

 

How very true your last sentence. I just had Avira alert on a NEW FP. A file I have had since last December when I uninstalled KAV. Avira has never alerted until a few minutes ago on KisKav6Remove.zip.

The truly sad and frustrating thing about all these FP's with Avira is two fold (1) Avira has these because they think being number one in detection is best for their users even at the expense of all the FP's. They evidently have not considered your cogent observation. I know I am in danger of being desensitized and I know the risks ...what about the vaulted average user that Avira wants so much to protect ...they are probably even more at risk of desensitization. (2) I submitted this to Avira via the suspicious files online submission page. I labeled it as "Possible FP" and tried to get a link for them where they could download it. I could still find it on the KAV FTP server but it wouldn't download but that doesn't surprise me as it was from last year. I got back a ridiculous report from Avira lab. In the report they stated:

"avp_remove.cmd MALWARE

The file 'avp_remove.cmd' has been determined to be 'MALWARE'. Our analysts named the threat BAT/KillAV.EG. The term "BAT/" denotes a virus in a Batch format. Batch file viruses execute commands from the commandline whereby the system can be modified.Detection will be added to our virus definition file (VDF) with one of the next updates."

Geez. I think I give up. I like Avira so much but I can't take this almost constant aggravation. Why in the world would Avira lab tell me that they are ADDING this detection when they already detect it and I told them I was submitting an FP! This happens EVERY TIME I submit an FP to Avira Labs. Something is very wrong with their submission process or the technicans can't read English and don't understand that I labeled this, and others, as FP's and the strangest thing is that Avira doesn't seem to even know what they detect already! How can they tell me they are ADDING detection for this, and other FP's I have submitted, when they already detect it? Don't they know the contents of their VDF?

Guard alerted FIVE TIMES on this innocent file while I was trying to reinstall Winamp. I kept telling Guard to ignore the file and it wouldn't. Guard messed up the installation of Winamp. I cannot understand how anyone can defend Avira, or the other vendors, who have a high rate of FP's. Avira detected Rising online antivirus scanner last week, now it detects KAV's removal tool, what is next? Will it detect itself as a virus?

http://forum.avira.com/thread.php?postid=269555#post269555

 

One has nothing to do with the other. Empirically, it is indisputable. Their aggressive heuristic algorithm results in some fp's and until they change it, it will remain as it is.

 

I could not agree more.

 

My oppinion on this test.
First congratulations for the first 3 competitors: Avira, NOD32, Kaspersky.

Now regarding this never-ending FPs subject...
Firstly all AVs should try to fix the FPs as quick as they receive them from a tester\user, etc.

Then, I think AVs giving FPs on system files should not get any certification at all. It's esential to verify the signature against important system files prior to release.

I admit Avira gives perhaps more FPs then an average scanner, but having a strong heuristic engine leads to FPs no matter what. I'm concerned about signature-based FPs... they shouldn't be there normaly for proper added signatures.
NOD32 was lucky to have 0 FPs in this test set. They have anyway a relativeley clean detection but they give FPs as well. A quick search on their forum will reveal many.

Now, regarding the certification levels I really require IBK to answer this.
Avira was penalized for 16 FPs lowering the level twice (from A+-> A, and A-> S). Almost the same with Bit Defender.
Avast had 9 FPs which is more then half of Avira's number. Normally it should have get an S, right ? But no, Avast gets no penalty. If this is fair then... no comment.

The same with AVK and McAfee that have 8 FPs. They got no penalties. I assume av-comparatives has something against Avira, Bit Defender or there are othe reasons for this unfair procedure.