AV-C On-Demand Comparative Test

False positives should be taken seriously whether on a corporate network or a home computer. For example, where I work I am responsible for 500 client computers. One FP can have significant financial impact on my company including, but not limited to: overtime, loss of customers, and damage to digital assets (data, etc.). I completely agree that detection rate is important but there is no reason to settle for compromise. For my home and my work network I demand both excellent detection and zero FP's.

At home the risk is similar but obviously much less widespread. However, for any product I use at home I still require accuracy in detection which is really what most vendors should strive for. In short, we are trusting these companies to precisely identify good and bad files for us, and any test of these products should demonstrate that ability as veraciously as possible.

 

[ However, for any product I use at home I still require accuracy in detection which is really what most vendors should strive for. In short, we are trusting these companies to precisely identify good and bad files for us, and any test of these products should demonstrate that ability as veraciously as possible]

I wish I had said that.
Regards,
Jerry

 

My own observations:

A pleasant surprise by Qihoo. Doing quite well, aside from the many false positives. I can't comment on those false positives, not all are equal and it's also relevant what happens when a FP is found. Automatic destruction of the file, or being asked what you want to do with the FP.
They didn't start from scratch, they had some help from Bitdefender and Kaspersky. It's a bit typical for Chinese companies, they try to copy and reverse engineer the foreign technology. It makes sense, why reinvent the wheel. It remains to be seen if Chinese companies can advance beyond the level of European/American companies. That requires true innovation.
But this fits in the trend that 'China' is moving towards producing high quality branded products. That's good for some, bad for others ...

My favorite AV, Avira is still doing well. Although the score for 'other malware/viruses' is rather low, which makes that questionable. (But given the sample set, most samples were 'other malware/viruses', see my comment at the bottom of this post). I wonder how it was tested. By default, many 'things' like 'fraudulent software', 'programs that violate the private domain' (just checking my Avira security suite) are not scanned for. It's nice if we could get some clarification on this issue.

McAfee is doing surpisingly well with its zero false positives, but this is only one test. I remember one particular destructive false positive that destructed Windows system files. I'm not familiar with the current version, but in the past updates were once-daily, updates would 'break' the AV/system and other major issues.

Webroot is doing badly. I don't understand why. Since it uses (?) Sophos' AV engine I don't understand its lower detections. I dropped the product a couple of years ago when they went from focusing on technology and service to marketing.

I must say I don't understand the sample set. (see graph)
Nearly 80 % trojans, and what are the 'other malware/viruses' ?
The sample set is of course extremely important.
I can understand there are many trojans, backdoors, some worms, but what about the 'real' viruses, spyware, adware ?? Is that sample say really representative, have the definitions of words like 'trojan' changed ?

 

In my eyes using words like virus, spyware, adware, rootkits and rogues is useless for the users of the product. It only has meaning for the virus lab employees to classify malware in order to easily create generic signatures etc.

Also it is not that by definition one type of malware will cause more troubles than another. For example, rootkits cause many troubles for detection and removal. But some rootkits can be sitting on your system doing basically nothing, and some spyware can be sending private details to the host of the malware. Next to that nowadays pc attacks use many sorts of malware, using exploits to open an backdoor, using a trojan dropper to install a trojan downloader which on return downloads a trojan horse and install a rootkit.

You already ofcourse see that more vendors are using the name malware instead of virus, since it includes all forms of viruses etc. For example Microsoft Antimalware Engine, Emsisoft Anti-Malware, Prevx Cloud Anti-Malware. Where in contrast most vendors still use the classical Antivirus and Internet Security classifications for their product. I would however rather use the word threat, which is in my eyes the best of describing the goal of security software, protecting you against threats (mind I love the name ThreatSense of ESET, however NOD32 is ofcourse very dated as name).

I like Prevx's business model which doesn't use Antivirus, Internet Security or Total Security/Global Protection classifications, but just let you opt for capabilities you want.

What I would like is seeing vendors dropping product names entirely, for example ESET Cybersecurity (for Mac) which perfectly describes it's function, no need for Antivirus, or just Prevx ofcourse.

What I also would like to see is totally customizable purchases, even more than Prevx, just pay for what you want.
A perfect example to describe this is BitDefender, that vendor doesn't need to use fancy product names (Antivirus Pro) since the company name in my eyes is already enough.
They could just make a solution to let users completely choose their product, for example give users the following choice:

[ ] essential protection against security threats (protection against malware: viruses, rootkits, spyware, adware, rogueware, trojans etc.)
[ ] firewall
[ ] antispam
[ ] parental control
[ ] privacy tools (file shredder, file encryption, file vault)
[ ] pc tuneup
[ ] backup
[ ] network and performance management

They could make preselected choices for average Joe's which reflect the current lineup to make it easier.
However I would then just pick the top three features and only pay for that and only install that. Which means I don't get bloatware I don't want, so I don't want to pay for either.

 

Fly,

The sample set used is representative of the types of new malware appearing according to Panda's quarterly report. The image is from page 9 of their January - March 2011 report.



One interesting quote from the report (also on page 9) is "in Q1 this year we have received an average of 195,463 files to analyze every day, 37.4% of which were new threats".

 

Interesting developments ... There used to be a lot of spyware and adware, initially not dealt with by the AVs.

So it seems they are just gone now, or have a different type of code.

 

No offense, but if your going entirely by closed lab test results and vendor information you probably don't know half of what you think you do..

I'd rather go look at tests with public samples that inadvertently walk past all these over-hyped HIPS and Heuristics/Signature engines, instead of believing mediocre results by commercial contracted entities..

Also, people in the ARK scene like to poke fun at how all these products only take minor obfuscation and discreet API usage to defeat, no fancy tricks needed. There are people defeating KIS 2012 HIPS, SONAR4, and BDIS 2011 without anything but encryption and code obfuscation..

 

Disappointing results for ESET indeed. I can see why ESET would have a high number of FPs ... ESET recently detected my HP Digital Imaging Monitor as an "autorun.Sz virus"

 

Send the FP to ESET so they can fix it:http://kb.eset.com/esetkb/index?page=content&id=SOLN141

 

Are you positive that it's detected in the root of CD/DVD? Please create a new thread in the ESET subforum if you want to continue discussing this topic.

 

avast very good
norton not good
avira good but still have problem when i download any thing from the net its slow very slow
kaspersky good but still have problem its heavy on the computer and slow the pc
bitdefender good but also slow the pc and slow browsing the net

 

The FP was sent to ESET automatically. I'm pretty sure it's a FP because others reported it here. ESET was 'unable to clean' the file, so everything still works. I think ESET may have fixed it already because after running subsequent on-demand scans, ESET found no threats.

Thank you for your follow-up, Marcos.

 

@ xandros

Re - Slow web

Switch off the web guards and you should notice a BIG difference

I use Avira v9 and it doesn't have a dedicated WG, but it DOES still scan web pages & downloads & is excellent & finding & blocking malware like that Also it does NOT slow my browsing in FF v3

 

Actually it was initially an FP in the sign. db 6016 (the current one is 6061) due to a special-crafted autorun.inf that nobody would expect to be used for legit purposes. Using autorun.inf instead of standard configuration ini files and putting lots of own stuff into it is never a good idea and such files are highly suspicious to antivirus scanners. If you're still having an issue with this, drop me a PM please.

 

go on the mcafee page on facebook, you can get a 6 months trial

 

Nothing in life is for free, is the free 6 months trial worth selling/sharing/distributing your personal data ?

 

good summary

 

I agree with you. Bitdefender had a seriuos update error that put all my system files in quarantine and also deleted some of them as well. I had to reinstall my system. Think this was at the end of 2009 or at the beginig of 2010. They gave a compansation to everyone affected as I remember. I got 1 yr extra on my license.

So FP is no joke. No mather if you call them critical or not.

 

Don t know if anyone caught this but the Avira tested was the free version.

 

Yeah they tested the Free version but Avira Premium and Free offer the same protection the only difference is that Avira Premium catches the malware before it touches your pc (Webguard) but the rest is the same.

 

@LethalBoy: Their detection is the same, not protection.

Sorry about that trjam, must've been too sleepy.

 

Question-
when AV-C does their on demand scans are any of the malware samples already executed? Or are all the malware in its non-executed and non-running state?

 

The samples are just scanned on-demand, not executed.

 

malware samples are executed in dynamic tests, but i dont know if samples are stored on local or external disks