AV Testing

Well, last night I put a few things to the test. I've done this before and got mixed results posted in the thread. Guys, if you don't like this kind of testing or don't believe it is worthwhile, then don't bother posting. Some of us enjoy testing and seeing results, even if it is with a small percentage of malware in this non-scientific manner. Thanks.

I was asked to test several of these as well, I brought several of the bigger named ones in as well for reference. I used 20 zero day malware links. The links were loaded in IE and anything it asked to download, was downloaded, even if IE said it may be unsafe. All of these AV's were installed and updated. If a pop up came up from the AV, block, clean or whichever action it suggested was performed. Once the links were complete (if that was possible), CCleaner was run to rid the system of any left over downloaded files, then the system was rebooted and then Hitman Pro was run to see what, if anything was left behind.

1) ParetoLogic Anti-Virus Plus (Paid)

I had never heard of this before being asked to test this one. Overall, the GUI was simple and easy to use. Nothing exciting for a paid AV. Once installed, there is no integration into Windows Security Center, so Windows still thinks you need an AV product. RAM usage runs at about 20MB. Quite low actually.

ParetoLogice AV did not bad. It dealt with all of the malware and not once did the system become unstable. Once a scan with HMP was complete, it found and wanted to fix the following:

Invalid security zone
2 pieces of malware
Trojan
Rootkit

http://www.paretologic.com/products/antivirusplus/index.aspx

2) CA Anti-Virus Plus (Paid)

I had heard of CA of course, as they've been around a while, but haven't seen much testing or talk about them. After testing, well, I might know why now. Someone also asked me to test this one. GUI is fine and RAM usage however pretty low at only about 9MB. That is where the good news about this product stops.

During the links, about 16 of them, I did not receive a single prompt from CA, so no idea what it was doing, if anything. With about 3 links left, the computer completely restarted (a certain piece of nasty malware). Once restarted, there were several programs open and present on the screen including some porno image. The mouse was not present and I couldn't find any way to make it work nor could I get any keys or the Windows button to respond. I finally simply had to give up. CA gets a big FAIL.

http://shop.ca.com/ca/products/antivirus/antivirus_plus.asp

3) Ad-Aware Internet Security (Free)

Had some major problems downloading this. The download never seemed to want to complete. It finally did though. Once installed, it updated and said the system had to be restarted. Wow, that would suck, having to restart after every definition update? Not sure if that is the case, but I restarted. The program did not start with Windows. Saw nothing in Task Manager and no icon in the system tray. I tried to start it manually and it just sat there loading. Finally got a message about it being unable to connect to some server and it shut down. I tried reloading it again and it just sat there forever. I finally simply gave up.

Perhaps they were having issues last night with their server? Not sure why it wouldn't start with Windows on a reboot though, as it's pretty important for your AV software to load itself.

http://www.lavasoft.com/products/ad_aware_free.php

So I figured since ParetoLogic left behind quite a few things and CA had such a hard time, that I would try some of the older and well known apps out there. I'm not going to report on the GUI or RAM usage or provide any links for these guys...we all know what they look like, how they run and where to find them.

4) AVAST AV (Free)

The system remained stable after the 20 links, but upon scanning HMP found 3 trojans present on the system. While the system was still operating fine, 3 trojans is not great considering only 20 links were used.

6) ESET Smart Security (Paid)

The system remained stable after the 20 links, but upon scanning HMP found a piece of malware and a trojan present on the system. Again, while the system was still operating fine, not a great score after only 20 links were used.

7) Comodo Internet Security (Free)

CIS would have gotten a perfect score if it had not allowed that one link to reboot the system. Regardless though, when it did reboot, that same porn program was not running and I could use the mouse and had full control, unlike with CA. So the malware caused a system reboot, but CIS dealt with it once the reboot happened, as I imagine anything it got running, was in the sandbox and gone upon reboot. There was nothing running in memory and nothing found with Hitman Pro.

So, last night, with those 20 links, CIS I would say was the winner and it's free. I'm surprised by the performance of some of the paid apps out there quite frankly. CA in particular was bad. Hope you enjoyed the reviews. Remember that there is nothing scientific about these reviews and you could load one of those AV's that performed badly and throw 100 links at it next time and it would stop everything. You just never know.

 

TheIgster I enjoy reading your posts.I would like to see your results and opinion of Bluepoint Security sometime.

 

Thanks forty...I will add it to the list of things to test...no problem.

 

Thanks!

 

I also never heard of ParetoLogic, the results of all the others didn't really surprise me.

Some suggestions, although you probably received most of them already. But these are the programs which are the most interesting for me.

Paid:

1. BitDefender Internet Security 2011 and F-Secure Internet Security 2011 (so we can compare them)
2. Norton AntiVirus/Internet Security 2011
3. McAfee VirusScan Plus/Internet Security 2011
4. G Data Anti-Virus/Internet Security 2011

Free:

1. Panda Cloud Antivirus Free Edition 1.3
2. AVG Anti-Virus Free Edition 2011
3. Microsoft Security Essentials 2.0

Nice job and keep up the good work!

 

That's why ESET should add a Behavior Blocker, to deal with the stuff that might get pass the Heuristics.

Keep up the great testing!

 

Thanks for the posts guys. Suggestions are all noted.

 

Nice job for Paretologic for its small user database (well small compared to some of the products we all know).

CA does not show threat alerts (from what I have been told), it only logs them in the main app which explains why you may have not saw any. They use to do this in the past and I have not heard any different recently. Kinda sad results however knowing the large use of CA products.

 

My 2 cents worth....all paid ones....

Vipre Premium
AVG Internet Security 2011
Bullguard Internet Security 2011
Webroot Internet Security Essentials 2011


Just curious.I have keys for them all,and need to see how they do.

By the way, Great Job!!!

 

Wow...really opened a can of worms here. I'm glad you all seem to appreciate the testing. I will do my very best to test as many as I can. I might be able to test some more tonight. Thanks guys!

 

It would be nice to see how AVG (i'm testing the pro version,as i have a license) handles.Good work TheIgster,let us know another results!

 

I would like to see how Avast paid av does if the browser is sandboxed, using the Avast virtualization feature.

 

Is there a trial for the Paid version?

 

Not surprised with the outstanding results from CIS.. Gr8 work and Thanks

 

This should be the trial.

http://www.avast.com/pro-antivirus#tab4

 

Interesting. I always expect CIS to do well but I agree, paid apps should bring more to the table.

 

OK...added.

 

I would be interested in seeing something that as far as I know has never been done.

Collect a fresh set of new links every day and do your best to bring collection and testing as close together as possible. The more consecutive days this can be done the better and more accurate the collective results will be.

Take the piles of samples compiled over that period of time and retest them against the same scanners all at once when the regular testing has been completed.

In the end you will have 2 sets of numbers:

1. Aggregate detection percentages based on the average per day for each AV.
2. Percentage based on each AV scanning the full pile of samples collected throughout the test.

I am expecting that the '0day' list order will not be the same as the 'after it no longer matters' list order. Even if the order does not change much I am expecting the actual percentages to be way off. It would do the community a great service to once and for all prove that testing of samples older than 1 day old is meaningless.

 

What I would like to see is many av's tested in Windows 7, 32 bit with UAC enabled. Also, have the latest MS updates as well as java, adobe, etc all updated. Then test- that would be a real world environment in my opinion. At least the results would be much more valid than testing in XP with no MS updates and/or outdated adobe, java, etc.

 

Absolutely.

 

I did IT for nearly a decade, this is not real world.

No one updates anything that is not a forced update. I fixed a PC through remote support last night with JRE 1.4.2 and acrobat 8. I have also never seen 7 32bit in the real world when it was not intentionally installed by the user. These users are advanced users at least and do not represent the average use. They buy their PCs from the store and all of those have 7 64. Vista 32bit I could see but 7 32bit is not a common user OS.

I would say XPSP3 vista 32bit and 7 64bit together would cover the vast majority of windows users.

 

EDIT to add:

Yes I agree that UAC on and current MS updates are OK for testing as this does represent the majority of users.

 

Thanks for taking your time to test these programs. I would like to see results for MSE and AVG as well.

 

nosirrah,

Some interesting thoughts and yes, most likely something that would be very useful. Unfortunately I'm not sure I would have to time to devote to something like that. If I was being paid, perhaps, but as it stands, this is just for fun in my spare time.

As for updates and UAC, etc. mentioned by other users here...I do my testing pretty much exclusively on a Windows 7 64 bit system. This is mostly what you see out there when you go to buy a new system these days. As for UAC, etc. this testing is about testing AV's and their protection. UAC prompts or not allowing the download defeats the purpose of testing. At least the type of testing I do.

 

great work indeed... Paretologic uses Kaspersky engine, so no surprise..