PCSL Total Protection Testing (2009 May's report)

Hello everyone,
I have finished the new May's report for PCSL Total Protection Testing.
http://www.pcsecuritylabs.net/news.php?readmore=29

And you can download report from here:
http://www.pcsecuritylabs.net/document/2009May.rar

I will be here discussing the details with all of you and your comments will be of importance to me

Jeffrey

 

Creating a readable report would be a nice start. I can't view the results in either Firefox or I.E.

 

Thks for the results, Jeff.

A suggestion: Since 12 products are almost neck-n-neck for the 5 star rating. Could you add details of what malware was missed by which product. So that one can evaluate better the difference between the products.

 

There should be an explanation on the website what exactly is...
- Detection Rate in Static Testing
- Total Detection Rate
- Final Score
- Award

But not... Marking system=（A+B）/C*100-lg(D+1)

Cheers

 

A2 and the false positives - a neverending story

 

You can check that in detection rate in the static testing

Hmmm, I will add manual in the package next time


Thank you for your suggestions. Good Luck

 

Hey, Panda did fair very well here (well? freakin' first ). Would you say that it's in the top by now and that their proactive features are really improving? P-CAV will probably be quite dangerous once Beta 2 is released. AV-Comparatives testing of Panda (ProtectStar reports) have been different when it comes to Panda...

 

Hi,

It seems that i am the only one unable to load PCSL Website, using OpenDNS as DNS provider



I have no idea what's wrong

Will try later but thanks for your work

Regards,

MaB

 

From those results it seems to illustrate difference in detection methods. For example, Avira and A2 seem to focus mainly on fingerprints or definitions while others like Bitdefender and Twister make heavy use of dynamic (non-signature detection as I understand it) detection. BTW what's up with Twister only having 1 FP!

 

LOL! Lately there is a reduction on Twister's FP. I myself have seen none for some months. They must be working on that. I 've also seen a reduction on the on-demand scores. My guess is, they have neglected signature adding, in order to put more manpower to the new version. As a result, the FDDS (the behavious blocker), has increased its share of detection (in the above test, the FDDS accounts for roughly 25+% of Twister's total detection!).

 

Thanks Jeffrey! Will check out the report asap.

 

Sorry to hear that, DNS problem of your ISP?

Waiting for your suggestions

 

Kaspersky was tested in automatic mode, isn't it?

 

it is

 

Hi Jeff,
That's just a number.
If you could display the exact malware missed. Then one can evaluate how good the protection of AV is. For ex: If ACME Anti Virus misses say conflicker.c sample then even though it has a 95% detection rate, its probably not all that good.

 

in proactive mode you will have complete power of KIS HIPS and detection/protection will be ~100% , you should consider running it in proactive mode for ("missed") "on execution" test next time...

 

Actually proactive mode is for advanced users, auto mode is for most of the users. Normanl users actually can not just whether it is malicious or not, anyway, through the test, KIS's auto mode is good and strong enough

 

You mean the exact missed samples' detailed information? But if one missed a lot, how can it be displayed

 

fair enough, but should be noted there is "reserve power" in KIS and protection can be further enhanced by adjusting KIS settings

EDIT:
BTW. this test is example in which way professional tests should go, if it is possible add for next testing small amount of "malware from the source", complete vector of infection to see particular antimalware solution in real environment, e.g. from malicious url to malware execution and disinfection...
many thanks for testing

 

That's why they offer proactive mode for the advanced users like you

 

Well, I thought it wouldn't be too hard for you. Since you seem to number your malware files with unique names. And if the AV is set to scan and delete/quarantine you will only be left with undetected samples in your malware folder.

So just see the files left and list out corresponding names. I think you could even right a script for it ( using grep ).

At this point, I am just making assumptions. So if your load is high, if possible list the missed malware you think were important. So that a reader can differentiate between the performance of all AVs getting 95+%.

 

well actually F-Secure should bump up a slot with 2010 and the Bitdefender engine instead of Kas 6. Along with their own engines to compliment it, looks like good times ahead.

 

Jeffrey, an idea to consider: It would be interesting to extend the analysis to pairs of anti-malware products, under the assumption that increasingly multiple security products are utilized for protection (e.g., a primary real-time anti-virus coupled with a secondary product for on-demand, manual scanning). Such an analysis would answer the question, “Which two products should I use in combination to maximize protection (and to minimize false positives)?” Obviously, not all products “play well” in combination with one another, so that practicality might limited the number the pairs examined.

Question: What is the process by which vendors are selected for inclusion in your analyses?

 

Jeff, thanks for your work and I enjoyed reading your test results.

I do have a question, though:

Is there a reason that you did not test AVG? It would have been nice to see how AVG compared to the other AVs tested, me-thinks.

Thanks in advance.

 

Hi Jeff,
thanks for your report.
A list of important false positive detected files would be interesting, too.
Regards,
Ford