AV-Test.org Review and Certification Report Q4 (2010)

smage,

Comodo reached 9.5.

 

Hmm...ok thanks.

 

Note that a sample size of only 20 was used for the “blocking of malware on or post execution” test. Thus, a difference in detecting a single malware instance could alter a product's test score by +/- 5%.

 

This ranking does not make any sense, I pick Gdata (take any other and will be the almost the same) to compare it with Comodo in the protection chapter.

Gdata Overall: 5/6
Protection against 0day malware: average 87 %
Bloking of malware: 65%
Detection 1: 100%
Detection 2: 100%
TOTAL average: 88%

Comodo Overall: 3/6
Protection against 0day malware: average 88,6%
Bloking of malware: 100%
Detection 1: 93%
Detection 2: 99%
TOTAL average: 95,15%

Avira overall: 4/6
TOTAL average: 84,4%

Norton overall: 5/6
TOTAL average: 96,2%

Webroot overall: 4.5/6
90.3%
95 %
92,6%
100%
TOTAL average: 94,4%

...

The same thing happens with any other combination of AV's, if you compare the tests using the same weighs for each test you will get a totally different ranking. It's even more weird the ranking if you compare Comodo with Webroot.

I can understand that they are using different weights for each part, I just wanted to show how easily the results can be changed giving advantage to some products over others.
I guess that avtest give all the credit to the detection part, the less important comparing with Protection and blocking (Dynamic and "Real test")

Still nobody knows what is the difference in the test btw the Protection test and Blocking test?... I would apreciate if somebody can give an answer.

In fact comparing the numbers of

This is not an AvsB it's just a random comparison while I was trying to understand better how avtest score the products.

 

After looking at it more, I have to agree about some of the summary scoring. While I think the individually detailed scores look very accurate, and similar to my own testing, I cant' figure out how they came up with what is a 3.5 vs a 5.5 Almost seems a little random.

 

does anybody have an experience (reg. perf impact on system) Comodo AV in CIS Suite??

bcoz i use Comodo Fw along with Avast AV and its far more lighter than KIS suite itself (i tested my self KIS trial for few days in my laptop) ...but in perf category KIS has edge over CIS....

guess ratings should be taken with grain of salt...

Thanks,
Harsha

 

What means reg. perf? registry performance? sorry

 

like how it impacts the system on daily usage -
- boot times
- browsing
- opening apps..

in my experience compared with Comodo + Avast combo, i can say that KIS is (based on my system, not necessarily true with others..)
- Boot times (too slower)
- browsing (not remember sorry)
- openning apps (bit slower)

Thanks,
Harsha.

 

I think that Comodo AV performance has not been tested still in a professional way, all I can tell you is that CIS wihout the AV it's transparent to the performance.

According to this the lightest free AV is avast.
http://www.av-comparatives.org/comparativesreviews/performance-tests

Some of the modules of Avast can be deactivated (not installed) and you will be protected anyway there was a post about this here in wilders

 

Thanks!
I guess CFW + Avast AV is a good option rather CIS suite bcoz u have the signatures of Avast and Comodo's Cloud scanner at your disposal...Which comes always handy when avast misses sth during my testing at MDL on day to day basis...

reg. Disabling, i'll search the forum...

-Harsha

 

My experience of Comodo without the AV-module is that it's extremely lightweighted while providing great protection; near perfect. I think that Comodo falls on the detection part where it isn't at all good, which is reflected in this test. That being said, even if Comodo fails to detect the malware, you're protected thanks to the "sandbox". The "sandbox" isn't really a true sandbox, but a virtualization technique based on what's in the cloud. I do not use Comodo as the whitelisting is lacking. The vendor does not seem to take PC games into consideration and it gives you too many pop-ups. That's just my personal experience. IMO, 'IMO' being the keyword here, Norton does a better job protecting you while not producing any annoying pop-ups and still have a great cloud detection. I cannot express and thoughts of the remaining suites tested as I haven't tested them. Neither do I use any of the products tested as I think they're unnecessary. Prevx would do just fine for me if I felt like using any conventional (although Prevx can't be seen as conventional) protection against malware.

 

You're not quite right in that statement. Microsoft Security Essentials is in that test as light as Avast when all the scores are summed up.

 

Sorry I just took a fast look to the report and I missed that detail, thanks for the correction.

 

Thanks Shadek for your comments.
reg. Usability i can confidently say that Comodo is very silent during my day to day usage and i don't get any pop-ups during the s/w installation (actually very very few and i don't play much games, last time i played is almost 6 moths ago) and i can say that Avast Behavior shield produces more pop-ups than Comodo...
like When i go into Inprivate mode in IE9 - i get a pop-up
and few more instances...

Thanks,
Harsha

 

A lot of people say told me they're not getting many pop-ups from Comodo, and I believe them (and you among them). Perhaps it's something on my end, even though I have a clean PC with just 36 processes running in the background. It simply does not suite me personally.

It's going to be an interesting year as more and more vendors move on to the 'cloud' and heading in the virtualization direction. We've yet to see the perfect technique but we're getting closer to that moment. So far, as anyone can observe in my signature, I think Sandboxie does the best job when configured to your own needs. Sandboxie 'invented' something new, and it's nice to see the vendors trying to beat its sandboxing technique. Competition is what brings the development of new, astonishing features to the light.

 

hmmm...Its something to do with games i guess...As you see i haven't played any games yet on CFW v5/v5.3 yet...probably thats the case..

Co-incidentally i have 56 processes running in the background..

Yes, Sandboxie did bring the revolution. Its just one maestro piece...
Ultimately its who the end users benefit from the positive competition.

Thanks,
Harsha.

 

Edit: See post 57, page 3 for enhanced chart.

Repair and "Usability" don't mean diddly squat when protecting your clean system. All Wilders members run clean systems, eh?

We all know MBAM, HMP and SAS are the tools to use for repair and one person's usability is another's pain in the...

Here's a compilation of the AV-Test scores that actually count for anything. (Hope I typed up all the data correctly. I triple checked. Honestly.)

 

*snip*

comodo missed 1% (over 200) wildlist sample detections in that test. i think it's safe to say that av-test weights wildlist misses pretty high and that missing 200+ threats in that test would cause them to rate a product lower. Webroot and G-Data (your examples) both didn't miss any wildlist detections and had decent/mostly-above-average scores in other sections, lending to their higher overall protection score.

 

I can't understand how AV-Test.org nowadays gives all the weight to an on-demand test, if they are gona give a 10% or 5% or even less weight to the important tests like the dynamic one we are comming back to the caverns.

 

I used to run Avira and CFW before changing into KIS, and everything was slower except for browsing (which has stayed as the same). Opening apps with Comodo + Avira was initially a bit faster, but later when I had hundreds of apps in the Defense+ application list, Comodo seemed to rewrite all of the app rules when writing new D+ rules, causing long slowdowns (over 10 seconds) when launching new applications. Does the newest version still do that?

Seems so. I think Comodo's protection score is too low when taking into account that it managed to block malware from infecting the system.

Congratz to the winners though. Good to see my AV get excellent results, especially after some tests where it has failed.

 

they also give high weight to dynamic tests. look at some of the failed products' protection scores. they may have gotten all wildlist threats, but with abysmal 0-day or dynamic detection scores, they fail pretty spectacularly.

 

guess it depends on the sample size, what was the sample size of the dynamic test compared to the on-demand test? (if it was a tiny sample size for the dynamic test then u really cannot give it too much weight since its too limited, but if the ondemand test had a very large sample size which i assume it did, u can put more weighting to it, if the dynamic test had like several hundred samples then it wuld be a diff story)

 

if you look at the reports, the sample size is given in each section.

0-day : 77
dynamic : 20
2-3 month old malware reference set : 642,693
wildlist set : 30,532

missing any percentage of the latter two means missing quite a lot of threats

 

The conclusion is that they made 4 tests and them the make 2 or 3 of them look irrelevant.

moral of a fable: the next time do it well or dont do it.

 

norton excellent