AV-Comparatives | Heuristic Behavioral protection test released!

That's what I get for trusting my memory instead of checking first. Just remembered reading about Vipre engine in Kingsoft thread. BTW, any idea what engine KS uses--or their own? They're not in any tests I've seen yet.

 

They use their own engine.

 

I am surprise at avast. All this while their heuristics is always below 50%, now they are at 70%. Not bad. Not bad.

I am also shock with Avira's heuristics. They have always maintain among like top 3 when it come to heuristics. But look like they slip.

On whole, most vendor improve their heuristic detection.

 

Hmm. I think I can conclude from all the real-world tests and this one that PCTools relies too much on the user making decisions. If all user-dependent situations were taken away they would score much higher in these tests.

 

Check the details of the test, it covers only samples of one day. Too bad that the test was not performed over a longer peroid, testing the samples from previous day. The scores could be entirely different one day later.

 

True.
I wonder which day av-c chose to test

 

you mean the samples appeared after the frozen database (aka zero day) e.g. Day +1, +2, +3, etc

 

Yes, the test is a 1 day freeze, so on the 3rd day, the product updates/detection definitions would be from day 2 and so on. Watching a larger period would make the test more reliable as the detection fluctuates every day.

 

Then I have to admit I dont understand the next statement...

 

yes, but If I am infected with a 0 day not detected, it makes no difference if Avira prompt me the day after, simply I'm infected !
Once again I agree with AV-C in their test metodology. A Heurstic test must see Av against 0 days virus and not 1 day or 2-3 days virus

Someone knows if test machine Operative System was a 32 or 64bit? Because Avira Proactive seems useless (see page 7), it does not help avira increasing detect %, and I know it does not work on 64bit systems

 

Can any one tell me how does Avast report if its heuristics detects a malware i.e Comodo if detects malware with heuristics the detection name contains the word "Heur", "Suspicious" so you know its an heur detection.

How does Avast detects with heur? I mean any way to identify if its an heur detection.

 

32-bit.

btw, the sentence "This test turned off signature protection" said in a post above by an user is wrong. Also generic signatures can catch new malware.

 

Thanks Mr. Clementi

 

I said that...

How can you test a product's heuristics specifically while leaving their signature components on as well? How do you know what's detecting what?

The whole point of specific type tests like this (aka non-whole product) is to test different aspects of products and their ability to function without other components of the product on.

 

Generic signatures that are designed to catch future variants from the same family... ...frozen database from a time before the sample was first seen. Generic signatures can surely be thought of as 'heuristics', as they are using previous experiences to solve the problem of future malware.

From 100,000 unique malware samples, the best most of us could do would be to make 100,000 signatures i.e. detection based purely on hashes. Talented AV/AM teams are able to get the total number of signatures much lower than this with generic signatures that catch a lot of known samples - as well as detecting future samples as well.

 

The test looks OK. At least from my experience with some of the contestants.
BTW, it's a shame about GFI and what they did to VIPRE...

 

The whole idea of retrospective tests is somewhat outdated, as detection methods you can freeze for such tests are only a fraction of protection methods todays av software does offer. The malware updates incredibly fast, if you want to test the real protection level, you need to reflect that. That's why I like the dynamic/real world test more, it includes all protection methods - if the test reflects how malware really infects systems (exploits etc.).

But even the dynamic/real world test has a speed problem: how long it took the tester to get the new malware sample, new URL? There is a time window of some hours (minutes?) between the malware initial release and the moment, the tester runs the test. Is the malware really zero day/minute or is it just new to the tester and in reality, was released days ago? Of course, how many users catch a new malware in the very moment it is released?

Because you cannot test the updated AV product from today against tomorrows malware (unless you have a time machine ;-) ), the idea of a retrospective test is to test today's malware against an old update of an AV product, in this case 1 day. It tries to simulate if your AV would catch the malware from one day in the future.

 

it is impossible to detect <malware newer than update> using typical signatures.

Detection methods used for catching new samples in this test are proactive, such as heuristics, generic signatures, behavior blockers