www.av-comparatives.org explained?

I found a link to this website www.av-comparatives.org which tests various AV programs but I don't understand some of the terms used.

First of all what's the difference between the

On-demand,
Retrospective, and
ProActive

Then on the Retrospective / ProActive test they have several sub tests like

ITW-samples
NEW" zoo-samples "
"already known" zoo-samples


Could someone please clarify what these terms mean so I know exactly what I'm looking at.

Thanks

 

On demand is when you initiate a scan manually. ITW means in the wild or malware which is in common circulation. Zoo refers to nasties which may not be currently on the loose but are resting in their cages. The important point is that Kaspersky is usually at the top of any test, thats why I use it.

 

Why do they all fair so badly for "ProActive detection of ITW-samples" ?

 

This is a way to try to rate the heuristics by rolling back the definitions of each program back to see what it would still detect.

 

If I understand things correctly, the on-demand tests challenge the AV packages with currently known viruses (in the various categories) using current definition databases. It's a measure of how up-to-date an AV is at handling known viruses - be they in-the-wild or zoo based.

The retrospective/proactive tests challenge the AV with viruses which have appeared after the date of the signature database. In the specific report shown - AV databases were from February 6 2004, but the virus sets examined were new examples which had been collected between Feb. 5 and May 5 2004. This is a very challlenging type of test and specifically attempts to gauge how well an AV package identifies samples that have yet to appear using samples which "will" appear, i.e. how well the heuristic component performs in real life.

Blue

 

But why are the AVs more successful at detecting Zoo samples with Heuristics than the ITW samples? Shouldn't the % detection rate for an antivirus program be about the same for ITW and Zoo?

 

Heuristics attempt to identify malware based on programatic behavioral characteristics. If the behavioral characteristics of the ITW and Zoo samples are functionally similar, the answer is probably; if they are not, the answer is no. It can go either way (higher for ITW or higher for Zoo) depending on the nature of the samples that make up the challenge group.

Blue

 

So is it just luck that NOD32 did so much better than the other AVs for ITW proactive detection, or is that a result of ESET specifically tailoring their heuristics to detect viruses likely to be found in the wild?