AV-Comparatives Results Online

Good if you can read Russian

 

not all products can update engine and signatures separatly, so, no, i can not do it. anyway imo it does not make sense, the tests shows the proactive capability the products had in august, thats it. all products improve constantly, so we will see the improvements done so far in the next retrospective tests.

 

Yeah, I agree. I don't understand the point in wanting to run current engines with older signatures to see how they might handle threats during the period in question. Doesn't that sort of defeat the point of the retrospective test? Isn't the retrospective test supposed to give a rough indication how an actual user of that product might have faired during the period in question against these new threats on day zero? Actual users don't have access to an AV engine from 3 months in the future, so why should this test provide such an advantage? If an engine makes radical improvements, on many fronts, then this will surely be reflected in the tests next time.

 

Don't they have to ask or agree to be tested?

 

There is a translation at the end of the topic

 

read the translated reply. Looks strangely defensive to me. I mean did he complain when KAV scored standard++++ (or whatever the highest rank is) in the last retrospective test?

In any case, I agree.

 

imo he does complain mainly about the wildlist (=used itw samples), not about av-comparatives.

 

hi

thats good stuff Andreas

thats the whole point of this test, to see how well antiviruses detect samples they dont have a signature for

i'm eagerly waiting to see drweb 4.33 in the next proactive test

i deleghted to see that Eugene shares my opinion about the usefulness of vb test

 

Agreed. But I believe the question at hand is... do you run such a test with the engine as it was at the start of the period or the engine as it was at the end of the period? I meant to say that I see no reason to use anything other than the engine as it was at the start of the period.

 

And he is not happy

 

If I remember right, you wrote some months ago that you'll add False Posititive tests to your av-tests.

I'm just a curious to know how much these ProActive methods are producing "FP:s". Just because there have been said that DrWeb is "famous" of it's "FP:s", which actually are only "suspicious" detections to need some further examination. After these samples were submitted to DrWeb and they have actually been clean, DrWeb has corrected these detections within a couple of hours. So, the main question is, how much these ProActive methods will actually produce "FP:s" with the tested scanners?

Best regards,
Firefighter!

 

The FP set is still under construction. It will be a quite large one and included in the test of next year. So we have to wait for it.

 

Thanks a lot for your answer! So, I remembered right as well.

Best regards,
Firefighter!

 

You may wish that he (E.K.) looks strangely defensive, but that's not true. When he critized that ItW test in the Av-Comparatives test, he gave this announcement after that Kaspersky got the best possible ranking, Advanced + in the Retrospective test 05-2005. Why? Here is a part of what E.K. wrote:

> These tests are not exactly correct. They took about 54 malicious programs for the last month, called it ItW and scanned anti-virus applications by old databases. Full crap… We add more ItW a day. So it was rather limited and restricted choice with worms and bots in it. But the real ItW list is wider, higher and longer…

Why are new malicious programs detected by old databases (about Pro-active defense):

1. heuristic code worked

(see, http://www.viruslist.com/en/analysis?pubid=174405517)

2. "generic detection" worked

3. old malicious program is packed by a new packer that was forced

4. the mask just concurred accidentally

We are beating everyone in point 3. With the bots (most widely spread malicious programs) we are doing rather fine. Still having the same problem with point 1.

– the better we detect, the more false positives we’ve got.

====================================================

So, when he referred those 54 ItW samples, he actually meant those 51 ItW samples that were in the Av-Comparatives.org Retrospective test 05-2005, not those 8, that were in the latest Retrospective test 11-2005, when Kaspersky got only Advanced.

Best regards,
Firefighter!

 

No, he refers to the one of November, where the wildlist contained 54 samples.

 

OK, I missunderstood this maybe because you used only 8 samples from that 54 NEW ItW samples.

Btw, is it better to leave this "ItW" category totally away in the future, when there are actually hundreds of NEW infections each day and not ONLY 54 in a month?

Best regards,
Firefighter!

 

Yeah, if you read the report and also the negative comments about the wildlist you know how useless the wildlist is and what my opinion about it is...

 

I read the EK piece too of course and I read it as negative about the AV-comparatives test, Andreas. Can you ask Eugene what he meant with his remarks to make things more clearer?

 

Why Av-comparatives used old versions of Norton, KAV and Bitdefender, but used the latest version of NOD32 in the last retrospective test?

 

An old version of NOD32 was used, too... The number what you see is 2.51.8.. it is version of program components but not version of virus database. Virus database was taken from 8.8.2005( I don't remember this number exactly but I think it is something like that)

 

But why IBk didn´t use bitdefender version 9, for exemple?

 

Was that version available Aug 5th 2005 when this test was conducted ?

 

BD 9 came out shortly after IBK started the test. I remember IBK saying he tested BD 9 on the side to see if it had better detection than BD 8, it didn't because both use the same engine & definitions.

Test seems fair to me

 

Yeah, always the same questions. The date format is even explained in the tables. Of course the latest avaiable versions available at that date were used, in accordance with the vendors. fastgame is right.

@edwin: done. he criticize both, because the wildlist is crap and tests which are based on the wildlist are crap too due that. But as my opinion about the wildlist is the same and if you look in the report you see that i stated that the itw category is meaningless this time, i do not feel the test attacked, as it contains also all the other new circulating malware that were around in that period. ;-)