AV-comparatives proactive test predictions

[risl]

There is a test listed for november 2007, so why don't we start speculating on this. Personally, I'm waiting to see if Dr.Web 4.44 origin detections or rootkit detections have any effect. F-prot would be also intresting with the Eldorado thing. Also Kaspersky had new heuristics if I remember correctly.

So, what are your opinions? ;)


(delete/lock if there is already a similar topic, didn't find it)

[Avena]

Hi,
Yeah, I think you can do it.I too planned to start speculating on this.:o

[IBK]

{QUOTE-> F-prot would be also intresting with the Eldorado thing. <-QUOTE}
no, the programs are from august.

[C.S.J]

i expect drweb to score quite well,

however...

i fear for the FP rate,

i.e. Standard, but if FP's are lowered, Advanced.

[HiTech_boy]

{QUOTE-> There is a test listed for november 2007, so why don't we start speculating on this. <-QUOTE}

Well , 17 days before the results are going to released , don't you think it is a little bit early for such thread ?

[Peter2150]

{QUOTE-> Well , 17 days before the results are going to released , don't you think it is a little bit early for such thread ? <-QUOTE}


Not only early, but of debateable value. It's to easy to become A vs B and then closed. We shall see for a bit.

[JerryM]

{QUOTE-> Not only early, but of debateable value. It's to easy to become A vs B and then closed. We shall see for a bit. <-QUOTE}

Surely this group of open minded, unbiased, and highly intelligent folks would not do that, would they????? ;D Or would they?

Regards,
Jerry

[Peter2150]

{QUOTE-> Surely this group of open minded, unbiased, and highly intelligent folks would not do that, would they????? ;D Or would they?

Regards,
Jerry <-QUOTE}

Asked and answered.;D

[trjam]

Avira, fast scan, many FPs. The surprise may be Norton.;)

[JerryM]

I doubt that there will be much of a change in positions from the former tests.
Jerry

[Firecat]

{QUOTE-> I doubt that there will be much of a change in positions from the former tests.
Jerry <-QUOTE}
Yeah, me too. I expect very few surprises this time around. :)

[Diver]

Go sit under a tree and don't think about it for 17 days.

[dawgg]

{QUOTE-> Also Kaspersky had new heuristics if I remember correctly. <-QUOTE}
That's allready been tested...
v6 (old heuristics)...9%
v7 (new heuristics)...35%
They were very new heuristics, don't know if there have been any major updates though... the heuristics on Kaspersky's beta servers have been updated... doubt these will be tested exclusively.

Guess we can only wait and see till after the results are published... then we can actually comment upon the results... speculation isnt going to get us anywhere... need the true facts (and statistics) :)

[C.S.J]

all think all results will differ from the last test.

i think there will be quite a few changes compared to last test.

[TonyW]

Personally, I avoid making predictions, instead preferring to see Andreas' results when they do come out.

[kdcdq]

The only prediction that I can make with confidence is that the newly-rated best AV products will be the only ones that I am not using right now....::)

[212eta]

I am anxious about Symantec, ESET, and BitDefender! ???
Let's wait & see...:shifty:


Uuuups!!! It will be another "famous" retrospective test...
Meaning, a repetitive test on old stuff like nod32 v2.7, BitDefender 10 etc.

Better check at chip.de, virusinfo.de, pcwelt.de etc.

[apm]

Bitdefender looks done very good (http://blog.chip.de/0-security-blog/microsoft-verbessert-seinen-virenscanner-20070821/)


http://blog.chip.de/0-security-blog/wp-content/uploads/2007/08/anti-virus-test-2007.png

[Diver]

I predict the winner will be elected President of the US.

[Arup]

{QUOTE-> I predict the winner will be elected President of the US. <-QUOTE}


In that case I nominate Dr. Web;)

[dawgg]

{QUOTE-> Bitdefender looks done very good (http://blog.chip.de/0-security-blog/microsoft-verbessert-seinen-virenscanner-20070821/)


http://blog.chip.de/0-security-blog/wp-content/uploads/2007/08/anti-virus-test-2007.png <-QUOTE}
Slightly off-topic isnt it?... Thats regarding overall detection. This thread's regarding detection of threats using 3 months old signatures.

[Sjoeii]

I predict Kaspersky, Symantec and DrWeb to do very well here.

[C.S.J]

if you disregard signature detections for these results, drweb should do quite well but seeing as it still includes both, I ain't too sure.

I still think advanced, unless it gets many fp's :)

[IBK]

there will no big surprises I would say :-\ . I expected other results (I thought they will all score much higher and also some single products are guessed would score much better than others, but I was wrong).
what is maybe interesting is that many products have more FP's by signature.
just wait 11 days.

[C.S.J]

Doesnt sound good for the doctor that Ibk but well see :)

[lucas1985]

{QUOTE-> what is maybe interesting is that many products have more FP's by signature. <-QUOTE}
A worrying trend I must say :(

[Banshee]

{QUOTE-> Originally Posted by IBK
what is maybe interesting is that many products have more FP's by signature. <-QUOTE}

Interesting indeed.I wonder if drweb will be the one with more FPs. Hard to beat as it is but never say never.

[C.S.J]

i stand by my earlier prediction, advanced but lowered to standard for FP's.

even more expected as the .origin is tested this time around and its fairly new.

but im sure it wont be the worst detection out there :)

{QUOTE-> Interesting indeed.I wonder if drweb will be the one with more FPs. Hard to beat as it is but never say never. <-QUOTE}
lets feed the troll (once again...)

[Banshee]

{QUOTE-> i stand by my earlier prediction, advanced but lowered to standard for FP's.

even more expected as the .origin is tested this time around and its fairly new.

but im sure it wont be the worst detection out there :)


lets feed the troll (once again...) <-QUOTE}



What I said was re:

"Originally Posted by IBK
what is maybe interesting is that many products have more FP's by signature."

If you have nothing else but "let's feed the troll" or similar asinine comments you should take them somewhere else.

[Sjoeii]

That' a surprise than. I would have expected otherwise

[C.S.J]

{QUOTE-> That' a surprise than. I would have expected otherwise <-QUOTE}
what do you expecr? :)

[trjam]

{QUOTE-> Doesnt sound good for the doctor that Ibk but well see :) <-QUOTE}
I would say you can bank on that one.

[Sjoeii]

{QUOTE-> what do you expecr? :) <-QUOTE}
I expected Symantec, Kaspersky and Drweb to do surprisingly well.

[Arup]

{QUOTE-> I expected Symantec, Kaspersky and Drweb to do surprisingly well. <-QUOTE}


What??? ??? ??? No Dr. Web.....:o :o

::) ::)
;) ;)

[ultragunnerdcl]

I think Kaspersky, Nod32 , Bitdefender & Avira would do very well in that test.;D

[Stefan Kurtzhals]

Judging from the last few tests, I think comparing % of detection with absolute numbers (false positives) doesn't make much sense. Aswell as not taking into consideration the relation of detection ratio and number of false positives. Or settings the boundaries with no recognizeable mathematically system. But that is just my personal opinion. Statistics are something wonderful... ::)

[De Hollander]

Oh crystal ball...., avk,avira,bitdefender,kaspersky,nod32,trustport ;D...

[MalwareDie]

I predict Avira will get knocked down to Standard because of fp's but will score high. I wouldn't be too surprised if Avira gets the most signature fp's.

[De Hollander]

Look how fast the days go by ;D Tomorrow perhaps "the big day"??

[C.S.J]

Saturday,

i predict many FP's from everyone.

[zfactor]

yeah im waiting to see this one also..

[C.S.J]

i think nod will do very well, their proactive detection is what makes up their detection. :)

it will score high, with low FP's, probably Advanced+

i think it will be the 'only winner' from this test.

[De Hollander]

Looks like you are right, about the date, there's a topic "Retrospective Test of November 2007" (av comparatives forum)


{QUOTE-> Saturday,

i predict many FP's from everyone. <-QUOTE}

[C.S.J]

i think in the past, he has released it a few days early, so you never know.

[vtn54]

Results summary is out ;D

Nod32 and Kaspersky get Advanced+ :thumb: :thumb: :thumb:

[Don johnson]

http://www.av-comparatives.org

[dawgg]

Congratulations to Kaspersky for such a significant improvement and NOD for retaining its advanced+ level for retrospective tests for so long.

Good to see most AVs have got the same (or higher) level since its last retrospective test... awaiting for statistics now :)

Edit 1: actually, v7 did get Advanced level in the last retrospective test

Edit 2:... so rude, how can i forget; a BIG thank you as always to AV-Comparatives team and IBK for you hard work and effort :-*

[C.S.J]

I still wish this test would exclude signature detections to show how good the heuristics really are

[Firecat]

AVG getting "Advanced" puts this huge smile on my face.....Grisoft seems to be making huge strides in getting their heuristics up to the mark; and their signature detection is not so bad either, making AVG Anti-Malware effectively the most cost effective product on the market today :D

I think a lot of products have been penalized for FPs; I would like to see the details....but lets just wait ;D

[dawgg]

{QUOTE-> I still wish this test would exclude signature detections to show how good the heuristics really are <-QUOTE}
but this way it also shows how good the signatures are... the signatures which can detect many variants. At the end of the day, all AVs have heuristics+signatures to try and combat 0-day malware... at least this way it shows the effectiveness of the combination of them both (rather than just the heuristics, because no AV only uses heuristics).
After all, retrospective test is to show detection of 0-day malware and in the real-world, users will have their AVs with heuristics and signature detection enabled :)

[C.S.J]

Drweb has, I think we can be sure of this.

Also avira

[JerryM]

I'm surprised at F-Secure, which is one of my favorites.
Agree that AVG has made some very good strides in the last couple of years.
Jerry

[Firecat]

{QUOTE-> I'm surprised at F-Secure, which is one of my favorites.
Agree that AVG has made some very good strides in the last couple of years.
Jerry <-QUOTE}
I think F-Secure's "Standard" rating is more due to FPs than actual proactive protection numbers....:)

[MalwareDie]

I wonder if Fortinet was penalized for too many fp's even though the heuristics were turned off lol... or was it because it scored too low.

[tec505]

Please can anyone tell me where can I find these results?

Thanks
Mike

[ggf31416]

This time the test used sampled obtained only 3 weeks after the tests, not 3 months. That explains some of the "too good to be true" results.

[eBBox]

The page just says "comming soon" when I go there >:(

[LowWaterMark]

{QUOTE-> Please can anyone tell me where can I find these results? <-QUOTE}It's just a summary table graphic at this moment. Start at their main page; click the bold blue Comparatives link in the center; scroll down a little to find the sentence "To get an overview of the comparatives, we summarized them in tables. You can find them here." Click the link on the word "here".

[trjam]

Kudos to Eset and Kaspersky. In a round robin fight, a good betting man would wager these are the 2 that would be left standing in the end.

[tec505]

{QUOTE-> It's just a summary table graphic at this moment. Start at their main page; click the bold blue Comparatives link in the center; scroll down a little to find the sentence "To get an overview of the comparatives, we summarized them in tables. You can find them here." Click the link on the word "here". <-QUOTE}


Found.

Thank U.

Mike

[Thankful]

Thanks to IBK and his team for the hard work they put in.
Interesting that Microsoft is becoming a player in the AV area (Advanced).

[C.S.J]

Yes, but its not all that surprising for what money can buy

[Steel]

Still the version 2.7 was tested. Whether the new version V 3 would topping the actual result of V. 2.7 ?

[tec505]

1st of all I would thank IBK and his team for doing these tests, which are extremely important.

2nd I agree with Thankful: Microsoft will be great and may be (or really) some small players could declare bankrupt!! :o

Mike

[WSFuser]

Its nice to see Kaspersky having excellent heuristics. Good job to them.

And thanks to IBK for doing these tests.

[Sjoeii]

{QUOTE-> Kudos to Eset and Kaspersky. In a round robin fight, a good betting man would wager these are the 2 that would be left standing in the end. <-QUOTE}
Correct!!!:thumb:

[wir.sing]

Leaving out FP's what percentage is required to score an Advanced+ and just a "normal" Advanced?

[IBK]

40,0%

[Sjoeii]

Than some of us have made some great improvements

[Diver]

IMO, this test is obsolete. The next phase of malware detection will be behavior based. Products like Symantec Endpoint 11 or Threatfire in combination with the right signatufe based AV are likely to be the standout performers.

[pykko]

hmm.... strange Avira has again many FPs. I've spotted a decrease in their number on my own experience, but who knows... ???
1 December is the big date, isn't it ? ;D

[C.S.J]

{QUOTE-> 40,0% <-QUOTE}
maybe drweb will get close to this advanced plus! Detection target but lowered for fps

Has 40 percent always been the target for the top rating?

[IBK]

yes, always

[MaB69]

{QUOTE-> hmm.... strange Avira has again many FPs. I've spotted a decrease in their number on my own experience, but who knows... ???
1 December is the big date, isn't it ? ;D <-QUOTE}

Excellent remark,

Please IBK what version of the search engine was installed : version in use when you made the test or the version downloaded with the three months old definitions ?

Thank you for all your work.

Regards,

MaB

PS : i'm not an Antivir Fanboy ;D

[MalwareDie]

Avira needs to fix the html heuristic or their product will always be penalized for too many fp's. Every fp I've had in a while are all from the html heuristic.

[C.S.J]

but isn't the fp test based on scanner detection only? MD

[dawgg]

Looks like the scanner detects some html using heuristics... there were a few FPs by AntiVir named HEUR/Exploit.HTML.. AntiVir probably scans it using this heuristics while the page is being downloaded or when its on the PC (temp folder)

[wir.sing]

{QUOTE-> 40,0% <-QUOTE}

Thanks for the answer. I really "like" your tests. Good to see someone putting all the effort in there to try to make a good test. I was wondering that if you had time you would be able to do a comparison of the free versions some vendors offer. Namely BD free, Antivir PE classic, Avast! free and AVG free. I think it would be interesting how they perform and also how they compare to the different paid version.

[dawgg]

{QUOTE-> Thanks for the answer. I really "like" your tests. Good to see someone putting all the effort in there to try to make a good test. I was wondering that if you had time you would be able to do a comparison of the free versions some vendors offer. Namely BD free, Antivir PE classic, Avast! free and AVG free. I think it would be interesting how they perform and also how they compare to the different paid version. <-QUOTE}
http://www.av-comparatives.org/forum/index.php?page=Thread&threadID=749

[zfactor]

where is everone seeing this i cant find it anywhere?? thanks

[WSFuser]

LWM explains:
{QUOTE-> {QUOTE-> Please can anyone tell me where can I find these results? <-QUOTE}
It's just a summary table graphic at this moment. Start at their main page; click the bold blue Comparatives link in the center; scroll down a little to find the sentence "To get an overview of the comparatives, we summarized them in tables. You can find them here." Click the link on the word "here". <-QUOTE}

[LowWaterMark]

And with images. ;)

1. Start at the main page:

http://www.av-comparatives.org

2. Click on the bold, blue Comparatives link in the middle of the page:

195534

3. On the next page, scroll down and click on the bold, blue word "here" as shown:

195535

[zfactor]

i found it yeah sorry i read most of the thread earlier and didnt see that post. thanks.. looking at it now

[Macstorm]

Congrats to the folks at Kaspersky Lab! :thumb: :thumb: :thumb:

[C.S.J]

{QUOTE-> Congrats to the folks at Kaspersky Lab! :thumb: :thumb: :thumb: <-QUOTE}
kaspersky has such a good signature detection rate though, i wonder what the percentage were heuristic detections :shifty:

[Macstorm]

{QUOTE-> kaspersky has such a good signature detection rate though, i wonder what the percentage were heuristic detections :shifty: <-QUOTE}
don't underestimate them mate, they have been working hard on heuristic detection lately. Also, have you ever heard of a little thing called 'proactive defense' module? ::) ;D

[C.S.J]

i know all that MAC,

but i wont be convinced, as kaspersky as a high signature detection.

i just think it would be nice to see how much were heuristic detections, maybe it will show this in the reports?

---
nod would be a different story, as their software is based soooo much around their heuristics, so its a little more genuine.

understand what i mean?

i just think it would be nice to see signature detections removed from ALL, then show percentages :)

[Macstorm]

yeah i know what you mean Chris :)

Let's wait 'til saturday, we all expect a very detailed report.

[IBK]

KAV without the new heuristic would have scored like eScan, which got Standard. what escan detected is mainly due generic signature detections.

[C.S.J]

k ibk, you posting them already yet? ;)

[wir.sing]

C.S.J. the easiest way to see how good the KAV heuristic is, is to look at the results of the May "Retrospective/ProActive Test". In that one KAV v.6 was tested. It recieved 9%. Version 6 didn't contain any "real" heuristics engine so these 9% were basically all signature detection.

Now Kaspersky released shortly before the test their new Version 7, which contains the same AV engine as Version 6 + a new heuristics engine. So IBK did a single product "Retrospective/ProActive Test" of KAV 7 shortly after the May test. It used to same testset as the May test and is according to IBKs report comparaible to the May one. So in this test KAV scored 35%.

So you can see their new heuristics is quite good. And back then it had just been released. Looking at the results from the November test it seems that Kaspersky has used the time since then to further improve their heuristics.

[zfactor]

very nice results from kaspersky, i knew nod would be at the top..

[dawgg]

{QUOTE-> it seems that Kaspersky has used the time since then to further improve their heuristics. <-QUOTE}
...Or the same heuristics performs better with the samples collected for this comparative :)
Either way, KAV's done well to achieve advanced+ and implement such a powerful emulator and its paid off (v6 compared to v7) and NOD's done well to retain its level for so long

Other AVs need work done... although, they may be attempting to improve their behavior blockers (checking files upon execution) rather than heuristics (checking files upon scanning)... they're both very effective methods and both have their advantages and disadvantages :)

[C.S.J]

{QUOTE-> C.S.J. the easiest way to see how good the KAV heuristic is, is to look at the results of the May "Retrospective/ProActive Test". In that one KAV v.6 was tested. It recieved 9%. Version 6 didn't contain any "real" heuristics engine so these 9% were basically all signature detection.

Now Kaspersky released shortly before the test their new Version 7, which contains the same AV engine as Version 6 + a new heuristics engine. So IBK did a single product "Retrospective/ProActive Test" of KAV 7 shortly after the May test. It used to same testset as the May test and is according to IBKs report comparaible to the May one. So in this test KAV scored 35%.

So you can see their new heuristics is quite good. And back then it had just been released. Looking at the results from the November test it seems that Kaspersky has used the time since then to further improve their heuristics. <-QUOTE}

thank you, but i know this already ;)

i was merely asking why signature detections are included in this particular test

its not really checking the unknown-malware technology, if there is a known signature for it.

just asking, thats all :)

[IBK]

its not a heuristic test - generic signatures etc. also provide proactive detection.

[C.S.J]

{QUOTE-> its not a heuristic test - generic signatures etc. also provide proactive detection. <-QUOTE}
yep your right, but you know people judge this test on heuristics, which can (could be...) misleading.

are there plans to do a heuristic test in the future?

[IBK]

whats the sense of that?

[C.S.J]

{QUOTE-> whats the sense of that? <-QUOTE}
you dont see any sense in testing the heuristics?

[dawgg]

Neither do I see a point in testing solely heuristics... the test should be to see effectiveness of detection of 0-day malware... and when there is a 0-day on your computer, you're equipped with the strength of the signatures+heuristics, not only heuristics.
Having only heuristic detection is unrealistic because nobody sets their AV to this.
Heuristics+signatures = realistic because thats the setup for all AVs and the protection it provides

(you're also often equipped with behavior blockers, but thats another test in itself... very time-consuming!)

[C.S.J]

ok, i see your point.

i was just asking :)

[IBK]

i do not see a sense to do not not count also generic detections. what matters for the user is to be protected, not if the malware is detected generically or by heuristic. also some products do not make clear distinction between what is detected heuristically and what not, so how would you test products for heuristic alone?

[wir.sing]

On the other hand, any plans to make a test of (just) behavior blockers components?

[IBK]

yes .


LowWaterMark: Now that the full report has been released, let's continue in the following thread:

Av-Comparatives Results (http://www.wilderssecurity.com/showthread.php?t=193122)


.