Test of AV's for the detection of contemporary polymorphous viruses

[Tommy]

Original webpage:
http://www.anti-malware.ru/

Babelfish translated

Testintroducion:
http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dnews%26newsid%3d513%26arc%3d0

Results of the test of antiviruses for the detection of the contemporary polymorphous viruses:
http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dpolymorphic

[C.S.J]

:thumb:

yep, another decent score for drweb.

silver is v.respectable,

whats interests me is, symantec and nod32 which usually get the 12 out of 12, didnt even better drweb.

all tests are different i suppose.

i aint surprised with the 3 gold awarded av's, as they are the ones with the high detections, so as-expected results.

but drweb impresses me further by still getting the silver with only a so-called 89% and 83% detection rates. ;)

[atlantis]

Avira v Kaspersky ( F-secure, GDATA ) who win? . For me winner is the best antivirus on this planet.

8)

[Zombini]

-{ Quote: "Avira v Kaspersky ( F-secure, GDATA ) who win? . For me winner is the best antivirus on this planet.

8)" }-

Amusing :-)

[Trespasser]

Avira AntiVir Classic is definitely impressive. All I can say is WOW...and it's free.

[Zimean]

Avira is rocking these dayz

[Sjoeii]

Very interesting

[ugly]

Very good results for Kaspersky.:thumb: ;D
As always on top.

[trjam]

Avira, leading the way ;)

[computer geek]

Well, mcafee isn't top of the list, but the results show it has a balanced detection. :thumb: Satisfied.

[Bunkhouse Buck]

-{ Quote: "Avira, leading the way ;)" }-

As it does most of the time...

[C.S.J]

-{ Quote: "As it does most of the time..." }-
well, i dont think it would for such things as removal, self protection, packer support etc.

there are many tests that 'could be done' and in all honestly, its only kaspersky who score high on them all, so if you rely purely on tests, kaspersky would easily be #1.

regards to this test, avira/kaspersky were expected to score high on this due to the high detections anyway, its simple mathematics, however... the ones to dissappoint are the top-tier-detections who score quite poorly, same goes for the ones who impress who are supposed to be 2nd-tier-detections who score well. etc.

[Bunkhouse Buck]

-{ Quote: "well, i dont think it would for such things as removal, self protection, packer support etc.

there are many tests that 'could be done' and in all honestly, its only kaspersky who score high on them all, so if you rely purely on tests, kaspersky would easily be #1.

regards to this test, avira/kaspersky were expected to score high on this due to the high detections anyway, its simple mathematics, however... the ones to dissappoint are the top-tier-detections who score quite poorly, same goes for the ones who impress who are supposed to be 2nd-tier-detections who score well. etc." }-

Back to Dr. Web, btw. I won't use Kaspersky because of a number of things, and really think the anti-spam module in Dr. Web is unsurpassed even though my ISP filters spam. I am getting a record amount of same, and Dr. Web gets at least 97% of it.

Avira may have better detection, but there are trade-offs with everything, and I like the good Dr. overall.

[TVH]

Only trade off for me with avira is the fp's when heuristics are set to high. Other then that it has excellent detection, fast scanning speed and its light. Great AV.

[Bunkhouse Buck]

-{ Quote: "Only trade off for me with avira is the fp's when heuristics are set to high. Other then that it has excellent detection, fast scanning speed and its light. Great AV." }-

I alternate between Dr. Web and Avira. I have never had a fp with Avira, contrary to what others have experienced.

[Firecat]

Whats interesting to me is that AVG seems to have put up a much better show than it did in previous AV-comparatives tests of polymorphic virus detection. Its a little bit strange, but AVG did claim improvements to polymorphic detection with every program update of AVG 7.5....:-\

[bellgamin]

Avira takes yet another blue. Ho-hum.:isay:

But I marvel at how well Avast did. Unexpectedly splendid!:thumb:

[031]

-{ Quote: "Avira takes yet another blue. Ho-hum.:isay:

But I marvel at how well Avast did. Unexpectedly splendid!:thumb:" }-
yes avast did great :) :) :) :)

[Diver]

Trend Micro looked pretty good. I guess that is one everybody loves to hate.

This forum is generating into an AV fan boy cheering section. You know, its only a computer program with either a free or modestly priced annual subscription, so why all the dedication and expressions of love and loyalty?

[Bunkhouse Buck]

-{ Quote: "Trend Micro looked pretty good. I guess that is one everybody loves to hate.

This forum is generating into an AV fan boy cheering section. You know, its only a computer program with either a free or modestly priced annual subscription, so why all the dedication and expressions of love and loyalty?" }-

Because as I have contended several times, there are likely a number of shills (hidden agenda promoters) for the AV they are fanboying (if I could coin a new term). But, we should be intelligent enough to have the ability to do some critical thinking and bypass the ********.

[Thankful]

In about seven days we are likely to see quite different results regarding polymorphic detection when AV-Comparatives results are released. Symantec has scored perfect results the last three tests. I look at the anti-malware.ru results with a fair amount of skepticism.

[Bunkhouse Buck]

-{ Quote: "In about seven days we may see quite different results regarding polymorphic detection when AV-Comparatives results are released. Symantec has scored perfect results the last three tests. I look at the anti-malware.ru results with a fair amount of skepticism." }-

You may be right, but the results can vary a lot depending on the test source. We often say that if your AV of choice scored well, the test was valid. If it didn't, the test was flawed.

[vlk]

-{ Quote: "In about seven days we are likely to see quite different results regarding polymorphic detection when AV-Comparatives results are released. Symantec has scored perfect results the last three tests. I look at the anti-malware.ru results with a fair amount of skepticism." }-

It should be noted that the test set for the Av-Comparatives.org poly test has not changed during the last three tests at all... (i.e. for ~2 years). Also, it contained some rather old / lab-only viruses.

The author of this new anti-malware.ru test at least used quite recent viruses, so the test result should be somewhat closer to reality...

Cheers
Vlk

[Thankful]

-{ Quote: "It should be noted that the test set for the Av-Comparatives.org poly test has not changed during the last three tests at all... (i.e. for ~2 years). Also, it contained some rather old / lab-only viruses.

The author of this new anti-malware.ru test at least used quite recent viruses, so the test result should be somewhat closer to reality...

Cheers
Vlk" }-
Thanks for pointing that out.

[Brian N]

Useless test, why don't they test DOS viruses instead? They're only slightly older than poly viruses.

[bellgamin]

-{ Quote: "Because as I have contended several times, there are likely a number of shills (hidden agenda promoters) for the AV they are fanboying (if I could coin a new term)." }-Probably NOT. Even worse than a shill is unsupported FUD.:dry:

[Bunkhouse Buck]

-{ Quote: "Probably NOT. Even worse than a shill is unsupported FUD.:dry:" }-

Your fear, uncertainty, and doubt. And, you don't need to shout.

I am telling people here the emperor has no clothes-and many can't handle it psychologically. True believers in a lot of the garbage that passes to the uncritical thinkers as something sagacious.

[bellgamin]

-{ Quote: "Because as I have contended several times, there are likely a number of shills (hidden agenda promoters) for the AV they are fanboying (if I could coin a new term). But, we should be intelligent enough to have the ability to do some critical thinking and bypass the ********." }-If you think that the Wilders Mods are incapable of dealing with shills & spam, then you should report it. That's what the exclamation mark in the upper right corner of a post is for. Otherwise, why demean an excellent forum with unsupported slurs and dire hints?>:(

Yes, there are members here who work for security software companies, or who test security software, or who moderate threads or forums for security software. They are well known to us. They are highly respected members. NOT shills!

[Bunkhouse Buck]

-{ Quote: "If you think that the Wilders Mods are incapable of dealing with shills & spam, then you should report it. That's what the exclamation mark in the upper right corner of a post is for. Otherwise, why demean an excellent forum with unsupported slurs and dire hints?>:(

Yes, there are members here who work for security software companies, or who test security software, or who moderate threads or forums for security software. They are well known to us. They are highly respected members. NOT shills!" }-

You and the moderators cannot possibly know all the players. My view is that you and several others, provide most of the assertions. Don't worry though, this place is a paranoia pit, and I have simply studied it for a year as a way to find humor and relieve stress!

[C.S.J]

the mods will never know if anyone works for an av, if the user does not want them to know.

a forum has always standard users, fanboys, staff, mods and admins, that is... what makes it a good forum :D

[Page42]

Thanks for posting the test results, Tommy.

I was pleased to see avast do so well. All the tables and figures made avast appear as if it would end up with a higher rating than a Silver Award and 25 points, but I guess it was that "low" score of 93.81% with the Allaple.4 family in Table 1 that dragged it down overall.

[Diver]

I am not ready to go so far as to call anyone a shill. Its more like they are overly excited about their choice of product. Its really such a waste.

There could be a lot of fruitful discussion here dealing with AV problems/bugs and testing methodology. Too bad so much of that is getting lost in the noise.

There was one very important comment here about the polymorphic virus sample sets being used by the various testers. That is the kind of stuff we need to know. Anyone can read the charts.

[Macstorm]

-{ Quote: "there are many tests that 'could be done' and in all honestly, its only kaspersky who score high on them all, so if you rely purely on tests, kaspersky would easily be #1." }-
I concur.

I fully rely on tests (and IME) ;)

[FRug]

@BrianN: err, maybe you should get your facts straight about the tested viruses first, before comparing them to DOS viruses. Those are very recent. Virut has made the Top10 in virustotal multiple times during the last 3 months with more than 30 different variants, the same goes for Allaple. The very first variant of Almanahe appeared in the 2nd quarter of last year, Twido (or Tvido) first appeared in November last year.

While this test of course has its limits due to the small amount of different variants/families (and probably the lack of verification whether the replicated samples are still running properly), it is not fitting to compare it to testing detection of DOS viruses.

[Bunkhouse Buck]

-{ Quote: "the mods will never know if anyone works for an av, if the user does not want them to know.

a forum has always standard users, fanboys, staff, mods and admins, that is... what makes it a good forum :D" }-

Gee, somebody with some sanity, and a clear unclouded perspective. And having made millions with computers (not bragging just a fact), I cut people a new asshole when I see posts that are clearly nonsensical and promotional in nature. If certain industry interests cannot take my heat, get out of the kitchen. If your products work as well as advertised, they will hold up to empirical testing and anecdotal user experience. Many do not on both levels, yet we see endless promulgation of mediocre (at best) AVs.

[Oldjim]

As a complete novice in relation to virus testing one thing puzzles me about this report. It states that a polymorphic virus cannot be detected by conventional signature tests and I would therefore have assumed that heuristics would have played a major part. However NOD32, which has one of the strongest heuristics, comes out worse than Kaspersky v7 which is definitely weaker in that area.
Can one of the experts advise where I have gone wrong.
Quick edit - I assume the dates given are incorrect -{ Quote: "Test was conducted in the machine under the operating system Windows xp SP2 in the period from 15 January through 20 February, 2007, in complete agreement with the methodology." }-

[Inspector Clouseau]

-{ Quote: "As a complete novice in relation to virus testing one thing puzzles me about this report. It states that a polymorphic virus cannot be detected by conventional signature tests and I would therefore have assumed that heuristics would have played a major part. However NOD32, which has one of the strongest heuristics, comes out worse than Kaspersky v7 which is definitely weaker in that area.
Can one of the experts advise where I have gone wrong.
Quick edit - I assume the dates given are incorrect" }-

Some Polymorphic Viruses using EPO; means that a emulator prolly never reaches that point where the virus gets activated. That means you have to write dedicated detection functions for some of the viruses. And besides that, most of the heuristic is trimmed to deal with the mass stuff, such as trojans etc.

[solcroft]

-{ Quote: "And having made millions with computers (not bragging just a fact), I cut people a new asshole when I see posts that are clearly nonsensical and promotional in nature." }-
I'm not surprised. By the nature of their work, shills have a vested interest in making sure they're the loudest (and even more preferably, only) huckster on the scene.

[bellgamin]

-{ Quote: "I'm not surprised. By the nature of their work, shills have a vested interest in making sure they're the loudest (and even more preferably, only) huckster on the scene." }-Agreed. And just who around here is behaving in such manner? No one. If someone does try to behave in such manner, all that anyone needs to do is press the exclamation button.

However, this forum is pretty much self-policing. In those rare cases when some bloke tried *shilly* schtuff (I love puns), the other posters quickly shot him down in flames.

Anyway, Solcroft, I hope your return means that your ISP has got you up and running again.

-{ Quote: "Some Polymorphic Viruses using EPO..." }-Can someone please explain what is meaning of "EPO"?

[lucas1985]

-{ Quote: "Can someone please explain what is meaning of "EPO"?" }-
Entry Point Obscuring/Obfuscating (http://www.google.com.ar/search?hl=en&q=EPO+viruses&meta=) :)

[bellgamin]

-{ Quote: "Entry Point Obscuring/Obfuscating (http://www.google.com.ar/search?hl=en&q=EPO+viruses&meta=)" }-Thanks Lucas. Oh my, that is really spooky stuff! A fascinating bunch of articles.

[lucas1985]

You're welcome :)

[solcroft]

-{ Quote: "Anyway, Solcroft, I hope your return means that your ISP has got you up and running again." }-
I'm accessing the net at uni, actually, where network policy is rather uppity about downloading executables, among other things. Thanks for the wishes though.

[A_Shabanov]

Now available official english version of this test on Anti-malware Test Lab official site:
http://www.anti-malware-test.com/?q=taxonomy/term/20

In the future we will try post immediately russian and english version.

[SecOmnius]

One more achievement for Avira ;D

[GES/POR]

-{ Quote: "Now available official english version of this test on Anti-malware Test Lab official site:
http://www.anti-malware-test.com/?q=taxonomy/term/20

In the future we will try post immediately russian and english version." }-

Can you include some others like f-prot and ikarus? Also how about that false alarm test?

[C.S.J]

-{ Quote: "Can you include some others like f-prot and ikarus? Also how about that false alarm test?" }-
hmm, its not coming next

packer support and keyloggers test are next. ;)

[GES/POR]

-{ Quote: "hmm, its not coming next

packer support and keyloggers test are next. ;)" }-

Tx Chris. Do you have a date for the p. s. n k. test? Sounds interresting.

[C.S.J]

nope, sorry... i dont.

they are in-progress though.

hope i aint breaking any barriers here, i apologize if i am :blink:

---

but, i too would like to see the 'dreaded false alarm test', because i have not seen them do one yet, and i HAVE seen a packer support one.

so, i suggest everyone go to Anti-Malware Test Lab (http://www.anti-malware-test.com/) and vote for the False Alarm test :D

[lucas1985]

-{ Quote: "packer support" }-
An anticipated failure, I'd say.

[C.S.J]

-{ Quote: "An anticipated failure, I'd say." }-
eh?

whys that? ;)

[lucas1985]

Because there isn't a reliable way to determine the unpacking abilities of an AV without knowing the inner workings of the engine.
Some food for thought here (http://www.wilderssecurity.com/showthread.php?t=175156)

[Macstorm]

-{ Quote: "One more achievement for Avira ;D" }-
Equal scores for Kaspersky & F-Secure ::)

[C.S.J]

-{ Quote: "Because there isn't a reliable way to determine the unpacking abilities of an AV without knowing the inner workings of the engine.
Some food for thought here (http://www.wilderssecurity.com/showthread.php?t=175156)" }-
ifs, buts and maybes.....

im glad there are other tests rather than 'mainly' just detection.

its nice to hear things from packers, keyloggers, self protection, rootkits and many of the other different things.

[lucas1985]

-{ Quote: "ifs, buts and maybes....." }-
:doubt: :what:
-{ Quote: "im glad there are other tests rather than 'mainly' just detection." }-
Testing of detection rates is very reliable, you detect or you don't. We can discuss the quality (it's a real virus or it's garbage) or importance (is a zoo, in the wild, rare virus?) of samples, but not a positive or negative detection.
-{ Quote: "its nice to hear things from packers, keyloggers, self protection, rootkits and many of the other different things." }-
Stefan's opinion (http://www.wilderssecurity.com/showpost.php?p=1199811&postcount=6) on testing of packer-based detection.

[C.S.J]

-{ Quote: "I don't get your point :doubt:
Stefan's opinion (http://www.wilderssecurity.com/showpost.php?p=1199811&postcount=6) on testing of packer-based detection." }-
thats regarding virusinfos test and how they label their results, nothing else.

[lucas1985]

Testing for the sake of testing is useless. Claiming that AV xxx supports packer aaa and AV yyy don't is very unreliable.

[C.S.J]

-{ Quote: "Testing for the sake of testing is useless. Claiming that AV xxx supports packer aaa and AV yyy don't is very unreliable." }-
i very much doubt the test will state such things,

they usually work and calculate using a point system, and then relevant awards are given out.

the work over at Anti-Malware are being recognised even more now, with Avira already adding the awards to their own website.

http://www.avira.com/en/company/awards.html

and drweb giving them a mention, here: http://info.drweb.com/show/3161/en

its another test to the list, whether people like them or not :D

----

they have taken my suggestion of a testing schedule and will post it as soon as one has been worked out ;)

[lucas1985]

We'll have to see, but I doubt very much that the conclusions of the packers tests will have some relevance.

[aigle]

-{ Quote: "Well, mcafee isn't top of the list, but the results show it has a balanced detection. :thumb: Satisfied." }-
Yes, too much balanced but the Balance is in the favour of viruses. :thumb: ;D

[Stefan Kurtzhals]

Because a packed malware is detected, you cannot say the product does support unpacking this packer.

Because a packed malware is not detected, you cannot say the product is not able to unpack this packer.

So how you want to test if a product does actually perform unpacking? Some products have detailed scan logs which indicate which packer they unpacked. But even those sometimes choose not to unpack a sample because it would be too slow.

To get back on topic, if a polymorphic virus detection is not 100%, it's a failure. Imagine a system infected with a polymorphic virus, thousand of files infected. The virus scanner only detects (and cleans) 99%, leaving 5-10 infected samples behind. The user will not be able to locate them - and will keep reinfect her/his computer again and again.
So, 99% for polymorphic viruses doesn't matter, it's not any good!
But also keep in mind, that replicating polymorphic viruses and creating a *good* set for testing is not a trivial task. Alot of the polymorphic viruses are buggy and destroy files during infection or create non-working decryption loops. So when some of the scanners in this test did not detect 100%, it may be that those samples are not working at all.

And yes, AVG 8 made a very big jump in detection of polymorphic viruses from version 7.5.

[solcroft]

-{ Quote: "Because a packed malware is detected, you cannot say the product does support unpacking this packer.

Because a packed malware is not detected, you cannot say the product is not able to unpack this packer." }-
Nope, you can't. But if you observe a scanner over a period of time against specific families and variants, or have ever explored its weaknesses in order to bypass it, you can usually get a reasonably good understanding of how it fares against this packer or that.

Still doesn't mean it's a scientific test, though, unfortunately...

-{ Quote: "And yes, AVG 8 made a very big jump in detection of polymorphic viruses from version 7.5." }-
I have to concur. They made a very big jump in detection is almost every type of malware, as far as I can see, and they seem to be quickly improving on script/exploit detection as well. A far cry indeed from the AVG 6/7 days, when it was worth exactly what you paid for it.

[The Hammer]

-{ Quote: ":thumb:


whats interests me is, symantec and nod32 which usually get the 12 out of 12, didnt even better drweb.

" }-An aberration to be sure.;)

[trjam]

Kudos to Avira, as always.:thumb:

[Bunkhouse Buck]

-{ Quote: "Kudos to Avira, as always.:thumb:" }-

Agreed. :thumb:

[Bunkhouse Buck]

-{ Quote: "well, i dont think it would for such things as removal, self protection, packer support etc.

there are many tests that 'could be done' and in all honestly, its only kaspersky who score high on them all, so if you rely purely on tests, kaspersky would easily be #1.

regards to this test, avira/kaspersky were expected to score high on this due to the high detections anyway, its simple mathematics, however... the ones to dissappoint are the top-tier-detections who score quite poorly, same goes for the ones who impress who are supposed to be 2nd-tier-detections who score well. etc." }-

The premise of the importance of after the fact is flawed. If you don't get infected in the first place, the rest is irrelevant. Avira has the highest probability of keeping malware off of my system- and that is what matters most to me.

[Arup]

-{ Quote: "The premise of the importance of after the fact is flawed. If you don't get infected in the first place, the rest is irrelevant. Avira has the highest probability of keeping malware off of my system- and that is what matters most to me." }-


The fact that Avira manages to top or come near the top for almost all tests thrown at it speaks volumes about the commitment of the Avira developers. In the end, not getting infected or being alerted to any malware is what a good AV is all about and in that regard. Avira rules.:thumb: