New AV-Test.org malware testing (Avira finished 1st, CA eTrust finished last)

[InfinityAz]

I just saw this on AppScout, unfortunately they don't give a link to the actual results (so if anyone knows please post). Here's the link to the text (with the actual text following it):

http://www.appscout.com/2007/05/antivirus_shootout_in_magdebur.php#more

AV-Test.org, based in Magdeburg, Germany, has just released results of an exhaustive malware-detection test. They threw over 600,000 malware samples at thirty-odd antivirus programs and measured how many each product detected. That's just detection - there was no effort in this test to measure whether the products could clean up the malware they found. The only real surprise to me is how well the best products did; quite a few reached 98% or even 99% detection.

The samples shook out into four distinct categories - Trojans, worms, backdoors, and bots (zombies). Only Windows-based threats were used, and only current threats (meaning they're no older than 12 months). Overall, the products were most successful at detecting worms and bots. The range of ability in detecting Trojans was quite a bit wider.

Limiting the list to products that we've reviewed in PC Magazine, the top scorer was Avira's Antivir, with 99% detection overall. F-Secure, Symantec, and Kaspersky all came close with 98% detection. Avast!, AVG, and BitDefender weren't far behind at 96%. From that point, though, scores start to trail off. Panda got 92%, Trend Micro got 91%, NOD32 88% and McAfee 87%. Microsoft's OneCare didn't come in dead last, as it did in an earlier test, but it only detected 81% of the threats.

Grisoft's Ewido Anti-spyware didn't do so well, detecting only 75% - but then, it's strictly an anti-spyware product, not an antivirus. As noted, AVG (also from Grisoft) scored 96%. VirusBuster, sold both as a standalone product and as the licensed antivirus in Agnitum's new security suite, only detected 73%. And bringing up the rear, Computer Associates eTrust-VET antivirus detected just 62%.

That's quite a range! And it would surely be a still greater spread if AV-Test had gone on to analyze how well the products removed what they detected. How well did your antivirus do?

Posted by: Neil Rubenking (PC Mag)

Results:
Avira's Antivir - 99%
F-Secure, Symantec, Kaspersky - 98%
Avast!, AVG, BitDefender - 96%
Panda - 92%
Trend Micro - 91%
NOD32 - 88%
McAfee - 87%
Microsoft OneCare - 81%
Ewido Anti-spyware - 75%
VirusBuster - 73%
Computer Associates eTrust-VET - 62%.

[NAMOR]

Wow Trend made a huge jump in detection when comparing this test to the PC world one.

[NAMOR]

I found this link on PCMAG.COM. I'm surprised at the detection of Rising.

http://www.pcmag.com/article2/0,1895,2135053,00.asp

-{ Quote: "

AV-Test.org Reports Stats from Antivirus Roundup
05.22.07

By Larry Seltzer

V-Test.org, an independent testing group at the Otto-von-Guericke-University (Magdeburg, Germany), tested 29 anti-malware products with a very large set of files (606,901 to be specific). The goal was to test detection capabilities only, not cleaning. Products were set with their most aggressive detection options, such as using all heuristics and testing inside archives.

In the test set were:

* 68,864 backdoors
* 407,487 Trojan Horses
* 47,891 bots (zombies)
* 82,659 worms

Some of the results:

* The best product, WebWasher, detected 99.83 percent, but this is a gateway product. The best desktop product, at 99.56 percent, is AVK 2007.
* The worst product, Computer Associates's eTrust-VET, detected 62.12 percent.
* The average product detected 86.95 percent, the median 90.97 percent.

All products were last updated on Friday, May 18th, prior to the test. Only current malware was used, meaning all samples were seen in the last 12 months. Only Win32 malware, not 16-bit Windows or DOS, was used, and all malware had to be functional, as opposed to corrupted or benign samples.
" }-

http://img.photobucket.com/albums/v219/NAMOR/PCMAG.png

[AshG]

This gives me some real food to chew on. I know there's no single uber-test that will give a definite answer as to which program is the best, and even the top tier testing sites have different rankings. The one thing I'm sure of is that NOD32 seems to be slowly sliding down the rankings. It's still at the top, but it's consistently at the bottom of the top lately and that concerns me.

My Eset license runs out in July, so I have a month and a half to make the hard choice - Kaspersky 7.0 or Eset v3. I've happily sent Eset my money ever since I stumbled onto v1 but it just hasn't been the same since the good Inspector moved on. Gah, decisions....

[Firecat]

Now THIS is interesting. Lots of great twists to the tale this time :D

1) AVG - 96%

I do not have any idea why AV-test uses AVG Pro instead of Anti-Malware in all its tests (at least this has been the case in all its past tests), but this score for AVG Pro means that the competitors had better start looking out. Even AVG Free will score the same in this particular test due to absence of adware/spyware samples. :D

2) Rising - 96.02%

Again AV-test is in stark contrast to what we saw on AV-comparatives about Rising, even if we discount the virus and otherOS malware as well as script malware categories from the AV-comparatives test. This AV is definitely something worth tracking. :D

But from a honest perspective, Rising has been working hard to add samples in the past few months. So, this detection rate is a good surprise to see. :)

3) BitDefender - 95.68%

BitDefender scored slightly less than I expected, but its still OK :P

4) Trend Micro - 90.97%

This AV differs from the PC World test results because the PC World test results had additional categories, mainly file viruses, polymorphic viruses, script malware etc. So speaking purely from a perspective of Trojans/Worms/Bots/BackDoors, it seems Trend is not so bad.

5) Dr.Web - 85.84%

:(:(:(:(:(:(:(:(:(

6) F-Prot - 85.27%

I don't know, this result seems very strange to me, did Marx test version 3.x again? I expected a bit better than 85% from F-Prot. :-\

7) NOD32 - 88.32%

This score is again somewhat inconsistent with AV-comparatives even if we discount the scores NOD32 achieved in the Viruses/Script malware/OtherOS malware categories :-\

Overall, it is VERY NICE to see regular tests coming out from AV-test. Gives regular eye openers. I hope AV-test continues to publish tests frequently throughout the year. :)

[Jarmo P]

Avira's AntiVir results are for the Premium edition I think, but they are consistent with most of the recent tests I have seen.
Have not seen any tests lately for AntiVir Classic though it might detect also some malware by heuristics that are not classified as viruses&trojans.

Same time I again got yesterday the large 1MB Avira update that did not require a reboot. And together with Sandboxie and Comodo running, Comodo started after the update telling me about invisible applications starting firefox instead Start.exe from SB, so Comodo was somehow in a mess, but reboot solved the problem.

NOD32 is definately on the downhill ride.

[NAMOR]

Maybe some can answer a simple question for me. How is it that some AV's tested on AV-Test.org and Av-comparatives have very similar detection % and others are completely off? If it's because they use different test beds, wouldn't we see more of a discrepancy between all of the AV in detection % instead of the 3 or so AV that have drastic differences.. You have the high-detection AV's that are comparable on both sites, then you have some mid-detection AV that are comparable, and even some lower-detection ones that are also comparable. But, then you have a few that are just all over the place in-terms of detection %'s.

[Durad]

For me not trustworthy test.

We calculated once that this collector is "collecting" several malwares every 10 minutes 24/7, 365. No sleeping, eating, working, relaxing etc... :)

He is using similar way to test detection rate as Virus.gr used. The diference is that Virus.gr is on some kind of black list while this one is not, guess why :)

For me this looks like a mini war between antivirus vendors and their secret weapons :D

[FRug]

Durad: av-test is THE most reputable test centre out there, they employ 15 people not doing this as hobbyists but as their main job for years (av-test exists since '91, as a company since '96).
Their test centre is running more than a hundred PCs and they have more than 60Terabyte of test data (malware/clean).

Comparing their infrastructure and professionalism (i.E. they check whether all files in the testset are actually still executable and not dead samples) to some greek VX kiddie with an attitude problem is ridiculous.

[Firecat]

-{ Quote: "some greek VX kiddie with an attitude problem" }-

Can you explain what you mean by "attitude problem"? ???

[The Hammer]

-{ Quote: "Can you explain what you mean by "attitude problem"? ???" }-LOL. I don't think FRug had you in mind. NObody thinks of you as a geek VX kiddie.

[C.S.J]

firecat dont forget, they are still testing 4.33

so to come within a few percent of nod32 and mcafee and beat f-prot, its not too bad :)

yayyy for panda

i too wish they would test removal and all the other little details.

[EliteKiller]

Overall I am very impressed with AVG's performance. Why would they test Ewido when it is not an AV, and why don't they refer to it as AVGAS? McAfee has better detection than the good Dr. :P

[Firecat]

-{ Quote: "LOL. I don't think FRug had you in mind. NObody thinks of you as a geek VX kiddie." }-
No, I was meaning to ask what he meant by saying VirusP had an "attitude problem". :)

[trjam]

Wow, Avast and Trend Micro are both better then Nod. I just dont see how they are going to turn things around. Not good at all.
Kudos to Avira, even with some current issues to work through.

[FRug]

-{ Quote: "No, I was meaning to ask what he meant by saying VirusP had an "attitude problem". :)" }-
1) he's a VXer freely exchanging stuff with anyone who can provide him with more samples, that alone is unprofessional and as such an "attitude" problem when it comes to computer security (at least on my list)
2) he doesn't listen to criticizm or tries to improve his testing methods. To me that also qualifies as "attitude problem"

[Blackcat]

-{ Quote: " F-Prot - 85.27%

I don't know, this result seems very strange to me, did Marx test version 3.x again? I expected a bit better than 85% from F-Prot. " }-
The big difference between this result and Command AV, suggests that the new version was tested.

[BlueZannetti]

To me, there's a bit of a disconnect on the testbed numbers.

607,000 samples, all within the past 12 months and supposedly fully functional. That means an average number (based on a 5 day workweek, no vacation days) of 607,000/260 ~ 2335 samples/day validated as functional. That's a major effort, possible, but quite large.

The other thing that is as one might hope and expect, on average the results largely agree with AV-comparatives. There a percent or three here and there above and below, but on average most results are in-line. However, both NOD32 and F-Prot are well outside the concordance achieved with all other products examined in both tests. The disagreement of both are a little more than two standard deviations away from the mean disagreement - which would suggest that a little test tweaking might be in order - one side side or the other. The primary difference, I believe, is the relative newness (past 12 months only) of the testbed for AV-Test.org

Blue

[MalwareDie]

I am quite suspicious of the 900 000 samples one month ago and now the 600 000 samples. I do not think all the samples were new.

[Diver]

Has anyone seen a response from Eset (or one of their boosters) explaining the rather poor showing of Nod32?

On the other hand, AVG (without anti malware I assume) had an unexpectedly good showing, compared to other tests.

[apm]

Avast! 584,574 96.32% is way above Nod32 536,043 88.32% & Dr Web 520,959 85.84% this time, impresive:o

[Legendkiller]

Wow,norton detected more samples then even kaspersky...:o :o

It seems norton has sustained its good performance since the test done last year....where it faired well.;) ;)

surprisingly nod and mcafee didn't rather well...:-\ :-\ ,i hope things will improve with the new versions...::) ::)

[Legendkiller]

and wait a minute....:lurking: :lurking:

avast and avg have done better trend,nod,mcafee,panda........i mean...8) 8)

[budfox]

Its good to see Fortinet finish high. I hate software Av's...they always seem to F'up XP kernals.


I hardware firewall/AV/IPS has zero impact on system stability/performance. Also all computers on the network are all protected.

If you like software av's I tried out AVK 2007 and really liked it. Make sure to turn off registry protection. It also has a really cool feature to disable realtime for x minutes (5mins to 8 hours i think). If you using a CPU intensive program, but you dont want to forget turning the reatime back on, this is a perfect thing for a AV program to have.

[C.S.J]

norton is still a massive player in the av market, and with improved 2007 version, they must be laughing, as they are back on top.

and yes avg as always, provides quality... and yes i still use the suite on the PC.
(kaspersky licence doesnt get used, and my trend expired)

[midway40]

Hasn't Norton always had a high detection rate but just used to be tough on computer resources? I hadn't kept up with it since when I last used it in '04.

And with eTrust's low detection rate they still offer that $1500 malware compensation? :wacko: (for those not familiar with this, see Firecat's post (http://www.wilderssecurity.com/showthread.php?t=172960&highlight=eTrust))

[Durad]

-{ Quote: "Hasn't Norton always had a high detection rate but just used to be tough on computer resources? I hadn't kept up with it since when I last used it in '04.

And with eTrust's low detection rate they still offer that $1500 malware compensation? :wacko: (for those not familiar with this, see Firecat's post (http://www.wilderssecurity.com/showthread.php?t=172960&highlight=eTrust))" }-

It used to be several years ago. It is improving as I could see.

WHen I test samples online at Virustotal Avast, Norton and AVG rerely detect someting. It could be just for the area where I live.

Files are usually detected by AntiVir, KAV based, NOD32, Panda heuristic, CounterSpy VIPRE and VBA32 heuristic. Very often WebWasher as well.

[Thankful]

If you believe these results, then F-Prot has taken a major step backward.
.... Highly unlikely.

[VikingStorm]

-{ Quote: "Hasn't Norton always had a high detection rate but just used to be tough on computer resources? I hadn't kept up with it since when I last used it in '04.

And with eTrust's low detection rate they still offer that $1500 malware compensation? :wacko: (for those not familiar with this, see Firecat's post (http://www.wilderssecurity.com/showthread.php?t=172960&highlight=eTrust))" }-
And they were also slow on updates, one of the last to finally get rid of only weekly updates for consumers (not sure how they kept that policy for so long). They are still the slowest to respond to new malware, which I don't understand why they would be, since they technically have more resources than everyone else.

[midway40]

Yeah I was aware of the slow updates. I had forgotten about when they used to never update the consumer versions on weekends except for high level threats.

Maybe they are depending on SONAR since at least in one instance it had blocked a trojan (Storm Worm) from spreading on Norton computers. I don't know if this is good or not, it really depends on how good SONAR is.

[mrhero]

For more detailed results, here is the link : http://www.sunbelt-software.com/ihs/alex/marx/detections_2007q2.htm

[ASpace]

Another crappy test !
Why ? Do you believe Symantec gets more than Kaspersky and NOD32 ?
Do you believe Rasing AV is better than NOD , Bit Defender ,Panda , the other competition ?

I don't ! Real world results show completely different numbers !

-{ Quote: "
I'll just copy and paste the same answer as in NOD32 subforum..." }-
I did the same

[RejZoR]

I'll just copy and paste the same answer as in NOD32 subforum...

Don't underestimate Symantec just because their NAV2004/2005/2006 line was crappy...

[JerryM]

-{ Quote: "Another crappy test !
Why ? Do you believe Symantec gets more than Kaspersky and NOD32 ?
Do you believe Rasing AV is better than NOD , Bit Defender ,Panda , the other competition ?

I don't ! Real world results show completely different numbers !" }-

Hi HTB,
I am curious as to how you determine "real world results" and the numbers associated with those results.

Thanks,
Jerry

[ASpace]

-{ Quote: "I am curious as to how you determine "real world results"" }-

Well , I have been working as an IT for 2 years and I do see the difference between products.These tests definitely does not match my experience although it is not big.

[JerryM]

-{ Quote: "Well , I have been working as an IT for 2 years and I do see the difference between products.These tests definitely does not match my experience although it is not big." }-


I do not in any way doubt your experience, but since you mentioned numbers, what are those in your experience?

Regards,
Jerry

[Firecat]

-{ Quote: "Hi HTB,
I am curious as to how you determine "real world results" and the numbers associated with those results.

Thanks,
Jerry" }-
"Real world results" is a word used by many vendors when their products score somewhat less than expected in reliable tests. No need to elaborate, but this has been put forward in the past.

@HiTech_Boy: So, if your personal experience is so good then maybe you should start your own AV-test. :)

I also have ~1000 samples, and I also see some things, but what I see in my sample set does not influence my opinion of any products in general....

Regarding Rising AV, I would like to ask whether anyone has sent Rising a few samples in the past. They don't respond to virus submissions via email, but unlike a few vendors they rapidly add definitions to their database. We're talking ~5000 samples in a week here (ask Pykko, he has submitted thousands in the past). The problem is that they do not have good sources in Europe and the Western World to acquire samples, and this is why detection rates are not up to par. They definitely have the technical capacity to be good in detection rates, but not the resources unfortunately.

However, Rising has been improving fast in recent days, and these results from AV-test suggest that AV-test's sample sources are quite varied when it comes to geography. Not that I'm sure about it, but putting two and two together from what someone at Rising told me and the results of Rising at AV-test and AV-comparatives, this seems obvious....

[MalwareDie]

Rising is not that good. you could ask some of the members of this forum that reside in China and they would tell you that. I do not and wil never trust AV-Test.

[Firecat]

-{ Quote: "I do not and wil never trust AV-Test." }-

The industry trusts it and you don't? ???
Wow, so suddenly Andreas Marx has become some German VX kiddie in the eyes of people....
But yeah, Rising surely displays some strange results, as does Ikarus and I have no idea why...

[MalwareDie]

Doesn't Ikarus have paranoid heuristics, like Fortinet? I think those 2's actual detection rates are lower than that and their detection rates are just being bumped up by their paranoid heuristics.

[Inspector Clouseau]

-{ Quote: "@HiTech_Boy: So, if your personal experience is so good then maybe you should start your own AV-test. :)
" }-

:o :o :o :o :o :o :o :o :o :o

[ASpace]

-{ Quote: "I do not in any way doubt your experience, but since you mentioned numbers" }-

Sorry , I wanted to say results ;D

-{ Quote: "So, if your personal experience is so good then maybe you should start your own AV-test" }-
2 years is not so much... ;D Nice joke ;D

-{ Quote: "I also have ~1000 samples" }-

Unlikely you and Pykko , I don't collect malware . Don't have time for such thing.

What I was talking about is that wherever I install NOD32 , it always finds much more than people's previous AV . Lots of Panda and Symantec users were scared when they saw their computer were so infiltrated . I mean , during these two year , I have never seen a case where Symantec and other "popular" vendors detect more than the small vendors .

I am more than happy with my current choice . What is important to me is the real world experience. Two weeks ago , while surfing the net , NOD saved me from a IRC worm (it detected is as a probably a variant of...) . I submitted it to VirusTotal and not many AVs detected it (including popular products with better results here) . So this is what I care , such numbers are not important when there is nothing to back up

Have a nice evening all :)

[lucas1985]

Inspector,
Did AV-Test.org test F-Prot 6?

[Inspector Clouseau]

i assume so

[pykko]

First of all this is a nice test and the results confirm my findings for several vendors:Symantec, Kaspersky, Rising, Avira, NOD32. I don't know about the others.
Norton is a very strong AV even if it has some things that users hate: resource hog and perhaps the old thought they're adding samples very slow. They've changed. They have at this time the best submission and adding samples system... mostly automated. ;)

-{ Quote: "
What I was talking about is that wherever I install NOD32 , it always finds much more than people's previous AV . Lots of Panda and Symantec users were scared when they saw their computer were so infiltrated . I mean , during these two year , I have never seen a case where Symantec and other "popular" vendors detect more than the small vendors . " }-
The same thing happened to me when I installed Kaspersky or Avira after NOD32. They find a lot of malwares missed by NOD32. ;) But that's not a point. Every AV will miss something other eventually will catch.

Now, I don't want to be misunderstood, but generally speaking when NOD32 is ranked very low you bash the tester. When nod32 is up everything is fine.

-{ Quote: "I am more than happy with my current choice . What is important to me is the real world experience. Two weeks ago , while surfing the net , NOD saved me from a IRC worm (it detected is as a probably a variant of...) . I submitted it to VirusTotal and not many AVs detected it (including popular products with better results here) . So this is what I care , such numbers are not important when there is nothing to back up " }-
Yes, the real world... it shows me similar results with av-test.org ones. Every AV has its up and downs. Maybe nod32 will get up again soon.

[C.S.J]

i personally dont use the 'results' but more of the ranking order of av's in the test.

many people can count percentages, but i honestly dont think the differences between the different companys are not as big as these tests (inc av-comparatives) show or lead people to believe.

i think it fits that you have norton/avira/kasperskys near the top.

then with bitdefender/nod32/trend/mcafee and panda

and then f-prot/drweb

sure, there are many above fprot and drweb and even more so, on the ones below them, but i dont believe the differences are not as big as these tests lead people to believe.

dare i use the words real world, i dont know... but i tend to go just off the ranking, rather than counting percentages.

its nice that drweb still have 4.44 to be tested as soon as its released, sometime soon and still v5 in a few months (beta) too.

[Technic]

Well said pykko.

Couple of things I would like to mention about Norton. Their latest products like N360 and NIS2007 are really light indeed. Detection rate is reasonable too.

So I have one suggestion: Please try these suckers before judging them (oh, and this is not for pykko).

Actually I am concerned why KIS/KAV products 5.x/6x/7.x are slowing my computer down more than Norton's products. Its very hard to find to answer to this, but Blue said something about heavy disk IO etc. I can confirm that. HDD is hyperactive, web browsing is slow and opening file folders are slow. Thats why Kaspersky wont get a pole position, unless they fix these issues. I really like the security feeling (more than feeling) provided by Kaspersky, but the rest...:gack:

My system is AMD Athlon 3200+ 1GIG ram.

[EliteKiller]

-{ Quote: "It also has a really cool feature to disable realtime for x minutes (5mins to 8 hours i think). If you using a CPU intensive program, but you dont want to forget turning the reatime back on, this is a perfect thing for a AV program to have." }-

Kaspersky has had that feature for quite some time. AVK uses a KAV engine.....;D

[rdsu]

The last results of avast! Pro and AVG Pro are excellent!!!

Even better when we can have a free AV with so good detection... :)

[The Hammer]

-{ Quote: "Has anyone seen a response from Eset (or one of their boosters) explaining the rather poor showing of Nod32?

On the other hand, AVG (without anti malware I assume) had an unexpectedly good showing, compared to other tests." }-There's no panic for this user.:P No plans to board a lifeboat and lower myself over the side of the ship. As far as explaining the showing of NOD32, I can't, as these things are generally too complicated and not my field or even my hobby. This kinda reminds me of a slow speed rollercoaster ride where NOD will be headed back up eventually. I also didn't detect any consternation or panic in the Inspectors posts either concerning his new baby (F-Prot). So I'll relax outside and have a beer.;D

@trjam. Like the new avatar.8)

[trjam]

can I have one of those beers.::)

[C.S.J]

yep, IC didnt look too worried, i thought he would have argued it, would have been more interesting to read :)

isnt that his usual way ;D lol

[The Hammer]

-{ Quote: "can I have one of those beers.::)" }-Sure. As we all know Canadian beer is superior to American brands so I know you'll enjoy it.;) I've done the testing so I can vouch for the results.;D

[flyrfan111]

I was also expecting the Inspector to expand our knowledge with his input.

[EliteKiller]

-{ Quote: "Another crappy test !
Why ? Do you believe Symantec gets more than Kaspersky and NOD32 ?
Do you believe Rasing AV is better than NOD , Bit Defender ,Panda , the other competition ?

I don't ! Real world results show completely different numbers !" }-
A few of the Castle Cops MIRT team stated the same thing on the SAS forums. :-* :wacko:

Obviously AV-Test.org and AV-Comparatives must be padding Symantec's results because it is impossible for them to be doing so well. :-\

-{ Quote: "@trjam. Like the new avatar.8)" }-
It looks like the flavor of the week is Kaspersky! ;) However I think my K is a lot sexier. :P

[C.S.J]

-{ Quote: ";) However I think my K is a lot sexier. :P" }-
i agree :thumb:

lol, doesnt beat mine though.... :blink: ;D

[Diver]

Its kind of hard to say what real world results are. From my point of view an AV must be trouble free. That means very few false alarms and no noticeable system slowdowns.

KAV never disappoints with the numbers, but can sometime be a pain to use, although less so on today's very fast computers.

I don't like finding half or a worm in the apple. One example is with Nod32. It scanned some file on my machine for several weeks and detected nothing until I ran the darn thing. Probably it was packed with something Nod32 could not open. Fortunately, I run as a limited user and nothing happened. I also restored a fresh image backup for good measure.

What I look for in tests done by others is consistency. It looks like Nod32 is not being consistent. Many others are. By the way, Eset recently had a 3 day update outage for trial users.

Nothing is perfect. The first release of ZASS was a disaster with the AV crashing all over the place. Symantec, which usually has the lowest count of false alarms in most tests just had a major mess up for its Chinese users. I suppose they only sold 246 copies and decided to show those pesky Reds something.

Finally, the real surprise is AVG. While this one sometimes does poorly on tests, it is a favorite of many, especially those involved in repairing computers. Len Silverman the technology columnist for the Houston Chronicle recommended using it on Vista, in part because Vista's native anti spyware program offered a good compliment. It costs nothing, updates reliably and is easy to use. Full system scans are very slow.

[Legendkiller]

norton has done consistently well for some time and to my great surprise people are now doubting AV-Test to not give the due credit to norton..

norton has really done some major improvements with their products and with their detection-rates....

so,people can choose to ignore norton's excellent performance,if they don't like it doing better than their favourite AV's...

[Durad]

-{ Quote: "Durad: av-test is THE most reputable test centre out there, they employ 15 people not doing this as hobbyists but as their main job for years (av-test exists since '91, as a company since '96).
Their test centre is running more than a hundred PCs and they have more than 60Terabyte of test data (malware/clean).

Comparing their infrastructure and professionalism (i.E. they check whether all files in the testset are actually still executable and not dead samples) to some greek VX kiddie with an attitude problem is ridiculous." }-


So who is paying these 15 people at av-test full time?

[jubilee]

the first 6 positions is not a big surprise

but

very good news about avast! and avg pro

bad news about bitdefender thow......

[BlueZannetti]

A direct comparision vs the most recent AV-comparatives is shown below for the overlapping entries. The difference column is a simple arithmetic difference, the normalized difference is centered on the sample mean (average difference) and normalized by the sample standard deviation.

As once might expect, the average difference of (AV-Comparatives) - (AV-Test) is positive - the testbed was restricted to material less than 12 months old, so detections should, on average, be lower and they are. A more detailed breakdown of the www.AV-Test.org results by category examined is provided here (http://www.pcwelt.de/news/sicherheit/81346/index2.html). The results there do imply that the lower detection rates of some products were not due to major issues in a single category, but reflected a global perspective of the testbed.

Still, owing to the large size of the testbed, the discrepancy seen for F-Prot and NOD32 is really not expected if the testbeds were globally equivalent. Now, for some products results are effectively equivalent, so there's a bit of a disconnect. For a much smaller testbed, these types of deviations should, in fact, be expected.

The simple occurrence of a deviation doesn't mean either result is "wrong", simply that they're measuring somewhat different attributes. Types of circumstances along these lines could include shifts in the geographic origin of the malware with associated somewhat different local coverage, non-malicious portions of the malware not being detected, and so on. One would have to examine the comprehensive scan results to assess why the results differ and whether that is a material difference.

Blue

[Doc Serenity]

I'm not an expert at anything related to computers.
But I do find this topic interesting.
Whether you look at the performance numbers or the rankings, it looks like Webwasher and AVK2007 are the top 2, closely followed by Avira.
Webwasher appears to be a corporate product and I haven't found anything from AVK in English so I don't see me going there.
I agree that there is always a difference between a controlled test and actual performance.
But I think that if I use one of the higher ranked av's with other products to give me a layered approach along with limiting user accounts, that I should be pretty safe.
I see the differences between the av testers but I doubt that any of them would be stacking the deck for some reason. That would be a quick way to lose any income they might be making.
And thanks to all of you for my ongoing education.
Doc

[NAMOR]

-{ Quote: "A direct comparision vs the most recent AV-comparatives is shown below for the overlapping entries. The difference column is a simple arithmetic difference, the normalized difference is centered on the sample mean (average difference) and normalized by the sample standard deviation.

As once might expect, the average difference of (AV-Comparatives) - (AV-Test) is positive - the testbed was restricted to material less than 12 months old, so detections should, on average, be lower and they are. A more detailed breakdown of the www.AV-Test.org results by category examined is provided here (http://www.pcwelt.de/news/sicherheit/81346/index2.html). The results there do imply that the lower detection rates of some products were not due to major issues in a single category, but reflected a global perspective of the testbed.

Still, owing to the large size of the testbed, the discrepancy seen for F-Prot and NOD32 is really not expected if the testbeds were globally equivalent. Now, for some products results are effectively equivalent, so there's a bit of a disconnect. For a much smaller testbed, these types of deviations should, in fact, be expected.

The simple occurrence of a deviation doesn't mean either result is "wrong", simply that they're measuring somewhat different attributes. Types of circumstances along these lines could include shifts in the geographic origin of the malware with associated somewhat different local coverage, non-malicious portions of the malware not being detected, and so on. One would have to examine the comprehensive scan results to assess why the results differ and whether that is a material difference.

Blue" }-


Thanks Blue for doing the comparison. Some of what you said brings me back to my old stat. class (which I hated). haha.

[yeuxbleus]

-{ Quote: "Now, I don't want to be misunderstood, but generally speaking when NOD32 is ranked very low you bash the tester. When nod32 is up everything is fine." }-

Yep. I've noticed the same.

-{ Quote: "Actually I am concerned why KIS/KAV products 5.x/6x/7.x are slowing my computer down..." }-

My processor is similar to yours:

-{ Quote: "...AMD Athlon 3200+ 1GIG ram." }-

and I don't experience slow downs with version 6.0.621. :)

[ablatt]

I wonder if the new NOD Beta 3 would do any better?

[Thankful]

The only way to resolve this issue is to do a comprehensive analysis of the results. Unlikely to happen.

[Blackcat]

Apparently AV-vendors who attended the Frisk conference were given (http://vba32.de/wbb2/thread.php?threadid=163) the malware before the test date.

[C.S.J]

thats totally not fair if this is the case........ :o

[CJsDad]

-{ Quote: "Apparently AV-vendors who attended the Frisk conference were given (http://vba32.de/wbb2/thread.php?threadid=163) the malware before the test date." }-

Anyway to find out which participants received the malware, did all of them or just a select few?

[Caimbeul]

-{ Quote: "Apparently AV-vendors who attended the Frisk conference were given (http://vba32.de/wbb2/thread.php?threadid=163) the malware before the test date." }-

If this is true the test is not worth a cent. Its an very unfair and important advantage given to some produkt companies! :thumbd:

-{ Quote: "Anyway to find out which participants received the malware, did all of them or just a select few?" }-

If i read the forum messages correctly only some participants have received the samples.

Anyone can proof this statement?

This would damage the credibility of av-test.org very seriously! :o

[Firecat]

-{ Quote: "Anyway to find out which participants received the malware, did all of them or just a select few?" }-
Add ~15GB worth of malware in 7 days? Next to impossible. It is more likely Andreas Marx/AV-test had already finished testing by the time the results were announced....Because in most cases the tests are done beforehand and results announced later. Besides, I doubt Andreas openly announced that "Hey, I'm releasing a test next week", so the question of whether vendors "optimized" their sample set is MOOT. I'm pretty darn sure Andreas is aware of this possibility and hence would have kept it in mind while doing the testing.

Besides, if this had been the case, many AV products that are not at the top would be so, for example, NOD32, F-Prot. Even AVG scored well in this test despite there being no Grisoft representative in the conference. :)

What Andrey explained in the VBA32 forum is just his theory, this doesn't mean anything about whether this is true and amounts to cheating or not. :)
I do not know Andreas Marx personally but I have seen a few of his writings and I do know he's far from stupid and is quite well oriented technically....

[Blackcat]

The first day of the conference was 15th May, the AV's were last updated on the 18th.

Even if vendors were given the samples on the first day would they have time to add them all?

Credit to Andreas to passing on the samples to vendors who did not attend.

[C.S.J]

-{ Quote: "Add ~15GB worth of malware in 7 days? Next to impossible. It is more likely Andreas Marx/AV-test had already finished testing by the time the results were announced....Because in most cases the tests are done beforehand and results announced later. Besides, I doubt Andreas openly announced that "Hey, I'm releasing a test next week", so the question of whether vendors "optimized" their sample set is MOOT. I'm pretty darn sure Andreas is aware of this possibility and hence would have kept it in mind while doing the testing.

Besides, if this had been the case, many AV products that are not at the top would be so, for example, NOD32, F-Prot. Even AVG scored well in this test despite there being no Grisoft representative in the conference. :)

What Andrey explained in the VBA32 forum is just his theory, this doesn't mean anything about whether this is true and amounts to cheating or not. :)" }-

i do agree that the tests were already done, however.....

its still an unfair advantage to the av companys that didnt go, as 15gb of new samples arrives at their doors for nothing, with no effort.

[IBK]

-{ Quote: "Even AVG scored well in this test despite there being no Grisoft representative in the conference. :)" }-
just a sidenote: there were at least 3 peoples from Grisoft there ;)

[ren]

hello,

i agree with firecat. and now that other compagny get a link, nothing seems to be unfair. it's just, perhaps, malwares taht av-test do not use anymore, malwares older than 12 month, dos and 16bit. who know.

[Firecat]

-{ Quote: "The first day of the conference was 15th May, the AV's were last updated on the 18th.

Even if vendors were given the samples on the first day would they have time to add them all?

Credit to Andreas to passing on the samples to vendors who did not attend." }-
The thing is that I'm pretty sure not everyone knew WHEN Andreas was going to publish the test results. Therefore, 15th and 16th May was in the conference, 17th May all analysts get home, and 18th May products were updated. How much time to add samples? One day, or maximum 2. The fact is that major analysts were at the conference, it is very difficult to send 15GB worth of samples to other company analysts via email, so the samples were likely received via disks or through download links (unlikely because the collection was already being distributed), so most likely it was distributed on DVDs.

So in the end, analysts had only the day of 17th and maybe some small part of 18th may to add definitions. How many would they add in that period? It would be very difficult for such additions to cause any significant impact on the test results.

[Firecat]

-{ Quote: "just a sidenote: there were at least 3 peoples from Grisoft there ;)" }-
Oh? :)

Sorry then, my mistake :)
Thanks for clarifying. ;D

[Firecat]

-{ Quote: "i do agree that the tests were already done, however.....

its still an unfair advantage to the av companys that didnt go, as 15gb of new samples arrives at their doors for nothing, with no effort." }-
The others also did get it, you know, with no effort on their part. Its not like VBA32 had to go request those samples from Marx, from what I see Marx gave those samples of his own will. :)

[Firecat]

-{ Quote: "hello,

i agree with firecat. and now that other compagny get a link, nothing seems to be unfair. it's just, perhaps, malwares taht av-test do not use anymore, malwares older than 12 month, dos and 16bit. who know." }-
ren,

you bring up a very interesting point. Why would Andreas Marx share samples at the conference before the actual test itself?

1) To show some of the problems AV-test has faced in the past and present with regards to sorting samples
2) To give some vendors the samples beforehand so as to reduce his workload later

Out of these scenario 1 is more likely and yes it is quite possible he may have showed older samples, because since these were used only to showcase problems, old or new doesn't really matter. The samples might still be important for companies which are new, i.e. for example VBA32.

Of course, if the reason is number 2, then I have explained this in my previous posts ;)

[dan_maran]

Much credit goes to Andreas Marx for his sharing of samples with the AV companies who didn't attend, this is a true testament of his willingness to help out the end users in their protection level(s).

As for sorting and adding/sorting 15gb worth of signatures in 3 days for an unannounced test, sorry no way.
Maybe md5/crc32 detection but no "real" detections.

Correct me if I am wrong Inspector/IBK/Stephan/AV experts in general :)

[Thankful]

The results seem inconsistent for NOD32 between the PC World Test
(http://www.pcworld.com/article/id,130869/article.html) and the most recent test (http://www.sunbelt-software.com/ihs/alex/marx/detections_2007q2.htm)

For example, Backdoors (95% (April) / 86.64% (May) ;
Bots (94/93.72) (the same) ; Trojans (89%/86.76%).

Inconsistencies exist with all the AV vendors.
Both tests use very large samples sizes.

[Durad]

-{ Quote: " The samples might still be important for companies which are new, i.e. for example VBA32.
" }-

VBA32 is not that new company ;) They are just new for English market.

[Firecat]

-{ Quote: "VBA32 is not that new company ;) They are just new for English market." }-
For that matter, neither is Rising. But they still deeply appreciate samples no matter how old or new they are, because not being in the English market for so long they have not quite paid focus for the malware spreading around in the English speaking regions. ;)

[Firecat]

As an update to this thread, I did contact Mr.Marx about it, and he was kind enough to explain in detail a few comments about this latest test from AV-test.org. Since the comments are quite detailed by nature, I think its better to do this post by post.

1) Regarding the somewhat "strange" detection rates of Rising, NOD32 and F-Prot when compared to AV-comparatives (for example):

-{ Quote: "
This test was entirely focussed on the detection of Win32 PE malware.
Due to the fact that all "old" samples are removed (we consider "old"
that the sample was older than 12 months or that it was not seen
within the last 12 months), many companies who have entered the
anti-virus market slightly later than the "big players", would
perform better when compared with tests where even very old samples
are included in the testbed. Rising, for example, is one of the new
players and they will not detect old (outdated) malware very well,
however, they are trying hard to ensure that their current detection
rates on important malware are going up.

We currently have more than 4 Mio. infected files in our malware
collection, however, we are only using malware which can still be
considered as a risk to users. Lets say, you are detecting 10% of the
3.4 Mio. "old" samples we have not used for the test, but 99% of the
new samples (0.6 Mio.), the product would be ranked as being worse,
as it's total detection score would be less than 25%. However, it's
indeed an effective protection against new threats!

Lets say, another company is detecting 98% of all "old" samples, but
only 45% of the new samples (which were still spreading within the
last 12 months), this scanner would detect almost 3.8 Mio files in
total or about 95% of all samples. Of course, this impression is
wrong, as you would rank a good scanner (first example) as bad and a
scanner which is getting worse (second example) still very good.
" }-

The statement makes good sense to me, and I agree that an AV which does well on detecting newer threats provides an effective protection for "today's malware". :)

More to come in following posts.

[Firecat]

2) Regarding the version of AVG tested (AVG Pro or Anti-Malware)/Some comments on product selection in general

-{ Quote: "
We tested an AVG version with anti-malware databases from Ewido. Please keep in mind that we at AV-Test doesn't make the decision which products and editions to test, but the magazines does it, after consultancy with the AV companies. Therefore, sometimes AVG Professional (as 'pure' AV solution) gets tested and sometimes, the magazines wants to review the other editions. However, we were also asked to test the scanner from Ewido separately, to see how good it performs in certain categories, even if it's more or less labelled as an anti-spyware-only product. So we did." }-

So basically, AVG's Anti-Malware edition was used, which would explain the quite good detection rates (not that AVG Pro is bad though). As you can also see, there is some interest from the magazines in the AVG AS product, so it is also tested. :)

Those editions and versions are tested which are requested by the magazines. :)

[Firecat]

3) I did ask about why AV-test does not test ArcaVir and also about why there are detection rate differences between F-Prot and Command AV. You can see the comments below:

- Regarding ArcaVir:
-{ Quote: "We don't have a current version of Arcavir right now which includes all important features we need to have in order to perform a test. For example, the logging is essential to see which files are malicious or infected (by what kind of malware?). We are not using any numbers displayed by the program, but only the report files of the scanner we parse and compare with the files we have in the database in order to check if all files are scanned, which are flagged as being infected or suspicious and the like. These data is also used to create cross-reference lists of malware names etc.
" }-
- Regarding F-Prot and Command AV:
-{ Quote: "In case of F-Prot, we have tested the new version 6.x, of course. We are also a bit surprised about the detection rates, but the version is new on the market and I'm sure that the team is intensively working on new detection routines, so the product will perform better in upcoming tests. Command, on the other hand, is still using the old F-Prot 3.x engine as OEM version, with different definition files. If you compare the Command and F-Prot results, you'll see that the new version is indeed an important milestone." }-
So basically F-Prot will get better over time, but I am wondering why Authentium has not upgraded to the 6.x engine. :-\

[Firecat]

4) I did also ask about whether AV-test would test Virus Chaser (its anyway like Command AV in relation to Dr.Web because it has its own definitions files along with Dr.Web's database). But the comments probably will apply to test any particular AV in general, as long as its more than just a clone. :)

-{ Quote: "Sure, we can, as long as it will fullfil all criteria to be tested -- the most important one are useful and complete report files, the other one is that the testing can be automated in some ways, as our malware collection is split into many different parts (4096 directories which are containing the files). Besides this, it should not crash (or at least not too often :-) ) when scanning larger sets of malware files." }-

[Firecat]

Mr.Marx also kindly took the time to address some of the other concerns that were being displayed/showed/posted here on Wilders.

-{ Quote: "
I also saw some remaining points regarding the samples used for the
testing. We are extensively using databases and related technologies
to track all samples, for example, to separation between "old" and
"new" malware is an automated, database-driven process. We're using
many e-mail and network honeypots and we have automated systems which
are browsing with unpatched browsers through the internet, in order
to collect as many malware files (plus all these ad- and spyware) as
possible. We can see that e-mail worms are not very common anymore,
we are getting a lot more malicious samples when browsing through the
web, one keyword might be "drive-by installations" using all kind of exploits.

These downloaded files and their behaviour is then analysed by other
automated systems in a secure environment, in order to find out what
they are doing, from which URLs they want to download further files,
which files are dropped and the like, in order to make the life as
tester a lot easier. Please note that all collected stuff is made
available to the AV companies on a daily basis as download." }-
-{ Quote: "
Furthermore, we are making collections of files which are considered
to be more important than others and which are gathered from other
sources available to the AV companies, too, usually on a monthly or
bi-monthly basis. For example, the 16 GB of RAR-compressed and
PGP-encrypted files I made available on a portable disk during the AV
Testing Workshop were uploaded to our sample server already on May 6,
2007. We informed the AV companies about the availability of the new
samples shortly after this, with the detail that they can also
receive the files during the workshop -- with the result that some
wanted to download the files (what migth take a few hours) and some
found it better to copy the files directly from my disk to their disk
(what should only take 2-3 minutes.) I've send a reminder out on May
23, 2007 that I'm going to delete the files now, in order to upload
new data soon. Please note that no AV company knew about this test
report or any deadlines in advance." }-
-{ Quote: "
This means, the comment from the VBA32 team that they performed so
bad in our test, because they did not had the samples, but all other
had, is pointless (see:
<http://vba32.de/wbb2/thread.php?threadid=163>). They had at least
three opportunities to get the files, on May 6 (after my first
announcement), during the AV Testing Workshop week (where they did
not attend) and then again on May 23, 2007 after my reminder. Maybe
they have already downloaded the files earlier, but a different
researcher was working on the files, so the one hand was not knowing
what the other one was doing (that's pure speculation, of course)?" }-

[Firecat]

And as the last paragraph of the email Andreas also wrote this :)

-{ Quote: "Besides this, I want to emphasize that it is not our task to feed the
AV companies which new malware files -- it is primarely their turn to
gather malware from all available sources and they should not only
rely on testers which are sending them everything what they do not detect." }-

And with this, I end my long series of posts :P ;D

I would like to thank Andreas Marx for taking the time to explain all this in detail, it is very highly appreciated. I hope these comments help people to understand this latest test a bit better. :)

Oh BTW, Mr.Marx has already seen this thread ;D

As you can see, the tests are still very darn reliable, IMO they're as reliable as it gets. :)

[Blackcat]

Thanks, Firecat for obtaining this information :thumb: . Very interesting reading.

Let's hope these emails are not pulled :P

[EliteKiller]

Firecat, thank you for taking the time to contact Andreas Marx and posting the Q&A session. IMO it would have been nice to see the results of this test PRIOR to him giving out the 16+GB worth of samples. In other words some companies had 1+ week to add the samples to their signatures before the test started.

[Firecat]

-{ Quote: "Let's hope these emails are not pulled :P" }-

I do have written permission from Mr.Marx to publish those comments, so I think the comments won't be pulled. :)

@EliteKiller: Mr.Marx already said that the 16GB worth of samples was already uploaded on their server at May 6 and all AV companies were informed of it shortly thereafter. The workshop was the "second chance" and the email sent on May 23 was a "last chance" email to inform vendors that the samples will be deleted soon, so its best to get them now. :)

[EliteKiller]

Firecat, I definitely understand the part about offering the samples before the review, which was the basis of my last post. All I was implying is that it would have been nice to see the review before the AV companies had a chance to add the 16GB worth of samples to their databases. IMHO it's no different than a professor giving the class all of the answers to the test a week or two in advance. This explains why so many had extremely high detection rates. Then again this also shows us which companies are slow to add samples. Does AV-Comparatives release their 500K samples to vendors and perform the review afterwards?

[IBK]

-{ Quote: "Does AV-Comparatives release their 500K samples to vendors and perform the review afterwards?" }-
no. .

[EliteKiller]

IBK, thanks for the swift reply. ;D I edited my previous post to include additional information right before you replied. What are your thoughts on AV-Test releasing 16GB worth samples in advance and performing the review afterwards?

[IBK]

I am not going to comment it. I do not agree with some other things, but different peoples have different opinions - it does not necessarly mean that one of the two peoples opinions are wrong.

[colt45allstar]

-{ Quote: "Sure. As we all know Canadian beer is superior to American brands so I know you'll enjoy it.;) I've done the testing so I can vouch for the results.;D" }-

Hardly most Canadian beer is overrated unless it's Unibroue.. their beers are awesome! American beer is more than Anheuiser Busch, Miller and Coors as well you know. American microbrews are near the best.. only topped by Belgian Ales ;D

Now then interesting results indeed. Avira has become quite impressive. It seems to be leading the way in detection with most tests and I applaud them for that.

I'm fine with Kaspersky and don't see myself ever switching again.. but if for whatever reason my mind was ever changed, Antivir would no doubt be the one I would try.

[Firecat]

For that matter, there is nothing to prove that the 16GB collections contained only the very same samples that were used in the test. It is important to note that Marx said the collections contained "more important samples, which are already available from other sources to the AV companies". :)

[EliteKiller]

-{ Quote: "For that matter, there is nothing to prove that the 16GB collections contained only the very same samples that were used in the test. It is important to note that Marx said the collections contained "more important samples". :)" }-
True, but there is no denying that 16GB is a enormous amount of samples, and the companies that were able to implement the "new" samples into their databases prior to the testing had an advantage. Why not release the samples after the tests are completed and finalized?

[Abeltje]

Those companies that were able to add them prior to the test might simply be better in adding new signatures. What's wrong with that? It's not that some companies got the samples and others didn't, everyone had equal chances. By the way, it should not be the task of antivirus product testers to supply samples to the vendors. I would expect that the vendors are able to add threats themselves. You can see who's best in doing so by looking at the results. And then arguing about whether some samples are maybe less important and you will hardly encounter them .. even if there is only a slight chance of getting infected by whatever action I take, I want to be protected by my AV, that's what I have it for. I find it alarming that NOD32 which I used for years now couldn't detect some 70000 samples of threats that someone was able to spot around on the internet during the last 12 month. I like Nod32 for many things, but I'm using Avira for the moment and hope that ESET will react to the recent tests (AV-comparative test wasn't that encouraging either).

[BlueZannetti]

-{ Quote: "The statement makes good sense to me, and I agree that an AV which does well on detecting newer threats provides an effective protection for "today's malware". :)" }-Yes, this does make sense, although I was initially surprised at the relative drop. Actually you can get some insight into the trend quantitatively by looking at differential changes in detection, which does provide a result consistent with the www.AV-Test.org results for NOD32 at least.

The basic estimation process is straightforward. Successive tests have an incremental increase in samples in each category. Let's say that Trojans increased by 100,000 samples, from 300,000 on a previous test to a current value of 400,000 total samples. Assume the current detection rate is 96%, while the previous detection rate was 98.5%. That means 16,000 samples were missed in the current test while 4,500 samples were missed in the previous test. In that time 100,000 samples were added to the testbed. You can't calculate the detection rate for the added samples with the information provided, but you can place upper and lower bounds on it. For the lower bound assumed all missed samples are in the new members of the testbed. This result is simply (100,000 - 16,000)/100,000 = 84%. For the upper bound, assume that none of the missed detections were fixed and remove them from the number of missed detections. This provides an upper bound of (100,000 - (16,000 - 4,500))/100,000 = 88.5% for the example calculation.

I won't provide the complete analysis for NOD32, but the bounding estimates are 90.7% and 94.2% for lower and upper bounds respectively for the entire testbed when this procedure is followed. I view as entirely consistent with the www.AV-Test.org result (88.32%) given the crude level of approximation that I've used. Note that this represents a 6 month window for my estimates.

Blue

[Firecat]

I sent off another e-mail to Andreas Marx about the doubts posed by some people regarding availability of the 16GBs worth of samples before the actual test itself. Mr.Marx was again very kind to provide these comments:

-{ Quote: "As I said previously: "Please note that all
collected stuff is made available to the AV companies on a daily
basis as download."

This means, every AV company can receive what we are seeing to be
spreading once a day, as download. These daily-released packages are
usually 100 to 250 MB large (packed and encrypted), containing about
1,000 to 3,500 unique new potential malware samples. After
verification (is it malware? what kind of malware?), these files are
put in our test collection for a period of usually around 12 months.
Of course, we have several different sources of samples, good
honeypot systems are just one part of the story.

Therefore, the 16 GB of files (btw, some GB are ad-/spyware samples
which are not even tested against!) would not have a meaningful
influence of the test results at all -- these were missed samples
from a different (previously performed) test and many of them were
available at the daily collections earlier.

As a tester, you should not just simply say "your product is not
good", but you should give every vendor a chance to react after a
test and send them the missed samples. A test should not only compare
and/or rank products, but one of the main goals should be to improve
the product quality." }-

And with that, I hope all doubts about this latest test are cleared. I thank Mr.Marx again for taking the time to clarify the various doubts presented by me as well as the forum posters here. His words make perfect sense, and to me, AV-test remains as reliable as ever. :)

[Thankful]

-{ Quote: "I sent off another e-mail to Andreas Marx about the doubts posed by some people regarding availability of the 16GBs worth of samples before the actual test itself. Mr.Marx was again very kind to provide these comments:



And with that, I hope all doubts about this latest test are cleared. I thank Mr.Marx again for taking the time to clarify the various doubts presented by me as well as the forum posters here. His words make perfect sense, and to me, AV-test remains as reliable as ever. :)" }-
Hardly. As I stated in an earlier post, the discrepancies for all AV vendors between the April and May tests by the same testing company (AV-Test) are just too great.

[MalwareDie]

-{ Quote: "Hardly. As I stated in an earlier post, the discrepancies for all AV vendors between the April and May tests by the same testing company (AV-Test) are just too great." }-

Agreed

[MalwareDie]

-{ Quote: "The industry trusts it and you don't? ???
Wow, so suddenly Andreas Marx has become some German VX kiddie in the eyes of people....
" }-


So does that mean that back in 2004 i should trust in Norton? It was far too bloated. Just because it is the most trusted in the industry doesn't mean that it should have my trust.

[C.S.J]

just because it used to be bloated, doesnt mean it offers less in protection.
so, yes you should trust it.

[Blackcat]

-{ Quote: " As a tester, you should not just simply say "your product is not good", but you should give every vendor a chance to react after a test and send them the missed samples. " }-
A pity that some other testing sites have set conditions for vendors to reach before they receive the missed samples. The "low-scoring" vendors therefore never see them again; Ad infinitum.
-{ Quote: " A test should not only compare and/or rankproducts, but one of the main goals should be to improve the product quality." }-
Completely agree :thumb:

[Thankful]

If we look at another specific example, for AV AVG, regarding Trojan detection, the detection for April was 91%, May 96%. This is a five percent difference. Based on a Trojan sample size of 407,487 for May, this represents an increase in Trojan detection for May of 20,374. This hardly seems possible.

[Zombini]

-{ Quote: "Do you believe Symantec gets more than Kaspersky and NOD32 ?" }-

Yes. Why not.

[solcroft]

-{ Quote: "A pity that some other testing sites have set conditions for vendors to reach before they receive the missed samples. The "low-scoring" vendors therefore never see them again; Ad infinitum." }-
I'll have to voice a differing opinion to that. Testers are testers, not servants of security vendors. As far as I'm concerned they've contributed enough by volunteering time, effort and money by bringing professionally-conducted tests to the public.

With my extremely limited knowledge of how the industry works, I do expect any vendor worth its salt to be able to collect their own malware samples without having to rely on testers to do the work for them.

[extratime]

Firecat thanks for asking Andreas Marx those questions, and a big thanks to Andreas for his candid and detailed responses.

It only bolsters their reputation as a testing body. It also makes me realize that there is a lot more malware out there than I could have imagined.

[flyrfan111]

-{ Quote: "I'll have to voice a differing opinion to that. Testers are testers, not servants of security vendors. As far as I'm concerned they've contributed enough by volunteering time, effort and money by bringing professionally-conducted tests to the public.

With my extremely limited knowledge of how the industry works, I do expect any vendor worth its salt to be able to collect their own malware samples without having to rely on testers to do the work for them." }-

Not always true. When you take a test in school isn't it better for you to know what you got wrong? More is learned from knowing what you got wrong than knowing what you got correct. It is only fair to inform vendors of what their product got wrong(or more accurately missed).

[solcroft]

-{ Quote: "Not always true. When you take a test in school isn't it better for you to know what you got wrong? More is learned from knowing what you got wrong than knowing what you got correct. It is only fair to inform vendors of what their product got wrong(or more accurately missed)." }-
The answer to your question is: undoubtedly. But the testers are not teachers, and neither are vendors little kids.

It would be in the vendor's best interests if they got the samples they missed; obviously they want all the malware they can get. However, if a vendor consistently displays a lack of malware-collecting resources to improve their product unless they are spoon-fed by testers, it says a lot about that vendor; at least, it does for me. Not to mention that testers aren't duty-bound to aid the less-capable vendors. It'd be nice of them, certainly; but I don't think it's a responsibility.

[C.S.J]

whether testers want to 'spoon-feed' the vendors, it doesnt matter.

i personally would rather have my av have the samples, than not.

[besafe]

The point of the test is to determine which vendors produce a quality product. If a certain vendor is poor at finding and detecting the most recent malware, that is good for the consumer to know.

If the testing organization is feeding the missed samples to the security companies, this could artifically inflate the test results and potentially mask a security vendors faults.

The security company should be doing the research and finding the malware samples, not the testing organization. I want to know which vendors are good at finding new malware, not which vendors are good at updating their signatures based on the test organizations samples.

[Firecat]

-{ Quote: "If we look at another specific example, for AV AVG, regarding Trojan detection, the detection for April was 91%, May 96%. This is a five percent difference. Based on a Trojan sample size of 407,487 for May, this represents an increase in Trojan detection for May of 20,374. This hardly seems possible." }-
Regarding AVG specifically, I have stated in another post that AVG Professional was used in the PC World Review while AVG Anti-Malware (which uses Ewido+AVG engines) was used this time. This would explain the difference in trojan detection rate. :)

Regarding sending samples to AV vendors, Andreas' view on the matter is highlighted in post #89. However, he also says that every vendor should be given a chance to improve regardless of detection rates by giving them the missed samples. Essentially it means that tester-provided samples should not be the only source for AV companies, but rather AV companies should gather samples by themselves. Only a combination of the two creates a good AV product (and improves existing ones). Though Andreas' sample set is HUGE, I'm sure no one on Earth has every piece of malware released on the Earth, therefore it is as much of the AV vendors' duty to collect their own samples as it is of the testers to allow vendors to verify test results as well as add undetected/missed samples to improve their overall detection. Because if an AV vendor relies solely on tester submitted samples, then there is every chance that by the time the tester submitted samples are added to the database, those samples are already old. And in paying attention to only these samples, the company has already missed current malware roaming around the Internet and hence has erred in terms of providing protection to its users from the latest malware.

[Firecat]

-{ Quote: "Regarding AVG specifically, I have stated in another post that AVG Professional was used in the PC World Review while AVG Anti-Malware (which uses Ewido+AVG engines) was used this time. This would explain the difference in trojan detection rate. :)

Regarding sending samples to AV vendors, Andreas' view on the matter is highlighted in post #89. However, he also says that every vendor should be given a chance to improve regardless of detection rates by giving them the missed samples. Essentially it means that tester-provided samples should not be the only source for AV companies, but rather AV companies should gather samples by themselves. Only a combination of the two creates a good AV product (and improves existing ones). Though Andreas' sample set is HUGE, I'm sure no one on Earth has every piece of malware released on the Earth, therefore it is as much of the AV vendors' duty to collect their own samples as it is of the testers to allow vendors to verify test results as well as add undetected/missed samples to improve their overall detection. Because if an AV vendor relies solely on tester submitted samples, then there is every chance that by the time the tester submitted samples are added to the database, those samples are already old. And in paying attention to only these samples, the company has already missed current malware roaming around the Internet and hence has erred in terms of providing protection to its users from the latest malware." }-
BTW regarding some other discrepancies people may have noticed between the PC World tests in April and the current test performed by AV-test.org, it should be noted that the PC World article "Top Antivirus Performers" was published on April 23, 2007, but the actual testing was finished quite a bit earlier than that date. The editors need time to write the actual review after looking at the lab results and of course it takes a couple of weeks to test out all the products to determine their opinion of the product on factors other than the detection rate (since detection rate alone is not always a factor for a potential customer).

And on top of this, one needs to know that it takes time to print the PC World magazine, so the truth is that there was a lot more time between the PC World results and the latest test from AV-test than just one month. :)

[Thankful]

-{ Quote: "BTW regarding some other discrepancies people may have noticed between the PC World tests in April and the current test performed by AV-test.org, it should be noted that the PC World article "Top Antivirus Performers" was published on April 23, 2007, but the actual testing was finished quite a bit earlier than that date. The editors need time to write the actual review after looking at the lab results and of course it takes a couple of weeks to test out all the products to determine their opinion of the product on factors other than the detection rate (since detection rate alone is not always a factor for a potential customer).

And on top of this, one needs to know that it takes time to print the PC World magazine, so the truth is that there was a lot more time between the PC World results and the latest test from AV-test than just one month. :)" }-
And the eight percent increase in Trojan detection rate for Avast! (88 to 95.94), how is that explained? This would be an increase of 32,354 Trojans in one month!! They must be pretty busy at Avast!

[Firecat]

-{ Quote: "And the eight percent increase in Trojan detection rate for Avast! (88 to 95.94), how is that explained? This would be an increase of 32,354 Trojans in one month!! They must be pretty busy at Avast!" }-
Again, the time difference was quite a bit longer than one month, its not like Avast! wouldn't improve in that period....

[solcroft]

-{ Quote: "And the eight percent increase in Trojan detection rate for Avast! (88 to 95.94), how is that explained? This would be an increase of 32,354 Trojans in one month!! They must be pretty busy at Avast!" }-
Obviously you are not doing a very good job of reading what people say before chiming in with your bit.

[Thankful]

-{ Quote: "Obviously you are not doing a very good job of reading what people say before chiming in with your bit." }-
Thanks. I've noticed your posts are always positive and supportive of others.

[Thankful]

-{ Quote: "Again, the time difference was quite a bit longer than one month, its not like Avast! wouldn't improve in that period...." }-
Even if the PCworld test was performed six months ago, that would mean Avast! would average an increase in Trojan detection of 5400 per month for six months.
Is this credible?

[IBK]

based on the nature of the used samples, its quite possible that they all get detected with just some few definitions..

[MalwareDie]

Generics?

[IBK]

you could call it generic

[Thankful]

-{ Quote: "based on the nature of the used samples, its quite possible that they all get detected with just some few definitions.." }-
Thanks.

[walking paradox]

The fact that the samples were available to the vendors before the test was conducted in principle undermines the validity of the test. Whether this actually skewed the results, we can't be sure. As was mentioned it probably wasn't enough time to add all the samples, but it only takes some of the vendors adding some of the samples to their signatures to skew the results. Granted all of them supposedly had equal access to the samples, but unless all the vendors added relatively the same amount of the sample malware to their database, then the results would be skewed. Am I missing something here?

[Firecat]

-{ Quote: "Even if the PCworld test was performed six months ago, that would mean Avast! would average an increase in Trojan detection of 5400 per month for six months.
Is this credible?" }-
Regarding discussions on whether any AV company can add 5000+ malware within a month, Mr.Marx has once again very kindly provided some comments :):

-{ Quote: "I saw that there are now discussions going on if a company can add
5,000 samples a month or so -- well, if they have developed some new,
good heuristics, this might work pretty well. Invest a lot of time in
generic signatures and you can easily catch 10,000+ malware files, if
done properly, so you will perform a lot better in test.

However, if your heuristic is bad, you might need to remove it, for
example, if the scan speed is too low or if too many false positives
are present. Then, the detection score might drop with one update to
the next...

This means, it's easily possible that products might score a lot
better or worse, within a short period of time. And nobody says that
what was detected in past might still be detectable in future. Some
more results about the reliability of AV scanners will be published
in near future, this includes facts like the ones mentioned above." }-

Lots of interesting comments from Mr.Marx above. I must say he has been very informative through all this. :)

It will be interesting to see the reliability tests which should be released in the near future. Also, from these statements it looks like Mr.Clementi is correct - scanners with good generics are probably detecting lots of malware. :)

[IBK]

of course its correct, but it is not exactly what i said/meant. but i will not go deeper into this.

[The Hammer]

-{ Quote: "Hardly most Canadian beer is overrated unless it's Unibroue.. their beers are awesome! American beer is more than Anheuiser Busch, Miller and Coors as well you know. American microbrews are near the best.. only topped by Belgian Ales ;D

" }-I've done the testing and stand by my results.;) :P ;D

[Firecat]

-{ Quote: "of course its correct, but it is not exactly what i said/meant. but i will not go deeper into this." }-
I don't know IBK, you have been quite vague throughout this thread. Every time I see your post I feel like you are thinking something (i.e. you have your reservations/doubts). :)
Either way, if you're thinking what I think you're thinking, then I'll have to refer you to your own post earlier:

-{ Quote: "....different peoples have different opinions - it does not necessarly mean that one of the two peoples opinions are wrong." }-

No offense intended towards you, so please don't take it offensively. And with that its best to drop the issue. :)

To me, AV-test remains one of the most reliable testing organizations out there (if not THE most reliable), and Andreas Marx's continued clarification of various doubts relating to the test only makes me trust AV-test all the more. :)

[IBK]

i do not criticize the test of andreas marx. after all, the results are quite similar to those of another tester, even if i would have expected more discrepancies based on the used test-set.
do not know what you think i am thinking, but as what i think has by far not even be mentioned in the thread, i am quite sure you do not know what i think :).
like i always stated, av-test is trustworthy and i am not doubting the results at all.

[Thankful]

-{ Quote: "Regarding discussions on whether any AV company can add 5000+ malware within a month, Mr.Marx has once again very kindly provided some comments :):



Lots of interesting comments from Mr.Marx above. I must say he has been very informative through all this. :)

It will be interesting to see the reliability tests which should be released in the near future. Also, from these statements it looks like Mr.Clementi is correct - scanners with good generics are probably detecting lots of malware. :)" }-
I appreciate the time you have spent on this thread.
I am also looking at the results from this website (May 10) which are more in line with those at AV-Comparatives (except for Symantec). http://www.virus.gr/fullxml/default.asp?id=110&mnu=110

[Firecat]

-{ Quote: "i do not criticize the test of andreas marx. after all, the results are quite similar to those of another tester, even if i would have expected more discrepancies based on the used test-set.
do not know what you think i am thinking, but as what i think has by far not even be mentioned in the thread, i am quite sure you do not know what i think :).
like i always stated, av-test is trustworthy and i am not doubting the results at all." }-
;D

OK then. Considering the way you mentioned generics in one of your previous posts, something came to my head about what you might be possibly thinking. Most likely I am wrong in what I was thinking. Again, sorry if I directly attacked you. :)

But do explain this to me:

-{ Quote: "the results are quite similar to those of another tester" }-

Isn't the only other really trustworthy test out there AV-comparatives? So, wouldn't this "another tester" be you (i.e. Andreas Clementi), or is there a third one newly added into the mix? ???

[IBK]

not at all. that results are totally different.
sorry to say, but virus.gr is completly unreliable source.

[IBK]

-{ Quote: "
Isn't the only other really trustworthy test out there AV-comparatives? So, wouldn't this "another tester" be you (i.e. Andreas Clementi), or is there a third one newly added into the mix? ???" }-
yes, i just do not wanted to promote myself :P.

[Firecat]

-{ Quote: "I appreciate the time you have spent on this thread.
I am also looking at the results from this website (May 10) which are more in line with those at AV-Comparatives (except for Symantec). http://www.virus.gr/fullxml/default.asp?id=110&mnu=110" }-
No, virus.gr has its own set of discrepancies. VirusP has put a respectable effort to clean up his sample set since the last virus.gr test, but I'm pretty sure there are still quite a bit of corrupted/harmless files still in his test.

In any case, the virus.gr test can be "interesting" for some users, but right now one cannot call it reliable, not just yet. :)

[Firefighter]

-{ Quote: "Another crappy test !
Why ? Do you believe Symantec gets more than Kaspersky and NOD32 ?
Do you believe Rasing AV is better than NOD , Bit Defender ,Panda , the other competition ?

I don't ! Real world results show completely different numbers !


I did the same" }-Actually, you (all) are talking about painkillers to a disease, that actually ever exists, if you just pick the right way to go! ;) Even without CLAMAV you can live it, if you just want as I do. :D

Best regards,
Firefighter!

[trjam]

-{ Quote: "Another crappy test !
Why ? Do you believe Symantec gets more than Kaspersky and NOD32 ?
Do you believe Rasing AV is better than NOD , Bit Defender ,Panda , the other competition ?

I don't ! Real world results show completely different numbers !


I did the same" }-
Actually I do believe it. I think it is you who cant fathom your precious pink diamond not being the rave of the AV world. Well, you both better wake up or it will be the zirconium mine for you. Why is it if your AV isnt one of the best, all tests results are crap. Hmmm. I look at CSJ standing by his product and reviews may be mixed, but you know, if it wasnt Avira, it would be Dr. Web. The time has come for Eset to come off their self proclaimed mountain and deal with directly working with, and listening to, their customers in order to make a better product. They can do this, but it will take a radical change of perception. Bah-Hum bug.

[walking paradox]

I am at a loss as to why the point I and others mentioned isn't being addressed more fully. Unless I'm missing something, giving the vendors the test samples before the actual test takes place renders the overall test methodologically flawed by introducing the potential of skewed results. I can't help but wonder why he didn't give them the sample malware after the test had been completed. That way he can assure the accuracy and reliability of the test, at least in this respect, and still contribute to the efforts of AV vendors.

[trjam]

Giving vendors test samples is like giving a bank robber the keys to the bank. It is assine. Show me someone who tests without giving the answers away and I agree, you may have some accuracy.

[MalwareDie]

That is an insult to IBK. he does not do that knid of thing.

[trjam]

I didnt say he did, but anyone whom does to me, the test is worthless. Wouldnt you rather see real world testing.

[MalwareDie]

O Okay. I misinterpreted your post.

[BlueZannetti]

-{ Quote: "I am at a loss as to why the point I and others mentioned isn't being addressed more fully. Unless I'm missing something, giving the vendors the test samples before the actual test takes place renders the overall test methodologically flawed by introducing the potential of skewed results." }-I guess the only comment that I'd make is that the result is largely consistent with the other major test that is frequently run (www.av-comparatives.org). This suggests that most of the final results are not skewed. As I mentioned above, you can place some bounding limits on the expected results between the tests by looking a sequential on-demand results from www.av-comparatives.org. A complete calculation for all common products using the av-comparatives August 2006 and Feb 2007 results is given below. A caveat however - don't overinterpret the results given below, they are crude estimates.

The bounding estimation is only of predictive use when it is "relatively" small. From a calculation perspective, a large range between the Estimated Min and Estimated Max values indicate instability in the detection rate over time. That could be due to a program undergoing either significant improvement or a major drop in performance with all these calculations dominated by what has transpired for the Trojans category. A quick inspection of the raw results at www.av-comparatives.org suggests that, except for McAfee and Norman, the large range values noted for Avast!/AVG/BitDefender/Dr. Web/F-Prot are due to improvements in performance. At the end of the day it really doesn't matter to a customer how this improvement occurred - but the estimates shown below were obtained prior to and independent of the www.AV-Test.org evaluation.

Finally, let's keep a grip on perspective. Unless one is a risk prone user, all these products possess sufficient performance.

Blue

[Tweakie]

-{ Quote: " A complete calculation for all common products using the av-comparatives August 2006 and Feb 2007 results is given below. A caveat however - don't overinterpret the results given below, they are crude estimates." }-

I'm sorry but I did not understand how you computed these estimates. I don't understand their meaning, either...

[BlueZannetti]

-{ Quote: "I'm sorry but I did not understand how you computed these estimates. I don't understand their meaning, either..." }-The calculations were described in an earlier post in this thread, here (http://www.wilderssecurity.com/showpost.php?p=1011361&postcount=101). It's an attempt to objectively answer the question of whether, aside from testbed sampling timeframe, the results of www.av-comparatives.org and the latest www.AV-Test.org AV comparison are the same.

Within reasonable and objective limits, I feel that the answer is yes. To obtain a more quantitative answer, a fairly extensive effort would be required and it's not really worth it. I realize that lot's of people read significance into razor thin differences in detection rates, and the numbers I estimated show a much larger uncertainty, but that's what they are.

Blue

[MalwareDie]

How does Avira, Norman, AVK, kaspersky, Symantec and F-Secure get an estimated max lower than both the av-test and av-comparative score?

[beethoven]

-{ Quote: "Unless I'm missing something, giving the vendors the test samples before the actual test takes place renders the overall test methodologically flawed by introducing the potential of skewed results." }-

As was explained several times already, the samples are being provided on a daily basis and are available irrespective of a test being conducted or not. Obviously those companies that avail themselves of the samples have an advantage but that does not skew the test. It just shows that some companies put more effort into collecting and integrating samples than others.

-{ Quote: "Every time I see your post I feel like you are thinking something (i.e. you have your reservations/doubts).
Either way, if you're thinking what I think you're thinking," }-

Well, I for one am not sure what Firecat is thinking IBK is thinking but I am thinking for those of us less telephathically developed, a bit more clarity would be a plus :o

[The Hammer]

-{ Quote: "A pity that some other testing sites have set conditions for vendors to reach before they receive the missed samples. The "low-scoring" vendors therefore never see them again; Ad infinitum.

" }-I'm sure the decision wasn't arbitrary and was carefully considered. Although I'd like to know from IBK what they are if he's willing to say that is.

[BlueZannetti]

-{ Quote: "How does Avira, Norman, AVK, kaspersky, Symantec and F-Secure get an estimated max lower than both the av-test and av-comparative score?" }-Like I said, the calculation is very crude and only a rough estimate. The known numbers from www.av-comparatives.org August 2006 and February 2007 results are: Total samples in each category (Trojans, Backdoors, etc.). This allows me to calculate a difference in each category. I assume this represents all new samples and that there has been no culling of bad samples from the testbed. If there has been trimming of bad samples from the testbed, this is obviously a low estimate.
Samples missed are also tallied in each test for each category. The worst case scenario is that all misses from August 2006 are now handled and that the misses now shown involve only the new samples introduced for Feb. 2007. Keep in mind that the new samples estimate is the basis for the calculation. The best case scenario (in terms of the new samples) is that none of the August 2006 misses were corrected which means that the Feb 2007 misses were inflated by an amount equal to that seen in August 2006. Obviously, some of the August 2006 misses will have been addressed, I simply have no way of knowing how many - hence correcting for all of the Aug 2006 misses will yield a high estimate.
Like I said, do not overinterpret. The estimates pertain to a partial testbed. The entire range of the estimates may not include either test result since it tries to recast the www.av-comparatives.org results into a partial test with a testbed somewhat analogous to that used by www.AV-Test.org. Since it attempts to mimic "new samples only", I'd expect it to come in low relative to the two parent cases (Aug. 2006 and Feb 2007). With respect to it's relation to the www.AV-Test.org results, there is no guarantee with respect to testbed overlap. In terms of being "new samples only", there is correspondence. Note also, the estimate "testbed" is not small. It is calculated to be comprised of roughly 176,000, of which 150,000 samples are Trojans and Backdoors. So these two categories dominate the results

I performed this calculation since to my initial eye, the results for NOD32 and F-Prot seemed a bit out of place relative to the others. After calculating these estimates, I don't believe that's the case. They do seem to reflect a trait seen in the www.av-comparatives.org tests as well, particular since the Estimated Min and Max are reasonably close in this case. One can argue what this means in terms for performance, what I would say one cannot argue (myself included) is that the www.AV-Test.org results are somehow skewed against NOD32. They appear to represent a current snapshot in time of immediate performance. The Estimated Min and Max for F-Prot are sufficiently different that one really cannot draw any conclusions using those values in that single case. However, by inference from the remainder of the comparisons, it's reasonable to conclude that the overall test is in control and that the results reflect current reality for F-Prot as well. Again, the disparities between the two tests were fairly stark for those two cases, but do seem to reflect the different natures of the two tests - i.e. all malware vs. only "new" malware.

Blue

[MalwareDie]

thanks for the explanation

[De Hollander]

-{ Quote: "i do not criticize the test of andreas marx. after all, the results are quite similar to those of another tester, even if i would have expected more discrepancies based on the used test-set.
do not know what you think i am thinking, but as what i think has by far not even be mentioned in the thread, i am quite sure you do not know what i think :).
like i always stated, av-test is trustworthy and i am not doubting the results at all." }-

@IBK

Wood that be your test...?

[Pfipps]

AV-test.org needs a meaningful rating system like AV-Comparatives (no certification, Standard, Advanced, Advanced +) so consumers don't get confused by the results.

As far as I am concerned, any anti-virus with the Advanced or Advanced + ratings is fine. All the other reasons are on a personal basis. They even point out that the Standard rating is sufficient if it has a ICSA certification - but I wouldn't take that risk.

[Pfipps]

The Sophos rating is scary. My college depends on Sophos! Mind you, the word on the street is that my college's IT department sucks :P .

[MalwareDie]

I wouldnt trust Sophos it is actually quite poor and its detection rate is inflated here.

[Pfipps]

-{ Quote: "I wouldnt trust Sophos it is actually quite poor and its detection rate is inflated here." }-

Why is it inflated?

[MalwareDie]

because Sophos' detection rate is not above 80% and mroe like 75% and that is not adequate.

[Firecat]

-{ Quote: "because Sophos' detection rate is not above 80% and mroe like 75% and that is not adequate." }-
Again you are exhibiting your distrust of AV-test.org....The fact is AV-test shows Sophos as getting around 80% while another test showed it as getting around 75%. But in either case Sophos is not so good, its better to use something else.

[Pfipps]

"Products were set with their most aggressive detection options, such as using all heuristics and testing inside archives."

I overlooked that one. It is important because I run NOD32 at max settings and I can't tell its running. With programs like Norton and Kaspersky, I can't stand the max settings. It's a huge issue, since it means that vendors like Panda, Alwil, and AVG may have better real life protection since they can be maxed out without much pc slowdown. Well, not necessarily better; but I know the top vendors would see their detection rates go down when on the usable settings (Am I wrong on this one?). The only exception I know of is Antivir, which is apparently quite fast anyway.

[C.S.J]

AV-TEST dont know what they are talking about,

drweb easily should score 99%, or probably even 100%

im never reading into these results again, they are such rubbish, do they even know what malware is?

::)

lol :D

[solcroft]

C.S.J,

It took me a few seconds before I realized the sarcasm.

;D

[Pfipps]

-{ Quote: "C.S.J,

It took me a few seconds before I realized the sarcasm.

;D" }-

Who was it directed to? Did my post sound like a fanboy message? Or am I too self-important? ;)

[C.S.J]

sarcasm, whats that? :shifty:

[masqueofhastur]

Is the WebWasher listed CyberGuard WebWasher?

[pilotart]

Although I have had this WebWasher anti-PopUp/Spyware Filter (http://www.freedownloadscenter.com/Network_and_Internet/Internet_Client_Suites/WebWasher.html) on my Win98 System for nearly ten years:thumb: (was one of the first such products), the one referred to in this test, is the Web Gateway Corporate Network AntiVirus/Security Protection (http://www.securecomputing.com/index.cfm?skey=22&lang=en) that includes8) the AntiVir AV Engine.