AV-Comparatives - Real-World Protection Test - April 2013

It is, if that same thing is also off for 99% of the users because it is set to off by default. Imagine users buying an AV product in good faith because it regularly scores well in testing, but on their PC's it has a lower detection ratio and less protection because they are on default settings which are lower than in testing and they are probably not even aware of having a higher infection risk.
If a tested AV vendor thinks it's not fair, they can consider changing the default settings, or even just tell AV-C to test their product with different settings(though I'm not a fan of that, unless it's tested with default settings as well.)

Please keep it respectful, you wouldn't like it either if you were accused of being paid by Avast to defend them and attack AV test groups.

It's a discussion forum, so when someone states their PoV, others can state theirs and give arguments and then in the end, hopefully new insight can be gained.

 

Amen.

Symantec was tested without the chance of choosing the settings like any other vendor. Notably, if Smart Definitions had been deactivated Symantec would had passed that (prehistoric) file detection test without a problem.

A sad AV-C vendetta.

 

Exactly, AV-C mentioned in the pdf that a Magazine covered the cost for including Symantec in the Performance test, so others do indeed want to see them tested.

Some how I think it's good that they are not included in the RWPT even if someone else would pay for it, as all the other vendors are take money out of their own pocket to get tested, while Symantec could be included for free only because someone else is paying. Though I don't mind that they are included in the Performance test.

 

I defend nothing but sometimes arguments and decisions are ok, even if they come from the bad testers. And some things there are quite fair - even if you don't recognize it. So let's concentrate on arguments

Core points:
(1) They test all in default settings. If some vendors use "weaker" default settings it is not the testers problem.
(2) We talk about stupid home user products. And the most of them rely on default settings. And the test ist called "real world testing" So really real world even in the settings point ;-)
(3) We don't talk about one product as you argue. Look at all the testet products and see how many don't have PUPs enabled in default settings.
(4) Who says that there are PUPs testet and not malware?
(5) If a program detects real malware only with PUPs detection enabled, it's bad designed.

 

Testing with default settings are grossly unfair to the products involved.
Security programs simply must be configured to the user,s own specifications and personal taste.

Releasing products with a fixed set of default settings may not be suitable for all the different computer specs and configurations out there.

 

I disagree since most people including me don't mess with the default settings.

 

Yes for on demand tests where Symantec will not do well, AV-C has no financial problems. Anyway I'm done with this discussion, just an observation.

 

Yep you may be right. I'm quite new to Avast, was using Comodo before. Avast is so quiet in terms of alerts, that's why I was asking.

 

Well IMO not only Symantec struggles in that test, but other vendors have no problem to be tested in that particular test even if they're not best at it since they want to participate in the RWPT and the File detection test is one of several tests included in the " test suite" afaik.

I don't want to know how you found that out. But whether AV-C got money or not is totally irrelevant

 

When it comes to av vendors and testing organisations,money is very much relevant.

 

I don't bite on that one, sorry

 

not a discussion but simply a fact to correct your observation.AV-C clearly mentioned in their report that "a magazine paid for symantec in on-demand/file detection test"

 

They are being paid for by magazines (FDT and performance tests). If and when someone funds AV-C for the dynamic testing (and chooses for results to be non-exclusive), they will appear. Otherwise; you get what you see, unless Symantec wants to show the world it's actually that good at even the real-world tests....

 

I dont see the point here...what if the program in question is mix of adware/rogue then every AV vendor will classify it differently,not poor designing its just how every AV vendor looks at it

The arguments against these small tests groups like Dennis,MRG etc will continue forever but nobody will point their finger on a well known big test group like AV-C,dont think I dont understand the logic

 

Than it's not a PUP.
Beside all this. I see what you want to point out, but nevertheless vendors choose their default settings and classification and many users use this default settings. So at least in this point the real-world protection test is real world and it's not the testers guilty if some products don't detect some things in default settings. And not only avast has disabled PUPs detection in default settings.

I think, even here on wilders, are enough (and even valid) arguments against some issues and design flaws also for the big tests like AV-C. But another point: If there are real arguments @IBK for example has always open ears and is open for discussions and clarifications.

But of course - even testing organisations are part of the big business, but thats ok. They and their results are on the other side used for marketing and so for business - so give them a part of the cake, they work for it. Even AMTSO is mostly a big business cooperation ;-)

As for the other point: I don't think it's a big vs. small test group discussion. At least one of the small groups you named has a very bad history...so no wonder many don't trust them. (but that's another topic)

 

whatever you may say to show it isnt what I am telling I can say that it is..atleast If I compare the threads of AV-C and AV-T and MRG,Dennis,west coast labs etc I can see where fingers are raised and where not AV-C may have been critized but not in the way others are being critized,I have my eyes open

 

Surely, you're right with your observation. But what's the reason? People don't critize AV-C so much
a.) only because of sympathy, or
b.) because they are little bit more transparent and there are not so much strange issues and things to critize as for example for MRG?

I believe b.)

 

Just a little heads up: The statistic for April isn't final yet. We disputed some of the samples that were previously considered as being undetected. So at least our results will change in the next few days.

 

Thanks for the heads up! Will look forward to the new result score!

 

Fabian if you don't mind my asking, why aren't you registerd on MalwareTips.com anymore?