AV-Comparatives - File Detection Test - March 2013

A response to a PC Magazine article, “Microsoft Outperforms Symantec in Antivirus Test"

Beyond the Headlines: Don’t be fooled by misleading security tests

 

I LOL'd this article. I did not even read it. I mean, a fact is a fact. a fact is stronger than one thousand words. Plain and simple. End of discussion. "Misleading"? In what means? Because AV-C wouldn't let Norton cherry-pick the specific test they like? Come on!
Now I admit symantec is still usable at the moment and likely is still better than microsoft in general; however, I feel they are going down-hill fairly fast. Look at the only comments below that article:

Comments
by huwyngr Norton Fighter on ‎04-15-2013 03:32 PM
Options

So AV-Test is not the same as AV-Comparatives ?



Confusing ....

 

An interesting article that people should read IMO:

All Mouth, No Trouser

The article is full of interesting information

 

Ok, there has been a 10 page full of back-n-forth arguments on symantec's performance and validity of on-demand tests and norton's sonar technology.

AV-C is a respectable org and it's test are generally seen favorably....but this test and subsequent arguments put forward by knowledgeable members seems to have created doubt in my mind about norton 1st time in many many years.

From my own experience, security is as much about the efficiency of the product as is about one's own habits.

Norton , imho, balances finely between modern protection modules and impact of the product on pc's performances.

I have rarely had any malware related problems in last few years either with norton or other products i have tried in between.

So, to cut straight to the point....does this test severely affects norton's image as AV-test did to microsoft here ? link

 

Following on the heels on Legendkiller's post just above, I just want to comment:

That the above posts are how I look at this recent "test" by AV-C.

I haven't used Norton since 2005, but I find what AV-C did by including them in a test that:

A) they asked not to be included in

B) did not pay (like the others did) to be included in, and

C) performed very poorly in (and it is well documented that this exact test is the reasoning that Symantec did not want to participate in the AV-C tests any longer)

seems like a cheap attempt to embarrass and hurt Norton's business, or perhaps even "force" (i.e. - extortion) Symantec to participate in "other" tests in the future in which they will perform very favorably in. All of which, as nine9s mentioned, could be a very valid reason for Symantec to engage in legal action against AV-C. Norton could perform extremely well in 9 of their 10 tests, but they just happened to choose the 1 test where Norton scored extremely poor to include in their tests?

Like I said, it's pretty well documented that the reason Norton pulled out of AV-C's tests was because of this particular test. Supposedly, Norton asked that it be performed in a certain way that would show how their product reacts to and treats the malware sample, but it involved going to lengths that AV-C didn't do for other antivirus companies and that they did not wish to go for Norton, so Norton removed themselves from the testing, which was their right. Neither company was wrong with how they handled it and for their stance in this instance. But this recent "we're going to include them even though they've asked not to be included, haven't paid to be included, and...oh yeah, just happened to perform fairly poorly compared to the others'....does seem unethical to me.

Shame, because I've always respected AV-C tests prior to this. And like I said, I don't use and haven't used Norton in years and don't intend to use it anytime soon in the future...but this wreaks of a low blow and a bit unethical on AV-C's part.

 

OK, let's say you are right about the a bit unethical part. But putting it simply - a test is a test and a result is a result, it is valid for all participants. It is true that Norton has other effective modules of protection but then most of the other vendors have other modules as well but this does not affect their signature detection, which is also important still. They have behaviour blockers, HIPS, you name it. Why the stress falls on Norton and why should they be treated differently to other AV vendors. If vendors claim to offer "all-round" protection, then they should endure all-round tests and not complain and whine how dishonestly they have been treated. Just my opinion. And if you are confident in your product - then there is no reason to fear "extortion".

 

I realize that your reaction is probably typical of most, and is just a sign of the times...

But we used to live in more honorable times where people, businesses, governments, etc. did what was RIGHT rather than what they "wanted" to. When somebody's word meant something. And when a company has specifically asked NOT to be included in a test, and they are included without their knowledge and without their consent anyway......well...that, my friend, is unethical. And that's really what this is all about.

How would you like it if somebody asked you if they could post an unflattering picture of you on the internet and you said that you didn't want them to but they did anyway? Especially when your competition is paying to have more flattering pictures of them posted over the internet? You'd feel a little violated and like they did you dirty, wouldn't you? People sue over stuff like that all the time. Simply because this is "Symantec/Norton...and they're big business....and big business can/should get screwed and that's OK" does NOT make it right. Let's say it affects their business negatively, and as a result, they lose business. Guess what that means? Yep...you got it. LAYOFFS. People losing jobs. Then how would AV-C look by doing something "unethical" like this that could affect people's livelihood?

 

Norton issued a blog about the test:
http://community.norton.com/t5/Nort...ooled-by-misleading-security/bc-p/944005#M708

 

Layoffs are pretty much relevant to the significance of Insight anyway....

They (AV-C) probably included the results because (hypothetically):

- They still have valid licenses
- Magazines would still request them and use such results
- If they are paid by the magazines; they aren't losing much money on adding Symantec to the testing

Most importantly, people want to see how Norton does in such a test. The opinion of a few (experts or otherwise) is not going to change that of the masses. In the case of a magazine that may want to utilize AV-Comparatives' expertise, of course they want to see how Norton is doing in an on-demand test.

 

Relevant reading for all regarding testing and marketing in general:

All Mouth, No Trouser

 

Very Relevant

 

I agree with you about the honourable times, people and business practices and while there is this side to the issue with Norton, there is another as well - business practices of advertising one thing and deliberately hiding others from your customers. And sometimes one thing evokes the same in return (not that I agree with that) but this is how things are. And about the comparison with my photo - I'm not a public person and Norton is one of the leaders on the market and customers want to know how it compares to other vendors. And implying that the test is in favour of the paying vendors I think is too much. I really don't think AV-Comparatives would risk their reputation over 1 test. I absolutely agree with you that publishing it despite Norton's wish is not ethical but I also stand by my comment about the test as such and the results.

 

Comments/Response on Symantec's blog post from PC Mag:

Symantec declared on-demand antivirus tests misleading

 

Well here we go with the complaints and excuses (results are just what they are) - and why misleading? Where have AV-Comparatives put more weight on this test than it actually is - a simple on-demand test? Who inflated it with so many heavy arguments, as if this is the ultimate AV test? If someone's interpretation of it is misleading that's another story. Why not just say well - we are not very good at that and that's it, we have other strengths. But to misplace the whole test just to prove your point and to act like a sulking child I think it's not in your favour.

 

I would think that all other anti-virus vendors would feel the same way as Norton and have very similar arguments.

Vlk backed Norton's position regarding on-demand scans.

Norton didn't request this test but the folks at AV-C decided to take it upon themselves to test Norton anyway. This only paints a partial picture which I am certain AV-C is aware of.

Since AV-C took the time to test File Detection, to be fair, they should also test Real World Protection.

Otherwise, this just looks like AV-C taking a cheap shot at Norton.

 

Agreed, it would serve AV-C well, to include NIS in the real-world test also (on their own cost) if they deem Norton too important not to test.

 

I think this one line from the PC Mag Article hits the nail (Symantec) on the head

""It is clearly stated, that only one feature of the product is tested," said Peter Stelzhammer, co-founder of AV-Comparatives. "If Symantec is thinking the file detection feature is worthless, why is it still included in the product?" Stelzhammer pointed out that file detection is needed for initial cleanup, and that PCs don't always have an Internet connection. Even so, "the test was run with full internet connection and Symantec cloud features have been granted access to their cloud."

http://securitywatch.pcmag.com/secu...declares-on-demand-antivirus-tests-misleading

I stopped using Symantec after watching two separate tests of NIS 2012 on Youtube where a downloaded rootkit completely disabled Symantec IS.

 

LOL
this is exactly what I suggested to those who thinks file detection is worthless: just disable your av's virus definition update and relying completely on behavior blocker.

 

Symantec and any other AV vendor do not think that file detection is worthless. They think that in addition to the file detection, there are also other measures to detect and stop malware and these measures are not includes in the file detection tests performed by AVC.

 

http://securitywatch.pcmag.com/secu...declares-on-demand-antivirus-tests-misleading

If Peter Stelzhammer from AVC really said this, I have no comments about this. They are really down and they really believe themselves. PCs nowadays always have Intenet connection, Symantec does include other protection features, and if a machine has already been infected, the on-demand scan by given product is not the best way to clean malware. Additionally, this is File-detection test, not protection or cleaning. If they claim that on-demand scanning is important for cleaning, then perform a cleaning test.

 

I'd say instead: Don’t be fooled by misleading "security" software.

 

Thanks for posting

 

I agree totally

 

That's the question that Symantec will never answer