AV-Comparatives - File Detection Test - March 2013

Annoying is putting it mildly. Most of the regular users of Norton (and I'm sure you would agree these are the vast majority) do not contemplate whether it is a false positive or not when a pop-up comes out from the Insight module saying WS Reputation 1. They see it as a real threat and labeling this to a small developer it also leaves a mark on his reputation, because he automatically becomes "shady", considering Norton's Reputation. Not to speak of the fact that WS reputation 1 detections are also automatically removed. So in my opinion a small developer is a hostage to Norton's way of handling unknown files and has to prove innocent - until then he is held guilty.

 

Thread needs to be renamed:

 

Not to beat a dead horse, but what was/is the main reason why Symantec don't want to test Norton in all the tests as all the other vendors that participate?

Is it that they don't want to score "average" on some tests and very good on others? If that is why, then I can't see why they make such a big deal of it, as no one of the other vendors shine in every singel test IMO.

 

I do not represent the Symantec official opinion, but I believe that SYMC don't want to participate in AVC because their products have changed the way of work. The way SYMC products protect now can hardly be measured by testing organizations' outdated methodologies. Symantec have invested a lot in proactive protection (real-time SONAR behavior analysis) and cloud based technologies (Insight). AV-Comparatives/AV-Test do not tests these or they do not do it in-depth, or they test with 100 samples, which is ridiculous when we take the amount of today's malware that appear every minute.

What they see is what I see, too, I believe - most users do not care about tests of organizations. Go on the street - stop 20 random people and ask them do you know what AV-Comparatives is .
However, those few who care (e.g. more savvy people - still users) blindly believe what these organizations say. And when AVC publishes tests with on-demand scans and the SYMC product is last, this definitely ruins their image. SYMC is big, SYMC is corporate. SYMC knows how the modern threats come to people's machines. On-demand scanning of files is not the way.

I still doubt why McAfee an Microsoft still care about these tests.. ? ..

Security is not just antivirus software, data protection is much more important than just antivirus. And Antivirus protection is ensured by a lot more measures than plain antivirus software and scanning. Malware comes this way (from Internet,emails, flash, exploits, etc) , not by putting some files on server and static scan them. Symantec knows this.

 

You know, Microsoft said something similar in response to their "poor" score in an AV-Test report.

 

The way Insight works, however, is tantamount to "guilty until proven innocent". This is different from behaviour blockers, which generally (unless set otherwise) have the opposite method of working.

I still say file detection is important because I have seen Norton miss when it comes to USB Flash Drives, HDDs, etc. I have also seen real malware infected files no longer giving the WS.Reputation.1 alert as more and more users downloaded and ran it (as there was no signature for it), meaning that an infected file had a decent reputation for a day or two before Symantec added a signature for it!

The best protection is one that uses both blacklisting and whitelisting. Norton's method creates unnecessary headaches and added costs (why should a developer of a small free app have to pay a somewhat significant amount to get his executables signed?)

There are so many articles on the WS.Reputation.1 detection:

http://www.mindworkshop.info/windows/the-norton-symantec-ws-reputation-1-false-positive/

When there are false positives, a company must reply immediately and not have to pay for "gold support". What Symantec has tried here with this whole Insight deal is to try and keep protection at a decent level while cutting as much staff as possible and still having users pay for "gold support". Why? To raise their margins - anyone who follows SYMC today knows they haven't been doing as well as they'd like (the recent reorganization is proof of that).

I don't have a problem with the protection offered - I do use PC Tools and Norton products, but I get the feeling that Symantec doesn't bother about the free and open web and small developers and wishes to raise it's bottom line by any means necessary. It just doesn't seem fair.

IMO, as long as Symantec continues to at least pass the tests, it is usable practically. If it doesn't make the baseline grade, it's probably better to use something else.

 

Well, it must be true, then, don't you think? Symantec and Microsoft are one of the biggest software vendors in the world - number 1 and number 2 (in security)

 

@er34 I somewhat agree, but then again. They may not like this one File-detection test, but this is one test out of several other tests in the "test suit" that AV-C got.

What I am saying is why can't Symantec look at it like "this is only one test that we tested badly in but there are 3 other tests by AV-C that we will test better in" so they can't stand to see one single bad test result from Norton (like the file detection test), even if the other tests by AV-C would show otherwise.

It's like they think that the RWPT is "outdated" too.
If I were them, I wouldn't care if the product tested badly in the file-detection test, if I still had the chance "show off" in the other tests. But unfortunately it seems they don't look at it that way wich is too bad IMO.

 

Well McAfee scored 99,3 in the RWPT test:https://www.wilderssecurity.com/showthread.php?t=344866

And so Symantec could possibly have done the same if they could stand to see Norton test less good in some tests like the File-detection test for example.

 

Hey.

Because if you think that this is only 1 test - most other people don't think this way. Marketing teams of competitive vendors take easily take this test and compare against Symantec. Massmedia takes the test results and reads them the way they want - do you see the title "Symantec got the worse results in reputable AV-Comparatives test". Enterprise rule number 1 is to avoid bad media reputation

Isn't it outdated, too ?

Most probably.

 

But that's only because they got tested once (could one say against their will) , if they were to apply for all the tests (wich they haven't done) and they would test very good in the RWPT the headlines would say otherwise I think, as they would probably not be worst in the test.

Depends on who you ask I guess

 

OK

 

Assuming you know the test very well (test setup, procedures etc), what exactly is outdated about it?

 

Just because Norton or anyone else misses a threat from a valid real world test like USB drive, HOW DOES THAT MAKE a static-scan a valid test ?

A million files in folder copied there file the product is off, is STILL NOT A VALID TEST.

Your argument is flawed, or am I missing something.

 

what do you mean?

 

He/she means that a million files in folder copied there while the product is off, is not a valid real-world scenario.

 

Let's say I know the procedure well , I know these testing organizations and I know malware and anti-malware well enough to know what might be valid and what not I really don't want to go into details because I have already posted this before in Wilders and it did not end well - arguing. People here don't understand some things I post.

I am not defending products and companies - I defend principles and good practices. I defend serious and reputable companies.

 

Once a company only focused on its profit rather than its customers, it will be a disaster. A lot of once-dominant products/companies were bought out by symantec then were ruined by symantec. a few examples are: Partition magic, ghost, drive image. The whole dev team of Ghost was disbanded a couple of years ago, that's why we see Ghost 15 forever. Symantec just anounced that there will be no ghost 16. I mean, why the heck do they do this?
Although this is not directly related to antivirus business, I feel symantec is doing the same thing to Norton AV. We'll see in a few years.

 

Imo, any "big" antivirus vendor that doesn't want to participate in av-c tests, they have not just something but a lot to hide, period.

So please don't come with alleged "detection technologies"

 

Nonsense.
There can be several reasons why a vendor will not participate in these tests.
The methodology used.
Financial reasons.
Personal disputes etc.

What on earth would the av vendors have to hide?
The suggestion is preposterous.

 

Check out what a former symantec employee said about the culture in this company (originally posted by firecat on page 5 of this thread):

 

Throw out a blind warning in File Insight is very easy to do, while actually checking the file for malicious code needs too much man power and efforts for Symantec. How smart they are, lol.

 

"Big" vendors, the "big" names that used to participate in these test and then quit from them just because they suddenly "didn't like" their methodology.

With the tons of money they collect from dupe users, of course they have the right to leave the tests anytime they wish. Just don't come here trying to justify their (real) poor performance in av-c with alleged "new detection technologies"

 

Ah, I forgot to say, exchanging "infected" usb sticks is one of the most common practices among my neighborhood, colleagues and friends... Guess how this "big" vendor with "super-duper technologies" fares against this simple challenge...

 

Read the next line after the one I quoted. They had to add a signature in those cases (so that the real-time scanner would catch it), because as the malware gained reputation, the WS.Reputation.1 detection disappeared. What I was saying is that there was a small window of time where Norton was not detecting anything on those files for whatever reason, and had to be compensated by adding a signature. This doesn't qualify as on-demand static scanning, but is definitely a plausible scenario (though rare). In any case, like I have described in previous posts, in general I haven't seen many PCs with Norton on it get easily infected, so what I'm saying is probably not as important as it may seem reading it

(Not sure how SONAR would have played into this, because I already knew these files were malware, so I was careful with handling them. In any case, this is one scenario where Insight doesn't yield the expected result).

I was just saying, there is still a reason for signature based analysis to exist, on-demand scan or not. Reputation based analysis can fail. The more signatures they add, the better a product is going to do in an on-demand test.....and conversely, the less they add, the worse it will do in a scenario like what I have mentioned.

However, I must say that if you like Norton, by all means continue using it - it's not that bad, the real alarm bell is when it starts failing some of these tests. The fact that it passes indicates it still is competent in the real world, so I would not advise anyone to switch based on just this one test result if they are happy with how Norton works. I am still using PC Tools, for example