New Detection Test - Dennis Labs

Yep that's pretty much where I stopped reading.

 

We could look forever and not find a perfect test. All of the testing organizations whether independent or not are flawed, and probably will never be perfect as nothing ever is. Symantec is definitely on the right path with this test but the question I have, and I don't know much about the AV-testing business, but if they didn't want to look suspect why not just promote this testing methodology to the testers and let them perform the test when they got to it without paying one testing company to do it right then. It has already been said that Symantec is consistently in the top tier of security products so I I don't see why the test needed to be performed right away as they have nothing to lose or gain either way. Maybe this is flawed and not how it works I don't know, it just seems like the logical thing to have been done if they didn't want to look suspectby the outcome.

 

"In the comparative, of those with the Advanced+ rating, only gdata scored higher than symantec."

I don't know one test from the other, but I can't help but wonder why the only av that scored higher than symantec was not tested by Dennis Labs for symantec. And that is where I have a problem.
Or I could be completely wrong-it may be there and I just didn't see it.
Hugger

 

I think it has everything to do with bang for buck. These tests are expensive, and every new product added to the test costs that much more. Symantec like everyone else I am sure has a budget for these tests, and they use that budget to select product that they compete with. GData is probably not seen as a competitor. I would be surprised if say a McAfee or a KIS was missing. GData is small fry!

My two yuans (cents are not worth much these days )

 

However, I think this Dennis Labs test is more informative than a simple on-demand test like AV-C. Detection is useless if prevention and cleaning are faulty

 

I'd say it has nothing to do with bang for the buck.

On Friday October 30, 2009, Symantec Corp. reported second-quarter net revenues of $1.47 billion. Net income for the second quarter was $150 million. Symantec ended the quarter with cash, cash equivalents and short-term investments of $2.33 billion. The company does not have any long term debt.

Mind you, this was for a 3 month period... $150 million income.

Just how expensive do you suppose these tests are?

 

Symantec suffered a net operating loss of $6,786 million for YE 2009. The small net income you cite for Q2 2010 hardly makes up for the huge loss suffered during the previous fiscal year.

 

It appears that there will be no resolution to the question of whether Symantec influenced, as a consequence of paying for the testing services, the outcome of the comparative through both (a) the selection of the anti-virus products tested as well as (b) the choice of malware samples tested -- despite the lack of supporting evidence, despite the quality of the company’s reputation, and despite the mammoth downside risk to the company that would be incurred if such deceit occurred and was exposed.

I encourage all who adhere to this “Symantec conspiracy theory” to be intellectually honest and consistent, and to therefore apply the same logic and standards to all other anti-virus manufacturers. For example, Prevx pays for the creation of the “Threats missed by other security vendors” statistics reported on its website and these statistics are not created by an independent organization. Thus, applying the same logic, one must question whether the numbers reported by Prevx are knowingly manipulated in order to enhance the perceived efficacy of their product.

Do I actually believe that Prevx is engaging in deceit? Of course not. But the financial arrangement and the lack of independence surround the creation of the statistics they report should be more than enough ammunition for many in this thread to turn their attention to Prevx and likewise express outrage at their misleading practices motivated by obvious conflicts of interest and a lack of transparency.

In my opinion, it is as foolish to believe that Symantec is deceitful as it is to believe the same of Prevx with respect to comparative statistics on anti-virus products. Yet, if you believe the former, then the latter is a logical consequence.

 

I would agree with that. As I stated earlier, if anyone but a Symantec employee had started this thread, we would have seen an entirely different take on this thread and test.

 

Except that we're only reporting that threats existed on user's PCs... nothing more, nothing less. This can obviously be proven unequivocally with the samples, which we are actively doing with some of the vendors currently.

I know it's been a few weeks, but please stop dragging Prevx into unnecessary arguments - what we provide is raw data, with no interpretation of it. It is an extremely simple formula: if product A is protecting the system and Malware B exists on the system, we report Malware B as being on a system protected by Product A, as summarized by our sentence at the top of the chart:

"The chart below shows how many infections Prevx 3.0 found yesterday, on PCs protected by security products from the following vendors:"

I'm honestly unsure how we could possibly phrase it any clearer, although there has only ever been one person who brings this up

No matter which test you look at, they will all be flawed in some way. Obviously, if Dennis Labs is a legitimate testing organization, they will have the data to back up their claims. I haven't looked through it in detail but although their sample set may only be of 40 samples, it is still a valid test (provided that what they said is true, of course, and I doubt Symantec would use a dubious testing organization).

Whether they went through 500,000 samples to find the 40 samples that produced the desired results is to be determined... but in the end, it is completely irrelevant - on those 40 samples, these are the results.

Given a few hours (depending on scan speed), any antivirus researcher could go down the list of each of the products mentioned in the Dennis Labs report and find 40 malicious files which are found by each product and found by none or nearly none of the others.

Testing organizations don't work for free because of the time it takes and the value which the tests provide. To reiterate - testing organizations don't work for free. If the exchange of money invalidates this test by Symantec, then it invalidates every other test performed (except for the few purely independent testers or truly independent tests done by magazines with their own funding which the community tends to invalidate because they aren't a "real" professional tester, but becoming a professional tester means working professionally, and that infers being paid )

I'm afraid P.E.T.A. is going to come after me for beating dead horses so frequently

 

I'll agree with that. I have been testing it today and it looks like a reasonably good product. However I have a license for 3 other products I thought looked good at first also... they all claim to be the best as well.

 

PrevxHelp, it appears that you completely missed my intended message, and I strongly encourage you to again read the post (#134). To be absolutely crystal clear, I am not criticizing Prevx! In fact, quite the opposite: I am using the respected reputation of Prevx to illustrate the fallacy of the arguments that are being made by some forum community members within this thread.

I have never heard anyone claim that Prevx’s reporting of the “Threats missed by other security vendors” is anything but completely truthful, and that’s the point: even though a company may have a financial and non-independent relationship to a set of anti-virus statistics, it does not logically follow that the credibility of the latter is invalidated by the former. If an individual accepts this assertion as true for Prevx, then by applying the same criteria and logic that person should reach the same conclusion about Symantec: namely, that their funding of the test isn’t grounds for rejecting the results. (One may have other objections to the test -- based on the methodology, for example -- but, that’s a different matter.)

If, on the other hand, a user adopts the perspective that the mere existence of a financial relationship between a company and the source producing anti-virus statistics invalidates the credibility of those numbers, then it would be necessary to reject the results of Symantec’s test in addition to those “threats missed” statistics reported by Prevx. Both actions, in my opinion, would be unwarranted and equally unwise.

Does this commentary help to clarify and to document the docile nature of the example?

P.S.: I have no intention of restarting a conversation about the “Threats missed by other security vendors” statistics issue!

 

This made me laugh so hard I fell out of my chair....


Do you guys seriously even practice what you preach because your company has proved otherwise in the past.



Edit.

There is a ENTIRE thread on prevx and that graph a simple search will bring it up, prevx was put Thur the ringer about that very thing. Just a FYI!

 

This is because all it is. Is Symantec patting them self's on the back expecting people to be in awe. When Symantec employs someone like AV Corp to do these kind of tests then I will read it and go from there, But when you employ some backdoor company under the disguise of independent testing its laughable at best.

 

Unfortunately, Symantec could employ the Almighty to perform the anti-virus comparative and it would still be subject to criticism among a subset of individuals, I suspect....

 

Thank you for the clarification While I have a mini-soapbox here, my point is that virtually all tests that take place do involve money exchanging hands. However, to stay away from fueling conspiracy theorists, I strongly doubt that, in the antivirus industry especially where it is almost always obvious when a file is detected or isn't, paying for a test from a legitimate vendor will influence the results (or that paying more would push them to provide better results). The payment is merely a means to perform the test.

Therefore, in many (not all) cases, it is possible for a vendor to pay for a test and still have the tester remain independent. As to if the vendor would subsequently publish a test which shines a not-so-pleasant light on them after they paid for it... well, I'll allow the reader to be the judge However, this issue is why users have negative attitudes towards vendor-funded tests: only the good ones end up getting published.

Phew

 

We are in agreement! As usual, I applaud your professionalism and willingness to be civil and considerate to your competition.

<Humor> Now, if only everyone else participating in this thread would come to see the meritorious wisdom of our perspective! </Humor>​

Very true, but this is a criticism of almost all research endeavors. For example, when a researcher (e.g., physicist) runs an experiment which “fails” (i.e., produces a result contrary to her/his own theory), that result almost never sees the light of day in an academic journal.

 

Absolutely it will be suspect if it is presented as this test was on this thread.

 

No... I firmly believe that test results would never see the light of day. So next time insted of blowing your money on some no name company. Use one that will do the job correct and has been for years.

 

"Unfortunately, Symantec could employ the Almighty to perform the anti-virus comparative and it would still be subject to criticism among a subset of individuals, I suspect...."

Depends on whose version of 'the Almighty' you're referring to.

 

Lets be honest,these tests aren't designed for the well informed people that frequent the likes of Wilders,their purpose is to wow the average user with it's 100% award and give them a contented glow as they reach for their credit card.

 

Warning this man speaks the truth, all AV vendors need to cover your ears until the bad man leaves.

 

I guess this thread served its purpose. Though I myself don't like the idea of vendors promoting sponsored tests that they won, it was enough to get me to test NIS 2010 myself. After finding that it didn't seem to have any problems with Windows 7 when all of the other products I have tried have at least 1 issue, I went ahead and just bought a 2 year license for this. This will be my first use of something Norton since 2003. I hope it works well for that long. If not I'll throw it on the pile with the other ones.

 

You should be absolutely embarrassed to post the results to this so called "test". Seriously, 40 samples? There are probably close to a million pieces of malware out there and you select 40 samples as a test bed? It is utterly ridiculous and I'm amazed that Symantec is that desperate. It doesn't matter that you paid for the test or not. It's like me stating I'm the fastest runner in the 100 meter run..... then you find out that I ran against 2 and 3 year olds.