AV-Test Results for Nov/Dec Now Available

...

Ha-ha. AV-Test falls into their own trap.

They say they see new malware every two second - then why do they select them and check against only against 100, when they see millions ?

When they are so much on the ball, when they are so new - why they test and use old technologies such as Windows XP (code started to be written in 1999) and Windows 7 (released 2009) ? It doesn't matter Windows XP/7 are most used - security comes from the OS and apps - they are old and not secure enough. Why don't they test Windows 8 and Windows Server 2012 ? Why they don't test all Microsoft technologies such as Smart Screen Filter?

And the most important - they get millions of new samples but notice - they select samples they think are "recent" and important. With 100 out of 1 million - this is pretty much zero.

This is biased - this is total untrue. How come they select?! This means that they decide which product will detect what and how much ! And they get only 0.0..% of what is seen new (according to them).

These words of theirs totally put them low as they must be!

 

Using millions of samples from one family, but only a handful for other families also creates a bias, although of a different kind.

AV Testing isn't easy. And I don't think i'll live long enough to see a truly "scientifically sound" test result that would accurately reflect the capabilities of AVs in a realistic scenario for all types of users.

It will be a lifelong struggle for testing organizations to improve their methodology. Regarding complexity and ambiguity, it is just not the same as testing most other consumer products.

 

Based on my own experience, MSE is not adequate to protect the novice user's computer. My aunt only had MSE on her computer. She used her computer for web surfing and email. Her computer consistently became infected with viruses. I suspect this had more to do with her grandchildren's use of the computer. Because I had to clean her computer for her, I finally asked her if I could put the Internet Security package I used on her computer. She agreed and I installed KIS. In the ensuing year that KIS has been protecting her computer, she has not had a single infection.

My brother's home computer is protected only by MSE and is used by his wife, two of his kids, and himself. It consistently becomes infected with viruses. I put KIS on two of his boys' laptop computers and they have never been infected since I installed KIS on them. The reason my brother won't install anything else on his home computer is because he reads the MS "information" and says that according to MS, an Internet Security package is a waste of money and not needed.

So, based on my own experience, I do not believe the information that MS released to excuse their poor test results. Others on this forum who claim to successfully use MSE only are more than likely using much safer internet practices than the average novice users like those I described above. It is the novice user that MSE is supposedly trying to protect, not the experienced or expert user.

 

From MS Response:
This post reviews AV-Test's results and their approach. In-depth details are provided below, but here are some key upfront data points to keep in mind:

1.AV-Test reports on samples hit/missed by category. We report (and prioritize our work) based on customer impact.
2.AV-Test's test results indicate that our products detected 72 percent of all "0-day malware" using a sample size of 100 pieces of malware. We know from telemetry from hundreds of millions of systems around the world that 99.997 percent of our customers hit with any 0-day did not encounter the malware samples tested in this test.
3.AV-Test's test results indicate that our products missed 9 percent of "recent malware" using a sample size of 216,000 pieces of malware. We know from telemetry that 94 percent of these missed malware samples were never encountered by any of our customers.

In other words, is he saying that user infection from 0-day malware is so rare these days that there is not really much point in an AV providing protection against such attacks? ie it would be a waste of resources.

I also think he is saying that they had 0.003% users infected by the 0-day malware samples. What is the user base of MSE? Still could be thousands of 0-day user infections worldwide and that's just from these 100 samples.

 

I'll just leave this here:
http://en.wikipedia.org/wiki/Scientific_method
http://en.wikipedia.org/wiki/Critical_thinking

Keep these in mind when evaluating claims in any dispute you come across in your life.

Trust me on that one.

 

There is a simpler reading of the MSE statement: AV-Test org have choosen malware samples that are not representative of the malware infections out there (based on hundreds of millions MSE users base).

 

Microsoft have the resources and I believe the talented programmers to create an excellent product and yet they don't. Is not like they are short on cash. Maybe they don't want to spoit the good relations with other AV companies.

 

Microsoft can never come up with a good product despite their resources.

 

Very true! Microsoft are the last entity I would trust for security / antivirus

I've had so many viruses slip through while having MSE on than any other AV their AV is a joke

 

I would not entirely agree with you there.Depends what sort of user you are and what sites you visit etc.
Would you care to elaborate which entity you would trust with security please.?

 

My experience shows that in the IT business world good business relationships are what matters the most.

 

I do respect your views but I wonder - you trust them and use the most important thing - the operating system from MS, but you don't trust them for ... ... .. security ?

If the operating system is not good/not secure no matter the application software - the whole computer system suffers.

Note = AVs are application software

 

Well said

 

Wow, what happened to Webroot! They have been the lightest AV available for a while now. Their usability score had a major drop! Why the change? 4.0 for usability is not bad, but its a major change from their last few test results. I'm still not happy with Eset's scores. They got outscored by most other vendors in protection, and repair.

 

4.0 is not bad at all,score would be much higher there were it not for the fp's.Fp's will kill your usability score unfortunately.My guess is most people would not encounter fp's at all.I have only ever had one and it was fixed within hours of contacting Webroot.

 

Yes this is something that Webroot is great with.

However, they need to work on the fp's for sure. I experience a fair amount of them both at work and home.

 

My comments on MSE. It could also be a case of guilt by association.

Most MSE users are unsophisticated PC users. Their browsers are most likely not properly configured and definitely not configured for max. security. Their application software updates are not current and probably not for the OS. Many do risky Internet behavior; peer to peer, no web site checker, etc. Finally, MSE probably isn't even updated on a regular basis.

So making absolute statements about number of people infected that use MSE are irrelevant.

 

I would suggest contacting Andreas Marx directly with any questions, doubts or comments you may have. You might just get an answer.

 

Statement from Symantec about PC Tools' failure on the PC Tools facebook page:

Let's see what happens next?

 

Your 'statement' could be made against the vast majority of so called 'unsophisticated' users out there – so not sure why you single out MSE users.

Consider this - most computers are packaged these days with a heap of bloatware (well, some may be useful) - including an AV - always a limited trial version. In my experience, MSE is NEVER packaged with retail computer sales – so how does it get onto one's PC? - either by choice, or it is recommended by someone.

With regard to results from sites such as AV Test – these are of undoubted interest to 'sophisticated' users – and the subject of endless debate. The results can, and are, used by marketing and\or IT experts as a yardstick of performance, etc. But does that mean a product at the bottom of list (with a fail) wont still provide adequate protection for most users? Hardly!
The reality is that out of the hundreds of millions of Windows users – only a tiny fraction ever get infected by malware in the wild.

And by the way – I note some Wilders people (whom you would think would be classed as 'sophisticated' users) - admitting that they have been infected by malware and then ranting against the products they use as the sole culprit of their misdeed!

 

I believe he's presuming that most inexperienced windows users are going to lean towards Microsoft products and services. As for the rest of his points, they are reasonably valid for probably at least 80% of the windows user-base. They don't update when they should, they don't change default settings, they click yes to any pop-up (even UAV) that appears, and they engage in other risky activities. So many of the reports against security applications like MSE could be dismissed based on user incompetence; however, I'd argue that any application or system that falls back on user incompetence as an excuse for their product or service failing should be ashamed of themselves. Expect that most of the people buying your product are idiots, and expect they are going to try stupid things. For example, like installing more then one firewall.

 

Have no idea where you get this 'presumption' from.
I requote, since you appear to have missed the point - ''In my experience, MSE is NEVER packaged with retail computer sales – so how does it get onto one's PC? - either by choice, or it is recommended by someone.''
Edit
Ooops! - sorry bout that! - I misread TO WHOM you were referring!

One would think that since an AV (other than MSE) is generally packaged with computer sales, that users would stick with what was already installed (eg, paying to keep what's already installed) – yet this doesn't seem to be the case judging by the huge MSE user base out there. (Of course, the reasons why MSE ends up on many users computers are many and varied - whether the reasons are valid or not, may be debatable)

Are you a marketing guru with an attitude like that – or just a cynic?

 

Why don't they test the full version of avast?

 

I don't know the official reason but I suspect that Avast wants this. Avast company user base consists of free program users mostly (AFAIK ~91% of all Avast program users use the free version). Additionally, the additional features the paid products provide do not increase the protection that high enough so that tests can detect it

 

Yes, I think I once read in a test report from AV-Comparatives that Avast wanted their free version tested.