new results from AV-Test.org (Q1/2008)

65k is not small, far from it

if ibk does use over 10 million, he should be slated for his method of lowering the ratings.

 

I am not sure where you fabricated this number from, but IBK's largest test set was the latest On Demand in August of 2007 and that test set was a total of 808,344. I am not sure a test set that large could even be tested, by the time you verified that each sample was valid and functional, the test would be irrelevant, it would take more than a few years to collect, sort and verify such a large test set.

 

Same Test-Results as ever. The known "good" AV's missing a few samples, the other a few little more.
Good to know that Freebees like Avast making their Job and you not to have spend some money for Security Software, because Nobody need such Placebos to lull the Mind.

 

he means the fp test.

 

Paul these large tests are flawed for the reasons you have given, they give zero clarification whether a av is good to the most important person, the customer.

 

touche Paul.

 

Seeing as F-Secure uses the Kaspersky engine amongst others, this figure tallies with the database record count of Kaspersky products at the time. That figure is somewhere over 527,000 now.

As we know, signature counts vary from vendor to vendor so this isn't a true reflection on how many computer viruses there are, especially when you consider the number of variants of one particular strain.

 

You can Test in thousand Ways and the Results are more or less always the same. It's really a hyper thing, but the user will be Infected, because Nobody show him how to get
a "really" secure PC.

@ Paul
If you have to use 10 Antivirusses -What's wrong?

 

Maybe nothing is wrong. Perhaps he likes to see how they work.

 

You can't be a true Wilders member unless you are running at least half a dozen antiviruses, hips, behavior blockers, firewalls...etc!!!!

 

Just to put things into perspective: Neither my employee nor I did request those results.But I am glad, that I got those results. All I know is, that Andreas Marx is currently in Spain, attending some Anti-Malware Task Force conference. My guess: AV-Test.org did test for the conference and didn't want those result go to waste.

To answer your first question: I will ask Andreas wether ad/spyware was included or not.

Now my very personal opinion:
I care about solid testing as well. That's why I gather as much information as possible, try to review lots of results and talk to people in my blog, in forums and at conferences like BlackHat and DefCon.

I don't know if Andreas over-hypes the numbers. How could I? But I wouldnt trust F-Secure either. (or any other Anti-Malware-Company for that matter.)

The biggest problem I have, is depending on externals sources for testing anti-malware-engines. That is because I have no way to gather a decent sample set. (Which should be always up to date.)

Question: Why does it matter if those samples are repacked or not? Don't repacked samples simply reflect the current situation? Just to make myself clear: Let's asume there were only 20 samples. 10 are the same malware but repacked, 10 are completely different. Scanner A detects all of the repacked samples, but only one the others. Scanner B detects only one repacked, but all of the others. Is Scanner B really better than Scanner A?

Last but not least: The ranking in my reviews of security software do not base only on pure detection rates. There are a lot of different issues which are important to our readers like: Does a security suite slow the system down? Are the alerts (firewall AND anti-virus) really understandable to non-tech humans?

Posting the results happend only for one reason: To engage discussions like this. My blog-entry ends with the following words: "My interpretation of the data is going to follow in a seperate blog-entry. I am curious: What program do you use? What is your opinion of the results?"

Thanks
Valentin

 

I'm doing all the other things, like hardening,very strong configured out my PC,alternative Browser with strong settings,too,contolling my clicking Fingers.
I use only a sandbox and an On-Demmand Scanner in Combination with well known Download-Sites.But that is here off Topic and my question was off Topic, too.

Most things of this we debate in the chip.de Forum ,too, and we debate about Valentins rewies, too, and a lot of people stand not behind this rewies and the Virusscanner Mindset, because its only money making and left the user in a false security feeling.

 

That is precisely what is wrong here. The concept that more is better. It's more-that's all.

 

It consistently does so and has done so for a good amount of time. There are many imitators- but Avira is the real deal.

 

congratulations to Avira. They rock again.

 

its funny you should use the word imitators Bunk,

there are 'very few' avs that aint an imitator.

most either use someone elses technology, or simply copy's it and calls it their own.

personally, id prefer a test of <1000 samples, of real-world threats and were manually checked for validity, then tested against the selection of AV's available, rather than 1 million un-tested samples, would this be easier and less time consuming, i dont know.

 

1) Marx is not overhyping the numbers. (see point 3, then you know why number of signatures do not equals to variants or multiple instances of same e.g. polymorphic malware)
2) IBK never said anything about 10 millions or any other number. beside that, ibk is one of the few testers which submits all false alarms to the vendors after the test in order that they can fix them.
3) when Marx says over 1 million of new samples of last 6 months, he means files with different md5. at e.g. f-secure they count the number of signatures; 1 signature may be able to cover 10000 of variants. when it said last 6 months, it could also be some samples which appeared for the first time already many years ago, but which he got somehow during the last 6 months again. there is lot of malware currently out there, keeping in mind that new variants etc. may be created automatically every second and be a potential risk for users (= its not malware which exists only in labs and which does not pose any risk in real world). you may just get one of those variants tomorrow, but you want to be protected against all variants, no matter if you get it in 10 minutes or in 12 minutes.
4) Marx probably did a very detailed report, but maybe Chip just summarized and published basic data without any details, so do not slap Marx for that. ask chip instead if they can give more details, in order that chip sees that users want and need the details (version, version number, signature dates, settings, size of various subsets, etc.).
5) Looking on how many new malware circulates and may be a risk fo users, many vendors are or are going to include behaviorbased protection technologies, in order to try to block malicious actions at least very shortly before you could get infected, e.g. when the malware is executed. that's why behaviorbased tests are needed to be done, along with the current on-access/on-demand tests.
5) I am still in Spain too, be back tomorrow. tired.

 

this i knew, or at least thought so.

so 2 FP's in a 65,000 FP-test is pretty darn good.

well, i think so.

and would not label it as 'Many FP's'

same goes for practically all of them, with the exception of Fortinet maybe

 

Within the antivirus testing tips report of May 2007 released by av-comparatives, it states that the set of clean files should number at least 10 million. That is where I am basing it off from.

 

yep, right you are.

it does say that, but i wonder if IBK uses 10 million+ for his tests.

no matter how many thousand, or million or whatever.... the amounts that are getting detected are practically NOTHING and no ratings should be lowered.

 

ratings SHOULD be lowered. but this year i will change a bit some things, and it will be easier to get higher scores (or in other words, FP's will not have much influence, as i do not think that there will be many FP's anymore).

 

ok, i will re-phrase.

ratings SHOULD be lowered, but if 30, or 40 or whatever is the amount for a 10 million test set (or even less than 1 million), they shouldnt.

AV-test have simply said they used 65,000 in their FP test, is there any particular reason why you IBK, dont disclose the info?

 

Hi and thanks for your insight. I am sorry but CHIP (or better I) did not get more details. But I already sent an email to Andreas Marx.

And most important: Have fun in Spain

 

So according to the overall results an average ,not techy user who sees the results thinks that avg and avast offer more complete protection than kaspersky,nod 32.I am quite surprised to see that even people on these site don't question this,since in my opinion this is not true.I hope that most people don't belive those test(not the metodology,or the doubt of the results ,i'm sure they are 100% accurate,but theiyr reflection to the real world(internet in this case )not the virus lab.