MRG Flash Tests 2011

F-Secure always performs average in these tests for some odd reason.

 

That's poifect! Thank you, Blues.

The results I'm seeing could not be described as "average".
F-Secure has FAILED on 27 of the 36 samples thrown at it, meaning they PASS 25% of the time.

 

And yet, it won 'Product of the year 2010' award at AVC.

 

I think we have to apply the same "layered" philosophy to test results as we do to security applications... i.e. no single test or series of tests are adequate. For an overall clear picture, layers of tests should be consulted. You think?

 

Not to take anything away from F-Secure and AVC is very highly regarded but a much different type of testing is done there. And just show that I'm not picking on F-Secure, look at my sig. My solution is currently just breaking even but 2011 is young.

 

AVC tests every malware they can find. MRG only tests very recent (zero-day) samples.

 

I fail to explain probably due to bad English.

If you go to hxxp://support.clean-mx.de/clean-mx/viruses frequently and look at the results on Virus Total will see that, for example, Avira (whose capabilities and results are well-known to me), as well as some others, outperform Vipre, and we have heard here, on this topic, that those malware can not be considered as a real 0-day malware compared to those that MRG uses. So, how Vipre can achieve so much better results with the latest, real 0-day on MRG? That does not make sense unless the Vipre is particularly good at detecting certain types of Malware that are often repeated in the MRG tests (keyloggers, for example)?

I was hoping that we will get some explanation on Vipre's results in MRG's testing, but we did not receive any, only more new results. The facts that Norton and Defense Wall are achieving good results on MRG does not mean anything to me regarding the Vipre results that do not match my personal experience or observations to Virus Total. If Vipre beats Avira on AV-Comparatives, I will believe. Until then, my apologies, but do not believe in MRG's testing.

 

I understand much better myself, Zimzi, now that you have explained it better. Thank you. It comes down to different tests, and different number of samples used, to put it in a nutshell. At least that is my way of looking at it.

 

Abundantly clear now thanks. By the way Vipre has yet to come out and play on AVC.

 

Read the the AVC methodology and answer this question. If you defined every single malware sample exactly 1 week after it existed how would you score on their test?

How would you score if you only defined all malware that has existed for 1 week or less?

 

Answer to question 1 is you would be at or approaching 100%.
Answer to question 2 is not as good as you would do in question 1.

So I have my own question: When will we see Vipre in AVC since The Inspector was Esets liaison to AVC when he wore the happy bites moniker and is now a Sunbelt VP?

 

I cant speak for them but I would not take part in a test where anyone could master "after it no longer matters" defs and score very very well.

 

Ok , so it's not your favorite test. I get that,but not everyone does great some barely reach Standard and some even fail.

 

If it "no longer matters", then users would be well protected by using only Malwarebytes.

 

We see and do our best to fill a hole in the current security landscape.

We do well because we do not try to do everything. We target where we see AV being the weakest and I personally feel we have have done well in that mission so far.

 

Well I for one can't wait for a end of year 2011 summary of results.

 

I think everyone on this forum would agree you do very well. My point is there is also validity to tests other than 0 day.

 

Its not so much that I have a problem with that concept in as much that I feel the average user can never understand what the percentages do and do not indicate.

Personally I would love to see an organization like MRG be well funded and testing 100+ 0day samples per day and tracking aggregate trends over time. I am a big fan of any testing that eliminates being able to prepare from the equation.

 

You have made a good point.

Unfortunately, the term "0-day malware" has become a marketing hype. Due to testing such as MRG's average users are given to believe that every 0-day malware is very dangerous. In reality this is often not the case, as the risk of certain malware is estimated not only on the basis that it is <24h, but on the very important criteria such as "reported infections" and, especially, "distribution potential". Many 0-day malware has a low risk because of the small distribution potential. So, for example, 1-month old malware with large distribution potential is more dangerous than 0-day malware with small distribution potential. In this sense, the term 0-day says very little about the real malware risk to the average users.

Of all the malware that we (my family) came across in the last year, year and a half, during the usual using of our computers, not a single one was the 0-day.

Through various superficial 0-day antimalware tests pc users are given to, choosing between antimalware A (detects 99% common malware, 70% 0-day malware) and antimalware B (92% common malware, 90% 0-day malware), choose antimalware B, which would be probably the wrong choice for the average users in this case.

This does not mean that antimalware software that are specialized for certain types of malware, such as, for example, keyloggers and other data-stealing malware, have no importance, but they can not be a replacement for all-around antimalware software.

 

In that case maybe you could ask MRG to test MBAM without IP Blocking?

 

Look at the flash test results,ESET and Avira is fighting against each other.

 

I'm second here. Would love to see something similar to it.

 

Me too. But I would love to see a test made every day out of all the new 50,000 - 80,000 malware that are released daily.

But I guess it's quite difficult finding all the 0-day malware.
Otherwise the MRG test set would have been bigger I think.

 

Can it be feasible/possible for an organization with the size of MRG

 

the important part of the quoted post being "well funded" - if they were, it would be entirely possible.