MRG Flash Tests 2011

That's fair. But, say someone else picks different 150 0-day threats each day. Will those, that perform 145/150, have the same results with new samples? Not really. The same way, those that perform 5/150, may now perform 145/150 with new samples.

May I be free to cite your question? "are you honestly saying that this means nothing?"

 

Doesn't it become interesting if X and Y continue to detect everything as where A, B and C (regularly/occasionally) drop the ball?

(Everything meant regarding these particular MRG Flash test setups, not other AV tests).

 

That is exactly why they don't do one massive pull and test all that day and call it a 'test'. Malware evolves over time and so does antimalware so multiple tests a week is the only way to aggregate a % that means anything at all.

Obviously the best of both worlds (150 samples a day confirmed to be new) would be better but that would take a lot of $ to pay the manpower, otherwise this is exactly what they would be doing.

Either way, I am glad that this is not yet another test that seems like all the other tests.

 

If you look at it like that, it could be said for almost every test.

 

This test is tough to rehearse for. It's not going to have alot of fans.You are either there or you're not.

 

I think that part's wrong because Malwarebtes' IP blocking succeeded.

 

Occasionally I test a few antimalware using more recent samples that is relatively easy to find on the Net. Vipre is great but can not achieve even close to 100%.

For me, the bigest mistery of MRG Product Comparison is how Vipre passes each test if:

a) Sveta using 0-day malware - "new and nasty stuff"
b) Vipre is installed with default settings - ("Allow Unknown Programs" by default)

MRG testing results are very difficult.

 

Fully agree. I hope further explanation by Sveta will be forthcoming.

 

Tests... tests... tests... stick to what works for you (and get away from what does not). I really don't care if avast! has missed almost every sample, I don't care if Panda scores so-so. I really do not care either if Vipre keeps a perfect score so far. For me, they've proven to be effective and enough. For me.
I don't get on the internet to go and look for a nasty that slips thru. No. I get on my computer to get things done. If you look for a way to bypass your setup (OS internals, 3rd party apps, virtualization), you will eventually end up finding it... eventually.
So please stop making a fuzz out of a sample missed or a sample detected.
If you belong to the Wilders community, you should be aware of the risks, pros and cons of your own setup. You should know how to stay safe and minimize infection vectors. Or else... start reading.

 

Agreed.Tests can show which is better but they actually far from perfect.

 

Fair enough. But your comments make me wonder why you are reading this thread in the first place.

 

lol,

may be he is inspiring us to also ignore the benefits of test


btw : i also agree with you atomomega

 

TDSS Rootkit 24 hours later. Good to see almost everyone adding detection.
http://malwareresearchgroup.com/2011/03/tdss-rootkit-24h-later/
Original Test:
http://malwareresearchgroup.com/2011/03/mrg-flash-test-3092011/

 

That is not their point, that same source has likely generated a minimum of 1 new variant an hour since it was pulled. They are attempting to demonstrate why 'after it no longer matters' testing does not tell you anything. If they were to take every sample in their test so far and held them all for even 1 day most vendors would show these same results and the results would be completely meaningless.

 

So vendors shouldn't add missed samples?

 

LOL I think you know that was not my point.

 

So, over time, heuristics and signatures will eventually catch up to the threat.
However, it is already too late. The original threat has already been missed.

 

Yes but there is a bigger systemic problem plaguing the whole industry.

If you run an AV and know what will make you look good in testing, where do you spend your research $s? If the hardest and most critical research results in very little visible results when subjected to conventional testing, do you still put more of an effort in where next to no one will notice just because you know your users need this the most?

 

The critical question is what are the chances a user will encounter one of the threats tested by MRG ? If the chances are reasonable, the uneducated user is in trouble since most AVs are not doing well against these threats.

 

Look at help logs, TDSS, spyeyes and zbot are everywhere. These are well funded projects with the best of the best malcoders. I hate to give credit to the bad guys buy they are a huge PITA to stay on top of.

The botnet black market relies on failed prevention combined with an update cycle that outpaces AV reaction times to remain a viable business.

Again, look at help forum logs, the majority have installed AV.

Layered + limited as a minimum would negate a huge chunk of those logs.

 

I like reading, and on top of that, I've been subscribed to this thread since the beginning

No, please don't take me wrong. I really respect the efforts and the work done by (almost) every test lab. The thing is that sometimes tests are (erroneously) taken as a mandatory requiremement to determine the real-world performance of an AV/AM product, which should not happen.

 

Layered and limited. I like that. Has a ring to it.
With SBIE Drop Rights and OA's Run Safer, I have the limited
and I also have the layered.
MBAM auto updates and auto scans.
Thanks for the excellent input, Bruce.

 

Yes. Bruce, thank you.

 

In the 2010 test archive it states ..

Methodology used in the Test:

3) Malware samples used are taken directly from MRG honeypots and analysed to verify their malicious nature.
4) The verified sample is uploaded to our test URL in readiness to be downloaded via Internet Explorer to the test systems which each has one of the 22 security applications installed.

That indicates to me that no AVs malicious web site blocking would count in these tests.

 

Plus 1.