MRG Flash Tests 2011

surprised to see immunet signature caught all of those.. kudos to them..keep up the good work

 

The summary above shows Eset missing exploit 3, when it actually passed all 4

 

Yes. Similarly, Kaspersky caught all 4 and is not displayed under detection for the first exploit.

 

that is a disapointment indeed for prevx

 

I don't think Prevx 3 blocks exploits, from past talks with PrevxHelp. Prevx 4 will change that.

I'm not sure if PrevxHelp meant all exploits, or just those that initiate via web browser?

Hopefully version 4 will change that behavior.

-edit-

Apparently, they all had HTTP as the infection vector. (I missed that part.).

 

According to Sveta:

"The purpose of MRG flash tests is to show users how security applications react to zero-day threats.

In these tests we will be using samples taken directly from our honeypots. After analyzing the samples to verify their malicious nature, they are moved to our labs and used in these tests.

We have created a simple methodology to emulate how these zero day threats could infect a system in the real world.

After verification, the samples are automatically uploaded to our test URL, where they are ready to be downloaded via Internet Explorer to test systems with active security software installed. The Security applications have three chances to detect threats:

1) Upon download

2) During an on demand scan

3) On execution in real time

The application under test is given a pass if it detects the sample in 1) or 2) above, or, if it successfully blocks its execution and prevents its action during 3)."

http://forums.malwareresearchgroup.com/viewtopic.php?f=32&t=451

 

Good work! Except that's for the 2010 series of their Flash tests.
Try this one:
-http://forums.malwareresearchgroup.com/viewtopic.php?f=18&t=561-
And since the 5/18 detections report was prefaced with "Please note that this test is just a preview of whats to come..." and not included in the running tally, it's probable they diverged from that strategy.

 

Hey,

Please, don't rush judgements about MBAM [which, by the way I run on my PC, paid version, but using it as on-demand scanner to back up NOD32 4.2].

MBAM is really good but...not against exploits but against the playloads delivered by these exploits.
And, talking about playloads, MBAM relies HEAVILY on their user-base to hunt for malware and submit samples to them. I'm registered on their forums and, for a while I used to hunt for malware on the Internet on daily basis and submit samples to them. Although, I had to stop doing so because it's time consuming and I've been very busy at work putting a lot of over-time.

What really surprises me is how bad SuperAntiSpyware [SAS] has been doing on these tests. I had thought they were on par with MBAM but I see they aren't.
I even thought about purchasing a license from them but now, after seeing this debacle I'm thinking twice before committing.


Regards,



Carlos

 

You are right

 

Yeah, they're somewhat careless in their reports. One of their forum members runs a spreadsheet and if you compare the MRG and Detects sheets you can see omissions are steady.
-https://docs.google.com/leaf?id=0BxamVvlZYmoyNmZhYTQ0MDEtMmY2OS00MzczLTg2MWEtOTU3Yzc2NDNmYjVj&sort=name&layout=list&num=50-
He even notes "Empty cells represent Not Reported or Failed."

Also an amber Passed indicates the user had to take an action and was applied to Avast's sandbox but there have been sandboxed detections that received a green Passed. Unless there's something about interacting (or the lack of) with Avast's sandbox that I'm not aware of.

But, MRG produces some really interesting and valuable data and our focus should be on the Passed and Failed. The methodology is valid IMHO and an inconsistency here and there on secondary and tertiary levels is OK considering the work load.

 

Don't worry, Carlos. I doubt that anyone who has been following these test results will rush judgement about MBAM.
MBAM has a phenomenal 96% success rating.
SAS has a very dismal 36% success rating.
It's easy to see where the money is better spent.

 

As I've already said before: SAS is overrated ...

 

Absolutely right!! +1

 

+2 ....................

 

WOW, оne of the rare Norton fails

 

My bad I presumed that this was an ongoing series of tests with unchanged methodology.

 

Indeed, with Clam signatures

 

Hahaha surprise!

 

Interesting read FWIW:
-http://blog.clamav.net/2011/03/top-5-misconceptions-about-clamav.html-
They never did respond to the anon comment re Immunet/SourceFire vs win32/SourceForge as ClamAV seems to be now on those two development branches. The win32 libraries are used by ClamWIN and Spyware Terminator and countless proprietary enterprise deployments.

I've observed the ClamAV win32 daily.cvd sig file is called daily.cld in Immunet and is updated several times daily in the Immunet Free I'm running. (One would assume the sigs are in the cloud before they migrate to the local database.) Similarly, bytecode.cvd is bytecode.cld (updated in the same intervals as the cvd) but main.cvd remains that.

But how soon the cld sigs make it to the win32 cvds, if at all, begs the question as to how close SourceFire and SourceForge co-exist. Could be that ClamAV is ClamAV, except when it's not.

 

MRG Flash Test 29/06/2011

http://malwareresearchgroup.com/2011/06/mrg-flash-test-29062011/

Avast!

 

MBAM!
Sunbelt!

 

Is Microsoft passing anything recently?

 

lol microsoft going down.. panda not dancing anymore avast doing good after the credit thingy they incorporated..

 

Avast!
EAM back on track D00ds
Malwarebytes performing just as always

 

With this and the last flash test, Norton starting to slide.
Eset holding up well.