[Thread split]MRG Flash Tests 2012

And more reliable.

 

Attention Attention!!!!

If you run any of the software that failed 12 hours later do yourself a favor and uninstall it. Even if you call customer service they will be asleep as evident by the results.

--------12 hours and 1 minute later as the sun is coming up---------

--------And security software companies across the world who only work 12 hours a day open for business-----------------------------------------


"Virus? What Virus?"


-------------Shortly Before Software company opened------------

-------------12 hours------------------

-------------As you check your bank account----------


"Where did all my money go"?

Keylogger strikes again.

 

It doesn't work like that. Many AV companies have automated systems and in some cases there are teams of virus analysts who work around the clock. Whatever product you use there'll always be a cat-and-mouse game. Customers shouldn't just uninstall a product because it happened to miss some samples in a test.

 

Well its good to know your so vulnerable Tony.

 

I found the results interesting, especially after the new variant appeared and only 13/42 vendors detecting it. The other point worth noting is that sample is now obsolete. Kinda makes detection of that sample moot.

 

Very interesting thread over there nosirrah!

 

there is one question which kept my mind busy ( look at my avatar )!

how reliable this MRG group and their tests are ?

are they as reliable as AV-C tests ?

please make it clear for me..

thnx

Regards,
Amin

 

I agree completely.

 

this has been discussed ad nauseum. i'd suggest searching (on google or wherever tickles your fancy) rather than derailing this thread with discussion about MRG's reliability.

 

Are they using the free or paid version of Avast to test?

 

http://www.mrg-effitas.com/current-tests/flash-test-results/

1. Avira AntiVir Premium
2. Avast Antivirus Professional
3. AVG Antivirus
4. BitDefender Antivirus
5. BluePoint Security
6. Emsisoft Anti-Malware
7. Eset Nod32 Antivirus
8. GFI Virpre Antivirus
9. Ikarus Virus Utilities
10. Kaspersky Anti-Virus
11. Malwarebytes Anti-Malware
12. McAfee Antivirus Plus
13. Microsoft Security Essentials
14. Panda Antivirus Pro
15. SoftSpehere DefenseWall
16. SourceFire Immunet Protect Plus/FireAmp
17. Symantec Norton Antivirus
18. Zemana Anti-Malware/Hitman Pro
19. Trend Micro Titanium Antivirus

Project Started: 29.06.2012

 

If you think about it a little more in depth any antivirus except those that have 100% rating so far are vulnerable and should all be uninstalled. Even those that passed 100% like EAM, I ran my own test on EAM with a freshly produced attack and it failed. A layered defense is probably best but even than I always bet on the malware to win out.

 

If you need 100% detection rate to keep an AV, don't bother installing one in the first place.

 

So very true.

 

As you have discovered, even those that score 100% on one test can fail in another. Time is of the essence and as Nossirah has shown in his own simple test with one sample, detection rates can change over the course of a few hours.

Layered protection obviously is key here to help with this situation. The AV industry talks of thousands of malware being out there, and yet, some of us will say what malware? simply because we don't come into contact with it*. At least I hope we don't have the misfortune to.

*Unless you deliberately search for it for testing purposes.

 

Yes, as it is important not only that you got home, but how you got home. Careful in what alleys you tread. BOOOOOOOOOOOOOOOO.

 

The same place you found my other test I started a new one. This time I am tracking exploit drops from popular site ebaumsworld.

FYI, you might want to update java and acrobat at the very least if you visit that site.

Generation 1 is already obsolete, it took 1 day.

If you cant find it, ping me.

 

New flash test results for 20.07.2012 (zero hour + 6 hours later)
http://www.blog.mrg-effitas.com/

 

In my opinion, MRG is not ready for prime time.

First they are using WIN 7 x86 for their platform. Run those test on a WIN 7 x64 platform and the results will be entirely different.

Next, they run the OS on a VM. "Real" test labs don't do that since they know many of today's malware can detect it's a VM and hide from detection.

 

20-07-2012
zero hour + 12
results available

 

Since MRG tend to test a small number of samples from specific families of malware, then it would be significant if any variants from these families were known to hide from VMs - only as far as it affects detection based on behaviour.

But keep in mind:
1. Executables behaving differently in a VM actually aids in detection since that is suspicious behaviour (the obfuscation just potentially makes it harder for researchers to track exactly what the malware does);
2. MRG can easily test each executable to make sure they can properly infect the virtual machines;
3. Only a small percentage of malware actually are VM/sandbox aware.

 

Why would a signature detection of downloaded malware differ between a x86 and x64 platform?

 

Now with these tests most Wilders will have a great insight on how time plays a super important role in detection rates. (Including me)

 

The next step will be to track per live source to pin down successful morph time frame (evasion of previous detections) VS. successful definitions update. If you are surprised at how fast poor collective detection turns into good collective detection you may be even more surprised at how must faster the successful morph rate is. I have been running a similar test on the malwarebytes forum, if anyone is interested please PM me.

 

Im checking those threads a few times every day.
Kinda interesting, usually when a malware "morphs" can't malware companies do like a generic signature that could detect the current and in any case future versions? (I know it won't be 100% but it doesn't looks like they are even trying)