MRG Flash Tests 2011

I'm still wondering about MD5 of the tested files. For instance, I've hunted for all Hiloti (aka Cimag) files from various sources that appeared within the last 2 months and didn't find a single one undetected by ESET (including today's variants). Quite the contrary, I've found a number of Cimags detected only by ESET and DrWeb from famous AVs. This raises some doubts about what and how was tested, hence I would kindly ask the authors to provide a list of hashes for the tested files for verification.

 

Looks O.K. now.

 

Nope. Panda's not there.

 

Email sent.

 

We are having some layout problems, everything should be OK now. We will rebuild the table and Panda will be back in it.

There was an error with McAfee's and Eset's results, it is fixed now.

Regards,
Sveta

 

Norton results changed too...
from 40-32-8
to 44-35-9
Thanks for fixing.

 

Hi Sveta,
would it be possible to share MD5 of the tested files so that AV vendors can verify the test results? This is one of the AMTSO principles for testing which make tests reputable.

 

I would kind of like that to go a step further and include a VT link for a very specific reason. Not only will get your MD5s but the extra VT data can be used to confirm many other things including first seen by VT date.

 

trjam's been there ,done that, got the T shirt I"m sure.

 

Sticking with Avira. But I do luv them all.

 

Yes. I like the idea of providing the VT links along with the test results. I even think it is pretty doable as the sample involved per test is at very minimum. It would be easier to track how fast the vendors are adding their detections and all antivirus vendors could get the samples with ease. just my 2 cents

 

VirusTotal results can be misleading to some people as they only show how files are detected by on-demand scanners but do not tell anything about real protection capabilities of particular security programs to protect the user against the given malware.

 

Based on current overall results, here are the AV rankings showing percent PASSED:
(Hopefully no miscalculations! Some are tied.)

1. DefenseWall (100%)
2. Malwarebytes (98%)
3. Emsisoft (91%)
4. Zemana (90%)
5. Coranti (83%)
6. Prevx (82%)
7. Norton (80%)
8. Sunbelt (80%)
9. BluePoint Security (75%)
10. GData (52%)
11. Immunet (52%)
12. Eset (52%)
13. Avira (48%)
14. Kaspersky (48%)
15. McAfee (41%)
16. SUPERAntispyware (39%)
17. Panda (39%)
18. Avast (36%)
19. BitDefender (36%)
20. F-Secure (30%)
21. AVG (30%)
22. Microsoft (18%)
23. PCTools (3%)

 

I didn't mention it for the scan results, the MD5 and first VTed date are what I am interested in.

 

In the latest test 4/12/11 the combined Avast/Bitdefender results don't match up with G-Data which uses both their engines.http://malwareresearchgroup.com/

 

I noticed this too, and brought it up a little earlier in this thread. Seems that MRG is using Avast V6, while G Data is still using an earlier engine, V4 I believe, though I may be wrong, I forgot to check it. When G Data releases the 2012 line-up, it'll include the Avast V6 engine, so the test results should reflect this.

 

On the latest test dated 4/12 avast! passes everything but Hiloti and TDSS shows orange instead of green. Anybody know why? Or maybe I am just colorblind.

 

they have caught on-execution by avast bb

 

Thanks. Looks like the Behavior Blocker is starting to do its job.

 

Orange probably means they got AutoSandboxed, after they have been flagged as suspicous based on the combination of heuristics/behavioral shield/community.

 

Yeah. It is definetely improved from v5.1. It catches some malware on execution in my sandboxie
Currently, i could not test it as my sandboxie wouldn't run any browser because of a bug related to trusteer. Waiting for v3.55 sandboxie.

 

New Test (4/13) http://malwareresearchgroup.com/2011/04/mrg-flash-test-4132011/
Congrats to AVG, Eset. Bad day for Prevx, Zemana.
Totals updated: http://malwareresearchgroup.com/malware-tests/flash-test-results/

 

Great to see MRG performing this type of "simple" but yet possible in the real world testing. Really becoming one of my favourite tests to follow. I hope they continue with this and maybe more nice testing, so we can just rely on AV-Comparatives/AV-Test.org/VirusBulletin for the bigger tests. In my eyes all other "professional" tests are useless.

Suggestion to the MRG website administrator, it would be nice to be able to sort the totals table on "Passed".

 

I asked the same "color" question on this thread about a month ago, and someone pointed to this post by Sveta.

 

4/13 Tests: