MRG Flash Tests 2011

I agree.

 

''well funded''...as av-comparatives and av-test or better than these two ones

 

I agree, there has always been a lot of criticism about MRG, but I find these flash tests really interesting. I hope they will only focus on these kind of tests, test products against aggressive malware with a short live, but which is actively spreading. For the reviews against huge sets of older malware we already have AV-Comparatives and VirusBulletin, then AV-Test.org, and also AV-Comparatives, can focus on dynamic tests.

 

Yes, indeed! For example: link 1, link 2

I agree. It is interesting to see how, for example, PREVX guys changed their opinions about MRG after their product "finally" achieved an excellent result in one of the MRG tests:

from link 1 to link 2

I do not know whether Sveta and Max perform any tests trying to earn some $$$ or just have good fun but want to send them the message to reduce the number of "passed" for Vipre. Apparently, they forgot that by default Vipre does not use proactive protection and therefore can not achieve such a great results with real 0-day malware.

If you do not believe me, just go to hxxp://support.clean-mx.de/clean-mx/viruses and see the results on Virus Total (small blue squares). You will see how Avira outperform Vipre. So, if Avira outperform Vipre with older malware how Vipre can be so much better with real 0-day malware on MRG's tests? You do not have to be virus expert to understand, it is elementary logic.

 

Thanks for that link, bookmarked

 

If by "proactive protection" you are referring to Vipre's Active Protection, then I disagree with you... it is on by default.

If you are referring to something else, please elaborate what you mean by "proactive protection".

 

Hi Zimzi,

If you read our methodology, detailed here - http://malwareresearchgroup.com/malware-tests/flash-test-results/ - you will see we test all applications with their default settings.

By Default VIPRE has “Active Protection” enabled with “notify me when known risks are blocked and quarantined” and “Check files when they are opened or copied” checked. Under the “Advanced” settings the “Allow unknown programs” is selected.

VIPRE is tested with these settings (see screen caps from our test system) and attained the results detailed in the report with these settings.



I do not understand what you mean when you say you “want to send them the message to reduce the number of "passed" for Vipre” – please explain why we need to change the results for VIPRE.

Please could you elaborate on your statement “Vipre does not use proactive protection and therefore can not achieve such a great results with real 0-day malware” as I don’t understand what you mean here. Are you suggesting MRG Effitas has published false results for VIPRE? Please be clear in your answer.

We process over 150,000 unique zero day – early life samples / day – compared to the 100 or so samples (which are not specifically zero day) which are available on the URL you posted. We run private efficacy assessments for clients all the time using many thousands of samples and find that generally, the flash tests match up with the deeper tests.

The crucial thing for vendors is to catch samples as near to their zero day as possible. This is not an easy task and it is for this reason that MRG Effitas supplies several vendors with tens of thousands of samples every day – see here for an example - http://www.superantispyware.com/media/Press_Release_MalwareResearchGroup.pdf

Regards,
Sveta

 

By deafult, except heuristic, Vipre Antivirus Premium does not use other proactive protections. There are the following settings (correct me if I'm wrong): no firewall enabled, unknown programs are allowed, prompting to act upon programs found with suspicious characteristics is unchecked, enabling of Host Intrusion Prevention System is unchecked. So, basicly, Vipre relies only on signatures and heuristic similar to, for example, Avira AntiVir Personal.

The founder of MRG, Sveta, said that Vipre detected samples by signature (mostly generic). Ok, we can imagine that Vipre has much better generic detection than Avira, NOD32, MSE, Kaspersky, Panda etc. but if you go to a website that I listed above you will see that even with older malware Vipre is outperformed by Avira (which I often mention only because, as a long time user, I'm familiar with its performances), as well as some others. So, how Vipre, under the same circumstances (relying only on signatures and heuristic) can be so much better than Avira, NOD32 or MSE etc. with real 0-day malware which were used on MRG's testing?

There must be some explanation. For example:

1. it is a mere coincidence due to the small number of samples. By the end of the test Vipre's percentage will fall somewhere near Avira, Kaspersky, NOD32 etc. which would be quite logical

2. during previous testing has been used more samples of specific types of malware (some sort of keyloggers, or similar) that Vipre particularly well detected

 

Hi Sveta,

I expected some professional explanations regarding the results achieved by Vipre? As an expert on viruses you certainly got some idea of ​​how the Vipre made ​​so well. It is not so good at Virus Total?

I really do not know if MRG has published false results. I hope not? If you got this impression it is probably due to my poor English.

I think it would be good for reliance in MRG's work to disclose more information about the people and organization itself. Where is MRG located (could we see photos of building/workingspace/people)? How many people work within the organization? Biographies of key people (university education, professional experience in the field of internet security)? Who are MRG's clients?

I'm asking for more information because must admit that I have a certain distrust regarding MRG. ~~ off-topic comment removed by LowWaterMark ~~

 

Virus Total makes use of CLI (command line interfaces), and not full versions. So, whenever one of the vendors that figure Virus Total may not detect an uploaded sample, it doesn't mean that the full version wouldn't.

I hope this clarifies it a bit.

 

The results on Virus Total perfectly coincide with my personal experience gained in testing Avira and Vipre. I listed Virus Total as something that people can check. I do not expect people to believe in everything I say, but I expect people not to believe in everything that someone else says.

 

What a person believes or not, is up to that person. But, that's besides the point. I just wanted to let you know that Virus Total does not make use of the full versions; that is, they don't have the same version we, the users, have. They have command line interface versions, and therefore lack some of the technology that's present in the versions we use.

I'm not saying anyone should trust or not trust MRG tests, AV-C tests, etc. Anyone is free to make their own judgement.

I just wanted to share a piece of information, that you could not be aware. I sincerely apologize for replying.

 

No need to apologize!?

As I already said, the results on Virus Total perfectly coincide with my personal experience, but, anyway, I'm curious to know, do you have specific information regarding Vipre's technology that is missing on Virus Total that is present in the desktop version?

 

I have not suggested that the information regarding MRG be published here, but on their site, but, anyway, I apologize if I violated some rules.

Generally I said everything I had on this topic so I will refrain from further discussing.

 

Sorry, I just thought you were mentioning I was blindly trusting this test (or any other), or somebody's word...

I don't have any specific information regarding Vipre (my first reply didn't have Vipre under consideration; rather a general consideration), but full versions do have other technologies that would be difficult to have in CLIs, such as behavior blockers, sandbox, etc.

I don't personally keep track on MRG tests (or any other to be honest), so I don't know how they perform tests or how they don't perform the tests.

My reply only had in mind to provide you with that information (CLIs), in case you didn't know already, and that could justify, perhaps, the discrepancy between results. Nothing more.

 

http://malwareresearchgroup.com/2011/04/mrg-flash-test-4042011/

 

Thanks. Coranti now 8/8. Norton, Prevx and Sunbelt with two misses each.
Emsisoft, Malwarebytes, DefenseWall, Zemana keep rolling along.
Totals here: http://malwareresearchgroup.com/malware-tests/flash-test-results/

 

Even with these two passes and two fails, Sunbelt is looking good overall... outpacing Norton and Prevx. I'd like to see Sunbelt maintain its position and not slip any. MalwareBytes really impresses. SUPERAntiSpyware continues to tank, passing only 35% of the tests. PC Tools has passed an embarrasing 3%. Yikes.

 

I really hate to say this, but these Flash tests are starting to make it look like free AVs are a bad idea (where the heck is Panda though? It says they participated.). Avira is weird, they're literally 50/50 on these. Avast, I adore you, but you've got to step it up. MSE looks like crap, surely they can't be that bad. I'm assuming MalwareBytes is tested using its real-time protection? If so, that 25 bucks I spent on it was money well spent. They always do good.

-edit-

I was looking at an older test when I made my comment about Panda not showing. I guess they aren't used in all tests?

 

I wish they would include AppGuard. Lol

 

I bought a couple MBAM licenses so I could run it real-time, and have scheduled scans and frequent updates. I was really impressed by their IP blocker website blocking. Then I decided to lighten up on my security software after I installed Sandboxie, and I dialed it back to on-demand only with daily updates and daily scans. But the point you make about MBAM can not be disputed... money well spent and they always do good in these tests. I think I'll turn the real-time protection module back on, and bump the updates to hourly.

Edit: terminology

 

They don't seem to call it "IP blocking" anymore, now it's "Website Protection". I'm not sure if it's just renamed or they tweaked/redesigned it, but it's a lot quieter than it used to be. I don't see any way to change updates to hourly either, just "download if available". I'm assuming it's the same exact thing. IP protection used to be the very first thing I turned off, it wouldn't leave you alone.

 

From what I can tell in the MRG forum thread, PC Tools was dropped because Threatfire is an optional component and isn't active unless you log in to your PC Tools account (which you get when you pay for it) and "activate" it. If that's true, it fails the MRG "default install" criteria as well as being beyond the scope of average user awareness.

I can't find where MRG accounts for dropping BluePoint.

Immunet has started to make a good showing even when BitDefender fails. I'd be curious to see what would result if they ran a separate test of Immunet with the Tetra engine disabled.

 

Heh, heh. They call it so many things. Here it is Website Blocking...

And you can schedule hourly updates per attached image...