MRG Team performes their first Real Time test!

I was the vote that put a close to this one because I wont ever sign off on a fake test . Here are the reasons I felt this test was fake (going by what was presented to me as to how it would go down) .

Our IP blocking was going to be completely bypassed removing 50% of our potential abilities . A test that collects data based on intentionally pre-crippling a product does not generate real data . If an app can prevent the sample from being downloaded to begin with , isn't that a critical failure to intentionally fail to report ?

The contestants were not competitors . Instead applications that would be used together were pitted against each other . Suites were also going to be pitted against components . Anyone displaying this general lack of knowledge when it comes to intended function clearly does not possess the required knowledge to run a test . If you don't understand the applications , how can you test them ?

The samples being used were both far too few as well as a list obviously designed to test an antimalware suite , not component products . Malwarebytes is designed to be an antivirus complement and there are other products billed to perform this same task . Anyone that knows anything knows that we have pushed the vast majority of these applications out of the help forums and this is for a specific reason . Help forums track AV failures (as the majority of victims have AV installed) and when it comes to this group of malware we clearly excel .

In short I was asked if I wanted to be in a test where half of our defense was going to be disabled competing against applications we are designed to be used with against a super small sample set . My answer was that I will NEVER support a fake test . I did not feel like being part of yet another "these cars perform poorly as boats" test and its about time that people do some of their own research and demand some more realistic testing .

 

Oh please........ always the same story with MBAM, nothing is good enough for you guys and everything is fake.
Give it a rest for once

What is Malwarebytes Anti-Malware if it is not Anti-Malware

 

So are you also saying a hybrid car being tested against a hummer off road is also valid ? How bout a 2D app in 3D content creation ? If you want to stick with security why not test a firewall to see how well it removes infections ?

If anyone wants to research my position they will find that it goes FAR back before MBAM and has never changed .

 

There are many valid points in this thread of why this test isn't exactly the best type of test around. People only ever read the final page of a thread.

 

But of course, but calling everything FAKE is not OK, if you don't like something or do not agree with something there are another ways to express yourself.
Personally I would like to see Malwarebytes in some test, but by the looks of it, that will never happen as no test seems to be suitable for them.

 

People can get malware URLs from multiple public places .

The apps we compete against are available from multiple download sites .

Its a little more work to both learn how the software works and to design a test specific to the intended function , a little too much work it seems .

 

It's actually quite difficult to test MBAM in a way that reflects it's performance in the real world.A common experience is to go to a system with an AV already installed that's been infected with a rogue for example.In these circumstances running MBAM has yielded very good results for me on many occasions.Reading through any tech forums will show that my own experience is mirrored by many others.
The problem is transferring these anecdotal successes into a scientific test scenario and the ambiguity of MBAM's description hasn't really helped.I for one wouldn't rely upon MBAM as a standalone,full-blown AV defence but the marketing spiel isn't clear as to it's optimal usage IMO.

 

Dear nosirrah,

I am familiar with your research as we are both members on certain websites and forums, I would like to see Malwarebytes tested for once, reviews are all good but many users think that MBAM is a substitute for AV/AM apps which we both know it is not. If this test is not adequate maybe some other will be

Regards,

Astech

 

It seems that on their forum that my membership is being delayed and I don't the time to wait so I will answer their request here instead :

I ask for 2 and only 2 things .

1. The contestants are competing for the same software slot (AV:AV , AV complement:AV complement , suite:suite) .

2. The contestants are installed , activate and updated FIRST and then the test malware is drawn from its home loaction on the web with effort made to replicate the circumstances . For example it is OK to directly download malware that the user needs to interact with (like codecs/rogues .....) but if it comes from an exploit the system should be sent to the exploit site with the exploitable software installed .

To the people with issues with my comments earlier I have a direct question to all of you . Would these two requests generate far more accurate data than the typical testing you have seen hundreds of times ?

 

I am in talks with their management and we have discussed some very fair terms for future testing . As this develops more data will be divulged so that everyone can see the process but at this point what we have discussed would both put everyone of the same level and does a good job representing the real world defense against live threats in realistic circumstances .

I am sorry if I came off harsh today but I am a bit fed up with some of the testing I have seem for many years now . My harsh words were far more aimed at testing in general than MRG .

 

I'm glad to hear that as many of us were under the impression that you had a problem with Malware Research Group and their tests.
Nice to see this solved

 

i agree with MBAM.