[Thread split]MRG Flash Tests 2012

LOL. Me too.

 

Although im Happy EAM did well (Go EAM!!!!) Its strange that their results are the opposite of what most results are on these products tested by other testers. I guess a pinch of salt is in order.

 

not a pinch ! a double handful..

that's amazing when bitdefender acts like this..

i'm still shocked !!

thumbs up !

 

Well when it comes To BitDefender, this is an example of that. Usually BitDefender scores high everywhere, not just in some random test. Yet here it scores low. Dosent make sense.

 

You read my mind.

 

Maybe this is due to Bitdefenders engine is involved in more antimalware solutions then I can count.
As these tests are built around very fresh samples, (if I am correctly informed)
I guess the virus makers make sure they can bypass the biggest vendors on the market, as seen in the current overall results.

/E

 

Wait a minute here, Are you saying the other test sites are using stale malware!!!!...Just goes to show your not safe any which way you go. Some of the best Here dont always score so high elsewhere. So if someone goes with a bitdefender they get 100% but yet here are vulnerable. And EAM here scores high, but no so much elsewhere. Seems from the results here and afar we need Defense Wall for 64 Bit.

 

I do not think they use "stale" malware, maybe not just as fresh as MRG does.
Then I think all samples are executed, this in turn will activate behavior blockers etc. in the different products. Meaning that some product will perform better in the MRG Flash test.
Maybe we will see Sveta in here to explain a bit regarding how they test, how fresh the samples are and so on.

I copied this interesting info from the latest test today:

"Hello everybody,
Flash tests are back. In the first 5 testes we concentrated on threats emerging from the Russian Federation, China, India and South America. In the one posted today and the next 4 we will focus our attention on threats emerging from the EU and North America. Once we finish the first 100 samples, we will focus on the entire “malware network “.
Regards,
Sveta"

/E

 

Just found the needed info:

http://www.mrg-effitas.com/current-tests/flash-test-results/

/E

 

again not acceptable.. even if they used fresh malwares , so how come MBAM did better than BD ? considering MBAM has no behavior blocker or HIPS.. whereas bitdefender has a robust one..

not acceptable.. whether or not you want to agree.. these tests are not as fair as AV-C tests. ( my opinion , don't wanna push it )

Regards,
Amin

 

The problem with any tests including this one and all the others is, what is meant by fresh malware? Do no others tests use fresh malware? How can bitdefender score well in all other tests. Yet not so well or average in this one. With such high scores, even just one of the others must have used fresh malware. Again this test should be taken with a grain of salt. Even me who uses EAM which scored perfect on the tests thus far, i take with a grain of salt.

 

Well, MBAM uses Shuriken heuristics module and adaptive signatures, you can read about it as Bruce from MBAM posted earlier in this thread.
What BD uses I don´t know, but so far not that robust.

I do not favor any software here, just viewing the facts.

/E

 

The problem still remains that if 5 tests say its robust and 1 says it isnt i question the results. Merely saying that this test uses fresh malware is not enough to justify the test as all tests can say they use fresh malware. But oh noes we use the freshest of the freshest malware. A product is accepted say by the community through use and independent studies of those products. Nothing is 100%, however with use and reviews a product usually attains a consistency. If it dosent is is usually dropped by users. Now yes the product may score 99% on one test and than 97% on another, but it holds a consistency. Now i dont use bitdefender, but to have such consistency in independent and reputable testing sites, along with a backing from many users, That it can score low on one test is already a red flag on that test. Not that the test could not be right. But it brings into question an array of things when it is the only test to rate a product such as bitdefender so low or average when it has rated high through many other tests.

 

I have only one short thing to say on this subject.

There is a gap between a sample that someone gives you and one you are testing at its live attack vector. Time exponentially widens this gap.

 

This is true. You can block old malware 100% and not block any just made ones. But Av-Tests states in their mission statement that their goal is to directly detect the latest malware. If they are doing so. Than Bitdefender should score on par with this test released by MRG. Yet it dosent. It scores very high by Av test standards. Some even score low on AV-Tests but score high on MRG test. If the security industry is so confused, im going to continue to bet on the malware to win out. ;0)

 

I hope Sveta starts doing some after the fact testing, say even just a 3 day delay on the same samples. I think that would explain a lot when it comes to what 'current' should actually mean. To be honest 12 hours old is often not current enough for the really bad stuff.

The fakealert/rogue infections are a very good example. If you have been watching these tests for quite some time you will see an unusually high failure rate for these compared to other infections. The people behind these work very hard on evasion as they know that if they don't dupe you right away, they likely wont at all and since these are highly visible and annoying infections they get reported like crazy so their life spans are very short.

 

I use amongst other things EAM. Now i test several products out as a hobby.

I tested this a short while ago and EAM missed it.

~ Removed de-linked Malware URL as per TOS ~

Yet it didnt miss a beat in the MRG test.

We just cant win!!! Ok im gonna go cry, save my seat.

 

Yes this would help tremendously because than it would show the gap in time. And truly up to the minute protection.

Bitdefender 10 minutes= Fail

Bitdefender 12 hours= Pass

 

Its been ages since I have tested over time changes in detection of something 0day but I thought it would be interesting for everyone to see what difference a day or two makes.

I know I cant post the logs here so I am doing the test over at the malwarebytes forum. I am not even going to link to it because I am not sure if that would be in violation but its easy enough to find over in general chat.

FYI this infection is the payload of a fake scanner collected through an actual real browsing session, no synthetic anything. The source is well known and between real world harvesting and VT sample sharing everyone should be on equal footing.

Obviously this is 1 sample and the results will be nothing more than interesting but I do hope that someone will take this idea and run with it using a much larger sample set.

 

Can you post or PM me the link with the thread? Nosirrah

 

NP, anyone that wants the link go ahead and PM me.

 

Very good idea!
I hope Sveta or Chris drops in, to hopefully pick up on this idea.

/E

 

Yea im gonna PM you cause i have MB Pro and would like to see the tests your conducting.

 

First one was a doosey. Low detection, later that day, more detect, still low detection rate. You'll even test a file or url and a program that is usually excellent will miss it. Yet a program who usually misses everything will catch it.

There is no protection from the Malware...bwhahahahahaha...

Ok back to your regularly scheduled program........

 

Two results for today

http://www.blog.mrg-effitas.com/mrg-effitas-flash-test-16-07-2012/

and twelve hours later

http://www.blog.mrg-effitas.com/mrg-effitas-flash-test-16-07-2012-12-hours-later/

Getting more interesting now