Interesting Emsisoft Tests

Hi Guys,

Interesting test at Emsisoft with recent malware...

http://www.emsisoft.com/en/software/scanner/

Mike

 

It would be interesting when it would not have been on their own website, tested by them self. Of course, when I test my own products, and place the test result on my own website, my products would be the winner...

As I stated in another post, it detected 42 as malware on my pc, even the 42 have been all FP... so its detection rate was 100%, and other AV 0%...

 

Interesting test results there,just a couple of points though.
If this test was against A2 own samples how come it didn't detect 100%?
Also testing MBAM and SAS against static samples doesn't give a true indication of their efficasy if that was how the test was performed,they're designed to detect running malware.

 

Sure. If it's based on their own malware collection - you expect them to get 100%. But the _rest_ of the results, aside from the top spot are still interesting

 

Re: Is a-squared Free 4.0 good?

http://www.emsisoft.de/de/software/scanner/

 

At the bottom...

Note

This test was an Emsi Software internal comparison and does not represent an objective assessment of the detection performance of a-squared Anti-Malware. It stands to reason that our own in-house product will produce the best detection rate against our own Malware samples. After an in-depth analysis some files were rated as harmless in the meantime. That's why a-squared Anti-Malware did not reach 100%.

 

A-2 really should not have listed.

Since they did add all the tested samples to there databace before testing..... its kind of like me sending 40.000 malware samples to ClamAV waiting 4 months then testing. OFC CLamAV would have 99.99% detection rate.

 

OK thanks for answering point 1,but why test MBAM and SAS in this way that they're not designed to be used?

 

I agree completely. The first place must be discounted - even in their own notes that say that "of course" they will come first. What's interesting for me is the rest of the pack.

 

Dunno. I guess if it's a test of scanners, you'd run a scanner. That's one for Emsisoft, not for me.

 

Yes but a test should show a true reflection of the capabilities of the products tested in order to be credible.I put it that had these malware samples been activated on the system in question,both would have achieved a far higher percentage.Yes they're both scanners,but they're designed to scan for running malware.

 

Four recent samples.

it021.exe - a-squared 4.0.0.101 2009.04.30 <- Not Detected - Win32.Malware.gen

6007_1.exe - a-squared 4.0.0.101 2009.04.30 <- Not Detected - a variant of Win32/Tinxy.AD

bb.jpg - a-squared 4.0.0.101 2009.04.29 <- Not Detected - Generic Dropper.cx

setupxv.exe - a-squared 4.0.0.101 2009.04.22 <- Not Detected - virus:FraudTool.Win32.MalwareRomovalBot.b

 

Did the other vendors catch these?

 

If I was of a suspicious persuasion I might think that showing MBAM and SAS (2 directly competing products) in such a poor light wouldn't be bad for business at all.

 

If a-squared came third, or last, doesn't bother me, it's only one test, so hopefully, no one will take it personally.

And the note explains it quite well - "This test was an Emsi Software internal comparison and does not represent an objective assessment of the detection performance of a-squared Anti-Malware. It stands to reason that our own in-house product will produce the best detection rate against our own Malware samples. After an in-depth analysis some files were rated as harmless in the meantime. That's why a-squared Anti-Malware did not reach 100%."

But just commenting on the one test - being in a two-week period, some of those AVs which aren't in the top group, may add malware signatures a few weeks later, as they might not have the staff and/or resources.

Launching malware would be an interesting test. I still think Emsisoft would do well in launching/preventing malware, many which might be missed by an on-demand scan, as their behaviour blocking (Mamutu/a-squared anti-malware) is a core component of their software.

But on-demand scanning, not bad - for a small company.

 

File it021.exe - Result: 4/40 (10.00%)

File 6007_1.exe - Result: 11/40 (27.50%)

File bb.jpg - Result: 10/40 (25.00%)

File setupxv.exe - Result: 2/40 (5.00%)

 

Typically, that's why companies publish results that show their product in a good light, and forget (or attack) those that don't.

I don't think anyone here would be surprised at Emsisoft's motives for publishing these results.

 

That would be a very interesting test indeed. It would be hard to do it with a high number of samples - but it would be interesting to pluck out some of the ones that few companies detected and run them through by hand to see if it made any meaningful difference to the results.

 

Nasty.

 

This is why the independence of the testers is of paramount importance when ascertaining the credibility of any test.

 

I installed some legitimate software the other night, but when I noticed it was trying to change browser settings, at least gave me an idea how a 'clean' file is behaving.

So sometimes it's not only malware you want to prevent, you want to prevent unwanted system changes. Well to me, that's important.

Anyway, although this is an on-demand scan, many may agree, Mamutu/emsisoft's behaviour blocker, is easy to use and provides strong protection.

 

Agreed 100%

 

*grin* I couldn't agree with you more.

 

And same applies to Comodo, that's an extremely good result considering its a new AV. And launching the malware missed, the rest would most likely, be stopped in its tracks.

 

When preparing this 'test' we tried to be as careful as possible to avoid arguments like "this test is fake" and "the test methods are questionable".

It was not out intention to give an overall rating on security products. Of course we can't do that independently as a competing company.

We just wanted to show the differences of the on demand scanners and the speed of updates. Several independent tests recently showed that the top 5 scanner detections differ only by 1%, but these tests didn't show if they missed the same 1% or different 1% of the samples.

Another thing we wanted to uncover is the fact that antispyware tools, even if they come with "+AV" modules, can't really compete with the top antivirus players on the market.

Regarding Malwarebytes: This freeware scanner tool (not speaking about the guard version) is highly recommended by many people (not only here), but I have not seen a single test of their scanner capabilities so far. Some independent testers I've spoken with, told me that it is not good enough to be included in their tests at all, but that's another story. Well, we were very surprised that it wasn't able to detect at least 50% of our samples. I guess most users like MBAM because of the scan speed, but it's new to me that it is made to detect only running malware. Most people use only the freeware scanner without guard imho.

Again, our test is not a 'security' test that shows how good a program can actually protect your computer. It is a pure scan engine test. Not more, not less.

Testing was done very carefully and reproducable. If any of the vendors needs a hash list of the missed samples, please send me a note.