MRG Flash Tests 2011

No more banking online in my family. I stopped 5 years ago, and it's not necessary, I still use my credit card though. This is what I do lately for great safety online:I use a Linux live CD, Lucid Puppy is great, it loads in 50 seconds and you can keep your configuration at every boot with a flash drive. Once you are done, that's it you take your CD out and nothing is left written to disk. Very effective, particularly against keyloggers, and... free. Any Linux live CD will do for that matter, but Puppy Linux loads in 50 seconds whereas the others might take several minutes before they are ready.

 

What I meant by imaging software, to restore an image of my OS previously created on a USB flash drive. Depending on the amount of data on the image, restorations can last anything between 7 and 18 minutes in my experience, but basically within 30 minutes of the 'accident' my system is back to the same original state. I have restored close to hundred images on several machines, and the reason has always been configurations mistakes, testing new OSs, bad installations, botched uninstalls etc, but never ever an infection. I think the danger of infection is real, but grossly overrated.

 

Single Product Test, DefenseWall blocked them all:
http://malwareresearchgroup.com/2011/11/17/single-product-flash-test-defensewall-hips-november-2011/

 

Very impressive!

 

It isn't that remarkable considering the policy restriction that DW employs. If it's untrusted, it'll not get through so it's hardly surprising malware will fail to pass by its defenses.

 

Protection being the name of the game , I'd say it's remarkable. I don't know what more you could ask for.

 

For the same reason, I'd also be interested to know how well AppGuard does on these tests.

 

Yes it would be interesting to see but I suspect that AppGuard would do quite well.

http://malwareresearchgroup.com/201...ash-tests-new-testing-specification/#comments

So perhaps sometime in the near future we may see some sort of testing by MRG of AppGuard.

 

Another test up - Webroot SecureAnywhere AV this time.

http://malwareresearchgroup.com/category/malwareproducttesting/

 

27/50 = 54% Pass rate for Webroot SecureAnywhere AV.

 

Embarrassing for former Prevx....

 

Your probably going to see a lot of results like that or worse from other products as well with the expanded test sets.

 

It sure would be great to see Sandboxie in these tests.

 

Webroot Secure Anywhere's detection rate was not as good as I thought it'd be. That's another reason why you should run it as another layer in your protection. It has great features for this purpose; SafeOnline and a light AV/AM which caches most but not all malware.

 

I wish MRG would run those same tests on Prevx to see if Webroot has watered down the protection.

 

We haven't - the entire Prevx team is still working at Webroot and it still has all of the same protection of Prevx.

If you'd like me to perform some Flash Tests, I can find 1,000 samples in the wild in the space of 30 minutes which are found 100% by WSA and 0% by any other vendor. Users shouldn't take the results of any one test as a definitive answer of how a product actually performs.

 

Yes, I would like to see that. It would be interesting information. I'll check back later for the results. Thanks.

 

The point is that any vendor can do this for any vendor. We just had a user write in asking if we could help with their infected PC - they had 135,000 infected files on it (all unique files) which their incumbent AV didn't find. As always, no AV is perfect.

As for these flash tests, I've asked around and no one within Webroot has received these samples from MRG so I couldn't even begin to guess what they are or where they came from or if any user actually ever saw them.

We stand behind our detection and we will fix any issue if we miss a sample - I don't see many (any?) other vendors standing behind their scan/cleanup engines like that...

Additionally, you can configure WSA to block unknown programs (which would cause it to always score 100%). That, along with other features of WSA and other AVs, means that users can control what they see.

I think Windows itself should be included in these flash tests with a policy set to deny execution from the folder c:\viruses\ and put the malware in that folder That would certainly score 100% every time!

 

Thats true. But not many vendors use that as marketing strategie like we can see for years at the nonsense graphs on PREVX homepage ...

Maybe every vendor should use such things? Look what we detected and what PREVX missed? Pure misleading of users and potential buyers!

And if we believe that MRG has choosen random samples and the total amount was only 100 - could we declare it as statistical hazard for the self named "world's Smallest, Fastest, Lightest and Strongest Security Software?"

 

That is a most fantastic claim for a MD5 checker.

Just for lulz back a while i had 12 ransom trojans from the same series c/o CleanMX and you detected all 12/12 after 24 hours

I took 1 of the files and added a funky character to the EOF and guess what>>no detection(must have broken the MD5 detection).

Sure you know by now Joe MD5 detection is retro(after the fact) so i bet you guys are flooded in your support channels with hosed customer machines

"Secure anywhere" seems to be a contradictionary description

 

So you're saying that PrevX or SecureAnywhere relies only in MD5?

 

We obviously don't just detect by MD5. Characters at the end of the file are handled differently - see: http://www.pcmag.com/article2/0,2817,2393678,00.asp:

"Next I checked its reaction to hand-modified versions of the same threats. For each sample I made a copy with a different filename, changed some non-executable bytes, and appended nulls to change the file size. Initially Webroot caught less than a third of these tweaked samples. However, they were all picked up after several rounds of checking for remaining threats."

 

too bad it doesnt work on x64 windows

 

Another Flash Test result up - Emsisoft AntiMalware this time and a 100% pass

http://malwareresearchgroup.com/category/malwareproducttesting/

 

May be just a naive question... is Emsisoft lucky with their test set or the sample is manipulated to show bad results for some and good results for others or some products are really lagging behind?