MRG Effitas 360 Degree Assessment & Certification Q4 2016

https://www.mrg-effitas.com/wp-content/uploads/2017/02/MRG-Effitas-360-Assessment-Q4-2016_wm.pdf

As usual, Windows Defender and Malwarebytes at the bottom of the heap.

 

This test also adds % detected via behavior analysis with Symantec scoring a whopping 15.56% for 360 malware and 26.47% for financial malware; telltale sign Symantec switching over to new machine learning algorithms.

Also appears signature/HIPS detection still quite effective.

 

LOL, like I said it's very dangerous to promote Win Defender like certain people do on this forum. And I suppose the new MBAM v3 would perform slightly better because it features an anti-ransomware component.

 

Not if its based on MBAR.

 

Good point, it didn't perform too well in most tests, I can't remember if people have already tested the new MBAM against ransomware. But it should perform well against exploits since it's based on MBAE. BTW, this test wasn't about exploits right? I suppose they simulated a scenario where people will download and run malicious apps that are hosted on certain websites.

 

Yes. Malware sample itself would have had to employ an exploit against a Win 10 or browser vulnerability. Report mentions use of 31 samples in the "Other" category w/o mentioning what they were.

 

Interesting result of Avast v12 (currently v17) specially taking into account that now it has AVG behaviour blocker and user base.

 

All I get is a "Page Not Found" from that URL.

 

Me too.

 

Yep, me too:

 

New link: https://www.mrg-effitas.com/wp-content/uploads/2017/02/MRG-Effitas-360-Assessment-Q4-2016.pdf

 

Kaspersky nailed another one

 

"You can bring a horse to water, but you can't make it drink."

 

Another test on the same day (2017/02/17):

https://www.mrg-effitas.com/recent-projects/our-projects/

 

Yeah. Two On-Line Banking Q4 2016 Tests. Can't figure out what is different between the two.

-EDIT- As far as I am concerned if the browser crashed during a botnet test, it is a pass grade. The only product to fully fail both botnet tests? You guessed it .......... Windows Defender.

 

Kaspersky, always Kasperky at the top.

 

Yawn - another test with parts of the native security disabled during test.
Testing online security and then test with zero-day capable URL-filter disabled. (Report 1,2 & 3)
Testing browser security and then having to use IE for years in a row on Win10 instead of Edge (Report 2 & 3)

 

URL filtering doesn't add protection, at least a lot, if an AV company knows that an URL is bad is because is able to detect the malware hosted on it and usually these URLs are uses as honeypots to get new malware samples

 

@guest :
Every single AV company will disagree with you on that.
And it appears that NSS Labs also disagrees with you : http://www.securityweek.com/microsoft-edge-tops-browser-protection-tests

 

Sorry but your link doesn't say anything different, phishing is not malware, and taking into account the current configuration of the modern browsers is almost impossible to be a victim of phishing, you have to be blind, so no mater what protection you have.

And if you are talking about websites with malware not phishing, the NSS labs test doesn't provide any valid input since it doesn't compare with AV, or show the real gap.

https://www.google.es/search?q=chro...&ie=UTF-8#safe=off&q=chrome unencrypted sites

https://www.thesslstore.com/blog/firefox-chrome-warning-about-insecure-login-pages/

What do you want to be fair? to test each product with the different browsers? or to test each product with it owns web filtering in use?

 

I just went though the 360 report again. No where did I see any statement that URL filtering was disabled for any tested security product. In fact, MRG stated all security apps were installed and run at default settings. That would mean that any URL filtering would be enabled if set on by default which I assume is the case for most.

 

@guest :
Please reread link again.
The NSS Labs report holds 220,918 socially engineered malware results AND 78,921 phishing results.
It's not only about phishing.
And they very clearly post block rates for both socially engineered malware results and for phishing results.

Since they test third party vendors with all modules active, then of course the native security should have all modules active.

 

Also please stay on topic. We are discussing the MRG tests; not the NSS Labs tests.

 

Didn't see that in this report. Nowhere it says that there was native security disabled or URL filter disabled. Also Edge was used for testing and not IE as you claim (more info in Appendix 1).

 

Three reports.
MRG Effitas Online Banking Certification Q4 2016 - two reports.
MRG Effitas 360 Assessment & Certification Q4 2016 - one report.
In two reports they used IE as I mentioned further up.
All testing institutions disables SmartScreen during testing since they can't get to samples otherwise.
Everyone that has used Windows 8.0, Windows 8.1 or Windows 10, knows what happens when you attempt to download a unknown/zero-day - in IE/Edge you get a big warning against it. With Chrome/Firefox you get the same big warning when you try run it after download.
The usual explanation from testing institutions are along the lines that SmartScreen are not WD, and they just test WD. In my book it's nonsense. Test Windows as a whole, since that is what a end user see. There's nothing "real world" about disabling parts of OS. SmartScreen has a system wide and a browser specific portion, both should be active in testing in order to be "real world".
Testing online security and then test with zero-day capable URL-filter disabled is ridiculous, when this has been official since 2015 : https://blogs.windows.com/msedgedev...en-drive-by-improvements/#VFHQbDvxrlxvf3wM.97
Testing browser security and then having to use IE for years in a row on Win10 instead of Edge is ridiculous, when this has been official since 2015 : https://blogs.windows.com/msedgedev...ge-module-code-integrity/#GRMFhVwMmcQu8hpI.97