MRG Effitas Real World Enterprise Security Exploit Prevention Test March 2015

Discussion in 'other anti-malware software' started by FleischmannTV, Apr 24, 2015.

  1. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    • Large number of in-the-wild exploits (300)
    • Diverse set of exploit kits (12)
    • Diverse set of exploits (10 different CVEs)
    • Internet Explorer, Firefox and Chrome exploits used
    • Large number of enterprise endpoint protection systems – 10 products in 12 different configuration
    • Use of in-the-wild in memory malware
    • Test with 0-day sample
    • Minimal delay between exploit acquire and test
    • Manual test and result analysis
    • Sponsored by Kaspersky Lab

    https://www.mrg-effitas.com/mrg-eff...-security-exploit-prevention-test-march-2015/
     
  2. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    20,590
    I am shocked. Kaspersky sponser, Kaspersky the winner.
     
  3. digmor crusher

    digmor crusher Registered Member

    Joined:
    Jul 6, 2012
    Posts:
    1,157
    Location:
    Canada
    Rinse and repeat. This test is kind of useless to me due to the low number of products tested.
     
  4. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    20,590
    Frankly digmor, it's just plain useless period
     
  5. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Sponsored test? Strange Kaspersky is doing very well in all testing organizations, not only they are wasting their money, but it doesn't do any good to their image... They've also conveniently left out all companies with free versions.
     
  6. ropchain

    ropchain Registered Member

    Joined:
    Mar 26, 2015
    Posts:
    335
    Well, I am sure that a company will only commission a test if it's sure that its product will end up being the winner. And if the results are too close to the results of other products, you just improve your mitigation software based on testing results during a review conducted by a so-called 'independent IT security research organisation'. (Hint: HMP.Alert b167 vs b174 ;). No, I am not against improving the mitigation capabilities of certain programs. But improving detection during an 'independent' review is a bit shady.)
     
  7. erikloman

    erikloman Developer

    Joined:
    Jun 4, 2009
    Posts:
    3,152
    Location:
    Hengelo, The Netherlands
    What free enterprise solutions?
     
  8. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    You are right! My bad, I didn't pay attention to this very important criterion. They have however left out quite a number of companies, I suppose a single sponsor can't pay for everybody...
     
  9. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    @Osaban

    Which endpoint products should have been included in your opinion? I see
    • Symantec
    • F-Secure
    • ESET
    • Sophos
    • McAfee
    • Trend Micro
    • Microsoft
    At least in terms of exploit protection the most relevant competitors are included. Of course consorts like Avira, Bitdefender and some others may be missing, but they wouldn't have scored well in this test. I understand that you can game a test by excluding your strongest competitors, but this was probably not the case here.
     
  10. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,899
    Location:
    localhost
    G-Data, Avira, Ikarus and Bitdefender are normally taken into account in business tests.
     
  11. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    Criticizing the absence of G-Data I can understand because they offer dedicated exploit protection. Avira, Ikarus and Bitdefender however don't. If they had been included, they would have performed worse than Kaspersky. Hence there was no motivation for Kaspersky to exclude them, rather the opposite.
     
  12. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    17,546
    Location:
    The Netherlands
    Well, I have to applaud MRG, finally they have given clear information about how all of these exploits were stopped. Also, real ITW malware was used if I'm correct, so there's nothing "artificial" about this test. So all in all, well done. And there 's nothing surprising about Kaspersky coming out on top, they have added a specialized anti-exploit module in the last years.
     
  13. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    Almost all of Symantec's blocks are attributed to blocking of HTML / Javascript. Could these kind of detections be avoided by using encrypted protocols?
     
  14. ropchain

    ropchain Registered Member

    Joined:
    Mar 26, 2015
    Posts:
    335
    The answer is: It depends
    On a network level you would need to perform SSL interception in order to inspect the network traffic.
    But I suspect that most modern AV suites are able to deal with this issue by using other methods.
     
  15. Cutting_Edgetech

    Cutting_Edgetech Registered Member

    Joined:
    Mar 30, 2006
    Posts:
    5,694
    Location:
    USA
    It's great to see an exploit test for AV's that finally tell how the exploit was blocked. I like the way MRG did this test. I wish Eset's exploit blocker would have been tested independently to see how it can do by itself. I would like to say thank you to Kafeine, and Ukatemi Technologies for providing the exploits for the test!
     
  16. One observation
    And the winner is . . . .(according the distribution of vulnerabilities graph on page 14): Chrome

    One question

    ... Did Kapersky had a hand in selecting which vulnabilities were included in the test?

    One inconsistancy
    In the surfright sponsored real world anti-exploit prevention with artificial tests produced by the sponsor (sorry this contradiction keeps being hilarious to me) MRG had no problem including competitors which did not have special anti-exploit mechanisms. Now they exclude some competitors because they are not designed to protect against exploits? Where is BD for example or Avast (remember Avast scored well in the also questionable real life exploit test of the chinese testing institute PCLS, commisioned by MBAE)?

    One conclusion
    The business model of sponsored tests is to provide "independent" evidence for the benefits and features of the sponsor's product.
    Since a sponsor does not want to spend money on a report which a competitor can use for marketing, we should not be surprised to see the sponsor ending top position.

    For a security forum these test are a blessing: otherwise we did not have something to comment and complain about. I enjoy reading the paparazzi tests from Russia (MRG) and China (PCLS) with love.
     
    Last edited by a moderator: Apr 27, 2015
  17. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Yes, Chrome and Firefox are tied :)
     
  18. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
  19. Yep saw that coming through the room hitting me in the face. But that is what makes Wilders fun.
    In Dutch we say: when you like to pull legs, you should not complain when someone returns the favor.
    :thumb:
     
  20. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    We have similar saying in Slovenia :thumb:
    I wonder why they never use some non-in-the-wild exploits for Chrome to test those tools? Would Chrome block successful execution using built-in sandbox and other mitigations and tests couldn't be properly run?
     
  21. Due to the bug bounty program there are no known in the wild exploits for nearly five years now for Chrome.
     
  22. ropchain

    ropchain Registered Member

    Joined:
    Mar 26, 2015
    Posts:
    335
    Not entirely true imho.
    It takes additional research to create a Chrome zero-day. (vuln in rendering engine + vuln in sandbox for example)

    But I think that this attack has a high potential of being able to serve like a Chrome exploit (Flash vuln + kernel vuln), but that's just a theory without any supporting evidence.
     
  23. ropchain

    ropchain Registered Member

    Joined:
    Mar 26, 2015
    Posts:
    335
    Well, it takes considerable effort to prepare a chrome exploit (even if it's using one-days) and that might not be worth the effort for MRG Effitas. Furthermore, I don't think that many researchers are willing to spend a week on writing a one-day Chrome exploit just to test mitigation software without any additional reward for that effort even though using a browser that does not employ a sandbox would also be sufficient enough.

    In the end it always comes down to effort vs. $$$.
     
    Last edited: Apr 27, 2015
  24. When it is so easy as you say, there would be heaps of nerds partying with their easy earned Chrome bug bounty. Mr JungHoon Lee earned € 225.000 at day two at last Pwn2Own, but the total paid on bug bounties was $557,500 USD So it takes far more additional research as you are suggesting.

    Besides the succesfull exploits demonstrated at the world tournament of white hat hackers Pwn@own, I have not seen a Chrome exploit in the wild, which was released earlier as the patch provided by Google.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.