Comparative Malware Protection Assessment

Discussion in 'other anti-virus software' started by itman, Apr 10, 2018.

  1. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    6,115
    Location:
    U.S.A.
    Of note is no endpoint vendor scored 100% in this test. Also the AI solutions were the lowest scorers. Cylance chose not to participate in the test.:rolleyes:

    https://www.mrg-effitas.com/wp-content/uploads/2018/04/MRG_Comparative_2018_February_report.pdf
     
  2. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    8,493
    Location:
    Slovenia, EU
    Good results from Sophos :thumb:
    Microsoft with Defender ATP also didn't want to participate. Too bad.
     
  3. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    10,221
    Location:
    The Netherlands
    Holy crap, SentinelOne and CrowdStrike performed very badly! I didn't expect this from "next gen" companies. And Symantec's behavior blocker seems to be pretty good. I also wonder why Cylance and M$ didn't want to participate, what are they so scared of? It would have been a nice chance to show of Win Def ATP. But very interesting test, thanks for posting itman. :thumb:
     
  4. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    6,115
    Location:
    U.S.A.
    A-V Comparatives informed me of a similar test they performed last fall. It was a commissioned test sponsored by Bitdefender: https://weblog.av-comparatives.org/advanced-endpoint-protection-test/ . In this test both Cylance and CloudStrike performed well. As such, one can only assume that testing methodology does play a factor in test results.
     
  5. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    10,221
    Location:
    The Netherlands
    Thanks, will do a bit of reading, and I honestly don't know. I mean, MRG's way of testing seems to be pretty straight forward, I doubt that bad results are because of MRG's testing methodology.
     
  6. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    6,115
    Location:
    U.S.A.
    By "methodology," I also was referring to malware samples used and specific tests performed. For example, the A-VC test used PowerShell scripts to test fileless malware. Cylance aced this test. The reason why? It employs a script blocker.o_O So as far as its behavior detection in this area goes, its capability is unknown.
     
  7. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    10,221
    Location:
    The Netherlands
    Yes, that's why Cylance was not included in the table. But very interesting results! SentinelOne and CrowdStrike have got some serious work to do. Same goes for Carbon Black. But at least they did have the guts to participate in the MRG test.

    I hope that Barkly, enSilo and Invincea will be included in future testing. And I still wonder why M$ didn't participate, I believe that based on what I've read, Win Def ATP is a pretty good product, but I hope it's not all talk, see link.

    https://cloudblogs.microsoft.com/mi...virus-is-the-most-deployed-in-the-enterprise/
     
  8. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,051
    Location:
    The land of no identity :D
    Meaning: "Hey, umm....we found that our URL blocker didn't work in this scenario, never mind that we do not really have an advanced scanner/behaviour analysis system. We fired most of our virus researchers years ago, now we have a good on-execution blocker only and nevermind that our product is useless if you do not give it access to our wide, wide servers - you should not be doing this, Symantec is good."

    For years, I have been stating that it's time this company bit the dust. That they have any market or mindshare after all the things they did in the past ten years is a miracle of sorts.
     
  9. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    10,221
    Location:
    The Netherlands
    LOL, good point. BTW, I totally forgot to comment on Sophos, seems like they performed poorly in the AV-Comparatives "PowerShell-based" exploits test. I hope Mark and/or Erik Loman can explain this. Seems like they were able to block exploits on Firefox, but not when Meterpreter is launched via non-browser exploits.
     
  10. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    6,115
    Location:
    U.S.A.
    For reference to others, you are referring to the link posted in reply #4.

    The test was not exclusively for Powershell based attacks but also included WMI, PSExec, Task Scheduler, EternalBlue, script and other method ATP based attacks. Probably the most extensive test in this area I have seen to date. Perhaps NVT should submit OSArmor to see how it performs against these tests.
     
  11. Azure Phoenix

    Azure Phoenix Registered Member

    Joined:
    Nov 22, 2014
    Posts:
    599
    It's probably for the best to wait until the latest version is officially released before submitting it.
     
Loading...