MRG Effitas Online Banking Browser Security Certification for Q1 2017

Discussion in 'other anti-virus software' started by Triple Helix, May 19, 2017 at 12:22 PM.

  1. Triple Helix

    Triple Helix Webroot Product Advisor

    Joined:
    Nov 20, 2004
    Posts:
    12,245
    Location:
    Ontario, Canada
    1. https://www.mrg-effitas.com/wp-content/uploads/2017/05/MRG-Effitas-Online-Banking-Certification_Q1_2017_Level_1_wm.pdf

    2. https://www.mrg-effitas.com/wp-content/uploads/2017/05/MRG-Effitas-Online-Banking-Certification_Q1_2017_Level_2_wm.pdf

    TH
     
  2. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    What is the difference from lvl 1 to 2 for webroot did they update the software to not to fail?
     
  3. Triple Helix

    Triple Helix Webroot Product Advisor

    Joined:
    Nov 20, 2004
    Posts:
    12,245
    Location:
    Ontario, Canada
    Read page 3 from the #2 report.
     
  4. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    I know but it doesn't explain what changed and why webroot improved it's score
    It's a methodology change, an error, a software update...?
     
    Last edited: May 19, 2017 at 3:00 PM
  5. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,910
    Location:
    The Netherlands
    LOL, here we go again. WD failed to spot 9 samples of the financial malware, but MRG probably disabled Win SmartScreen, so it's unfair to Win Defender. That's what all of the fanboys will say. Good to see that HMPA blocked both banking trojans in the botnet test.
     
  6. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    Do you know that the malware that can be detected by SS is basically already in WD signatures?
     
  7. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,910
    Location:
    The Netherlands
    Wait, say what? Where did you read this? So Win SS will in fact never improve WD's detection rate? It just keeps getting funnier, if this is true of course.
     
  8. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    It still can protect you from phishing sites but basically everything related with malware should be almost the same. If a site is blocked because it contains malware is because Microsoft know that malware and is already in their signatures.

    As far as I have been told Microsoft have now everything centralize in an anti-malware cloud.
     
  9. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,910
    Location:
    The Netherlands
    OK so all of this discussion was basically useless? LOL, I wonder what excuse they will come up with next. To be honest, if I would have to criticize one thing about MRG testing, it's that they consider it a pass if security tools manage to block a malicious URL. I'm not interested in that, I would only like to know if they can block malware both pre and post execution, not if they can stop users from downloading malware.
     
  10. Nightwalker

    Nightwalker Registered Member

    Joined:
    Nov 7, 2008
    Posts:
    876
    He is just confusing Cloud protection (Block at first sight/reactionary response with machine learning) with SmartScreen (application reputation/whitelist).

    http://info.microsoft.com/rs/157-GQE-382/images/Windows Defender ML Whitepaper.docx

    More info here:
    https://docs.microsoft.com/en-us/wi...ns-in-windows-10#windows-defender-smartscreen
     
    Last edited: May 19, 2017 at 5:06 PM
  11. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    4,012
    Location:
    U.S.A.
    No comment. My OBP - 4/2016 replies sum up what I feel about these MRG tests.
     
  12. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    I am not confusing anything.
    How long do you think that does it takes if SS detects a suspicious file that has been checked or uploaded to their the cloud and found that it was malware until WD is able to detect it?
    Minutes, not even a second according with MS data? What is going to impact in a test? Almost non existent

    If they haven't merge SS in WD is because is a good way to feed their cloud with files from people using other AVs

    All the files go to the same pot
     
    Last edited: May 19, 2017 at 5:47 PM
  13. Nightwalker

    Nightwalker Registered Member

    Joined:
    Nov 7, 2008
    Posts:
    876
    Ofcourse you are, SS is a whitelist approach method, if a file lacks a safe reputation it will block execution entirely, while cloud detection do the opposite (block if it is a already know bad file), please refer to Microsoft whitepaper and the link that I posted above.

    It is obvious that it will impact in tests and real world protection, Whitelist done right is very powerful, it is default-deny. (yes, SmartScreen isnt perfect, far from it)

    Do you know about Avast! hardened mode? It is a similar approach.
     
  14. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    From where do you think SS takes the reputational info of a file?
     
  15. Nightwalker

    Nightwalker Registered Member

    Joined:
    Nov 7, 2008
    Posts:
    876
    I will quote Microsoft about this:

     
  16. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    Thanks, I know I am right

    . If the app lacks a reputation or is known to be malicious, SmartScreen warns the user or blocks execution entirely,

    Is known to be malicious means that it's already detected by WD/Cloud engine so SS is redundant.
    If it lacks of reputation means that is using a subset of rules from the ML/AI engine to determine the reputation, and reputation doesn't mean that is an unknown file. The number of cases where a file can have a bad reputation and the entire AI determines that the file is good so only SS will protect you will be minimal.

    Regarding URL if ms knows that a site is bad or has a low reputation is because it has already all the files of that site in their cloud and found them unsafe, so WD will detect it anyway.
     
  17. Nightwalker

    Nightwalker Registered Member

    Joined:
    Nov 7, 2008
    Posts:
    876
    You ignored the whitelist approach:

    "
    • Checking downloaded files against a list of reported malicious software sites and programs known to be unsafe. If it finds a match, SmartScreen shows a warning to let the user know that the site might be malicious.

    • Checking downloaded files against a list of files that are well known and downloaded by many Windows users. If the file isn't on that list, SmartScreen shows a warning, advising caution."

    I think you are regarding Windows Defender very high thinking that SmartScreen whitelist + aplication reputation will give minimal differences in protection, but thats okay, We can just agree to disagree.

    We are massive offtopic anyway, so I will recommend this tool, it makes SmartScreen much more useful.

    https://github.com/AndyFul/Run-By-Smartscreen
     
    Last edited: May 19, 2017 at 6:27 PM
  18. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,896
    Sometimes you can't teach an old dog new tricks. You may want to help them understand why they are incorrect, but they will dismiss your knowledge as "being a fanboy".

    Meanwhile people that have a clue already know that SmartScreen blocks everything without a reputation, and thus, 99% of malware.
     
  19. Hiltihome

    Hiltihome Registered Member

    Joined:
    Jul 5, 2013
    Posts:
    659
    Location:
    Baden Germany
    Hhm,
    there are so many threads, where fan-boys and MS-haters refer to SS.
    Testing companies disable SS, AV-vendors hate SS...

    I my opinion third party AVs are going down the tube.

    SS is Microsoft's answer to the challenge of cyber criminals, and they will succeed,
    because MS has the biggest cloud and user base.
     
  20. Umbra

    Umbra Registered Member

    Joined:
    Feb 10, 2011
    Posts:
    3,374
    Location:
    Europe then Asia
    Not just SS now, but WD Security Center; it already has WD+SS, coming very soon will be some of EMET mitigations techniques (see insiders builds), i don't even mention WD ATP...
    so WDSC is the new security platform of MS; and as you said, classic 3rd Party Vendors will go down the tube..we won't need them except for some special features.
     
  21. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,372
    That white list is just the reputation.

    How much time it will take to ms to finally classify those files flagged by SS (these will have priority over unknown files) to truly determine that are good or bad? Running them in a sanbox or with an analyst or by other means? Hours? A day?
    How old is the malware used in the testd? Hours, days...
    This is why the differences in real world or in a test are almost non existent.

    If SS is not being used in AVC and MS haven't complain (AVC is in constant communication with vendors) is most probably because they agree with the methodology.
    If there is a group of people that want to think that there is a conspiracy is up to them.

    If WD have improve from 90 lows to 90 highs is because his AV was crap and now is decent and SS has nothing to do with it other than feeding the cloud with files. Probably they prefer this way rather than having tons of FP due to SS.
     
  22. Umbra

    Umbra Registered Member

    Joined:
    Feb 10, 2011
    Posts:
    3,374
    Location:
    Europe then Asia
    who cares if it is even years, i don't execute any files i don't know. Before executing a file you probably have done some research, you don't execute every files you see right?
    cloud is just a confirmation (or not) of what i think about the file.

    real world malwares are rarely 0-days ; they are "old" ones. it is why WD scores well in prevalence tests and badly in 0-days one.

    MS doesn't care of tests in general, they have no financial gains unlike 3rd party ones. They give free basic protection for all.
    you still think about WD as a standalone feature like MSE is, you are plain wrong. We are in win10 CU now so tests must be done accordingly using WDSC not WD.

    that is not the discussion if WD is good or not, it is about if SS should be used in the tests. since we are on Win10 CU the answer is yes.
     
  23. clubhouse1

    clubhouse1 Registered Member

    Joined:
    Sep 26, 2013
    Posts:
    913
    Location:
    UK
    These threads are always a riot of laughs!
     
  24. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,910
    Location:
    The Netherlands
    I believe lordraiden made a good point. Basically it's like this:

    Win Def = blacklisting (signatures and heuristics)
    Win SS = blacklisting + URL Filter + white-listing

    It's very likely that Win Def will be able to detect all malware that Win SS can detect with blacklisting + URL Filter. So this means that Win SS will probably not improve the detection of Win Def. The only advantage that it gives is white-listing. But in my book, white-listing shouldn't be allowed in these kind of tests. Anyone can make a list of 500 popular apps, and all other apps are not allowed to run, no matter if they are malicious or not. I believe Avast also offers this white-listing feature, but I suppose it's not enabled by default.
     
  25. Triple Helix

    Triple Helix Webroot Product Advisor

    Joined:
    Nov 20, 2004
    Posts:
    12,245
    Location:
    Ontario, Canada
    They used v9.0.15.40 which is the current release so that tells me a few minor changes in the Cloud, but what's surprising to me MRG contacted all vendors and Webroot was the only one to deal with the issues MRG presented to them.
     
Loading...