Windows Defender Is Becoming the Powerful Antivirus That Windows 10 Needs

Discussion in 'other anti-virus software' started by Secondmineboy, Jan 30, 2016.

  1. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Endgame employs a local ML engine. Therefore it doesn't have to go to the cloud for additional scanning:
    On the other hand, WD's initial ML scan is done by its cloud servers.

    In the Bad Rabbit incident, both initially detected at the ML scan stage. The difference is that Endgame's ML engine appears to be more "sensitive;" enough to classify it positively as malware at this detection stage.

    Bottom line - it doesn't matter in the least if AI scanning is deployed but the algorithms employed aren't adequate. Also, the solution is not as Microsoft's recommended; lowering the sensitivity threshold thereby dramatically increasing false positive incidents. The solution is to improve WD's ML algorithms.
     
  2. stapp

    stapp Global Moderator

    Joined:
    Jan 12, 2006
    Posts:
    23,936
    Location:
    UK
    https://www.ghacks.net/2017/12/18/microsoft-changes-windows-defender-path-on-windows-10/
     
  3. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Another AV not SRP - friendly. I hope that's just another SNAFU from MS.
     
  4. paulderdash

    paulderdash Registered Member

    Joined:
    Dec 27, 2013
    Posts:
    4,639
    Location:
    Under a bushel ...
    That update doesn't seem to have reached my machine yet. But yes, I suppose that will need an exception in (SRP) AppGuard ...
     
  5. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Note the following if your using security software that monitors driver loading:
    What is a bit strange is I see no WD drivers loaded when using Winobj to verify loaded drivers. Since I only use WD for periodic scanning, perhaps drivers are loaded at boot time and then unloaded when WD realtime scanning is not enabled.
     
  6. plat1098

    plat1098 Guest

    Likewise, Antimalware Service Executable moved from Windows Processes to Background Process. Possibly the reason?
     
  7. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    17,546
    Location:
    The Netherlands
    Exactly, and this is what I'm looking for, I believe Invincea is also capable of this. There is nothing really impressive about using the cloud to scan files. However, Win Defender did classify it as suspicious with it's pre-execution behavioral monitoring component, but it needed the cloud to come up with a verdict, to avoid false positives. And post-execution protection is also important, so you still need a HIPS/behavior blocker for blocking malware that somehow manage to bypass AV/Cloud AV.
     
  8. Tarantula

    Tarantula Guest

  9. guest

    guest Guest

  10. TairikuOkami

    TairikuOkami Registered Member

    Joined:
    Oct 10, 2005
    Posts:
    3,418
    Location:
    Slovakia
    I remove "%ProgramData%\Microsoft\Windows Defender" to save ~500MB, because WD's definitions download there, time to stop. But I guess, this update applies to Insider only? :doubt:
     
  11. guest

    guest Guest

    Nope, im not Insider and i got it.
     
  12. TairikuOkami

    TairikuOkami Registered Member

    Joined:
    Oct 10, 2005
    Posts:
    3,418
    Location:
    Slovakia
    Interesting, MS providing Insider updates to RTM users. I wonder, if it is possible to download KB4052623 manually? WD says, it is updated, I still have the version 16299.15.
     
  13. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    About the two quoted posts above.
    I noticed that Microsoft reached out to Carlos Perez even before he posted his blogs on the subject.
    This feature will surely be constantly tightened more and more.
    In just the short time alone since those blog post was posted, the entire Windows Defender antimalware platform has been updated once, the Windows Defender antimalware engine has been updated twice and since the machine learning model that is responsible for detecting obfuscated scripts are retrained many times each week in cloud and then updated on your local client, then that machine learning model has been updated 30-40-50 times since time of blog post.
    And all will continue to be updated and retrained.
    Quote are from here.

    As for this part, I would definitely not call it a mixed bag.
    A feature that in its first implementation reduces the tools an attacker has available in those areas by two-thirds. That is a huge plus.
    And as the platform gets constantly updated, also here will the attackers toolbox be shrinking and shrinking.

    Windows Defender Exploit Guard brought all of these HIPS features to the protection stack on Windows 10.
    Sure - Microsoft could have gone all out and just implemented "Block the world"-HIPS rules.
    That would have made all the 12-years old screaming of joy when they do their YouTube-"security testing" where they download some "malware"-collection that are actually 50% clean files just renamed to Super-mega-über-dangerous-rootkit.exe and then click-click-click through the list of files while screaming OMG OMG OMG into the microphone throughout their security "evaluation" videos. :rolleyes:
    But every grown up competent researcher, every enterprise user and every home user in the world would have hated it - because in the real world you also have to actually be productive.

    Instead Microsoft did the intelligent thing - implement HIPS without exposing rule making and prompts answering to end users.
    Then gradually tightening rules and their reach as upgrades continues to roll out.
    That's the way to do this in 2017, soon to be 2018, if you want to make HIPS usable to a broad user base.

    Anyone that has used a classic HIPS will agree that, while powerful, you end up spending more time on rules then getting anything else done.
    Instead with Attack Surface Reduction rules, you pick the areas you want to restrict and let the feature handle rules. Both at initial setup and also every time the feature are updated with stricter rules and further reach.
    I am certain that every end user out there prefers this instead of manual rule tinkering.
     
  14. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Quote above about the browser implemented SmartScreen are not entirely correct.
    Local cache file look-up and then cloud look-up if not in cache file and SmartScreens logic finds connection suspicious.
    Link : https://blogs.windows.com/msedgedev/2015/12/16/smartscreen-drive-by-improvements/

    But this quote above about the system-wide SmartScreen are correct.

    As for your comments on certificates.
    They have huge plusses, but also pitfalls. The hammer needs to drop quickly upon sign of misdoing.

    Also, as Microsoft said at Microsoft Ignite 2017 - Windows Defender for example doesn't allow anything due to digital signatures exactly due to the fact that if you suffer a breach then in a heartbeat your certificate can become your enemy.
    For the same reason they said not to build whitelists.
    Let WD learn your workflow instead.

    When combining the many layers in the native security, you get the best of both worlds. :thumb:
     
  15. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Actually you misread the blog post.
    SmartScreen and therefore the new Network Protection in Windows Defender Exploit Guard that expands SmartScreen to all outbound connections from all processes, are 100% capable of dealing with HTTPS also.
    The reference to "HTTP, TCP and IP" in blog post, are referring to network layers. Application layer, Transport layer, Network layer.
    SmartScreen has been handling HTTPS perfectly for 10 years now and the new Network Protection handles HTTPS just as perfect.
    Scroll down a little further in the same blog post you just read, and read again.
    Or read the documentation on Microsoft Docs.
    Also, since SmartScreen has been with us for 10 years now, every user ought to know that it handles HTTPS perfectly fine. :)
     
  16. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    What I was saying in this post was, that False Positives aren't just False Positives. You need to count in impact.
    Block Outlook.com and you impact many. Block Willy's webpage with pictures of his ant farm, and impact are small.

    But we agree that it's better to err on the side of caution.
    Having a site/download occasionally blocked for 10-20 minutes until FP corrected, are better then dealing with consequences of phishing or dubious files locally.

    Also since looking at FP numbers has become the new black, one should also look at what is behind those numbers.
    AV-Comparatives makes some great reports, that are a lot more transparent then those from many other test labs.

    False Positives are, when faced, categorized by AV-Comparatives into 4 categories as listed below :
    "Very low" = fewer than a hundred users = range from 1-99 users.
    "Low" = several hundreds of users = range from 100-999 users.
    "Medium" = several thousands of users = range from 1000-9999 users.
    "High" = several tens of thousands of users = range from 10.000 to 99999 users.

    False Positives aren't just False Positives when you add in impact.
    In order to be fair against every vendor, lets just use the medians of each category.
    "Very Low"~50, "Low"~500, "Medium"~5000 and "High"~50000.
    If we look at four random vendors and AV-Comparatives findings of FP from each of these vendors and do the math, it looks like this :
    (I will make the third-party vendors anonymous to not start a Wilders war - people can look in report themselves and make the link between numbers below and vendor name :))

    E1-anonymous : 10 Very-Low, 2 Low, 4 Medium, 1 High = 10*50+2*500+4*5000+1*50000 = 71.500 users affected.

    E2-anonymous : 1 Very-Low, 0 Low, 0 Medium, 1 High = 1*50+0*500+0*5000+1*50000 = 50.050 users affected.

    F-anonymous : 104 Very Low, 26 Low, 2 Medium, 0 High = 104*50+26*500+2*5000+0*50000 = 28.200 affected users.

    Microsoft : 47 Very Low, 1 Low, 0 Medium, 0 High = 47*50+1*500+0*5000+0*50000 = 2.850 affected users.

    So Microsoft with 48 FPs actually only affected ~2.850 users.
    And F-anonymous, the vendor with the most FPs - 132 of them, affected ~28.200 users.

    BUT - the E2-anonymous vendor with only 2 FPs actually affected ~50.050 users, and E1-anonymous with 17 FPs affected ~71.500 users !!

    These numbers of impact are much more important in my opinion.
     
  17. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    You misunderstood the Microsoft blog post.
    The new Microsoft blog post are about the fourth layer in Windows Defender.

    Windows Defender's layered machine learning models.png

    Microsofts data shows that Windows Defender users face around 90 billion potentially malicious encounters per day !! Every single day. Every week. Year in, year out.

    Microsofts data also shows that Windows Defender handles 97 % of everything, using only layer one.
    Layer one are strictly the local Windows Defender client on your pc with its local machine learning models, behavior based detection algorithms, generics and heuristics. The entire Windows Defender Behavioral Analysis engine.
    And a lot more, that are all included in the local client.
    97% using only local capabilities with instant/milliseconds protection.

    For the remaining 3% Windows Defender will move to layer two and query its cloud backend with metadata on suspicious files. Cloud will process these data with machine learning rules and return a verdict in milliseconds.

    A tiny percentage of suspicious encounters are still inconclusive, and Windows Defender moves to layer three. Here the suspicious file are uploaded and analyzed in cloud. Suspicious files are still locked and blocked on your local pc all the way through layer one, two and three.
    Now with layer three, you have verdict back in seconds.

    We know from the Microsoft whitepaper, that "only" 4.5 million files needs to be uploaded each day. Meaning the number of queries that has made it to layer three.
    Now if we do the math on those numbers :
    90 billion encounters every day and 97% of that handled strictly by local client on your pc.
    3% = 2.7 billion encounters makes it to layer two.
    4.5 million files = 4.5 million / 90 billion * 100 = 0.005% makes it to layer three.
    Or put another way - 99.995 % of everything are handled on layer one and two.

    There are not any public numbers on layer four in Windows Defender's layered detection model.
    But when 99.995% are handled at layer one and two, only 0.005% needs to move to layer three - then it must be a lot less then 0.005% that then needs to move to layer four.

    And as this new blog post shows, even for this tiny portion - a lot less than 0.005% - of encounters that needs to move to layer four - than still there was a verdict ready to block worldwide in just 14 minutes !

    If a person are not impressed by that, then I guess that person must spend a lot of time in some parallel universe where research are light years ahead of what we have here on poor earth. :thumbd:

    Read all of these blog posts and the white paper to understand all the layers of Windows Defender :
    Blog post - Antivirus evolved
    Blog post - Windows Defender Antivirus cloud protection service: Advanced real-time defense against never-before-seen malware
    Blog post - Detonating a bad rabbit: Windows Defender Antivirus and layered machine learning defenses
    White paper - The Evolution of Malware Prevention (register to download)
     
  18. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    No.
    The local Windows Defender client on your pc has machine learning models, behavior based detection algorithms, generics and heuristics.
    It has the entire Windows Defender Behavioral Analysis engine including memory scanner locally on client on pc.
    And a lot more as mentioned earlier in thread, that are all included in the local client. On your pc. Fully functional. Without cloud.

    You need to read the three Microsoft blog posts and the white paper linked to in my reply to Rasheed.

    I know you are often confusing the third layer in Windows Defender as being being the only Machine Learning engine. It is not.
    There are Machine Learning engines on all five layers in Windows Defender, locally in client on pc and on all cloud layers, all of them running different algorithms targeting different areas.
    Not only that, there are many, many different ML models on every layer.
    As you can read in the white paper, then on layer two - which is where Windows Defender start using its cloud - on that layer alone there are 167 models running. And that was as of early summer 2017. Even more progress has been made since then.

    I also know that you on occasion has posted that the Behavioral Analysis engine in WD are cloud based, and then posted a link to Block-at-First-Sight.
    But that is not correct.
    Behavioral Analysis engine and memory scanner are part of the local client on your pc, tracking behaviors from file-based as well as file-less attacks locally on pc.
    The Block-at-First-Sight feature are something completely different.
    It is a combination of all the layers that gives Windows Defender these capabilities. But the part of Block-at-First-Sight that you reference, are layer three - the stage where a file are uploaded.
    In order for anything to get to that, it has already been through Machine Learning models and Behavioral Analysis on local pc, then through +167 models in first cloud stage (layer two in the Windows Defender funnel), and THEN the tiny percentage that are still suspicious are uploaded on layer three.

    Anyway, just read the four links I posted in my reply to Rasheed. :)
     
  19. Brocke

    Brocke Registered Member

    Joined:
    Mar 16, 2008
    Posts:
    2,306
    Location:
    USA,IA
    Has anyone documented the steps on setting wd other than what MS has given? Like a step by step how to sort of.
     
  20. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    The update to the Windows Defender antimalware platform, are not insider only.
    It is for every 1703 and 1709 installation. :)
     
  21. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Oh, sweet joy. Once again Microsoft brings more goodness to Windows 10 Pro users.

    As Dave Weston tweets :
    https://mobile.twitter.com/dwizzzleMSFT/status/943356105165598721

    Amazing. This will bring security to a whole new level. :thumb::thumb:
     
  22. WildByDesign

    WildByDesign Registered Member

    Joined:
    Sep 24, 2013
    Posts:
    2,587
    Location:
    Toronto, Canada
    @Martin_C Great stuff there regarding Windows Defender Application Guard for Pro users. Lightweight virtualization for browsers (Edge, so far) is an interesting direction. There was some talk that it may eventually open up to other browsers as well which would be good at some point in the future as well.

    I may have to fire up a VM for this new Insider build, although I don't recall if the virtualization for WDAG can be used within a VM. I'll have to test and find out. I believe it uses nested virtualization for VM in VM essentially.
     
  23. OverDivine

    OverDivine Registered Member

    Joined:
    Jan 16, 2009
    Posts:
    24
    the only downside of windows defender application guard is that i cannot use an adblocker. it uses 1gb+ ram btw but i don't mind
     
  24. Azure Phoenix

    Azure Phoenix Registered Member

    Joined:
    Nov 22, 2014
    Posts:
    1,556
    Are you referring to the adblocker extensions? If so, do you know if Adguard for Windows would work better?
     
  25. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Appears MS had updated the diagram shown. When I initially viewed it specifically showed that HTTPS traffic was not being monitored; only HTTP traffic.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.