AV-Comparatives: Real-World Protection Test February-June 2018

My sincere hope in RS5 is that Microsoft has finally "detached" WDEG ASR from Windows Defender. So when WD is bypassed by malware as has been pointed out is fairly trivial to do, WDEG ASR is still functional.

 

In regards to the ad hoc test on malwaretips.com of Win 10 native SmartScreen w/WD disabled, the same would apply when using a third party AV since WD will be disabled by default.

For reference if one monitors SmartScreen network activity, it does indeed go to the Microsoft cloud for unknown process execution validation. One might infer that it is doing the equal of WD's "block at first sight" cloud AI scanning. It is not. The SmartScreen connection is to scan the process against the constantly updated SmartScreen white/black lists on Microsoft's servers. The local resident like copies are only refresh on a periodic basis.

On the other hand if WD's periodic scanning is enabled, WD will submit any unknown status file found when the scan is run to the AI cloud for a quick scan via block at first sight detection.

The problem is:

1. If the unknown file is malicious, previously executed, and not detected by your third party AV prior to the periodic scan, you're already infected.
2. Periodic scanning by default only performs a quick scan. So if the unknown malware is not resident in an area scanned, it won't be submitted to the AI cloud for scanning.

-EDIT- The above comments apply to the instance where you have SmartScreen set to "Warn" and you allowed the download.

 

thank you

 

I just got done doing some testing in regards to SmartScreen on Win 10 1803 in regards to both IE11 and Edge. For reference, specifically the unknown and malicious download tests using the SmartScreen validator here: https://demo.smartscreen.msft.net/.

To begin with, both Edge and IE11 use the same blacklist and whitelist reference files; a periodically updated large one and a smaller cache one that is updated when the browser is opened. As such, you would expect that both browsers would perform the same way when detecting unknown and malicious download files. They do not.

Of note is the Microsoft test web site verbiage states that the tests for unknown and malicious download files applies to execution only. So it appears Microsoft has covered "their butt" on that regard. When using IE11, both file categories will throw a warning if a "Save" selection is attempted. Such is not the case for Edge; the files are saved without an alert being generated. Both file types will throw an alert upon file execution since that is being monitored by Win 10 native SmartScreen processing.

From the above, it can be concluded that IE11 with its SmartScreen option enabled is the only browser that can block downloads for these two file types. Why Edge's SmartScreen enabled processing does not work the same way is a mystery. One possible explanation is Edge is designed to interface directly with Windows Defender. Someone using WD can perform these tests and determine if WD will block the saving of the downloads; I am skeptical that it will.

The above lack of Edge's download detection has major malware implications. Again, native SmartScreen only monitors files with "the mark of the web" associated with them. As previously noted and confirmed by another poster, this can be "stripped" from a file download. If this was to occur for an unknown 0-day malware download that was subsequently run, you are now 100% dependent upon your AV solution detecting it which is doubtful. Also doubtful is if WD's "block at first sight" cloud scanning would be done since that is triggered by native SmartScreen file unknown status.

Why AV labs don't test for such things as this is also a "mystery."

 

Is that the Zone.Identifier ADS?

 

For those who want to get the most out of Smartscreen, Andy Ful's Hard_Configurator will install for you a right-click option, to force checking by Smartscreen. It won't work for script files, but it will force Smartscreen to check an .exe file that it would otherwise ignore.

 

Yes.

 

On this regard, as anyone done any testing on its impact or not on WD's "block at first sight" cloud scanning?

 

Any Ful says on the other forum, in response to your question:
"What are they interested in?
The right-click option in Explorer to force checking by SmartScreen ('Run As SmartScreen' or 'Run By SmartScreen') depends only on SmartScreen settings. If SmartScreen for applications is turned on then this feature is fully functional (even when Defender is completely turned off).
SmartScreen is independent of the "block at first sight" feature."

 

So... all I need to do is to restore a clean image, right? What's the problem with that concept?

 

Nothing.

 

As far as Andy Ful's Hard_Configurator, it appears to correct many of Win 10's default native SmartScreen issues.

My issues with it are:

1. It is dependent upon SRP as many of these third part solutions. Effective with Win 10 1803, SRP has been deprecated. Which means it is no longer actively supported and its existing features may no longer function properly in future Win 10 upgrades:

https://docs.microsoft.com/en-us/windows/deployment/planning/windows-10-1803-removed-features

2. It is a GitHub product that must be installed. I for one "draw the line" when it comes to using like open source software that can affect Win OS built-in security protections.

 

What Microsoft needs to do is fix Win 10 native SmartScreen to work as a fully functional reputational scanner applicable to executable file types.

This can be done as follows:

1. The WD engine is loaded at boot time similar that currently done if periodic scanning is enabled.
2. "Block at first sight" cloud scanning is done on any unknown and previously un-scanned executable status file regardless of its origin.
3. The above processing is done regardless of if WD is set to realtime mode. This is doable since the "block at first sight" processing exists as a front-end to the main WD scan engine. This front-end of course is triggered by native SmartScreen existing processing. If a third-party AV solution is employed, SmartScreen just throws one of three alerts; unknown, suspicious, or malicious.

Problem solved. Well, not really. "Hell will freeze over" before Microsoft will do this.

 

If Microsoft ever does that, just imagine the absurd amount of cry and lawsuits from third party security vendors.

 

Continuing my comments and directed at Microsoft, I see a profit opportunity for you in regards to the above "enhanced" SmartScreen processing. I for one would be willing to pay a reasonable monthly subscription for such capability as long as it was in line with what was tolerable for a retail user. Given the millions of Win 10 licenses out there, this should more than offset the exponential load increase on your Azure AI network.

I see this being offered in a both a base and enhanced version. The base low cost version would be similar to that in existence in Windows Defender; a quick analysis by the AI servers. For the enhanced version/s, well the "sky's the limit on those;" possibly to the degree present in WD's ATP protection.

Other possibilities are offering volume licensing deals to AV vendors and provide an interface into SmartScreen that would allow then incorporate the verdict determination into their existing products.

Bottom line - unbundle your excellent cloud scanning AI capability from your not so good Windows Defender solution.

 

Just the opposite; especially for the base version for what I described above. For starters, it would get them "off the hook" in regards to user decision interaction since it is SmartScreen provoking that activity, not their product. In reality, it is no different from existing SmartScreen detection and required user interaction. That is "block" or "warn" based on individual app settings.

 

1 Hard_Configurator is installable for convenience's sake, but you can really just plop the folder on the proper path and it will work.
2 I don't think there are any other GUI solutions that work with modern versions of Windows and are based on default/deny SRP. So you don't have a real lot of options besides rolling up your sleeves and configuring SRP yourself in registry or GPO.
3 SRP is still used by a lot of businesses, so I don't think Microsoft will kill it so fast. It should be good for the foreseeable future, despite the fact that it is not being actively developed.

 

I was doing some PC house cleaning today and came across one of my prior Win 10 native SmartScreen bypasses I developed. It's very trivial and still works on ver. 1803 so I am posting here for anyone who wishes to duplicate it.

To begin with we are using the unknown and malicious test files from the SmartScreen test site referenced previously: https://demo.smartscreen.msft.net/ . I recommend using Edge for this test since it has been previously noted the test files download unimpeded by SmartScreen.

1. Click on the above link which will take you to the SmartScreen demo test site.

2. Perform either the unknown or malicious, or both if you prefer, download tests. If you performed the unknown test, a file named freevideo.exe should be present in your download folder. If you performed the malicious test, a file named knownmalicious.exe should be present in your download folder. For this bypass POC, I will be using the knownmalicious.exe file.

3. Copy/move the knownmalicious.exe in your download folder to C:\Users\XXXXX\AppData\Local\Temp folder where XXXXX is your logon id.

4. Double click on the knownmalicious.exe in the Temp folder. The SmartScreen alert screen will pop up noting a malicious file has been blocked.

Now that we have verified that SmartScreen normally detects execution of our test file, we will write our bypass code.

5. Open Notepad and copy the following script code to the open file. Obviously, change XXXXX to your logon id.

START "" C:\Users\XXXX\AppData\Local\Temp\knownmalicious.exe
​

6. Save the file as SSbypass.bat to where ever you want.

7. Double click on the .bat file to run it.​

"Not a peep" from SmartScreen. I'll let you figure out why this works. For those using Andy Ful's Hard_Configurator version, please do check if the bypass works with it.

Bypasses like this are primary reason why SmartScreen validation needs to be done upon file download and not just execution.

 

I see you have done a lot of writing about SmartScreen, but to me it's very simple. Every time that SS warns about "unrecognized app", it should be marked as a fail, no matter if it's malware or non-malicious. If it's blocked with the "block on first sight" feature then it should be tested for false positives.

https://www.howtogeek.com/320711/what-is-smartscreen-and-why-is-it-running-on-my-pc/

 

Lucky for WD users, Microsoft does rely on win 10 native SmartScreen detection to trigger WD "block at first sight" processing. This is illustrated by going to this test site: https://demo.wd.microsoft.com/Page/CloudBlock , reading what is posted there, and downloading the test file. Interesting, Eset blocks their download as "a variant of Generik.KDIQJCH trojan." Appears WD now works as many other AV's in that prior scanned files are "marked" as such and unscanned files will be submitted to the cloud after local heuristic analysis is performed.

A similar WD test can be performed by going to the AMTSO desktop features web site and performing the Cloudcar test.

Also WD's high incident on user interaction decision required is not the same as a false positive detection. The later is a positive verdict rendering requiring no user interaction. My take on the user interaction issue is that WD's default cloud scanning duration settings are to brief to allow the AI engine to detect with high confidence that malware activity exists.

 

TWO (2) pages for WD ...….

The WD thread is not here but there =
https://www.wilderssecurity.com/thr...erful-antivirus-that-windows-10-needs.383448/

 

even the thread's title is insulting

 

Here's the solution. We need to create a thread "Windows Defender Is Not the Powerful Antivirus That Windows 10 Needs."

 

or just rename that thread to "Windows Defender" and maybe newbies won't be pulled into a dream world where they think they're actually protected.

 

I sure would love for them to include Cylance Home next time I would be very curious how it stacks up.