May 2016 Windows 10 tests at av-test.org

Referring to other articles on SmartScreen, I believe the author was just referring to the initially connection it makes to MS servers to validate the download

 

@itman :

No, that is not correct.

As both @Minimalist and @TonyW mentions, SmartScreen has been OS systemwide since Windows 8.
And in IE and Edge.
And in both the Store and in every single app you install and/or run.
And in both Outlook, Outlook.com and in Exchange.
And on both desktop, mobile and Xbox.

Targeting phishing, shady sites, spam, low-rep or unknown files, malware hosts, malwartising and drive-by/zero-days/exploits.

It's completely ridiculous that testing institutions disables all that before testing the native security.

 

SmartScreen doesn't do any MITM.
Part of it are integrated in browser and therefore doesn't need to.

The person that wrote the article on that site you quote from, does not have a clue what he is talking about. Those kind of articles was posted everywhere back in early Win8-days, where tech medias tried to lure people into believing that reputation databases was dangerous.

 

I still disagree. If you are testing Windows Defender, you are testing Windows Defender. And no, a firewall test would not involve Windows Defender. I do understand that Microsoft intends for you to use all of their included defenses, but SmartScreen if not disabled continues to run with any other product but should not be left on for testing. By the same reasoning you could say for example that Norton would have scored higher if SmartScreen was left on. That may be correct, but it does not make Norton a better solution than what it already offered, it didn't do the work. If disabling Windows Defender also disabled SmartScreen as an included component then I would be inclined to agree with you, but it does not therefore I cannot. It's generous of Microsoft to include it for free and I am sure there are people there that work very hard on it, but the first thing I do on a clean install of Windows 10 is disable it with group policy and add a better solution, which in my opinion is a paid one. But even for the sake of argument, or a lack thereof, I'm pretty sure that even with SmartScreen, it would still fall well short of the top tier solutions.

 

@xxJackxx :

What you use or does not use, what you disable through group policies or does not disable or whether you may think something is better or worse then something else are completely irrelevant.

Testing security are not about giving high scores to whatever product you might use.

It ought to be about measuring the level of protection a user have.

The category that testing institutions always point fingers at are "webfiltering" - URL-filtering, zero-day detection.

The same testing institutions always disables SmartScreen.

In case anyone wonders, then take a look at SmartScreen description in link on page 1 of this very thread.
It will tell you - SmartScreen : URL-filtering, zero-day detection, reputation database.

Windows Defender still does very good on its own. Always hovering around 99,8 % on prevalent malware and usually fluctuating between 88-95 % on zero-days in the various tests.

But disabling the module that specifically handles URL-filtering and zero-day detection, and then complain about URL-filtering and zero-day detection ought to be better - that doesn't make sense.

Regardless of whatever product you might use, then surely you can accept that all IT security are modular designed.
Modules from one vendor might be able to coexist with modules from another vendor.

But no matter what vendor you test, then they need to be tested as that vendor designed their solution. With all their modules active.

And considering that all the testing institutions write pages about how mighty advanced their monitoring systems are, then monitoring one more module and adding a single column that lists that reading - that really ought to be within their reach.

 

I do not disagree wit that, the reason I did not name what I use (Norton was an example, I could have just made up a name). Leaving SmartScreen on would only be valid if it were left on for all products, at which point it would give inconsistent results as some of the other products would catch the same things first. If it were marketed as "Windows Defender Internet Security with Windows Firewall, SmartScreen and IE/Edge" (as they are all more effective with Microsoft's own browsers), then your modular point might be something I would agree with. I get what you are saying but I really don't think you are going to get them to change the way they test.

 

@xxJackxx :

No, we can agree that they will probably not change their way of testing, because these test reports are not made to inform the end user but as a tool for marketing.

What we do not agree on, are that the security modules in Windows need to be marketed any different then they already are.

Windows protects itself perfectly fine.
Especially when evaluated as a whole.

What the testing institutions ought to do, are to stop disabling parts of the native security and instead test "Windows 10 x64 with default settings" and call its entry exactly that in the report.

The third-party solutions called "Windows 10 x64 with default settings and product XYZ in default settings".

And then add a column to each entry showing SmartScreen blocks.

A lot of people will be very surprised by how much SmartScreen blocks also if you have third-party solutions installed.

Such reports would show a realistic scenario like those the end user will actually experience.

Now we just need a testing institution that has the courage to be the first that starts doing realistic testing.

 

I wouldn't mind seeing that test, but I think it should be a standalone test without involving any third party vendors. Then the results could be viewed and not debated, regardless of what they were.

 

Why without third-party vendors ??

The third-party vendors has been very busy year in and year out, linking to test results where Windows native security was partly disabled and the third-party vendors therefore had an advantage.

Seeing them pitted against a fully functional OS allowed to use its full protection capabilities, means that third-parties would not be able to claim that it was just a bad day/ a good day/ something good or bad about testset or whatever statements would be released after the test was public.

Indeed it would be interesting.

 

This

 

Smartscreen only protects Edge and Explorer from drive by attacks and is not system wide.

 

Sorry, but that is wrong!

I have had SmartScreen block .exes from installing from my Downloads folder several times.

 

Microsoft Edge and Internet Explorer 11.

 

As I said above.
... Downloaded a program using Cyberfox > right click > Run as Admin > SmartScreen blocked.

http://www.pcworld.com/article/2971...vacy-in-windows-10-piece-by-piece.html?page=2

Also - http://www.groovypost.com/howto/turn-off-smartscreen-filter-windows-10/

 

Today, we’re happy to announce that with the latest Windows 10 updates, we’ve extended SmartScreen to include protection from drive-by attacks in Microsoft Edge and Internet Explorer 11.
Read more at https://blogs.windows.com/msedgedev...en-drive-by-improvements/#mpeuGvUpZMmToR9s.99

 

You don't have to believe me, but I am telling you SmartScreen Filter protects more than just Edge and IE11. I am sure @Martin_C , or someone else, will confirm what I am saying. You're welcome to your opinion but you're wrong!

 

https://blogs.msdn.microsoft.com/ie...nded-validation-ev-code-signing-certificates/

Edited to include a link to the original post instead of to a quote.

 

I believe you but did not get why they say...we’ve extended SmartScreen to include protection from drive-by attacks in Microsoft Edge and Internet Explorer 11.

 

Yes on Windows 7. On Windows 8 and later it is system wide.

 

Thanks Minimalist.

 

I believe there's a huge misunderstanding going on in this thread.

I don't think @waters meant that SmartScreen wasn't system-wide, but rather that its drive-by-attack protection was only applicable for IE and Edge.

It will still protect you if you execute something from your download folder like @Krusty13 said. But it won't protect you from drive-by-attack if you are using anything besides Edge or Internet Explorer. At the very least that's what I'm understanding.

 

Hi @waters.

Your statements are incorrect.

As has been mentioned before in this thread and similar threads, SmartScreen has been OS systemwide since Windows 8.0
And in IE and Edge.
And in both the Store and in every single app you install and/or run.
And in both Outlook, Outlook.com and in Exchange.
And on both desktop, mobile and Xbox.

Targeting phishing, shady sites, spam, low-rep or unknown files, malware, malware hosts, malwartising and drive-by/zero-days/exploits.

Official information on Windows 8.0 : https://technet.microsoft.com/en-us/library/dn283963(v=ws.11).aspx

Scroll down and read that SmartScreen are implemented in core and functioning systemwide. In browsers AND on desktop.

Official information on Windows 8.1 : https://technet.microsoft.com/en-us/windows/jj983723.aspx

Scroll down and read about SmartScreen being systemwide.

Official information on Windows 10 : https://technet.microsoft.com/itpro/windows/keep-secure/windows-10-security-guide

Scroll down and read about SmartScreen being systemwide.

Official video demonstration of the difference between Windows 7 SmartScreen and the SmartScreen present in Windows 8.0, Windows 8.1 and Windows 10 : https://technet.microsoft.com/en-us...-insights-module-7-smartscreen-filtering.aspx

Play and watch SmartScreen being systemwide in all Windows editions since Windows 8.0

Official information on SmartScreen enhancements now also targeting drive-by/zero-days/exploits : https://blogs.windows.com/msedgedev/2015/12/16/smartscreen-drive-by-improvements/

SmartScreen are extremely efficient.

As has been mentioned before : Windows Defender does very good on its own. Always hovering around 99,8 % on prevalent malware and usually fluctuating between 88-95 % on zero-days in the various tests.

Now add SmartScreen that specifically targets phishing, shady sites, spam, low-rep or unknown files, malware, malware hosts, malwartising and drive-by/zero-days/exploits.

That is how an actual Windows 10 installation works. Windows Defender and SmartScreen in combination. Very efficient combination.

But the testing institutions don't want you to know that, so they turn off part of the native security during testing in order to be able to make their graphs where it looks like the native security are ranking lower.

In the real world, it's a lot better then they try to make you believe.

 

Martin,

What great links! As you can see from above, I struggled and still didn't find anything close to what you've provided. Kudos!


Thank you.

Dave

 

Hi Dave,

Thanks.
I'm glad the links was helpful.
Microsoft usually always documents everything thoroughly.

Martin.

 

smart screen is system wide on 8+. Only on 7 and below it is browser only.