risks of using a/v HTTPS interception scanning

The real question is if the AV's performing scanning of HTTPS traffic can detect APT's such as the one noted below.

The NDIS miniport network adapter filters the AV's use in the majority of cases create an interface that allow their real-time scan engines and, possibly heuristics, to scan both encrypted(if allowed) and unencrypted network traffic. In other words, signature analysis is the primary detection method. So if the malware is 0-day, it zips right though a network based analysis:

Criminals Delivering Vawtrack Banking Trojan

The malvertising campaign discovered by Trend Micro researchers lasted until December 31 and affected users located mainly in Japan.

People in Japan were delivered malicious ads that redirect them to a malicious website serving up malware over encrypted HTTPS using a Let's Encrypt-issued certificate.

The malicious website used the Angler Exploit Kit in order to infect victims’ computers with the nasty Vawtrack banking trojan, which is specially designed to raid their online bank accounts.

Before installing the Let's Encrypt certificate, the attackers behind this campaign compromised an unnamed legitimate web server and set up their own subdomain for the server's website, said Joseph Chen, Fraud Researcher at Trend Micro.

The cyber crooks then installed the Let's Encrypt cert on the compromised server and hosted a malicious advertisement (also contained anti-antivirus code) from that subdomain.

Ref.: http://thehackernews.com/2016/01/fr...urce=THNLS&utm_medium=BelowLS&utm_campaign=LS
​

-EDIT-

Also worth noting is network based scanning is ineffective against packed and/or obfuscated malware that is increasingly being used to avoid detection. That malware only uncloaks once it has been loaded into the browser' memory.

​

 

Another important distinction that needs to be made.

Just because encrypted SSL browser scanning is not being done does not imply that HTTPS download traffic is not being scanned; at least in Eset's case. All other HTTPS download traffic is being scanned at the network level. You can test your own security solution for the same by doing a HTTPS download from the Eicar.org web site.

And as we all know, malware does have to be downloaded in some fashion to run on the targeted PC.

 

It is true that remote content must be retrieved by the client before it can be executed on/by the client. If there is no inspection/MITMing of the client<->server SSL/TLS connection there is no scanning "at the network level". It may occur elsewhere, but we have to determine where it would happen for all the different scenarios. I doubt I appreciate all of them, but as I touched upon in the other thread, I can think of several other types of opportunities:

1. It can happen because an AV extension is installed in the browser. Which may or may not apply in a given scenario. I'm inclined to think that an AV that is designed for HTTP/HTTPS scanning would be unlikely to install a browser extension that performs the same (duplicative) scanning operations as the HTTP/HTTPS network traffic scanner.
   
2. It can happen because the browser directly or indirectly triggers use of an AV scanning API. This would depend on whether the browser is coded to do that and for the specific type of content/operation in question and an AV is hooked up to handle such an invocation.
   
3. Filesystem activity alone can trigger it, but only if the operation results in filesystem activity and the file format is understood/processed by the AV and the AV reliably has the ability to block execution/processing before it happens.

I think there are at least two somewhat different "download" scenarios we have to think about. One is the typical file download (exe, bat, etc). Where the browser has to not only retrieve the file but save it to disk in its native format including extension. So that it can be opened by an external application or user. A favorable case. The other type of download would be where remote content is retrieved and processed by the browser itself (html, js, ...). The browser doesn't have to cache such content to disk, and in some configurations a browser won't. Even if it does, it may use a non-native-file-format caching scheme that the AV software can't process correctly. Which makes results browser dependent.

The first AMTSO EICAR test is manual and causes a:

• Content-Disposition: attachment; filename="eicar.com"
• Content-Type: application/octet-stream

download which triggers a Save As dialog (non-blocked behavior to understand what is going on). The second AMTSO EICAR test causes the same thing but via a drive-by technique rather than manual trigger. I don't think those are sufficient test cases. For testing all the scenarios where remotely hosted content may be retrieved and executed. For vanilla file downloads, they might be.

 

As far as an AV extension installed in the browser to scan data after it has been unencrypted, that is not going to happen since it would entail modifying the browser. AV vendors won't go there. The closest to this that currently exists is BitDefender's free Safe Pay ver.. BitDefender used an old ver. of Chromium and modified it. Finally, this browser was designed for banking activities and not general browsing. As such, doubt it does any SSL protocol interception at all.

As far as an AV scanning API interface, that exists in Win 10. However my quick read about it leaves me to believe that it does not apply to scanning of anything directly inside the browser itself.

The only way that AVs will be allowed scan browser data directly is if the browser manufacturers provide a backdoor for them to do so. That isn't going to happen unless someone forces them to. Also overall, it is not desirable since it will be only a matter of time till malware authors figure out a way to exploit the backdoor.

The bottom line is the AV vendors have applied the least costly way to deal with the issue. Akin to a "Band Aid" solution rather that doing it properly as I noted previously.

 

Just how effective is network filtering of web traffic and is it really needed? Nothing betting in this regard than analyzing actual real world experience.

I use IE11 with Smart Screen filtering enabled. I also use a third party DNS provider. I use Emsisoft Anti-malware that has an aggressive IP/URL blocker. I also use Eset Samrt Security than has a network web filter. Finally, I do not use any other web site rep. scanners at browser level. Nor am I cautious about what web sites I visit.

I have had Eset SS installed over a year and during that time I used its SSL protocol scanning approx. 3/4 of the time. During this period, I never once was alerted by the web filter of any HTTPS malicious activity. The overall count of HTTP based malicious activity detected by Eset's web filter was 10 at most. Roughly 7 of those were detected by IP or URL blacklisting. Only 3 were actually detected by the real-time scan engine at the network level as malicious and those were all Trojans.

EAM on the other hand detected much more with its IP/URL blacklist. Most of those were privacy based but a number were flagged as malicious. I don't have exact counts since EAM does not log by category what it detected.

Based on the above, my overall conclusion is your most effective protection against web based malware is using a good third party DNS provider. Beyond that using a security solution with aggressive IP/URL blocking couple with the browser's like protection will provide adequate web protection.

Note: my primary reason for using Eset is its firewall/IPS, HIPS, advanced memory scanning, and exploit protection.

 

I think that depends on what is being scanned and what the browser allows via API. IIRC, one afternoon I was playing around with one of my Firefox extensions and wanted to apply regular expressions to retrieved Javascript files (to look for some strings associated with tracking) and was able to. I can't remember if I just scanned or blocked/neutered at the same time. While there probably are some issues in this area and that affects decisions, I suspect one reason AV vendors don't want to go there is because they have to deal with N different browser APIs which change from time to time. Although I have apprehensions about the moves towards common browser APIs for extensions/apps, that could make things easier on antimalware vendors who want to use extensions.

Well I also mentioned those two APIs supported earlier, so ruling in/out a particular browser's use of those would also have to be done. In my mind anyway. Those implementing AVs or researching things carefully would already know the details. You didn't reply to that mention so I don't know whether you have knowledge of where/where-not those earlier interfaces come into play(?).

I don't know what you mean by "scanning of anything directly inside the browser itself" and why you think a browser backdoor is necessary. I believe the concept is that antimalware software can register itself as a scanner, and applications... if/when they want to... then have a way to find and ASK a scanner to assess content. Just because a browser or other application would ask for scanning doesn't mean it issue free. Where used, the browser would be exposing things to the scanner (and not just in a piecemeal fashion: the AMSI "supports the notion of a session so that antimalware vendors can correlate different scan requests") so some thought would have to be given to infosec/privacy consequences if scanner is cloudware. However, some similar concerns would exist anytime browser retrieved content gets exposed to a scanner.

Is your concern that malware could achieve the permissions necessary to pose as a scanner and a browser wouldn't be able to tell that it was handing content to malware? I don't know how well that is protected against. I don't like locked-down software because it hampers my security objectives. I do like options though, and would point out that a browser could have a "use AMSI: Yes/No" setting or perhaps even something more fine-grained.

EDIT: LOL, you posted again while I was taking my time to reply. One note about your second post: DNS based blocking would be applied before HTTPS connection establishment, so you can't make accurate assessments about the efficacy of MITM scanning while DNS blocking is in use. Same idea applies to other combinations. You have to test things individually. However, yes, DNS based blocking is beneficial... precisely because it is early, but also because it is heavy-handed and reduces the risk that you might miss one of potentially numerous threats at the blocked host.

 

The apps have to initiate the call to WIN 10 AMSI as noted below:

Windows 10 will have a new mechanism that will allow software developers to integrate their applications with whatever antimalware programs exist on users’ computers.

Ref.: https://msdn.microsoft.com/en-us/library/windows/desktop/dn889587(v=vs.85).aspx
​

Whereas eventually browser developers might utilize this for their Win 10 vers., it is high unlikely they will allow passing of any encrypted data.

 

I also believe it is just a matter of time until legal action is brought against retail AV vendors performing SSL protocol scanning. Many vendors are still not performing it properly. Eset for example doesn't detect revoked certificates. The scenarios I see play out in court to name a few are:

1. A sizable financial fraud takes place. The bank or financial concern of the affected user does a forensic analysis on the individual's PC and can show one of more SSL encryption protections are not being properly implemented. The bank refuses to reimburse the individual for the loss. The individual hires an attorney who also finds other like affected persons and files a class action law suit against the AV manufacturer.

2. A eager attorney with a tech savvy connections in made aware the AV vendor SSL protocol scanning is violating one or more of his nation's privacy laws. He files a class action law suit. Even in the U.S. with its weak individual privacy laws, it is illegal to intercept any encrypted traffic for virtually all healthcare related web sites. As far as I can determine, almost all AV vendors who do SSL protocol scanning are in violation since they are not automatically excluding said sites. The situation in Europe is much larger in scope due to its broader and more strict individual privacy laws.

And many more scenarios ....................

Add to that that many AV vendors are now enabling SSL protocol scanning as a default in their products which makes them even more culpable legally.

 

The solutions I see as follows, not all of them but these are just possibilities.

1 - plugin on browser to scan page before its parsed, plugins like ublock origin do this e.g.
2 - no scanning directly of web content but rely on other protections such as memory exploit blocking, behaviour blockers, file scanning as content hits browser cache and blacklisting known url's. emsisoft has proven effective in this regard. Especially when combined with something like HMPA.

Also i think they safe from legal action for as long as they allow the end user to disable the module, which I think they all do currently. Even then its a user's own choice to use that vendor. The only dodgy thing legally is perhaps they should show a disclaimer on installation to warn the end user that they will be intercepting TLS content.

 

I'd lean toward more restrictive wording such as: clear, conspicuous, unavoidable disclosure and explicit consent. Partly because there will usually, if not always, be other features/aspects that deserve similarly serious treatment.

However, some and perhaps even more focus should be on the APIs that are used to inspect/MITM protected communications. That should be a control point as well. Example: I don't think AV software, or other applications, should be able to silently install root certificates. Although fully defending against that would be difficult in the context of a general purpose computer which allows the owner/admin freedom to control and troubleshoot their device, that doesn't mean that API developers should throw their hands up and blow off doing what can be done.

Edit to clarify: That last point is not meant to promote lock-out for all users. If some users want their system(s) to completely eliminate any possibility of MITMing, that is their choice. On the flip side, if some want their system(s) to support/use MITMing, that is their choice. I simply wish for the user to be well informed (so that they can think things through and make a decision before being affected) and for which ever option they choose to be as safe/robust as possible.

 

https://www.helpnetsecurity.com/2017/03/08/https-interception-dilemma/

 

SOS in my book. Corps monitor all employee communications including encrypted ones for legal liability and fraud reasons and will always do so. Any employer that allows a BYOD to connect to the corp network in any form deserves to be hacked.