I'm a bit troubled about a basic security conflict, and would like to get an update on what other people do, and what's best practice for risk reduction in 2014. Malware uses SSL and is often inserted into legitimate sites; so no site and no SSL traffic is truly trusted. Antivirus/anti-malware packages can be set to scan https/SSL traffic as well as ordinary http traffic, a function which seems as essential as ordinary http scanning for malware detection. However, by design and quite correctly, the SSL protocol is intended to be sensitive to MITM interception, in order to ensure the correct remote party is in being communicated with. (Specifically, if SSL didn't "really" need scanning then the same logic would apply equally to unencrypted http and simplifies to arguing that all traffic scanning is non-beneficial and traffic scanning technologies should always be disabled) So this is the conflict: If I enable A/V traffic scanning, then I'll have probable detection of common malicious or suspicious traffic as it arrives in the system, which is very desirable. But enabling scanning breaks every SSL connection's certificate chain, so the browser can't validate whether the remote party is as claimed and the communication as intended. I could allow the browser to detect incorrect certificate chain issues by disabling traffic scanning - but then the browser is more vulnerable to rogue issues that could be detected by scanning incoming and outgoing traffic. It seems by design of SSL, I have to choose either to not identify rogue traffic, or not identify rogue certificates, and freely allow one or other other vulnerability? Advice appreciated?
I don't know if there is a solution to this problem that would meet your needs. I personally disable https scanning and leave AV to scan files when they are written to disk. I know that it would be better to intercept malicious traffic before it hits the disk, but I just prefer to use https without MITM. You might try to use an AV that has some kind of browser integration. This way traffic could be scanned after it gets decrypted by browser and before it is written to disk. I don't know in details how those addons work and I hope somebody else could give you better advice.
If you setup/allow something to MITM your secure connections, I think that MITM would have to handle the server side connections and security thereof. You'd be relying on the MITM to support appropriate cipher suites and order, handle all certificate checking including revocation checking, and create visual alerts/prompts if there is any kind of a problem. Seems to me that a MITM could do that well, but I wouldn't encourage any assumptions on that front. I think you'd want to understand how it works (does it utilize its own routines or OS provided features, ...) and do some testing (to verify it actually detects the various types of problems it should) before relying upon it. Your browser likely has some of its own protection options (malicious URL checking, checking downloaded program files, ...) which can be used without adding anything. You'd want to compare the pros/cons of the browser's features to the pros/cons of which ever browser-external AV MITM component, or browser-internal AV extension, you're considering. An ordinary file download can trigger an AV check through filesystem activity. If not when the file is created then at least when it is subsequently opened. So you should always have decent coverage there, and I would expect the best coverage to come from AV software. When it comes to content processed/displayed by the browser itself, it gets trickier. If a) your browser officially supports a "write to disk, then read back, then process" type model of allowing AV software to check content via filesystem activity, and b) your AV software officially supports that approach (Does it perform all of the same checks as would be performed in a MITM scenario?), then you'd have some coverage on that front as well. I'm not sure how common and viable this approach is today. One simple experiment would be to find, or configure, a webserver from which you can via HTTPS load EICAR test string bearing html, javascript, etc files in your browser. You could compare results with AV proxy/MITM/extension enabled and then again with such components disabled. To see what the full up behavior is, and then also determine if the AV software can block browser loads based on filesystem activity alone.
What this suggests is that ideally the https pathway should not be MITM'ed, for obvious reasons, and other approaches preferred. It sounds like views lean towards trusting disk interception. But isn't that a dead end hope from the start? Browsers routinely use RAM cache: Modern browsers may (probably do) use RAM-only cache whose size depends on the device, browser, and stipulated caching policy (META/HTTP headers) rather than the user. Malware will be unaffected if run from RAM cache by a vulnerable browser or clicked by a naïve user. So two layers of security are now stripped away - the malware won't be detected in traffic and also won't be likely to be detected on disk. Protection is extremely thin, it would be reliant on "unusual activity" detection by the A/V upon malware execution, as the last resort. There are dangers inherent in, and W3C policies against, arbitrarily saving SSL traffic to disk: Disk detection assumes the browser saves everything to disk. But this is against HTTP protocol/specification, which strictly states how http data should be cached and in particular when it shouldn't be cached. Often the remote site will instruct the browser not to cache items of traffic because saving to disk would result in sensitive, privacy breaching material being accidentally held that shouldn't be. As well, most browsers are generally configured to not cache all SSL on the basis that if it's private enough to merit SSL, it probably shouldn't be cached by default. So SSL scanning replacement via "save to disk" relies on something that's in directly conflict with privacy/security demands and browser conceptualisation, which can require a substantial part of https data to never be written to disk (as cache or otherwise). In-place or external malware otherwise quickly detected by the nature of their traffic activities won't be detected. Malware that is already in place, which evaded detection or is on another networked device, may be quickest detected by stereotype attempts to communicate home or across the network, or perform suspicious or illegal network activities. Some networked devices can't run A/V and many devices may be "open" (wise or unwise isn't the issue) and for these scanning is essential. I'd like to believe otherwise, but I can't see how there remains a valid basis for believing that malware/phishing via https (or even normal http) will reliably be detected by disk scan/file activity scan if traffic scan is disabled. It looks like a non-starter to mitigate that way?
Some vendor HTTPS scanning statements: http://blogs.mcafee.com/business/https-scanning-uncovering-hidden-malware http://wsa.sophos.com/swa_docs/ws1000/tasks/ConfigGlobalPolHTTPSScanning.html Some Wilders HTTP (not HTTPS) scanning threads: https://www.wilderssecurity.com/threads/is-http-scanning-webguard-necessary.184856/ https://www.wilderssecurity.com/thr...-add-web-scanning-to-their-av-product.184789/ https://www.wilderssecurity.com/thr...real-world-protection-provided-by-avs.272407/ https://www.wilderssecurity.com/threads/http-scanning-necessity-or-just-a-security-blanket.174155/ https://www.wilderssecurity.com/threads/web-and-http-scanning.207321/ https://www.wilderssecurity.com/threads/real-time-file-system-protection-vs-http-scanning.195436/
Thanks for the links. Of all those, this was the post that stood out by Ondrej Vlcek ("vlk") of Avast in 2006. Much commonsense. He stated even back then and for plaintext http, not to rely on disk scanning to compensate for absence of traffic scanning, since it implied trust that the browser won't be subject to exploitable vulnerabilities from data not saved to disk. Since then, it's more, not less likely that a substantial amount of traffic data won't be saved to disk before being processed. Reasons include no-cache HTTP/META headers (privacy/staleness reasons), avoidance of disk-caching sensitive data by browsers (which SSL is presumed to be), RAM based caching, early stage browser analysis of incoming traffic for optimisation/prioritisation before caching. If we need to ask whether malware can execute before being saved to disk, the answer is obvious. Malware can execute prior to saving to disk, or when nothing will be saved to disk. With SSL websites (for several reasons) and browsers that can use RAM cache, much more traffic won't ever be saved to disk by design. If one looks at data and expert posts in those threads, they enforce the original question, they don't really help to answer it. They seem to say both that disk based scanning probably can't substitute for full traffic scanning, but full traffic scanning isn't possible without breaking many sites' certificate chain, creating impersonation vulnerabilities instead. The rest is no real hard data or expert opinions how to mitigate or which vulnerability of the two an end user is advised to allow exposure to, on an SSL website, if mitigation isn't possible. But that's adding nothing to the basic question which started the thread - "It seems by design of SSL, I have to choose either to not identify rogue traffic, or not identify rogue certificates, and freely allow one or other other vulnerability - advice appreciated?" (** I ignore http here because "significant" sites of all kinds are much more SSL based - the amount of time people spend on SSL sites or rely on them is comparatively large: whether the measure is high volume such as Google/Facebook/Microsoft/Wikipedia/EBay/Amazon, major government/health/education/tax portals such as irs/gov/edu/gov.uk, non-http protocols such as IMAP, or sites handling personal/financial matters such as: bank/email/online_shop/insurer/phone_provider/doctor. Basically the more "important" and more "popular" sites and traffic are much more likely SSL also.)
Hi guys, HTTPS scanning is indeed very useful as many sites are moving to HTTPS-by-default (or even HTTPS-only). Of course, this applies mainly to higher-profile sites but history has shown that even such sites commonly become attacks vectors (as hackers break into them and inject malicious code). I think that the value of a HTTPS scanner really depends on its actual implementation. Most of HTTPS scanners today are indeed MitM (technically), but their actual implementation varies vastly. The first generation HTTPS scanners were simply hijacking all traffic and using the same cert for signing. This had a lot of negative UX consequences, and in many cases, didn't work at all (think e.g. about certificate pinning in Chrome). The second generation is different: it uses the original cert for the site, but changes the root of it (i.e. the scanner re-signs the whole chain). This works much better from the UX perspective -- as browser doesn't really notice anything -- but it still represents some security risk if implemented incorrectly. The implementation must make sure it perfectly mimics all the possible states -- expired, revoked, invalid etc. Also, naturally, it must generate the root certs ad hoc on the device and never reveal them outside of that box -- otherwise, an attacker could use them to orchestrate yet another MitM attack downstream. Plus, there's a few more implementation quirks that need to be solved (especially related to the HTTPS protocol also being commonly used as a secure communication channel inside apps, in various non-standard ways). But generally, I see a lot of value. FYI, the upcoming Avast 2015 (including Avast Free) includes a full-blown 2nd-gen HTTPS scanner, and it's turned on by default. Cheers, Vlk
Thanks vlk. As an aside, how far are we (ie major AV/antimalware/browser/standards creators) along, to working at fixing this? Because the usual advice "out there" as can be seen by googling, is "Oh, just disable SSL scanning, you don't need it and this fixes it"..... Avast might have it solved, but I think it's more than a problem just one company should fix? Is this an AV sector issue, or a standards/IETF issue, or a browser creators issue? Is there much that can be done to motivate progress?
No, it is still very much guerilla-style. There were attempts to move the functionality into browser extensions, as this would be a cleaner way to do it, but given the dismal state of the browser extension ecosystem (all the bad toolbars etc. -- sometimes even distributed by AV apps) this didn't really lead anywhere.
Hi VLK, I checked Avast 2015 (build 2208 ) and it gives no warning on a site with a revoked certificate when HTTPS scanning is enabled.(Tested with FF33.1 and IE11).
