HTTP scanning and real world protection provided by AVs

[Gullible Jones]

Just something that came to mind today: tests of AV heuristics and signature detection might not be a good indication of how effective an AV is in real life.

A fair amount of malware is spread by USB sticks where that sort of thing would come into play. But as far as I know, the majority of it comes from the web via driveby downloads (as I've mentioned previosly).

So... I'm thinking the real world efficacy may depend heavily on how good the AV's HTTP scanning is. Not in the least because, once malware actually executes, it can bypass most antiviruses.

An example: from what I understand, Avira and AVG both have HTTP scanners. Let's say Avira is the better one at recognizing malware in tests (IIRC it is). Okay, fine, it's probably better at detecting malware on the hard drive or on removable media, or maybe even nabbing it when it executes.

But suppose Grisoft has put a lot more effort into making LinkScanner better. Couldn't this really turn the tables? If AVG is better at finding malicious scripts on web pages before they do their dirty work, it will (I think) have a higher chance of actually preventing the damage from occuring, even if it's not as good against malware on execution, or detecting inactive malware on a storage medium.

Has anyone done tests of antiviruses based exclusively on their HTTP/web scanning ability, to ascertain what really works best for preventing badware from touching a machine in the first place? If so, what AVs came out on top? And does what I'm saying even make sense? :P

[OlegSych]

+1
What we see in 0days tests: many AV block malware by blocking whole WEB-site. But is all PE-samples from this site will be detected (for example AV was installed when system already infected)?

[Konata Izumi]

I'd go with Avast Web Shield for HTTP scanning. ^^

[Gullible Jones]

-{ Quote: "+1
What we see in 0days tests: many AV block malware by blocking whole WEB-site. But is all PE-samples from this site will be detected (for example AV was installed when system already infected)?" }-

Somehow I'm not surprised.

Re Avast I don't know how good its HTTP scanning is, it seems to be less featureful than AVG LinkScanner but beyond that I'm not sure.

[AvinashR]

-{ Quote: "Somehow I'm not surprised.

Re Avast I don't know how good its HTTP scanning is, it seems to be less featureful than AVG LinkScanner but beyond that I'm not sure." }-

I guess now a days each and every AV provides you some kind of HTTP Scanning !!

And see if anything bypasses your security setup then you can't do much things..You have to remove it then by using second opinion scanners.

[Konata Izumi]

-{ Quote: "Somehow I'm not surprised.

Re Avast I don't know how good its HTTP scanning is, it seems to be less featureful than AVG LinkScanner but beyond that I'm not sure." }-

Avast don't show site ratings unlike AVG Linkscanner
but based on experience Avast Webshield http scanning is better than AVG Linkscanner.

[Gullible Jones]

Though really, all of them seem to be absolutely terrible against rogue AV autoinstalls. I was recently in a discussion with a fellow who got Antivirus XP 2010 or somesuch, and Avast didn't even see it coming.

[Konata Izumi]

-{ Quote: "Though really, all of them seem to be absolutely terrible against rogue AV autoinstalls. I was recently in a discussion with a fellow who got Antivirus XP 2010 or somesuch, and Avast didn't even see it coming." }-

I think it will! If he also has Network Shield enabled. an up-to-date one!

[Ibrad]

I think it HTTP may add some protection for vendors (depending on the vendor). Trend Micro has great web protection but can't always detect the threats from the site it blocked.

[Gullible Jones]

-{ Quote: "I think it will! If he also has Network Shield enabled. an up-to-date one!" }-

He probably did, it is enabled by default.

[mvario]

-{ Quote: "He probably did, it is enabled by default." }-

Scanning for PUPs isn't enabled by default and that may have an effect.

[andyman35]

-{ Quote: "Scanning for PUPs isn't enabled by default and that may have an effect." }-
I'd love to hear an explanation from Avira and Avast as to why neither of them enable PUP monitoring by default.???

[Gullible Jones]

... Yeah. So would I. To be brutally frank, that is STUPID.

[Kees1958]

-{ Quote: "Just something that came to mind today: tests of AV heuristics and signature detection might not be a good indication of how effective an AV is in real life.

A fair amount of malware is spread by USB sticks where that sort of thing would come into play. But as far as I know, the majority of it comes from the web via driveby downloads (as I've mentioned previosly).

So... I'm thinking the real world efficacy may depend heavily on how good the AV's HTTP scanning is. Not in the least because, once malware actually executes, it can bypass most antiviruses.

An example: from what I understand, Avira and AVG both have HTTP scanners. Let's say Avira is the better one at recognizing malware in tests (IIRC it is). Okay, fine, it's probably better at detecting malware on the hard drive or on removable media, or maybe even nabbing it when it executes.

But suppose Grisoft has put a lot more effort into making LinkScanner better. Couldn't this really turn the tables? If AVG is better at finding malicious scripts on web pages before they do their dirty work, it will (I think) have a higher chance of actually preventing the damage from occuring, even if it's not as good against malware on execution, or detecting inactive malware on a storage medium.

Has anyone done tests of antiviruses based exclusively on their HTTP/web scanning ability, to ascertain what really works best for preventing badware from touching a machine in the first place? If so, what AVs came out on top? And does what I'm saying even make sense? :P" }-

I have a friend who uses AVG linkscanner to stay out of risky places and really appreciates IE8 smartscreen and download checker. As AV he uses Avast (behavioral and file shield), becasue Avast has safe mode scanning and acquired a lot of knowledge of GMER. He also argues that Avira often turns out best, but he trust AVG linkscanner better for prevention, same applies on removal he has more trust in Avast safe mode scan and GMER knowledge.

Problem with these setups is that it are intellectual exercises, it makes sense, but it is hard to find proof or tests to back this up.

Regards Kees

[Konata Izumi]

-{ Quote: "... Yeah. So would I. To be brutally frank, that is STUPID." }-

umm.. maybe to avoid FPs?:shifty:

[CloneRanger]

Using Avira personal i've found it's heurisitcs are excellent at intercepting malware/scripts etc, before Anything actually gets downloaded to do any damage :thumb:

This is without a so called HTTP scanning engine, because it's not included in the free version. Never found the need for it with Avira's heurisitcs, even when i used the Full version, so no potential slowdowns :)

GMER, as good as it is, is only an after the fact app. Prevention is always better ;D

[0strodamus]

HTTP scanning is overkill IMHO. The file scanner will intercept and prevent the creation of the malware file on disk and that is enough. Other than F-Secure's scanning method (it doesn't use a proxy), the HTTP scanner's proxy will also greatly diminish your ability to control outgoing connections with your firewall.

[Kees1958]

-{ Quote: "HTTP scanning is overkill IMHO. The file scanner will intercept and prevent the creation of the malware file on disk and that is enough. Other than F-Secure's scanning method (it doesn't use a proxy), the HTTP scanner's proxy will also greatly diminish your ability to control outgoing connections with your firewall." }-

Well for anyone containing the browser (DefenseWall, Sandboxie, OA free run safer, PrevX safe online, etc) or using Chrome with --safer-plugins I agree.

Regards Kees

[AvinashR]

-{ Quote: "HTTP scanning is overkill IMHO. The file scanner will intercept and prevent the creation of the malware file on disk and that is enough. Other than F-Secure's scanning method (it doesn't use a proxy), the HTTP scanner's proxy will also greatly diminish your ability to control outgoing connections with your firewall." }-

+1

I agree with you, there is no need of HTTP Scanning if you have Real time file scanner because it will definitely intercept any malware which is creating himself on you hard disk drive...Secondly you can configure SRP or AppLocker to be safe from the drive-by malwares.

No need to have HTTP Scanning it will surely slowdown your browsing...

[i_g]

-{ Quote: "I agree with you, there is no need of HTTP Scanning if you have Real time file scanner because it will definitely intercept any malware which is creating himself on you hard disk drive..." }-
But it won't prevent a malicious code, injected by an exploit on a crafted webpage into your browser's memory, from running. OK, maybe it wouldn't be able to create its files on disk to be started next time (provided the file on disk is actually detected by the antivirus - which may or may be not, independently of whether the exploit itself is detected) - but since it's running, it's able to send your private data out. But if you're OK with it... enjoy ;)

-{ Quote: "... Yeah. So would I. To be brutally frank, that is STUPID." }-
I believe you should make clear (for yourself) what exactly is clasified as PUP first.

[YeOldeStonecat]

-{ Quote: "Just something that came to mind today: tests of AV heuristics and signature detection might not be a good indication of how effective an AV is in real life.

A fair amount of malware is spread by USB sticks where that sort of thing would come into play. But as far as I know, the majority of it comes from the web via driveby downloads (as I've mentioned previosly)." }-
This is why I really enjoyed the "Dynamics" test done at AV-Comparatives...do a search in their tests for Dynamics 2009. It's more "real world"....such as you suggest.

This is also one of the reasons I believe in UTM appliances at the edge of a network, like Untangle..especially business networks. Gone are the days of just a plain NAT router. Have a UTM appliance that gets the scanning done at the gateway, using its own processor, not adding a performance hit to workstations.

[ALiasEX]

I couldn't post this last night after I typed it. Here it is unchanged:

"Has anyone done tests of antiviruses based exclusively on their HTTP/web scanning ability"

No but they have conducted tests using all components provided by the tested products to see "what really works best for preventing badware from touching a machine in the first place?" Unfortunately, they have been limited so far.

Only 100 samples:

http://av-comparatives.org/comparativesreviews/dynamic-tests

Missing some popular vendors:

http://blogs.pcmag.com/securitywatch/2009/12/av-testorg_releases_real-world.php

[AvinashR]

-{ Quote: "But it won't prevent a malicious code, injected by an exploit on a crafted webpage into your browser's memory, from running. OK, maybe it wouldn't be able to create its files on disk to be started next time (provided the file on disk is actually detected by the antivirus - which may or may be not, independently of whether the exploit itself is detected) - but since it's running, it's able to send your private data out. But if you're OK with it... enjoy ;)


I believe you should make clear (for yourself) what exactly is clasified as PUP first." }-

So here you would like to say that an HTTP Scanning will prevent you from these kind of crafted webpages and drive by malicious codes...I don't think so that this will also protect you. I am not at all agree with this....

[SweX]

I wanted PCA to get an HTTP scanning feature so PCA get's more complete:D
But Pbust said it wasn't going to be added any soon unfortunately:( .

[i_g]

-{ Quote: "So here you would like to say that an HTTP Scanning will prevent you from these kind of crafted webpages and drive by malicious codes...I don't think so that this will also protect you. I am not at all agree with this...." }-
Whether it protects you or not, that depends on the particular exploit, antivirus, etc. But it could protect you, yes.

What I'm trying to say is that you are wrong if you think everything has to be written to disk; the malicious action may occur in memory only - where the file scanner cannot protect you, no matter how good it is.

[AvinashR]

-{ Quote: "Whether it protects you or not, that depends on the particular exploit, antivirus, etc. But it could protect you, yes.

What I'm trying to say is that you are wrong if you think everything has to be written to disk; the malicious action may occur in memory only - where the file scanner cannot protect you, no matter how good it is." }-

The same can apply to HTTP Scanners too...It really depends upon the the particular exploit or code.

[i_g]

So you're saying that the antivirus itself can be vulnerable... well, sure, but if you trust the antivirus scanner less than your browser, then you should probably change your antivirus ;)

Scanning of the HTTP stream for viruses certainly contains less code (i.e. less possibilities of an exploitable bug) than the whole browser (where the vulnerability can occur anywhere "higher", not just in the HTTP processing - in the rendering of the HTML elements, in JavaScript engine, in any plugins or addons loaded into the browser's process - Flash, Acrobat, ...) - so I'd rather take the chances with the antivirus scanner.

[AvinashR]

-{ Quote: "So you're saying that the antivirus itself can be vulnerable... well, sure, but if you trust the antivirus scanner less than your browser, then you should probably change your antivirus ;)

Scanning of the HTTP stream for viruses certainly contains less code (i.e. less possibilities of an exploitable bug) than the whole browser (where the vulnerability can occur anywhere "higher", not just in the HTTP processing - in the rendering of the HTML elements, in JavaScript engine, in any plugins or addons loaded into the browser's process - Flash, Acrobat, ...) - so I'd rather take the chances with the antivirus scanner." }-

See, what i want say that even a good HTTP scanner can be bypassed...This is not a rocket science for malware writers. And once their code bypass this thing they can do what they want to do....A very good example are fuds and crypters which a are very well written to bypass all these scanners and sanboxes...:lurking:

[Gullible Jones]

Bypass scans yes. Bypass sandboxes, not so easily AFAIK. An encrypted trojan may be able to avoid detection by an AV scan but if it's executed in a sandbox, it's still in the sandbox when it runs, encrypted or not.

[Konata Izumi]

-{ Quote: "Bypass scans yes. Bypass sandboxes, not so easily AFAIK. An encrypted trojan may be able to avoid detection by an AV scan but if it's executed in a sandbox, it's still in the sandbox when it runs, encrypted or not." }-

right
:isay: :thumb:

[Gullible Jones]

... Or not. I just read a raymond.cc article mentioning "crypters" that could force their way out of a sandbox. Probably not ITW but :o

[AvinashR]

-{ Quote: "... Or not. I just read a raymond.cc article mentioning "crypters" that could force their way out of a sandbox. Probably not ITW but :o" }-

They cannot bypass them, but refuses to run inside them....:)