AV-Comparatives Firewall Test 03/2014

I spent the last day trying to replicate their results but haven't been able to do so to be honest. In general both ECHO REPLY as well as NetBIOS ports are considered "restricted". That means, even if you allowed the application who serves them by accident, unless the computer or the network that sent the request is marked as trusted, packets will be dropped. RDP is disabled by default in Windows and it needs to be enabled manually, which is why there is no default restricted port rule for it. But both NetBIOS as well as pings should have been blocked no matter what.

I asked the AVC staff for more detailed information. In general they are very cooperative and easy to work with, so I hope we find out what went wrong in their tests so we can fix it.

 

Yes it can be turned off in System properties - Advanced System Settings - Remote tab. You can disable Remote Assistance and Remote Desktop...

hqsec

 

Of course...it can be disabled similar as other remote or sharing services like access, registry, help, WebFolders and WebClient, folders sharing, all "Live packet" components, WMP sharing, etc.
This test is very controversial.

 

You can and RDP is turned off by default. You have to turn it on explicitly.

 

I await that eagerly

 

No, the test is way too limited. I'm not so sure about the methodology of test. Maybe it has some validity to it, or maybe not.

 

Sure it can be turned off and people can use linux instead, but this test was about default settings and all products used by common users, like 99% of the time.

 

Nice to know Fabian is working on it.

 

This test put the cat amongst the pidgeons A worthwhile test

By default Avast IS doesn't disable the Windows Firewall, so both FW's run in parallel

So much for the age old "advice" Not to run more than 1 FW together

Microsoft’s Windows FW built into Windows 7 got a perfect score

Also take note that generally, Home editions of Windows are affected less than Professional/Business/Enterprise versions of Windows

 

I wonder if Avast's fw isn't simply a Windows fw interface

 

@ Joxx

Could be ? Somebody must know though !

 

The published test is too restrictive to serve as a basis for any decision making. FW testing is a complex issue, and just reporting on 3 parameters (ping, RDP access and file share) is not sufficient.

Just to drive my point home: if one looks at Windows FW vs. ZoneAlarm, it is very clear from the published tests that Windows FW is a much better FW.
However, ZoneAlarm services are installed very deeply into the OS kernel, and it is very difficult to dislodge them.
OTOH, WFW is running at a higher level, and if you have a program on your computer that runs as a local proxy (some well known AV programs do that in order to better analyze suspected code), and there is a hacker code on your system that accesses the Internet through that program, it will not get detected by WFW. ZA will detect it.

BUT, I myself run WFW and not ZA (although I have 3 PRO licenses for it), because ZA is a resource hog while WFW is much lighter - and this is a big plus for my specific configuration.

 

Yes, if there are other computers/devices connected to the same router.

 

I don't understand the need to question the methodology by bringing up the topic of outbound control. It is clear that the purpose of the test was to check the effectiveness of firewall in protecting against unsolicited inbound access on potentially hostile networks such as the public ones in airport, hotel and cafe.

As for the results, it is hardly surprising that Windows Firewall fare well when network location is set to Public. Good to see a few vendors pass...especially those that choose to use WFP. Disappointing to see some of them fail in providing a minimum equivalent level of protection (and to add insult to injury, you pay for the license) Too many place a misguided emphasis on leak tests (a race to no end) while disregarding the most important function of a firewall.

 

I couldn't agree more!

 

It is not healthy to decouple inbound from outbound control.

Take this operational scenario of a typical FW as an example:
1. Your browser sends out a http request on port 80 to some external server.
2. FW catches the browser outbound request and pop-ups an authorization request. You approve.
3. The remote server answers the request and you get an INCOMING response.
4. No popup alarm from the FW for this inbound traffic.
Why no popup alarm for (4)? because the FW knows that YOU asked for that data on THAT port, so the response is, by implication, wanted.

So, if you have a rogue program that injects itself into the memory space of an active browser of yours, it can conceivably sends out hacker's requests that will get an unblocked incoming response. You want to catch this too.

 

Well, I know that. The role of firewall is to regulate traffic both in and out. What you mentioned falls under the category of HIPS...that is if you want to prevent that rogue program from doing its deed. But that is stupid because there is no proper isolation among desktop apps (leaktests prove my point). If a rogue program is the concern, it should not execute in the 1st place.

In fact, you don't even need a rogue program for such things to happen. Your browser itself is the main threatgate. All it takes is browser vuln or a rogue website. XSS or SQL injection.

Common Network Security Misconceptions: Firewalls Exposed (thanks to MrBrian for sharing the link)

As Marcus Ranum puts it, wire cutters are the ultimate firewalls. A firewall is a risk reduction system; not a risk mitigation system.

http://www.ranum.com/security/computer_security/papers/a1-firewall/
http://www.ranum.com/fun/bsu/ultimatefirewall/index.html

 

Is more the opposite...

Is unhealthy, annoying, whatever to coupe inbound and outbound!

Packet filter and process control are different worlds...

 

I had something like that happen to me last year, but it was an exploit involving windows media player, that online armor did not stop because I had it set to automatically allow digitally signed programs to do things to the system. You could make the argument that I could have changed that setting, but the popups from legitimate system processes make this nonviable, because it will ask for permission for everything. The pop-ups are almost worse than the security issue. The only reason I caught the problem though was the online armor logs -- which were tremendously helpful. In the end I opted to wipe everything and start over again. But more alarmingly is that the firewalls allow remote desktop connections on networks that are designated public, which is a huge security issue, and frankly embarrassing for any firewall vendor. Even if you filter outbound connections very well, you need to provide the same type of protection inbound -- I do not want to be objectively more insecure using security software I have paid for, than the free alternative that comes with windows.

 

I have not completely read the report yet, but it contains links to Youtube videos which show the testing. For example they mention that Outpost still fails with Netbios explicitly blocked, but in the video you can see that Outpost is in Permanent Auto-learn mode, which afaik allows everything that it doesn't detect as a threat.

 

I don't think so. I just looked at that youtube thing, and saw on the left side Outpost icon BLUE. Blue=Rules Wizard=it asks. Allow Most and Allow all are orange and red - I just recheked those colors on Win7 Outpost.
That video goes so fast you have no time to read all the settings they showed, but the icon color really was blue.

Edit - we don't know what mode it was in. Because Autolearn mode can only be done when you're in Rules wizard, which is blue and there is no indication of learning on that icon. Oh, well, back to my green BlockMost.

 

The Outpost window has Auto-learn mode in the title everytime it is shown, and you can also see it's Permanent when they go to settings twice. Set the video to 1080p and watch fullscreen, it's clearly readable. And if it goes too fast, there is always the pause button

 

OMG! You are absolutely correct. Permanent Autolearn on the setup screen. Unbelieveable.

 

If Auto-Learn is the Outpost default setting then its not really a surprising result as all products are tested at default settings. See page 4, first paragraph.