Firewall Scorecard

After a few months of testing firewalls, I have decided to summarize the results. My rating system is from 1 to 5, with 5 being the highest. To rate a 5, the firewall must be a keeper. That is, it is in use on at least one system that you have the right to choose software. Rather than use stars, I have decided to use the ^ character. Why a ^ ? Well, the ^ makes me think of a shark's fin and a great firewall will have a ^^^^^ rating, or five sharks. After all, I do love scuba diving and sharks clean up the seas in their own way, like a firewall cleans up bad packets.

^^^^^
Kerio 2.15
CHX-1
These are the only firewalls i have in use right now. Kerio on my main machine and CHX-1 on some other low powered boxes for which I do not want to take the time to set up an application aware firewall. I am able to use the same rules for each machine running CHX-1. The ability to specify TCP flags and pseudo stateful UDP are nice as well. I like Kerio because it is the easiest to understand rule based firewall with the best interface of all of them, and the right balance of simple application control without excessive user interaction.

^^^^
Look n Stop
This one needs a bit more testing before I get comfortable with it, and possibly a new release that takes the various beta drivers to release status. It is the next one that I intend to look at carefully.
8Signs
Very good interface, but CHX-1 is preferred due to the UDP pseudo stateful feature.

^^^
Zone Alarm
Has performance issues when many connections are open. Simple things like specifying allowed SMTP addresses are very difficult. It is the best choice for beginners who want more than the XP firewall. User interaction is excessive when advanced program controls are turned on.
Jetico 1.55
Promising, but not ready for prime time yet. This could become a keeper if the developers add all of the usual features that most firewalls have and cut down on user interaction.

^^
Sygate
The proxy issue is a killer for me.
Tiny 6.X
When I am ready to design the replacement for the space shuttle, I will be ready for this firewall. It is just too complex.

Waiting in the wings:
I really need to take some time with Outpost Pro, but have not got around to it.

Your own ratings are invited using ^, *, or whatever.

 

I don't know about the FW for Tiny because it screwed up my system every time I tried to install it, but I have installed all the other Tiny modules (Windows Security, Integrity Guard) and until April 16 you can get Tiny 2005 Pro for $49 I believe (I got the discount code just by e-mailing their sales team)...pretty awesome deal considering it can basically be made to replicate most of the functionality of Prevx, RegDefend, Process Guard Free, and alot of overlap with a2 IDS (alert on driver/service installation, code injection)

Back on topic I give Injoy ^^^^^

 

I would rate ZoneAlarm Pro much higher, near the top, but I'm not the one doing the rating, hehe ..

 

Jetico is good but needs honing, Kerio 2.15 is the yardstick all firewalls should be looking for. Zone Alarm free is quite good, specially for begineers but the same simplicity can also lead to holes in it. NetVeda also shows lots of promise and probably is a very good and free alternative to Zone Alarm Pro with far more features and controls.

 

i dunno why, but i put off using outpost for a long time too. i must have read something bad about it way back when. anyways, i would have to give outpost pro 2.5 a ^^^^^ for now. out of all the firewalls i used it seemed to be the best fit for me and my computer set-up.

 

for me, i would rate outpost ^^^^^, then look n stop ^^^^. i dont rele like other firewalls except maybe syagte, but its outdated with a few bugs and above all its application-based.

 

I wanted to wait for a few people to comment before I did . The opening post , seems to me , is basically rated on ease of use . Then again , I do not know . Zone Alarm is as easy as easy gets but , it got 3 ^s . On the other hand , Tiny got 2 . Tiny is probably the best overall firewall on that list . Ease of use : ZA . Protection level : Tiny Best of both worlds : Outpost Pro . Outpost is as good as Tiny . Just that Tiny adds a few things that no other firewall has . ZAP protection is not in the same class anymore . Very sad as they were extremely good at one point . And please do not make the mistake that I rate Outpost high because I use it . WRONG ! I use it because it is THAT good . Hope that helps

 

Its not just about ease of use. It is about effectiveness and transparency. It is the ability to understand what is going on that makes a security application transparent. If it is too confusing, mis-configuration is a major possibility.

Beauty is in the eye of the beholder. For example, Outpost Pro which I can't seem to get around to exploring is a favorite of many around here. Remember, the ^^^^^ rating is for keepers. If you can keep it and be happy, that gets it a ^^^^^. It is going to be a different answer for different users.

My choices also reflect a lack of comfort with the current crop of advanced application controls. It is my view that if the user if presented with too many firewall warnings, eventually they get numb from it, and will allow a Trojan to install when a Java or IE box asks the question "do you trust xxx banking login thief". I have left many posts on this board questioning the inability of many of these security solutions to differentiate between benign activity and harmful activity. The warnings for benign activity happen so much more often than for harmful activity that it has caused me to consider these approaches to be useless, and the entire pursuit of firewall leak testing to be futile, considering the current state of the art.

This does not mean folks should not try these tests. Its just that there is no substitute for user awareness. In circumstances where the user is not aware (children/teens), the machine needs to be locked down using strategies similar to those used on enterprise networks or kiosk browsers.

So, don't be concerned if my ideas are different from yours, the point is to find out what all the ideas are.

 

Pretty good list Diver..

I would rate mine generally as follows:

5 sharks:
CHX-I
Jetico
Kerio 2.1.5
BlackIce

4 sharks:
8Signs/VisNetic
Kerio 4
Outpost Pro

3 sharks:
Look N Stop
Tiny 6.5 Pro
ZoneAlarm Pro
Sygate Pro

2 sharks:
Anything else not listed above except (1) below

1 shark:
Norton Personal Firewall

I'll try and use anything 3 and above for a while, but 2 or less I'm not interested in. I'm sure I've missed a good one, but those are what I can think of right now..

 

I'm really surprised that sygate hasn't been rated more sharks by you guys. To anyone not affected by the local proxy issue it seems a good one, especially the free version given the price.

 

Sygate is pretty good, but the proxy issue is a killer for some. I do like it, but I also like quite a few others even more. It's good though, in my humble opinion..

 

still waiting for the new version, it might rank up there. i think that 5.5 is a little dated tho, and i don't feel safe using it.

 

They are unbelievably slow in updating.. 5.6 free has been out how long now, and there's still no 5.6 Pro? Don't know why.. And they don't seem to care much about the proxy issue or they would have done something about it by now...

 

In Sygate forum, there is a official response from their tech about the proxy issue, they are supposed to be working on it, surprisingly, been a long time but till now, it is yet to be solved. Shame, otherwise another good alternative to Kerio 2.15 and the pro version comes with IDS which is regularly updated by Sygate. I had to let go of it as my love for Avast would not make me switch to another AV specially when there are so many alternative firewalls out there.

Would just like to add that Jetico, when develped would be among the best out there, the reason, it has good sandboxing with very low resource requirment compared to other firewalls with sandboxing and has the best packet inspection among its peers. Best of all, it is still free with no expiration limit so even if it goes paid, one can still use the older version free of cost like we do with Kerio 2.15

 

^^^ 1/2
NetVeda

^^^
Jetico

 

I pretty much agree with Diver's sentiments, even though I have not tried all of those firewalls listed. I too believe transparency is tremedously important. Sometimes the worst firewalls are not those with no user control or full control but "semi-user control" with flakey (hey hey: flake get it?) documentation about who or what is making the decisions. I find it quite amusing that many who absolutely hate Kerio 2x because it is too difficult for them agree that it has the best interface. If you look at the official Kerio forums, even those who believe Kerio 4x is great keep making comparisons with features of Kerio 2x which should be included, and not just limited to the user interface. Go figure.

(After that "flakey" remark I almost wish I filled this post with as many diver associations as possible, while making my argument)

 

I just find funny how different the peoples' opinions may be on this topic. For my part for example, I woud nearly inverse the most popular ratings. I don't even consider Jetico, CHX-I or Kerio. Jetico is still beta for me; after many tries I came to conclusion that it would more likely screw my system than protect it. Perhaps it will be number one inthe future, but not yet. CHX-I has become popular recently. Great, but it is only packet filter destinated to protect servers, and is not a general purpose firewall. Sorry, but I wont change my opinion on this. The Visnetic/8Signs /InJoy is the same category. Kerio is good, easy, but very old and unsupported, has really become obsolete in comparison.
If the winner is the keeper, in my case it's Tiny. No other may compare to that one in terms of protection. Agree it's complex, but after you have managed to configure it (about one month work), you may forget it and use just like ZA.
I don'know how, but it pops up warnings before PG on my system, and thats a reference.
Second to Tiny but a way down, I would rate Outopst (quite efficient and much easier).
Next (three sharks) woul be LnS and Sygate. I dont rate ZA beacause I haven't used it for years.

 

Excellent post ! I think you are on the money with that .

 

Quoting Diver
My choices also reflect a lack of comfort with the current crop of advanced application controls. It is my view that if the user if presented with too many firewall warnings, eventually they get numb from it, and will allow a Trojan to install when a Java or IE box asks the question "do you trust xxx banking login thief". I have left many posts on this board questioning the inability of many of these security solutions to differentiate between benign activity and harmful activity. The warnings for benign activity happen so much more often than for harmful activity that it has caused me to consider these approaches to be useless, and the entire pursuit of firewall leak testing to be futile, considering the current state of the art.

If'n I had my druthers....I'd druther have a 50/50 chance of guessing right
on a pop up....then having no chance at all.

 

My view is if there are only a limited number of warnings to respond to, the user has a fighting chance of getting it right almost all of the time. Whem there are too many, the odds do drop to 50%.

If you would rather have a 50% chance on 100 warnings out of which only one is for real, have at it. I would rather eliminate the unnecessary "application A has started application B" stuf so I have a chance to concentrate on the occasional "this website is about to hose your computer, do you trust us" warning, and get it right every time.

Just think about it, where is the malware coming from?

 

Since TCP/IP packet technology has not gone through rapid change since Kerio 2.15, dont think age of firewall has to do with anything, the simplicity of set up of Kerio 2.15 ruleset creation is its beauty and security, properly configured Kerio would be hard to bypass then and even today. Unlike anti virus engines which need updates frequently due to new patterns and algorithm of viruses, a packet has no such pretensions, a bad packet is a bad packet today, a port sniffer or scanner would do what it did four years back.

Kerio 2.15 will continue to stay favorites of many including yours truly here due to its interface and the fact that people like gwion, BZ and many others continue to provide real world free support at forums like this and DSL, this can not be replicated by any tech department. The turnaround time for a Kerio related question is by far the fastest. As for outbound protection goes, a good anti virus and Trojan scanner plus some common sense will go far in protection than any intrusive sandbox which would pop up every few seconds on minor perceived threats. Just imagine a SYSADMIN handling all the calls for all the pop ups, the company would go out of business as they would have to employ far more SYSADMINs than workers.

 

It would be better to catch it before it ever gets on your machine..

 

Yes, I agree, my comments about Kerio age concerned application rather than TCP/IP filter component. For overall protection, just have a look at the firewallleaktester results and compare with others. It's app controll is really not sufficient. On the other hand, if it's only TCP/IP filter that you want, you'd better go with CHX-I or 8signs anyway.

It's a matter of question which was already posed here : do you want a firewall to be easy or efficient ?
The majority of ratings here confirms that the ease of use and the transparency is most important criterium. Thats why Kerio still remains so popular. Its your choice and of many others here and I respect it. For my part, I also like it of course, but If I have to choose, I take efficiency.

I don't think however that any AV/AT scanner would ever offer you better protection than a good, well configured sandbox combined with PG. And believe me, it need not to be very intrusive to be efficient. It does not pop up as frequently as you migth think. In fact, with few exceptions, it only happens on new installs and at first use of new progs.
As for the SYSADMIN question, ok, I would not take Tiny for a company with dozens of dumb employees if I was an Admin .
But here, I am on a personal standalone PC and I am my own hotline, as the majority in this forum.

 

Dont you think there would be newer ways of beating out sandbox as well. CHX-1 cant be compared with Kerio, they are entirely different, on top of that, CHX-1 cant be used in a Gateway/NAT enviroment.

I feel seperate sandbox applications like Process Guard/SSM and others do a real nice job in preventing malware execution and all the firewall should do is provide with an outbound access warning as well in inbound attacks.

 

I would second this - the ability to filter traffic by application is critical for those concerned about trojan activity. If you have locked your PC down to the point where no new software, drivers or services can be installed then this is not an issue, but otherwise CHX-I should be considered for use on gateways/proxy servers rather than end-user PCs.

InJoy is worth considering as a competitor to CHX-I since it offers traffic-limiting (e.g. you can restrict P2P traffic on your network to ensure that it does not interfere with web browsing) and pretty graphs.

Configuring Tiny well does require knowledge not only of Tiny itself, but of the Windows system and every application you have installed (what files do they need access to, which registry keys?). However it can then replace a firewall, Process Guard, Reg Defend and (with track and reverse) an uninstaller also, so it should be worthwhile for the intrepid (not me, I'm a coward... ).

I would suggest not reading too much into this - it may come down to issues like installation order or startup order. What matters is that the prompt occurs before the program starts and both should suffice here.

While this is a justifiable point, it is not just the advances of TCP/IP that has to be considered but also Windows itself - any new vulnerabilities found in its network subsystem have to be countered and (with the latest leaktests) some form of process monitoring has to be done also.