5 Popular Firewalls - Efficiency Test

Well, thanks for trying anyway!

 

Indeed well said

 

Yes so far Look 'n' Stop (Pure Firewall) is a winner in efficiency category
So... if you have separate HIPS application or just feel that you don't need one, then the best idea will be install Pure Firewall (without gazillions of addons/features, which will slow your system down and in fact you don't need them too).
These days on the market there are many "x"in1 products but nobody say that they are useful, every security approach needs to be reviewed and chosen for the individual user because everyone uses computer to smth else. Security applications should gives tangible benefits not be against the user in any aspect of pure security, system resources or convenience of usage.


Creer.

 

It's very true Peter and I agree with you. One of reason why I did this tests was my personal curiosity how efficient all this firewalls are on my machine and whether in fact I chose the lightest solution for me... because of this... why not share the results of tests with other Wilders... maybe someone will find this helpful

 

I'm curious on what Kerio 2.15 shows

 

Impossible to test this on Windows 7, not supported for years.
I also remember that on Windows XP I had problem with Kerio 2.1.5 driver which caused BSOD and memory dumping... but in fact Kerio was/is very light if it works on your computer without any issues.

 

Well, as said... no workie on W7; however I tested on XP SP3 and it's actually "heavier" than LnS (which - unlike Kerio - is still actively developped). LnS is a no-BS applications/stateful firewall w/ IPv6 support, extremely configurable but usable almost out-of-the-box if you wish, highly recommended.

 

I do not agree with Peter. In the instance of these particular tests, they are useful for folks who have limited system resources (small RAM, slow cpu etc) or who want maximum zip for playing games.

Moreover, the average user is NOT going to try as many firewall options as Creer has done.

Finally (and I'm sure this isn't the case with Pete) those who decry a particular test are usually folks whose favorite proggie didn't do well.

Peter mentioned that these forum have become enamored of testing. I agree. However, I consider that these forums have ALSO developed a sizable coterie of those who decry any & all tests, simply so that those folks can pontificate and play their favorite spiel again & again & again -- which by now has become boilerplate for many of them. I am one of those "decriers", at times. I simply looove to bang on Matousec's silly tests at every possible opportunity.

 

Is it just me or my eyes that in the first test for Online Armor DefenseWall Hips was also installed?
your screenie showed so... and if so exactly that test is worth nuts...

 

Ah...good eye Brumm..I missed that

 

Me three miss it.

 

I forgot to mention - DW was disabled (HIPS protection was disabled) during OA tests. The processes belong to DW there was in fact but this is how it looks in Windows Task Manager even if DW protection is disabled, so don't worry about this.

 

Hello All:

Yes, it is hard not to become an advocate for the tools we ourselves have become attached to. Yes again to, feeling upset if our selected tool(s) do poorly in somebody elses tests. Some feel they have to defend as if they were being attacked. Testing is the only way facts can be produced that are measurable. Very different than opinions however well intended. Opinions are easy to produce, test results require work and time. As well it requires courage to publish them and take any heat that comes from advocates.

But the bottom line in scientific testing:

1. Are the results repeatable in someone elses lab using identical conditions? So this means repeating the tests in a setup identical to Creers.
2. Are the differences between these products statistically significant?

Nobody can exactly match Creers setup.

All this aside, I liked the tests and if I were looking to replace my FW+HIPS combo I would use his hard work and results as an ordered list for trials on my set up.

Take care!

 

Hi Pete:

Yes we all have preferences. But I wonder about this left over driver concern? Here is Creer's quote that indicates he had an image restore twixt tests.

I would like to know if the image was FW /HIPS free and used between each vendor and the "new" install done clean

 

~Phrase removed~creer, but im pretty sure you mean you have 1 cpu with 2 cores

 

Yes, image backup was FW/HIPS free & used between each vendor & the new install done clean, however... in case of first two tests I mean OA and DWPF test - I used image backup with already DWPF installed. Why? I did this test few days earlier for needs of this thread and at that time I did not think about doing more extensive tests as I've finally done in this thread... I was too lazy to do new tests for OA and DWPF and I thought if protection in both will be disabled during test there will be no real difference and negative impact.
During test of OA, DWPF protection was completely disabled and during DWPF test I disabled OA protection. Now I see my bad I can admit to the mistake I made because only that don't make mistakes, who do nothing. I will re-test OA and DWPF again with the same clean image backup file which have been used for other tests. Until that I removed screens with results for OA and DWPF from this thread.

 

Yes, of course, one CPU with two cores.

 

yea sorry about that phrase. didnt know i couldnt say that lol

hey but creer, can you test the HIPS of outpost firewall free 2009 and the HIPS of online armor free? and compare?

 

I like comparisons like this. I believe that a grain or twelve of salt needs to be taken with them though.

In my experience, most comparisons do not empirically prove much. The whole "which browser is fastest" is a perfect example. Lab tests can show one thing, but what my eye sees and what I feel is often not the same.

Firewalls are a queer sort of tool. Some are undeniably lighter on resources. Some are undeniably more complex. But usually it seems that people use the one that they understand or can naviagate the best. What use is the "best" firewall if it is insanely complex to create rules. It would be more produtive (and likely make everything work) if you chose a firewall that was maybe only 90% as good, but that you could manage well.

The best part about comparisons, especially those by a fellow user and not a paid source, is that you get to see some more "real world" scenarios, as well as just seeing products that you may not have known about.

I don't suppose "fanboys" will ever stop thier wanting to convince others of how good thier "fave" product is. In fact, the stronger a "fanboy" base, the more interested I often am in investigating. Afterall, there is a reason why someone absolutely loves a specific product. It may not fit everyones taste, but it is certainly better than the majority of testers saying "it is garbage, stay away".

The problems arise when a "fanboy" keeps a closed mind to other similar offerings. When this happens, that door is closed and you might as well go elsewhere.

Sul.

 

i read those comparisons - no more - no less.
my view is more like Peters - must be usable HERE.

 

Well said sir! I think you are being hard on yourself this was not an "error" per se just a small experimental adjustment to improve on your results. The thread will be the better for it!

The base image was / is an excellent idea and the amount of work a bit more but that is what testing means work and clarity.

I for one am quite happy to wait patiently for your upgraded tests.

 

Ahh well what a lot of pointless critism

Facts purely black and white spoken

1. Any Windows PC is more or less unique, because Windows (unlike Apple) does not control the hardware. At OS install drivers could be replaced by generic ones or old hardware specific versions installed.

2. As soon as you start unsing your PC, you will get floating dll files, invalid/old reference in driver locations, useless leftovers of de-installed programs, invalid updates, old drivers, etc

Black and white conclusion:
So any repeatable test (EVEN on a clean image) is not representative for my PC, therefore I can state (although untested and completely irrelevant) my setup is the most effective setup. It is fast it is smooth and offcourse safest (again not able to back this up, but what the heck, most of us give opinions and do not provide proof or arguments)


Personal impression (sort of Ying and Yang version)
Yes I like the genius of the AND and do not like the tiranny of the OR. Creer thanks for the test, pitty you did not de-install DW when testing OA, but what the heck it gives a nice impression. Because you reversed the condition when testing DWPF (disabling OA). So they were more or less equally handicapped.

I personally have tested 6 version of the DW firewall (different engines) and can state that the most advanced version creates same overhead (in terms of user experience) as default Windows XP firewall (that is with HIPS ON !). DW seams to eat more CPU seconds than Wndows FW service, but correct CPU usage is very difficicult to measure exactly (can be hidden under parent/calling processes).

Good job


Regards Kees

 

It seems to me that this thread is along the same lines as the popular one started by TheIgster in the AV section which was suddenly terminated by one of the mods. Is there a fundamental difference or will this thread eventually suffer the same fate?

 

Hi Kees:


Agreed, the tests here are not representative of my setup either. I'm on W7 64 bit just to drive home the point that you and others have made.

I see these types of "independent" user tests as very useful even with the flaws in them that we all make.

I would use them as a starting point for selecting tools and testing them one by one on my setup.


Now what we need is a user based thread on:

5 popular firewalls-effectiveness test.

There would be a minor challenge of course of agreeing on the criteria to measure effectiveness.

Last time I tried that it was 3 years ago and I had to take a sabbatical!