5 Popular Firewalls - Efficiency Test

Discussion in 'other firewalls' started by Creer, Apr 25, 2010.

Thread Status:
Not open for further replies.
  1. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,889
    Location:
    localhost
    Well, thanks for trying anyway! :thumb:
     
  2. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,889
    Location:
    localhost
    Indeed well said :thumb: :thumb: :thumb:
     
  3. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    Yes so far Look 'n' Stop (Pure Firewall) is a winner in efficiency category :D
    So... if you have separate HIPS application or just feel that you don't need one, then the best idea will be install Pure Firewall (without gazillions of addons/features, which will slow your system down and in fact you don't need them too).
    These days on the market there are many "x"in1 products but nobody say that they are useful, every security approach needs to be reviewed and chosen for the individual user because everyone uses computer to smth else. Security applications should gives tangible benefits not be against the user in any aspect of pure security, system resources or convenience of usage.


    Creer.
     
    Last edited: Apr 26, 2010
  4. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    It's very true Peter and I agree with you. One of reason why I did this tests was my personal curiosity how efficient all this firewalls are on my machine and whether in fact I chose the lightest solution for me... because of this... why not share the results of tests with other Wilders... maybe someone will find this helpful :)
     
    Last edited: Apr 26, 2010
  5. Yakuman

    Yakuman Registered Member

    Joined:
    Aug 5, 2008
    Posts:
    75
    I'm curious on what Kerio 2.15 shows :)
     
  6. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    Impossible to test this on Windows 7, not supported for years.
    I also remember that on Windows XP I had problem with Kerio 2.1.5 driver which caused BSOD and memory dumping... but in fact Kerio was/is very light if it works on your computer without any issues.
     
    Last edited: Apr 26, 2010
  7. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Well, as said... no workie on W7; however I tested on XP SP3 and it's actually "heavier" than LnS (which - unlike Kerio - is still actively developped). LnS is a no-BS applications/stateful firewall w/ IPv6 support, extremely configurable but usable almost out-of-the-box if you wish, highly recommended. :thumb: :cool:
     
  8. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    6,660
    Location:
    Hawaii
    I do not agree with Peter. In the instance of these particular tests, they are useful for folks who have limited system resources (small RAM, slow cpu etc) or who want maximum zip for playing games.

    Moreover, the average user is NOT going to try as many firewall options as Creer has done.

    Finally (and I'm sure this isn't the case with Pete) those who decry a particular test are usually folks whose favorite proggie didn't do well.

    Peter mentioned that these forum have become enamored of testing. I agree. However, I consider that these forums have ALSO developed a sizable coterie of those who decry any & all tests, simply so that those folks can pontificate and play their favorite spiel again & again & again -- which by now has become boilerplate for many of them. I am one of those "decriers", at times. I simply looove to bang on Matousec's silly tests at every possible opportunity.
     
  9. Brummelchen

    Brummelchen Registered Member

    Joined:
    Jan 3, 2009
    Posts:
    2,998
    Is it just me or my eyes that in the first test for Online Armor DefenseWall Hips was also installed?
    your screenie showed so... and if so exactly that test is worth nuts...
     
  10. hayc59

    hayc59 Updates Team

    Joined:
    Oct 29, 2008
    Posts:
    2,621
    Location:
    Deep Into The Blues Baby..
    Ah...good eye Brumm..I missed that o_O
     
  11. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    6,660
    Location:
    Hawaii
    Me three miss it.
     
  12. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    I forgot to mention - DW was disabled (HIPS protection was disabled) during OA tests. The processes belong to DW there was in fact but this is how it looks in Windows Task Manager even if DW protection is disabled, so don't worry about this.
     
  13. Escalader

    Escalader Registered Member

    Joined:
    Dec 12, 2005
    Posts:
    3,710
    Location:
    Land of the Mooses
    Hello All:

    Yes, it is hard not to become an advocate for the tools we ourselves have become attached to. Yes again to, feeling upset if our selected tool(s) do poorly in somebody elses tests. Some feel they have to defend as if they were being attacked. Testing is the only way facts can be produced that are measurable. Very different than opinions however well intended. Opinions are easy to produce, test results require work and time. As well it requires courage to publish them and take any heat that comes from advocates.

    But the bottom line in scientific testing:
    1. Are the results repeatable in someone elses lab using identical conditions? So this means repeating the tests in a setup identical to Creers.
    2. Are the differences between these products statistically significant?
    Nobody can exactly match Creers setup.

    All this aside, I liked the tests and if I were looking to replace my FW+HIPS combo I would use his hard work and results as an ordered list for trials on my set up.

    Take care!
     
    Last edited: Apr 27, 2010
  14. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    20,520
    To be honest do I have a product preference. Yep. But I didn't look at the actual results close enough to know how any one of the products really did.

    Also I note one of the comments, that there were two products but one was disabled. It should be removed, as even though disabled it's drivers are there, and the impact of that is at best unknown.
     
  15. Escalader

    Escalader Registered Member

    Joined:
    Dec 12, 2005
    Posts:
    3,710
    Location:
    Land of the Mooses
    Hi Pete:

    Yes we all have preferences. But I wonder about this left over driver concern? Here is Creer's quote that indicates he had an image restore twixt tests.

    I would like to know if the image was FW /HIPS free and used between each vendor and the "new" install done clean
     
  16. wutsup

    wutsup Registered Member

    Joined:
    Sep 20, 2009
    Posts:
    634
    Location:
    United States
    ~Phrase removed~creer, but im pretty sure you mean you have 1 cpu with 2 cores :D
     
    Last edited by a moderator: Apr 27, 2010
  17. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    Yes, image backup was FW/HIPS free & used between each vendor & the new install done clean, however... in case of first two tests I mean OA and DWPF test - I used image backup with already DWPF installed. Why? I did this test few days earlier for needs of this thread and at that time I did not think about doing more extensive tests as I've finally done in this thread... I was too lazy to do new tests for OA and DWPF and I thought if protection in both will be disabled during test there will be no real difference and negative impact.
    During test of OA, DWPF protection was completely disabled and during DWPF test I disabled OA protection. Now I see my bad I can admit to the mistake I made because only that don't make mistakes, who do nothing. I will re-test OA and DWPF again with the same clean image backup file which have been used for other tests. Until that I removed screens with results for OA and DWPF from this thread.
     
    Last edited: Apr 27, 2010
  18. Creer

    Creer Registered Member

    Joined:
    Jun 29, 2008
    Posts:
    1,345
    Yes, of course, one CPU with two cores.
     
  19. wutsup

    wutsup Registered Member

    Joined:
    Sep 20, 2009
    Posts:
    634
    Location:
    United States
    yea sorry about that phrase. didnt know i couldnt say that lol

    hey but creer, can you test the HIPS of outpost firewall free 2009 and the HIPS of online armor free? and compare?
     
  20. Sully

    Sully Registered Member

    Joined:
    Dec 23, 2005
    Posts:
    3,719
    I like comparisons like this. I believe that a grain or twelve of salt needs to be taken with them though.

    In my experience, most comparisons do not empirically prove much. The whole "which browser is fastest" is a perfect example. Lab tests can show one thing, but what my eye sees and what I feel is often not the same.

    Firewalls are a queer sort of tool. Some are undeniably lighter on resources. Some are undeniably more complex. But usually it seems that people use the one that they understand or can naviagate the best. What use is the "best" firewall if it is insanely complex to create rules. It would be more produtive (and likely make everything work) if you chose a firewall that was maybe only 90% as good, but that you could manage well.

    The best part about comparisons, especially those by a fellow user and not a paid source, is that you get to see some more "real world" scenarios, as well as just seeing products that you may not have known about.

    I don't suppose "fanboys" will ever stop thier wanting to convince others of how good thier "fave" product is. In fact, the stronger a "fanboy" base, the more interested I often am in investigating. Afterall, there is a reason why someone absolutely loves a specific product. It may not fit everyones taste, but it is certainly better than the majority of testers saying "it is garbage, stay away".

    The problems arise when a "fanboy" keeps a closed mind to other similar offerings. When this happens, that door is closed and you might as well go elsewhere.

    Sul.
     
  21. Brummelchen

    Brummelchen Registered Member

    Joined:
    Jan 3, 2009
    Posts:
    2,998
    i read those comparisons - no more - no less.
    my view is more like Peters - must be usable HERE.
     
  22. Escalader

    Escalader Registered Member

    Joined:
    Dec 12, 2005
    Posts:
    3,710
    Location:
    Land of the Mooses

    Well said sir! I think you are being hard on yourself this was not an "error" per se just a small experimental adjustment to improve on your results.;) The thread will be the better for it! :D

    The base image was / is an excellent idea and the amount of work a bit more but that is what testing means work and clarity. :eek:

    I for one am quite happy to wait patiently for your upgraded tests. :)
     
  23. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Ahh well what a lot of pointless critism

    Facts purely black and white spoken

    1. Any Windows PC is more or less unique, because Windows (unlike Apple) does not control the hardware. At OS install drivers could be replaced by generic ones or old hardware specific versions installed.

    2. As soon as you start unsing your PC, you will get floating dll files, invalid/old reference in driver locations, useless leftovers of de-installed programs, invalid updates, old drivers, etc

    Black and white conclusion:
    So any repeatable test (EVEN on a clean image) is not representative for my PC, therefore I can state (although untested and completely irrelevant) my setup is the most effective setup. It is fast it is smooth and offcourse safest (again not able to back this up, but what the heck, most of us give opinions and do not provide proof or arguments)


    Personal impression (sort of Ying and Yang version)
    Yes I like the genius of the AND and do not like the tiranny of the OR. Creer thanks for the test, pitty you did not de-install DW when testing OA, but what the heck it gives a nice impression. Because you reversed the condition when testing DWPF (disabling OA). So they were more or less equally handicapped.

    I personally have tested 6 version of the DW firewall (different engines) and can state that the most advanced version creates same overhead (in terms of user experience) as default Windows XP firewall (that is with HIPS ON !). DW seams to eat more CPU seconds than Wndows FW service, but correct CPU usage is very difficicult to measure exactly (can be hidden under parent/calling processes).

    Good job :thumb:


    Regards Kees
     
    Last edited: Apr 28, 2010
  24. pjb024

    pjb024 Registered Member

    Joined:
    Feb 24, 2005
    Posts:
    351
    Location:
    Leeds, UK
    It seems to me that this thread is along the same lines as the popular one started by TheIgster in the AV section which was suddenly terminated by one of the mods. Is there a fundamental difference or will this thread eventually suffer the same fate?
     
  25. Escalader

    Escalader Registered Member

    Joined:
    Dec 12, 2005
    Posts:
    3,710
    Location:
    Land of the Mooses

    Hi Kees::thumb:


    Agreed, the tests here are not representative of my setup either. I'm on W7 64 bit just to drive home the point that you and others have made.

    I see these types of "independent" user tests as very useful even with the flaws in them that we all make.

    I would use them as a starting point for selecting tools and testing them one by one on my setup.


    Now what we need is a user based thread on:

    5 popular firewalls-effectiveness test.

    There would be a minor challenge of course of agreeing on the criteria to measure effectiveness.

    Last time I tried that it was 3 years ago and I had to take a sabbatical!
     
Loading...
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.