New Firewall Tests from Leak Tester

Discussion in 'other firewalls' started by Antarctica, Mar 11, 2006.

Thread Status:
Not open for further replies.
  1. Antarctica

    Antarctica Registered Member

    Joined:
    Feb 25, 2003
    Posts:
    1,617
    Location:
    Canada
  2. Chuck57

    Chuck57 Registered Member

    Joined:
    Sep 2, 2002
    Posts:
    1,422
    Location:
    New Mexico, USA
    Looks to me like Outpost and LnS got the same score, but Outpost did edge out LnS, barely, in the leaktests. I think either one is a great firewall.

    I've never tried Jetico nor paid much attention to anything about Jetico, so can't comment on its win.

    *edit* On looking at the list again, if you consider the vertical line - (generic block) which means the leaktest is stopped before it can access the network, it looks like ZA Pro actually stopped more leaks than any of them by 14 to 12.
     
    Last edited: Mar 11, 2006
  3. StevieO

    StevieO Registered Member

    Joined:
    Feb 2, 2006
    Posts:
    1,067
    As always very good tests to have and to be able to reference etc, thanks gkweb.

    I tried both Breakouts and Jumper on ZA Free on 98SE just to see what might happen, and they didn't even launch. As they are for XP etc it isn't surprising, but sometimes exploits can cross over, so it's nice to know i'm safe from those. I've used all your tests Apps several times in the past, and others too, and am able to block 99% of them with ZA Free in high security mode and running Winsonar.

    This latest round of tests made me look into Jetico for the first time, so well done to them ! I DL the help file to read, which was very indepth and i would need more time to study it properly to set it all up correctly, if i choose to try it. But i'm not sure given my present system config, that i would gain any benefits from spending all that time doing so and installing it ? Unless somebody can explain how i may improve my security by installing it.


    StevieO
     
  4. hemkop

    hemkop Registered Member

    Joined:
    Feb 7, 2006
    Posts:
    61
    Well it seem that bitdefender has also improved the firewall.
    And specially on version internet security, there it is a button "Stealth Mode"

     
  5. flyrfan111

    flyrfan111 Registered Member

    Joined:
    Jun 1, 2004
    Posts:
    1,224
    ZA doesn't get the total points for AWFT as it only stopped 8 of the 10 subtests in that test. It also got a generic block on 1 of the remaining 2 and it missed one altogether. By counting the way you are, you would be counting ZA'a generic block of one test and it's blocking 8 of 10 tests of the same quality as the checkmark for Outpost, LnS and Jetico for stopping all 10. That would obviously be unfair to elevate ZA to the same level as the 3 that did stop all 10 correctly. ZA is a good product though as it was the only firewall that managed to stop the breakout1 test. The score is accurate though as each test a firewall passed is worth one point. A generic block is worth 1/2 a point.
     
  6. Wolfe

    Wolfe Registered Member

    Joined:
    Jan 23, 2005
    Posts:
    160
    Good stuff. The only thing that does bother me is the fact the tester is heavily involved with GhostSecurity - thus putting his impartiallity at stake.
     
  7. nicM

    nicM nico-nico

    Joined:
    Jul 15, 2004
    Posts:
    631
    Location:
    France
    Where can you find a single trace of partiality when Ghostwall - or any Ghost Security product - is not included in the tests? o_O

    Cheers,
    nicM
     
  8. Alphalutra1

    Alphalutra1 Registered Member

    Joined:
    Dec 17, 2005
    Posts:
    1,160
    Location:
    127.0.0.0/255.0.0.0
    nicM, I think he is refering to gkweb's preference for a HIPS is AppDefend. I think he believes there is biasim since he is a beta tester of the product and also produced their procX. However, it should not really affect partiality since gkweb also beta tests jetico, DCS, and many other companies products.

    Alphalutra1
     
  9. nicM

    nicM nico-nico

    Joined:
    Jul 15, 2004
    Posts:
    631
    Location:
    France
    OK, but we're not talking about the leaktest comparative anymore there - or I'm missing something?

    I was reading the post I quoted in the context of theses firewalls tests (thus my reaction ;) ) , not about the software advice section.

    Cheers,
    nicM
     
  10. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,026
    Location:
    The Netherlands
    Nice to see that they have updated the test results, and also nice to see that ZA Pro isn´t doing that bad. Of course other HIPS should be able to fill the gap. ;)
     
  11. Phazor

    Phazor Registered Member

    Joined:
    Jun 27, 2002
    Posts:
    111
    Were these firewalls tested using the programs default ruleset or rulesets created by the tester?
     
  12. JRCATES

    JRCATES Registered Member

    Joined:
    Apr 7, 2005
    Posts:
    1,203
    Location:
    USA
    I PM'd gkweb (firewall tester), and asked him that exact question (if the tests were performed on firewalls as they are presented to the user "out of the box", or if the settings were tweaked for maximum performance). Here is the reply he sent to me:

    "Yes I have tweaked the firewalls to their highest settings, every feature was enabled, everything maxed out. Global filtering rules removed if any, to be sure to be asked about any network activity.

    Out of the box settings, generally and depending of the firewall, are weaker. Out of the box settings are generally purposefully not set to high to not ask too much popups to the user.
    "
     
  13. Phazor

    Phazor Registered Member

    Joined:
    Jun 27, 2002
    Posts:
    111
    So basically for the average joe, the tests really mean nothing then.
    Unless of course the user is a firewall guru and can create the same settings and rules as what was used for the test.

    I think the test maybe should have been the other way around. Test it "out of the box" for the average joe. Give us a idea how bad the firewalls work in that way.
     
  14. JRCATES

    JRCATES Registered Member

    Joined:
    Apr 7, 2005
    Posts:
    1,203
    Location:
    USA
    You're welcome :blink:

    Well, I'm just passing along the info...even though I agree that perhaps there should be TWO sets of tests: One for the "average Joe" (which I am one of as well), and one for the most knowledgeable and advanced users.

    But please....don't kill the messenger cause you don't like the message! ;) :p
     
  15. Phazor

    Phazor Registered Member

    Joined:
    Jun 27, 2002
    Posts:
    111
    So JR where exactly did that leave you with your firewall of choice?... Since the tests really arn't directed towards the average joe and his average ruleset firewall..:doubt:
     
  16. JRCATES

    JRCATES Registered Member

    Joined:
    Apr 7, 2005
    Posts:
    1,203
    Location:
    USA
    Probably in the same position as you - UNDECIDED :p

    Actually, I am trialing LNS on a test machine currently.....and if Blackspear's settings and "Advanced Options" being checked are "sufficiently adequate" and are what is considered to be the "optimal settings", then it is relatively easy enough to understand and properly configure. Don't know about any specific "additional rules", though.....

    Hey, here's an idea for gkweb to consider for future tests, though (and to add as a disclaimer for the current round of tests, if he can remember and wouldn't mind adding this additional info):

    Perhaps WHATEVER SETTINGS were used to test each firewall could be added in a "Settings and Configuration Description" section. He could have a link for each firewall, basically explaining WHICH measures and actions were taken to "tweak" each firewall to configure it to it's "maximum/optimal security setting".

    Call me crazy, and I know I'm asking (or suggesting) for extra work, time and effort to be made on behalf of the tester....but I think it would help "average" (and perhaps even more advanced) users to properly set-up the firewall of their choice......
     
  17. gkweb

    gkweb Expert Firewall Tester

    Joined:
    Aug 29, 2003
    Posts:
    1,932
    Location:
    FRANCE, Rouen (76)
    Hello,

    "out of the box" test was done in the past, and there was two tables, one "highest settings" and one "out of the box". It has been removed since due to the big extra work it involves.

    Currently there is 18 leaktests (26 real tests, AWFT has 6 tests, Wallbreaker 4 tests, etc...), 15 firewall tested, each test triple checked (really), that gives : 26 * 15 = 390 tests (triple check = 390 * 3 = 1000+ tests). If you add the "out of the box" table, you reach 2000+ tests. Given the fact that I will add other firewalls, I let you imagine the amount of work required.

    I am not paid to do such tests, I do them in my spare time, and I do not have so much time.

    The highest settings shows the true security potential of the firewall, I see it as a vital information. Highest settings are not so hard to do as you seem to think, basically enable every features, and disable automatic allowances.

    If you want more information about "out of the box" settings, you can read more there :
    http://www.firewallleaktester.com/advices.htm#01

    Regards,
    gkweb.
     
  18. Infinity

    Infinity Registered Member

    Joined:
    May 31, 2004
    Posts:
    2,651
    first of all, thanx gkweb for testing all those fw's. may I ask why Tiny isn't included in the test?
     
  19. starfish_001

    starfish_001 Registered Member

    Joined:
    Jan 31, 2005
    Posts:
    1,041

    I would have like to have seen that as well
     
  20. Devil's Advocate

    Devil's Advocate Registered Member

    Joined:
    Feb 5, 2006
    Posts:
    549
    I'm not interested in you doing out of the box tests, you can forget those if you like, but you can at least tell us what the 'highest settings' you used are. It's kind of like doing an antivirus test but refusing to tell how the AV was setup, what signatures were used, what samples were used etc. It makes it hard for other people to replicate your findings if they so choose to do.

    That said I agree with the following

    True. I got about the same results you have with a few firewalls, so clearly, your 'highest settings' isn't some arcane art beyond all but firewall gurus.
     
  21. gkweb

    gkweb Expert Firewall Tester

    Joined:
    Aug 29, 2003
    Posts:
    1,932
    Location:
    FRANCE, Rouen (76)
    Hello,

    To be honest, I didn't realize that some people was wanting that much "firewall tutorial settings", and I understand your concerns.

    I have added that to my "To Do" list, but honestly I don't know when I will be able to do it. If I do a page for each firewall, taking screenshots and commenting what to do, if I have 5 screenshot * 14 firewalls, I will have 70 screenshots + comments, that is a big work also, in addition to the tests.

    Just for information, today I have passed my entire day on things related to my website. I have results to check with what people tell me, I am in contact with firewall vendors about current private firewalls build which should be updated next month, I am grabbing and compiling every people opinion on various forums and am preparing to modify some things on my website, etc... I have planned to do some personal things today and I didn't even have the time.

    So I take seriously your request, but to achieve what you (and many other) want, either it will be asking again some time, or either I will have to request help from other people (of course checking the tutorial before validating it).
    For now I cannot do it since I am working on my next website update, but be sure I will not forget it :)

    In the meanwhile, I can suggest to anyone interested to check the official firewall forum and request there how to tighten their firewall security.

    Regards,
    gkweb.

    EDIT : about Tiny, it is in the reward page, and so cannot be in the scoreboard. But the next website update should change this point, and I plan to test it later.
     
  22. starfish_001

    starfish_001 Registered Member

    Joined:
    Jan 31, 2005
    Posts:
    1,041
    sorry I don't understand surely tiny warrants testing even if it was rewarded in 2004o_O
     
  23. <DreamCatcher>

    <DreamCatcher> Registered Member

    Joined:
    Jan 6, 2006
    Posts:
    154
    After looking at the good tests by 'gkweb' I have to say that it has kinda changed my view of Nortons firewall, compared to what I used to think of it.
     
  24. sweater

    sweater Registered Member

    Joined:
    Jun 24, 2005
    Posts:
    1,674
    Location:
    Philippines, the Political Dynasty Capital of the
    I am not surprised why Jetico becomes no. 1 in the test. I have tried and used this firewall in the past and it "bombarded" me with lots of pop-ups, I know that its just one of the normal character of the best protection software nowadays and that is to inform pc user what really is going on inside the system. Getting indepth on it this firewall was really designed for advanced users as I cannot understand clearly how it works and how to configure it but I know that it's one of the best. Still, I don't have enough patience on its machine gun like pop-ups then I have to uninstall it and never have returned on using it again. :oops:

    AppDefend looks more colorful than ProcessGuard that's why gkweb choose this thing (just kidding :D ) Seriously, even if AppDefend was still in beta stages it already looks very promising and offers a lots of things that probably PG misses. Its Network access control was I think was one of its best features not found on PG. I was anticipating that PG will improve from its present protection capabilities and I was hoping that it'll also adds some more features not found in AppDefend.
     
  25. puddingalien

    puddingalien Registered Member

    Joined:
    Dec 16, 2005
    Posts:
    21
    If the settings could be shown for, say, the top 3 firewalls in the test, maybe that would be a more reasonable request for now?

    Of course, if the firewall uses an almost constant 90 or 100% cpu (and also gives a mound of pop-ups that regular users can't understand) then who needs adware: I just installed it, myself.
    (points at Outpost :mad: :thumbd: :eek: )

    So, it would be interesting to see in the test the amount of resources used by each firewall.o_O :gack:
     
Loading...
Thread Status:
Not open for further replies.