Matousec RETEST !! COMODO & Online Armor DO NOT pass 100% !!

Discussion in 'other firewalls' started by Happy-Dude, May 17, 2008.

Thread Status:
Not open for further replies.
  1. Happy-Dude

    Happy-Dude Registered Member

    Joined:
    Aug 28, 2006
    Posts:
    54
    Location:
    United States of America
    Heya guys !!

    Matousec updated their tests today (05-17-08 ) and decided to retest COMODO and Online Armor for their latest rounds of test to see if they still score 100%. Apparently, they did not, but COMODO FWv3 still came out on top (95%) !! Online Armor now comes out at 89%, and their team response told of working on fixes to their latest versions.

    http://www.matousec.com/projects/firewall-challenge/
    http://www.matousec.com/projects/firewall-challenge/results.php
     
    Last edited: May 17, 2008
  2. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,057
    Oh gosh. Now I am only 89% protected against theoretical threats. The only thing that comes to mind is the famous line from the movie Gone with the Wind.
     
  3. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Oh, my! They didn't reach 100%? I 'm afraid to ask what my Kerio 2 scored then. :D

    Instead of looking only at leak test results, you should look at the new "PerfTCP" and "PerfUDP" tests, to realise how much this insane leak test competition has impacted the basic job of a firewall, handling packets.

    It's why i 've gone back to Kerio 2. Browsing and p2p is visibly faster compared to the "leak test champions"...
     
  4. wat0114

    wat0114 Guest

    Oh well, here we go again, more scrambling by these vendors to "patch" their products so they can re-claim their perfect scores :rolleyes:
     
  5. InfinityAz

    InfinityAz Registered Member

    Joined:
    Jul 23, 2005
    Posts:
    828
    Location:
    Arizona
    It amazes me how often firewalls are updated now and it seems often in response to these leak tests. When was the last time Windows firewall was updated?
     
  6. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Seems knowone cares about leaktets anymore :D
     
  7. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    It's more amazing to see where this leads to. It's funny, but reading Matousec's site, it appears that they broke something while updating previous releases, so they now fail in tests that they used to pass. They try to "pass" a new POC and this results on braking something else. This is getting ridiculous. I wanted to use Comodo in these days, but seems that my browsing is sluggish. Using a previous version is discouraging because of the bugs. OA has its own problems too, if you visit their forum. They sure excell in Matousec's tests, but i only want something light and simple to use as outbound control behind a router and this seems to be the last thing the current devs think of.

    Instead of concentrating on bugfixing, they are obsessed with "strengthening the HIPS part" of the firewall. Yeah, ok. In the meantime, i 'd rather use Kerio 2 which makes your browsing and p2p fly with 0% CPU usage.
     
  8. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Oh, don't worry, vendors care! That's why some have the option to buy through Matousec's link now. It's profitable for everybody. Getting a "100%" logo from Matousec, is a fast pubblicity to put on the vendor's site, just like VB for antivirus. On the other hand, vendors, have interest in Matousec to continue such tests, specially if their product is designed for such tests. So i would guess that both sides have economical and reputation gains from leak testing. :D
     
  9. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    17,878
    Location:
    New England
    Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs.
     
  10. Dwarden

    Dwarden Registered Member

    Joined:
    Apr 11, 2003
    Posts:
    176
    Location:
    Czech Republic
    webroot packet performance packets is quite amazing ...
    and seems both AO and CF failing new Socksniff ..
     
  11. pykko

    pykko Registered Member

    Joined:
    Apr 27, 2005
    Posts:
    2,236
    Location:
    Romania...and walking to heaven
    Agree 100%. :thumb:
     
  12. 19monty64

    19monty64 Registered Member

    Joined:
    Apr 10, 2006
    Posts:
    1,302
    Location:
    Nunya, BZ
    When they do this I may start using a firewall again. :D
     
  13. nhamilton

    nhamilton Registered Member

    Joined:
    Jul 31, 2007
    Posts:
    61
    I can understand the reason for leak tests ... in that malware could use the techniques to communicate using a valid app and not having the firewall pick up on it. I think the new TCP and UDP performance is great .. I just do not understand why a firewall should pick up keyloggers. Fine for a secruity suite or other apps .. Who care if an app can read my keystrokes, it is only if the attacker can get that data that matters.
     
  14. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    I wonder if they can afford to. There are those of us who know leaktests for what they are, but the majority of hopelessly clueless newbies have based their whole conceptions of a "good" firewall around Matousec and his silly tests. Forfeiting Matousec would probably be tantamount to forfeiting a major part of their customers' goodwill.
     
  15. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    There is nothing to worry about. I took a look at the results, the main part of the fails are keyloggers. It was declared that free version doesn't have antikeylogging protection except the very basic one. But my beta (and I think yours as well) passes all of them quite well :)
     
  16. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,057
    In case you missed it, I was being...ah.. slightly sarcastic. I know your right, but in any case don't really care. Before something can leak it has to be installed and run. Also in case you didn't know the line from Gone with the Wind is "Frankly my dear, I don't give a damn"

    Pete
     
  17. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    Hmm but why you using then product which flag/main/proud of feature is leakproof capability?:rolleyes:
     
  18. Bouvier

    Bouvier Registered Member

    Joined:
    Apr 10, 2008
    Posts:
    1
    Now THAT is a clever advise, I hope OA- and Comodo programmers will be that clever too, unfortunatelly I expect them to go for the gold, because that draws the masses to them.

    Being an OA-user myself, I still think it is one of the most flaky softwares I ever used.:doubt:
     
  19. boonie

    boonie Registered Member

    Joined:
    Aug 5, 2007
    Posts:
    238
    LOL. Actually, the first one that popped into my head was "After all, tomorrow is another day." (Another day = Another leaktest).
     
  20. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Peter uses paid version which stops all the keyloggers. Free version is officially declared not to have advanced antikeylogging. But these are irrelevant details. I can understand leaktesting, I can understand stability tests, I just cannot understand how firewalls leak protection relates to antikeylogging. It seems Matousec has lost my respect in some degree. Let us say 12.5%.
     
    Last edited: May 17, 2008
  21. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    :eek: we can rephrase that question: Why an Firewall have antikeylogger feature?
     
  22. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,047
    Location:
    Saudi Arabia/ Pakistan
    I totally agree! :thumb: :thumb:

    This testing is going to be insane now.
     
  23. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    This is completely different question. If you are uncomfortable with the name (firewall) let you call it security suit. As for me I prefer to have "all in one". I hate a lot of different software with different settings with different support. This is why at the moment I use only OA AV+ and nothing else. And even with my risky behaviour (no UAC, no LUA, admin account, downloading and starting everything I find interesting) I'm quite satisfied with the protection level OA AV+ introduces. The only other "security" s/w I use is FD-ISR for a fast recovery in case something went wrong. But I never regarded OA as just firewall, this is Firewall + HIPS + antivirus (AV+ version). So my answer to your queston is "I (paid user) want it". Another question is about leaktesting. Leak is a process when sensitive information is transferred out of your computer. So I do not see how keylogger can be regarded as leak in case it cannot transfer its log outside.
     
    Last edited: May 17, 2008
  24. nhamilton

    nhamilton Registered Member

    Joined:
    Jul 31, 2007
    Posts:
    61
    the two new test in level 1, with the performance of TCP and UDP are actually good and should have more weight. The anti keylogging is just insane.
     
  25. arran

    arran Registered Member

    Joined:
    Feb 5, 2008
    Posts:
    1,139

    if your p2p is slow it means you haven't configured comodo properly, I have no speed issues with comodo 3 and it does have some inbound protection with advanced packet filtering rules you can setup.

    Anyway the main reason why these firewalls is focusing more on leak tests is because most of us here allready have inbound protection.

    "Inbound Protection"

    1. Router firewall
    2. admuncher or Proxomitron
    3. Firefox No Script
    4. web antivirus.
     
Loading...
Thread Status:
Not open for further replies.