New Matousec Firewall Challenge

Discussion in 'other firewalls' started by lordraiden, Nov 28, 2008.

Thread Status:
Not open for further replies.
  1. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,085
    Last edited: Nov 28, 2008
  2. firzen771

    firzen771 Registered Member

    Joined:
    Oct 29, 2007
    Posts:
    4,815
    Location:
    Canada
    i dunno if u know this but Mamutu is a behavior blocker, not leaktest blocker. and tbh i have no idea why they would put Mamutu in a test like this, its pointless. Behaviour blockers are not built to stop leaks, just ACTUAL malicious behaviour.
     
  3. Antarctica

    Antarctica Registered Member

    Joined:
    Feb 25, 2003
    Posts:
    1,620
    Location:
    Canada
    I taught Mamutu was a Behavior Blocker.o_O
     
  4. firzen771

    firzen771 Registered Member

    Joined:
    Oct 29, 2007
    Posts:
    4,815
    Location:
    Canada
    it is a behaviour blocker, thats why its pointless to include in a test like this, but im sure somebody will see this and think Mamutu is a terrible product simply because of this leaktest.
     
  5. Leolas

    Leolas Registered Member

    Joined:
    Jun 18, 2008
    Posts:
    58
    Location:
    Modena, Italy
    Happy for OA, but this time no HIPS/firewall has a really excellent score.

    I'd have expected to see more softwares over 90%
     
  6. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Exactly. It is the most "Not recommended" firewall. o_O A company is getting negative pubblicity on one of their products for "failing" in a test, where the product shouldn't be competing in the first place. :blink:
     
  7. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Hmm, i think comodo forgot about leaktests or something. They got lower than pctools. Atleat my KIS is doing great :).
     
  8. firzen771

    firzen771 Registered Member

    Joined:
    Oct 29, 2007
    Posts:
    4,815
    Location:
    Canada
    OMG, somebody is finally worrying about actual effectiveness over gettin 99% on a leakesto_Oo_O
     
  9. Leolas

    Leolas Registered Member

    Joined:
    Jun 18, 2008
    Posts:
    58
    Location:
    Modena, Italy
    And so why would they release leak tests as well?

    I think a program shouldn't be developed "around" a leak-tests, but if a soft can pass a leak test, it also means that this soft can protect against malwares that work like the test.

    But yes, having a 100% on matousec also means a lot of advertising
     
  10. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    While good performance on the leaktests doesn't prove s/w is the same good against real malware, bad performance in leaktests definitely says that s/w can be easily bypassed and it's protection and monitoring is weak. Leaktests are nothing, but bypassing techniques. Another question is why Comodo releases its own leaktests if it doesn't care ? Oh, I've got it .. they only care about their own leaktests, because they know how to protect against them ! :)
     
  11. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Perhaps this one should be merged with the other thread. And yeah, examples like this actually prove Matousec really completely lost sanity... These tests don't make sense; their are testing less and less of firewall functionality and now even include products that are neither firewall nor HIPS. Ugh. o_O :thumbd: :gack:
     
  12. SystemJunkie

    SystemJunkie Resident Conspiracy Theorist

    Joined:
    Mar 3, 2006
    Posts:
    1,500
    Location:
    Germany
    I agree with this, Mamutu is like a shallow shield that does not exist. Comodo lost ground, their peak time seems to be over since they have started this security suite. Very good to see the efforts of PCTools Firewall I was always sure that this would become a firewall with lots potential especially if you make the correct settings.
     
    Last edited: Nov 28, 2008
  13. Carver

    Carver Registered Member

    Joined:
    Feb 5, 2006
    Posts:
    1,827
    Location:
    USA
    Yes I do believe it has, I am now using OA Free. The web site www.matousec.com list the latest free Firewall as Online Armor Personal Firewall 2.1.0.131 FreeFREE, but the GUI of the Firewall itself says 3.0.0.190.Free. it seems that matousec got the version wrong.
     
  14. 3xist

    3xist Guest

    Hi Alex.

    I noticed CIS got 84%. Were all products tested by default or set on highest possible settings? If highest possible, If you set CIS to Proactive Security, it will achieve higher results. This can be done by right clicking system tray icon Configuration>Proactive Security. This activates ALL Defense+ Settings, Because by default only some D+ Settings are active due to AV architecture, and some other technology to help improve usability. Anyway Proactive Security should really bring it back up again.

    But if it's default... don't matter. :) Cangrats on the higher products! :)
     
  15. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,085
    Interpretation of results:
    However, Online Armor Personal Firewall 2.1.0.131 Free was not tested against the latest version of the tests, hence it is likely that its comparable result would be worse.
     
  16. Leolas

    Leolas Registered Member

    Joined:
    Jun 18, 2008
    Posts:
    58
    Location:
    Modena, Italy
    My question is: why didn't he test v3, then? o_O
     
  17. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    Outpost failed at very dangerous spots, driver installations (read - rootkit installations), I am actually very glad to see that results, as I know Agnitum will add extra (hmm... essential) protection after this tests... Matousec should check ADS in next challenge update ;)
    Also I cant believe Comodo failed at crash, kill ... tests, there must be something wrong in configuration or there is bug maybe in latest build...

    Edit: closer look shows some bugs when Outpost tried to block unwanted detected behavior
     
    Last edited: Nov 28, 2008
  18. subset

    subset Registered Member

    Joined:
    Nov 17, 2007
    Posts:
    825
    Location:
    Austria
    From the website:
    "The products are configured to their highest usable security settings and tested with this configuration only."
    http://www.matousec.com/projects/firewall-challenge/#methodology-rules
    Apart from whatever "highest usable security settings" could possibly mean... I would be surprised if Mr. Matousec does not share his chosen settings at least with the vendors of a tested program. o_O

    Cheers
     
  19. 3xist

    3xist Guest

    Okay.

    Well here is what we have so far from the forums:

    So there is a small bug causing CIS's results, and an update will come hopefully early next week.
     
  20. Kyle1420

    Kyle1420 Registered Member

    Joined:
    May 27, 2008
    Posts:
    479
  21. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I always said this is not a problem to build paranoid product that can catch everything, can stop everything (including good harmless programs), but to be completely unusable. The main problem is not to catch everything, but to catch everything and be usable at the same time. Comodo is very good with the higest settings, but with those settings it is completely unusable. And if it is set to low popups mode, it goes insecure. This is very huge desing flaw. Most people do not understand what is the difference between different Comodo modes and are not protected thinking they are.
     
  22. 3xist

    3xist Guest

    Hi Alex.

    As mentioned before there is a small bug causing these results. Should be fixed next week.
     
  23. deanmartin

    deanmartin Registered Member

    Joined:
    Sep 6, 2007
    Posts:
    231
    Location:
    USA/KY
    Mamutu ?, Next round Threatfire & Spybot Search & Destroy.
     
  24. Rain_Train

    Rain_Train Registered Member

    Joined:
    Aug 27, 2008
    Posts:
    139
    I'm surprised about PC Tools Firewall. Isn't there a version 5 in the making? Does Matousec test beta products?
     
  25. 3xist

    3xist Guest

    Oh, by the way the small bug in CIS has just been fixed! :)
     
Loading...
Thread Status:
Not open for further replies.