Did you choose your firewall according to Matousec ?

Discussion in 'other firewalls' started by bollity, May 10, 2009.

Thread Status:
Not open for further replies.
  1. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I did my research on matousec tests. After I did it I realized that the tests are developed very professional and the tests themselves are perforemd very professional. So for now, until some product scores at least 80% at matousec it is out of my attention too. Not because I mindlessly trust matousec, but because I value the tests themselves and find them much better than just "oh, I installed this and that and I'm happy" exlamations. From the other side I have not a time to test EVERY security out there by myself. I do it only for the software that fits my initial requirements.

    And yes, you can laugh visiting matousec site, but this only does mean you are not professional enough to understand a real value of the tests for security. All the tests use absolutely real system vulnerabilities that can be (and mosly are) exploited in the real malwares.
     
  2. the Tester

    the Tester Registered Member

    Joined:
    Jul 28, 2002
    Posts:
    2,854
    Location:
    The Gateway to the Blue Hills,WI.
    I don't choose a firewall according to Matousec test results.
    I generally select a firewall from positive feedback here at Wilder's.
    Whether I keep it or not has more to do with port scan results at Shields Up and how the firewall runs on my computer.
     
  3. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814



    That part made me chuckle, Reminds me of the Lamb following the master to slaughter. But meh what do I know. :rolleyes:
     
  4. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Certainly not. I know Look'n'Stop is a good firewall, and had it long before I even knew of the existence of Matousec. According to his tests it is a bad firewall, but they are really testing HIPS applications rather than pure firewalls.
     
  5. LoneWolf

    LoneWolf Registered Member

    Joined:
    Jan 2, 2006
    Posts:
    3,782
    Short answer: No
     
  6. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Do not hesitate to discuss ANY particular test and the ways it can be exploited in a real malware. C'mon, please. I do like to discuss technical things and I hate when people just discuss their feelings :)
     
  7. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    http://www.matousec.com/projects/proactive-security-challenge/results.php
    "Proactive Security Challenge".

    Matousec do not say LnS is a bad firewall, it only says LnS has nothing to do with Proactive Security and has a weak outbound protection and selfdefense.

    ===
    However, it is crucial to know what does it mean if a product succeeds in our tests and what does it mean if it fails. Before you start interpreting the results, you should be familiar with the information on the index page, especially with the methodology and rules. You should also know what kind of products do we test before you start to interpret the results. We have received a lot of reactions from people who are not familiar with that information and simply do not understand the results and misinterpret them. All the tested products have one common feature – the application-based security model. In combination with their packet filtering capabilities, the tested products attempt to block attacks from other machines on the network as well as attacks performed by malicious codes that might run inside the protected machine. This is definitely not an unusual situation. People who use email clients, instant messengers, or web browsers face attacks that exploit the vulnerabilities in this kind of software very often. It happens that a malicious code gets inside the machine. And then it may try to install itself silently to the system, to steal users' data or sniff their passwords, or to join the target machine to the botnet. This is what the products we test want to prevent. This is why they are used. The problem is that although the goal is common, not all the products implement a sufficient protection.
    ===

    I wonder, why people try to interpret the results not being ready to ? (not spending just 10 mins to read about how to interpret the results)
     
  8. Keyboard_Commando

    Keyboard_Commando Registered Member

    Joined:
    Mar 6, 2009
    Posts:
    690
    When I first bought a computer ... seeking guidance on todays computer security apps - YES. I'd probably follow a website like Matousec.com's results and choose accordingly. Obviously going for the top spot applications.

    I've since learned SOME of these established advisories abuse their clients/readers' trust. So now - NO.

    I Research research research what I let onto my computer now. I trust security enthusiasts that like to pick these things apart. I now prefer the cynical approach.

    I do think there is a place for reviewing sites, such as Matousec.com, though. Be nice if some of these sites could remain somewhat independent of the Bull-Shhh.
     
  9. Dwarden

    Dwarden Registered Member

    Joined:
    Apr 11, 2003
    Posts:
    177
    Location:
    Czech Republic
    malware authors today take any working PoC (proof of concept) code, tear it apeart and implement either unchanged or drastically improved and redone to fit theirs needs ...
    so if there is somewhere 'working' attack vector even as part of some 'test' consider it already being part of badware creation suite ...

    claiming M tests are useless is laughly and selfish as it equals claiming any other tests are useless ...
    they got purpose as part of complex mosaic ...
     
  10. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814
    Yes John.. Marketing for 1000$ please. :cautious:

    Don't get me wrong test have there place in how they work and what they are used for. But lets not forget this is the same site that would change numbers to fit his own agenda or pocketbook. I have no problem with tests, Just sites that change how things look depending on how much of a "Investment" you feel like paying to them. or adjusting your numbers and running out of date software if you did not pay.

    This site has been a joke from day one. If this site is your one stop spot for firewalls your going to need more then a firewall to protect you.
     
    Last edited by a moderator: May 15, 2009
  11. arran

    arran Registered Member

    Joined:
    Feb 5, 2008
    Posts:
    1,156
    No I didn't choose my firewall according to matousec results.

    matousec is based on Leak tests. It is mainly for Noobs who expect to get infected with Malware, and they don't want the malware to make any out going connections.

    But a lot of people here at Wilders can prevent malware infection from happening in the first place.

    So my question is. If we don't have any malware on our pc's trying to make outgoing connections then why do we need a firewall with strong out bound protection?

    It would be better instead to use a firewall which can filter the packets of the allowed traffic on the allowed ports, like LNS.
     
  12. nielsson

    nielsson Registered Member

    Joined:
    May 13, 2009
    Posts:
    18
    You want to limit even legit applications traffic out as much as possible to prevent security compromise from hardcore hackers.
     
  13. SamSpade

    SamSpade Registered Member

    Joined:
    Oct 22, 2006
    Posts:
    415

    I think we can see from those who have chimed in here on this question that no one uses Matousec as the *only* source for making a decision; and we also see that the overwhelming majority of people most certainly do NOT depend on *only* a firewall for protection.

    The fact that most "firewalls" people use today are in fact only *one part* of their overall security solutions proves also that "firewalls" of today are not your older brother's firewalls anymore. People who prefer a "pure firewall", with no HIPS or antivirus or other stuff, will also have supplemental strategies for protecting their systems. Those who prefer "suite-type firewalls", which do include behavior blockers, HIPS, AVs, and/or other measures do so because it's what they see as the better way.

    In either case, Matousec is showing what these most widely used "firewalls" can and cannot do. It's up to people to choose what kind of firewall they will employ to do the kind of security they wish to have. So, in that way, Matousec does provide a vauable service in showing what the popular "firewalls" do and do not do. From there it is up to each to choose what will do with that information.

    Sam
     
  14. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814
    In a way it provides misinformation as the scoring system they have only scores hips not the firewall so in all sense of the word they are misleading people. Putting firewalls in a "Fail" section that do there job well only because they don't have his VISION of what a firewall should be. :cautious:

    Hence why they have now renamed the tests to Security test or some stupid thing like that and dropped firewall from it, but for years this was not the case.
     
  15. arran

    arran Registered Member

    Joined:
    Feb 5, 2008
    Posts:
    1,156
    But most people on these forums do have the same type as my setup, not by the same product but in terms of how strong the security setup is.

    I totally agree that you should limit legit applications traffic as much as possible. However you don't normally need strong out bound protection to limit legit applications, any firewall can limit legit applications. Generally Its only malware which tries to bypass firewalls by obscure ways.
     
  16. SamSpade

    SamSpade Registered Member

    Joined:
    Oct 22, 2006
    Posts:
    415
    OK, I see your point. Matousec was calling it a "firewall test" when in fact it has been much more than a firewall test ( ... *depending on* how we define a "firewall". :blink: ) Matousec was defining a "firewall" as an app which not only does connection control and/or packet filtering, and other strictly "firewall" concerns (uhh-oh), he was also using such issues as "calling home" or making unauthorized *outbound* connections; and I see his point on that, too.

    Well, now that the new people who bought out Matousec have done a re-name of the service to something it is hopefully more accurate, it should remove most/all of the criticism over the term "firewall test". Let's hope!!

    "Toward a more perfect union.... "

    Sam


    |||
     
  17. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753

    Actually if you look at Matousec's test, he defined as firewall, ANYTHING, that would do everything EXCEPT from connection control and packet filtering. This is how Mamutu ended up there. Or SSM which was "reccommended" was doing what real packet filtering? Or it was just a simply access control for application outbound? Did SSM stealth ports? Can Mamutu filter ANYTHING?

    These were the tests:

    Level 1 – Breakout2, Coat, ECHOtest, Kill1, Kill2, Leaktest, PerfTCP, PerfUDP, Tooleaky, Wallbreaker1, Yalta
    Level 2 – AWFT1, DNStest, Ghost, Jumper, Kill3, Kill3b, Kill6, Wallbreaker3, Wallbreaker4
    Level 3 – AWFT3, AWFT4, DNStester, Kernel1, Kill3f, Kill4, Kill7, SSS2, Suspend1, Thermite
    Level 4 – CopyCat, CPIL, CPILSuite1, Kernel1b, Keylog1, Kill3e, Kill8, Kill9, SSS, Suspend2
    Level 5 – Breakout1, CPILSuite2, Crash1, Crash2, Crash3, Crash4, Kernel2, Kernel3, Keylog2, Kill3c, Kill3d, VBStest
    Level 6 – CPILSuite3, Crash5, Crash6, DDEtest, ECHOtest2, FireHole, Flank, Kernel4, Keylog3, Keylog4, Kill10, Kill11, Runner
    Level 7 – BITStest, FireHole2, Keylog5, Keylog6, Kill12, OSfwbypass, Runner2, Schedtest, SSS3
    Level 8 – Kernel4b, Kernel5, Keylog7, Kill5, NewClass, Schedtest2, SockSnif, SSS4
    Level 9 – Crash7, Driver Verifier
    Level 10 – BSODhook, ShadowHook

    Of them, the tests are:

    - Leak tests (how to hijack another application/ exploit system processes/ parent process control)
    - Kill/termination tests (same as above)
    - Keylogging tests (!!!)
    - Hooking tests
    - Driver verification test
    - Driver installation tests

    Probably only the "DNSTester" is really relative to what we once knew that a firewall is.

    The TCP/UDP Perf tests, which were actually interesting to see how the firewall affects your connectivity in terms of sheer performance, were actually dropped (!).


    I mean, if the above battery of tests, represents the adeguate tests to test firewalls, what are the adeguate tests to rank HIPSo_O? If the HIPS can protect from DNS spoofing, stealth ports, stop fragmented packets, DOS and protect the ARP cacheo_O? Good grief!
     
  18. Dwarden

    Dwarden Registered Member

    Joined:
    Apr 11, 2003
    Posts:
    177
    Location:
    Czech Republic
    i fully agree that the definition of the test (mainly just the name part) is not 'perfect' at all ...

    yet ...

    i wonder were there any major suggestions what 'truely' firewall tests should be included into such test suite ?

    like packet spoofing or w/e ?
     
  19. cqpreson

    cqpreson Registered Member

    Joined:
    May 18, 2009
    Posts:
    348
    Location:
    China
    I only know the newest firewall from Matousec as usual.As for me,I like to know firewall's quality through my own use.
     
  20. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814
    The other problem I remember Matousec having is that it would not really test to see if any information got out. Alot of the firewalls that score so low would actually score very high because they would block the attempt for it to gain access to the web hence the firewall blocking like it should. But the only thing they test is if it installs and runs not if it is stopped. Their "testing" has so many holes in it that I will never take that site seriously.
     
  21. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I think you have mixed matousec tests with something else (may be testmypc ?)

    As far as I remember they are the most accurate available leaktests out there.
     
  22. TairikuOkami

    TairikuOkami Registered Member

    Joined:
    Oct 10, 2005
    Posts:
    3,418
    Location:
    Slovakia
    I tried Comodo Firewall based on its test, but not anymore, since I do not take leak test seriously anymore. I use Windows Firewall, obviously not #1 there. :rolleyes:
     
  23. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814

    No.. It is Matousec. But testmypc also has done nearly the same thing if not exact. Now I don't know if they still run there tests that way or not as I don't keep on top of his Testing.
     
  24. MikeNash

    MikeNash Security Expert

    Joined:
    Jun 9, 2005
    Posts:
    1,658
    Location:
    Sydney, Australia
    TestMyPC is not the same as Matousec. The tests are similar, but that's where the story ends. See my post to this thread for more details - I'll avoid a repost.
     
  25. Fajo

    Fajo Registered Member

    Joined:
    Jun 13, 2008
    Posts:
    1,814

    Mike.. I'm not comparing the 2 tests. I'm just talking about the end result of how things are scored. :D
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.