New Matousec Firewall Challenge

Discussion in 'other firewalls' started by guest, Nov 28, 2008.

Thread Status:
Not open for further replies.
  1. hammerman

    hammerman Registered Member

    Joined:
    Jul 14, 2007
    Posts:
    283
    Location:
    UK
    I totally agree with your views and have sent Matousec an e-mail arguing that it is totally unfair for Mamutu to be included in a 'firewall' test.

    Having said that, would you agree that some tests would apply to Mamutu such as the keylogger tests. Presumably you would expect Mamutu to perform well against these.
     
  2. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    So how to test Mamutu? On what triggers it respond? Does anyone test it against real malware? And can I safe surf and execute my downloads, or should I pick my executions eg. trojan downloaders I can execute and rootkits never etc.?
     
    Last edited: Nov 29, 2008
  3. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Someone mentioned earlier that Matousec's tests are aimless. No, they aren't. He seeks to become the "Av comparatives" for "non antivirus" products. And this, in a way or another, brings money income...

    WinPatrol Plus is supposed to protect too from some keyloggers and trojans. So, what? It's gonna be included to this sharade too?

    Why must just ANY application pass through "testing" from Matouseco_O It is obbligatory?

    If someone wants to see if Mamutu is useful to him or not, he can do something simple and more realistic. Put Returnil or similar on his PC , go and download some malware and see if Mamutu intercepts it. It's much more realistic that running leak tests.

    But of course, today, where everything is about money and the customer must be a brainless humanoid without personal opinion about a product without having to rely onto a "test", Matousec sounds "indispensible"...

    What will be of us without testers? Oh my!

    PS. I am not a Mamutu or other Emsisoft product user. I am just fed up with "tests" and people running like sheep to cheer about their "winner" product, but who oddly enough don't feel so secure as the test would imply, so they add another gazillion security applications. :blink:
     
  4. guest

    guest Guest

    First of all i want to say that thanks to matousec all win.
    Thanks to him comodo has discover today some bugs and they are going to release an update the next week to get 100% score.

    Other apps like like outpost firewall, OA, ZoneAlarm,... take this test for improve their software, thanks to this the final user wins.

    For all of this, I want to give thanks to Matousec for his work.

    I want to ask.
    What kind of security offers Mamutu if it can not be able to block that another application close it, like a virus or any malware.?

    This leaktest represents "all" the ways that the virus, malware... take to infect a computer. No?
    What kind of behaviour blocker is Mamutu if it can be able to detects this ways...?

    Maybe this test are not for mamutu but I think that emisoft have a lot of to learn from this results and at least fix bug (because this in a security software is a bug) that another app can close it.
     
    Last edited by a moderator: Nov 29, 2008
  5. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Ah, well... Until now, I still had some weak feeling I might be missing something about their tests; now it's completely clear. The guy's just an incompetent person who's earning money via his flawed "tests" and that's the only purpose of the entire show, exactly as I've suspected for quite some time.

    Oh well, time to move on. Indeed, absolutely no point in wasting more time with this. :rolleyes: o_O :gack: :thumbd:
     
  6. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    Problem is that there is no standard "QoS" procedure, there is no benchmark or anything else which will determine quality for non signature based protection programs, when you purchase any of that kind of software you actually bought "cat in the sack".

    When I see Matousec tests I see some tries in that direction, is he wrong person to do this I dont know, but his attempt to establish benchmark is step in something that noone is done yet, and it should be respected.
     
  7. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    In opposite to loud bashihg I hear I try to have a common sense. That is to say SSM and DSA performed better than let us say WebRoot firewall (true firewall). Generally I don't care about rating, but some tests are "generic" for every security product (for example crash and kill tests) and I regard them "mandatory". If product fails mandatory tests than there is nothing to talk about.
     
  8. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Right; the ideal security product per Matousec is one that cannot be ever killed. I.e. one that requires a user to pull the power cord to be able to reboot the box. Seriously, about 1/3 of the tests are (vastly local) termination tests, that's a wonderful firewall testing methodology, isn't it? And yeah, lets drop the TCP/UDP performance tests for a good measure, that's been the only relevant test left wrt firewall functionality. Well done. :thumbd: :thumbd: :thumbd:
     
  9. 3xist

    3xist Guest

  10. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Well, I really don't care whether they retest the new build or not; wouldn't even waste time with this any more if I were a CIS developer. Though I appreciate Comodo devs frankly stating they are not changing things just to pass completely broken and irrelevant "tests". :thumb:
     
  11. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    It is obviouse that matou tests are security oriented. Peformance has nothing to do with security. As for me, I can live with some slowdown, because there is nothing critical with 0.5s difference in a page load when I visit Wilders, for example. But since I have a lot of sensitive info on my computer I could not live with security that can be killed, crashed or bypassed using WELL KNOWN tricks.
     
  12. 3xist

    3xist Guest

    I completely agree with you. I like Comodo because of that, They like protecting there end users and not worrying about "leak tests". It's good they stand strong and conduct them selves like that.
     
  13. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Huh? So, a security suite that makes your box unusable (like certain yellow products) just rocks, right? It doesn't matter that you won't be able to use your box in a productive way any more, because it will take 15 minutes to boot and that it will take ages to start any application and your system will be perpetually short of RAM. Wow, actually if you bury the entire box into 30ft layer of concrete, that's the ultimate security. Noone will ever be able to misuse it. Now, lets get real and focus on the need of a real user - install similar junk on a bunch of boxes and prepare yourself to run fast when users turn the box on. :rolleyes:
     
  14. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    If Comodo can be bypassed by published technique, it is hardly can be said it cares about its users. Neither it does care about Comodo usability. It only cares about PR, and this is why they release their own leak tests with the only aim to make you think Comodo is the best in passing leaktests.
     
  15. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Yeah, such as system reboot or user logout... Wow, what a dangerous technique. :D :D :D Or by sniffing unencrypted traffic which you can do from any other box without any trouble. What a great firewall testing. :rolleyes:
     
  16. Leolas

    Leolas Registered Member

    Joined:
    Jun 18, 2008
    Posts:
    58
    Location:
    Modena, Italy
    Well, call it HIPS testing, if you prefer, but it's always a testing..

    BTW, you're making fun of the reboot/logut testing, but when a virus doesn't let you to log in your system (or always log out) it's very annoying ;)
     
  17. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,514
    Location:
    Annie's Pub
    Security is a serious issue and need serious treatment. Matousec isn't the appropriate person to guarantee such, he clearly disqualified himself :rolleyes:

    Regards,

    Smokey
     
  18. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Nope. I do not use yellow products, I use the products I trust in and performance in leaktests is important criteria (though, this is not enough, because I also value usability, which some good test passers lack).
     
  19. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Kill1
    kill2
    Kill3b
    Kill3f
    Suspend1
    Kill3e
    Kill9
    Suspend2
    Crash1
    Crash2
    Crash3
    Crash4
    Crash6
    Kill12
    Kill5
    Crash7

    These are the tests Comodo failed. Do you see here something about reboot or logoff ?
     
  20. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    I'd suggest re-reading this post and the Comodo forums thread referred to therein.
     
  21. Leolas

    Leolas Registered Member

    Joined:
    Jun 18, 2008
    Posts:
    58
    Location:
    Modena, Italy
    No wait: you can say that Matousek (the man) isn't a serious tester, and that he's testing software he shouldn't test (he says it's a firewall challenge, even if it's an HIPS challenge to be more publicized), but matousec's tests are very good tests: he's the only independent tester of HIPS, and we have to thank him, if now a lot of firewalls contain an HIPS module, and are much much safer than they were some years ago. ;)
     
  22. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I don't need demagogy, sorry. I see the result and this is enough for me. I accept there could be mistake in the code, yes, but the fact that if not Matou test this mistake would live there for nobody knows what time is frightening. I always regarded Comodo as "making show" product. The more I see the more I believe I'm right. Loud show, low usability. The only benefit is it is "free forever".

    PS. BTW, Comodo should be thankful to Matou for finding bugs in their code.
     
  23. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    Frankly, i thought we buried this on the keylogger tests....
     
  24. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I dare to say his test suit is the most professional suit available. And this is open source. His methodology is questionable, yes, but his professional level can hadly be questionable. 84 open sorce excellently coded tests. Is there anything to compare to ? Is here anybody who is able to make something in the line ? It is easy to critisize, it is hard to do better.
     
  25. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Hmm, can we just comment on the methods instead of the results of the products?

    ~Comments removed. - Ron~

    Matousec should really remove mamutu and retest cis. If he doesn't listen to what the developers want than his tests are obviously more about money than fairness. I bet he would remove mamutu if they paid him. He might as well join toptenreviews.com :).
     
    Last edited by a moderator: Nov 29, 2008
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.