AntiLeak racing insanity?

Discussion in 'other firewalls' started by pandlouk, Jun 18, 2008.

Thread Status:
Not open for further replies.
  1. pandlouk

    pandlouk Registered Member

    Joined:
    Jul 15, 2007
    Posts:
    2,572
    Matousec Tests, have definitily brought insanity in the firewall competition.

    Do not get me wrong, I continue to believe that some basic antileak capability is necessary mainly for giving some basic information/instracting the user on how the windows applications interract with its other. This can teach the users to avoid installing cracks or at least, if they continue to use them to try isolating them as possible. Also it can help the people to be more aware that even legitimate products phone home, without their explicit permition (most of them do not include this info, not in the help and not in the EULA).

    But all this thing it has gotten out of hand. Instead of firewalls now we have a bunch of hips applications with basic firewall ability. Hips are good for the people that can understand their various warnings and know how to troubleshoot their computer in case theymake a wrong choice. They are not designed for the majority of the users.

    Firewall are transformed in powerfull hips applications that leave the users unprotected; not because they do not work as they should but because most of the users do not know what to answer at the various pop-ups. So the users give wrong answers and they toggle the protection or they acccidentally lockup their programs and in the worst case their pcs.

    For not talking about the various incompatibilities, the BSODs that they are causing and the constant access on the hard disks.

    1-2 years ago, I was helping the others to clean their pcs from malware. Now I help the others to "clean" the mess that themselves have created with the "help" of the antimalware/hips applications (antiviruses, antispywares, firewalls).

    Matousec's firewall challenged is only a hips challenged. Nothing more and nothing else. It has nothing to do with their firewalls capabilities.

    The users are the only ones that do not benifit from those tests. The only ones that benifit are matousec and the security vendors. It is only a great pubblicity system that has nothing to do with the filtering of the network traffic.

    The best firewall according to the matousec tests, would be faronics AE (or any other antiexecutable application).

    ps. just a small rant

    Panagiotis
     
  2. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,059
    I totally agree.
     
  3. Phant0m

    Phant0m Registered Member

    Joined:
    Jun 7, 2003
    Posts:
    3,684
    Location:
    Canada
    I pretty much feel the same way.


    Regards,
    Phant0m``
     
  4. Someone

    Someone Registered Member

    Joined:
    Jan 18, 2008
    Posts:
    1,106
    Hi

    I agree 100% too.

    I think what the firewall vendors really need to do is get together and all agree that Matousec is getting too extreme now and stop participating in the tests.
     
  5. wrathchild

    wrathchild Registered Member

    Joined:
    Feb 7, 2008
    Posts:
    170
    Location:
    Neoplantesis
    Good observation :thumb:
     
  6. Tunerz

    Tunerz Registered Member

    Joined:
    Jun 12, 2007
    Posts:
    96
    Location:
    Philippines
    You have said it very well. Such results begin to mislead users with those POC results that gives negative images to firewalls rated in their "protection level". In my opinion, those POC leaktests shouldn't be given too much significance in a way that their results would also be the same when tested with real malware. Shouldn't it be somewhat doubtful for this leaktests to perfectly replicate complex malware behavior? Malware writers should have begun doing a major overhaul of methods in trying to bypass outbound protection of firewalls if those POC leaktests had just perfectly duplicated malware behavior. A true malware may not be completely compared with POCs. There would always be a difference, either major or minor, between the two.
     
  7. chrisretusn

    chrisretusn Registered Member

    Joined:
    Jun 16, 2004
    Posts:
    1,322
    Location:
    Philippines
    :thumb: Great post!

    I suspect that many vendors are simply hard coding against the specific "leak test" and not the actual threat anyway so they will pass.
     
  8. jrmhng

    jrmhng Registered Member

    Joined:
    Nov 4, 2007
    Posts:
    1,268
    Location:
    Australia
    So everyone agrees. How boring! We need someone who is absolutely convinced that having HIPS features in a firewall is a must and that any firewall that isnt completely leakproof is useless and the developers need to be named and shamed for their poor programing! :p:p
     
  9. hany3

    hany3 Registered Member

    Joined:
    Dec 2, 2007
    Posts:
    207
    firewall leak testing is nothing but HIPS testing
    all the advanced firewalls like comoda , OA ect.. are nothing without their hips module

    evidences of my opinion

    1- currently the 2nd place in matousec firewall challenge is "prosecurity" 93% which is not even a firewall , and i think this is funny :D
    also safety system monitor "hips only " is ranked as "good
    while most of the pure firewalls marked as "none"

    2-in comodo 3 with the D++ disabled , it's very very easily killable by the task manager , this the point which inforced comodo vendor to maintain some of the D+ activated even if u choosed the basic firewall without hips
    can be very easily terminated by the weakest trojan in the world
    so comodo without its hips module , is weaker than the built in windows xp firewall

    3- such leak test challenge measure the hips power of the firewall , and ignore completely other essential aspects such as

    A) flexability , usability and ease of use
    B) bugs and conflicts with othe softwares "some firewalls like outpost and comodo sometimes their bugs cause problems "eg. BSODs" which may be worse than a virus attack or those that can be caused by a hacker on ur pc
    c) memory usage and slow boot time" like zonalarm and OA "


    4-if u are using a AV
    so the alarm will be , virus detected
    and this is very easy , does not need advanced user
    and even ur kid will simply answer " delete"

    but the hips protection in such firewalls are fully user dependent , they only need a very advanced and experienced user
    u can face alarm xxx.exe is trying to do xxx process
    is this normal or not o_O
    i don't know
    should i allow o_O
    should i deny o_O
    can i call my experienced friend at home and ask him this difficult Question
    so , what will u do if this alarm appeared hundreds of times every day
    will u call him 100 times


    any thing which is user dependent is liable to mistakes
    it's very easy that u may answer a Q , wrong answer

    tha lazy click syndrome ,
    u just do nothing but
    allow
    allow
    allow
    allow
    allow

    may be between these "allow"s , u did a mistake
    may be u allowed a trojan to connect to outside ur pc
    the firewall gave u the alarm , but u "or even ur kid" answered the wrong answer and u got hacked "theoritically" in the presence of the number one firewall in the leak test ranking


    and sometimes i heard a funny comment from a comodo user who get bored form ansering such hundreds of alarms every now and then
    he said " if i really want to be bothered , i'll simply go downstairs and talk to my wife":D :D better than keep answering such endless Question alarms


    the fact simply is "suck leak tests measure the ability of the hips module in the firewall to alarm u about certain process in ur pc , and in the same time they ignore completely , the quality of the user's answer , is there's a possibility that the user answer the Q by the wrong answer o_O? , sure it's possible
     
    Last edited: Jun 18, 2008
  10. CoolWebSearch

    CoolWebSearch Registered Member

    Joined:
    Sep 30, 2007
    Posts:
    1,217
    Not to mention that CFP 3.0 which is supposed to be the best in Matousec's testing failed against programs with stolen rights to connect to the internet and also failed to block 2 Trojans of 10 of them from phoning home (by "PC Welt" magazine)-again real stuff some leak-tests.
     
  11. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,650
    Location:
    Hawaii
    There is a "Rogue Software" list. Someone needs to start a list of "Rogue Security Software Tests", with Matousec's nonsense first on the list.
     
  12. Diver

    Diver Registered Member

    Joined:
    Feb 6, 2005
    Posts:
    1,444
    Location:
    Deep Underwater
    I have been saying leak testing is a waste of time for years. Most users would be fine with the windows built in firewall, especially the one in Vista.

    If you want to see how badly some of these HIPS enabled firewalls muck up your system, run GMER with and without the firewall installed. Even with D+ turned off, Comodo 3 hooks into every running program multiple times.

    You might also check your commit charge with and without the firewall installed (not applicable to Vista) and then run a CPU benchmark like SuperPi on a with and without basis.
     
  13. ccsito

    ccsito Registered Member

    Joined:
    Jul 27, 2006
    Posts:
    1,579
    Location:
    Nation's Capital
    This is all about marketing. Same spiel that Bill Gates has been pulling since the 1980's. :p
     
  14. solarpowered candle

    solarpowered candle Registered Member

    Joined:
    Jan 9, 2003
    Posts:
    1,181
    Location:
    new zealand
    pandlouk deserves a cookie.
     
    Last edited: Jun 18, 2008
  15. dmenace

    dmenace Registered Member

    Joined:
    Nov 29, 2006
    Posts:
    275
    As the creator of System Shutdown Simulator (SSS),
    I believe leaktests are important in testing firewall and HIPS.
    In the future we may add Sandbox leaktests...

    Are people in this thread saying they don't want further leaktests... You have no interest whatsoever in SSS?

    So I'll just disappear...:'( :'(
     
  16. nhamilton

    nhamilton Registered Member

    Joined:
    Jul 31, 2007
    Posts:
    61
    don't disappear .. I think if people really think about it they would like the leak tests .. I believe it is more a reaction to matousec. Enougth people will refer to there results that vendors can not ignore them. But I belive they put way to much importance to the leak test and not enougth to the underlying firewall.

    Adding things like keyloggers, which have nothing to do with firewalls in my opion. I would expect to score a firewall it should be something like

    15% kill ability
    15% leak ability
    20% performance
    50% Network/application control ..

    where with the matousec results it is like 60% leak, 35% kill ability ...
     
  17. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Agree. Matousec is a waste of time. Ignore <click>
     
  18. TairikuOkami

    TairikuOkami Registered Member

    Joined:
    Oct 10, 2005
    Posts:
    2,509
    Location:
    Slovakia
    dmenace, antileak tessting in essential, but not, when it misleads people people by thinking, that it applies to firewall only. If he would rename it to antimalware aplications testing, then it would be adequate, so far it proofs only one thing, that firewall are useless as long as you have a good HIPS, which prevents malware even to start, so it can not connect to the internet. I myself am happy with a poor Windows Firewall and with no malware, nor antimallware aplications in my PC.
     
  19. dmenace

    dmenace Registered Member

    Joined:
    Nov 29, 2006
    Posts:
    275
    I understand your views precisely especially regarding leaktests being more for HIPS software than firewalls.

    This is particularly relevent when matousec classifys SSM as a firewall when it has no inbound filtering at all.

    Its just that when a user is behind a SPI/NAT firewall inbound performance is a bit useless... hence the focus on outbound performance.

    However I totally agree that I would rather have a firewall with advanced network control / logging capabilities rather than leaktest performance.

    An interesting debate!
     
  20. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    I can imagine the whole thing with leaktests started when router boxes took charge of the inbound protection. These come now with NAT, SPI and whatnot, so firewall vendors had to concentrate on the aspect of security a router could not touch - application outbound control (well, they have to make their living too). As this involved executable injections, many came with various PoCs to challenge this approach. Matousec collected these PoCs and conducted some tests, that is all. It is up to you to interpret it right.

    No. It's the other way round - it is not Matousec's fault that the users are generally ignorant and cannot distinguish between HIPS and firewall features. As it goes for any other test, a certain level of knowledge is needed to interpret it right. A test is a test, not an education tool. If you don't understand it either learn or steer clear.

    Cheers,
     
  21. emperordarius

    emperordarius Registered Member

    Joined:
    Apr 27, 2008
    Posts:
    1,218
    Location:
    Who cares
    Solution: Besides Leak testing, make a new testing consisting in:
    Putting a firewall in a system, gathering some hackers, trying to get access into the system using different ways, seeing how the firewall reacts :D
     
  22. pandlouk

    pandlouk Registered Member

    Joined:
    Jul 15, 2007
    Posts:
    2,572
    I agree, leaktests are important for both firewalls and hips.

    But for the firewall part there must be involved also somekind of network traffic.

    Matousec is misleading the users. For example SSM, comodo (with the firewall component disabled), etc. will pass those tests with flying colors.
    So where does it stand the title "firewall challenge"? It is totally unfair to the other firewalls (which might do a far better job in filtering the network traffic).
     
  23. Mrkvonic

    Mrkvonic Linux Systems Expert

    Joined:
    May 9, 2005
    Posts:
    8,703
    Hello,

    This has been done many times - it's boring.

    Remember the Vista, Mac, Ubuntu test? Well, no OS fell for brute hacking.
    The TCP/IP protocol has pretty much been unchanged since 1980, so it's rather boring ...

    Mrk
     
  24. pandlouk

    pandlouk Registered Member

    Joined:
    Jul 15, 2007
    Posts:
    2,572
    But if you consider that wifi connections are becoming the standard, personal firewalls become more necessary than ever.

    My thoughts are exactly the same. Network control and extend logging should be the primary fuction of a firewall.
     
  25. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    I fully agree with this.
    I emphasize 0% keyloggers.
     
Thread Status:
Not open for further replies.