Using leak tests to evaluate firewall effectiveness (2007 article)

Discussion in 'other firewalls' started by MrBrian, Jan 31, 2014.

Thread Status:
Not open for further replies.
  1. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
  2. Brummelchen

    Brummelchen Registered Member

    Joined:
    Jan 3, 2009
    Posts:
    1,732
    independent of content - ~snipped~

    first part - stealthed ports are a myth - pretty useless to avoid hackers and intrusions.

    second part does not have anything in common with a firewall - it describe something called HIPS. ofc HIPS is part of modern "security suites".

    major problem is that most users can not handle such complicated software and leave it on default settings or on auto.

    BTW the origin of "firewall" describes a piece of hardware outside - "in the line of fire" - to avoid trouble between a compromised system and its defense line.

    2c
     
    Last edited by a moderator: Feb 1, 2014
  3. kronckew

    kronckew Registered Member

    Joined:
    Aug 27, 2006
    Posts:
    209
    Location:
    CSA Consulate, Glos., UK
    Last edited: Feb 1, 2014
  4. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,723
    Location:
    localhost
    Yeap :thumb: Already 2 years in IT security terms is an eternity. IMO no point to post such outdated references. :)
     
  5. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    Correct me if I'm mistaken: that article is a backgrounder on the topic, providing a taxonomy of how leaks can occur. If any information in there is no longer relevant, please explain precisely why. If there are new ways to leak that aren't covered in that taxonomy, it would be great to see a reference.
     
    Last edited: Feb 1, 2014
  6. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,723
    Location:
    localhost
    As already said these leaks are outdated they go back from the pre-financial malware era. The attacks have become more sophisticated (see MRG financial malware tests) so running those leak tests to evaluate effectives of firewalls would not simply reflect the actual strength of the firewall to protect from real malware.

    Then you will have one strand of users that would simply say that the work of a firewall is packet filtering not HIPS or BB or sandboxing. So to evaluate the effectiveness of a firewall you test against malformed packets, flooding, etc. And here you can read tons of information (past posts) on why matousec tests are not representative.

    Finally, another strand of users will simply say that whatever protection of leaks you have its already too late. Infections must be prevented not blocked so the focus should be more on proactive than reactive tools.
     
  7. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    An accurate assessment. Stealth was somewhat useful back when applications and operating systems weren't constantly calling home, checking for updates. When dialup was the norm and your IP changed many times a day, a system that didn't announce itself could stay hidden for a while. Years ago when I used dialup, I had a batch file that was launched by the connection interface. It launched ID Blaster and randomized the unique identifiers of my system. When my IP changed, so did its identifiers. On DSL and other modern internet services, you can have the same IP for weeks, months, sometimes years, even when they're supposed to be dynamically assigned.

    Leaktests served a purpose when they were created. People like Matousec perverted them into marketing tools to "rate" firewalls, all the while ignoring the most important aspect of a firewall, its configuration. Kerio 2.1.5 is not regarded as a good firewall against leaktests. With its default rules, it fails most of them. By fine tuning the rules, I made it pass 2/3 of them. Blocking Internet Explorer defeated a few more. Leaktests weren't created to "rate" firewalls. They were intended to help people tighten their configuration by exposing weaknesses in them.
     
  8. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    For those that didn't like the article, which of these statements do you believe?
    1. Today's leak tests are good. The article is simply outdated.
    2. Today's leak tests are not good because they don't use the techniques used in real malware.
    3. Leak tests theoretically can never be good. (Why?)
     
  9. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    I understand that it's debatable whether a firewall should be "pure" or should include HIPS features also. My interest in the article was more along the lines of how leaks can occur and how you can test your security setup for them.
     
  10. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    The answer will depend on how one defines "firewall". I don't consider HIPS to be a firewall, even when integrated with one.
     
  11. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    There's the difference. Firewall or security setup. For testing a security package, leaktests can be useful. Using the PCAudit2 leaktest for example, it can be defeated using firewall rules that control loopback traffic on a per-application level. If that firewall has a rule that allows all loopback traffic (or doesn't control local traffic at all), it will fail the test. The test can also be defeated by preventing it from setting its hook with a HIPS. That begs the question of why one would tell the HIPS to allow an app to run, then use it to interfere with its operation. Other than a leaktest, what application would you treat that way?

    The information on the page is still basically valid, but it is dated and incomplete.
     
  12. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    Do you have an opinion on whether today's publicly available leaktests are a good test of the techniques today's malware use?
     
  13. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,723
    Location:
    localhost
    You can simply compare the results of, for example, AV-test.org on real malware and the results of matousec on leaktest. You will see that are considerable divergences. A champion in protection can be miserable in leaktests. What is then more relevant?
     
  14. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    My first preference would be to keep malware from executing in the first place. But I also have an interest in which security solutions perform well in outbound protection and behavioral detection.
     
  15. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,723
    Location:
    localhost
    Then consider that outbound protection do not necessarily mean not allowing the outbound but to protect your sensitive data. For example, by just serve junk data to stealers or simply jamming it. This is a common approach from several products nowadays. Again, you can take leaktests in consideration but their weight in terms assessing the ability of a product to protect the users is rather limited. Best to focus on the source that the end of pipe.
     
  16. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    I haven't looked at leaktests lately. Many of them probably won't run at all on my system, too old. My biggest issue with leaktests is in their usage. From a security policy perspective, the test procedures are a contradiction. It allows an app to run but interferes with its operation. When both user applications and leaktests are allowed to run, what criteria is used to decide which can set a global hook and which can't? Which can establish localhost (loopback) connections and which is prevented from doing so? For me to even run a leaktest, I have to put SSM into administrator mode and answer a bunch of prompts. I have to violate my core security policy just to try them. The same would apply to malware. In user mode, all I'd see is "Access Denied". If a malware equivalent was downloaded by another app and that app tried to execute it, It'll be silently denied. I'll see nothing. For me, there's no connection to normal usage.
     
  17. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    Do you make an exception for outbound control because it's easier to know what to allow (assuming you're not allowing apps to check for updates)?
     
  18. noone_particular

    noone_particular Registered Member

    Joined:
    Aug 8, 2008
    Posts:
    3,798
    Not including investigative tools and utilities that aren't part of normal usage, my ruleset allows 2 applications to connect directly to the internet, Tor and one instance of Proxomitron. All other internet apps are forced through them.
     
  19. TairikuOkami

    TairikuOkami Registered Member

    Joined:
    Oct 10, 2005
    Posts:
    2,508
    Location:
    Slovakia
    The article is still valid. Firewalls did not get updated, theirs HIPS did. Once the firewall works, there is nothing to update, but it needs to be updated, so it can be sold. Just look at Jetico Firewall, it was outdated for years and still was #1, but people are persuaded, that only updated products are secure. It does not apply for OS either. :rolleyes:
     
    Last edited: Feb 2, 2014
  20. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    I've compiled a list of techniques from http://www.matousec.com/projects/proactive-security-challenge-64/results.php#detailed-results:

    1. Autorun test techniques: registry location exploitation, DLL injection, disk location exploitation

    2. Leak-test techniques: in-process data substitution, direct network access, trusted process manipulation, registry location exploitation, COM interface exploitation, parent process control bypassing, code injection, remote thread creation, remote thread manipulation, network management API exploitation, windows messages exploitation, DLL injection, file/directory manipulation, binary planting, windows/event hooking exploitation, DDE exploitation, system service exploitation, indirect network access, system object manipulation, Windows Management Instrumentation API exploitation

    3. Self-defense test techniques: file/directory manipulation, remote process manipulation, remote thread creation, remote thread manipulation, system object manipulation, registry key/value manipulation, remote process memory manipulation, code injection, trusted process manipulation, system service exploitation, remote process handles manipulation, registry location exploitation, windows messages exploitation

    4. Spying test techniques: windows/event hooking exploitation, registry location exploitation, keyboard API exploitation, windows clipboard API exploitation, network API exploitation, graphics API exploitation

    5. Other test techniques: file/directory manipulation, system service exploitation, Windows Filtering Platform API exploitation
     
  21. wat0114

    wat0114 Registered Member

    Joined:
    Aug 5, 2012
    Posts:
    1,983
    Location:
    Canada
    I haven't looked at all these latest leak tests that could be used. the feeling I've had over the years, though, is that nothing elaborate is ever used. It's just a malicious process that attempts outbound comms, which most any properly configured application firewall will easily stop.

    Take Wilders member Rmus who has posted so many screenshots of malware attempting to communicate to the outside world in his testing, and his retro Kerio 2.x stops them all. That really seems to prove nothing overly fancy is used by hackers to bypass firewalls.
     
  22. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    Examples:
    Inside Trojan.Clampi: Bypassing your Local Firewall
    King of Spam: Festi botnet analysis
    Malware Update with Windows Update
    Backdoor:W32/Hupigon
    Troj/Salload-D
    Trojan:Win32/AgentBypass.gen!K
    Case study: TDSS Rootkit
    Analysis of a dll injector – Trojan.Win32.Inject.dnz
    Analysis Reveals Flame Malware’s Process Injection Tricks
     
    Last edited: Feb 2, 2014
  23. wat0114

    wat0114 Registered Member

    Joined:
    Aug 5, 2012
    Posts:
    1,983
    Location:
    Canada
    Thanks MrBrian. I didn't look at them all but it seems the few I looked briefly at are contingent upon running with elevated rights to be successful. Also if someone's going to deliberately execute one without knowledge of what it is, it's game over anyway. It's like you mentioned earlier: "don't allow them to run in the first place". I remember in Rmus' examples he's running with limited rights, so there are no elaborate injections taking place or registry writes, or malicious files with common names like svchost.exe or iexplore.exe, etc...A setup with limited rights and especially fortified further with application whitelsiting is going to limit these malicious bypass attempts to the mainstream types which are easily stopped with a properly configured basic application firewall.
     
  24. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    You're welcome :).

    Some of these would be stopped by limited user privileges (assuming no privilege elevation is possible), but I think some also would not be stopped. Consider what happens when a legit application encounters an exploit.
     
  25. wat0114

    wat0114 Registered Member

    Joined:
    Aug 5, 2012
    Posts:
    1,983
    Location:
    Canada
    Sure, this could be debated at length as to how successful this type of malware encounter could be, but it really all comes down to one's defenses in place.
     
Loading...
Thread Status:
Not open for further replies.