Using leak tests to evaluate firewall effectiveness (2007 article)

From http://www.securelist.com/en/analysis/204791977/Using_leak_tests_to_evaluate_firewall_effectiveness:

 

independent of content - ~snipped~

first part - stealthed ports are a myth - pretty useless to avoid hackers and intrusions.

second part does not have anything in common with a firewall - it describe something called HIPS. ofc HIPS is part of modern "security suites".

major problem is that most users can not handle such complicated software and leave it on default settings or on auto.

BTW the origin of "firewall" describes a piece of hardware outside - "in the line of fire" - to avoid trouble between a compromised system and its defense line.

2c

 

that article is 5+ years old. software firewalls have evolved since then and include defence against the majority of exploits.
Outpost has had anti-leak protection for many years now.
http://i153.photobucket.com/albums/s231/kronckew/stuff/Capture003-1.jpg

 

Yeap Already 2 years in IT security terms is an eternity. IMO no point to post such outdated references.

 

Correct me if I'm mistaken: that article is a backgrounder on the topic, providing a taxonomy of how leaks can occur. If any information in there is no longer relevant, please explain precisely why. If there are new ways to leak that aren't covered in that taxonomy, it would be great to see a reference.

 

As already said these leaks are outdated they go back from the pre-financial malware era. The attacks have become more sophisticated (see MRG financial malware tests) so running those leak tests to evaluate effectives of firewalls would not simply reflect the actual strength of the firewall to protect from real malware.

Then you will have one strand of users that would simply say that the work of a firewall is packet filtering not HIPS or BB or sandboxing. So to evaluate the effectiveness of a firewall you test against malformed packets, flooding, etc. And here you can read tons of information (past posts) on why matousec tests are not representative.

Finally, another strand of users will simply say that whatever protection of leaks you have its already too late. Infections must be prevented not blocked so the focus should be more on proactive than reactive tools.

 

An accurate assessment. Stealth was somewhat useful back when applications and operating systems weren't constantly calling home, checking for updates. When dialup was the norm and your IP changed many times a day, a system that didn't announce itself could stay hidden for a while. Years ago when I used dialup, I had a batch file that was launched by the connection interface. It launched ID Blaster and randomized the unique identifiers of my system. When my IP changed, so did its identifiers. On DSL and other modern internet services, you can have the same IP for weeks, months, sometimes years, even when they're supposed to be dynamically assigned.

Leaktests served a purpose when they were created. People like Matousec perverted them into marketing tools to "rate" firewalls, all the while ignoring the most important aspect of a firewall, its configuration. Kerio 2.1.5 is not regarded as a good firewall against leaktests. With its default rules, it fails most of them. By fine tuning the rules, I made it pass 2/3 of them. Blocking Internet Explorer defeated a few more. Leaktests weren't created to "rate" firewalls. They were intended to help people tighten their configuration by exposing weaknesses in them.

 

For those that didn't like the article, which of these statements do you believe?
1. Today's leak tests are good. The article is simply outdated.
2. Today's leak tests are not good because they don't use the techniques used in real malware.
3. Leak tests theoretically can never be good. (Why?)

 

I understand that it's debatable whether a firewall should be "pure" or should include HIPS features also. My interest in the article was more along the lines of how leaks can occur and how you can test your security setup for them.

 

The answer will depend on how one defines "firewall". I don't consider HIPS to be a firewall, even when integrated with one.

 

There's the difference. Firewall or security setup. For testing a security package, leaktests can be useful. Using the PCAudit2 leaktest for example, it can be defeated using firewall rules that control loopback traffic on a per-application level. If that firewall has a rule that allows all loopback traffic (or doesn't control local traffic at all), it will fail the test. The test can also be defeated by preventing it from setting its hook with a HIPS. That begs the question of why one would tell the HIPS to allow an app to run, then use it to interfere with its operation. Other than a leaktest, what application would you treat that way?

The information on the page is still basically valid, but it is dated and incomplete.

 

Do you have an opinion on whether today's publicly available leaktests are a good test of the techniques today's malware use?

 

You can simply compare the results of, for example, AV-test.org on real malware and the results of matousec on leaktest. You will see that are considerable divergences. A champion in protection can be miserable in leaktests. What is then more relevant?

 

My first preference would be to keep malware from executing in the first place. But I also have an interest in which security solutions perform well in outbound protection and behavioral detection.

 

Then consider that outbound protection do not necessarily mean not allowing the outbound but to protect your sensitive data. For example, by just serve junk data to stealers or simply jamming it. This is a common approach from several products nowadays. Again, you can take leaktests in consideration but their weight in terms assessing the ability of a product to protect the users is rather limited. Best to focus on the source that the end of pipe.

 

I haven't looked at leaktests lately. Many of them probably won't run at all on my system, too old. My biggest issue with leaktests is in their usage. From a security policy perspective, the test procedures are a contradiction. It allows an app to run but interferes with its operation. When both user applications and leaktests are allowed to run, what criteria is used to decide which can set a global hook and which can't? Which can establish localhost (loopback) connections and which is prevented from doing so? For me to even run a leaktest, I have to put SSM into administrator mode and answer a bunch of prompts. I have to violate my core security policy just to try them. The same would apply to malware. In user mode, all I'd see is "Access Denied". If a malware equivalent was downloaded by another app and that app tried to execute it, It'll be silently denied. I'll see nothing. For me, there's no connection to normal usage.

 

Do you make an exception for outbound control because it's easier to know what to allow (assuming you're not allowing apps to check for updates)?

 

Not including investigative tools and utilities that aren't part of normal usage, my ruleset allows 2 applications to connect directly to the internet, Tor and one instance of Proxomitron. All other internet apps are forced through them.

 

The article is still valid. Firewalls did not get updated, theirs HIPS did. Once the firewall works, there is nothing to update, but it needs to be updated, so it can be sold. Just look at Jetico Firewall, it was outdated for years and still was #1, but people are persuaded, that only updated products are secure. It does not apply for OS either.

 

I've compiled a list of techniques from http://www.matousec.com/projects/proactive-security-challenge-64/results.php#detailed-results:

1. Autorun test techniques: registry location exploitation, DLL injection, disk location exploitation

2. Leak-test techniques: in-process data substitution, direct network access, trusted process manipulation, registry location exploitation, COM interface exploitation, parent process control bypassing, code injection, remote thread creation, remote thread manipulation, network management API exploitation, windows messages exploitation, DLL injection, file/directory manipulation, binary planting, windows/event hooking exploitation, DDE exploitation, system service exploitation, indirect network access, system object manipulation, Windows Management Instrumentation API exploitation

3. Self-defense test techniques: file/directory manipulation, remote process manipulation, remote thread creation, remote thread manipulation, system object manipulation, registry key/value manipulation, remote process memory manipulation, code injection, trusted process manipulation, system service exploitation, remote process handles manipulation, registry location exploitation, windows messages exploitation

4. Spying test techniques: windows/event hooking exploitation, registry location exploitation, keyboard API exploitation, windows clipboard API exploitation, network API exploitation, graphics API exploitation

5. Other test techniques: file/directory manipulation, system service exploitation, Windows Filtering Platform API exploitation

 

I haven't looked at all these latest leak tests that could be used. the feeling I've had over the years, though, is that nothing elaborate is ever used. It's just a malicious process that attempts outbound comms, which most any properly configured application firewall will easily stop.

Take Wilders member Rmus who has posted so many screenshots of malware attempting to communicate to the outside world in his testing, and his retro Kerio 2.x stops them all. That really seems to prove nothing overly fancy is used by hackers to bypass firewalls.

 

Examples:
Inside Trojan.Clampi: Bypassing your Local Firewall
King of Spam: Festi botnet analysis
Malware Update with Windows Update
Backdoor:W32/Hupigon
Troj/Salload-D
Trojan:Win32/AgentBypass.gen!K
Case study: TDSS Rootkit
Analysis of a dll injector – Trojan.Win32.Inject.dnz
Analysis Reveals Flame Malware’s Process Injection Tricks

 

Thanks MrBrian. I didn't look at them all but it seems the few I looked briefly at are contingent upon running with elevated rights to be successful. Also if someone's going to deliberately execute one without knowledge of what it is, it's game over anyway. It's like you mentioned earlier: "don't allow them to run in the first place". I remember in Rmus' examples he's running with limited rights, so there are no elaborate injections taking place or registry writes, or malicious files with common names like svchost.exe or iexplore.exe, etc...A setup with limited rights and especially fortified further with application whitelsiting is going to limit these malicious bypass attempts to the mainstream types which are easily stopped with a properly configured basic application firewall.

 

You're welcome .

Some of these would be stopped by limited user privileges (assuming no privilege elevation is possible), but I think some also would not be stopped. Consider what happens when a legit application encounters an exploit.

 

Sure, this could be debated at length as to how successful this type of malware encounter could be, but it really all comes down to one's defenses in place.