Leaktests: Should they be considered in selecting a Firewall?

I was going to respond to many of the points in Paranoid2000's latest post, but it seems that we both are trying argue different points from unrelated forms of reasoning. I think it would be better if I did what I thought of doing originally. That is, I intend to release an online article on this subject in the near future. Hopefully, it will be easy to see my line of reasoning so my views can be picked apart cleanly. It would also be easier for me to determine which assumptions are wrong if they are criticised.

Many of the criticisms of my point of view were somewhat valid, but it would be difficult to rebutt them without explaining my line of reasoning beforehand. Hence, the point of an article to bring up each of these issues.

 

I have an idea (not original) that when you perform these leak tests on your own system, you are taking an unnecesarry risk. I don't know if this is a legitimate concern, but it seems to me there is a potential for someone either gathering information about your vulnerabilities, or, for all I know, even installing malware.

I assume all of the large commercial firewall producers do have people in there developement team that keep abreast of the latest vulnerabilities. I just hope that the developer's don't forego refining their product in favor of getting a gold star on a leak test.

I do wonder where these leaks come from and why they were not addressed prior to the release of a leak test. Is it that these are o/s quirks than no one could have predicted comes along, or is it that at the present time, for whatever reason, the firewall vender's have decided to wait for the disclosure of leaks to the public before updating their products.

I doubt anyone believes that all of these developers are as horrified as the end-user might be to find out how vulnerable he is. Not to rattle anyone's cage, but it seems to me that by the time the finish developing, testing, and distributing there products, most if not all of them, are aware of numberous bugs that they wish were not present. Not that they don't care. The cost of fixing these bugs have to be weighed against just how badly the public is going to perceive them. In the case of a not-known-to-the-public vulnerability, I would suggest that the negative impact (at present) is nil.

With so much research, it is surprising that no one has solved this problem. How can the malware writers be ahead of the security writers? How can it be easier to discover unknown vulnerabilities and then write custom and comparitively small programs that can find unknown computers with unknown security measurers that will adapt to many differnent implimentations. Perform (in most cases) a function that goes counter to (in most cases) professionally developed programs which have the luxory of having been selected for compatability with the system, then go on to neutralize their unknown in advance adversary, and perform tasks which should be being monitored?

Some of it can be blamed on poor program writing. But, and this is just my uniformed guess, the explanation is simpler. Probably, your computer is essentially secure. Only, for one reason or another you do not use the tools you have in such a was as is consistant with computer security.

I'm the worst offender. Paranoid has pointed to my ignoring the limited access users limited priveledges as protection for certain bad events. how many people have all their contacts saved in a way that invites exploitation. How often do you change your passwords? are they strong passwords? Still using Outlook Explorer are you? how many are using (microsoft doesn't count) secure encryption? Proxy servers? VPN? Ever consider powering down your external modems and routers when not in use. The list of stuff I see people doing goes on and on, and I know relatively little about windows, and much less about browsers and firewalls.

I figure these firewall people say, "...true, but the vulnerability still would not have effected them had they simply..."

I actually heard software guys say, well if they don't know 'X', they shouldn't own a computer. - Not very charitable!

My approach is just like the firewall makers in my imagined scenario. 'Oh, yeah? Well, I'll stop doing it when I get burned!'

Ghost, if you don't mind would you post here, or pm me when you finish your article. Always nice to find out what else I am doing to compromise my computer!

(wow, this is really, really strong coffee!)


-Handsoff

 

Not suprised at all and there in lies the dilema, the definition and expectations of a firewall nowadays will be as varied as the number of users.

What I look for first in a firewall, software or hardware, would be more in line with the traditional definition. A good stateful firewall which allows for full control of network traffic both inbound and outbound from the system/network. For software firewalls application control is secondary, if I use it.

So for me, leaktests are not a consideration in selecting a firewall. I prefer to focus on other measures/policies to deal with potential malware and my firewall to be just a firewall.

Regards,

CrazyM

 

Maybe I am too dense but I have not really understood the issue here especially in light of the discussions held at BBR Forums some time ago.

1.As far as kerio 2.1.5 , the matter was discussed in detail some time back at BBR Forums when BZ commented that "leaktests usually deal with something other than the firewall features" and leaktests "deal with things a real firewall does not deal with normally, a 3rd party program like a sandbox would have to run that protection. 3rd party sandboxing applications, or programs that have sandboxing built in are the ones that pass the most tests."

2.These discussions had been joined in by ghost16825 as also by gkweb - the author of 'firewallleaktester.com' - who said that " you have necessarely applications on your side allowed to do a kind of traffic, and the leaktests just show that these allowances can be hijacked, the fact it is often IE targetted is just because it is more easy to do, the leaktests aren't malwares, they are proof of concept, a real malware would be tricker of course. ." These discussions could be accessed at :

http://www.broadbandreports.com/forum/remark,10656737

3. The FAQ at firewallleaktester.com says that none of FWs pass everything, but by adding many security layers such as AntiVirus, AntiTrojan, AntiSpyware, and the most important, sandboxe and/or process integrity.In answer to vendor claims of passing all leaktests it is said that "Either your firewall vendor is just making marketting advertising, which is understandable after all, or either your product is not just a personal firewall, but rather a sandboxe including a firewall (such as Tiny, BlackIce, or BitGuard)."The documents are at :

www.firewallleaktester.com

4.The point that the original poster raised is of relevance here : as what the FW is exactly supposed to do . If it is supposed to deal with all types and kinds of intrusions\extrusions then clearly software FWs like KPFW are failures . However if we accept that FWs are basically packet filters and are characterized by its limitations as also by its capabilities then there is no problem.The latter position , from the above, broadly appears to have been already accepted by both the FW advocates as also of leaktest authors.

In view of the above the issue appears to have become redundant to me . Anyway,this is my opinion wharever it is worth.

 

what about LnS? afaik its just a personal firewall yet it blocks the majority of tests. is it just secure rules or does some of its advanced features have something to do with it?

 

If by this you mean the question has been answered as well as is it need be, then I agree.

If you mean to say the question is irrelevent, then not.

Many people want to know what makes a particular firewall better than the next, how do they choose. You can't blame people for hoping they can improve their odds for finding the one that will work best for them.

I'll bet more than one reader is thinking: 'just give me a name!'.


-HandsOff

 

Basically as far as my understanding goes , on a outward access the firewall records a note to self in its "state table" that there's an outbound (SYN, SYN/ACK, ACK, RST, etc.) alongwith header details like its destination address ,port etc. A packet is allowed in when it either corresponds to an outgoing one or matches one of the rules . In a FW like KPFW stateful inspection means examination of ports, IP's, and headers (and the flags they contain). A lot of enterprise systems analyze the entire packet including the payload. That's a sort of "real" stateful inspection .

Thus the difference in the performance of different FWs lies , besides the ruleset, on their inbuilt hard coding as also on the level of IP/TCP stack at which they operate.But the point being made earlier was that it may not be the primary job of a FW to cover for the lacunae, majority of which have been introduced by a standard non-compliant browser like IE - though . ofcourse more the coverage given by the FW in this regard the better it would be for the user.

It is pretty late and am signing off with this.Maybe ghost or some of the others would elaborate farther .

 

LnS uses a separate Application sandbox-like module to do this.

 

It seems we have 2 possible discussions going on.


(1) Should firewalls block leaktests?

and

(2) Should your whole security setup (firewall,AV,AT,system firewall/IDP/HIPS etc) block the leak tests?

From the discussion here, I would guess that most people in this thread would say Yes to question 2. Anyone say No?

There is some dispute over question (1) depending on whether the poster prefers to use something else to handle leak tests, but that isn't really important else for this consideration.

For the typical user questions (1) =(2) because they don't use anything besides a AV and a firewall so if your view is 'Yes' for (2), it has to be (Yes) for (1) too.

Most of these "leaktests" are frightfully simple and by no means do I think that those known are even close to what is possible by even a moderately clever (by wilders standards) attack.

Some would blame Microsoft, <insert standard rant of the microsoft "Architecture" even though the ranter generally doesn't know what this mean>, others would say it is an unavoidable that once a malware runs you are dead since you can't possibly think of the millions of possible things it can do.

That said, monitoring child-parent exes start, would block many of these, preventing process mantipulation + component monitoring would block most of the rest.

Only a very small component could be part of what a typical packet filter can address.

 

1. In answer to the question which FW is best the FAQ at the firewallleaktester.com says :

"Good question ! Since everyone needs is different, the firewall which will fits the best to your needs will be different than the firewall which fits better to the needs of your neighboor. Almost all of the firewalls offers a very good protection at the network layer to protect you from worms or script kiddies, what will makes the difference is all of the features around, and eventually, depending on your system power, the ressources used (someone with a P4 3.2Ghz and 1Go RAM will not care of the ressources that his firewall can eat, but try to think to a P-III 500Mhz user with 128Mo of RAM)."

2. A layered approach to security has therefore to be adopted by all of us instead of expecting one single all-potent application to do the job.

>>firewalls =whole security setup (firewall,AV,AT,system firewall/IDP/HIPS) ??

 

what ever security software u have is ur security setup so whether u have only an AV and FW or if u have an all-in-one security suite, theyre both considered ur entire security setup. some people want their firewall to block leaktests since they want to keep a minimal set of software.

 

FatalChaos (post #6) writes that it depends on whether or not you have other programs that will block the leaktests.

CrazyM (post #28 ) thinks of a firewall in the traditional definition, and for him, leaktests are not a consideration in selecting a firewall

Paranoid2000 (post #7) feels that a firewall successfully blocking these tests is a last line of defence should malware scanners fail.

And many other interesting observations.

Another consideration about the tests: Not everyone thinks through the situation as many here have, and part of the dilemma I’ve observed with some people is that when they discover these leaktests, they are led to believe on first impression that because they are "firewall" leaktests, that the security breech is solely a firewall problem.

Well, they were designed to test firewalls, but suppose for a moment the word "firewall" were removed, and the tests were just called "leaktests." Suppose that they were described as tests showing that under certain conditions, unauthorized network traffic can exit the computer, followed by suggestions as to the different ways of blocking these exploits.

As a matter of fact, if you browse the firewall leaktest site that yogishree refers to (post #29) you will find some interesting bits of information. He mentions the FAQ, and included in it is this:
-------------------------------------
Generally adding other security layers will catch the leaktests before they reach your firewall.
-------------------------------------

On the main page, the reader is referred to the "Advices" page where you will find this:
----------------------------------------------
Conclusion : your firewall is running on an operating system, which is the real reason for "firewall leaks." So, "leaks" are mainly windows leaks, not really firewall ones. Thus, you should take care of your windows setup instead of only tweaking your firewall.
-----------------------------------------------

If the user reads carefully, it will be clear that even though these are firewall tests, the firewall is just one component to consider in the security setup.

Also on this site is a wonderful paper, "Anti-leaktest-guide," on the Documents page. The first section describes the tests and then lists a firewall that blocks the test. The second section suggests another application that blocks the tests. The site's Software page discusses some of these "other" applications.

After studying these, people will be better informed, can carefully evaluate their test results and decide how they want to set up protective measures, and be in a much better position to conclude how much weight to give these leaktests when selecting a firewall.

regards,

-rich
________________
~~Be ALERT!!! ~~

 

There is a saying that goes something like:

When a man's only tool is a hammer, everything starts to look like nails.


__- HandsOff______________

 

here's tooleaky's take on this issue (a little on the old side perhaps)
http://tooleaky.zensoft.com

personally i would expect a firewall to be leakproof - the whole point of switching from the XP2 firewall to a third party firewall is that in theory you are gaining outbound protection - in practice this doesn't always seem to be the case so why not just stick with XP2 and save the money and the hassle (or just use a router)

having said that i also regard the firewall as part of a layered protection system - it is perhaps a little unfair to expect a firewall to stop everything and so the addition of a good Realtime AV/trojan detector + Software to prevent a trojan from running like Antihook/Processguard/OnlineArmour would seem to be a sensible route. A team effort.

However with the increasing sophistication of trojans using rootkits etc it remains to be seen as to whether even a whole team can stop the likes of Hacker Defender Gold etc.

Ultimately ONE COMPONENT in your security system HAS to be able to DEFEAT the BEST otherwise you might as well throw the whole lot in the trash can - Because the BEST is YET to COME.

 

well to me the layered protection is best, and its purpose is to prevent hacker and malware from getting into my pc and tho its not fool proof, i feel secure knowing im giving the bad guys a hard time and that the less determined hackers/malware hackers would go for easier victims.

 

personally i would like to see a firewall with a RADAR screen sweeping incoming and outgoing packets (with appropriate sound effects) it may not improve detection rates but it would sure look cool - maybe impress the girls

you could watch packets from China and Japan pinging away merrily on the scope (could replace the pinball game)

 

The probability of the risk and/or the consequences of this action occuring may be more than acceptable. In which case there's no good reason to throw the whole lot in the trash can. As for future threats, I believe it is overstated how different these will be from current threats. Most future threats will be new forms of a historic class of vulnerabilities rather than a unique type of threat due to the new technology itself.

 

it's ok ghost - i was just feeling in a melodramatic mood

but there are some very sneaky trojan writers out there - i do wonder just how long current security software can hold em off - i can't imagine that trojan writers are just going to give up and say "oh well processguard is just too good for us - lets give up and play battleships instead" or "oh dear NOD is so clever it's pointless trying to defeat it - we really should stop this hacking lark and sell widgets"

Shadow Walker the new wave

 

"As far as kerio 2.1.5 , the matter was discussed in detail some time back at BBR Forums when ...

1. What is BBR forums?

2. Besides Wilders and Castle Cops where are some other good forums and sources of info. My head is spinning already. I may as well pile it on.

3. It seems that firewalling inbound is really easy. XP SP2 is probably up to the task alone. A router certainly is and both is overkill. That is where I am at now. I use a NAT router and XP SP2 firewall.

Conclusion 1)
If I use AV, anti-Trojan and antimalware I may stop bad things from getting onto my computer that I ALLOWED to get through the firewall perimeter. If the malware does get in through the perimeter and the other measures such as AV do not take care of it, then there is a new problem - outbound communication.

If I am right so far, The only remaining task is to keep exploitive software that: 1) got in and 2) stayed in, despite AV, AT, etc... from calling my sensitive data back to somewhere else.

The best firewalls perform sort of well on these "test of concept" leaktests which are not "clever" compared to a real world attack. It would seem that better outbound protection is in order. Firewalls are possibly/probably not enough. Then what is ? We are not all network experts. I want to learn but I do not want this to be a hobby. It is far too paranoid and boring. I want to shore up, check the forums once a week and get out of this obsession that has held me for about a month now.

So back to point:

Specifically, assuming that bad software is on a PC running windows and for the time being AV, AT and the like are not killing it, what is the best way to control its outbound communications?

Some posters seem to suggest that CHX-I, Process Guard, antihook and those types of programs have a role. This statement from a post caught my eye, "-Using CHX with Jammer and Antihook for outbound protection, my system never felt better, have tried out almost all the major firewalls out there, all have their pros and cons, CHX scores over all of them in my case.".

If I understand the use of these programs right - they seem to make outbound firewalling unnecessary from a security standpoint. Outbound firewalling may still be nice to see which Windows services, App. service call out, etc...

Thanks,
B.

 

And yet hackerdefender gold doesn't have any functions built in to elude personal firewalls ........

 

if that's the case then we have nothing to worry about - even the simplest firewall will stop it.

 

Hi,

The problem is simple: since Microsoft keeps its architecture, it will always be a cate and mouse game, especially because Windows kernel is not well documented.
Consequently, many things (escalation, evasion, attacks like shatter, exploits etc) are possible...

As far as i know, Shadow Walker has been defeated.
Generally, a protection of the hardware/physical memory can be enough against exploit using Direct Kernel Object Manipulation (DKOM): ProcessGuard and System Safety Monitor can be helpful in this case.

In a few days (perhaps weeks), an user/Rin3 rootkit will be released from
China, and also a rootkit detector/profiler from the Xfocus chinese team.

For the King version of HxDef, a good protocol analyser will detect all the connections.
Really afraid?

(....)

Regards

 

personally i have complete and utter faith in the forces of light in their eternal battle with the dark side (but you won't catch me going broadband)

 

Actually, I wasn't pontificating, I do not know enough to do so.

So... Take my previous post as a series of questions.

I am asking how to control malware that is on a windows PC. We will assume that AV, AT, anti-spyware and all are on the computer but not able to detect or remove the malware. If this malware has the ability to latch onto internet explorer or other benign looking windows processes, how can I keep it from sending sensitive data outbound? A firewall alert will probably look benign enough for people like me to "allow" and check the box that says "make rule" or "remember choice".

There are a few posts like this thread where the efficacy or even need of the personal firewall as an outbound connection control or as an app. control is debated. I gather that the best ones do an decent job. Decent really is not enough. I do not want the weight and eventual BSODs of a 3rd party firewall if it really does not add much to my NAT router and xp SP2 firewall.

Is there a better way and do you have to be a geek to use it?

B.

 

hi - brjoon - my first thoughts are that if you have a trojan on your computer that has not been detected by any scanning or prevention software then i don't see anything that can stop it. to stop it you have to detect it. at least with a 3rd party firewall you have a chance - because it "might" alert you that "something" is trying to send data out. no security software is 100% perfect there is always a chance that something will get thru all your layers.

what percentage of trojans/spyware is truly undectable i wouldn't know but i would have thought it was relatively small.