Leak Testing - Where is the Value

When leak testing started there were only a few examples of malware that used any of these concepts and it was believed at the time these were targeted attacks. That was before the malware explosion.

There are many different leak tests. If I counted right Matousec has 66 tests. Because these are not actual malware they are properly classified as proof of concept.

My question is, out of these many leak tests, which are in wide use by contemporary malware?

Some other things to consider:

1, The majority of PC's use either the default XP (and now Vista) firewall lacking outbound control.

2. The most widely used third party firewall is Zone Alarm Free which has only minimal leak control.

3. As best as I can determine, totally disabling the firewall is the most common technique used by malware writers to facilitate communitation.

4. Gkweb makes a point over at firewallleaktester that any firewall where svchost communication on ports 80 and 443 is not limited to seven (for the US, there may be others) Microsoft update address ranges may be bypassed using background intelligent transfer service. I have have found his advice useful previously. In some firewalls this limitation is easy to implement, in others very difficult.

Points 1 though 3 suggest that there is enough low hanging fruit that malware authors don't have to bother with complex strategies to fool firewalls. Point 4 suggests using BITS may be the method of choice. However, the famous Storm worm uses a modified P2P protocol, and IRC is popular as well.

Some of these tests are also likely to be more difficult to implement or keep hidden than others which could reduce their use by malware authors.

So, anyone, where are the real risks, and is it BITS or something else?

 

Personally (and I'm far from any expert), I don't put much stock in software firewall's capabilities to stop a true malware attack. From some of the things I've read, firewalls are no different than AVs in that there is probably ways to circumvent them all.

Personally, I'm more of the mind of prevention rather than mitigation. I invest my $$$'s in a quality AV/AS/AM set, practice safe hex using FF with NoScript and AdBlock, and I have a behavior blocker as a low-resource cost backup plan.

I do use a firewall (Vista firewall, with the outbound control turned on), but I don't expect it to be the piece of software that truly protects me should I get something on my PC.

 

IMO the only protection to be gained from any of these "security" software is the protection of time invested in PC configuration.
A blocked threat means that I can put off re-installing my OS for one more day.
I remember reading about Microsoft's NT4 receiving C2 security clearance, only to discover later that this was ONLY with the network cable UNPLUGGED.

The only truly safe system is one that is not connected to a network, and again, IMO, anyone who believes that they can obtain true security by installing all of these resource gobbling softwares is really fooling themselves.

 

Well... That is true for malware that just screws up your PC, but for things that run silently stealing passwords, data, etc, once its on its on until you reformat.

 

My opinion of it is, there will always be some way for malware to get out, so don't put your full trust in any software firewall for this purpose, period. That's not to say you shouldn't use one and attempt to cover 90% of the situations, but I'd never assume that I'm 100% covered..

I also believe that the prevention approach is best too. An educated and smart user can eliminate a lot of security software..

 

Exactly, I agree 100%.

For certain (NOT ALL), financial activities I fire up a VMWare Server (free) session where I have a clean XP installation and where I use IE to go to ONE site; since I only go to one site, I have no need for a firewall in the session.
When I'm done, I just reset the image.
I would expect that this level of isolation could also be achieved by having OS installations on separate drives.
By exercising discipline and maintaining complete isolation of activities in separate environments, I don't have to worry if any software failed the leaktest of the week.

EDIT: BTW, the VMWare software has a memory overhead of about 50mb, but I can run the VM with only 96MB by shutting off unnecessary services in the session, so that's 150MB total for a completely isolated environment.

 

While researching this question I ran across some comments by Symantec that most malware bypasses firewalls either by disabling them or sending allow messages. Hmmmm...

My concern with using Vmware for banking only is if there is a keylogger on the host, it will read all keystrokes sent to the guest. There are probably some other strategies that would be more foolproof.

 

Point well taken, which is why I do NOT browse with the host.
This line of action is of course based on the premise that the safest box is an unconnected box.

I have three separate VMWare sessions: one for banking, one for P2P, and one for browsing.
Of course 96MB is too small for a general browsing session, so for that VM I allot 512MB; P2P runs great with only 96MB.


I believe that in this age of multi-gigabyte, multi-core CPU machines, OS virtualization with multiple environments is the way forward, rather than trying to do everything, and therefore be forced to try to protect everything, in a single environment.

 

Actually, I was thinking of suggesting that, but I got lazy and did not. Virtualization is definitely the wave of the future. Some even advocate a small kernel supporting various OS functions in separate virtualized environments.

 

Sorry if I was drifting dangerously OT, Diver, but I think Virtualization as a security option does not get enough airplay.
Most suggest firewalls based on personal preferences or point to leaktests, with very little conclusive proof that these applications have ever saved anyone from anything!

So in an effort to try to get back to the topic, Leak Testing - Where is the Value:
IMO, the answer is there is practically NO VALUE, for the reasons previously stated.

VMWare Server is free to use, and it is in use in more financial institutions than any of these leaktest passing "personal" firewalls.

I will be quiet now, thanks!

 

A good HIPS along with a strong incoming firewall will prevent probably about 99.9% of attacks if the user knows how to use their security software. Sure nothing will ever be full proof, but you can get very close.

The value in leak teasting to me is more the principle. If a firewall is going to control outgoing communications at all it mise well do it right.

 

Aha, but how do we know what is right?
Diver remarked that Matousec has 66 leaktests; how do we know there aren't any other ways to circumvent the protection of a firewall?
Next year there may be 100 leaktests!
Even Matousec has stated that they need to update their firewall tests, but are currently too busy with other tasks.
Outpost 4 failed miserably on the first round of leaktests, after which they stated that leaktests are essentially meaningless.
Outpost 2008 is now on top of the leaktest game; guess what?
Agnitum now proudly claims their firewall is superior based on the new results, so apparently, the leaktest which they previously derided is now valid!

An interesting cycle is clear to me; is it clear to you?

 

While I am a leak test skeptic, I would not go so far as to say they are worthless. A firewall with outbound control might catch something missed by a signature AV. However, its much better to prevent infection in the first place by operating system hardening. In my view preventing leaks is a side benefit of OS hardening.

HIPS, anti leak strategies, LUA/UAC, software execution policy, anti executable are all variations on the same theme of blocking dangerous activity. The differences are in whether the measures alert the user, how far the measures go, and how likely the measures are to interfere with legitimate activity. The later requires that the utility be trained, and the user make many decisions along the way.

Which brings me back to the begining. Are we doing all of this stuff in some kind of a shotgun approach or can we target what is important in some kind of intelligent way. Perhaps only a few of the 66 leak tests are important.

It might that my wish is for a smart firewall. Something that knows the difference between malware and legitimate programs without having alert for every possible combination of program and parent program. It just seems nuts to me that printing a text file should set off an alarm.

 

As someone who spent a few years working as a developer on Big Iron at a financial institution that managed 250 billion in assets, who had to have a dedicated machine just to log on to provide support, IMHO, with all due respect, the BEST protection is maintaining separate OS environments.

As they say, to each his own.

 

As I mentioned above nothing is full proof, but you can get very close. Getting as close as possible is what I consider "right".

About Outpost, to me that is an indication of their company and not very important when compared to the "cycle".

If it is possible that there will always be an unknown leak test then it is also possible that all leaks can be plugged. Not to be fake and positive, but why be negative?

 

The way I look at it a HIPS is better at preventing any infection than leak tests, but in my case leak tests are a part of my HIPS

 

Symantec's firewall provides this intelligent recognition - processes determined as malicious will have network access blocked for them. Micropoint's firewall also intelligently recognizes benign processes, and prompts for the ones it can't tell. Many firewalls today also implement whitelisting and learning modes to cut down on the initial popups.

 

Hello,

Leak Testing has several values:

- Prove that Windows is a troublesome OS.
- Earn someone good money.

But it is meaningless for testing firewalls. Leaktests exploit the system and a firewall sits on top of a system, so if the system is leaking, the firewall will be leaking too. That's all.

Mrk

 

Hello,

As I often said many times in the past, for me leaktests value is to demonstrate than firewalls (without HIPS) cannot handle everything and will always be bypassed by one way or another. It was to fight against marketting advertises such as "100% secure firewall" and things like that.

I think the point is well taken nowadays, as the answer naturally shown to be HIPS rather than firewalls themselves. Of course now some firewalls include an HIPS module as well.

Just my personal opinion about the subject, but everyone can have it's own, I won't disagree

Regards,
gkweb.

 

I happen to agree too. I think you're right on the money.... There will always be a way around or through any software firewall...

 

And now because of your testing efforts and those of Matousec, we have a selection of firewalls to choose from that include HIPS features as well as design principles based on Matousec’s Ideal Design of Personal Firewall concepts, such as: Outpost, Online Armor, Jetico, and Zone Alarm to name a few.

In Matousec's case, several vendors - especially those I mentioned - have endeavored to improve their products to include some or all of the design principles Matousec lists.

These firewalls may not guarantee 100% security, but at least they provide some defenses other products don't include, and if that's what some people want, then that's great. At least there are products that include HIPS-like features for those who seek them.

The way I see it, this is just another one of those threads where it is hip, trendy, fashionable and cool for those to declare leaktesting and leak-resistant products are pointless.

 

I don't see it that way at all. I think one could just as easily say the same thing about those who declare that leak testing is the thing to do now and the only way to go... Let's use some common sense and intelligence, and just admit that there is no 100% solution. The answer probably lies somewhere in the middle, where there is practical value, and not at either extreme position.

 

I already stated there is no 100% solution. But what some people fail to realize is that this leaktesting has resulted in keeping security product vendors "on their toes", directly benefiting us - the customers - with a choice of products which include some useful features and enhancements we might want to have if we want more than just a bare bones firewall.

 

Some people also fail to realize that this leaktest has also introduced much unnecessary paranoia and forced vendors to introduce more and more bloat and unneeded features to their products.

It floats both ways, my good sir, not just the the way you want to see.

 

A red herring criticism if there ever was one. Highly opinionated and impossible to back up. Its actually contradictory on its face. Unnecessary paranoia does not force vendors to do anything, its a bonanza.

While stopping short of actual offensive language, this has an offensive tone and certainly the post you replied to did not deserve this. Its only purpose is to annoy the person quoted and bait him into a fruitless debate filled with more garbage such as the first quote. I believe the practice is know as Trolling, is it not?