credential of matousec's tests ?

Whatever the reason, in the 2 years that I've been using it, I have found Sygate PFP to provide excellent protection and during that time I installed a large number of apps, many of which wanted to 'call home' on a regular basis. So hopefully you understand why it's difficult for me to attach real-world meaning to Matousec's tests.

 

You don't need much outbound protection to control legitimate programs, as they don't attempt to bypass the firewall deliberately. You need more outbound protection if you are ultra-paranoid or if you want stronger protection against malware not detected by your AV.

 

Let me put some light on this rough topic. Don't forget that I'm not a security expert, so I can make some mistakes

The key words in this thread are: "security strategy", "firewalls" and "leaktests". If we define them at first, this will be easy to understand.

- Security strategy
There isn't a security strategy which fits all patterns of computing. Each PC user must develop his/her own security scheme which should be based on his/her knowledge, discipline, availability of resources and patience. A basic layout should include backup (which requires its own policy), a router, password management (another strategy to develop) , a antimalware app, a personal firewall and common sense/safe hex. A more sofisticated/advanced strategy may add rollback solutions, HIPS, integrity checkers, more antimalware apps, hardening, etc.
A smart, rational, well developed scheme requires a basic understanding of how malware is able to install itself into a system.
Understanding Computer Infections I
Understanding Computer Infections II
Understanding Computer Infections III
When you understand how malware infects your PC, you'll be able to tackle leaktests with your security scheme. It doesn't matter if this is a job for your firewall or HIPS.

- Firewall
Firewalls are applications designed to filter network packets. Nothing more. But wait, network packets are created by applications, so a personal firewall should keep some track of the applications generating network activity. Then, we can classify personal firewalls according to their ability to control applications requiring access to network:
* Firewall with null control of applications (aka "pure" packet filters): Ghostwall and CHX-I.
* Firewalls with limited control of applications (name of executable, path, MD5 checksum): Kerio 2.1.5, Filseclab.
* Firewalls with advanced control of applications (but with varying degrees of success): all the others (LnS, Comodo, SKPF, Jetico, Blink, Outpost)
Are firewalls with null/limited control of applications weaker than the others? Absolutely no. Your security setup (in a layered approach) will take care of the "deficiencies" of the chosen firewall.

- Leaktests
Leaktests are PoC -proof of concept code- which use Windows design vulnerabilities to bypass firewall's control policy in a smart way (i.e. no brute force attacks). Some of the leaktests techniques are already used by malware. But leaktests first need to execute. So, a execution interceptor (usually a HIPS) will block any leaktest (or malware) before it's loaded in memory. But, leaktests are usually tested against firewall, which must discern between legitimate and malicious activity of files already loaded in memory (i.e. the user has given it execution permissions).
So, who needs to pass leaktests? THE USER A firewall or HIPS which are "leaktest-aware" may help, but it's up to you to pass then, because when malware is executed, it's generally too late.

 

Hello,

No problem controlling outbound. It's OK. Something you even want it. Outbound control of legitimate programs.

Now comes the question what can your firewall do when subjected to malware? And this is really the question of what can malware do on Windows.

Answer: everything.

Therefore, to 100% control outbound for every possible type of code, you need total system control. This means that the only 100% firewall is one that completely patches the kernel, takes control of it and becomes the sole I/O filter for what you do.

Otherwise, there will always be some aspect of the system that you firewall will not be able to monitor.

Example: malware installs its own socket. Your firewall might not be able to monitor this at all.

The problem with leaktests is that they address an important issue that happens AFTER you infect your machine. Like I said, swallow 10gr of thalium and see how you cope.

The point is to keep malicious code off your computer and then, your worry with outbound is restricted to normal applications using normal protocols, and there you do not need any leaktests.

How to keep malicious code off your machine?

This is open to debate. Some will say HIPS, some will say anti-virus + anti-spyware, some will say anti-executables, software policies, limited user, alternative browser etc. All legitimate ways.

Choose the one best suited to your skill, understanding and needs.

If you are very paranoid and do not trust yourself too much, limited user or anti-executable is probably the best choice. If you are a geek and like to be in control, go for HIPS. If you just know what you do, firewall is enough. And so forth.

Mrk

 

What passes through my firewall of straw, will be killed by my security softwares, if not my frozen snapshot will kill it.
So Matousec's tests don't tell me anything new, but it's good for a social chat. Matousec's tests are just another way of earning money.
The brilliant bad guys always manage to break any software, no matter how many times it has been tested, even by Matousec.

 

I totally agree. gkweb's testing is done professionally and unbiased. He just gives the straight goods. Matousec wants $$ for bug reports and slams products that don't meet his standards on how a firewall should work.

 

Excellent post Mrk. That's it. I'm tending towards that exact post.