Matousec Test Updated

Yes,I totally agree with you.Firewall is a Network Defender.But matousec's test includes HIPS's protection.And matousec's rusult can give users false advice.

 

Any results can give the users false advice, because any results are limited in this or other way. But let us be fair, even being far from perfectness those results are less misleading than PR stuff vendors keep on their web-pages. Still, you don't blame the vendors, but you blame the person who did really great job trying to develop his own independent testing. The project popularity and the vendors responses witness he really did well.

 

I don't want to blame the testers.They are great.But if they can pay more attention to details like distinguish pure firewall from HIPS.It can be more helpful for users.The test can be better.
Vendors due to want to sell more their products,so they would like to give users advantages rather than faults.It is normal.

 

Well the project is popular because it was the only existing "firewall test" the last 3 years.

Why do you think matousec did not changed the name of the test to "HIPS test" or "Proactive test" until recently? He changed it only when his site was well known and started to lose popularity because of the previous name (how can someone justify the firewall name for testing threatfire?).

The vendors who do well on the test since they have HIPS in their products praise it.

But vendors with products with no full hips, do not...

Panagiotis

 

I just wonder, why nobody else cared to run with something like this, but "properly cooked"

 

You mean a firewall test?

For the exact same reason that matousec does not do it. It is extremly time comsuming.

Or have you ever seen any log, protocol restriction, etc. comparisons in his tests?

And yet, he build his reputation with a firewall test that was never a firewall test.

Panagiotis

 

Well, I accept your point, still I think the term itself ("firewall") is very very open to discussions. Taking it literally you can come to a very wide range of functionality. Though, taking it historically the range can be narrowed down

 

Actually the term "firewall" in the security industry is well defined for the last 25+ years and cannot be subject of personal interpretations.

Panagiotis

 

http://en.wikipedia.org/wiki/Firewall

I do not find it very well defined. But if you post some more definitions it will be welcome.

 

Kaspersky improved.

 

I agree, in the wikipedia article is not well defined.
The definition of a firewall is the following:
"A Firewall is a system which limits network access between two or more networks."

Panagiotis

 

Neither your definition nor the wikipedia article specifies that a firewall should have self-protection though, and I really don't understand why if I have a HIPS that can protect all files, then my firewall must be able to protect itself as well, otherwise it is not a firewall?

 

Hum, but why should I buy this definition ? It doesn't match just a single idea I share. Is it just because you like it or are accustomed to ?

 

You should buy this definition because this is what it is.
No network security professional will argue against this definition.

That is just the way it is, whether it matches your idea or not.

 

This is plain simple. No decent security software should rely on OTHER security software to protect itself or perform the other BASIC security tasks. Otherwise it is forced to reference in EULA that it is only can perform security tasks when it is protected by something else. But if you need only packet filter (in opposite to security software), you can get it, of course. Though, what does it have common with security ? It has NOTHING common with security, it's just a nice (tool/toy) to pleasure your taste.

 

Hum. I let other buy the things because they "are what they are". To buy anything myself I need more valid reasons

 

- You asked for the definition and I provided it to you. If you buy it or not, is something that you should answer, not me.
- I do not really care of the ideas that you share about what a firewall is. AFAIAC, you can even believe that is a pc on a fireplace.

Panagiotis

 

I don't argue anything you say. But let us return to where we started from. I said "Firewall definition is very questionable". And it seems you were not able to provide a commonly accepted definition. Yes, you have provided some definition you like, but since its origin is not too clear we cannot take it as a base

 

Alex, it is the definition used in network security industry.
for example:
http://hitachi-id.com/concepts/firewall.html

Panagiotis

 

What I find interesting with these results, is that when there's a tie in scores, the paid for program triumphs.
As can be seen with Kaspersky/Comodo & Outpost(paid)/Online Armor(free):
http://img10.imageshack.us/img10/8379/matousecbias.png

One would normally assume when a tie occurs the order is organized alphabetical; which is how Matousec seems to do it with anything below a 'Protection level' of "Excellent".

Here is the order correctly alphabetized:
http://img41.imageshack.us/img41/2706/matousecbiascorrected.png

 

Do we really have a thread here arguing about what a firewall is ? Really? About the precise definition?

The definition provided by Panagiotis above is a pretty accepted definition of what most people understand a firewall to be.

Alex_s makes a good point that security programs should be able to protect themselves - but sadly, the good points he makes are very often lost due to his style of "debate"

Matousec should have renamed his Firewall Challenge a long time ago. Or, he should have at least only have tested products which the average guy in the street understands to be a firewall. This much is clear and obvious to me at least.

 

About the precise definition? No.
But IMO this does help in comparing the generally accepted definition against Matousec's definition.

 

Haha!

 

From Matousec web site:

It seems their attempt to "mitigate misunderstanding and criticism" has failed miserably.

 

I was discussing 'leak tests' the other day with a friend who said, If it can't get in, it can't get out.

For me, a more realistic test would be to put these killtests so-called into remote code execution (drive-by download) exploits on a test site and see if they can be prevented from installing/running.

----
rich