Scot Finnie awards Online Armor firewall of the year 2008

I agree that the addition of a router somewhat clouds the picture. A router would tend to protect any firewall incoming. In fact there are some members of this forum who feel that a router & the Windows Firewall offers decent protection. I noted also that while Scott cautions against the use of ZA Free he himself did not compare Zone Alarm Pro to Online Armor Pro.

 

Incoming protection is only what the router provides. AFAIK, the router has no bearing on those leaktests, unless i'm missing something.

 

So what are supposed to do with our notebook computers? OTOH, if you don't have a router in your home or office, you need one.

 

If you read Scotts review "I have also performed a set of my own security tests on Online Armor 2.1, Comodo 3, and some of the other firewalls I considered along the way. The latest versions of Online Armor 2.1 and Comodo 3 offer superb protection when used properly".

How can he have carried out a proper test when the router was doing a lot of the "protecting" work?

A proper test is when the wall is is not aided by anything else,how can you otherwise isolate what is actually giving the "superb protection" or part thereof.

 

The router has no bearing on Matousec's leaktests. The tools used to perform these tests are run on the test pc to challenge the outbound filtering capabilities of the software firewall being tested. A quote from his Firewall Challenge page:

Routers provide only inbound protection. Not that this is an inadequate service that they provide, but they do nothing to filter these tests.

 

Scott tested more than leaktests-I am talking about inbound not outbound,are you saying a firewall is only concerned with leaktests

You probably are aware that on the inbound side a router and a firewall overlap,this is obviously the concern-forget the leak stuff.

How can anyone know what portion of the inbound was intercepted by the router.

Just for example how many times have people mentioned that a port scan was different with and without a router

 

No, most certainly not. I'm going on Matousec's tests, which test only outbound filterihg, so the router plays no part here. As for Scot's tests, I'm not aware of what he did. Do you have a link to his tests? If he tested inbound capabilities, I'm very interested in how he did that.

Yes, very aware. I find it bizzare that he would test a software fw's inbound filtering through a router The router would most certainly stop everything, unless it is poorly configured.

From your post #24:

So it was Scot who stated that and not Matousec? Sorry, I wondered about that and no wonder I could not find it anywhere on Matousec's site

 

Watt,

In the first post,a link is given to the review and Scott states the following;

"This evaluation assumes that the software firewall is running behind a hardware router or broadband “modem” that offers network address translation (NAT) and stateful packet inspection (SPI), or in other words, a hardware firewall."

This is stated in the second paragraph below the sub heading "Why Programs Were or Weren’t Tested"

Bizzare is quite the right word !!

By all accounts OA offers superb all round protection,am not querying that it deserved to win and I wish them every success.

What is queried is Scotts methodology.

 

Thanks Hairy, I found it. I've taken some of his points regarding these tests.

From scot's Newsletter Blog; http://blog.scotsnewsletter.com/

He also mentions before that:

I'm curious how he determines the effectiveness of inbound filtering on software firewalls, especially through a router with SPI, no less I will ask him via email.

 

I bet that we could offer a small prize if someone finds a firewall test when the tester uses Nmap, a packet builder and related tools.

 

Sure, I'll throw in a few bucks. The Canadian dollar is holding its own pretty well these days


Assuming it's up to date, here's Scot's testing methodology:

http://www.scotsnewsletter.com/fw_test.htm

in there he states:

This contradicts what he says in his recent blog.

 

wat0114,

You have to conclude without the slightest doubt that his methodology for the tests we are now discussing,is as in the link-ie behind a router.

Otherwise he wouldnt have specified it!

Why complicate a straight forward issue?

we could also say maybe its a typographical error

 

Sorry, I was only trying to seek answers

 

I purchased OA three months ago without looking at the requirements. I missed the part where Vista wasn't supported

I paid for something I can't use, I'm still waiting for Vista support...... and waiting...

 

Why you did not ask for a refund. Mike is a very nice guy and I am sure he would have accomodate you.

 

I have read Scot Finnie's newsletter for some years, and have usually found them of interest. He tends to get hung up on particular products at times and dismisses others out of hand because he experiences problems - on the other hand, he may go to some lengths to contact developers and raise issues - I think he has a low tolerance for being inconvenienced, and comes across as quite grumpy at times.

I am not sure about his security testing methods - I think he is coming at it from the perspective of an average user, ie. what do I have to go through to get this software working and does it impact on the performance of my computer. He always seems on the search for the ultimate security product.

His is certainly not the last word. I have disagreed with his conclusions on a number of occasions - where he has found problems, I did not.

 

FWIW and for anyone who cares (maybe Harry?) a few points on Scot's views on the importance of inbound filtering (he answered my email querry):

• He places little concern on inbound issues.
• He recommends using a hardware firewall because they are cheap, easy and effective.
• When he does test software firewall's inbound filtering, he does so without a NAT or hardware firewall and uses a 3rd party service to run a port probe test.
• He feels inbound filtering is less important for end users and more important for enterprises.

[

 

Good you went to all the trouble .
His statements really didnt change my original doubts as to the testing effectiveness of running all the tests behind a router,without testing the inbound filtering capabilities of each candidate
He really shouldnt assume all or most of his readers would be prepared to buy a hardware firewall,most unlikely
If he explained his suppositions in another newsletter it would make interesting reading.
Anyway,what do you think?

 

Personally, I quite agree with what he says, especially about inbound not being as important for end users as it is for enterprise. I don't know that he assumes his readers would be prepared to buy a router, only that he recommends using one. If he hasn't already voiced these opinions in an earlier newsletter, he probably should.

 

I disagree. Inbound protection is the most important part of a firewall. After all, the whole reason of using a filter is keeping the "bad" packets (and bad guys) out. It's true that a personal computer can be configured and tweaked in such a way that there are no opened ports, and so the need for inbound filtering disapears. But how many home users know (or are willing) to do it? It is also true that some home users have a router with firewall capabilities, but there are situations when this is not possible. In these situations, the only line of defense is a personal firewall with good inbound protection.

 

Indeed it is, only that inbound filtering is definitely more important for enterprises, for obvious reasons. I'm still hoping someone can test for and provide results for thorough inbound filtering effectiveness on persoanl firewalls.

 

Because I'm hoping the Vista version is right around the corner... is it?

 

Firewall of the Year 2008? We ain't reached the end of 2008 so how can he decide that in only 3 months?

Perhaps it should be Firewall of the Year 2007.

 

As far as I see this logic it recommends something for use in 2008 basing on 2007 experience. In 2007 it was something of the year 2007 based on the experience of 2006, etc ..

 

What kind of test would you like to see? What should be tested about inbound filtering? Usually, inbound filtering works well, unless there is a bug in the firewall driver (like the fragmented packet problem in Kerio 2.1.5), and I suspect that in this case the problem would be found very quickly.