New firewall test site

Why do you think the tests are wrongly made VC, in my mind over lan testing is the best way, no isp firewall rules to confuse probes. I may not be currenty presenting then in the best way but I am working on that day by day.

That said you do need to point out exactly what you think is wrong.

 

Like I said, you don't know all the features of each firewalls, and since you don't show the configurations used, we don't know exactly what are you doing and if is right!

Why not, first implement all the website with the complete information, and than publish it to us analyze it? That will avoid a lot of these discussions...

for example:
- why you say that Zone Alarm (Free or Pro) doesn't have Self protection, when they have it!? You disable the service when you are the Admin for the applications... You said that Sygate Pro prevent us from that, I really don't know that since you didn't show the configs...
I can't disable the ZA service...
- why you didn't config correctly the CHX for your tests?

I can find other things, but I'm only using CHX (inbound) and ZA (Free and Pro) (outbound)...

Why you didn't try to talk with the firewalls developers and ask them for the best settings for each firewall?

Other useful thing, is to make a comparation between the configurations out of box and the bests ones...

 

Your right

 

Piolyte,

Your site was discussed in the Outpost forum New Firewall Test Site thread - in addition to the points raised there I would put forward the following suggestions for your consideration:

Intrusion Detection System
A Yes/No doesn't really seem adequate here. Those that need an IDS (people running servers, rather than end-users) would be looking for signature matching (ideally with Snort compatibility to allow new ones to be added) while some personal firewalls treat any unsolicited incoming connection as an intrusion. A grading system like None/Simple/Complex may be a better choice.

Web/Content Filtering
Offered by many firewalls, this can be a real security benefit if more specialised software is not used. A grading system again would seem best - or just listing the categories that can be filtered (ActiveX, Cookies, Java, etc).

Port Stealing
From your description (flooding to corrupt a switch/router/bridge's ARP tables) I would suggest that this isn't an issue that any firewall could protect against - it would be up to the network switch to detect and counter this. As for exploit potential, a user would have to be connected to the same port on the same switch as the attacking machine - since the majority of ISPs would/should have their servers on a separate network segment from their end-users, this only seems a plausible attack with cable-based ISPs which have their users arranged in one big LAN, if connected to a vulnerable switch.

Self Protection
Some firewalls are able to disable the network connection if terminated abnormally (e.g. Sygate and Outpost 3.0) - I'd suggest that such cases be considered as a pass since they (a) block any malware access and (b) alert the user that something is amiss.

Suggested Extra Test
Check firewall protection during Windows startup/shutdown ("cradle to grave protection") to see if there is any time window in which systems are unprotected. Startup in particular can be a problem since so much other security software (not to mention malware) will try to be the "first starter" which could mean a delay for some firewalls in providing protection, so having a "standard configuration" PC with just a free anti-virus and a "loaded" one with plentiful other security software may be needed to provide a fuller picture.

I would say though, that your site looks most promising and does provide some useful new perspectives on firewall testing - keep up the good work!

 

"Self Protection

Does firewall protect itself from being shutdown enough to allow packets through?"

Maybe you could expand on this some, here is simple example:

Ability to prevent termination of Firewall Service.

Ability to prevent termination of Firewall GUI.

Ability to stop all traffic when Firewall is not running.

 

I also think it shoudl be mentioned that termination could be guarded by Process Guard free, which is easy to use, and free. I mean it's really not that big of a deal if you ask me that the firewall can be terminated...those are probably the least of my concerns from a security standpoint.