Matousec Firewall Challenge = new Test

Fine !

I'm getting slowly sick of this Marketing fart called "Matousec", now.....
any idea when you should start getting the horrible bugs out ?

Any idea when your users will be able to play some media file instead of your tool stalling the system for 15 minutes before opening it ?

It's fine to lure customers in, it's even better to listen to them once you lured them in.

 

First of all, Lucas was right in post #62 - my mistake for not reading that part from Matousec. Second, I was just trying to emphasize the fact that both for HIPS and firewalls the rules are very important, and that the score can vary from configuration to configuration.

PS: As for Kerio, I was refering to the old style test, when all firewalls were tested against all leaktest - and I belive it's not impossible to achieve 90% (this is a problem with most leaktests, they depend too much on using IE - use another browser and block IE => pass the tests ).

 

why do OA execution alerts don't count ?

Clarify, please. I have completely missed your point. I just say that Comodo fails memory tampering done by dnstest and you can check it yourself.

 

There may be nothing wrong with your tests. Just:

1.) publish the tests
2.) publish your methodology
3.) make your tests reproducable.

 

Comodo has the same execution alerts. Any malware can fail if it cant even run.....

 

This bug has already been fixed in 119. It existed only in build 112 for a short time and we fixed it the day it was reported. Just had to run 119 through testing prior to release.

See here, for what else changed: http://support.tallemu.com/vbforum/showpost.php?p=32912&postcount=7

By the way - we do listen - and respond - in fact, we've been criticised for wasting too much time being polite to trolls, fanboys, competive vendors and folks who are just downright rude.

Mike

 

Then you have missed my point.

Many legal programs start other legal programs. There is nothing wrong in the fact program a starts program b. The "wrong" appears when program a tries to tamper trusted program b. Either via OLE, DDE, memory modification, code modification or commandline.

Alert "program a tries to start program b" tells too few to a user.
And alert "program a tries to modify memory of program b" tells much more.

 

I mean explorer.exe execution alerts which always come up.

 

I'm still missing you. What is wrong with starting explorer ? Any program can start explorer just to open a folder. There is nothing wrong in starting explorer in most cases, so this alert is meaningless in most cases.

 

There are many more avenues of attack used in his leaktests than just iexplore.exe. Not only that, Kerio 2.1.5 would fail miserably in the kill tests. Having said that, in no way am I implying it is an inferior application filtering firewall. It is actually, at least imo, a beautifully conceived product, one of the the most user-friendly for creating very granular rules, light-as-a-feather firewalls ever designed If it is paired with a good HIPS, you will have a powerful security package, that is if the leaktests matter to you. Otherwise it is, on its own, an excellent application filtering firewall for pre-Vista systems.

 

Online Armor 2.0.119 has scored 100%
on the newest and toughest Matousec.

 

Sure. It has. Another goos news is Comodo guys have already congatulated OA guys with this result. And now they are friends !

 

Hi Mike,

Yep, that probably covers about 90% of us

 

Comodo and Tall Emu are competitors. I'd like to think that we could be gentlemen about it and we will certainly try to keep it that way for our part.

Of course, we are competitors and we will try always to get the top position, and they will try to do the same - and when it's by a small percentage point - does it really matter so much?

As Coolio has pointed out - this competition keeps everyone on their toes.

 

Perhaps suprisingly, most folks I've come across are polite and considerate. Everyone loses their temper and gets frustrated - even those tend to calm down once they realise that they're being listened to and their problems are being looked at.

 

For some reason I'm unable to get to Matousec site on my system. Have tried even matousec.com and no-go. My system is Dell XPS 420 Vista Premium, Avast Home and Webroot FW. Is it maybe a security setting?

 

Did you try disabling your firewall or any other software that might be blocking the site to see if that makes a difference?

I cannot get to the Matousec site either. My problem is that when my router is pulling the IP address, I cannot get to the site but when I bypass the router and connect directly to the cable modem, then I get a different IP range and I am able to get to it. It seems that the IP range I get with the router is being blocked.

I don't worry about it because I can always come here to find the results.

 

I tried it without the FW and with SAS Pro off still no-go. Will try resetting cable modem see what happens. Maybe something changed by SP1?

 

Well I switched over to the drive with XP Pro and still not able to get Matousec. Must be something about the Cable Modem or this Dell.

 

It would be pointless, because the tests I made are relevant just for the rules I choose (which are very specific for my network config).

 

This is also OK. But then your tests have only interest for you and nobody else.

 

You are right. I was curious so I emailed Matousec Support. They informed me that the tets are run under an admin account because of a poll that indicates 84% of pc users run under an admin account.

**note**: edited message to conform to forum rules. Sorry admins!

 

I find it commendable that the information responsible for the following bulletin was given to Matousec and Mike Nash by ailef and MaratR who are volunteers and members of the Comodo Computer Security Testing Group.

2008-03-25: "We have received an email from ailef and MaratR with information about a security weakness in Online Armor Personal Firewall 2.1.0.112 Free that was tested in our challenge recently. We have successfully verified the information that the tested version of Online Armor automatically allows various privileged actions if it receives no response from the user in a few minutes after the alert is shown. We would like to thank ailef for his findings, we would like to apologize to our visitors and other vendors for possibly wrong results in case of Online Armor."

"We have contacted the vendor of Online Armor and received the information that the latest version of this product, Online Armor Personal Firewall 2.1.0.119 Free, does not suffer from the problem any more. To solve the problem with possibly wrong results, the vendor ordered a paid testing of its product. We have tested Online Armor Personal Firewall 2.1.0.119 Free and found that the security hole was fixed and also that it passes all current Firewall Challenge tests. Online Armor is thus the first product with the perfect result in Firewall Challenge tests. We are going to implement new tests to the testing system in next months and try to violate its perfect score."

http://www.matousec.com/projects/firewall-challenge/

Al

 

Continuing.

Another difference is in wallbreaker3 test

if you allow in Comodo initial alert "wallbreaker3 starting cmd.exe" then test fails
if you allow in OA the same alert OA pops another red alert where it shows the commandline and where you can see what does wallbreaker tries actually do. In commandline you can see the URL where leaktest tries to connect.

more later

 

execution alerts dont count!!!! And comodo does these same alerts but they are disaabled easily.