Matousec RETEST !! COMODO & Online Armor DO NOT pass 100% !!

[Happy-Dude]

Heya guys !!

Matousec updated their tests today (05-17-08 ) and decided to retest COMODO and Online Armor for their latest rounds of test to see if they still score 100%. Apparently, they did not, but COMODO FWv3 still came out on top (95%) !! Online Armor now comes out at 89%, and their team response told of working on fixes to their latest versions.

-{ Quote: "# 2008-05-17: Results for these products and versions were published:

* Comodo Firewall Pro 3.0.22.349
* F-Secure Internet Security 2008 8.00.101
* Lavasoft Personal Firewall 3.0.2293.8822
* Online Armor Personal Firewall 2.1.0.131 Free
* Panda Internet Security 2008 12.01.00
* Trend Micro Internet Security 2008 16.10.0.1106
* Webroot Desktop Firewall 5.5.10.20

After the implementation of several new tests, we have decided to retest Comodo Firewall Pro and Online Armor Personal Firewall Free to see if they can still score 100% in our challenge. Both these products lost the perfect score, but Comodo Firewall Pro remained on the first position in our challenge. The odd thing is that the new version of Comodo Firewall Pro did not pass two tests, namely SSS and SSS4, that its previously tested version passed. This firewall has no problems with our new keylogger tests but failed SockSnif test. It also lost a few points in performance tests but its results are still excellent and almost perfect, 95% in total. Online Armor Personal Firewall Free lost more points, especially in keylogger tests. The classification of its protection is now Very good with 89%, close to the Excellent protection, which starts on 90%.

Lavasoft Personal Firewall uses the engine of Agnitum's Outpost and its score for today is 70%. Webroot Desktop Firewall, a free product that uses the old version of the engine of Privatefirewall, scored 60%.

F-Secure Internet Security 2008, Panda Internet Security 2008 and Trend Micro Internet Security 2008 are security suits with very high hardware requirements but their protection is noticeably worse than of other products tested today. " }-

http://www.matousec.com/projects/firewall-challenge/
http://www.matousec.com/projects/firewall-challenge/results.php

[Peter2150]

Oh gosh. Now I am only 89% protected against theoretical threats. The only thing that comes to mind is the famous line from the movie Gone with the Wind.

[Fuzzfas]

Oh, my! They didn't reach 100%? I 'm afraid to ask what my Kerio 2 scored then. ;D

Instead of looking only at leak test results, you should look at the new "PerfTCP" and "PerfUDP" tests, to realise how much this insane leak test competition has impacted the basic job of a firewall, handling packets.

It's why i 've gone back to Kerio 2. Browsing and p2p is visibly faster compared to the "leak test champions"...

[wat0114]

Oh well, here we go again, more scrambling by these vendors to "patch" their products so they can re-claim their perfect scores ::)

[InfinityAz]

-{ Quote: "...to realise how much this insane leak test competition has impacted the basic job of a firewall, handling packets." }-

It amazes me how often firewalls are updated now and it seems often in response to these leak tests. When was the last time Windows firewall was updated?

[Coolio10]

Seems knowone cares about leaktets anymore :D

[Fuzzfas]

-{ Quote: "It amazes me how often firewalls are updated now and it seems often in response to these leak tests." }-

It's more amazing to see where this leads to. It's funny, but reading Matousec's site, it appears that they broke something while updating previous releases, so they now fail in tests that they used to pass. They try to "pass" a new POC and this results on braking something else. This is getting ridiculous. I wanted to use Comodo in these days, but seems that my browsing is sluggish. Using a previous version is discouraging because of the bugs. OA has its own problems too, if you visit their forum. They sure excell in Matousec's tests, but i only want something light and simple to use as outbound control behind a router and this seems to be the last thing the current devs think of.

Instead of concentrating on bugfixing, they are obsessed with "strengthening the HIPS part" of the firewall. Yeah, ok. In the meantime, i 'd rather use Kerio 2 which makes your browsing and p2p fly with 0% CPU usage.

[Fuzzfas]

-{ Quote: "Seems knowone cares about leaktets anymore :D" }-

Oh, don't worry, vendors care! That's why some have the option to buy through Matousec's link now. It's profitable for everybody. Getting a "100%" logo from Matousec, is a fast pubblicity to put on the vendor's site, just like VB for antivirus. On the other hand, vendors, have interest in Matousec to continue such tests, specially if their product is designed for such tests. So i would guess that both sides have economical and reputation gains from leak testing. ;D

[LowWaterMark]

Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs.

[Dwarden]

webroot packet performance packets is quite amazing ...
and seems both AO and CF failing new Socksniff ..

[pykko]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-
Agree 100%. :thumb:

[19monty64]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-
When they do this I may start using a firewall again. ;D

[nhamilton]

I can understand the reason for leak tests ... in that malware could use the techniques to communicate using a valid app and not having the firewall pick up on it. I think the new TCP and UDP performance is great .. I just do not understand why a firewall should pick up keyloggers. Fine for a secruity suite or other apps .. Who care if an app can read my keystrokes, it is only if the attacker can get that data that matters.

[solcroft]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-
I wonder if they can afford to. There are those of us who know leaktests for what they are, but the majority of hopelessly clueless newbies have based their whole conceptions of a "good" firewall around Matousec and his silly tests. Forfeiting Matousec would probably be tantamount to forfeiting a major part of their customers' goodwill.

[alex_s]

-{ Quote: "Oh gosh. Now I am only 89% protected against theoretical threats. The only thing that comes to mind is the famous line from the movie Gone with the Wind." }-
There is nothing to worry about. I took a look at the results, the main part of the fails are keyloggers. It was declared that free version doesn't have antikeylogging protection except the very basic one. But my beta (and I think yours as well) passes all of them quite well :)

[Peter2150]

-{ Quote: "There is nothing to worry about. I took a look at the results, the main part of the fails are keyloggers. It was declared that free version doesn't have antikeylogging protection except the very basic one. But my beta (and I think yours as well) passes all of them quite well :)" }-

In case you missed it, I was being...ah.. slightly sarcastic. I know your right, but in any case don't really care. Before something can leak it has to be installed and run. Also in case you didn't know the line from Gone with the Wind is "Frankly my dear, I don't give a damn"

Pete

[Einsturzende]

-{ Quote: "In case you missed it, I was being...ah.. slightly sarcastic. I know your right, but in any case don't really care. Before something can leak it has to be installed and run. Also in case you didn't know the line from Gone with the Wind is "Frankly my dear, I don't give a damn"

Pete" }-
Hmm but why you using then product which flag/main/proud of feature is leakproof capability?::)

[Bouvier]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-

Now THAT is a clever advise, I hope OA- and Comodo programmers will be that clever too, unfortunatelly I expect them to go for the gold, because that draws the masses to them.

Being an OA-user myself, I still think it is one of the most flaky softwares I ever used.:doubt:

[boonie]

-{ Quote: "The only thing that comes to mind is the famous line from the movie Gone with the Wind." }-

LOL. Actually, the first one that popped into my head was "After all, tomorrow is another day." (Another day = Another leaktest).

[alex_s]

-{ Quote: "Hmm but why you using then product which flag/main/proud of feature is leakproof capability?::)" }-

Peter uses paid version which stops all the keyloggers. Free version is officially declared not to have advanced antikeylogging. But these are irrelevant details. I can understand leaktesting, I can understand stability tests, I just cannot understand how firewalls leak protection relates to antikeylogging. It seems Matousec has lost my respect in some degree. Let us say 12.5%.

[Einsturzende]

-{ Quote: "Peter uses paid version which stops all the keyloggers. Free versins is officially declared not to have advanced antikeylogging. But these are irrelevant details. I can understand leaktesting, I can understand stability tests, I just cannot understand how leak protection relates to antikeylogging. It seems Matousec has lost my respect in some degree. Let us say 12.5%." }-
:o we can rephrase that question: Why an Firewall have antikeylogger feature?

[aigle]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-

I totally agree! :thumb: :thumb:

This testing is going to be insane now.

[alex_s]

-{ Quote: ":o we can rephrase that question: Why an Firewall have antikeylogger feature?" }-

This is completely different question. If you are uncomfortable with the name (firewall) let you call it security suit. As for me I prefer to have "all in one". I hate a lot of different software with different settings with different support. This is why at the moment I use only OA AV+ and nothing else. And even with my risky behaviour (no UAC, no LUA, admin account, downloading and starting everything I find interesting) I'm quite satisfied with the protection level OA AV+ introduces. The only other "security" s/w I use is FD-ISR for a fast recovery in case something went wrong. But I never regarded OA as just firewall, this is Firewall + HIPS + antivirus (AV+ version). So my answer to your queston is "I (paid user) want it". Another question is about leaktesting. Leak is a process when sensitive information is transferred out of your computer. So I do not see how keylogger can be regarded as leak in case it cannot transfer its log outside.

[nhamilton]

-{ Quote: "I totally agree! :thumb: :thumb:

This testing is going to be insane now." }-

the two new test in level 1, with the performance of TCP and UDP are actually good and should have more weight. The anti keylogging is just insane.

[arran]

-{ Quote: "Oh, my! They didn't reach 100%? I 'm afraid to ask what my Kerio 2 scored then. ;D

Instead of looking only at leak test results, you should look at the new "PerfTCP" and "PerfUDP" tests, to realise how much this insane leak test competition has impacted the basic job of a firewall, handling packets.

It's why i 've gone back to Kerio 2. Browsing and p2p is visibly faster compared to the "leak test champions"..." }-


if your p2p is slow it means you haven't configured comodo properly, I have no speed issues with comodo 3 and it does have some inbound protection with advanced packet filtering rules you can setup.

Anyway the main reason why these firewalls is focusing more on leak tests is because most of us here allready have inbound protection.

"Inbound Protection"

1. Router firewall
2. admuncher or Proxomitron
3. Firefox No Script
4. web antivirus.

[Fuzzfas]

-{ Quote: "if your p2p is slow it means you haven't configured comodo properly, I have no speed issues with comodo 3 and it does have some inbound protection with advanced packet filtering rules you can setup..." }-

Neither p2p nor browsing is slow per se. But compared to Kerio 2, everything is sluggish. With Kerio 2 pages load in snappier way and in p2p connections seem to be handled faster at the same time without impacting browsing so much.

It's not about configuration, i know how to configure emule ports. People just don't want to understand that when something does 10 things at the same time, it is likely to do them slower than something that only does 1 thing at a time.

I think the TCP/UDP results even at matousec are a proof of what i am talking about. I bet you that if he had tested Kerio 2, the perf tests would have been both close to 100%. Some people just can't perceive that 15% drop. I can, it's a matter of "feeling".

"PerfTCP
Test type: Performance test
Scoring: The performance reduction is crucial for the score of this test. The product scores 100% if the network performance was not reduced below 90% of the original performance. This means that the test "tolerates" 10% performance consumption. If the performance is less than 90% of the original performance then the score is computed proportionally (e.g. 50% test score means that the performance was reduced to 45%)."

"PerfUDP
Test type: Performance test
Scoring: The performance reduction is crucial for the score of this test. The product scores 100% if the network performance was not reduced below 90% of the original performance. This means that the test "tolerates" 10% performance consumption. If the performance is less than 90% of the original performance then the score is computed proportionally (e.g. 50% test score means that the performance was reduced to 45%)."

http://www.matousec.com/projects/firewall-challenge/level.php?num=1


Results for Comodo. PerfTCP: 81
PerrfUDP: 84

I didn't expect Matousec's tests to tell me that, that's why i keep going back to Kerio 2.

[Einsturzende]

-{ Quote: "This is completely different question. If you are uncomfortable with the name (firewall) let you call it security suit. As for me I prefer to have "all in one". I hate a lot of different software with different settings with different support. This is why at the moment I use only OA AV+ and nothing else. And even with my risky behaviour (no UAC, no LUA, admin account, downloading and starting everything I find interesting) I'm quite satisfied with the protection level OA AV+ introduces. The only other "security" s/w I use is FD-ISR for a fast recovery in case something went wrong. But I never regarded OA as just firewall, this is Firewall + HIPS + antivirus (AV+ version). So my answer to your queston is "I (paid user) want it". Another question is about leaktesting. Leak is a process when sensitive information is transferred out of your computer. So I do not see how keylogger can be regarded as leak in case it cannot transfer its log outside." }-

Yes I completely agree with your statement, security evolves and there is no more completely visible line among once strictly dedicated products, I expect Matousec will change name of his project soon, his project evolve too.
Leaktests only is not enough for testing such evolved products, I find Matousec as driving force for advanced security products, without him and gkweb we will even now have packet filters on our PCs instead Personal FWs.
BTW, why Tallemu testing only its free product?

[Fuzzfas]

-{ Quote: ":o we can rephrase that question: Why an Firewall have antikeylogger feature?" }-

Because it has become a vicious circle. It's like the egg that made the hen or the hen laid the egg?

Matouosec started with leak tests. Some vendors chose to follow the "way of the leak" in order to promote their products and show that their firewall is better than the others because has this extra ability. As they followed this way, Matousec got more credit too. People at download.com are now judging firewalls according to Matousec. You read reviews saying "See Matousec why this firewall is bad!".

Now it's pretty much a trap where the vendors put themselves. If they started putting keylogging in their firewalls first, Matousec was happy to oblidge and incluse keylogging tests too, so to sort out the "best of the best". If Matousec was the first to introduce keylogging tests, the vendors that chose the "way of the leak", are now bound to accomodate Matousec's leaks in order to maintain their "leak supremacy" on which they built good part of their reputation and pubblicity.

Hopefully, the vendors that already are judged as "poor" by Matousec, won't be interested in this game and will concentrate on improving the firewall itself, instead of just adding ways to pass every single leak test Matousec will conceive.

[alex_s]

-{ Quote: "Yes I completely agree with your statement, security evolves and there is no more completely visible line among once strictly dedicated products, I expect Matousec will change name of his project soon, his project evolve too.
Leaktests only is not enough for testing such evolved products, I find Matousec as driving force for advanced security product, without him we will even now have packet filters on our PCs instead Personal FWs.
BTW, why Tallemu testing only its free product?" }-

I agree with your main line. Matousec tests are useful and technically interesting. Though, his latest move is beyond my understanding and while I'm not giving up his whole project I have internally decreased the value of his rating for me.

As for the marketing strategy Tall Emu provides I can say nothing. I'm rather a tech person and I'm just a user, even being a beta guy. And I'm afraid this a bit OT, sorry :)

[Peter2150]

To answer a couple of questions thown at me. First I use OA paid, because I consider it a darn fine product, period.

As to the firewall part, it should be noted when OA first started it didn't even have the firewall. Then when Mike started playing with the idea of adding a firewall, he committed to making it the best it could be. IMHO he has succeeded.

Pete

[Peter2150]

-{ Quote: "LOL. Actually, the first one that popped into my head was "After all, tomorrow is another day." (Another day = Another leaktest)." }-

ROFL. Excellent boonie.

[dendrobates]

I agree on a lot of previous posters here, that if you end op highest in the ranking here, you get a Firewall that is terrible.

When i tested Comodo for a few weeks, i noticed that i better could hire someone to handle the extremely nagging, unworkable pop-ups.

For the record, CPF has run for 3 days learning before it was used.
Even after weeks you still get these False Positive-alike terrible pop-ups.
So it is very likely that when you get a pop-up alert which is real, you don't read the text anymore.

From all the security software i have ever tested, it came #2 on
my noisiest nag top 3
And how should be people ever be able to work with this and
know how to handle these popups if Security isn't their job or their hobby?

Another thing is, that near my opinion Firewalls should be firewalls and not
trying to be anti malware software or hips, it is a bad combi

So i prefer a hardware firewall outsite my pc, a hips and a good anti malware, backup sw etc.

BTW if you nag with a popup for every BIT of disk i/o you get a 100% rating in these tests !!

:)

[Stem]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-

I do hope members (and vendors) have seen and fully noted the above statement.

At one time I had full respect for Matousec, that was due to the testing of security on the internal level (if that software would/could crash the system,.. possibly due to attack,.. or simply due to bad coding from that vendor). Unfortunately, I now see Matousec going for "leaktests" and now actually creating their own.

As I have mentioned before. Unless I allow crap on my PC, why should I worry about it getting out (for whatever reason)

Due to such as "Matousec", vendors are spending too much time (IMHO) on prevention of leaktests, I am sure they can provide/introduce better security to their products that will actually benefit end users far better.

[Firebytes]

Personally, as far as me caring about products that pass leaktests; firewalls are waaaaaaaaaaay down the priority list behind plumbing, condoms, feminine hygiene products, inkpens, and diapers to mention just a few. Now one of those leaking can really ruin a person's day. :shifty: ;) ;D

[gerardwil]

roflastic;D

Gerard

[Joliet Jake]

Does Matousec invent these tests? By this I mean the tools, keyloggers etc he uses to test the firewalls.

[Stem]

-{ Quote: "Personally, as far as me caring about products that pass leaktests; firewalls are waaaaaaaaaaay down the priority list" }-I agree. but users/public are easily confused~(mainly scared by such)

[arran]

I agree with most people here that these software firewalls have little "Inbound" protection that they mainly focus on leak tests etc.

But the reason why I use comodo 3 is because it adds an extra layer of security to my setup. If any malware happens to gets passed my av,sandboxie,
and web filtering Comodo 3 will prevent the malware from executing and running and also prevent it from connecting to the internet.

[MikeNash]

-{ Quote: "I do hope members (and vendors) have seen and fully noted the above statement." }-

Seen and fully noted.

We've been recently criticised for releasing software before it was ready to keep the number one spot (whereas before, we used to get in trouble for long release cycles :) ). We won't be making that mistake again.

The paid version of OA will pass those tests (I think socksnif and one keylogger will fail; plus the two BSOD's in the report). The current beta of OA I believe passes all of them (not sure on socksnif, will have to check it), including the BSOD fix.

We're not going to rush a release out. We're in the middle of some changes, and I've promised that our help will be updated in line with the next release.

If paid users of OA are concerned about the test results, they may request access to the beta forum (through the thread at the Online Armor forums) and get the beta versions of our software.

Having looked at the results it's exactly as I'd expect - OA Free does not offer keylogger protection, so it's going to lose points.

I will probably ask Matousec to test the paid (current release version) of Online Armor - now that he has added the keylogger tests there's a clear difference between OA Free and OA Paid, where before they performed the same in the tests so there didnt seem much point.


Mike

[Peter2150]

-{ Quote: "
But the reason why I use comodo 3 is because it adds an extra layer of security to my setup. If any malware happens to gets passed my av,sandboxie,
and web filtering Comodo 3 will prevent the malware from executing and running and also prevent it from connecting to the internet." }-

Those are the same reasons I use OA. I think both programs accomplish the same thing.

[WSFuser]

Are Comodo and OA the only firewalls tested with PerfTCP/PerfUDP? The reports for other firewall dont mention those two utilities.

BTW what does OA's score of 66% on PerfUDP mean in layman terms?

[MikeNash]

-{ Quote: "Are Comodo and OA the only firewalls tested with PerfTCP/PerfUDP? The reports for other firewall dont mention those two utilities.

BTW what does OA's score of 66% on PerfUDP mean in layman terms?" }-

It exposed a UDP related issue in performance in OA. It's been addressed.

[arran]

I tried Online Armor once but it conflicts with the best ad filtering software on the market "admuncher" so I went straight back to Comodo.

[Pedro]

-{ Quote: "the two new test in level 1, with the performance of TCP and UDP are actually good and should have more weight. The anti keylogging is just insane." }-
Indeed, ridiculous.
It's not even "leaktest", it's malware generic detection, or something like that.

At the very least he could do those tests separately.

[MikeNash]

-{ Quote: "I tried Online Armor once but it conflicts with the best ad filtering software on the market "admuncher" so I went straight back to Comodo." }-

If you grab the latest version of OA and Admuncher, you'll find they work together quite nicely now :)

[hammerman]

-{ Quote: "Not being a person that cares about leaktests, I encourage both vendors (well, actually all firewall vendors involved in the testing) to ignore this latest round. Work on making your products fast, stable and user friendly for a while, and forget these POCs." }-
I couldn't agree more :thumb::thumb::thumb:

Been saying this for a long time. These tests have become an unwanted distraction. Any vendor that goes out of it's way to try and regain it's position will lose my respect, FWIW.

[arran]

-{ Quote: "I couldn't agree more :thumb::thumb::thumb:

Been saying this for a long time. These tests have become an unwanted distraction. Any vendor that goes out of it's way to try and regain it's position will lose my respect, FWIW." }-

I don't think it is urgent that the venders need to fix these failed areas either, but at the same time I think it is good that sites like Matousec do test these firewalls because it keeps the venders on their Toes. They need to be kept on their Toes and keep making their firewall products better because malware is allways changing and adapting like this for example.


new shape shifting malware.
http://itnews.com.au/News/76128,shapeshifting-malware-hits-the-web.aspx

anyway I'm not worried OA and comodo will proabaly in the near future release updates fixes for the part in the tests they failed on.

[Mrkvonic]

Hello,

Here's a leaktest for you:
Delete all your personal files. What? What? PFW didn't protect them? What!

Each leaktest only proves that you should first infect yourself. Very convenient.

Vendors, go for STABILITY first. People want stable, reliable products, not a mishmash of 300MB of bugs and patches.

Mrk

P.S. Is mishmash a word?

[alex_s]

-{ Quote: "People want stable, reliable products, not a mishmash of 300MB of bugs and patches." }-
If users only needed stable and reliable products they would stay with MSDOS.

[subset]

Hi,

the problem is that especially small vendors get overly excited because of this fully synthetic test results.

The big vendors like Symantec, Trend Micro or McAfee don't apparently care about their results nor are they responsive to Mr. Matousek. :-X

Even the responses at the Kaspersky forums are like: "85%, pretty good... next!"

Not "Good gracious! Oh my God! Oh, not again! We have failed! Highest priority! Fix it! Fix it! Fix it!" :lurking:
Instead of... next!

Cheers

[alex_s]

-{ Quote: "Hi,

the problem is that especially small vendors get overly excited because of this fully synthetic test results.

The big vendors like Symantec, Trend Micro or McAfee don't apparently care about their results nor are they responsive to Mr. Matousek. :-X

Even the responses at the Kaspersky forums are like: "85%, pretty good... next!"

Not "Good gracious! Oh my God! Oh, not again! We have failed! Highest priority! Fix it! Fix it! Fix it!" :lurking:
Instead of... next!

Cheers" }-

I'm not marketing guru, but it seems quite obviouse for me that a big vendor with a big income and brand name can afford itself much more independent marketing strategy. Small vendor without brand-name is very dependent on everything public. Otherwise it risks to end up with a very good, but known to nobody product that will never cover expences. BTW, I do not think that Kaspersky lab is too dependent on their KIS. I think their main income is KAV, well know and still well rated antivirus.

[Kees1958]

Hi all,

I fully understand the ironic comments of the wilders members.

For classical HIPS and FireWall's it makes sense to combine their functionality. The most user intrusive part of both: is application control. This application centric control (including whitelists) is the rational behind this trend. To differentiate FireWalla with these features, testing had to be pushed further. It is just a pity it has taken a ridiculeous direction.

The test pictures FireWall concentrating on their core task as inferior quality products. PC users may have other approaches in obtaining those goals.

For instance
- On Vista, use VistaFireWall control for outbound control and ThreatFire to deal with intrusions
- on XP, use Sunbelt Kerio FW (with a nice NIDS and easy image execution control when checking behavior control, but automaticcally allowing startup) and ThreatFire

Above setups are 10 times easier than the 10+ numbero uno ranked Comodo, but I bet you loose only little security with this immense gain of user friendliness and simplicity.

Regards Kees

[Joliet Jake]

Firewall vendors should get together and agree a single course of action with regards to online tests where they're required to pay for retests.

The business model of Matousec is plainly obvious, scare the pants off vendors by pitting one against the other and releasing a bright and shiny league table every few weeks which results in vendors rushing out their latest 'fix'.

The biggest beneficiary here is matousec imo.

If vendors agreed to stop dancing to his tune and instead develop their product in a mature fashion I don't believe anyone's security will be compromised.

[Saraceno]

-{ Quote: "Firewall vendors should get together and agree a single course of action with regards to online tests where they're required to pay for retests.

The business model of Matousec is plainly obvious, scare the pants off vendors by pitting one against the other and releasing a bright and shiny league table every few weeks which results in vendors rushing out their latest 'fix'.

The biggest beneficiary here is matousec imo.

If vendors agreed to stop dancing to his tune and instead develop their product in a mature fashion I don't believe anyone's security will be compromised." }-

Hopefully Online Armor and others will read your post a few times over. These 'hypothetical' one-in-a-million chance of infection tests are getting out of hand.

The only thing leaking is the firewall vendors' pockets.

[Einsturzende]

-{ Quote: "

I will probably ask Matousec to test the paid (current release version) of Online Armor - now that he has added the keylogger tests there's a clear difference between OA Free and OA Paid, where before they performed the same in the tests so there didnt seem much point.


Mike" }-
Optimistic question ;) :
Why not adding better keylogger detection in free version?

It is confusing now, some keyloggers can be detected and some not, someone will thinking OA free have poor keylog detection implementation.
Half solutions are always bad...

[Saraceno]

Mike's working hard, he's gotta put 'food on your family'.

You can't get all the features for free. ;)

[Dwarden]

Matousec should (or more like MUST) add new simple test to Level 1:

When firewall supports SNORT/advanced SNORT rules it PASSED and if NOT it FAILED ...

deadly serious about this...
no mercy with vendors ignoring it :)

[FatalChaos]

I just noticed that for outpost 2008, perfudp and perftcp are listed as N/A, as they are for some other firewalls. Anyone wanna shed any light on this one? Is it just not possible to test the performance level of some firewalls?

[ggf31416]

-{ Quote: "I just noticed that for outpost 2008, perfudp and perftcp are listed as N/A, as they are for some other firewalls. Anyone wanna shed any light on this one? Is it just not possible to test the performance level of some firewalls?" }-
The products with score /62 were not retested yet.

[Einsturzende]

-{ Quote: "The products with score /62 were not retested yet." }-
Matousec already tested imaginary build, Agnitum needs time to send him another imaginary version of Outpost which will pass all those keylogg tests, WE DON'T KNOW YET HOW GOOD OR BAD IS OUTPOST (according to Matousec testing)

[Einsturzende]

-{ Quote: "Mike's working hard, he's gotta put 'food on your family'.

You can't get all the features for free. ;)" }-

Yes, you can get but not from him, anyway that is not my point at all, my point is give us free keylogg protection or remove all keylogg protection from free version, half protection is good for nothing...

[Fuzzfas]

-{ Quote: "I just noticed that for outpost 2008, perfudp and perftcp are listed as N/A, as they are for some other firewalls. Anyone wanna shed any light on this one? Is it just not possible to test the performance level of some firewalls?" }-

No test here, but i will take my crystal ball and say that Outpost will score over 90% at least in the PerfTCP test. I am trying the firewall now and i feel the connections quick. Everything seems very responsive. If i am proven to be correct by Matousec, i will think of opening the "Fuzzfas feel-the-network test". ;D

[dmenace]

I can see why people dislike the leaktest concept...

BUT... without leaktests any discussion about firewall performance will be extremely boring, bland and too technical.

Since inbound protection is usually carried out by a SPI NAT router (yes I know there are exceptions)

Outbound protection is a much more relevent aspect to test. If malware manages to sneak into your pc, you would still want to stop it from connecting to net and send private info.

IMHO, firewalls were the earliest form of HIPS / 0-day protect software in the respect that they would detect outbound connections by unknown software - a very suspicious action most malware does.

~~~

On another note:

Comodo 3 (in matousecs latest testing) still fails SSS? I thought that was fixed a while ago. ???

Cheers,
Denis.