Matousec RETEST !! COMODO & Online Armor DO NOT pass 100% !!

Neither p2p nor browsing is slow per se. But compared to Kerio 2, everything is sluggish. With Kerio 2 pages load in snappier way and in p2p connections seem to be handled faster at the same time without impacting browsing so much.

It's not about configuration, i know how to configure emule ports. People just don't want to understand that when something does 10 things at the same time, it is likely to do them slower than something that only does 1 thing at a time.

I think the TCP/UDP results even at matousec are a proof of what i am talking about. I bet you that if he had tested Kerio 2, the perf tests would have been both close to 100%. Some people just can't perceive that 15% drop. I can, it's a matter of "feeling".

"PerfTCP
Test type: Performance test
Scoring: The performance reduction is crucial for the score of this test. The product scores 100% if the network performance was not reduced below 90% of the original performance. This means that the test "tolerates" 10% performance consumption. If the performance is less than 90% of the original performance then the score is computed proportionally (e.g. 50% test score means that the performance was reduced to 45%)."

"PerfUDP
Test type: Performance test
Scoring: The performance reduction is crucial for the score of this test. The product scores 100% if the network performance was not reduced below 90% of the original performance. This means that the test "tolerates" 10% performance consumption. If the performance is less than 90% of the original performance then the score is computed proportionally (e.g. 50% test score means that the performance was reduced to 45%)."

http://www.matousec.com/projects/firewall-challenge/level.php?num=1


Results for Comodo. PerfTCP: 81
PerrfUDP: 84

I didn't expect Matousec's tests to tell me that, that's why i keep going back to Kerio 2.

 

Yes I completely agree with your statement, security evolves and there is no more completely visible line among once strictly dedicated products, I expect Matousec will change name of his project soon, his project evolve too.
Leaktests only is not enough for testing such evolved products, I find Matousec as driving force for advanced security products, without him and gkweb we will even now have packet filters on our PCs instead Personal FWs.
BTW, why Tallemu testing only its free product?

 

Because it has become a vicious circle. It's like the egg that made the hen or the hen laid the egg?

Matouosec started with leak tests. Some vendors chose to follow the "way of the leak" in order to promote their products and show that their firewall is better than the others because has this extra ability. As they followed this way, Matousec got more credit too. People at download.com are now judging firewalls according to Matousec. You read reviews saying "See Matousec why this firewall is bad!".

Now it's pretty much a trap where the vendors put themselves. If they started putting keylogging in their firewalls first, Matousec was happy to oblidge and incluse keylogging tests too, so to sort out the "best of the best". If Matousec was the first to introduce keylogging tests, the vendors that chose the "way of the leak", are now bound to accomodate Matousec's leaks in order to maintain their "leak supremacy" on which they built good part of their reputation and pubblicity.

Hopefully, the vendors that already are judged as "poor" by Matousec, won't be interested in this game and will concentrate on improving the firewall itself, instead of just adding ways to pass every single leak test Matousec will conceive.

 

I agree with your main line. Matousec tests are useful and technically interesting. Though, his latest move is beyond my understanding and while I'm not giving up his whole project I have internally decreased the value of his rating for me.

As for the marketing strategy Tall Emu provides I can say nothing. I'm rather a tech person and I'm just a user, even being a beta guy. And I'm afraid this a bit OT, sorry

 

I agree on a lot of previous posters here, that if you end op highest in the ranking here, you get a Firewall that is terrible.

When i tested Comodo for a few weeks, i noticed that i better could hire someone to handle the extremely nagging, unworkable pop-ups.

For the record, CPF has run for 3 days learning before it was used.
Even after weeks you still get these False Positive-alike terrible pop-ups.
So it is very likely that when you get a pop-up alert which is real, you don't read the text anymore.

From all the security software i have ever tested, it came #2 on
my noisiest nag top 3
And how should be people ever be able to work with this and
know how to handle these popups if Security isn't their job or their hobby?

Another thing is, that near my opinion Firewalls should be firewalls and not
trying to be anti malware software or hips, it is a bad combi

So i prefer a hardware firewall outsite my pc, a hips and a good anti malware, backup sw etc.

BTW if you nag with a popup for every BIT of disk i/o you get a 100% rating in these tests !!

 

I do hope members (and vendors) have seen and fully noted the above statement.

At one time I had full respect for Matousec, that was due to the testing of security on the internal level (if that software would/could crash the system,.. possibly due to attack,.. or simply due to bad coding from that vendor). Unfortunately, I now see Matousec going for "leaktests" and now actually creating their own.

As I have mentioned before. Unless I allow crap on my PC, why should I worry about it getting out (for whatever reason)

Due to such as "Matousec", vendors are spending too much time (IMHO) on prevention of leaktests, I am sure they can provide/introduce better security to their products that will actually benefit end users far better.

 

Personally, as far as me caring about products that pass leaktests; firewalls are waaaaaaaaaaay down the priority list behind plumbing, condoms, feminine hygiene products, inkpens, and diapers to mention just a few. Now one of those leaking can really ruin a person's day.

 

roflastic

Gerard

 

Does Matousec invent these tests? By this I mean the tools, keyloggers etc he uses to test the firewalls.

 

I agree. but users/public are easily confused~(mainly scared by such)

 

I agree with most people here that these software firewalls have little "Inbound" protection that they mainly focus on leak tests etc.

But the reason why I use comodo 3 is because it adds an extra layer of security to my setup. If any malware happens to gets passed my av,sandboxie,
and web filtering Comodo 3 will prevent the malware from executing and running and also prevent it from connecting to the internet.

 

Seen and fully noted.

We've been recently criticised for releasing software before it was ready to keep the number one spot (whereas before, we used to get in trouble for long release cycles ). We won't be making that mistake again.

The paid version of OA will pass those tests (I think socksnif and one keylogger will fail; plus the two BSOD's in the report). The current beta of OA I believe passes all of them (not sure on socksnif, will have to check it), including the BSOD fix.

We're not going to rush a release out. We're in the middle of some changes, and I've promised that our help will be updated in line with the next release.

If paid users of OA are concerned about the test results, they may request access to the beta forum (through the thread at the Online Armor forums) and get the beta versions of our software.

Having looked at the results it's exactly as I'd expect - OA Free does not offer keylogger protection, so it's going to lose points.

I will probably ask Matousec to test the paid (current release version) of Online Armor - now that he has added the keylogger tests there's a clear difference between OA Free and OA Paid, where before they performed the same in the tests so there didnt seem much point.


Mike

 

Are Comodo and OA the only firewalls tested with PerfTCP/PerfUDP? The reports for other firewall dont mention those two utilities.

BTW what does OA's score of 66% on PerfUDP mean in layman terms?

 

It exposed a UDP related issue in performance in OA. It's been addressed.

 

I tried Online Armor once but it conflicts with the best ad filtering software on the market "admuncher" so I went straight back to Comodo.

 

Indeed, ridiculous.
It's not even "leaktest", it's malware generic detection, or something like that.

At the very least he could do those tests separately.

 

If you grab the latest version of OA and Admuncher, you'll find they work together quite nicely now

 

I couldn't agree more

Been saying this for a long time. These tests have become an unwanted distraction. Any vendor that goes out of it's way to try and regain it's position will lose my respect, FWIW.

 

I don't think it is urgent that the venders need to fix these failed areas either, but at the same time I think it is good that sites like Matousec do test these firewalls because it keeps the venders on their Toes. They need to be kept on their Toes and keep making their firewall products better because malware is allways changing and adapting like this for example.


new shape shifting malware.
http://itnews.com.au/News/76128,shapeshifting-malware-hits-the-web.aspx

anyway I'm not worried OA and comodo will proabaly in the near future release updates fixes for the part in the tests they failed on.

 

Hello,

Here's a leaktest for you:
Delete all your personal files. What? What? PFW didn't protect them? What!

Each leaktest only proves that you should first infect yourself. Very convenient.

Vendors, go for STABILITY first. People want stable, reliable products, not a mishmash of 300MB of bugs and patches.

Mrk

P.S. Is mishmash a word?

 

If users only needed stable and reliable products they would stay with MSDOS.

 

Hi,

the problem is that especially small vendors get overly excited because of this fully synthetic test results.

The big vendors like Symantec, Trend Micro or McAfee don't apparently care about their results nor are they responsive to Mr. Matousek.

Even the responses at the Kaspersky forums are like: "85%, pretty good... next!"

Not "Good gracious! Oh my God! Oh, not again! We have failed! Highest priority! Fix it! Fix it! Fix it!"
Instead of... next!

Cheers