New Matousec Firewall Challenge

BlueZannetti, sorry about that waiting for approval to post
response...G

 

You've missed the point. Air-bugs are not leaktests, but means by which security applications stop and disable leaktests. In the both cases they are "preventive" means.

 

According to Melih, matousec made a mistake in the testing which put comodo a lot lower than it should be. Comodo is now in 2nd place and according to him the bug fix version will put comodo back in 1st place but he refuses to retest.

And alex, you claim the high scores of comodo don't matter if it sacrifices usability, but the av will cover whatever the firewall misses so there is really no loss of security with other modes by default. It's the same situation as Mamutu, they test the firewall/hips part which Mamutu does not have. Matousec excludes the comodo av from the tests.

Look at the quote below.

 

I'd say it's very childish to think that something ended just because you have lost interest

 

Seems like the names of the Comodo configurations confuse many people, including Matousec. Last time I looked CIS firewall came with the default setting of "Optimum Security", which means the best security, right? No, that's the one that provides more usability, and the most secure is called "Proactive Security". Unless I am confused yet again.

 

Hi All

Does it not strike you that all of this is a "pure shambles".

It takes a "flawed" Matousec test to uncover a bug in Comodos Firewall HIPS.

Talk about the crippled relying on the infirm.

With all the resources that Comodo can deploy, NON it appears, go into quality control. We are fed B....S... from the Comodo machine, until of course it is found out. Then to cap it all we use a tin pot organisation (Matousec) who not only makes mistakes but includes the free version of OnLine Armor in 2nd place. BUT in the very small print says that the latest version was not tested and if it were would likely score lower results.

Why is there any further debate. It makes Wilders/Comodo forums/the security industry a laughing stock.

Frustrated

Terry

 

My congrats to all the Comodo likers !

But you know, fast retest is a boo-boo thanks to Melih. So have a temper to wait for the next 3 monthes. As Matousec stated "too fast restests turn it to the free quolity testing service, which is unacceptable"

 

Yeah, retests should have a certain time period like av-comparatives.

 

As i said before, how can the testing be fair if he conducts tests at different times. Imagine AV-Comparatives testing av's at different times!
Even being truthful about this makes his tests more flawed than people would think.

 

Optimum - the amount or degree of something that is most favorable to some end

In other words a mix between security and usability.

Proactive - acting in anticipation of future problems, needs, or changes

In other words reacting to problems earlier than later (alert after a bit of damage has been done-im guessing)

 

Not because I've lost interest but b/c the test lost even the small remaining part of credibility and relevance to firewalls testing...

 

This is just your private opinion supported by opinion of some other Wildres people. Most people do not think about it and some other take in account the tests themselves rather than political aspects. From the technical point of view Matou tests are great. Politic is something else, the thing without clear criteria, so hardly discussable. In politic you only can say "like" or "dislike". In tech if you see a way to terminate a process and this way is reproducable, then there is nothing to argue about.

 

Coolio10,
I understand now that "optimum" is a mix between security and usability-just not what it means in English, iaw many would expect optimum security to mean best security (see the dictionary examples). As opposed to perhaps "optimum mix between security and usability". And to tell whether Proactive is better than Optimum or not takes a little help. And the confusion is certainly real.

 

Don't you think the "levels of security" is nonsence ?

Security must be full in any mode. If in some mode security is not full, it only means this level is "insecure". But the vendors are so sneaky inventing confusive words

 

Yes, for most users I would expect that how these things work out of the box is more useful. But if Matousec says he will use the most secure mode, Comodo deserves the benefit of that. Tests that have the participants telling us that the tester didn't use it right leave me with a cold feeling that we have a whole lot of company not using it right out there. Modes that say something-like P2P or gaming are sometimes useful, but not just slogan type names. At least Matousec seems to stir things up and get them some visibility and some interesting forum comments.

 

From the technical point of view the tests suck goat's nuts wrt firewall testing. Some of them are plain broken even as HIPS test. Ways to terminate a process are largely documented, that doesn't mean they make a useful firewall test. And putting apps that are neither firewall nor HIPS to this test is the last straw here.

But this debate with you apparently goes nowhere, so I'm leaving it at that.

A professional tester will ask the vendor before and during testing if unsure, not fix the results after they've published the results with huge hype and after the vendors had to point out the flaws. A professional tester will also withdraw a product from testing when it ceases to exist (cough Prosecurity cough) or when the vendors does not wish to participate in similar comparison tests (for obvious reasons in addition, as was the case with Mamutu). A professional tester will not publish a ranking of products with results coming from different testing methodology and try to hide behind some ridiculous smallprint and claims about lack of time.

But the thing is - Matousec is not a professional tester apparently.

 

It's just NOT a firewall challenge.

 

This is not technical point of view, this is political point of view. Yes, politically it may look wrong, but I'm out of this discussion. Technically if test failed it is failed. This info is useful for thoughtful users and vendors, not for the people who only see "rating" column and how the project is "called".

 

Granted, that i find the terms "optimum" and whatever the other is called, unfortunate, because it can confuse the user , although during installation the other option has written below "maximum security", this is hilarious.

All Matousec had to do in order to be "professional" was to go to D+ settings and make sure all boxes are ticked. It's what i always do when i install Comodo, just to make sure that the installation went smoothly. Apparently, Matousec, chose the "optimal security" (reccommended) during installation and then didn't touch any other setting. He went directly to run the leak tests...

Exactly... It's also ironic that Comodo itself came up with an "all at one" leaktest launcher, but doesn't control that a new release, isn't "broken" in a past leaktest. It's also ironic that they pubblish a leaktest "suite" , while they state that leak tests aren't their objective. Well, they sure give the wrong message to their userbase. Not that i think that Comodo isn't great. And i don't care if it fails 5 leak tests, as long as it has simple execution protection. But they should adopt a more coherent policy. And quick bugfixes are welcome and nice. But they should at least leak test their releases to make sure D+ isn't broken somewhere, since they also provide the clt.exe...

 

Technically if you test whether a car can bake a cake, you are an idiot. This is exactly what Matousec did with Mamutu.

Technically if you fail to test half of the features of a firewall (like Matousec who's testing HIPS and not firewall features) your test sucks goat's nuts.

 

This is what I talk about, you care about "names", not the important facts.

I do not see how many Mamutu scored, I don't care. I see it failed "kill1" and "kill2" tests from level 1, which are "generic" for any security product. This does mean Mamutu can be easily killed. This is important fact. Knowing it I would hardly use Mamutu. Once again, not because it scored 2%, but because it can be easily killed.

 

It has obvious commercial impact on said vendors, like it or not. Thoughtful vendors regard this as a problem.
Solution, bloat the firewall like the others.. no more firewall updates.. newest keylogger method added to the firewall..

That's how i see it.

 

Yeah, you apparently don't care about any facts and instead are just pushing Matousec propaganda here. For sure stuff like "ECHOtest - finds out whether your firewall filters ICMP traffic" is an excellent test for a behaviour blocker.

Debate over, have a nice day.

 

Any behaviour blocker MUST protect itself in the first place. If it is killed by "bad behaviour" it is not behaviour blocker, it is nothing, because after it is killed it can block nothing.

 

Enough is enough. May I suggest that you actually read the methodology and the results before posting further noise here? Mamutu failed the test not because it didn't block the two kill tests, but because it didn't "pass" other tests totally irrelevant for this class of products, such as the above ICMP traffic blocking or the Leaktest test that "finds out whether your firewall filters outbound TCP traffic" or the Yalta test which "checks whether your firewall filters outbound UDP traffic".

If you still don't understand why Mamutu doesn't belong to Matousec tests after reading the above, then I'd ask you to refrain from further commenting on this thread as you have absolutely nothing useful to add.