Mamutu Scored Last In Latest Firewall Challenge

Latest Results;

http://www.matousec.com/projects/firewall-challenge/results.php

 

I saw it and i was honestly wondering, what on earth was Mamutu doing in a firewall challenge...

 

You ain't the only one wondering that ...

 

They could as well have put ThreatFire there... Seriously, this Matousec testing is more and more WTF kind of stuff.

 

AGREED, matousec mustve been smoking some crazy **** that night.

 

I am sure, that if you notify Matousec about it, he will apologise to you for forgeting to include TF.

They 'd better change the whole thing into "Leak tests vs. Security applications". And think that he dropped the most useful test, the TCP and UDP perf test. Of course, when you catch all the leak tests, who cares if the firewall itself sucks and has poor performance in TCP and/or UDP. The firewall part is secondary. Leak tests rule!

 

Lol, yeah! He forgot EQSecure too with my ruleset

 

very tru, it would have been nice to see how alcyon's ruleset compared to normal EQsecure without them, or even just having EQsecure on the test would have been interesting if nothing else. id rather see a HIPS products on the test rather than a freakin behav blocker, lol

 

You could also ask what ProSecurity is doing in a Firewall challenge since its a HIPS and not a firewall. Mamutu is an IDS similar to a HIPS. Mamutu's score is hilarious.

 

yes, but mamutu is a BEHAVIOR blocker... at least a HIPS is more similar to these types of tests than a BEHAVIOR blocker considering leaks or mostly blocked by HIPS component of all the firewalls on that list.

 

Maybe they could throw in sandboxie as a firewall next time.

 

Matousec has gone over the rainbow with this one. Running out of software firewalls to test, is he? With the ludicrous inclusion of this fine behavior blocker, any credibility he may have had has been lost in my not so humble opinion.

 

Mamutu in a firewall test? Clearly they don't know what they are testing. Makes one wonder how seriously to take their tests...

 

I thoroughly just enjoyed reading like never before, every single one of your replies and opinions over this.

And i have only to add this one of mine: Is that guy gone off his rocker or what?

 

I totally agree with comments made that Mamutu has no place in these tests. However, I would have liked to have seen some results for the KeyLog tests which do apply. After all, it is claimed that Mamutu protects against keylogging activities but I haven't seen any evidence to support this. Has anybody tested Mamutu against these? If it's claimed Mamutu detects keylogging activities but doesn't, what else does it not do?

Anybody know where there are any test results for the behaviour blockers? I really need convincing about Mamutu.

 

I think Matousec should broaden the scope of their tests, as too many good programs are left out in the cold.

In the next test I wanna see how well MS Word and Minesweeper performs in their firewall leak-tests

 

Mamutu is absolutely misplaced in this test.

Mamutu is not a firewall, nor a leak blocker. It's a pure behavior blocker, made to detect real malware infections.

I've already asked Matousec to remove Mamutu from the test as it turns our product unnecessarily in bad light.

 

I think they are planning to include a selsction of widescreen TFT monitors in their next firewall test.

On a serious note, it is almost scary that they have included Mamutu. I hope they remove it - would have thought the vendor would have some legal redress against them for misrepresentation if they dont.

 

Without a doubt. The comparison tests are way out of order IMHO.

As far as light, MAMUTU is in the highest level of light that Behavioral Blockers can attain and theres obviously more to come from this amazing specialized program.

But a firewall test?

Pfffffft, placing a pure BB as MAMUTU in that test must surely have been deliberatedly sanctioned as a contestant in that manner in some effort to confuse the public as well as attempt to discount how really very reliable it really is in what it's security monitoring is chiefly designed to address.

That's as far off base as adding say FileMappbyBB as another competitor. Totally out of it's authentic context of the security it's fashioned to deal with without a doubt IMO.

 

On another note - and leaving the leak mania aside as it's already been debated zillions of times - I'm seriously annoyed by the Matousec's obsession with the termination tests. Almost every level has a good 1/3 of those, and sadly vast majority of them tests local process termination. I guess the "ideal" firewall in Matousec's view is a one that cannot be terminated at all - leaving the users with hung system and requiring them to press a reset button or pull the power cord?! Ugh...

 

A while back I emailed Matousec asking to include Sandboxie set to only allow the browser to use internet resources.

They stated - no - as the setting couldn't be set through the gui which was fair enough.

Now the setting can be set through the gui and their answer is still no?

 

Here is Matousec response to emsisoft feedback;


¨We are testing a specific kind of security software for which we defined the term "personal firewall". A product must meet some fixed criteria in order to be included to our project. The main criterion is to implement a process-based security. Firewall Challenge is designed to test personal firewalls, HIPS products, behavior blockers and other behavior based systems. Mamutu met all the required criteria and hence there was no why not to include Mamutu to our project after we received several requests from our visitors. All the products included to our project implement similar features. These security features are tested in our project. We believe that using a set of open tests is the only objective way to compare all the products that implement the very same features. There are various tests used in Firewall Challenge, only a part of the used testing suite is based on leak-tests.¨


Its funny looking at the vendor responses, LńS Firewall said they are not HIPS, Mamutu said they are not a firewall.

I use both, and I´m not sweating over it.

 

Sums things up perfectly.

 

I don't use Mamutu but I agree with the sentiment. Matousec is non-professional, aimless organization. Their tests are totally pointless.

"Lets load a kernel-driver and lets see how security products can stop it from sending out network traffic."... sureee!! Who are they kidding.

Ofcourse, you always have some upstart vendors that always chase the latest fad... trying to keep up with each and every test. Comodo, take the message - "You are wasting your time".

Matousec sucks.