Latest Matousec Sec Challenge Results(Firewalls)

http://www.matousec.com/projects/proactive-security-challenge-64/results.php

 

Here below you have a dedicated thread with all updates:
Matousec: Proactive Security Challenge 64 (bits)
May be MOD can move this message into it. So we keep all updates in one thread

 

Well done outpost.2nd place.

 

Matousec is more of a complete security vulnerability test than a firewall test. A good packet filter Firewall would not have a chance against these kind of test. A good packet filter Firewall like Look 'n' Stop could fail the test, and still be one of the best products in it's category. The reason being is that the product is designed for an entirely different purpose.

A good HIPS normally do well against these type of test. I'm wondering how some other products like Appguard, VoodooShield, and Faronics AE would do against these type of test. I'm not sure how one could conduct the test against these products since these products are designed in many cases to stop the executable from ever executing. So the way in which the test are conducted may have to be modified to make the testing possible. I could be wrong though since I have not read in depth on how the executables are executed. For instance, in Appguard's case if an attempt was made to execute the executables from the userspace then they should be automatically blocked from executing if the executable is unsigned. If Appguard was set to lockdown mode then the executable would be blocked from executing from the userspace even if the executable was signed. If Appguard was set to high level of protection then a signed executable would be allowed to execute from the userspace, but it would be guarded so it should only execute with limited rights. So in many cases according to their testing methodology their executable may never be allowed to execute in the first place. I'm not sure if this makes a difference in their test, but I wonder why some of the products such Appguard, VoodooShield, and Faronics AE have not been included in the test. Maybe its money or maybe their test are not designed for such products. I've wondered about this for some time now. I guess I need to read into detail how Matousec perform their test.

I would also like to see how Sandboxie would do in these test. Sandboxes should also perform well against these types of test since their policies should not allow the threat to manipulate the actual system. In other words it should contain the threat inside the sandbox.

 

I'm with you Beethoven1770.. check this out..

http://www.matousec.com/

 

i dont care about paid results. never ever wondered why comodo is anytime on first place?

Privatefirewall 7.0.28.1 is really outdated (7.0.29.1 is present)
ThreatFire has lost support over a year ago and is still listed.

that list cant be serious without a note about thoses issues.

 

Junk, makes me wonder why people fall for this crap?

 

Because people always want to know, which is the best, which is in the top 10, but it is understandable since people do not go to Wilders for an advice, they ask Google.

 

According to the test results, this might have something to do with it:

How many other tricks like this are missed by the tester? Although matousec is controversial around these parts, I find value in the testing the do.

 

Folks would get better advice asking next doors cat about firewalls than relying on this having found it via Google.

 

It's nothing new that matousec "test" has not much to do with reality, compares apples and pears, is easy to fool and mostly a marketing machine for some ... Some interesting posts about it here and so on.

 

I don,t see OA there

 

Tall emu the original developer of OA had a financial dispute with matousec and so OA is not admitted anymore until a financial agreement is in place but from what i remember OA was always second place.

 

I believe OA, and Comodo bounced back and forth between 1st, and 2nd place.

 

It is always possible to encrypt packet and avoid a good packet filter

 

Such "independent" testers depend on vendors money...

 

Pay more and receive more seems to be the order of things involving independent tests.

 

hi
Of course this methodology will make no sense against pure firewall that focused on packet filtring https://www.wilderssecurity.com/showthread.php?p=1902320#post1902320
Hiyack what others have done before for free (Gkweb/firewallleaktester, HIPS devs tools, HIPS tests mostly), promote products that guive you a few dollars more and make money with an HERESY of firewall testing concept...

Independency is not compatible with money and interest conflicts.
This methodology is for the most part an HIPS assessment abilities of some various products (Security suites, av, firewall, HIPS suites), and is far from a proffessional testing way (pentesting with intrusion and data leak via tunneling for instance).

Rgds

 

Thanks

 

Your most welcome.!

 

so you can see what drives matousec... independant? yes sure....

 

Uff... Every time the same discussion. That's why I suggested post #2 to keep it always in the same thread as users can refer already to the hundreds of posts about it.

 

When defensewall 3.0 came out (was in alpha), Ilya tried to included DW at matousec. Would be a nice testimonial of its protection (also a product specific test AV-Comparatives scoring 100% just before launch of DW 3.0). At that time I tested DW 3.0 prerelease against the Matousec suite. It scored a full 100%.

Matousec does not want to take the entry point into regard. Even good old windows throws a warning when you try to run something from the internet. So sandbox / containment programs (HIPS) based on these policies are not welcome at Matousec. Matousec test products when companies do not ask for it. When a product has a firewall it is not included because it not according Matousec's testing policies.

IMO Matousec should not have market exposure here at Wilders, see for instance https://www.wilderssecurity.com/showthread.php?t=344241 Let's all refuse to post about Matousec and apply a 'cordon sanitair'

 

But you can download matousec's tests and check DefenseWall firewall
I think results will be fine