New Matousec Tests - 2010

Another Proactive Security Challenge milestone – it now contains 148 tests (2010/01/04 10:30)
http://www.matousec.com/

We have implemented several major changes in our Proactive Security Challenge:

1) 64 new tests have been implemented to SSTS. The main focus was to cover files and folders protection, registry keys and values protection and control of system autorun entries. Several tests of other kind were implemented too. A functionality of a few tests has been changed.

2) Scoring have changed in case of many tests. There are currently no 5%, 10% or 50% penalties, every failure leads to score of 0% for the given test. This affects many termination tests, SSS, SSS2, SSS3, Driver Verifier, BSODhook and ShadowHook tests.

3) The vendors now have a right of only one free testing request per six months for all their products, previously they had two such requests for every product. This change is enforced by the extensive amount of the new tests.

4) SSTS now contains the new Configurator tool. This tool is handy for creating the configuration files, especially for the settings required by the new tests related to file and registry protection.

The vendors are now given five weeks to review their products' protections. The tests with the new version of the suite and new rules will start after that.

The further development will focus on implementing new testing suite for Windows 7 on 64-bit platform.


Security Software Testing Suite
http://www.matousec.com/projects/security-software-testing-suite/

Changelog

* 2010-01-04: Major improvements and changes. Many new tests were implemented. The main focus was to cover the areas of Windows OS security that were not covered by previous versions. The suite now contains tests that check protection of disk files and folders, registry keys and values and various places in the system that can be used to create autorun entries and several new tests of different kind. Several bugfixes and changes were implemented. SSS, SSS2 and SSS3 were simplified and now check only the ability to perform unwanted reboot/user log out. The current number of the tests in the suite is 145. The suite is now supported by the SSTS Configurator tool that makes it easier to create the correct configuration file.

 

Interesting. Waiting to see the first testing results. Hope to be out soon.

 

Windows vulnarabilities and attack methods are a never ending game. It will be interesting to see the results. I think these tests will push security products to strive hard. But what is the practical implication of such products remains unknown.

 

I want to see how well works the HIPS on win7 x64 and the difference with 32bit

 

Yes me too.

 

Sounds great I can't wait to see some results.

 

How does x64 SBIE fare under these tests compared to 32 bit?

 

That's a big problem with Matousec's tests.
He does not give evidence of the practical value of his tests.

At the Firewall Leak Tester site were comments like "Trojans that use this technique: W32.Welchia.Worm, The Beast", which can be still found here (Techniques employed by leak-testing software):
http://www.matousec.com/info/articles/introduction-firewall-leak-testing.php

Apart from that I think everyone who performs comparison tests with real malware will see, that apps like Mamutu or ThreatFire do way better than Matousec's results indicate.
Therefore I don't think his tests have a great practical value.

Cheers

 

Welcome to the land of common sense and reason.

 

With run/internet restrictions in place the several I tried couldn't execute.

 

I'm afraid we will not meet many testers in this land.

Cheers

 

At least is the only serious test on internet, those 148 test are ways that the malware use for infect the computer, if an intelligent HIPS can block this, it would be a very good security app.
Also this tests helps to find bugs in the code:

http://forums.comodo.com/leak-testi...l-challenge-t30896.0.html;msg222603#msg222603

http://forums.comodo.com/feedbackco...e-milestone-t49428.0.html;msg355458#msg355458