Matousec: Proactive Security Challenge 64 (bits)

Proactive security challange 64 bit

http://www.matousec.com/projects/proactive-security-challenge-64/results.php

Passing
Comodo - 94 %
Privatefirewall - 88 %
Kaspersky Internet Security - 86 %
Outpost Security Suite Pro - 86 %

Failing
VirusBuster Internet Security - 71 %
Jetico Personal Firewall - 59 %
ESET Smart Security - 51 %

<50%
ZoneAlarm Extreme Security - 43 %
Total Defense Internet Security - 21 %
avast! Internet Security - 15 %
Dr.Web Security Space - 11 %

<10%
- Avira
- Bitdefender
- Norton
- PC Tools
- Webroot
- G- Data
- Panda
- AVG
- Bullguard
- F-secure

Numbers speak for themselves. However is really disheartening to see so many companies claiming 64 bit "compatibility" and failing to provide adequate protection. It's also interesting to see that just because you rent your software doesnt mean that you will get better protection. Private FW score #2 and Outpost score #4. And so much rentware simply failed..

 

Re: Proactive security challange 64 bit

I believe Matousec only tests the HIPS capabilities of products. These are not really reliable and I think the usefulness of Matousec has been questioned on this forum for a long time.

 

Re: Proactive security challange 64 bit

Of course they are testing HIPS. It is proactive security challenge.

 

Re: Proactive security challange 64 bit

So one of the "failing" firewalls can be better or the same as the top rated ones"? I don't care about HIPS testing as I run AppGuard, and EXE Radar pro, with SBIE. Come on, test firewalls only! That's what I need to know about in a "Firewall test". Thank you.

 

Re: Proactive security challange 64 bit

I'm just saying I don't think it's very real world. There is no way BitDefender, etc. would do that poorly against threats, even zero-day stuff.

 

Re: Proactive security challange 64 bit

They no longer test OA?

 

Re: Proactive security challange 64 bit

They ("matousec") , OA decided not to pay the ransom fee.

 

Re: Proactive security challange 64 bit

They use Internet Security suites; disable all components except the firewall and then see if the firewall, on its own, blocks their test suite. No wonder many products are not doing well. With all components enabled every product there will do better than what their results are suggesting.

 

Re: Proactive security challange 64 bit

cheap

 

Re: Proactive security challange 64 bit

Thought it's the other way around?
They do not test "Firewall" only HIPS.

 

Re: Proactive security challange 64 bit

Exactly my point, far from real world. Most security suites don't have HIPS, so it makes sense the main ones like COMODO, PFW do really well, since they were made to stop those kinds of attacks. But kudos to Kaspersky for doing so well; I'm guessing it will be AV of the year again.

 

Re: Proactive security challange 64 bit

Also keep in mind, that many products may actually be a lot better than what the rating suggests. The last time I checked, unless you pay Matousec, they will stop testing your product as soon as you fail one of the levels. All skipped levels will automatically count as failed. So if they have 10 levels and your product will fail the very first one, but would have passed all others, you will end up with a < 10% rating, although you would have ended up with a > 90% rating if they had continued testing.

 

Re: Proactive security challange 64 bit

Just what I said! Sorry I did not dumb it down.

 

Re: Proactive security challange 64 bit

just remember something that when online armor was tested not even the super comodo did better than OA

 

Re: Proactive security challange 64 bit

Bad move for Emsisoft.
OA was always among the best.
They should pay cause it is advertisement for the product.
You always have to invest some ammount of money in marketing.

 

Proactive security challange 64 bit

Agree.

Matousec is charging a Testing Fee
like other Security software Testing Organizations.

-Classic Firewalls fail...
-Firewalls+HIPS prevail...

 

Re: Proactive security challange 64 bit

Exactly, he is not really testing firewalls, he is testing HIPS, so his results are a pile of ... hmm, a junk.

 

Re: Proactive security challange 64 bit

A junk if look at it as a fw test, it's an HIPS test.
The only thing wrong with this test is that not all the products are HIPS or have HIPS capabilities.

 

Re: Proactive security challange 64 bit

Pros of Matousec approach:
1. Proactive testing - it is a good test of ability of a product to protect from unknown threats.
2. You can check their results with their test suit.

Cons:
1. As it was already told by Fabian Wosar in previous post, Matousec stops testing a product as soon as it fails one of the levels. And all further levels are automatically counted as failed. This is a great drawback of the methodology.
2. They have strange approach to choose products. When there was only 32-bit challenge I asked them by email why they omit DefenseWall. They answered that they refused to test DW only because a user cannot tweak some options there.

Matousec tests are rather ambivalent, like everything else in our world.

 

Re: Proactive security challange 64 bit

~ Off Topic Comment Removed ~
Still, i don't really believe this, because if it was true, Comodo would have reached the high score because it failed only in the last test(s). It would be a tremendous coincidence that the best security suites failed exactly at the very last tests. This being the only way to explain what you wrote.

 

Re: Proactive security challange 64 bit

~ Off Topic Quote Removed ~

You can read about this on their site. In paragraph "Testing levels".

 

Re: Proactive security challange 64 bit

That is a shame really, for if he did do so I'm sure his results would have been the same as mine when I downloaded the 32- bit version of SSTS, in which DefenseWall passed all tests on Xp for a 100% score. Again this was only a crude test that I did and I'm in no way an expert on these matters, but I am/was very pleased/satisfied with my results.

 

Re: Proactive security challange 64 bit

I didn't try Matousec tests myself but I know other guys did that for DefenseWall. It passed all 32-bit tests.

 

Proactive security challange 64 bit

Let's clarify things a little bit:

-Matousec is NOT solely testing HIPS.

-Matousec is testing Firewalls WITH HIPS and Firewalls WITHOUT HIPS.

Now, if the Firewalls WITH HIPS
are scoring more than
Firewalls WITHOUT HIPS
is it a big surprise?

-Isn't it what someone should normally expect?