Matousec: Proactive Security Challenge 64 (bits)

Discussion in 'other security issues & news' started by guest, Dec 11, 2011.

Thread Status:
Not open for further replies.
  1. guest

    guest Guest

    Proactive Security Challenge 64 (2011/12/01 11:26)

    Proactive Security Challenge 64 starts and it is not just an update to another platform.

    Yes, one of the changes is that the 32-bit Windows XP SP3 platform used in the original Proactive Security Challenge is replaced with 64-bit Windows 7 SP1 platform, but that is not the only change. Naturally, Proactive Security Challenge 64 uses Security Software Testing Suite 64 that we presented last time. The new suite comes with lots of improvements, new and harder tests. The structure of levels has been carefully redesigned so that in Proactive Security Challenge 64 the very bad products will still score very low, sometimes possibly little better than in the original Proactive Security Challenge; the products of medium quality should score better, sometimes much better; and the very good and excellent products will have it much harder to achieve the perfect score. So, for most of the products, the challenge is easier and will allow better scoring, but for the top products it will be harder to achieve the same score as before, at least at the beginning. In other words, the new suite and the new system of levels is better balanced than before.

    We start with just four products, but in a couple of weeks there will be more. The results of Jetico Personal Firewall confirm that it is not that easy to get really high score. The performance of the new ESET Smart Security 5 suggests that its vendor might want to join the club but it will take some time before its engine matures. The results of McAfee Total Protection and Panda Global Protection are not surprising. Unless a big change happens here, they will be on the bottom of the result table. See the first results of Proactive Security Challenge 64 yourself.

    permalink
     
  2. wat0114

    wat0114 Guest

    It's kinda funny they rate ESET SS very poor, yet display a prominent advertisement for it to the left of the results :rolleyes:

    Jetico has not been updated by the developer since last December :thumbd:
     
  3. J_L

    J_L Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    8,516
    It's more relevant, I guess. Where's all the other participants?
     
  4. guest

    guest Guest

    "We start with just four products, but in a couple of weeks there will be more." ~Matousec
     
  5. Night_Raven

    Night_Raven Registered Member

    Joined:
    Apr 2, 2006
    Posts:
    388
    The tests themselves are good but it's too bad they still keep this absurd level structure, as it destroys all objectivity. Have to to tests myself if I want accurate and true results.
     
  6. guest

    guest Guest

    Quite agree, the level structure is just silly.
     
  7. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,148
    Yeah, I don't see the point in the level structure either. Malware doesn't come in levels... you don't need one infection before you can get another.
     
  8. Night_Raven

    Night_Raven Registered Member

    Joined:
    Apr 2, 2006
    Posts:
    388
    Indeed. It's as if AV-Comparatives don't test a certain antivirus program against trojans unless it scores well against viruses first. It makes absolutely no sense from a logical and/or practical standpoint. It does make sense from a marketing standpoint, though, and that's all that matters nowadays. Unfortunately.
     
  9. guest

    guest Guest

    Latest news

    2011-12-21: New results have been published for:
    AVG Internet Security 2012.0.1890
    avast! Internet Security 6.0.1367
    Avira Internet Security 2012 12.0.0.832
    PC Tools Internet Security 2012 9.0.0.898

    This time we wanted to give you a product that scores better than others, possibly attacks the 80% bar. This is why we included PC Tools Internet Security in this update. It was a big and unpleasant surprise that its performance in Proactive Security Challenge 64 was extremely poor. If you remember its excellent results from Proactive Security Challenge, you must be surprised too. On 64-bit Windows 7 SP1 platform, the tested version of PC Tools Internet Security 2012 succeeded against just a few tests and its final score is as low as 6 %. We hope that the related issues in its protection will be solved quickly and that PC Tools Internet Security will strike back among the top products soon.

    Since PC Tools Internet Security performed poorly, it was avast! Internet Security that finished with the best score among the products tested this time. Yet still its performance is not good enough. It offers only a basic protection against some of the attacking techniques leaving malware plenty of ways to bypass its protection. Its final score is 15 %, still deep in the red zone.

    AVG Internet Security and Avira Internet Security remain consistent in their poor results. Nothing new on this field – 3% score for AVG, 4% for Avira. Both products pass only the most trivial tests.

    Source: http://www.matousec.com/projects/proactive-security-challenge-64/#latest-news
     
  10. Baserk

    Baserk Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    1,317
    Location:
    AmstelodamUM
    Or as in a real-world translation;
    On Win7 SP1 x64, PC Tools failed at test nr. 7 out of a hundred, so we decided to stop testing and thus give it a score of 6% effectiveness because it didn't achieve any scores at tests 7 to 100.
    Meh, bizarro world level structure testing indeed.
     
  11. guest

    guest Guest

    Latest news

    2011-01-13: New results have been published for:
    Comodo Internet Security Premium 5.9.219863.2196
    F-Secure Internet Security 2012 12.44.110
    G Data TotalSecurity 2012 22.1.0.0
    Privatefirewall 7.0.25.5

    Comodo Internet Security did it again! There were few tests that it failed this time and it is probably just a matter of time for this product to reach the perfect score in our challenge again. Comodo Internet Security outclassed all products that have been tested in Proactive Security Challenge 64 so far. With 94% score its position on the top of the table is stronger than ever before. Congratulations!

    Privatefirewall looked very promising on lower levels, but harder tests on higher levels did not allow this popular free product to get to the top. 56% score currently means 3rd place in Proactive Security Challenge 64. Privatefirewall's core is solid, its self-protection is almost perfect and the autorun protection control is very good. Having long term experience with this product, we expect that Privatefirewall will be improved in the future and will strengthen its position on the top of the result table.

    F-Secure Internet Security 2012 and G Data TotalSecurity 2012 did not surprise us. Although G Data TotalSecurity performed little better than F-Secure Internet Security, its final score is as low as 5 %. The implementation of many of its features is incomplete – for example the autorun control or the self-defense features. F-Secure Internet Security then passed even less tests and finished with 3% score.

    Source:
    http://www.matousec.com/projects/proactive-security-challenge-64/#latest-news
    http://www.matousec.com/projects/proactive-security-challenge-64/results.php
     
  12. Night_Raven

    Night_Raven Registered Member

    Joined:
    Apr 2, 2006
    Posts:
    388
    That's odd. I tested Comodo Firewall (v5.8 though) myself and got a significantly lower score than that. Also, results were different with the enhanced protection disabled/enabled. Smells fishy to me. I suspect there is something going on behind the scenes between Matousec and Comodo.
     
  13. guest

    guest Guest

    Nothing odd, nothing fishy, your "suspicion" is baseless. I'm fairly confident that you're ignoring Matousec's methodology and rules. To reproduce Matousec's results, you should follow Matousec's methodology and rules - step by step - and you should also test the same product version they tested.

    Go study these links:
    http://www.matousec.com/projects/proactive-security-challenge-64/#methodology-rules
    And especially:
    http://www.matousec.com/projects/proactive-security-challenge-64/faq.php
     
  14. Night_Raven

    Night_Raven Registered Member

    Joined:
    Apr 2, 2006
    Posts:
    388
    I'm aware of their methodology. Difference was I used the latest available version at the time. However, I find it rather impossible for a product to gain ~25% over a single version.
    I might retest the latest version when I have the time.

    Also, considering Comodo's reputation, past, and even present, I don't think it's at all baseless for me to suspect some foul play.
     
  15. CloneRanger

    CloneRanger Registered Member

    Joined:
    Jan 4, 2006
    Posts:
    4,833
    The good thing is, you can DL & run ALL the tests yourself & see how your Apps fare. I've run them ALL in the past, & a Lot more besides & was more than happy with the result, on this older OS & Apps :D :thumb:

    *

    No, but it "appears" to be holding it's own in spite of that :) Not all older stuff is past it's sell by date ;)
     
  16. wat0114

    wat0114 Guest

    Yes, I agree it does very well. It's really an excellent firewall in so many ways. I've a license for the latest version, but I haven't used it for a while, having settled on Windows fw. The lack of development concerns me not because it can't pass all the Matousec tests (I don't really care much about those), but because it indicates the developer may not be focusing on it, which either means he's going to drop it alltogether, or fall behind in keeping it functional with current and future Windows versions, where a monthly patch could, at some point, interfere with the fw's functionality.
     
  17. guest

    guest Guest

    Remember to read this: http://www.matousec.com/projects/security-software-testing-suite-64/

     
  18. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    Not much products tested :D
     
  19. guest

    guest Guest

    Latest news
    • 2012-01-16: Responses from Comodo Security Solutions, Inc., the vendor of Comodo Internet Security Premium, and PWI, Inc., the vendor of Privatefirewall, have been added.


    Comodo Security Solutions, Inc. – the vendor of Comodo Internet Security Premium

    2012-01-16 (Comodo Internet Security Premium 5.9.219863.2196 scored 94 % and took 1st place): Thank you for including our product in your latest tests. Your open-source, open methodology tests are always very well thought out and unique. The latest results indicate the fact that there is a reason for us to be able to offer a 500 USD virus free guarantee in all operating systems. Keep up the good work!
    Egemen TAS,
    Director, Desktop Security Products,
    COMODO


    PWI, Inc. – the vendor of Privatefirewall

    2012-01-16 (Privatefirewall 7.0.25.5 scored 56 % and took 3rd place): While a #3 ranking is not very satisfying when combined with the Protection level earned by Privatefirewall in this initial round of the new x64 PSC, we are aware of the work required on our end to dramatically improve our performance and the effort to do so is already underway. We thank the Matousec organization for their continued commitment to helping vendors improve their products and ensuring that the consumers and businesses that rely on them are able to realize the greatest levels of system protection.
    Greg Salvato
    CEO - Privacyware

    Source: http://www.matousec.com/projects/proactive-security-challenge-64/#latest-news
     
  20. J_L

    J_L Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    8,516
    Great to see more results and responses.
     
  21. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,731
    Location:
    localhost
    Updated: http://www.matousec.com/projects/proactive-security-challenge-64/

     
  22. xxJackxx

    xxJackxx Registered Member

    Joined:
    Oct 23, 2008
    Posts:
    4,052
    Location:
    USA
    Interesting that as poorly as ESET did, they still beat Norton by more than 3 times over. I'd like to see the Russian vendors added to this, they usually do pretty well.
     
  23. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    This is a HIPS test but still interesting. :D
     
  24. hayc59

    hayc59 Updates Team

    Joined:
    Oct 29, 2008
    Posts:
    2,138
    Location:
    R.I.P. Roger(roddy32)
    Oh So Cool NOT
     
  25. guest

    guest Guest

    Source.
     
Loading...
Thread Status:
Not open for further replies.