Matousec: Proactive Security Challenge 64 (bits)

Discussion in 'other anti-malware software' started by fax, Jun 22, 2013.

Thread Status:
Not open for further replies.
  1. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,726
    Location:
    localhost
    This thread looks locked.... :rolleyes:
    Matousec: Proactive Security Challenge 64 (bits)

    ... so I am posting a new thread with the same title.

    http://www.matousec.com/projects/proactive-security-challenge-64/
     
  2. The Red Moon

    The Red Moon Registered Member

    Joined:
    May 17, 2012
    Posts:
    3,871
    Not much difference from any other results...same old..same old.

    Some of the products tested seem out of date too.
    I see nothing about this testing that is of any relevance.

    Thanks.
     
  3. 0strodamus

    0strodamus Registered Member

    Joined:
    Aug 23, 2009
    Posts:
    1,047
    Location:
    United Surveillance States
    I was hoping to see Outpost 8.1 and Spyshelter 8.4 tested.
     
  4. tomdy2k

    tomdy2k Registered Member

    Joined:
    Jun 6, 2010
    Posts:
    174
    Bitdefender 2013 fails...Hard to believe:rolleyes:
     
  5. 0strodamus

    0strodamus Registered Member

    Joined:
    Aug 23, 2009
    Posts:
    1,047
    Location:
    United Surveillance States
    Does Bitdefender 2013 have a strong HIPS?
     
  6. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    FYI It's not an Antivirus test, but a HIPS test.
     
  7. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    3,764
    Location:
    Outer space
  8. Baserk

    Baserk Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    1,317
    Location:
    AmstelodamUM
    ^That's because of Matousec testing procedure. link
    A product needs to pass 11 different levels of tests. Only after passing 50% of all tests in level 1, you qualify for level 2.
    If you fail in level 1 (less than 50% score), testing stops and hence, you have failed to pass all tests in levels 2 to 11.
    And that's why you see so many 'none/very poor' protection scores.

    edit; Some argue this results in poor representation of actual protection, others argue that a security program that can't even qualify 50% of the first run of tests, deserves 'poor standing'.
    Me, I'm not touching that with a 20-foot pole.
     
    Last edited: Jul 15, 2013
  9. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    3,764
    Location:
    Outer space
    Yes, that is likely, but Matousec tests at maximum settings afaik, and on the Anti-malware.ru test, on maximum settings, BD scores 100% against both basic and complex techniques, so you'd expect it to reach something higher than level 2 on Matousec.
     
  10. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,726
    Location:
    localhost
    http://www.matousec.com/projects/proactive-security-challenge-64/
     
  11. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    Last edited: Aug 8, 2013
  12. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    That appears to be "outbound" test. Is that just leak test or what and how does it compare with Matousec's tests. I've been trying the demo of BD based in part on Matousec's last published test of BD that gave it a 97% Excellent rating. That was for BD Total Security 2011. What,if anything has changed in BD to account for the latest horrid scoreo_O Or is it a change in Matousec's testing procedure? For some reason I have not been able to download the BD pdf, but if it failed the self protection test what good is it? I dropped Norton last year for that reason and switched to KIS. Looks like I'll be staying with KIS.
     
    Last edited: Aug 8, 2013
  13. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    If a firewall stealths all your service ports do you still have to be concerned if it gets a very low score on Matousec's testing results?

    If your av suite has excellent detection, if the firewall doesn't pass one of Matousec's tests would the antivirus part of the suite protect your system and data?
     
  14. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    3,764
    Location:
    Outer space
    If you check the detailed report from Anti-Malware.ru(largely in English: -http://www.anti-malware.ru/files/adm/test/firewall_test_outbound_2013.xlsx-) and look at the methods, they look like general HIPS test to me, just like Matousec. Of course if you can defeat a HIPS, then you can also form unwanted outbound connections, so both test are in a way also outbound tests.

    About the BD results: Mastousec's rating/testing method with different levels is debatable, so perhaps the difference would have been less big if they tested it on all levels. However, I just saw, the big difference with BD 2011 test and 2013 is, that 2013 is tested on 64 bit, so perhaps they are having trouble with 64 bit. The good results from Anti-Malware.ru are from 32 bit, so that seems to support my suspicions. I don't really know BD products well but afaik the HIPS is disabled by default and probably not many of their customers enable it, so it's development has low priority. Norton also has a disabled by default HIPS hidden away in the settings and theirs doesn't work well on 64 bit either.


    Only products with a HIPS component score well on Matousec's tests, so if yours doesn't score well that doesn't mean you have to start worrying. However, Anti-virus' base protection is primarily based on black-listing,(though a lot have added new proactive modules) so you should use one that has good proactive modules; generic exploit prevention, sandboxing, advanced behavioral protection, limiting rights of untrusted files, white-listing etc. It differs with the vendor and to make it more complicated they usually use different terms and give it some cool sounding name so you don't know what it actually does and technical explanations on their sites are usually limited. There is usually more info on their forums and here on Wilders of course. If you find it too technical or too much trouble, most AV tests called Real World Protection test usually have methods that allow a product to use it's proactive technologies, so results of those tests can give a good view. If you find a test from a less known organization, make sure they execute their samples and the machine has internet access so proactive modules can be tested. You could of course also add some stand-alone proactive program like Sandboxie, Defensewall, AppGuard, NVT EXE Radar, VoodooShield etc.
     
  15. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    I asked a similar question on the Bit Defender Forum (firewall section) without specifically mentioning Matousec.. I had a generic statement to the effect that it had done very poorly on a recent test published on a website that is fairly well known for testing firewalls. I also noted that some posters on a security website I frequent were surprised at the BD results and that they could be described as a failure.

    They deleted my question. LOL

    Truth hurts I guess.

    There are numerous posts on the BD forum about the 2013 firewall being shut off with no warning. The forum moderators claim that the problem has been fixed in the latest edition ("2014" though unnamed). Wondering what version Matousec used.
     
  16. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    BoerenkoolMetWorst,

    Thank you so much for taking the time to write the above post. It was very informative, analytical, and shows a deep understanding of the subject by you.
     
  17. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    2,969
    Location:
    U.S.A.
    I always viewed Matousec's testing as leak testing; nothing more, nothing less. A modern version of Comodo's old CLT tests. If your the type that frets over any possible outbound leak on your PC, the tests are relevant. I have other things to worry about such as making sure nothing inbound gets into my PC that is unwanted and/or malicious.
     
  18. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    So would you feel comfortable using BD because it stealths all ports. Would that protect it from inbound threats or is stealth just a gimmick as i have heard others say.
     
  19. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    oOps My Bad. My question is still there. Interesting is the fact that there is a BD 2013 forum and a "New" Bitdefender forum. Why are av companies like Norton and BD moving away from branding their new products with a year?
     
  20. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    2,969
    Location:
    U.S.A.
    Stealthing ports falls into the category of "minimizing attack surfaces" category. Simply put if an attacker can't "see you', it makes it more difficult to attack you. It does not guarantee that you can't be attacked. Note: today with the wide use of broadband routers, port stealthing is done primarily by router's firewall. So if the software firewall doesn't do it, your still protected.

    Another mitigation in this category is ICMP. By turning off echo reply for router and/or PC firewall, your PC will not respond to a hacker's ping request.
     
  21. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    3,764
    Location:
    Outer space
    Glad to have helped, and thanks for the compliment :)
     
  22. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,026
    Location:
    The Netherlands
    I´m still waiting for the ultimate HIPS which can pass all of these Matousec leaktests. :D

    But the whole website is really unclear. All these testing levels are so annoying.

    Plus it would be nice if you had a separate page with a quick description of all leaktests used.

    Also, what´s up with the result table (see link), it looks so silly. o_O

    http://www.matousec.com/projects/proactive-security-challenge-64/level.php?num=11
     
  23. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    2,969
    Location:
    U.S.A.
  24. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,026
    Location:
    The Netherlands
  25. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,726
    Location:
    localhost
    http://www.matousec.com/projects/proactive-security-challenge-64/

     
Loading...
Thread Status:
Not open for further replies.