AV-Comparatives Real-World Protection Test Overall Report (August-November)

Discussion in 'other anti-virus software' started by Gobbler, Dec 15, 2014.

  1. Gobbler

    Gobbler Registered Member

    Joined:
    Jul 30, 2010
    Posts:
    270
  2. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    5,054
    Thank you for sharing. Nice results from top scorers :thumb:
     
  3. tgell

    tgell Registered Member

    Joined:
    Nov 12, 2004
    Posts:
    1,073
    Qihoo 360 IS still doing good. I wonder if avast's results include NG. November test was with avast! 10. November improved over October. Hopefully it will continue to improve.
     
  4. anon

    anon Registered Member

    Joined:
    Dec 27, 2012
    Posts:
    4,087
  5. iforget

    iforget Registered Member

    Joined:
    Mar 29, 2012
    Posts:
    15
    Location:
    Riderville, Canada
    While I am always interested and enjoy these tests I would like to see more information such as.

    Numbers on how prevalent these malware are.
    Where geographically they are prevalent.
    % on how many malware were blocked by url filter, blocked on download, blocked on execution and blocked after execution. This would show the differences in an antivirus that has good url filtering but poor protection from plugging in an infected thumb drive etc.
    These changes would make these kind of tests a lot more relevant and informative to me.
     
  6. Mayahana

    Mayahana Banned

    Joined:
    Sep 13, 2014
    Posts:
    2,220
    Trend is really cleaning up.. Wait for 2015, when the 2016 product launches, and some of the other things are fleshed out going on behind the scenes. I personally feel the 2015 product is very good, but needs a few things ironed out, which I am really hoping happens during the beta for 2016.
     
  7. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,071
    Location:
    Germany
    Trend is still trending (yes, I love this stupid pun).
     
  8. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,954
    Location:
    DC Metro Area
    I see the rollover chart as misleading. It would be better if it emphasized as numbered percentages the number of complete compromises that were not detected or blocked. Not including user dependent intervention in the over-all positive percentage scores does not seem appropriate, The user dependent choices are usually obvious. Why should a product that allows more compromises without warning appear on the roll-over scores to be better than a product that blocked or detected and warned of a higher number of threats but required more user choices, for which it typically gives a recommendation, which usually is an obvious choice in any event? The results have an atypical ratio of comprises vs. blocked+user intervention.

    IMHO, for Wilders followers, the best product is the one that had the least amount of outright comprises, rather than those that "blocked" the most. Just my 2 cents.
     
    Last edited: Dec 15, 2014
  9. J_L

    J_L Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    8,516
    Noticeable improvement for Avast on November... Guess they weren't just developing side attractions after all.
     
  10. Yuki2718

    Yuki2718 Registered Member

    Joined:
    Aug 15, 2014
    Posts:
    1,257
    User dependent should be given penalty, otherwise AV which flags everything suspicious can earn perfect score except FP test, and AVC is not mainly focused on those security geeks but general public who often don't know security and sometimes even override "recommended" action, especially when the product is FP prone.

    Also don't confuse warning with user dependent. In AVC test, they firstly choose wrong decision and if product still finally blocked the threat it is counted as blocked, not user dependent.
    So user dependent really means user dependent, if user choose wrong decision he will be infected even if it was just a miss-click.

    Anyway, I don't see that needs as we can easily add user dependent rate to blocked rate.
     
  11. ArchiveX

    ArchiveX Registered Member

    Joined:
    Apr 7, 2014
    Posts:
    1,015
    Qihoo - 360 ! :thumb:
     
  12. DX2

    DX2 Guest

    Is Qihoo 360 the same as 360 English? And does it use the same AV engines? If so were all the AVs active during this test?
     
  13. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,954
    Location:
    DC Metro Area
    Thanks for the info. :)

    Guess I'm a bit confused which I'd rather have--a product that only has .3% complete compromises and 1%+ user dependent or one that has .6% complete compromises and no user dependent. I think it can be argued that the former offers better protection. Not much of an argument I admit, but still somewhat of a valid point IMHO. Weird cuz the two I'm looking at use the same engine, one using an additional engine also.

    And what is Trend doing that the others are not. They have been topping every recent test?
     
    Last edited: Dec 16, 2014
  14. Cruise

    Cruise Registered Member

    Joined:
    Jun 10, 2010
    Posts:
    1,024
    Location:
    USA
    Hey guys, help me to understand something here... BitDefender scored in the top grouping, but Lavasoft's Ad-Aware (which uses the BD engine) was near the bottom ...how does that figure?
     
  15. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,954
    Location:
    DC Metro Area
    Not sure , but even on it's website Lavasoft says "Ad-Aware will closely match Bitdefender's impressive test results in independent lab tests. With such a powerful engine at its core Ad-Aware 11 will provide serious protection against even the most advanced threats."

    Of course here it didn't even come close-something is really off. Perhaps Lava Soft doesn't offer updated definitions as frequently or uses a different set. Does seem extremely
    incongruent. Does LavaSoft use BF's cloud data?

    It may be that Ad Aware only uses BD's engine for file scanning.

    http://www.lavasoft.com/products/compare.php
     
  16. garrett76

    garrett76 Registered Member

    Joined:
    Mar 18, 2014
    Posts:
    210
    Two things: Ad-Aware free (which is the version tested by Av-Comparatives) does not have a web-protection against malicious websites (unless you install the web companion application) and, above all, it does not have the active virus control, which is the behaviour blocker of Bitdefender. The latter makes a big difference in catching unknown malwares during the execution.
     
  17. FOXP2

    FOXP2 Guest

    garret76 correctly beat me to it as I was composing this off-line. :) But to expound further...

    What you need to understand is you (an just about every one else) think generically: Ad-Aware, Bitdefender, period. It's Ad-Aware Free vs Bitdender Internet $ecurity - I hate those complicated technical details.

    There is much, much more to Bitdefender and their licensed SDK than the "Bitdefender engine."

    AAF uses the bdcore.dll (aka the "engine") which has real-time signature scanning and B-Have heuristics. A downloading file will write, re-named, to a system temp folder for an analyze with both, and upon passing, copying it to your download folder with the correct name.

    BDIS and Ad-Aware Pro and Total adds Active Virus Control (AVC) which is a monitor that attaches separately full-time to every running process, a firewall with IDS, and Web/phishing filtering. I've posted up more detail on AVC, BD's flagship protection, in a couple of other posts here. Ad-Aware Personal does not have the firewall/IDS.

    AAF's Web/phishing filtering is an opt-in during install that installs their stand-alone Web Companion, but it's not BD-based and uses independently compiled lists. AV-C does not clarify if the WC is/was used during their testing.

    For about eight months on two systems I'm running Pro and Personal, I've observed Ad-Aware's BD signature updates (which occur on the average every 1.5 hours) and they're nearly 100% in sync with BD's timeline and rarely more than an hour or two behind. The small incremental updates are checked for by default every hour in all of Ad-Aware's products.

    Three days ago I installed BD Windows 8 Security on a test rig running Win 10 TP and in the dozen or so times I've checked the updates so far, Ad-Aware matched every time. The contents of the plugins folder in AA mirrors that of those in BD's.

    While a delta 10% in the Compromised metric being a close match is somewhat wishful and up for discussion, it most certainly is not "didn't even come close."

    Finally, Ad-Aware also has Lavasoft's stuff in the mix (there's a boatload of engine libraries running in real-time) that present additional layers of protection after the download, as does bdcore.dll in Free (plus the extras in the paid versions). I stand to be corrected on this, but I don't believe this particular AV-C test addresses that. UPDATE: I read AV-C's pdf on this test process and I stand corrected.

    Cheers.
     
    Last edited by a moderator: Dec 16, 2014
  18. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,818
    Location:
    Innsbruck (Austria)
    the Lavasoft SafeBrowsing add-on (preselected) which aims to block malicious websites is used.
     
  19. FOXP2

    FOXP2 Guest

    @ IBK :thumb:

    Web Companion recently replaced the Security Toolbar, both being referred to as "Lavasoft SafeBrowsing" by the Ad-Aware installer opt-in. The Toolbar was a, well, toolbar useful only to a few supported browsers. The Companion uses their proprietary TCP service and filters all URL traffic via the local proxy and is a far more efficient solution which relies on a local database updated daily as well as a Lavasoft cloud server(s) for real-time.

    http://toolbar.lavasoft.com/ - http://webcompanion.com/

    So, your test... using Toolbar or Companion?

    Thank you!
     
  20. Cruise

    Cruise Registered Member

    Joined:
    Jun 10, 2010
    Posts:
    1,024
    Location:
    USA
    So I would ask you, since Lavasoft uses the BD engine can you explain the large difference between the Bitdefender and Ad-Aware scores?
     
  21. FOXP2

    FOXP2 Guest

    Lavasoft's Safe Browsing has nothing whatsover to do with Bitdefender.
    And if posts #16 and #17 above didn't explain anything, then for you there is no explanation. :'(
     
  22. Cruise

    Cruise Registered Member

    Joined:
    Jun 10, 2010
    Posts:
    1,024
    Location:
    USA
    Judging from your (very informative) post #17, I got the sense that they both utilize the very same AV engine and signatures, so that's why I'm still perplexed. Did I misunderstand your meaning?
     
  23. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    Note: eScan uses Active Virus Control (as well as the BD cloud), yet often does not have the same detection rate in real-world protection tests. It is, thus, not just dependent on usage of AVC, as it seems to involve interplay of AVC with other components in the suite.
     
  24. nine9s

    nine9s Registered Member

    Joined:
    Feb 8, 2013
    Posts:
    265
    Location:
    USA
    Why are user dependent results so discounted? I like the idea of Emsisoft alerting me versus automatically doing stuff.
     
  25. Yuki2718

    Yuki2718 Registered Member

    Joined:
    Aug 15, 2014
    Posts:
    1,257
    Too sad, why nobody mention about network-based IPS except FOXP2's #17 (but only 1 line). Probably NIPS is the least recognized feature in modern AV/SS in this forum.
    According to Symantec, 44% of detection by SEP is by antivirus engine (sig and heur). Next, IPS is 42%. Insight 9%, and SONAR only 5%. Past (before 2013) retrospective test results also shows that actually BB don't occupy much of blocking rate.

    IBK explained me that most major AV can well protect against known exploit, however at the same time also confirmed there's difference in effectiveness against obfuscation technique for exploit. Considering in dynamic test even 1% rate makes difference, such de-obfuscation capability is important factor to explain results.

    I have to say BD's anti-exploit capability is not excellent according to all exploit tests available from 2012 to 2014. However, I think it's still much better than those not-big name such as eScan or Lavasoft, probably they even can't protect against some of not-well-obfuscated exploit.
     
    Last edited: Dec 17, 2014
Loading...