DTL Home Anti-Virus Protection Annual Report 2015

Discussion in 'other anti-virus software' started by IBK, Oct 7, 2015.

  1. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
  2. J_L

    J_L Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    8,738
    DTL? Oh... Still not Windows 10 test though, just another W7 one.
     
  3. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Interesting. First recent AV lab report I have seen that "bombed" Bitdefender. If this report holds true, doesn't bode well for all the AVs using the Bitdefender engine.
     
  4. haakon

    haakon Guest

    Not Windows 10? Recent? This from the 4-digit post count members. :rolleyes: And yet another furthering of the "Bitdefender engine" nescience.

    This report, on the DTL home page, is dated 07/10/15. Which is somewhat convoluted considering:

    On page two:
    Document version 1. 01. Written 15th April 2015. Edited 7th October 2015: Corrected typographical errors.

    Page three:
    The tests were conducted regularly during the previous year (2014)...

    How relevant would Car & Driver's reporting on 2014 cars be when the 2016 models are in the showrooms? And assuming nothing in the driver's landscape has changed in any way.

    This report is an interesting read and that's all it is. It certainly is not about anything 2015.
     
  5. Oldjim

    Oldjim Registered Member

    Joined:
    Sep 7, 2005
    Posts:
    99
    What the heck does this mean
    The way I read this is that the other tests weren't with a fully patched system which makes the test results almost totally useless
     
  6. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Yes that was confusing for me too. So they were running unpatched OS and were trying to see which AVs would protect against exploits. I wonder what would be results for all AVs on fully patched OS.
     
  7. Rolo42

    Rolo42 Registered Member

    Joined:
    Jan 22, 2012
    Posts:
    571
    Location:
    USA
    It means that, for the test, all updates are frozen to the same date to keep a level playing field; it wouldn't be accurate to test one product updated, say, 1 January and another product updated 30 January.

    As far as age of the test goes, it establishes reliability. Do you want a product that performs well every month, every year or is "the best" this quarter but lackluster otherwise?
     
  8. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Cutting through all the noise that are in this test and focusing on the essence of this test - then the conclusion are : Patching works.

    That's it. Everything else are irrelevant.

    Test are done through 2014.
    Test are done on Win7 that has only had SP1 installed and nothing else. Not a single Microsoft update applied.

    Some products did better, some did less - totally irrelevant.

    The Windows 7 SP1 system with MSE, but without ANY Microsoft updates got a protection score of 58%

    The important thing to notice are that during this test in 2014 they also had a single system running that where allowed to be kept fully updated with all Microsoft updates.
    This always fully updated Win7 SP1 system with MSE installed, achieved a protection score of 99% while not having any false positives at all.

    Perfect !!

    This is the thing that all those users that love to complain about the mandatory and cumulative updating in Windows 10 better notice.

    This is why Microsoft have chosen to change how updating are done.

    Having the entire user base always fully updated are a huge benefit for the security of the entire Windows ecosystem.
     
  9. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    That's true, but then again why only one AV tested with OS patched and others not? That doesn't seem fair to other AVs, does it? If updating raised protection of MSE from 58 to 99%, then other AV's would probably get 100% if tested on patched OS.
     
  10. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Depends what you're testing. If it's security effectiveness against exploits, then you want to do so on an unpatched system.
     
  11. Rolo42

    Rolo42 Registered Member

    Joined:
    Jan 22, 2012
    Posts:
    571
    Location:
    USA
    They didn't:
    Since they did this, they tested MSE on an updated system in addition to testing it the same as all the third party products.
     
  12. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    OK, so we can ignore MSE's 99% protection result (58% is comparable result) as this only shows how patching OS will improve security and doesn't tell us anything about AV efficiency.
     
  13. Krusty

    Krusty Registered Member

    Joined:
    Feb 3, 2012
    Posts:
    10,210
    Location:
    Among the gum trees
    MS probably know on an unpatched system what is vulnerable and being exploited. They provided the patches so their AV had less to do... maybe?
     
  14. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Why ignore it ??

    As the report clearly states - Patching works !

    As the report also clearly states :
    And that combination of a FULLY patched system with MSE gave a protection score of 99%

    Security are a moving target.

    If you are driving in your brand new car, do you then need a blacksmith riding along just in case the horse drops a shoe ??

    If anyone are still confused by this test, then they really ought to read the full report and not just a quick glance at the charts.
    Because it highlights a subject that are massively in the media currently - updating and keeping yourself fully patched and how this benefits your security.
     
  15. Oldjim

    Oldjim Registered Member

    Joined:
    Sep 7, 2005
    Posts:
    99
    In addition to Windows being unpatched after SP1 this is, in my opinion, even more critical
    So they presumably patched all these on the fully patched system
     
  16. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Title of that report is "Antivirus protection and performance". Patching has nothing to do with it. Especially when only one contender was tested with fully patched system. It might show that patching works but doesn't show which AV works best under fully patched OS. And we're back at the title of report.
     
  17. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    As with everything else in life, you really have to extract the meaningful essence.

    It has no relevance who did what on a unpatched system.

    What will you use the information for ??

    Do you honestly believe any of the tested vendors support not updating your OS for a year ??

    Do you honestly think they care about that portion of the report ??

    They will tell anyone who show up with a question on helpdesk, to ALWAYS patch. To always be updated.

    Not everything in life are a "mine is bigger than yours"-contest.

    So many of the test we see from testing institutions are flawed in one way or the other - mainly because they fail to test and rank according to prevalence.

    This test has a irrelevant portion about protection on an unpatched system.

    But it has a very valuable portion about the importance of patching and how it affects security of the OS.

    Focus on that and try and forget the additional ranking.

    The content are more important than the title.
     
  18. Rolo42

    Rolo42 Registered Member

    Joined:
    Jan 22, 2012
    Posts:
    571
    Location:
    USA
    I think that would be a most foolish thing to do. Unless you make it a habit of rarely updating your OS (which is also most foolish), why would you do that? Out of the box, Windows provided 99% protection for no more cost that what you paid for the OS. And with less overhead. I would think this would have great appeal to one named, "Minimalist"! :)

    Patching has everything to do with it:
    • After URL filtering, patching is the greatest guard against exploits as it eliminates vulnerabilities
    • If you're going to test AV software, then test AV software without its reliance on the OS contributing to the effort
    • Since an updated OS provided 99% protection, how much credit is inaccurately given to 3rd party software?
     
  19. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Yes, I do employ updates as part of my minimalist setup and don't even use real-time anitimalware at the moment. :) But that was not my point and I think that you know it :)
    Testing all AVs on unpatched OS and then give one of them chance after updating OS is IMO not really good AV testing procedure. It would be better to test all of them on patched OS.
    So I will rephrase my statement: "We can ignore MSE's 99% protection result when we try to compare how different AVs perform in this tests."

    But we can have different opinion. It's just an AV test, not a big deal :)
     
  20. Rolo42

    Rolo42 Registered Member

    Joined:
    Jan 22, 2012
    Posts:
    571
    Location:
    USA
    That's isn't what they did; like others have said, read the entire report and take it in context.
     
  21. ttomm1946

    ttomm1946 Registered Member

    Joined:
    Jul 23, 2014
    Posts:
    217
    I was really surprised on the mediocre Bit Defender score
     
  22. ttomm1946

    ttomm1946 Registered Member

    Joined:
    Jul 23, 2014
    Posts:
    217
    they only tested webroot for impact and not protection..
     
  23. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    Oh, I bet they tested it for protection as well, yet once again it was decided that the methodology didn't reflect the blablabla.
     
  24. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Actually, I believe this test is more than suspect.

    If you read the methodology section, Dennis states it used malicious URLs and downloads but gives no details as to what these were. They even stated that they will not make these public or available to any AV vendor tested.

    Then there is the 99% effectiveness of MSE on a patched system. When was the last time for any recent AV lab test did MSE score anywhere near that figure? Those tests are always done on fully patched systems.

    If I "read between the lines" of this report, I think Dennis wants us to believe that this was a test simulating average usage PC activity. The problem is defining exactly what that is.
     
  25. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    This is simply not true.

    If you read the report it clearly states :
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.