Dennis Technology Labs 2013 Q4 Report

Discussion in 'other anti-virus software' started by Inside Out, Dec 19, 2013.

Thread Status:
Not open for further replies.
  1. aztony

    aztony Registered Member

    Joined:
    Sep 9, 2012
    Posts:
    737
    Location:
    The Valley Arizona
    Give this man a cigar!
     
  2. xxJackxx

    xxJackxx Registered Member

    Joined:
    Oct 23, 2008
    Posts:
    8,625
    Location:
    USA
    Glad to see ESET constantly improving in recent times. Version 7 is impressive. Good to see Kaspersky doing well also. The current patch d for the 2014 version is actually pretty stable at this point. I'm surprised to see Norton do that well. I was reminded the other day that they recently laid off a bunch of employees. Must be why the quality of version 21 (2014) seems to be slipping. I own licenses for all 3 and had to remove all copies of Norton. I liked it to start with but there have been too many bugs introduced. Overall an interesting report but mostly what I expected to see.
     
  3. zfactor

    zfactor Registered Member

    Joined:
    Mar 10, 2005
    Posts:
    6,102
    Location:
    on my zx10-r
    imo norton was awesome for 2009 and kinda great for 2010 and after that downhill from there
     
  4. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    according to this test it offers best protection?? but not quite best overall package
     
  5. larryb52

    larryb52 Registered Member

    Joined:
    Feb 16, 2006
    Posts:
    1,131
    yea I know Dennis great guy not sure to trust his tests tho, seems anyone with a name and a website can put up so called results and people believe them like they are gospel , look for my tests next week :) sorry these tests are getting out of line...
     
  6. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
    Does Dennis even have any involvement in Dennis Technology Labs? One person who is involved, as the Technical Director, is a founding member of the Anti-Malware Testing Standards Organisation and is the chairman of AMTSO's Board of Directors.
     
  7. si_ed

    si_ed Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    54
    Hello everyone. I trust you all had a nice break over the Christmas/New Year period.

    As usual I will address the interesting points raised following one of our tests.

    * "So few vendors"

    It's true. I wish we had more resources to test all of the products that people care about. We focus on low level detail, not vast numbers of products or malware samples. It's just not possible for us to include everyone's favourite product. However, every vendor is welcome to ask us to include their product in our tests. Put pressure on them and I'm sure you will not be disappointed.

    * "Dennis Labs have always been sponsored by Symantec in the past. Lately they declare they are unsponsored... Symantec doesn't show as No 1 in the final score, but if one looks at all their charts Symantec is always No 1 most of the time.It also remains to be seen why so many vendors were left out if the test is really unsponsored."

    We test products that are really popular. We have to be picky because of the work involved (see the answer above). We perform bespoke work for Symantec and almost every major anti-malware vendor you can name. That has no effect on the results that we produce.

    The logs that we provide to the vendors are so detailed that it would be far more effort for us to cheat than it would be to do the work for real. Ethics aside, it simply would make no financial sense for us to change results in the favour of Symantec, Kaspersky, ESET, McAfee etc etc...

    Standard response to accusations of corruption: produce some evidence if you think you have a case. (PS. You don't have evidence or a case).

    Also, are you sure that Symantec's products come in at the number one spot in the majority of cases? Check the last 18 months worth of reports. You may be surprised. As far as I can tell, out of the four 2013 tests the Symantec Norton product came first only once (Kaspersky came first the other three times) and in 2012 Norton came first in one out of two tests (Kaspersky came first in the other test).

    * "What's the Protected score for a test machine without any AV?"

    The answer is 'zero'. We only use threats that will work fully against the target system. We are testing the abilities of the security products to protect the system, so the threats need to work in every case. We have a load of identical systems available, all but one running anti-malware software. The one lacking protection will be compromised 100% of the time.

    * "yea I know Dennis great guy not sure to trust his tests tho..."

    No one working at Dennis Technology Labs is actually called 'Dennis'. I PM'd you asking for clarification about your point before the New Year but I'm yet to receive an answer.

    Dennis Technology Labs is part of Dennis Publishing Ltd, which is named after the owner Felix Dennis (http://www.felixdennis.com). I am certain he would be the first to admit that he knows nothing about testing security software.

    Best wishes to all for 2014,
    Simon
     
  8. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    From Wikipedia, the free encyclopedia

    http://www.opswat.com/about/media/reports/antivirus-october-2013

    Over 100 million customers is considered not really popular.
     
  9. MrBrian

    MrBrian Registered Member

    Joined:
    Feb 24, 2008
    Posts:
    6,032
    Location:
    USA
    Thank you for answering :).
     
  10. si_ed

    si_ed Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    54
    Not wishing to argue, but Opswat is just one metric. And, as far as consumers are concerned, I'm not convinced that its market share reports provide the sort of information you suggest that it does. In Opswat's own words,

    "Please note that OPSWAT is not a research institution and makes no claims on accuracy of this data in the real world marketplace; this report aims to distribute the unique data collected to inspire public discussion, not to make any claims as to why changes have occurred".

    In addition, as far as I understand it, Opswat usually measures the number of products that are *uninstalled*. So, in some respects, if Opswat says that Product X is popular is may actually be saying that it is very unpopular!

    In the report that you link to Opswat has updated it methodology and now includes metrics from its Gears tool. I doubt that many consumers use this "network security management platform for IT and security professionals".

    Si.
     
  11. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Likewise not wishing to argue, but what is your metric to evaluate popularity?
     
  12. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    Cant believe i missed this test! :D
     
  13. si_ed

    si_ed Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    54
    Dennis Publishing is the UK's largest publisher of computer magazines. Our readership goes some way to educating us about which products they know and care about. We receive letters and responses to surveys, although we've not run a survey for some time.

    I don't have exact market share figures but I would be surprised if Avira had a larger global market share than the other products in the test. But of course I could be wrong.

    Regardless, as I mentioned before, any vendor is welcome to engage with us, including Avira. Those guys are aware of us and our work thanks to our joint involvement in AMTSO.

    Si.
     
  14. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    So basically your metric is your readership in the UK ignoring the rest of the vast world of users... Whatever you think of OSPWAT, they seem to give a more reliable mirror of what the real situation is, not just a vague " I don't have the exact market share figures" and for a person who claims to be very accurate with its testings that seems to be a bit of a contradiction. Good luck with your enterprise, but I for for one can't take you seriously.
     
  15. si_ed

    si_ed Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    54
  16. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    :thumb:
     
  17. agoretsky

    agoretsky Eset Staff Account

    Joined:
    Apr 4, 2006
    Posts:
    4,033
    Location:
    California
    Hello,

    Downloaded a copy of the pre-released paper and had a glance through it. Good stuff; definitely not something to snooze though.

    Regards,

    Aryeh Goretsky

     
  18. Rompin Raider

    Rompin Raider Registered Member

    Joined:
    May 6, 2010
    Posts:
    1,254
    Location:
    Texas
    Good info...thanks!
     
  19. DoctorPC

    DoctorPC Banned

    Joined:
    Jan 9, 2014
    Posts:
    813
    Nevermind the bugs, 100% CPU spikes, and other annoyances that make it impossible to deploy.
     
  20. si_ed

    si_ed Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    54
    Thanks Aryeh. Good to hear from you! It's been a while. Did you mean this document?:
    https://www.wilderssecurity.com/showthread.php?t=358840
     
  21. agoretsky

    agoretsky Eset Staff Account

    Joined:
    Apr 4, 2006
    Posts:
    4,033
    Location:
    California
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.