Why do these two tests have so many differences ?

Discussion in 'other anti-virus software' started by aluckystar, Jan 19, 2007.

Thread Status:
Not open for further replies.
  1. aluckystar

    aluckystar Registered Member

    Joined:
    May 30, 2006
    Posts:
    66
    Location:
    Paris of the East
    I found a site of antivirus software testing recently, it is called “Malware-Test”. You can see it here.

    The test team was founded by some experts once worked for Trendmicro.

    I found that the result of this test had many differences with the test done by AV-Comparatives.

    For instance, NOD32 v2.5 , in AV-Comparatives , the detection percentage is 98,61% . At Malware-Test , the result is 49.51%.

    Another example, Kaspersky and Dr.web both performs good in these two tests. Kaspersky's detection percentage is higher at AV-Comparatives, Dr.web's detection percentage is higher than Kaspersky at Malware-Test.

    After read thest two tests ,you can find the statistics had so many differences. Some differences are very obvious. Can someone tell me why o_O?

    PS:Malware-Test provide a kind of test that shows the treatment rate of anti-virus software, I think this is very useful. I hope AV-Comparatives could also do this kind of tests. Thank you.:)
     
  2. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    :thumb: Unfortunately malware-test.com is not a trustworthy source and their results are extremely skewed to say the least.
     
  3. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    "some experts" ... "once worked for Trendmicro" ...

    As what? The test is flawed to maximum.
     
  4. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    I found this info on the website:

    About us

    Anyway, the tests are done using samples got from a honeypot. Which is why the test is inaccurate, or so I think.
     
    Last edited by a moderator: Jan 19, 2007
  5. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
  6. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    if you look at http://malware-test.com/smf/index.php?action=dlattach;topic=3125.0;attach=47 you see that
    a) some products (if not all) were just trial versions, which means they had not the full potential as the full versions (= it is not true that they had the best possible settings enabled, cause in some trial versions that is not possible) and
    b) not all products were updated the same day.
     
  7. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    701
    Samples from honeypots? I hope they carefully sort out the garbage/damaged samples - which make up most of the files you get from this source.

    Obviously they don't do that - the NOD32 detection rate is a good indicator for this.
     
  8. PaulBB

    PaulBB Registered Member

    Joined:
    Jan 2, 2006
    Posts:
    708
    I don't know who's behind the Malware-Test, i don't know who's behind AV-Comparatives, I'm not working for anyone of them etc...but since i'm a member of Wilders forums i observed something interesting: Any test that doesn't rate NOD32 detection rate 98% is inaccurate and not a trustworthy source. How does it sounds ? :gack:
     
  9. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    You want an honest answer? That sounds stupid.

    Why would someone working for another av company than ESET (Example: Stefan and myself) write their doubt's? We're not working there. Or in my case no longer anymore. Don't you think your statement is quite "bold"? I mean you should have noticed that somebody else confirmed that too, not only the usually NOD users. That should ring somehow a bell.
     
  10. rothko

    rothko Registered Member

    Joined:
    Jan 12, 2005
    Posts:
    579
    Location:
    UK
    if any of the top tier AVs scored such a low result then it would be cause to doubt the test, it isn't just a NOD32 thing.
     
  11. malcan

    malcan Registered Member

    Joined:
    Dec 17, 2006
    Posts:
    22
    The virus trends in Asia could be diffrent from in Australia, I think.
    I thought IBK works for them.o_O
     
  12. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    I am the "chief" of av-comparatives :D
     
  13. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    And if you will look at all the reports at AV Comparitives done from last year, you will see certain ones where Nod wasnt the winner. So that pretty much blows your theory out of the water.
     
  14. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    Hmm Nod didnt win this one. IBK does a great job at ensuring things are as fair and equal as can be. In his summazation of all the products for the year, yes Nod won, but he also stated that Avira will stand a great chance of taking that honor this coming year if they keep up the good work.
     
  15. ianlai

    ianlai Registered Member

    Joined:
    Aug 14, 2006
    Posts:
    6
    well done last year

    cheers:D
     
  16. malcan

    malcan Registered Member

    Joined:
    Dec 17, 2006
    Posts:
    22
    So, you own av-comparatives. Cool. :D
    Sorry for my ignorance. :oops:
    Keep up your great work! :D
     
Loading...
Thread Status:
Not open for further replies.