New AV-Test.org malware testing (Avira finished 1st, CA eTrust finished last)

Discussion in 'other anti-virus software' started by InfinityAz, May 23, 2007.

Thread Status:
Not open for further replies.
  1. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    The thing is that I'm pretty sure not everyone knew WHEN Andreas was going to publish the test results. Therefore, 15th and 16th May was in the conference, 17th May all analysts get home, and 18th May products were updated. How much time to add samples? One day, or maximum 2. The fact is that major analysts were at the conference, it is very difficult to send 15GB worth of samples to other company analysts via email, so the samples were likely received via disks or through download links (unlikely because the collection was already being distributed), so most likely it was distributed on DVDs.

    So in the end, analysts had only the day of 17th and maybe some small part of 18th may to add definitions. How many would they add in that period? It would be very difficult for such additions to cause any significant impact on the test results.
     
  2. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    Oh? :)

    Sorry then, my mistake :)
    Thanks for clarifying. :D
     
  3. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    The others also did get it, you know, with no effort on their part. Its not like VBA32 had to go request those samples from Marx, from what I see Marx gave those samples of his own will. :)
     
  4. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    ren,

    you bring up a very interesting point. Why would Andreas Marx share samples at the conference before the actual test itself?

    1) To show some of the problems AV-test has faced in the past and present with regards to sorting samples
    2) To give some vendors the samples beforehand so as to reduce his workload later

    Out of these scenario 1 is more likely and yes it is quite possible he may have showed older samples, because since these were used only to showcase problems, old or new doesn't really matter. The samples might still be important for companies which are new, i.e. for example VBA32.

    Of course, if the reason is number 2, then I have explained this in my previous posts ;)
     
  5. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    Much credit goes to Andreas Marx for his sharing of samples with the AV companies who didn't attend, this is a true testament of his willingness to help out the end users in their protection level(s).

    As for sorting and adding/sorting 15gb worth of signatures in 3 days for an unannounced test, sorry no way.
    Maybe md5/crc32 detection but no "real" detections.

    Correct me if I am wrong Inspector/IBK/Stephan/AV experts in general :)
     
  6. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    5,906
    Location:
    New York City
  7. Durad

    Durad Registered Member

    Joined:
    Aug 13, 2005
    Posts:
    594
    Location:
    Canada
    VBA32 is not that new company ;) They are just new for English market.
     
  8. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    For that matter, neither is Rising. But they still deeply appreciate samples no matter how old or new they are, because not being in the English market for so long they have not quite paid focus for the malware spreading around in the English speaking regions. ;)
     
  9. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    As an update to this thread, I did contact Mr.Marx about it, and he was kind enough to explain in detail a few comments about this latest test from AV-test.org. Since the comments are quite detailed by nature, I think its better to do this post by post.

    1) Regarding the somewhat "strange" detection rates of Rising, NOD32 and F-Prot when compared to AV-comparatives (for example):

    The statement makes good sense to me, and I agree that an AV which does well on detecting newer threats provides an effective protection for "today's malware". :)

    More to come in following posts.
     
  10. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    2) Regarding the version of AVG tested (AVG Pro or Anti-Malware)/Some comments on product selection in general

    So basically, AVG's Anti-Malware edition was used, which would explain the quite good detection rates (not that AVG Pro is bad though). As you can also see, there is some interest from the magazines in the AVG AS product, so it is also tested. :)

    Those editions and versions are tested which are requested by the magazines. :)
     
    Last edited: May 25, 2007
  11. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    3) I did ask about why AV-test does not test ArcaVir and also about why there are detection rate differences between F-Prot and Command AV. You can see the comments below:

    - Regarding ArcaVir:
    - Regarding F-Prot and Command AV:
    So basically F-Prot will get better over time, but I am wondering why Authentium has not upgraded to the 6.x engine. :doubt:
     
  12. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    4) I did also ask about whether AV-test would test Virus Chaser (its anyway like Command AV in relation to Dr.Web because it has its own definitions files along with Dr.Web's database). But the comments probably will apply to test any particular AV in general, as long as its more than just a clone. :)

     
  13. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    Mr.Marx also kindly took the time to address some of the other concerns that were being displayed/showed/posted here on Wilders.

     
  14. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    And as the last paragraph of the email Andreas also wrote this :)

    And with this, I end my long series of posts :p :D

    I would like to thank Andreas Marx for taking the time to explain all this in detail, it is very highly appreciated. I hope these comments help people to understand this latest test a bit better. :)

    Oh BTW, Mr.Marx has already seen this thread :D

    As you can see, the tests are still very darn reliable, IMO they're as reliable as it gets. :)
     
  15. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,024
    Location:
    Christchurch, UK
    Thanks, Firecat for obtaining this information :thumb: . Very interesting reading.

    Let's hope these emails are not pulled :p
     
  16. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    Firecat, thank you for taking the time to contact Andreas Marx and posting the Q&A session. IMO it would have been nice to see the results of this test PRIOR to him giving out the 16+GB worth of samples. In other words some companies had 1+ week to add the samples to their signatures before the test started.
     
  17. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    I do have written permission from Mr.Marx to publish those comments, so I think the comments won't be pulled. :)

    @EliteKiller: Mr.Marx already said that the 16GB worth of samples was already uploaded on their server at May 6 and all AV companies were informed of it shortly thereafter. The workshop was the "second chance" and the email sent on May 23 was a "last chance" email to inform vendors that the samples will be deleted soon, so its best to get them now. :)
     
  18. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    Firecat, I definitely understand the part about offering the samples before the review, which was the basis of my last post. All I was implying is that it would have been nice to see the review before the AV companies had a chance to add the 16GB worth of samples to their databases. IMHO it's no different than a professor giving the class all of the answers to the test a week or two in advance. This explains why so many had extremely high detection rates. Then again this also shows us which companies are slow to add samples. Does AV-Comparatives release their 500K samples to vendors and perform the review afterwards?
     
  19. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,873
    Location:
    Innsbruck (Austria)
    no. .
     
  20. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    IBK, thanks for the swift reply. :D I edited my previous post to include additional information right before you replied. What are your thoughts on AV-Test releasing 16GB worth samples in advance and performing the review afterwards?
     
  21. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,873
    Location:
    Innsbruck (Austria)
    I am not going to comment it. I do not agree with some other things, but different peoples have different opinions - it does not necessarly mean that one of the two peoples opinions are wrong.
     
  22. colt45allstar

    colt45allstar Registered Member

    Joined:
    Jun 9, 2006
    Posts:
    65
    Hardly most Canadian beer is overrated unless it's Unibroue.. their beers are awesome! American beer is more than Anheuiser Busch, Miller and Coors as well you know. American microbrews are near the best.. only topped by Belgian Ales :D

    Now then interesting results indeed. Avira has become quite impressive. It seems to be leading the way in detection with most tests and I applaud them for that.

    I'm fine with Kaspersky and don't see myself ever switching again.. but if for whatever reason my mind was ever changed, Antivir would no doubt be the one I would try.
     
  23. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,225
    Location:
    The land of no identity :D
    For that matter, there is nothing to prove that the 16GB collections contained only the very same samples that were used in the test. It is important to note that Marx said the collections contained "more important samples, which are already available from other sources to the AV companies". :)
     
    Last edited: May 25, 2007
  24. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    True, but there is no denying that 16GB is a enormous amount of samples, and the companies that were able to implement the "new" samples into their databases prior to the testing had an advantage. Why not release the samples after the tests are completed and finalized?
     
  25. Abeltje

    Abeltje Registered Member

    Joined:
    Aug 24, 2006
    Posts:
    156
    Location:
    Netherlands
    Those companies that were able to add them prior to the test might simply be better in adding new signatures. What's wrong with that? It's not that some companies got the samples and others didn't, everyone had equal chances. By the way, it should not be the task of antivirus product testers to supply samples to the vendors. I would expect that the vendors are able to add threats themselves. You can see who's best in doing so by looking at the results. And then arguing about whether some samples are maybe less important and you will hardly encounter them .. even if there is only a slight chance of getting infected by whatever action I take, I want to be protected by my AV, that's what I have it for. I find it alarming that NOD32 which I used for years now couldn't detect some 70000 samples of threats that someone was able to spot around on the internet during the last 12 month. I like Nod32 for many things, but I'm using Avira for the moment and hope that ESET will react to the recent tests (AV-comparative test wasn't that encouraging either).
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.