AV-Comparatives June (May 2007) Results (Retrospective / Proactive Tests)

Discussion in 'other anti-virus software' started by AshG, May 29, 2007.

Thread Status:
Not open for further replies.
  1. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,024
    Location:
    Christchurch, UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Avast and Norman continue their improvement over the last 12-15 months.
     
  2. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    I might be way off mark here, but I see it like this: the malware in that sample would be detected by the given signatures named, but seeing as this is a proactive test, no signatures were used. Right or not?
     
  3. Londonbeat

    Londonbeat Registered Member

    Joined:
    Sep 21, 2006
    Posts:
    350
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Using the heuristics would probably not be recommended for a home user, but such paranoid heuristics could have advantages when used on a gateway.
     
  4. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    note that a proactive test is not a pure heuristic test. also (generic) signatures detections are for example counted as proactive detection.
     
  5. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Thanks for clearing that up. :)
     
  6. lucas1985

    lucas1985 Retired Moderator

    Joined:
    Nov 9, 2006
    Posts:
    4,047
    Location:
    France, May 1968
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    At the gateway level, this is acceptable.
    Thanks.
     
  7. profhsg

    profhsg Registered Member

    Joined:
    May 18, 2004
    Posts:
    145
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Where are all of you finding the actual report. When I go to the AV-Comparatives site at this time, the report doesn't seem to be up yet. Only the table.
     
  8. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    click refresh.
     
  9. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Well it basically raises the question for the "Award"-Triggers. My (personal!) opinion is that it is somehow "unfair" to rate AVG "Standard" and for example Bitdefender as well "Standard". AVG has a PA detection rate of 8%. Please excuse me, but that is close to "nothing". And it has "many" false positives.

    Now take a look at Bitdefender: it has 48% PA detection. That's a six times better detection rate. And... ...drum rolls... also "many" false positives. To be more concrete here, Bitdefender has 2 fp's less than AVG!

    There we go: We know now "for sure" that Bitdefender has

    1. Less false positives and
    2. That the detection rate is MUCH MUCH HIGHER (6 times!) than AVG's
    3. still they got the same "Award"

    I mean hey, we should also somehow judge the detection itself. Otherwise this would be a pure "false positive" test. For example all what scores below 25% Detection gets Standard. Because where do you draw the line? If something scores 1% - or lets be nasty and say 0% - and has "a few" false positives, would that also be "Standard"? If yes, that would be total unfair in regards to the other which also reached that "award" but maybe with much better detection rate and even less false positives (as the Bitdefender vs. AVG example proves).
     
  10. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    you make a good point IC

    i still think f-prot, drweb, bitdefender should be advanced,
    sure fprot and drweb and 26/36 (i think) FP's, but over a 20,000+ set, whats the percentage here?

    and as much as i hate to say it, avira deserves an advanced+, 18 fp's... again, its nothing over the set.
     
  11. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    I'm honored and delighted to hear that. Especially from you :D
     
  12. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    yes good point, but next time may i ask you to tell me your opinion in the four weeks time you and others had to discuss it with me (to eventually fix/improve something), instead of waiting for the final release... :rolleyes:
     
  13. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    just change it :)
     
  14. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    so you should be, i dont usually agree with the ENEMY! of F-prot, competing with my spider grrrr *lol*
     
  15. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Believe it or not, i didn't have the time to take a closer look to the results in the preview. I had to prepare speeches and was basically really busy with the workshop thingy. During the workshop you might have noticed that during the night there was some loud "noise" since you sleeped in my home office room. ;) Several av guys followed me home after our "drinking evening" because they thought that my home is the hotel. I don't think i need to mention the physical condition during that time. It wouldn't make sense to read the report, even the next day :p So.... well... I finally saw it today "closer". :D
     
  16. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    np ;)
    will put your point on the discussion panel, so it might maybe be applied in the next retrospective test, depending from the outcome of the discussion.
     
  17. JerryM

    JerryM Registered Member

    Joined:
    Aug 31, 2003
    Posts:
    4,306
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Re my former post. Won't anyone answer the question?

    Looking at Kaspersky the FPs were generated by Bit Defender Package. What is that package? I would have to assume that it is not a second AV application, but what is it?

    Thanks,
    Jerry
     
  18. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    its a file related to BitDefender.
     
  19. faenil

    faenil Registered Member

    Joined:
    Oct 25, 2006
    Posts:
    88
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    I've been using nod32 for 2 years...but I want to give a try to AntiVir...taking into account that it has got a better on demand detection...and better trojans-backdoors heuristic detection...
     
  20. one111

    one111 Registered Member

    Joined:
    Apr 6, 2005
    Posts:
    92
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    As far as I can see Avira stands head and heels over all the rest even according to this test. Check out the detection rates and you'll see that they're on top.
    The FP's have to be taken care of, but don't forget a FP can be quarantined
    and checked out whereas if a virus or trojan gets by it can destroy your computer!
    I'll stick with Avira
     
  21. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,251
    Location:
    The land of no identity :D
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    I know the deal with Dr.Web. This time, for some reason the report does not include the detail on whether it is a heuristic of signature detection, whereas in previous reports this information was there. Anyway, in Dr.Web engine there are three types of detections:

    1) "Pure" heuristic detection (BACKDOOR.Trojan, DLOADER.Trojan, etc.)
    2) Heuristic detection based on known behaviour after looking at signature ("modification of <malware name>", "<malwarename>.origin" with the 4.44 version)
    3) Signature detection ("<Malware name>", "<Malware name>.based")

    In earlier reports, there was a bit more detail regarding these detections from Dr.Web. This time, however, the detections are not detailed enough. Usually in Dr.Web, heuristic detection is preceded by "probably...", unless it is the second case described above, in which case it will be detected as "modification of <malware name>". In this report, the preceding sentences for the FP detections for all AVs have been removed.

    For example, if you have a heuristic detection from Dr.Web which says "probably infected with BACKDOOR.Trojan", then the November 2006 test reported the detection correctly. However, in this month's report, it is simply reported as "BACKDOOR.Trojan" instead of "probably infected with BACKDOOR.Trojan". This does not make the test any less reliable, but it makes reading the FP details of various AVs a bit more difficult for the discerning users.

    Also, for example, sometimes you may get a detection from Dr.Web like "modification of Win32.Swaduk.6891" (for example). Even in this case, the November 2006 test reported the detection correctly, but considering that for each AV the preceding text to the detection has been removed, "modification of Win32.Swaduk.6891" could very well have been reported in this particular report as simply "Win32.Swaduk.6891", which is potentially confusing heuristic detections with signature detections.

    That being said, the point of the FP tests is to show users how many FPs are being made by an AV, because if there are too many FPs made due to any AV's heuristic then the heuristic technology is not worth it, especially for some corporates and OEMs who have chosen to do this with some products (example: Webroot with Spy Sweeper+AV, NWI with Virus Chaser). At the same time, too many FPs even without heuristic is also not good for many people, hence they would avoid such products. Therefore, I wouldn't call this a major issue.

    The "Pure" heuristic FP detections from Dr.Web in this latest AV-comparatives report are as follows:

    Because of reasons mentioned above, the below detections mentioned in the latest report can be either Real signature detections, or detected by Dr.Web via variant heuristic as "modification of <malware name>"

    *After comparing the FP detections from Dr.Web between November 2006 and May 2007, it seems these detections are most likely detected by Dr.Web as "modification of..." type detections.

    I am not 100% sure about the following detections:

    I hope this was of some help in differentiating the heuristic detections from the signature ones...
     
    Last edited: May 31, 2007
  22. JerryM

    JerryM Registered Member

    Joined:
    Aug 31, 2003
    Posts:
    4,306
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    Thanks, but I wonder why it would be on the system. I know that 2 AVs do not work well together so I feel sure it is not BD AV. There is a BD Anti Rootkit. Maybe that is the one, but I just wonder why BD applications would be on the computer?

    Thanks,
    Jerry
     
  23. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    @Firecat: I will definitly reinclude more details in future retrospective tests. it was a bad idea to reduce the information. seems like many peoples liked it the way it was done before.
     
  24. SteveS335

    SteveS335 Registered Member

    Joined:
    Jan 16, 2007
    Posts:
    43
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    I think this line
    Needs to be removed from the test result.

    The only way anyone can make sense of these comparatives is to study the data and the report.

    The reasons have been posted above.
     
  25. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

    it also says: Please read carefully also the test reports and the other documents on our website to understand how to interpret the results.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.