A little request of explanation about votes on av-comparatives

Discussion in 'other anti-virus software' started by Lucas Malor, Oct 18, 2007.

Thread Status:
Not open for further replies.
  1. Lucas Malor

    Lucas Malor Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    7
    I want to do some little question, without any intention to start polemics.

    I noticed two antiviruses, AVG Anti-Malware and BitDefender Prof.+, are scored by Av-Comparatives as "Advanced+":

    http://www.av-comparatives.org/

    IMO they not deserve this vote, because:

    - number of polimorfic viruses detected by AVG Anti-Malware are only 3 of 12.
    - the percent of Script viruses/malware detected by AVG Anti-Malware is only 77,66%
    - the percent of not categorized malwares detected by BitDefender Prof.+ is only 75,07%

    Ok, the total detection rate is high (97,75% and 97,51%), but IMO they are not comparable with Nod32 (12 of 12, 97,60%)

    A second question: can you write in the Retrospective test results also the percent of false positives and the scanning time? It could be useful also to report these info for the normal test.

    The last question: do you have done the retrospective tests with the max available heuristic, or with the program default value?

    PS: my compliments for the good job :)
     
    Last edited by a moderator: Oct 18, 2007
  2. Frank the Perv

    Frank the Perv Banned

    Joined:
    Dec 16, 2005
    Posts:
    882
    Location:
    Virginia, USA
    po·lem·ic (pə-lěm'ĭk) Pronunciation Key
    n.

    A controversial argument, especially one refuting or attacking a specific opinion or doctrine.

    A person engaged in or inclined to controversy, argument, or refutation.
     
  3. Macstorm

    Macstorm Registered Member

    Joined:
    Mar 7, 2005
    Posts:
    2,531
    Location:
    Sneffels volcano
    You just have started the III world war ;)

    I bet you are a nod32 fan! :rolleyes:
    .
     
  4. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    No, actually a Perv I think.;)
     
  5. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    you could argue lots of points, but this is the way it is.

    Advanced + for Nod32, yet it still missed 15,000 malware ?

    id like to see IBK do a removal test, that checks just how much of the threat(S) were removed after detection.
     
  6. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    (1) no [scan speed on infected files? nonsense]; you can find the scan speed on clean files in the report. You can find the list of FP's in the report, a percentage is not (and will not be) given.
    (2) highest settings, as stated in the report.
     
  7. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,050
    It will be ended very quickly, if this thread goes in that direction.

    Pete
     
  8. ggf31416

    ggf31416 Registered Member

    Joined:
    Aug 20, 2006
    Posts:
    314
    Location:
    Uruguay
    I don't represent AV-comparatives

    The main test is total detection, not polymorphic detection. And IMHO 12 viruses is not enough to get reliable results.

    Script viruses are only 2% of the total test-set. Other OS (mac, linux, etc) malware (not "not categorized malwares") are less than 0.35% of the test-set

    Why not?

    Read the report. You will find the number of FP and the scanning speed.

    Again, read the report: "The same products, with the same best possible detection settings that the scan engines had in the last comparatives, were used for this test"
     
  9. larryb52

    larryb52 Registered Member

    Joined:
    Feb 16, 2006
    Posts:
    1,126
    why is this thread around, I'm surprised it's still open...
     
  10. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    hey IBK,

    if Fortinet recieved over 1000 false alarms, this would indicate a 'minimum' test set of 1000 for the false alarm test, right?

    if so, why is 36 (ie. drweb) enough to lower the certification, also... the other AV's that were lowered too.
     
  11. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    17,877
    Location:
    New England
  12. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
  13. Lucas Malor

    Lucas Malor Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    7
    Thanks for the answers!

    Well, you are right :p I forgot On-Demand scan is tested only on virus files :oops:

    Thanks :)

    I don't understand what you mean. In the report is written:

    IMO failing the detection of 9 polymorphic "families" is very bad for AVG. One polymorphic virus was never detected, as the report shows...

    Whops, you are right :p

    I read them, but I think it could be good if these values are showed also in the Online results, between parenthesis :) I think these values are interesting also for "unskilled" ones :)

    I want to ask a request: it's possible to do also a retrospective test for the default antivirus level of heuristic? It would be very useful if you want to know if it's better to raise (or lower) the default setting :)
     
  14. Lucas Malor

    Lucas Malor Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    7
    No answers?

    Mmmmmhh.... nothing? o_O
     
  15. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    nothing.
     
  16. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    that means a no, maybe it also means a bad day too :p
     
  17. GES/POR

    GES/POR Registered Member

    Joined:
    Nov 26, 2006
    Posts:
    1,490
    Location:
    Armacham
    IBK keep ya head up :thumb:
     
  18. Lucas Malor

    Lucas Malor Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    7
    Yes....

    Or a great rudeness :mad:

    I only asked an enlightenment and I've done two little suggestions. And IMHO the second one (a retrospective test also with default heuristic level) is very valid.

    I tried to sweep aside polemical comments (posted even if I started this thread asking moderation), but I think this go too far. I can ask a calm and mature answer? or I'm asking too much?

    I am unable to describe the very stupid way I was welcomed. It's usual to threat here in this way all newbies, all controversial, all suggests? I thought av-comparatives is a serious project, and I usually suggest to take a look to its site. Now I'm starting to think the contrary. I hope some av-comparative member can change my impression, because actually I'm very disappointed.
     
  19. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Re: Yes....

    It is, indeed. Because not every customer visits a forum after a purchase and just uses "out of the box" settings (default heuristic)

    There are several (valid) reasons not running all the time with highest heuristic settings. For example:

    System Impact (Slows down because more files driven thru advanced/additional heuristic)
    False Positive Problem (Especially a serious problem for corporates)
    and last but not least because the customer trusts your default settings as sufficient.
     
  20. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    Re: Thanks for the answers!

    it is usually better to raise it.

    P.S.: "nothing", "no comment", ":blink:", etc. means usually "no" or that I currently do not want to reply. Even if I usually reply in all forums to various questions, I am not obligated to reply in forums; if you need an answer it is sometimes a bit more probably that I answer to emails. It is nothing personal and I do not look at the number of posts of the poster.
     
    Last edited: Oct 25, 2007
  21. Lucas Malor

    Lucas Malor Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    7
    Well, so I prefer you don't reply, instead of some sarcastic reply (because it was sarcastic....).
    "Asking is allowed, answering is polite".

    Thanks anyway to you and Inspector Clouseau (it's what I thought about. Scanning time + false positives comparison for different levels could help you to choose the right heuristic for your purpose).
    My idea it's only an idea, if someone of av-comparatives wants to apply or consider it, ok, otherwise amen.

    So long.
     
  22. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    My "no reply" was not understood by you, so I posted something ;). [yes, I am often sarcastic]
    About if it will be applied: maybe in future, but not yet.
     
  23. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    lol, i did tell him it meant no. :rolleyes:

    i personally think the same test should be done with default aswell, but it just adds to the work for you IBK o_O

    but before any other request, i want a removal one.
     
Loading...
Thread Status:
Not open for further replies.