AV Performance Statistics

Discussion in 'other anti-virus software' started by Blackcat, Jan 18, 2007.

Thread Status:
Not open for further replies.
  1. lucas1985

    lucas1985 Retired Moderator

    Joined:
    Nov 9, 2006
    Posts:
    4,047
    Location:
    France, May 1968
    Folks, I think that the tree is stopping you to view the forest. Discard engines known to generate many FP and known to flag any runtime packed executable (Fortinet, Panda, eSafe, Sophos, CAT-QuickHeal). Discard little known/new engines (Ikarus, Sunbelt, UNA). Now read the statistics again:
    -Antivir.
    -BitDefender.
    -PrevX.
    -Kaspersky, NOD 32.
    -VBA32.
    -Dr. Web.
    -Norman.
    -McAfee.
    -AVG, Avast.
    Doesn´t that have a certain sense?
    Now, my doubts:
    -why is NOD 32 behind BD and in the same level as Kaspersky? It is in the level of Antivir.
    -why is Norman behind VBA32 and Dr. Web? I think that it is between both.
    -why Kaspersky scores so high? They are quick at adding signatures but they heuristics are not so good.
    -where is F-Prot?
     
  2. lodore

    lodore Registered Member

    Joined:
    Jun 22, 2006
    Posts:
    9,064
    that does have some sence.
    but bitdefender does have hourly updates like kaspersky does and very good heristics so that could give it the edge in this random test.
    lodore
     
  3. lucas1985

    lucas1985 Retired Moderator

    Joined:
    Nov 9, 2006
    Posts:
    4,047
    Location:
    France, May 1968
    I´m not questioning the score of BD, they have worked hard enough to occupy that position. But NOD 32 should be in the top-three (Antivir, BD, NOD 32) and not together with Kaspersky.
    Also, the PrevX score is nice, the automated research in each customer and the central database seem to produce good results. I´m not sure how corrupted samples affects PrevX, though.
     
  4. lodore

    lodore Registered Member

    Joined:
    Jun 22, 2006
    Posts:
    9,064
    im quite surprised kaspersky and nod32 arent higher up in the chart.
    lodore
     
  5. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    I need more information from the maintainer of this page in order to better judge its validity.

    Now "all detected malware" - does this mean even packed/crypted files that are erroneously flagged by say eScan are recorded as "malware"?

    If so this would be a major flaw in the already inaccurate detection percentages.
     
    Last edited: Jan 19, 2007
  6. fcukdat

    fcukdat Registered Member

    Joined:
    Feb 20, 2005
    Posts:
    569
    Location:
    England,UK
    With reguards packers and malware surely there is one or or 2 vendor employees here that receive detected files from the VirusTotal service and would be privvy to whether these packed files are infact dud or *live* malwares ;)

    This would cover some of the bases:)
     
  7. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    im not surprised by nods place, but let me state i dont think this test is accurate in any way at all.......

    if i were to be surprised, it would be mcafees.
     
  8. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    I find it interesting that Fortinent is ranked #1 on two different "independent" reviews that obviously have suspicious testing methods w/ skewed results. I doubt AV-Comparitives will even consider it for their next review. :blink:
     
  9. lucas1985

    lucas1985 Retired Moderator

    Joined:
    Nov 9, 2006
    Posts:
    4,047
    Location:
    France, May 1968
    Fortinet will be included in the 2007 tests of AV-Comparatives.
     
  10. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    Excellent. At least we'll finally have a current credible review of their product. Maybe someone can start a poll on which certification level it will reach. ;)
     
  11. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,218
    Location:
    The land of no identity :D
    No need for that, detection rates are good enough, the results will speak for themselves. :)

    I do, however, suspect that Fortinet will produce lots of FPs.
     
  12. elgallego

    elgallego Registered Member

    Joined:
    Jan 26, 2007
    Posts:
    1
    Here is study of false positive detection of runtime packed files. eSafe, Fortinet, Ikarus, Quickhel and Sunbelt:thumbd: say notepad.exe is infected if it is packed with UPX, ASPack or Yoda:
    http://blog.hispasec.com/laboratorio/189
    (in spanish only)
     
  13. fcukdat

    fcukdat Registered Member

    Joined:
    Feb 20, 2005
    Posts:
    569
    Location:
    England,UK
    Finally a break from bot collecting,sorry for the delay but here's my sincere reply to all that question the reliability of those results.

    Ok we will have to differ on our views on the merits of reported data
    This is my defence of how i interpret the results and the debunking in this thread that is evident.

    I personally accounted for a pecentage of those files uploaded to VT service,i would estimate well over 4 figures in files since the birth of MIRT in November06.

    Do i upload my specialized packed tools....do i upload legitimate packed files as with debunking in this thread ?

    No on both accounts,i upload files imported by malware infections fo mal checking and if 10 or more of the DB's are'nt calling them out i'm uploading to MIRT(700+ todate).This results in the loss of about 80% of checked malwares but the upload's still happened@VT.

    Sometimes the only detections are by these so called *packer* sensitive heuristic's at that point in time the suspect heur's have checked a malware file for their user's potentially.

    Maybe this is not precision science but surely checkpoints for new malware such as this are not a bad thing ?

    In finalising i dare suggest that the amount of actual malware files uploaded to VT against legitmate packed files is probaly a massive percentage not acknowledged by anyone here.

    So for some carry on debunking like politicians,if the results don't go your way at the polls try spinning your way out of it :rolleyes:

    I'm back off to continue uploading suspect malware files :)
     
    Last edited: Jan 30, 2007
  14. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,102
    Location:
    North Carolina USA
    Again, Again, detection is one thing, detection and removal is another. Until someone can show this for the truth, everything else means crap.
     
  15. Durad

    Durad Registered Member

    Joined:
    Aug 13, 2005
    Posts:
    594
    Location:
    Canada
    3 months ago I sent HijackThis.zip to Fortinet because it was detected as malware. Also 4-5 months ago I did the same for McAfee,,
     
  16. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
  17. Dr. Lucien Sanchez

    Dr. Lucien Sanchez Registered Member

    Joined:
    Dec 16, 2006
    Posts:
    44
    Location:
    England
    AntiVirus Graph.

    I was wondering about the results of this graph. It seems to show the likes of NOD32 and Kaspersky as being extremely average. Anyone want to comment? I was under the impression that NOD32 and Kaspersky were as good as it gets as far as anti virus is concerned.
     
  18. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: AntiVirus Graph.

    in a word, 'bullshit' :D

    thats my opinion, keep with nod and kaspersky or whatever, it is better than that graph says. w
     
  19. kof

    kof Registered Member

    Joined:
    Jan 8, 2007
    Posts:
    56
    Re: AntiVirus Graph.

    ...
    truth
     
  20. plantextract

    plantextract Registered Member

    Joined:
    Feb 13, 2007
    Posts:
    392
    Re: AntiVirus Graph.

    i agree. first of all fortinet detects anything packed with certain run time packers as suspicious, panda isn't that vicious but also has some detections like that.
    a good look would be at the av-comparatives test.
     
  21. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: AntiVirus Graph.

    sure av-comp is a good test and always has interesting results, but i like those av-test.org ones, they test a hell of a lot av's, when is there next test results published, anyone know?
     
  22. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    702
    Re: AntiVirus Graph.

    The graph is interesting, but the test samples contain lots of false positives and garbage executables. Also, the test doesn't show you the high false positive rate some of the scanners have, they are running in "paranoid" mode on Virus Total. Also, the scan results are incorrectly rated. For example, information messages in the scan log are rated as detection.
     
  23. Macstorm

    Macstorm Registered Member

    Joined:
    Mar 7, 2005
    Posts:
    2,641
    Location:
    Sneffels volcano
    Re: AntiVirus Graph.

    Who said biased graph? :D
     
  24. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Re: AntiVirus Graph.

    it is unprofessional and all bs do not take anything from it at face value.
     
  25. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    Re: AntiVirus Graph.

    It doesn't look like it's been updated since February 9 anyway.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.