AV Performance Statistics

Discussion in 'other anti-virus software' started by Blackcat, Jan 18, 2007.

Thread Status:
Not open for further replies.
  1. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,102
    Location:
    North Carolina USA
    Re: AntiVirus Graph.

    ROTFL :)

    I was thinking the same thing.;)
     
  2. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,238
    Location:
    Sydney, Australia
    This is interesting/disturbing: AV Ratings: shocking

    I am pretty sure about the bonafides of this rating system, but as of today my set-up aint looking so hot :gack:

    http://winnow.oitc.com/AntiVirusPerformance.html

    Bit of a slap in the face for a few other ; ahem; popular AV suites. :cautious:

    Prevx for one has in the past been happy to display this site as a reference to how well they are going; bites them in the ass now. :(

    This obviously may change day to day and week to week.

    (disclaimer : licensed PX user: entitled to carp on a bit now and then :D )
     
    Last edited: Mar 21, 2007
  3. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    Re: This is interesting/distirbing: AV Ratings: shocking

    hi longboard,

    this is already on the forum a few times, yes its complete bullshit, dont pay no attention to it. *lol* :D

    crazy results o_O
     
  4. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    Re: This is interesting/distirbing: AV Ratings: shocking

    If the process of sampling is good (random, etc.), then if they were to analyse the past data for FP's and corrupted files, etc. , and revise these statistics, we could get good info, although with a delay (analysis period).

    Aproximated info on 0 day threats.
     
  5. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,238
    Location:
    Sydney, Australia
    Re: This is interesting/distirbing: AV Ratings: shocking

    @CSJ
    Why so?
     
  6. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Obviously they are not using proper samples as Fortinet can just flag certian extensions with scanning the content. Fortinet cannot do so well unless they do that.
     
  7. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,238
    Location:
    Sydney, Australia
    OK, my ...cough ahem.. 'sensationalist headline' post/thread got shifted here

    (memo to self: search and search :oops: save the mods some time cleaning up after me)

    I have now read the whole thread and fwiw think fcukdat's post #38, and not withstanding this from Mr Kurtzahls
    and IBK's comments, has real relevance.

    I cant dispute methodology Q's from the experts but methodology is always going to be questioned.

    No-one seems to have absolutely debunked these results.

    Be interesting if any "affected vendors" commented.
     
  8. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,238
    Location:
    Sydney, Australia
    For any Discerning Doubters of "generic" packer detection I suggest you look at ronjor's latest:
    https://www.wilderssecurity.com/showthread.php?t=169383

    follow the in link the article to
    http://www.secureworks.com/research/threats/gozi/?threat=gozi

    Read it and weep for us all my friends:
    :eek:
    Serious stuff indeed.
    Regards

    Edit: PS: might be interesting if that website with the crappy data could provide a month by month or annual table. ??
    Ok getting a bit overexcited now... over and out. ;)
     
  9. BrainWarp

    BrainWarp Registered Member

    Joined:
    Aug 26, 2004
    Posts:
    289
    So who will be the first to donate to this study's methodology .:rolleyes:
     
  10. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
  11. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Experts... i don't think so. EliteKiller you arguments crushed them but they are stupid to know it.
     
  12. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,251
    Location:
    The land of no identity :D
    Seems a lot of the MIRT guys don't like NAV or AV-comparatives much...:doubt:
    Also looks like some Throttler guy posted something...
     
  13. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    I hate to dig up older threads but I thought some may find this interesting. Many times here we have discussed packer detection pros and cons, some are for and some are against that is fine it is all in the good of competition.

    Below is another CON. This is just for information and not to start a huge fight as more then just generic packer detecting AVs missed. It is also a single file that by itself is harmless and not much faith can be put in one file, as stated this is just an example.

    Our two favorite packer detectors (still)produced some "nice" results. In reverse this time.

    http://www.zer0-tec.net/downloads/UPX-Comp.pdf

    *Disclaimer* these results are NOT posted to show who detected what but how it was detected, as this file by itself is harmless so the results are worthless in that respect. This file was unpacked for usage reasons and not any malicious intent as this is a file which is known as PUP :) This is just to show another Con of packer detection(s).
     
  14. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    702
    There is practically no AV left by now that does not packer/cryptor based detection. No point in discussing this anymore. :rolleyes:
     
  15. i_g

    i_g Registered Member

    Joined:
    Aug 30, 2006
    Posts:
    133
    The fact that some AV detected the packed file but didn't detect the unpacked one does not mean that it's detecting the packer!
    The signature may simply have been taken from the packed sample, sometimes even for a good reason. (For example, some malware comes with a packed version of a common SysInternals utility, (ab)using it for something. Now, detecting the original (unpacked) utility itself is certainly not a good idea - but detecting the packed variant, which is not its usual form, might be.)
     
  16. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    This was posted in relevance to the thread, and certain vendors whom rely on this method too heavily.

    @Stefan- I understand that, I was simply showing an example in reverse.

    @i_g-
    Did you read the whole thread? I think you would see where I was coming from if you did. IF I didn't make myself clear I apologize, I never stated that they were ONLY detecting the packer in this example. I was simply using it as an example to show that 2 subjects brought up in this thread obviously don't look past UPX.
     
  17. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Yub. And just to add more thoughts: Mail-Gateways. For spammed runtime compressed Malware you USUALLY SHOULD ADD a compressed signature. That speeds up email scanning rapidly! Just imagine what happens if a email gateway has to unpack/emulate 1000's of files per minute instead of just saying it's malware and we don't need to spend more cpu power on that. That decides if a email server goes down during an outbreak or if it stays alive!
     
  18. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    I guess I really didn't make myself clear in this one :)
    I was speaking in respect to home/end user versions not Gateways/mail servers etc.
    Inparticular Fortinet and eScan although I am suprised some others didn't get it as well packed since it is a PUP and these take light in the corporate world.
    Again I apologize for the lack of clarity in my statement(s).
     
  19. proll

    proll Registered Member

    Joined:
    Aug 9, 2005
    Posts:
    56
    I doubt if you are really an AV expert .

    Panda's desktop engine never report any packers.This because the VirusTotal use an Command Line Engine of Panda,and it will report Packer.

    In fact,Some antivirus tool's Desktop products will report Packers ,such as VBA32,Avira,ForClient,and so on.
     
  20. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,102
    Location:
    North Carolina USA
    I agree. Just who does he think he is, calling himself an AV Expert.:rolleyes: :) ;)
    ROTFL
     
  21. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    702
    proll, seems Panda disagrees with you ;-)

    http://research.pandasoftware.com/b.../Mal_2800_ware_2900_formation-statistics.aspx

     
  22. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    98031
    Thanks for the link Stefan, and thanks to Pandalabs for releasing the packer detections for PeiD.
     
  23. proll

    proll Registered Member

    Joined:
    Aug 9, 2005
    Posts:
    56

    Yep,I did never say it's good or bad to report a Packer in black list.

    BUT,Some software's heuristic is fake,the heurstic detection is almost based on the Packer,it can do noting with out the Packer.

    Such as Avxxx, and so on:thumbd:

    That's not amazing,All of "AV expert" should must know this.
     
    Last edited: Jun 28, 2007
  24. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Stefan doesnt need you to tell him cuz he already knows this.
     
  25. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    702
    It's interesting how some people know more about AV heuristics than the people who wrote them. :)

    proll, it doesn't come to your mind that the heuristic was designed to be exactly like it is? Because of various reasons and limitations? That the person who designed it is very well aware of *all* the implications of the choices?
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.