Malwares bypassed Eset?

Discussion in 'other anti-malware software' started by pegas, Jan 9, 2009.

Thread Status:
Not open for further replies.
  1. pegas

    pegas Registered Member

    Joined:
    May 22, 2008
    Posts:
    2,016
    Hi folks,
    I am a long term satisfied user of ESS and I have nothing to complain about yet. However I am curious to get some explanation of information provided on http://www.prevx.com/avgraph/12/Eset.html
    I am not drawing any bad conclusion as regards Eset, I would rather have a clear picture ...
     
  2. nonoise

    nonoise Registered Member

    Joined:
    Jun 6, 2008
    Posts:
    322
    i find it hard to believe microsoft security products did 7 times better than eset. more popular the av soultion, higher the score.
     
  3. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,856
    That doesn't guarantee that they weren't detected.

    These reports seem automated, I myself use nod32 and do my fair sharing of browsing through malware for testing. If I was using Prevx it would think that just because I have a ton of malware on my system for dev testing that ESET isn't catching it?

    That chart proves that malware is on the system not that it wasn't detected/cleaned.
     
  4. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    Hello,
    (This post showed up on my monitoring so I'll respond here to clarify what it means :))

    These graphs are automatically generated every day based on the number of infections reported by our CSI scanner. We look at the Security Center (WMI) data to see what AV the user is using and then that is correlated and turned into this data.

    The infection counts are completely raw data - we have a lot of ESET users, and therefore they will statistically miss more (as there are just more users). For instance, just because Trend doesn't look to miss too many, that doesn't mean that could mean that they are either blocking far more or that they have far fewer users (in this case, it is the latter).

    The graphs do take some reading and interpretation to draw your own conclusions. We received backlash for interpreting the results ourselves and showing percentages so we decided to keep the data as raw as possible.

    Please let me know if you have any further questions :)
     
  5. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,856
    Exactly how does your software confirm that they were missed by the anti-virus?
     
  6. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    If our software detects malware on the system, we consider it missed by their existing AV.

    Before you ask - we account for false positives as well and if we do find a file as a false positive, the scores are retroactively corrected. We have a relatively low false positive rate but feel free to account for a variation of +/- 1% in the scores (which is many orders of magnitude higher than any AV's false positive rate).
     
  7. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,856
    The fact that you assume that malware on a system == the AV missed it just proves my point. I know ESET is more popular in the developer world, and a lot of developers will keep their stack of malware for testing. This can't be used as an AV comparative in any way.

    A little OT: I've searched all over your website for a malware submission email but have failed to find it. I tend to send new missed malware in mass mail to popular AV's (mainly VirusTotal ones). Does a Prevx email address exist?
     
  8. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    Well, "CSI", our scanner, does not look for idle threats in dormant folders - it looks for registered system files, active programs, open/loaded files, rootkits, and other forms of malware entry points so I do not think it would stumble upon a stack of malware just sitting in a folder and, even if it did, we prevent "abuse" in our system by limiting infection logging to ~200 infections per user so no individual user could try and sway results one way or another.

    We're in the process of developing a new submission system, but for now I'll PM you the address of one of our researchers :)
     
  9. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,856
    So 1 infection replicating 200 files/registry entries and such would be individually counted as 200 threats?

    Thanks for the email.
     
  10. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    After detecting a sample, we calculate a 1-to-1 hash on the file and use that in the infection count. Therefore, if there are 200 duplicates of one file across the system, they will be counted as one infection. However, if there are 200 modifications of one infection they would be counted as 200 (but at what point does it stop being a modification and start becoming a variant? :D)

    We also do not factor in detected registry entries in the count of infections anywhere - the reports consist of only file based infections.
     
  11. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,856
    I see, thanks.

    Your results differentiate a little between the graphs and the table. It seems to show on average that 1 infection is missed 1 time, showing that it's more likely "time between infection and database update" than "completely missed variant". Why does the graph list over 5k yet the table shows around 30?
     
  12. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    The table below the bars is a very diluted version of the complete results. We have algorithms which run in the background of the database to find popular files that users might be interested in and then we prioritize those files to show in the vendor charts, if they were missed by that vendor.

    So, the 30 files there are just picked out of the list of thousands - many of which are probably completely randomly named so they wouldn't be useful to users.
     
  13. pbw3

    pbw3 Registered Member

    Joined:
    Nov 12, 2007
    Posts:
    113
    Location:
    UK
    Ignoring the "malware on a machine may not be a threat" issue, would it not be useful to include, alongside the total threats, "average number of threats per PC" as part of the info, to put the data into context..?? If daily was too brief, it might be weekly..

    It could be interesting - although it might then also be more controversial and hence open to abuse.... I wonder if that might also generally show a fairly low average number of threats per PC..!??
     
  14. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    Yes, that would be interesting and I'll forward it onto the web team. I think rather than seeing an average number of threats across ALL users (as that would be diluted by the number of clean users which scan), it might be more useful to keep it in perspective of the number of threats per infected computer.

    Managing the current state of a PC is a difficult task to do, as the user can manually clean up their PC after seeing our scan results or they could just never rescan so the results will never be 100% accurate but IMO it is a very unique set of statistics.
     
  15. pbw3

    pbw3 Registered Member

    Joined:
    Nov 12, 2007
    Posts:
    113
    Location:
    UK
    I must admit that I thought including clean PC's would be the more useful stat overall; but whichever, avoiding abuse could be difficult..:)
     
  16. GreenWhite

    GreenWhite Registered Member

    Joined:
    Nov 23, 2004
    Posts:
    110
    Why are numbers so representative ?

    Think Satyam.
     
  17. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    Numbers are just one component for a user to consider when choosing a product. There are a large number of other aspects to consider and our chart is merely informational and its intention is to show users which products may benefit the most by having our products installed alongside them.

    For instance, if your AV managed to miss 0 threats which we found, there would be no benefit (in terms of detection) to add our products on top.

    However, no AV finds 100% so we highly recommend using multiple products and we're trying to help users understand that to improve their security, they really NEED to use multiple products (and vendors need to allow this by not monopolizing the security of the user's computer).
     
  18. Killtek

    Killtek Registered Member

    Joined:
    Feb 22, 2007
    Posts:
    100
    I always found those charts confusing. :D I guess you can interpret them anyway you want... For today it looks like Panda antivirus had the least detected malware by CSI. This means Panda did a better job detecting overall and needed less help from PrevX CSI detection... confusing isn't it? I guess I'll look into buying Panda then :)
     
  19. pbw3

    pbw3 Registered Member

    Joined:
    Nov 12, 2007
    Posts:
    113
    Location:
    UK
    Actually, I am being a bit slow here.. A really interesting stat would be the percentage of infected computers by AV type (and of course you have full access to this from the scans!). But then I suspect this is becoming more useful to users than as a sales tool..:)
     
  20. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    However, it can also mean that there are less Panda users :) I agree that they can be confusing but they are just data and if we do interpret the data for the users, then it looses a level of granularity and then people blame us for just throwing out random statistics :doubt:
     
  21. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    Yes, I agree that this alternative would be useful however we did have this before and were criticized very heavily because it just looked like we were showing "random" statistics so we decided on the current format which provides completely transparent statistics.

    However, I will let the web team know that there is interest for the old format and see if we can do anything to meld the two :)
     
  22. pbw3

    pbw3 Registered Member

    Joined:
    Nov 12, 2007
    Posts:
    113
    Location:
    UK
    - Number of scans
    - Number of infected PC's
    - Total number of threats

    are all the same, ie raw data, without any "interpretation"..;)

    However, I can certainly see how it might be criticized as controversial..!!
     
  23. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,047
    Location:
    Saudi Arabia/ Pakistan
    Ìnstead of giving the no of infectiosn, it will be better to give a ratio of no. of infection to the no. of users of that security product.

    Otherwise if a software is used by more people, it,s score becomes worse automatically.
     
Loading...
Thread Status:
Not open for further replies.