New VirusP AV-test 5-2003!

Discussion in 'other anti-virus software' started by Firefighter, May 19, 2003.

Thread Status:
Not open for further replies.
  1. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Hi again! VirusP has published his 3:rd AV (+some AT:s) -test made 4.-12. May 2003.

    The 50795 malware samples were chosen using VS2000 according to Kaspersky, F-Prot, RAV and McAfee antivirus programs. Each malware sample was unique by malware name, meaning that AT LEAST 1 antivirus programs detected it as a new malware.

    ALL malware samples were unpacked and the only samples that were kept were the ones that were packed using external-dos-packers (that means not winzip, winrar, winace etc).

    The malware samples had the correct file extension using a special program (Renexts) and were unique, according to checksum32 filesize.

    All "fake" malware samples were removed, as well as "garbage" files.


    Here are the results of "the Jury of Greece"!


    Summary detection rate of checked 50 795 malvares:


    1. 99.67% - F-Secure version 5.40

    2. 99.55% - Kaspersky version 4.0.5.37

    3. 97.66% - e-Scan Pro version 2.5.181.5

    4. 97.14% - McAfee version 7.00.5000

    5. 95.18% - RAV version 8.6.104

    6. 92.92% - F-Prot version 3.13

    7. 90.59% - PC-Cillin version 2003 10.01.1039

    8. 90.01% - Norton version 2003 Professional

    9. 89.37% - Sophos version 3.69

    10. 89.23% - Dr. Web version 4.29c

    11. 88.75% - Panda Titanium version 2.04.04

    12. 87.09% - Command version 4.75.0

    13. 86.85% - Avast version 4.0.202

    14. 84.68% - BullGuard version 3.5

    15. 83.26% - BitDefender version 6.4.3

    16. 79.31% - AntiVir version 6.19.09.60

    17. 78.48% - Vexira version 2.06.00.01

    18. 75.32% - Norman version 5.50

    19. 74.29% - Nod version 1.405 -

    20. 73.98% - Solo version 2.5

    21. 73.98% - Fire version 2.7

    22. 66.85% - AVG version 6.0.478

    23. 63.08% - E-Trust version 6.1.4.0

    24. 62.44% - Ikarus version 5.05

    25. 60.16% - VirusBuster version 10.00.88928

    26. 53.65% - Protector Plus version 7.2.E01

    27. 49.80% - ViRobot Expert version 4.0

    28. 45.69% - V3Pro Deluxe version SP2

    29. 41.29% - Gladiator version 3.5.0

    30. 40.42% - VirScan Plus version 12.784

    31. 37.79% - RHVBS version 3.95.487

    32. 33.66% - Quick Heal version 6.09

    33. 28.87% - Digital Patrol version 4.0.65

    34. 24.27% - Wave version 2.0

    35. 12.38% - PestPatrol version 4.2.0.33

    36. 11.77% - TDS version 3.2.0

    37. 10.10% - AntiTrojan Shield version 1.0.0.16

    38. 9.16% - PC DoorGuard version 3.0.0.6

    39. 7.85% - Trojan Remover version 5.0.3

    40. 7.04% - Anti Trojan version 5.5.408

    41. 4.06% - Tauscan version 1.6.0723

    42. 3.77% - The Cleaner version 3.5.3517

    43. 3.56% - Hacker Eliminator (former LockDown Millenium) version 1.2

    44. 1.82% - Trojan Hunter version 3.5.707

    45. 1.37% - IP Armor version 5.40.0112

    VirusBuster II Crashed in the test, so there were no result. :D :D


    "The truth is out there, but it hurts!"

    Best Regards,
    Firefighter!
     
  2. Paul Wilders

    Paul Wilders Administrator

    Joined:
    Jul 1, 2001
    Posts:
    12,472
    Location:
    The Netherlands
    Could you define "malware" ;).

    regards.

    paul
     
  3. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Thanks Firefighter, But ...

    All the AT's did badly so I suspect they were not mainly Trojans in the test i.e. Viruses - Chalk & Cheese come to mind. ;)
     
  4. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To Forum Administrator from Firefighter!

    I chanced the "virus" word from that original site to "malware", because in that term there were such kind of things that were for excample in RAV total database!

    The original site is calling such kind of things as viruses and malwares, what I am calling now as malwares only:

    "The virus samples were divided into these categories, according to the type of the virus :

    File = BeOS, FreeBSD, Linux, Palm, OS2, Unix, BinaryImage, BAS viruses.
    MS-DOS = MS-DOS and HLL*. viruses.
    Windows = Win.*.* viruses.
    Macro = Macro and Formula viruses.
    Malware = DoS, Constructors, Exploit, Flooders, Hoax, Jokes, Nukers, Sniffers, Spoofers, Virus Construction Tools, Virus Tools, Corrupted, Droppers, Intended, PolyEngines.
    Script = BAT, Corel, HTML, Java, Scripts, VBS, WBS, Worms, PHP, Perl viruses.
    Trojans-Backdoors = Trojan and Backdoor viruses".


    "The truth is out there, but it hurts!"

    Best Regards,
    Firefighter!
     
  5. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To Pilli from Firefighter!

    We have to remember, that those 50 795 checked malwares were picked from Kaspersky's, F-Prot's, RAV's and McAfee's database randomly, if I understood the original test report right!

    About that picked sample, there were all kind of bad stuff!

    Total number of trojans and backdoors were 8 943.

    Rank of Anti-Trojans against 8 943 trojans and backdoors.

    1. 73.42% - Digital Patrol 4.0.65

    2. 55.17% - TDS version 3.2.0

    3. 41.55% - AntiTrojan Shield version 1.0.0.16

    4. 37.20% - Trojan Remover version 5.0.3

    5. 36.21% - PC DoorGuard version 3.0.0.6

    6. 32.01% - PestPatrol version 4.2.0.33

    7. 30.75% - Anti Trojan version 5.5.408

    8. 22.89% - Tauscan version 1.6.0723

    9. 19.53% - The Cleaner version 3.5.3517

    10. 15.87% - Hacker Eliminator (former LockDown Millenium) version 1.2

    11. 7.31% - Trojan Hunter version 3.5.707

    12. 6.79% - IP Armor version 5.40.0112

    We have to remember that there will never be an 100% proof result, when the checked trojans and backdoors were picked from 4 different AV-program's database.

    Inversely, we don't know how good detection rate AV-programs have, when all those checked trojans and backdoors are from 4 - 6 AT-program's database! :D


    "The truth is out there, but it hurts!"

    Best Regards,
    Firefighter!
     
  6. Pilli

    Pilli Registered Member

    Joined:
    Feb 13, 2002
    Posts:
    6,217
    Location:
    Hampshire UK
    Thanks FF, It would be interesting to know.
    1. What AT databases were used
    2. Whether the scanners were just set to their default settings.

    I have a feeling that comprehensive testing af AT's is, possibly, more complex than of AV's - That is why it is difficult to find valid reviews ;)

    "The truth maybe out there but is obscured by many layers of deep & smelly stuff" :'(
     
  7. VirusP

    VirusP Registered Member

    Joined:
    Mar 23, 2003
    Posts:
    22
    Location:
    Athens, Greece
    The 50795 malware samples were chosen using VS2000 according to Kaspersky, F-Prot, RAV and McAfee antivirus programs.
    The scanners were set to full scanning capabilities e.g. full heuristics.
     
  8. Paul Wilders

    Paul Wilders Administrator

    Joined:
    Jul 1, 2001
    Posts:
    12,472
    Location:
    The Netherlands
    VirusP,

    Please post results from your upcoming new test - upcoming Saturday, if I'm not mistaken.

    regards.

    paul
     
  9. I_lack_commonsense

    I_lack_commonsense Registered Member

    Joined:
    Jan 22, 2003
    Posts:
    44
    Is it just coincidence that those 4 came out on the top 6?
    Do they have a distinct advantage since the samples were chosen according to those programs, or is that irrelevant?
     
  10. VirusP

    VirusP Registered Member

    Joined:
    Mar 23, 2003
    Posts:
    22
    Location:
    Athens, Greece
    I will make updated tests (meaning i will test the new GAV 4 as well) on Saturday or Sunday. If i forget to pub the results in here, please bother to take a look at my website, ok? ;)
     
  11. VirusP

    VirusP Registered Member

    Joined:
    Mar 23, 2003
    Posts:
    22
    Location:
    Athens, Greece
    I used these 4 programs because they have the biggest -atmo- unique-virus-names list compared to the other ones. The fact that they are among the top programs was taken under consideration, still, this was not the main reason for their use. Although it seems like it, i don't think that using these 4 for the making of the vx database improved their ratios considerably.
     
  12. Paul Wilders

    Paul Wilders Administrator

    Joined:
    Jul 1, 2001
    Posts:
    12,472
    Location:
    The Netherlands
    VirusP,

    I'm sure you'll do this approx. 4,000 community of registered users the honour of posting your results ;).

    regards.

    paul
     
  13. VirusP

    VirusP Registered Member

    Joined:
    Mar 23, 2003
    Posts:
    22
    Location:
    Athens, Greece
    :) :)
     
  14. Jason_DiamondCS

    Jason_DiamondCS Former DCS Moderator

    Joined:
    Nov 11, 2002
    Posts:
    1,046
    Location:
    Perth, Western Australia
    It seems to me if you pick virus's to use from program's database's then the results are going to be in favour of whatever database's you picked them from. Unless you can assume that the other AV programs DO NOT detect virus's other then the ones in those databases you selected, which I don't think is that good a thing to assume.

    To put it another way, if 25% of the samples were taken from NOD's database and NOD still got 75% of the other database's malware samples (one could say this is highly probable,maths wise), it's detection rate would jump by 6% up to 81%. Whats 6% between friends ;) .

    Unless there is something I am missing here isn't what I am saying correct VirusP ?

    *edit* I just chose NOD because it was close to 75% detection rate and easier to do the maths on, I don't favour any particular AV program :)

    -Jason-
     
  15. VirusP

    VirusP Registered Member

    Joined:
    Mar 23, 2003
    Posts:
    22
    Location:
    Athens, Greece
    Maybe you misunderstood me. I explained that i put together the virus lists from four av programs, meaning that even if i had used nod, i still would have to add many samples nod finds as the same and other av as unique. So, i might would have added 250-500 more samples tops, which i don't think is a significant number compared to the 50795 samples used. And to be more accurate, nod's unique virus list is alot smaller than RAV's or KAV's or F-Prot's, so there still would be many more samples i would have to add that nod would find as non-unique.
     
  16. Jonas

    Jonas Registered Member

    Joined:
    Oct 30, 2002
    Posts:
    46
    What is this Digital Patrol out doing TDS? I have never heard of Digital Patrol before these tests results were published. As much time as i spend on security boards i am surprized that this AT has not come up before. Is this AT really anygood or is this test rubbish?

    Peace,
    Jonas
     
  17. root

    root Registered Member

    Joined:
    Feb 19, 2002
    Posts:
    1,723
    Location:
    Missouri, USA
    Hi Jonas. Out of fairness to VirusP, I don't think we should consider his efforts rubbish.
    It does not seem to me that this test is aimed at nor suitable for comparing the efficiency of ATs in detecting trojans.
    It is rather a broad look at how various programs detect a myriad of different types of malware.
    I think this testing can have some information for certain people, such as myself, but I would not use the results of this test alone to determine what security product I will use.
    I too never heard much about Digital Patrol before this, but in an effort to be open minded, I think I personally will check this program out a little more as it does seem to have some redeeming qualities, so to speak.
     
  18. Jonas

    Jonas Registered Member

    Joined:
    Oct 30, 2002
    Posts:
    46
    Root,
    thanks for your reply. i did not mean to come accross as closed minded. Indeed, i have spent about 30 minutes reading about the program. I would be interested to hear your thoughts on Digital Patrol as well as its "redeeming qualities" :p

    peace,
    jonas
     
  19. JimIT

    JimIT Registered Member

    Joined:
    Jan 22, 2003
    Posts:
    1,035
    Location:
    Denton, Texas
    VirusP,

    If you have the time (or want to exert the energy), I'd be interested to know how Trojanshield would do in this test, as I purchased it a while back.

    I'll leave my personal opinion of it out of this post, but it would be nice to have an actual "review" of it, as I cannot find an up to date test on it anywhere.

    Thanks for the consideration. :)
     
  20. root

    root Registered Member

    Joined:
    Feb 19, 2002
    Posts:
    1,723
    Location:
    Missouri, USA
    Jonas, I'll try to look it over and post something.
    I'm in the middle of trying to write something up on Outpost 2.
    By the way, I wasn't insinuating you were closed minded. :D
    I was referring to a problem I have with forming opinions without all the facts. :rolleyes: Emphasis on "I" there.
    I'm outa here for the nite.
    Have a good one.
     
  21. Technodrome

    Technodrome Security Expert

    Joined:
    Feb 13, 2002
    Posts:
    2,140
    Location:
    New York
    Digital Patrol is Rusian AntiVirus "STOP"

    http://www.proantivirus.com/



    Technodrome
     
  22. controler

    controler Guest

    Or for those that can't read Russian

    http://www.antiviraldp.com/
     
  23. Jason_DiamondCS

    Jason_DiamondCS Former DCS Moderator

    Joined:
    Nov 11, 2002
    Posts:
    1,046
    Location:
    Perth, Western Australia
    Hi VirusP,
    isn't your test saying "Here is the malware these 4 programs can detect, how good are you at detecting them". I must admit that this in some ways is a lot better then most AV tests but it is still flawed. Shouldn't a test be made on malware samples not picked from any database, but ranked on their danger to end user. So even if you used 52000 random malware samples they are ranked accordingly in their danger and reflect a "score" based on that. I think that having a large database is always good to pick up old (but working) malware, but what good is it having malware samples which don't work on current systems, or are sub PARTS of malware, etc. Those samples should be rated lower if not removed from the test.

    It's important to RANK the best programs in the security field but if you want to be "the most trustworthy" as according to your site, I suggest you come up with a better way to test. You will find the only way to get a better test is to put a LOT of time into it, to research the malware.

    From your posts on DSLR I can tell you like KAV a lot, do you use KAV's database when collecting malware? How much does KAV detect of your whole sample set? Also on DSLR you stated you disliked all anti-trojan only programs, does your dislike of AT's have anything to do with all AT's getting a poor score in the trojan tests or is it because of the poor score you dislike them? :)

    -Jason-
     
  24. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    To Technodrome from Firefighter!

    I have to add some corrections to your statement about Digital Patrol.

    I have asked some questions from Digital Patrol support, and get some answers at the same day. Here they are!


    You wrote:

    Quest. 1. Does Digital Patrol have any conflicts with such antivirus programs as Kaspersky 4.0, DrWeb 4.29c, RAV AntiVirus or F-secure antivirus?

    Repl. 1. Doesn't.

    Quest. 2. Is there any manual downloadable in the web?

    Repl. 2. Help System available on web (http://www.antiviraldp.com/help/) and DP program (file dpatrol.chm)

    Quest. 3. Is it possible to have two Anti-Trojans at the same time in one PC, I mean for example Trojan Remover and Digital Patrol at the same time?

    Notice: By the way, Trojan Remover hasn't any resident scanner at all, it scans only after you have rebooted your PC!

    Repl. 3. Yes, it is possible.

    --
    AntiviralDP.com
    http://www.antiviraldp.com

    And here is one evidence more about that. http://www.antiviraldp.com/anti-trojan.htm

    After that all, I think Digital Patrol is an Anti-Trojan, not Anti-Virus!

    Besides, we all have seen how good such kind of AV:s as KAV or DrWeb are on the Anti-Virus field, why not then Digital Patrol on the Anti-Trojan field? The russians are very good programers in my mind! :D


    "The truth is out there, but it hurts!"

    Best Regards,
    Firefighter!
     
  25. xor

    xor Guest

    Íå òîëüêî ðóññêèå, ÷òî ðîäèëèñü â Ìîñêâå ìîãóò ïðîãðàììèðîâàòü è àíàëèçèðîâàòü òðîÿíñêèõ êîíåé, òàê æå ýòî ìîãóò ðóññêèå ðîäèâøèåñÿ â Ãåðìàíè. :D :D :D
     
Loading...
Thread Status:
Not open for further replies.