New VirusP AV-test 5-2003!

Hi again! VirusP has published his 3:rd AV (+some AT:s) -test made 4.-12. May 2003.

The 50795 malware samples were chosen using VS2000 according to Kaspersky, F-Prot, RAV and McAfee antivirus programs. Each malware sample was unique by malware name, meaning that AT LEAST 1 antivirus programs detected it as a new malware.

ALL malware samples were unpacked and the only samples that were kept were the ones that were packed using external-dos-packers (that means not winzip, winrar, winace etc).

The malware samples had the correct file extension using a special program (Renexts) and were unique, according to checksum32 filesize.

All "fake" malware samples were removed, as well as "garbage" files.


Here are the results of "the Jury of Greece"!


Summary detection rate of checked 50 795 malvares:


1. 99.67% - F-Secure version 5.40

2. 99.55% - Kaspersky version 4.0.5.37

3. 97.66% - e-Scan Pro version 2.5.181.5

4. 97.14% - McAfee version 7.00.5000

5. 95.18% - RAV version 8.6.104

6. 92.92% - F-Prot version 3.13

7. 90.59% - PC-Cillin version 2003 10.01.1039

8. 90.01% - Norton version 2003 Professional

9. 89.37% - Sophos version 3.69

10. 89.23% - Dr. Web version 4.29c

11. 88.75% - Panda Titanium version 2.04.04

12. 87.09% - Command version 4.75.0

13. 86.85% - Avast version 4.0.202

14. 84.68% - BullGuard version 3.5

15. 83.26% - BitDefender version 6.4.3

16. 79.31% - AntiVir version 6.19.09.60

17. 78.48% - Vexira version 2.06.00.01

18. 75.32% - Norman version 5.50

19. 74.29% - Nod version 1.405 -

20. 73.98% - Solo version 2.5

21. 73.98% - Fire version 2.7

22. 66.85% - AVG version 6.0.478

23. 63.08% - E-Trust version 6.1.4.0

24. 62.44% - Ikarus version 5.05

25. 60.16% - VirusBuster version 10.00.88928

26. 53.65% - Protector Plus version 7.2.E01

27. 49.80% - ViRobot Expert version 4.0

28. 45.69% - V3Pro Deluxe version SP2

29. 41.29% - Gladiator version 3.5.0

30. 40.42% - VirScan Plus version 12.784

31. 37.79% - RHVBS version 3.95.487

32. 33.66% - Quick Heal version 6.09

33. 28.87% - Digital Patrol version 4.0.65

34. 24.27% - Wave version 2.0

35. 12.38% - PestPatrol version 4.2.0.33

36. 11.77% - TDS version 3.2.0

37. 10.10% - AntiTrojan Shield version 1.0.0.16

38. 9.16% - PC DoorGuard version 3.0.0.6

39. 7.85% - Trojan Remover version 5.0.3

40. 7.04% - Anti Trojan version 5.5.408

41. 4.06% - Tauscan version 1.6.0723

42. 3.77% - The Cleaner version 3.5.3517

43. 3.56% - Hacker Eliminator (former LockDown Millenium) version 1.2

44. 1.82% - Trojan Hunter version 3.5.707

45. 1.37% - IP Armor version 5.40.0112

VirusBuster II Crashed in the test, so there were no result.


"The truth is out there, but it hurts!"

Best Regards,
Firefighter!

 

Thanks Firefighter, But ...

All the AT's did badly so I suspect they were not mainly Trojans in the test i.e. Viruses - Chalk & Cheese come to mind.

 

To Forum Administrator from Firefighter!

I chanced the "virus" word from that original site to "malware", because in that term there were such kind of things that were for excample in RAV total database!

The original site is calling such kind of things as viruses and malwares, what I am calling now as malwares only:

"The virus samples were divided into these categories, according to the type of the virus :

File = BeOS, FreeBSD, Linux, Palm, OS2, Unix, BinaryImage, BAS viruses.
MS-DOS = MS-DOS and HLL*. viruses.
Windows = Win.*.* viruses.
Macro = Macro and Formula viruses.
Malware = DoS, Constructors, Exploit, Flooders, Hoax, Jokes, Nukers, Sniffers, Spoofers, Virus Construction Tools, Virus Tools, Corrupted, Droppers, Intended, PolyEngines.
Script = BAT, Corel, HTML, Java, Scripts, VBS, WBS, Worms, PHP, Perl viruses.
Trojans-Backdoors = Trojan and Backdoor viruses".


"The truth is out there, but it hurts!"

Best Regards,
Firefighter!

 

To Pilli from Firefighter!

We have to remember, that those 50 795 checked malwares were picked from Kaspersky's, F-Prot's, RAV's and McAfee's database randomly, if I understood the original test report right!

About that picked sample, there were all kind of bad stuff!

Total number of trojans and backdoors were 8 943.

Rank of Anti-Trojans against 8 943 trojans and backdoors.

1. 73.42% - Digital Patrol 4.0.65

2. 55.17% - TDS version 3.2.0

3. 41.55% - AntiTrojan Shield version 1.0.0.16

4. 37.20% - Trojan Remover version 5.0.3

5. 36.21% - PC DoorGuard version 3.0.0.6

6. 32.01% - PestPatrol version 4.2.0.33

7. 30.75% - Anti Trojan version 5.5.408

8. 22.89% - Tauscan version 1.6.0723

9. 19.53% - The Cleaner version 3.5.3517

10. 15.87% - Hacker Eliminator (former LockDown Millenium) version 1.2

11. 7.31% - Trojan Hunter version 3.5.707

12. 6.79% - IP Armor version 5.40.0112

We have to remember that there will never be an 100% proof result, when the checked trojans and backdoors were picked from 4 different AV-program's database.

Inversely, we don't know how good detection rate AV-programs have, when all those checked trojans and backdoors are from 4 - 6 AT-program's database!


"The truth is out there, but it hurts!"

Best Regards,
Firefighter!

 

Thanks FF, It would be interesting to know.
1. What AT databases were used
2. Whether the scanners were just set to their default settings.

I have a feeling that comprehensive testing af AT's is, possibly, more complex than of AV's - That is why it is difficult to find valid reviews

"The truth maybe out there but is obscured by many layers of deep & smelly stuff"

 

The 50795 malware samples were chosen using VS2000 according to Kaspersky, F-Prot, RAV and McAfee antivirus programs.
The scanners were set to full scanning capabilities e.g. full heuristics.

 

Is it just coincidence that those 4 came out on the top 6?
Do they have a distinct advantage since the samples were chosen according to those programs, or is that irrelevant?

 

I will make updated tests (meaning i will test the new GAV 4 as well) on Saturday or Sunday. If i forget to pub the results in here, please bother to take a look at my website, ok?

 

I used these 4 programs because they have the biggest -atmo- unique-virus-names list compared to the other ones. The fact that they are among the top programs was taken under consideration, still, this was not the main reason for their use. Although it seems like it, i don't think that using these 4 for the making of the vx database improved their ratios considerably.

 

It seems to me if you pick virus's to use from program's database's then the results are going to be in favour of whatever database's you picked them from. Unless you can assume that the other AV programs DO NOT detect virus's other then the ones in those databases you selected, which I don't think is that good a thing to assume.

To put it another way, if 25% of the samples were taken from NOD's database and NOD still got 75% of the other database's malware samples (one could say this is highly probable,maths wise), it's detection rate would jump by 6% up to 81%. Whats 6% between friends .

Unless there is something I am missing here isn't what I am saying correct VirusP ?

*edit* I just chose NOD because it was close to 75% detection rate and easier to do the maths on, I don't favour any particular AV program

-Jason-

 

Maybe you misunderstood me. I explained that i put together the virus lists from four av programs, meaning that even if i had used nod, i still would have to add many samples nod finds as the same and other av as unique. So, i might would have added 250-500 more samples tops, which i don't think is a significant number compared to the 50795 samples used. And to be more accurate, nod's unique virus list is alot smaller than RAV's or KAV's or F-Prot's, so there still would be many more samples i would have to add that nod would find as non-unique.

 

What is this Digital Patrol out doing TDS? I have never heard of Digital Patrol before these tests results were published. As much time as i spend on security boards i am surprized that this AT has not come up before. Is this AT really anygood or is this test rubbish?

Peace,
Jonas

 

Hi Jonas. Out of fairness to VirusP, I don't think we should consider his efforts rubbish.
It does not seem to me that this test is aimed at nor suitable for comparing the efficiency of ATs in detecting trojans.
It is rather a broad look at how various programs detect a myriad of different types of malware.
I think this testing can have some information for certain people, such as myself, but I would not use the results of this test alone to determine what security product I will use.
I too never heard much about Digital Patrol before this, but in an effort to be open minded, I think I personally will check this program out a little more as it does seem to have some redeeming qualities, so to speak.

 

Root,
thanks for your reply. i did not mean to come accross as closed minded. Indeed, i have spent about 30 minutes reading about the program. I would be interested to hear your thoughts on Digital Patrol as well as its "redeeming qualities"

peace,
jonas

 

VirusP,

If you have the time (or want to exert the energy), I'd be interested to know how Trojanshield would do in this test, as I purchased it a while back.

I'll leave my personal opinion of it out of this post, but it would be nice to have an actual "review" of it, as I cannot find an up to date test on it anywhere.

Thanks for the consideration.

 

Jonas, I'll try to look it over and post something.
I'm in the middle of trying to write something up on Outpost 2.
By the way, I wasn't insinuating you were closed minded.
I was referring to a problem I have with forming opinions without all the facts. Emphasis on "I" there.
I'm outa here for the nite.
Have a good one.

 

Digital Patrol is Rusian AntiVirus "STOP"

http://www.proantivirus.com/



Technodrome

 

Or for those that can't read Russian

http://www.antiviraldp.com/

 

Hi VirusP,
isn't your test saying "Here is the malware these 4 programs can detect, how good are you at detecting them". I must admit that this in some ways is a lot better then most AV tests but it is still flawed. Shouldn't a test be made on malware samples not picked from any database, but ranked on their danger to end user. So even if you used 52000 random malware samples they are ranked accordingly in their danger and reflect a "score" based on that. I think that having a large database is always good to pick up old (but working) malware, but what good is it having malware samples which don't work on current systems, or are sub PARTS of malware, etc. Those samples should be rated lower if not removed from the test.

It's important to RANK the best programs in the security field but if you want to be "the most trustworthy" as according to your site, I suggest you come up with a better way to test. You will find the only way to get a better test is to put a LOT of time into it, to research the malware.

From your posts on DSLR I can tell you like KAV a lot, do you use KAV's database when collecting malware? How much does KAV detect of your whole sample set? Also on DSLR you stated you disliked all anti-trojan only programs, does your dislike of AT's have anything to do with all AT's getting a poor score in the trojan tests or is it because of the poor score you dislike them?

-Jason-

 

To Technodrome from Firefighter!

I have to add some corrections to your statement about Digital Patrol.

I have asked some questions from Digital Patrol support, and get some answers at the same day. Here they are!


You wrote:

Quest. 1. Does Digital Patrol have any conflicts with such antivirus programs as Kaspersky 4.0, DrWeb 4.29c, RAV AntiVirus or F-secure antivirus?

Repl. 1. Doesn't.

Quest. 2. Is there any manual downloadable in the web?

Repl. 2. Help System available on web (http://www.antiviraldp.com/help/) and DP program (file dpatrol.chm)

Quest. 3. Is it possible to have two Anti-Trojans at the same time in one PC, I mean for example Trojan Remover and Digital Patrol at the same time?

Notice: By the way, Trojan Remover hasn't any resident scanner at all, it scans only after you have rebooted your PC!

Repl. 3. Yes, it is possible.

--
AntiviralDP.com
http://www.antiviraldp.com

And here is one evidence more about that. http://www.antiviraldp.com/anti-trojan.htm

After that all, I think Digital Patrol is an Anti-Trojan, not Anti-Virus!

Besides, we all have seen how good such kind of AV:s as KAV or DrWeb are on the Anti-Virus field, why not then Digital Patrol on the Anti-Trojan field? The russians are very good programers in my mind!


"The truth is out there, but it hurts!"

Best Regards,
Firefighter!

 

Íå òîëüêî ðóññêèå, ÷òî ðîäèëèñü â Ìîñêâå ìîãóò ïðîãðàììèðîâàòü è àíàëèçèðîâàòü òðîÿíñêèõ êîíåé, òàê æå ýòî ìîãóò ðóññêèå ðîäèâøèåñÿ â Ãåðìàíè.