I understand that DCS is quite critical about AV/AT tests. I wonder whether this test will be accepted by them. It seems to me that the testers tested everything except the detection rate of the scanners ... " Having completed the literature review we saw no point in repeating tests that had already been carried out by other reviewers. Instead we chose to run tests covering three areas that had not previously been well covered. The first was a signature file currency test. The second, a scanner speed test. The third involved an assessment of the level of technical support offered by each vendor."
Hi ano1, You are probably correct but it seemed a little better technically than most AT reviews I have seen but unfortunately AT reviews appear to be rather limited. At least the writer is honest about the test limitations:
Interesting review, but nothing new - although I think this is the first review of its kind. But TDS is one of only four anti-trojan programs that have been around since the very beginning (along with BOClean, Lockdown, and The Cleaner), and TDS is the only anti-trojan program with daily database updates and a fulltime analyst for detections, so we're not surprised I can elaborate though on our signature counts. The earliest backup we have of the primary TDS database (and this is just primaries, not total references!) is 1018 signatures, back in mid-2000 (before most other anti-trojan programs were even a dream) So here they are ... (month-year): Trojan count 03-1998: 1 (NetBus 1.2, followed shortly by Back Orifice 1.2, Acid Shivers, and Sokets de Troie) 08-2000: 1018 04-2001: 1500 09-2001: 2000 11-2001: 2500 02-2002: 3000 03-2002: 3500 05-2002: 4000 } 06-2002: 4500 } most of todays antitrojans seem to be about here or below 07-2002: 5000 } 09-2002: 5500 10-2002: 6000 12-2002: 6500 02-2003: 7000 03-2003: 7500 05-2003: 8000 06-2003: 8500 08-2003: 9000 09-2003: 9500 10-2003: 10,000 11-2003: 10,500 01-2004: 11,000 You'll notice that in some months (such as 08-09 2003) there were over half a thousand PRIMARY trojans added - equivalant to 16 new trojans every day of that month. How other anti-trojan scanners were able to keep up-to-date without a fulltime analyst during those busy times, I don't know. Maybe they didn't ... So the question is - how many years and how many thousands of trojans behind TDS is your anti-trojan?
I'll have to email the webmaster about the second set of results. Beast 2.x and Assasin 2 are both detected, both are Client only until you actually execute it and generate a trojan server. So if Client scanning was off that would be the reason why they werent detected in those results. I find it hard to believe TDS missed these 2 trojans, which were added immediately after release. How any antitrojan can miss such popular trojans even a few days after release, let alone 3 months later is worrying. This is my only problem with this sites currency review, I would prefer testing 1 day after release, and then a week later. A month is a LONG time to wait
THEN THERE THE BLAZE TEST LOL step one roam all the darkside of the net and whatch tds go off lol step to get beer pizza and get ready to bug the heck out of the tds family with lots of newbie qustions lol step 3 go to um will places where you get free stuff dowenload all of it from a shadey source and and try to run tds into the ground step 4 go alinate a bunch of hackers see if you can manage to stay alive lol not recomended for the weak of heart or your hard drive lol now go looking for free porn and download all those exe files that look like pictures media files and web pages lol now thats a real test
I dont know if this is a legitimate request for tests like this one (considering the nature of trojans), but why not execute the trojans as well (instead of just the one instance for BOClean, which is understandable). And maybe test how well the products remove/clean the trojans in question. I would definitely find it interesting to how each AT respond to a trojan infection... if it is detected as soon as it is executed, or on the next reboot, or maybe if it is only detected when the on demand scanner is activated. And then see how "efficiently" it removes an infection.
Rerun2, how would you execute any trojan with exec protection up? Would be rare, wouldn't it? I mean: for TDS to be able to detect it, a trojan doesn't need to have been running and infecting your system in order to be detected by TDS. Call it a safety prevention. But be sure the nasties have been tried for their activities, so the lab guys of any anti-trojan know.
It probably wouldn't execute, which would be nice to see/know. But how about those trojans that are not picked up TDS' execution protection, but is detected through its memory object scan. And how about polymorphic trojans. Antitrojan "A" might be able to detect the archived version of a polymorphic trojan that was directly downloaded from the developer's site or perhaps a few that were submitted to them, but one would have to generate and execute some random servers to see if it is still able to detect them and how well it is able to handle them. I guess that is my point, that it might be nice to see some kind of distinction between just detecting an archived sample and stopping and cleaning up a "live" sample (though on demand detection usually equates to resident protection as well for the malware in question). I agree. You mention the nasties activities as well, which is a good point. Because maybe it is more difficult to measure a particular AT's success against trojans when their behaviors and payloads are less predictable than that of other types of malware if executed. Im not sure though, I am just throwing out some random thoughts
You might like to read this nice thread about a particular polymorphic trojan detection. We name it the golden thread here at the Wilders forums, as you see various developers working together for our security. Interesting read! http://www.wilderssecurity.com/showthread.php?t=8499