The "Real world" anti-trojan tools comparison

[spy1]

( A 're-test' from PCFlank) : http://www.pcflank.com/art26a.htm

*I can almost guarantee no one's going to like the 'results' - OR some of the comments made about TDS!

[count_dracula_]

What's wrong with the results?

Seem OK to me.

[UNICRON]

Well I think the TrojanHunter people are the ones who are going to cry foul on this one. TDS was among the top few and it was pretty close.

PC flank did pick TDS as the AT of choice for advanced users but not n00bs, which is consistant with what DCS promotes, although I wonder at the logic of pcflank doing so it Tauscan beat TDS in their tests? Why would they pick a product with a lower detection rate for an advanced user? They call TDS "cracker-tool" looking lol! That was why I bought it ;)

But once again we will be entertained by the complaints of bias I am sure.

"However, the Tauscan is not able to scan compressed files."

How then can it have a better detection rate than some of the other ATs? I have never tried Tauscan so I can't say much about how good it is, but I did not know it can't scan compressed files. Is this really true? Obviously, compressed files were not part of the test then, which makes me wonder what else wasn't part of the tests? DCS will have some grounds to say that the test doesn't try to detect trojans running in memory ect. Since the files available for download were all zipped, I guess they had to unzip them all before scanning to give tauscan a chance? Can someone confirm or deny that Tauscan can't scan compressed files? I would feel compelled to give them the benefit of the doubt on this one.

All in all, another poor test by non-experts. The rankings are meaningless once again. I am surprised at the results of TrojanHunter more than anything. I'm sure its developer will not be happy and I don't blame him.

[count_dracula_]

I'm new to this security lark, so I'm be speaking out of place here, but why should I believe people here who say that the tests are a joke and that they were not performed by an expert. What makes you an expert? And why are any tests you can do more valid then the ones performed by pcflank?

All these security forums are the same, anything with a gui that looks like it was designed by a blind man with no understanding of interface design gets high marks, anything that does an equal job but has the misfortune to be made with a decent gui gets branded as only suitable for newbies, or for retards who don't know anything.

I use NIS 2000 (I know it's not an AV, but illustrates my point). Everywhere I go I see people saying that it's bloated and slow, I have an off the shelf PC from a high street retailer and not the kind of behemoth monstrosities everyone here seems to have self built, but I haven't had any problems with NIS. There has been no slow down and it doesn't consume any more resources than any other similar product i have tried after having it 'recommended' to me. But whenever anyone mentions they use NIS they get laughed at and told to use so called better products like Look and stop. Jeez, if ever there was a program designed by a blind man that's it.

It's the same with this AT, anytime anyone says an AT with a good GUI like Tauscan does better then TDS with it's crappy GUI (it barely even qualifies to be called one), then the reviews are 'obviously' amateurs or their testing is floored. What a surprise.

[UNICRON]

you make some good points. I will agree that TDS's interface isn't its strong point (more like a glorified dos prompt), and that just because an interface is nice doen't mean the program is crappy.

However, the interface is less important to advanced users than it is to n00bs, so advanced users are less likely to favor a product solely onthat basis. many of these tests are not troajn detection tests, but n00b freindly tests. That is great for n00bs but hardly worth reading for advanced users.

Real detetction is more important to an advanced user, and n00b friendly is more important to a n00b. Logically, if a n00b can't properly operate a product, it stands to reason that that product won't be used to its full potential, and therefore operate as an inferior product. Perhaps this is why tests are done with the default settings, and so many people put importance on default settings. Many people never change them, so it appears that a real world test should be done with these settings. I disagree, but who am I? lol ;)

Basically, simple advice in these matters is do some research, test a few products, and pick one that suites your needs and are comortable with. Don't worry about the snickers from quatsi-experts (like me ;). Also since almost always there are people who have something to gain from the results of these tests, be a bit skeptical of any results. There is no one product that hasn't been rated the best and the worst in different tests so really finding out what one is best can be harder than you think. Even here, opinions of all the quatsi-experts are very divided on the subject of AV, AT, firewalls, proxies, ad-blockers etc. This is probably a good thing.

[Randy_Bell]

No way is Tauscan superior to TDS-3, which is the most sophisticated antitrojan software in the world!! And I'm not a TDS-3 user, I'm a TrojanHunter user!!

Once again, two Russian products have come out on top in PC Flank's antitrojan comparison. Could there be a Russian bias there, considering the location and origins of PC Flank?

And I have trouble taking seriously the dead-last rating that TH received!! Really PC Flank, you can do better than this!! ::)

[Paul Wilders]

This test is not worth mentioning IMHO. Merely Tauscan is unable to cope with polymorpic trojans, as stated by the CEO from Agnitum (by email to us, december 2001), tells it all.

I could comment on a lot more - but there really is no need for that, other than this test makes me laugh out loud.

regards,

paul

[controler]

I do not have even close thew writing and editorial skill displayed here
by the moderators admins ect but any of us can see
count_dracula_
has good points and is a gifted writer also.
Most of those that know me here, know I am by far not new to the computer sceen. I do beta testing for some perty big companies and go deep into the GUI and testing. Since I am a software junki,
I try all kinds of software besides the main beta testing I do and
do quick nit picking. My first impression is how well does the software do and how nocely done is the user interface. What makes the software I am looking at good for the common household user?
TDS is a great program and it really doesn't take that much extra to
kick it up a knotch in detecting more nasties but as the TDS crew admits, TDS is as was made for the advanced user.
Since I have not seen anything on the new TDS-4, I can't comment on that. I am guessing the user interface will be much improved and more suitable to the "newbi"

[Rickster]

Still can’t figure out from all this what specifically was wrong with the test. Didn’t phase my opinion of TDS – in fact, add polymorphics and other advanced detection attributes with the few click process of keeping an eye on my entire system and TDS is orbits far above anything out there. All it indicates is that for that particular test bed, this was the result - nothing beyond that – and not the limits of product capability. I suppose I’m more curious about why it makes you laugh Paul – not in terms of total capability against the sum of all of threats, because that would be laughable – but as it applies to what was specifically tested. As for others, is there evidence the results were altered or tainted? PCFlank’s credibility hinges on that point – so if observers think it’s bogus, lay it out for us. Otherwise it just smacks of defending an inferior purchase decision – and nobody likes to think they did that – even if they did. I’m with Unicron in being totally surprised at the Trojan Hunter result, especially after all the good things I’ve heard about it.

[Paul Wilders]

Rickster,

{QUOTE-> All it indicates is that for that particular test bed, this was the result - nothing beyond that – and not the limits of product capability <-QUOTE}

Exactly my point. Although I'm not familiar with the test bed used, it's plain for all to see it's an "out of the box" test bed. Since nowadays techniques in use by nasties can and and are far more complex, I cannot look upon this test as a reliable one. Limits of capability do play a major role - just for that reason. Laughing comes near to crying here; many will use this test as a guidance...

regards,

paul

[Gavin - DiamondCS]

Hi Everyone,

Don't have time to read all the forum now, after work perhaps.. lots of trojans to work on as always :)

I made a comment on the TDS private forums regarding this, and will duplicate it here :

I have received notification of this test. The results are somewhat hard to go by. They have downloaded a lot of trojans..

However, the download site contains a lot of source, clients, tools, fakers, nukers, spammers and other tools which are not applicable for detection by most of these programs - especially not TDS-3. We are unsure if Client detection was turned on, and whether any of the tested trojans were in the categories not detected by TDS-3.

We do not add source, nor do we add installation programs for a trojan. RAT.NerTe for instance, comes with an installation program which extracts the trojan server, and the client has a full install program. These are not themselves trojans. Install the client and server, and TDS sure will detect both

For this reason, I have replied for more information and offered to give detailed information about each file on the site. "Real world tests" should mean exactly that, you dont get infected by source, a spammer, or a client installation program.

I would like to add they downloaded "256 random trojans"
A few questions in my mind - mostly did they analyse them as I would, studying and executing them to see exactly what they are ? Probably not, or all programs would have scored higher I feel. I know I have 99% of the trojans because I have known of and downloaded from the same place for the last 15 months. One I did miss has arrived recently, GreekHackers Rat, which is now promptly fully analysed and added - it is a Y3K Rat rip, and was already generically detected by TDS in memory.

[count_dracula_]

PC Flank made an error, Tauscan can scan compressed files

http://www.agnitum.com/forum/showthread.php?s=&threadid=4375

[Gavin - DiamondCS]

I would just like to add I was not surprised with the results of other programs, but that after taking into consideration my previous post ::)

I have a test machine, I execute these things and that is what is needed to know what are trojans and what are not :)

Also, some companies do not seem to actively search for trojans like I do.. AV generally just receive submissions as far as I can tell - some smaller companies do not have the time to develop their software AND spend the time I do to find and download and analyse lots of trojans AND handle support AND sales AND website all at once.

Luckily we can, I often take some time at home to hunt for and get more trojans.. what can I say it interests me and I like to make sure TDS has a hell of a lot of trojans to use its scan engines on :)

TDS4 will have many new things, and new users will be happy :)

[Smokey]

{QUOTE-> quoting: Gavin / DiamondCS link=board=25;threadid=2170;start=0#15525 date=1025873503]

TDS4 will have many new things, and new users will be happy :)
<-QUOTE}

What about "old" users? ::)

Ciao,

Smokey

[controler]

Ok one quick thing ha ha different for me huh?

I do believe the INSTALL program should be caught as well.
true it is just sitting there till executed but the file still remains
a threat...
That is like saying we say a nuke go buy on a truck,driven by bad guys,but untill it explodes, we have no problem.

Sorry

[wizard]

{QUOTE-> quoting: Smokey link=board=25;threadid=2170;start=0#15529 date=1025877045]

What about "old" users? ::)

Ciao,

Smokey
<-QUOTE}

I think they'll be happy to as they get TDS 4 for free. :)

wizard

[UNICRON]

{QUOTE-> quoting: Smokey link=board=25;threadid=2170;start=0#15529 date=1025877045]
{QUOTE-> quoting: Gavin / DiamondCS link=board=25;threadid=2170;start=0#15525 date=1025873503]

TDS4 will have many new things, and new users will be happy :)
<-QUOTE}

What about "old" users? ::)

Ciao,

Smokey
<-QUOTE}

free upgrades for current users

[controler]

Users sounds good LOL

I use TDS-3 but am using the trial version yet

Because I am using, I get to update to the Free Trial TDS-4 version
Yehhaaaaaaaa !!!!!!!!!!!!!!!

just teasing again guys..

[Gavin - DiamondCS]

{QUOTE-> quoting: controler link=board=25;threadid=2170;start=0#15541 date=1025883438]
Ok one quick thing ha ha different for me huh?

I do believe the INSTALL program should be caught as well.
true it is just sitting there till executed but the file still remains
a threat...
That is like saying we say a nuke go buy on a truck,driven by bad guys,but untill it explodes, we have no problem.

Sorry
<-QUOTE}

The install program I mention runs visible. It asks if you would like to install NerTe client. Once it has finished, no files are then executed, you are left with NerTe client, and the shortcuts.

This is not dangerous. If a trojan installer is a dropper, if it is in any way dangerous to you, be sure I will add it to detection appropriately !

Newbie TDS users will like the upcoming products we are preparing for them. Registered users will receive a full upgrade for their paid product. Can't say much more sorry :)

[Randy_Bell]

The following is an excerpt from Magnus Mischel's post at TrojanHunter Forum:

http://www.misec.net/cgi-bin/yabb/YaBB.cgi?board=TrojanHunter&action=display&num=1026264092

Should TrojanHunter detect Clients & EditServers?
« on: July 10th, 2002, 2:21am »

--------------------------------------------------------------------------------
Before I begin, I'd like to clarify just what is meant by a trojan client and a trojan "editserver".

Client: Remote access trojans require that the attacker have some way to take control of a compromised computer. In almost all cases, this is done with the use of a trojan client. The client is simply an ordinary program that the attacker uses to connect to the server and do such things as download files, take screenshots etc. Trojan clients, unlike trojan servers, are harmless. Unless you are an evildoer who goes about compromising peoples' computer systems or are a trojan researcher, you won't have a trojan client on your computer.

EditServer: Most of the newer trojans come with an EditServer. This is a program that can be used to configure the trojan server. For example, the server could be configured to send information to an attacker whenever the server starts on a computer. Some trojans only come with the EditServer, and the EditServer is then used to create the actual trojan server from scratch. EditServers are also harmless and you won't have them on your computer unless you are a hacker or trojan researcher.

Now to the question: Should TrojanHunter detect these harmless files? TrojanHunter currently only detects actual threats such as trojan servers. Most other trojan scanners also detect clients and editservers. One reason why this question seems important is the following: If someone decides to evaluate trojan scanners by downloading zip files with trojans in them he will in most cases find the following in a typical zip file: A trojan client, an EditServer and a trojan server. Only the trojan server is an actual threat here, and in some cases it won't even be in the zip file as the creator expects the hacker to create it using the EditServer. The problem, then, is this: If the "trojan files" are scanned, then TrojanHunter will detect only the actual threat - the trojan server. It will not detect the client or EditServer. The result could be interpreted in such a way that TrojanHunter only detects 33% of all trojan files. Of course, anyone who has some more detailed knowledge about trojans and how they work will know that this conclusion is grossly inaccurate.

With these points in mind, I would like everyones opinion on whether or not TrojanHunter should detect files of this nature. Thanks!

[Paul Wilders]

Hi Randy,

{QUOTE-> With these points in mind, I would like everyones opinion on whether or not TrojanHunter should detect files of this nature. <-QUOTE}

A matter of marketing in the end, I suppose. Nowadays, there's a tendency to include all - the PSC/BOClean guys are working on a version not only detecting (activating) servers. Reason? Marketing. Costs? Hugh. Why? to expand the range of potential customers. Since IMHO most potential customers don't know about all this stuff (clients, EditServers, servers) and stumble across tests in which all those are included, the majority might tend to demand an anti-trojan detecting all.

Thus, it's up to the AT vendors to make up their choice: a more "bloathed" software, coming with a far bigger potential user market, or a leaner product, missing a (big?) part of the potential customers out there. Goes for all anti-trojan software - not in particular TH.

regards.

paul

[Randy_Bell]

{QUOTE-> quoting: Forum Admin link=board=25;threadid=2170;start=15#17643 date=1027454029]Thus, it's up to the AT vendors to make up their choice: a more "bloathed" software, coming with a far bigger potential user market, or a leaner product, missing a (big?) part of the potential customers out there. Goes for all anti-trojan software - not in particular TH.
<-QUOTE}

When I expressed my opinion to Magnus' original post at the TH forum, I cast my vote for product purity and single-mindedness -- advocating leaving TH the way it is. Personally I doubt whether expanding the TH ruleset to included non-threats like clients and editservers will improve TH's market position, but I could be wrong. Still others in the TH forum were on the opposing side, recommending that Magnus yield to obvious market realities. There seems to be an even split of opinion, whether Magnus should change TH to include detection of these non-threats. :)

As a loyal TH user, I still feel that TH should remain the way it is now. I've personally tested TH on many trojans myself, and I think the protection I get from TH is much better than the pessimistic conclusion reached by PC Flank and others.

Also, until all this bad publicity happened, Magnus was busy working on version 2.6 of TH, the newest version -- and I fear this will divert him from more important work. :)

[Paul Wilders]

marti,

No doubt anyone is entitled to his own opinion. Just keep in mind, threads in security forums (wether this one or Magnus own forum) is far from the "average Joe" opinion - and that's the aimed market. Those looking for a security app fairly never will visit any of these forums. They decide on other merits: overall tests, online or published in magazines.

{QUOTE-> Also, until all this bad publicity happened, Magnus was busy working on version 2.6 of TH, the newest version -- and I fear this will divert him from more important work <-QUOTE}

Magnus will manage. We have been with him from day one, testing alpha and beta versions from the first version from TH. If he doesn't succeed: this is no kid stuff anymore; just plain business. Marketing is a major issue - regardless if the software is top notch, mediocre or inferior. Perfect marketing will sell a mediocre app in hugh volumes - wrong marketing decisions will kill top notch software. It's a though world out there - regardless what software we're talking about.

That being said: "live and let live" is my personal adagium. There's room for all good software, although competition might be hard. I do wish TH and Magnus all the best - as I do wish all good software vendors the best.

regards.

paul

[wizard]

{QUOTE-> quoting: Randy_Bell link=board=25;threadid=2170;start=15#17630 date=1027446819]With these points in mind, I would like everyones opinion on whether or not TrojanHunter should detect files of this nature. Thanks!
<-QUOTE}

Just a simple answer. If TrojanHunter is tested by AV-Test.org for example it would fail the backdoor test because in this test set clients and edit servers are included.

In Germany nearly every print magazin review refers back to these tests. So what choice does a company really have not to include these files and get a good review?

wizard

wizard

[DrSeltsam]

TrojanHunter uses the filesize to generate the signature - so trojanhunter would fail if a pure backdoor or trojan test will be done, too.

[root]

Randy, I just stumbled across this again and thought I would add my 2 cents.
I am somewhat knowlegable of security and malware and have tried most AVs, ATs, and Firewalls around. I also tend to follow testing at all the testing sites I am aware of. I would like to make one point about that. If I see a particular product consistantly in the upper part of the results across a wide range of testers, then I am going to think there is some merit to believing that that particular product will do a good job of detecting malware. The converse is true if I see a product consistantly coming up with poor test results. This is not the only criteria I use to select a product, but it is one. I have neither the time nor the inclination to check each and every sites testing methods and parameters.
Another point is that if an edit server is part of a trojan and not something I am likely to have on my system unless it came with a trojan, then YES, I would want my AT to pick up on that and allow me to make the decission whether it is malware or not. To me that would just be an added way to try and detect trojans.
I am a TDS user, but have nothing against TH, Tauscan, BO Clean, and a couple of the other ATs. It seems they all can do a pretty good job. For some, TDS is not the right choice as it is a little complicated.
I understand some people tending to be purists.
What I do not understand is practicing purity to the point of excluding added features that may indeed enhance the product, without taking away from its effectiveness to do its primary job.
I am only posting this to say, I would like to see TH score well in the tests because I do think it is a great product for many people. Those many people will not choose it if it lags in too many comparison tests.

[DrSeltsam]

there are a couple of tests on rokop-security.de ;o).

[the Tester]

I am a newbie in comparison to most ppl in this forum.I will say a few things about this topic from a newbie's perspective.When I decided to get an AT program,I looked here and read the rankings,reviews.I also surfed a lot of forums.I went with Trojan Hunter(I do like this program) because I could trial it.I liked what I read about TDS-3,but I lack the savvy and experience to truly use it the way it should be used.Boclean didn't have a trial download that I could find.Ease of use does weigh heavily in the matter for someone like myself.I also like what I see about ANTS 3.0 in this forum.The bottom line for me is this:I seek opinions from the sites like wilders.org because I learn a lot about programs from someone that knows.Tests in my opinion can be very biased.True independent tests are a good tool if you know where they are.