TDS3 - Indenedent Tests

[aAlan Magee]

;D
Some good results prove, once again, why TDS3 is a gem.

http://www.staff.uiuc.edu/~ehowes/trojans/tr-tests.htm

BTW Since these reults were published Trojan Hunter & BoClean have issued an update to correct their failure.
I wonder how they would fair with a big selection?

[Vampirefo]

I believe this test to be more accurate. http://www.pcflank.com/art17d.htm ;)

[TonyKlein]

-{ Quote: "I wonder how they would fair with a big selection?" }-

Almost certainly a lot better.

This test, although independent, is far from being exhaustive or representative, and happened to hit on a '"ASP file wrapper" memory offset problem with a specific obsolete version of ASPack', which was promptly dealt with by an update.

For a balanced opinion *you needn't go further than this site: *http://www.wilders.org/anti_trojans.htm

Cheers,

[wizard]

-{ Quote: "I believe this test to be more accurate. http://www.pcflank.com/art17d.htm ;)" }-

Do you? I am working partly for the german review site Rokop-Security. This sited published a lot of independent reviews (also for Trojaner-Info). All the test Rokop published (mostly with new/different testsets) Tauscan scores never beat KAV.

As PC-Flank does not offer any further information about their testset I can not comment it. But it is rather strange to see that score of Tauscan.

wizard

[Vampirefo]

-{ Quote: "
But it is rather strange to see that score of Tauscan.

wizard" }-

LOL, why? *Did you expect to see TDS-3 on top? You don't have to like or agree with results, but to me they are 100% more accurate then the other test posted, to me any test that tests only a small selected number of Trojans is just a waste of time, and shouldn't even be done, again my opinion.

If you have some test results, post them, and I will glance at them when I get time, and give you my honest opinion of the results.

[wizard]

As I wrote before at any tests from Rokop-Security we have not seen in any test that Tauscan has beaten KAV or get the top score.

Rokop-Security has a testset with a view thousands samples. Roman normaly uses a testset about 100 trojans out of the full test set. Testing many trojans does not say anything about quality of the test. The main problem is that you can take 1000 trojans but how realistic are such tests? For example Bionet Trojan 3.15 was only available a few hours for download. So every trojan user who would liked to use Bionet trojan will only 3.18 for download. Yes you can make a testset with Bionet 3.15, 3.16, 3.17 but the only in a real situation these version would not be used because of bugs. Another big failure most testers do when they say they use a big trojan test set is to use also clients and edit servers to push the number. As both are *not dangerous and would never get on a victims PC it is unrealistic to include them in a testset. But there are many more problems regarding a functional testset. Also the trojans should be tested if they are really working. Some samples never work correct but are still detected by some programs.

And the most problematic think with testing a huge testset is that most programs 'cheat' with their statistics. If you only compare the statistical results and not each entry in the logfile you will get false results.

I hope that PC Flank is/was aware of all those problems. :)

If you want to take a look at some results of Rokop-Security: At Becky's General forum there is link which translates the german Trojaner Info site. :)

wizard

[Vampirefo]

I agree with you, that venders do indeed cheat,And when I seen the author of a certain Trojan Scanner peddling the results posted by the other poster to discredit, the results, I posted, I quickly disregarded those results as tainted, useless.

[Vampirefo]

-{ Quote: "As I hope that PC Flank is/was aware of all those problems. :)
wizard" }-

I am sure the do, they are an honest site, I test my firewall on their site as well, they just post the facts, like them or not. And if the results were not true, I am sure some Companies would have complained none have, cause there is nothing to complain about.

[Paul Wilders]

Vampirefo,

You want a software vendors opinion? Here's one for you, coming from the CEO from Agnitum/Tauscan, dated December 15th 2001 - concerning polymorphic trojans:


-{ Quote: "Unfortunately Tauscan is not able to detect Polymorpic trojans. Its engine needs to be re-written from the scratch

Best regards
Mikhail Zakhryapin
Director, Agnitum Ltd.
http://www.agnitum.com" }-

It's safe to conclude:

a) PCFlank did not use polymorphic trojans when testing;

b) Tauscan is not able to cope with these - as none "pattern file scanners only" are.

c) There are alternative anti-trojans who indeed are able to cope with polymorphism. TDS is amongst them.

Plain to see for everyone, this has nothing to do with honesty - but all with testing circumstances and the willingness from uninformed people to depend on sites as PCFlank to make up their mind in regard to what anti-trojan software to use.

Glad to be of help *;)

regards.

paul

[Vampirefo]

Sorry I am not following you, are you saying TDS-3 did or did not score 75%, I didn't say nor claim Tauscan detects Polymorpic Trojans, I let the results speak for themselves, TDS-3 on this test scored 3rd place, are you suggesting this is untrue?

[Paul Wilders]

No problem, Vamp, I'll give it another try *;).

You stated:

-{ Quote: "I am sure the do, they are an honest site.." }-

I'm referring to the above in my post above. Honesty is not at stake; the way PCFlank performs tests is.

-{ Quote: "I test my firewall on their site as well.." }-

That's nice to know, but of no relevance here.

-{ Quote: "they just post the facts, like them or not." }-

Depends on what one calls "facts". It's plain to see for all, PCFlank at least "forgot" to imply a very serious breed of nasties in their test: polymorphic trojans.

-{ Quote: "And if the results were not true, I am sure some Companies would have complained none have, cause there is nothing to complain about." }-

I just provided you with info coming from the CEO from Agnitum/Tauscan concerning just one company view. If CEO's would bother to react on each and every test on the net, they would have no time left in a day.

Bottom line: PCFlank test is unreliable, and should be no guideline for decision making - regardless what anti-trojan software we are talking about.

Glad to help you out *;)

regards.

paul

[Gavin - DiamondCS]

I have contacted the PCFlank testers some time ago, and again recently. They do admit their test is very much outdated, and have now said there may well be a new test soon, about a month. If they do contact me I will make a post here. If you take a look at the total references/primary trojans count in their screenshot and compare to today's figures for TDS you would see a huge difference.

Then
1804 Primaries, 7342 references

Now
3566 Primaries, 11774 references

Also, I would have to say these (Eric Howes) are the best tests if something more other than just SubSeven was used, but it is still a very good indication. Almost all trojan users don't use 20k .COM destructive trojans, nor do they use worms scripts or otherwise, they use uploaders or webdownloaders, and a custom packed full featured trojan. To find custom trojans, you need memory scans among other things