TDS3 - Indenedent Tests

Discussion in 'Trojan Defence Suite' started by aAlan Magee, Mar 17, 2002.

Thread Status:
Not open for further replies.
  1. aAlan Magee

    aAlan Magee Guest

    :D
    Some good results prove, once again, why TDS3 is a gem.

    http://www.staff.uiuc.edu/~ehowes/trojans/tr-tests.htm

    BTW Since these reults were published Trojan Hunter & BoClean have issued an update to correct their failure.
    I wonder how they would fair with a big selection?
     
  2. Vampirefo

    Vampirefo Guest

    I believe this test to be more accurate. http://www.pcflank.com/art17d.htm ;)
     
  3. TonyKlein

    TonyKlein Security Expert

    Joined:
    Feb 9, 2002
    Posts:
    4,347
    Location:
    The Netherlands
    Almost certainly a lot better.

    This test, although independent, is far from being exhaustive or representative, and happened to hit on a '"ASP file wrapper" memory offset problem with a specific obsolete version of ASPack', which was promptly dealt with by an update.

    For a balanced opinion  you needn't go further than this site:  http://www.wilders.org/anti_trojans.htm

    Cheers,
     
  4. wizard

    wizard Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    818
    Location:
    Europe - Germany - Duesseldorf
    Do you? I am working partly for the german review site Rokop-Security. This sited published a lot of independent reviews (also for Trojaner-Info). All the test Rokop published (mostly with new/different testsets) Tauscan scores never beat KAV.

    As PC-Flank does not offer any further information about their testset I can not comment it. But it is rather strange to see that score of Tauscan.

    wizard
     
  5. Vampirefo

    Vampirefo Guest

    LOL, why?  Did you expect to see TDS-3 on top? You don't have to like or agree with results, but to me they are 100% more accurate then the other test posted, to me any test that tests only a small selected number of Trojans is just a waste of time, and shouldn't even be done, again my opinion.

    If you have some test results, post them, and I will glance at them when I get time, and give you my honest opinion of the results.
     
  6. wizard

    wizard Registered Member

    Joined:
    Feb 9, 2002
    Posts:
    818
    Location:
    Europe - Germany - Duesseldorf
    As I wrote before at any tests from Rokop-Security we have not seen in any test that Tauscan has beaten KAV or get the top score.

    Rokop-Security has a testset with a view thousands samples. Roman normaly uses a testset about 100 trojans out of the full test set. Testing many trojans does not say anything about quality of the test. The main problem is that you can take 1000 trojans but how realistic are such tests? For example Bionet Trojan 3.15 was only available a few hours for download. So every trojan user who would liked to use Bionet trojan will only 3.18 for download. Yes you can make a testset with Bionet 3.15, 3.16, 3.17 but the only in a real situation these version would not be used because of bugs. Another big failure most testers do when they say they use a big trojan test set is to use also clients and edit servers to push the number. As both are  not dangerous and would never get on a victims PC it is unrealistic to include them in a testset. But there are many more problems regarding a functional testset. Also the trojans should be tested if they are really working. Some samples never work correct but are still detected by some programs.

    And the most problematic think with testing a huge testset is that most programs 'cheat' with their statistics. If you only compare the statistical results and not each entry in the logfile you will get false results.

    I hope that PC Flank is/was aware of all those problems. :)

    If you want to take a look at some results of Rokop-Security: At Becky's General forum there is link which translates the german Trojaner Info site. :)

    wizard
     
  7. Vampirefo

    Vampirefo Guest

    I agree with you, that venders do indeed cheat,And when I seen the author of a certain Trojan Scanner peddling the results posted by the other poster to discredit, the results, I posted, I quickly disregarded those results as tainted, useless.
     
  8. Vampirefo

    Vampirefo Guest

    I am sure the do, they are an honest site, I test my firewall on their site as well, they just post the facts, like them or not. And if the results were not true, I am sure some Companies would have complained none have, cause there is nothing to complain about.
     
  9. Paul Wilders

    Paul Wilders Administrator

    Joined:
    Jul 1, 2001
    Posts:
    12,472
    Location:
    The Netherlands
    Vampirefo,

    You want a software vendors opinion? Here's one for you, coming from the CEO from Agnitum/Tauscan, dated December 15th 2001 - concerning polymorphic trojans:


    It's safe to conclude:

    a) PCFlank did not use polymorphic trojans when testing;

    b) Tauscan is not able to cope with these - as none "pattern file scanners only" are.

    c) There are alternative anti-trojans who indeed are able to cope with polymorphism. TDS is amongst them.

    Plain to see for everyone, this has nothing to do with honesty - but all with testing circumstances and the willingness from uninformed people to depend on sites as PCFlank to make up their mind in regard to what anti-trojan software to use.

    Glad to be of help  ;)

    regards.

    paul
     
  10. Vampirefo

    Vampirefo Guest

    Sorry I am not following you, are you saying TDS-3 did or did not score 75%, I didn't say nor claim Tauscan detects Polymorpic Trojans, I let the results speak for themselves, TDS-3 on this test scored 3rd place, are you suggesting this is untrue?
     
  11. Paul Wilders

    Paul Wilders Administrator

    Joined:
    Jul 1, 2001
    Posts:
    12,472
    Location:
    The Netherlands
    No problem, Vamp, I'll give it another try  ;).

    You stated:

    I'm referring to the above in my post above. Honesty is not at stake; the way PCFlank performs tests is.

    That's nice to know, but of no relevance here.

    Depends on what one calls "facts". It's plain to see for all, PCFlank at least "forgot" to imply a very serious breed of nasties in their test: polymorphic trojans.

    I just provided you with info coming from the CEO from Agnitum/Tauscan concerning just one company view. If CEO's would bother to react on each and every test on the net, they would have no time left in a day.

    Bottom line: PCFlank test is unreliable, and should be no guideline for decision making - regardless what anti-trojan software we are talking about.

    Glad to help you out  ;)

    regards.

    paul
     
  12. Gavin - DiamondCS

    Gavin - DiamondCS Former DCS Moderator

    Joined:
    Feb 10, 2002
    Posts:
    2,080
    Location:
    Perth, Western Australia
    I have contacted the PCFlank testers some time ago, and again recently. They do admit their test is very much outdated, and have now said there may well be a new test soon, about a month. If they do contact me I will make a post here. If you take a look at the total references/primary trojans count in their screenshot and compare to today's figures for TDS you would see a huge difference.

    Then
    1804 Primaries, 7342 references

    Now
    3566 Primaries, 11774 references

    Also, I would have to say these (Eric Howes) are the best tests if something more other than just SubSeven was used, but it is still a very good indication. Almost all trojan users don't use 20k .COM destructive trojans, nor do they use worms scripts or otherwise, they use uploaders or webdownloaders, and a custom packed full featured trojan. To find custom trojans, you need memory scans among other things
     
Thread Status:
Not open for further replies.