*Update* - Informal Trojan Detection Test # 2

[Eric L. Howes]

Hi All:

The "Informal Trojan Detection Test # 2" web page has been updated to incorporate the results of a round of re-testing for BOClean and TrojanHunter. This re-testing was performed with a series of updates to BOClean and TrojanHunter that were released in response to issues raised in the original round of tests that was posted on 3/15.

You can read the results of this new series of tests here:

http://www.staff.uiuc.edu/~ehowes/trojans/tr-tests-2.htm

If you haven't done so already, please take the time to read the "Disclaimers & Limitations" section at the bottom of the page. I would also urge you you read the "Note On Re-testing" for an explanation of the updates to the page made since its initial release.

I hope you find these tests interesting and useful. As always, comments, questions, and criticisms are welcome.

Eric L. Howes
eburger68@yahoo.com

[Liquid_Fish]

Thanks for sharing this Eric. * Keep it up!

[Marsman]

Eric,

Are your tests interesting & useful? *Yes, most definitely because your tests uncovered some possible vulnerabilities that were up until this point obviously dismissed or overlooked by several industry leaders. *I have always found outside testing, pro/con and user product reviews very interesting and informative but just as important to me is the way in which these companies deal with the not so favorable reviews. *I feel your time & effort has directly contributed to better BOClean & TrojanHunter products going forward. *

Sincere thanks,
Mars Man *;)

[wizard]

No offense against Trojan Hunter or Magnus but the retest of TrojanHunter was a little bit 'unfair' regarding the file scan. If you allowed all other vendors to add the signatures of the packed servers all the results look a little bit nicer. ;)

Also the 'retest' gave the two vendors the chance to modify their memory signatures also. So the test can not show anymore if the memory detection also works with packed trojans. I was suprized to see that BOClean fails. I saw something like that for the test I wrote last year for Rokop-Security on one sample but I found not enough time to investigate more in this issue. Maybe I should start a research on that once again.

Anyway Eric: Very nice test. I have never seen such a clear and detailed and well documentated test for a long time. It leaves no question open. So I am looking forward to your nexts tests. :)

wizard

[puff-m-d]

Just a little something that came to mind....

Even tho the retests were on the unfair side, the quick response from both BoClean and TrojanHunter should be commended. *It shows that they listen to feedback from both tests like this one and users of their product.

Also TrojanHunter is fairly new and Magnus is diligently working on getting a strong signature file made.

Anyway, just a thought......

Kent

[wizard]

Of course Magnus and Kevin/Nancy do a very good job in support. :) Magnus is always interested to get any samples missed by Trojan Hunter in any test he knows about. :) I think just another positive aspect of such test is when program does not get 100% the vendor mostly gets the change to get the missed samples and improve the quality of his product. :)

wizard

[Liquid_Fish]

I kind of agree with wizard here, just because I would like to see how the app performs before its known to be tested. *I also think given the lack of depth of the tests, the retest was important just to test all the detection scenarios.

[Eric L. Howes]

Hi:

Thanks for checking the updated version of the page.

wizard & Liquid_Fish:

You're right. The re-testing was in some ways "unfair." The test had significant limitations on it from the get-go, though, and once it became apparent that my Sub7 2.13 MUIE trojan was not quite was I had presumed it to be, I decided it best to go ahead with the re-testing and then lay all the cards on the table in the "Note On Re-testing" section.

I also decided not to deep-six the original results -- they are still discussed on the page -- but to place those initial results in a larger context.

Finally, I couldn't ignore the several folks who apparently disregarded everything I said in the "Disclaimers and Limitations" (despite my throwing up red flags all over the place) and swiftly concluded that the test demonstrated that X anti-trojan application was worthless and Y anti-virus application was pure gold.

It was an interesting ride, in any case. Hope you found it to be so, too.

Best,

Eric L. Howes

[Liquid_Fish]

{QUOTE-> It was an interesting ride, in any case. Hope you found it to be so, too. <-QUOTE}

Definatly!! *Thanks for sharing your tests with the world.

[octogen]

And thanks for being open to criticism without being defensive. *It is tests like yours that bring various issues to light so that they can be addressed. *We could all benefit, especially when we get positive and quick responses like we did from Kevin/Nancy and Magnus. *Good work!