New anti trojan scanner test

There is a new anti trojan scanner test published today.

http://www.rokop-security.de/Test/Test13/hauptteil_test13.html

It is in german language and here is a short explanation:

Testset 1 contains 120 trojans

Testset 2 contains the same 120 trojans but every trojan that was not already packed with a runtime packer was now packed with one.

Interessting to see how the detection rates would decrease when runtime packers are used. KAV scores because it uses a special engine to unpack the trojan.

Remember the second test regarded only file scanning. Some anti trojan programs offer special techniques to find such packed trojans like mutex scanning or process memory scanning. Such techniques were not tested.

wizard

 

Where's BOCLEAN?

 

BOClean was not tested for two reason. The first one was a technical reason: BOClean does not offer file scanning and this test was about file scanning only.

The second reason is that Privacy Software did not want their product reviewed/tested by Rokop-Security.

wizard

 

I don't know their reasons, but from face value, that is a shame.  Constructive criticism can go a long way to better a product.

 

I don't know their reasons, but from face value, that is a shame.  Constructive criticism can go a long way to better a product.
-----------------------------------------------------------------------

BOCLEAN & tds-3 are the 2 best ,bar none.............test or not!

 

I have had nothing but excellent detection with BOClean...

And their support is incredible - i.e. how ALL companies' support departments should be like.

 

Hi,

Man am I reading the test results right? Are they saying  the best AT is KAV?  I find it hard to believe it would beat TDS-3 at anything related to Trojans.

So is my TDS-3 keeping me as safe as I thought it was?

Thanks
Logan

 

Hi Logan5,

you do not need to be worry about the test result. The trojans TDS-3 missed have been submited to DiamondCS and should already  in the latest signature files.

TDS-3 and KAV are both very good in trojan protection.
On test KAV is better and another test TDS-3 scores.

The pro of KAV is the so called "unpacking engine" which really improves detection which is based on file scanning only. But TDS-3 offers a lot more possibilities to detect known and unknown trojans: process memory scanning, mutex scanning, heuristic,...

KAV can only detect "known" trojans. So TDS-3 is still a very good investment.

wizard

 

Wizard,

Hey thanks

Logan

 

Logan if it helps, I have seen tests in the past on AT scanners where TDS-3 always ranked near the bottom.  I stopped following these tests on AT scanners because they historically aren't very accurate to begin with, not to mention a little dishonest.

Sorry, I can no longer point you to such tests, as I don't have links to them.

What do I run?

TDS-3

 

Re: Another flawed anti trojan scanner test

There are very few anti-trojan comparisons that aren't bias, and the rest are often very flawed. There are too many considerations that simply weren't considered when it comes to anti-trojan software comparison ... here are just a few.

- In the case of the above test, why did they only test 120 trojans, when for example our database has many thousand? No scanner can positively identify all trojans as no anti-trojan/anti-virus company can possibly get a hold of all trojans ever released, so where those 120 specially selected to make one scanner look better against another?

- Why weren't thousands of (rather than just over a hundred) trojans scanned? Only this would give a truer representation of how well the scanners were performing.

- Why is there only one test - file scanning? Assumingly
because this is the only common link between each scanner. However, there are some anti-trojan systems that are resident-only and do not have scan capability, so they couldn't be compared in such a test.

- In many cases, all a trojan has to do to evade detection (on NTFS - WinNT, 2K, XP, etc) is to move itself into an Alternate Data Stream (see http://www.diamondcs.com.au/web/streams/streams.htm). As of last year TDS was the only anti-trojan scanner that was capable of detecting, enumerating and scanning inside all NTFS Alternate Data Streams and providing enhanced filtering and cleanup options, yet you will never see this in a comparison - why? Because virtually every scanner will fail and thus cannot even be represented in such test.

- The same applies with Mutexes - little flags in memory that trojans use to determine whether they are already running -- and if a trojan uses mutexes to determine if it is resident then why shouldn't anti-trojan systems also? Surprisingly, TDS is the only one that can, and still the only scanner of any type that ever has.

- Trojans can be packed using custom compressors that cannot be decompressed by _any_ existing anti-virus or anti-trojan system. This is where memory scanning comes in. Mutex scanning is just one memory test, but TDS also scans memory objects and process memory (where the _decompressed_ trojan is running and ready to be scanned and easily detected).
 
- Out of all of the scanners tested, which ones provide Execution Protection? In other words, which scanners can block trojans BEFORE they execute, and which ones wait until the trojan is running before they can be detected (eg. when it's too late). TDS has Execution Protection, but again you'll never see this in any comparison! On a side-note, we've completed our kernel-level drivers for TDS4/WG4 Execution Protection, making TDS4/WG4 the only anti-worm/anti-trojan systems that can intercept file execution at kernel-level.

On a final note ... you just can't compare cute little sports cars with tanks!

And ....... RADIUS4, our 4th generation detection engine (over 5 years in the making, completely rewritten from the ground up, with every single byte optimised), is coming.

Best regards,
Wayne

 

Wayne - I had to swing back by this thread after checking everything else here to tell you this -

While not currently a user of TDS myself, I want you to know that your statement above was probably the most intelligent, informative and properly-handled response to that type of question that I've ever seen anywhere.

Both the knowledge and restraint you displayed make that post of yours worthy of display in your own 'FAQ's' page and in your advertising for the program - don't change a word, just use it as is. Pete

 

which ones provide Execution Protection?

What other trojan programs has execution protection? Have been considering  TD3 -- Boclean -- Trojan Hunter.

Thanks for the help
Scott

 

[td]What other trojan programs has execution protection? Have been considering  TD3 -- Boclean -- Trojan Hunter.
[/td]

Just TDS3.  Its the one you want.

 

Re: Another flawed anti trojan scanner test

>- In the case of the above test, why did they only test
>120 trojans, when for example our database has
>many thousand? No scanner can positively identify all
>trojans as no anti-trojan/anti-virus company can
>possibly get a hold of all trojans ever released, so
>where those 120 specially selected to make one
>scanner look better against another?

Nope. It's not Rokop's "style". There is a statement why this trojans were chosen:

Bei der Zusammenstellung der Testsets wurde darauf geachtet, daß eine ausgewogene Trojanermischung entstand. Sowohl sehr verbreitete, als auch eher seltene Exemplare sind vorhanden. Auch wahre “Dauerbrenner” und “Geheimtips” wurden berücksichtigt.

Rokop writes that he tried to get a good "mix". He used very wide spreaded trojans and some rare one. He also used some "secret tipps" and "evergreens".

>- Why weren't thousands of (rather than just over a
>hundred) trojans scanned? Only this would give a
>truer representation of how well the scanners were
>performing.

Do you speak about trojans or about backdoors? Most Anti-Trojan Tools are only Anti-Backdoor tools ;o).

>- Why is there only one test - file scanning?
>Assumingly because this is the only common link
>between each scanner. However, there are some anti-
>trojan systems that are resident-only and do not have
>scan capability, so they couldn't be compared in such a
>test.

There is only ONE secure way to prevent a system for trojan infections. This is file scanning and nothing else. With all other detections the trojan was active and one or two seconds might be enough to disable ALL anti virus and anti trojan systems ;o).

>- As of last year TDS was the only anti-trojan scanner
>that was capable of detecting, enumerating and
>scanning inside all NTFS Alternate Data Streams and
>providing enhanced filtering and cleanup options, yet
>you will never see this in a comparison - why? Because
>virtually every scanner will fail and thus cannot even
>be represented in such test.

Nonsense. Rokop tries to be a representable and practical test. Do you know only ONE backdoor that hides himself into an ADS? By the way - ANTS 2.1 released LAST year is also able to scan inside ADS' ;o).

>- Trojans can be packed using custom compressors
>that cannot be decompressed by _any_ existing anti-
>virus or anti-trojan system. This is where memory
>scanning comes in. Mutex scanning is just one memory
>test, but TDS also scans memory objects and process
>memory (where the _decompressed_ trojan is running
>and ready to be scanned and easily detected).

As i said. The only way to prevent an infection is file scanning. What if the trojan hooks openprocess and you aren't able to open the process? Or what if the trojan simply terminate tds?

There will be a way to disable a defense system EVERYTIME. BOClean for example patches the export table of the kernel32.dll to hook the tool help api. All you have to do is to read the original addresses of the exports from the kernel32.dll from DISK and patch the kernel32.dll in memory. Tadaaaaaaaa - boclean will be listed in every processviewer.

Do you want to run a anti trojan tool as a service? What if a trojan simply deactivate it in the registry? Or what if it kills the service from an other system level?

This might all happen if a trojan is able to start. And what is the only thing that can prevent this? Yes, you are right - file scanning ).

>- Out of all of the scanners tested, which ones provide
>Execution Protection? In other words, which scanners
>can block trojans BEFORE they execute, and which
>ones wait until the trojan is running before they can
>be detected (eg. when it's too late). TDS has
>Execution Protection, but again you'll never see this in
>any comparison!

Not the features of the programms were compared only the "power" of the file scan engine.

>On a side-note, we've completed our
>kernel-level drivers for TDS4/WG4 Execution
>Protection, making TDS4/WG4 the only anti-worm/anti-
>trojan systems that can intercept file execution at
>kernel-level.

Hehe - did i ever tell you about ANTS 3.0? *fg* TDS4/WG4 is NOT the only system ). ANTS 3.0 can monitor and block registry, filesystem and network activities in REALTIME. If you want you can get a little "demonstration" ).

>And ....... RADIUS4, our 4th generation detection
>engine (over 5 years in the making, completely
>rewritten from the ground up, with every single byte
>optimised), is coming.

Hehe - the time you needed to code an engine has nothing to do with its quality. Compare KAV and NOD32. Do you think the NOD32 engine is worse then the KAV engine? I don't think so. By the way:

Don't mix up the scan engine and the unpacking engine ;o).

Adieu, Andreas

 

Andreas, please don't forget any of these trojans that terminate processes simply poll the process list for say TDS-3.EXE and terminate if running.

If a file scan reveals nothing.. this could happen with any scanner at all. An unknown/new packer. Nothing can detect and then unpack every new unknown packer - if it could then nothing would ever get past a system with unpack engine - and they DO get by them.

If TDS-3 was terminated, renaming it to HI.EXE, running it, running a process memory/object/mutex scan would reveal the trojan. Even if it was an unknown totally new trojan, and none of the many generic signatures I have for process memory caught it (there are a lot, new trojans are often detectable this way before any analysis goes ahead), no ICQ notify string for example, nothing, it would STILL be caught thanks to some process memory signatures I have added just for this possible scenario. They are very powerful, and I will not elaborate as to exactly how they work, but they are effective against all current process killers.

Kill it. Rename TDS-3.EXE. File scanning will NEVER be a complete solution, saying so is naive.. for the reason above that unknown packers will always be created. They are created just for trojans at times..

Best Regards

 

Hi all. Interesting comments and the thread is informative.
Let me say that I personally think that TDS 3 is the superior AT. I have never used BO Clean but have heard much praise for it, so it must be good.
Having said that, I also strongly believe that most of the AV/AT tests I have seen for the last couple of years shows that AVP/Kav is a most excellent Product. For it to consistantly come in at the top or close to the top says a lot for me. But even more important is how well a product works for people in real life situations. I have used AVP for years and it has served me well. I have put it on other peoples machines and cleaned up all kinds of malware that other AV/AT programs have missed. (Never found anything missed by TDS 3 though.)
I just think that AVP/KAV should get the credit it deserves, so I put my 2 cents in most of these threads. Not to criticize others preferences, not to try to say AVP/KAV is perfect, for it is not.
I use AVP 3.5 right now and TDS 3, both resident. I have no problems with resources. (I use a backup AV scanner for downloads such as NOD32)
Wayne indeed addressed the issue of competent testing very well. Most tests leave a lot to be desired. I am still impressed with the track record of AVP/KAV in both AV and AT tests. It is a unique product, as is TDS 3 and both deserve appreciation in my eyes.
Best regards to all.