Unpacking ability of some AV's

Interesting results from the Russian anti-malware site where the unpacking abilities of an AV's RTM and on-demand scanner were measured.

The Nimda A virus was packed by 20 different packers and then the files were scanned by 13 different Antivirus programs.

The packers included;

1. ZIP self-extracting archive (SFX)
2. RAR SFX
3. ASPack 2.12
4. ASProtect 1.23 RC4 build 08.07
5. exe32pack 1.42
6. EXECryptor 2.0
7. ExeStealth 3.04
8. FSG 2.0
9. MEW11 SE 1.2
10. MoleBox 2.3.3
11. Morphine 2.7
12. Packman 0.0.0.1
13. PECompact2 2.55
14. Pe-pack 1.0
15. Petite 2.3
16. UPX y..2shchSh
17. WWPack32 1.20
18. Yoda'.s Crypter 1.3
19. Yoda'.s Protector 1..0b
20. (Win).UPack 0.27 beta

And the AV's tested were;

# Symantec AntiVirus Corporate Edition 10.0.0.359 (SAV) with engine 103.0.2.7
# Trend Micro PC-cillin Internet Security 2005 with engine 7.510.1002
# McAfee VirusScan Professional 2005 (9.0) with engine 4.4.00
# Sophos Anti-Virus 5.0.3 (SAV)
# Kaspersky Anti-Virus Personal Pro 5.0.14 (KAV)
# Eset NOD32 Antivirus System 2.12.3
# CA eTrust EZ Antivirus 6.2.1.1 (CAI) with engine 11.5.0.0
# Norman Virus Control 5.80 with engine5.82.01
# BitDefender 8 Standard with engine 7.01620
# Panda Titanium Antivirus 2005 (4.02.00)
# AVG Anti-Virus 7.0 Professional (7.0.323)
# Dr.Web Scanner for Windows 95-XP v4.32b
# Hauri ViRobot Expert 4.0 with engine 2005-0


The top 5 AV's for unpacking were;

1. Kaspersky - 86%
2. BitDefender - 67%
3. Sophos - 57%
4. Trend Micro And McAfee - 55%
5. Dr.Web - 48%

These results can be added to those over at Scheinsicherheit's and over at MyCity.

Overall, using the results over at these 3 sites, a clear pattern can be seen between very good and very poor unpackers. Kaspersky is still tops.

Moreover, the fast scanning speed of some AV's, NOD, CSAV, eTrust must be due, partly at least, to their relatively poor unpacking ability.

 

Wow! NOD32 is abysmal at unpacking. Why does it have such a good rep?

 

Because it's not all "just" in unpacking capabilities...
Also strange they haven't tested avast!

 

But the NOD results are terrible, that's a fact. Even if it's 'just' about unpacking. Kaspersky seems to be doing very good in all aspects, so Rejzor, just like you I'm testing out their 2006 line

 

Maybe a stupid question, but if you double clicked a packed malware not detected, how can you prevent infections with an av?

Best regards,
Firefighter!

 

The AV has signatures doesn't it ? and what happens when its unpacked ?

Doesn't matter how many packers are supported, if the AV doesn't have a signature for the malware its not going to get detected.

 

Probally memory-scanner?

 

I meant actually that if a sample is detected by a signature, but not detected when it's runtimepacked. How you can prevent the infection after it's double clicked?

Best regards,
Firefighter!

 

I know what you mean my point is how many of these AV's rely on ondemand only, real time protection kicks in and nabs it.

 

This test is from 6 June, 2005, so would be nice to see the results with last versions of the AV's...

If this is true, NOD32 is really very bad in this area...

I also would like to see avast! on this test...

 

Site translated to English:
http://babelfish.altavista.com/babe...alware.ru/index.phtml?part=compare&anid=packs

 

This test seems to be totally biased and must have been carried out by an amateur. If NOD32 detected only 5% then how would you explain that advanced heuristics catches almost every FUNCTIONAL file uploaded to Jotti's when most of the others AV actually miss it?

 

I agree with you, Marcos.
Very strange that NOD32 have a great detection with only 5%...

 

How does the tester know if the Antivirus company just didn't pack most common ITW malware (and Nimda is such malware) and packed it on their own - addings lots of signatures for the differently packed malware?

That is the case for Trend Micro I bet, their unpacking is very minimalistic.
For NOD32, on the other hand, I think the tester forgot to activate the Advanced Heuristic which enables better/generic unpacking.
While working on AntiVir's unpacking, I compared quite a few other virus scanners. The test results look quite the same as here, except for the two examples I mentioned above. McAfee has more unpackers and should have scored higher aswell. Dr.Web also has added lots of unpacking.

 

In fact that is not true -- realtime protection uses the same signatures that manual detection uses; and if the manual scanner cannot detect a runtime-packed malware, then the realtime scanner cannot detect it either.

 

I think that I understood why you just mentioned that FUNCTIONAL. Just today I copied the first 2 x 100 snapshots in Jotti's and the list is in here. But be calm, my journey continues.

FF av-test 2 x 100 from Jotti's 12.-13. Nov. 2005:

Set 1+2 ---- Set 1 ----- Set 2

60.5 % ----- 63 % ----- 58 % -- Vba32
60.0 % ----- 62 % ----- 58 % -- DrWeb 4.33
57.5 % ----- 56 % ----- 59 % -- Kaspersky
46.0 % ----- 50 % ----- 42 % -- BitDefender
44.0 % ----- 45 % ----- 43 % -- NOD32
40.5 % ----- 45 % ----- 36 % -- Fortinet
39.5 % ----- 38 % ----- 41 % -- ArcaVir
36.5 % ----- 40 % ----- 33 % -- AntiVir
23.5 % ----- 27 % ----- 20 % -- AVG
22.5 % ----- 20 % ----- 25 % -- ClamAV
16.0 % ----- 17 % ----- 15 % -- Avast
16.0 % ----- 18 % ----- 14 % -- F-Prot
14.0 % ----- 16 % ----- 12 % -- Norman VC
_9.5 % ----- 10 % ----- _9 % -- UNA


Best regards,
Firefighter!

 

Hmmm... neither F-Prot nor AntiVir were tested. Too bad. Ah well, my *personal proctologist* (the good Doctor Web) did well, and (per Stefan) is even better as of 4.33.

 

Why NOD's supporters automatically claim "bias" when ever a test shows NOD to be weak in an area?

It is quite common knowledge that the unpacking abilities of NOD's realtime scanner is it's weakness. That is one of the reasons it is so light and fast in use.

It realies on catching things once they are upacked, rather that checking them before.

 

Have you checked out the replies to your last post by any chance?

 

Patrician did you ignore this post by an expert who is not employed by Eset as Marcos is and in fact works for the competition? Besides wasn't your Nod licence due in November? Aren't you now free to play with any other Av?

 

Well biased or not, upset NOD32 users or not. The score for NOD32 is way way too low. I don't get it, NOD32 almost for sure unpacks ZIP SFX, UPX, both Yoda's packers/crypters, MEW, FSG and Morphine (as from what i know). Now how they scored just 1 detection? And i wonder if they even tested if all repacked samples are actually working (this is even more important for Sandbox based scanners).
I was betatesting some eMule mod and devs packed it with UPack. Guess what? App executed but terminated itself silently after few seconds (just because of Upack,non packed worked as it should). So my point is that these beta and some even very alpha version crypters and packers could simply corrupt the original file. And result is simply way of because we know NOD32 in version 2.5 has some serious detection capabilities in all areas shown also in AV-Comapratives test...

 

Propably you don't understand which Forum this is?

Best regards,
Firefighter!

 

Woa! Slow down, I wasn't complaining about NOD unpacking, I was just stating that the results of that test shouldn't be a suprise. NOD's unpacking ability is pretty weak compared with others on the market because it works a different way.

This isn't a bad thing as such, it's just a different approach. One I don't like I admit, but if NOD users are happy having the possibilty of having nasties on their HDD's and have faith that NOD's realtime scanner will catch them if activated then that's great for them.

Personally I prefer my AV to stop nasties even getting to my HDD. But that's just my personal preference and not everybody feels the same way.

 

See post 15 for comments from one of the competition.

 

I did not ignore it at all. I couldn't follow or fully understand the point that the poster was making due to bad use of English I'm afraid.

And I fully understand that English isn't his primary language and I am not in anyway critesising his English skill.