Malwaretestlab Crypter vs Antivirus Test

[maymoons]

-{ Quote: "
Malwaretestlab Crypter vs Antivirus Test
General Info
Crypter tools are dangerous malwares that are known by antivirus and they make it impossible to identify the viruses. These malwares can be formed as brand new and not identified malwares by using these tools.
Theoretically the number of malwares is 10.000.000 all over the world and we assume that we have an antivirus which can define all of the malwares. If this antivirus does not define one Crypter tool, you can use all of the malwares except caught by antivirus. Sure, not every Crypter support every malware file.
In this test, an identified virus is unidentified by 93 Crypter and success of catching OnDemand and OnExecute of antivirus are measured.
The virus used in the test is Trojan. Win32.Filecoder.c ( MD5: BFB2FF61F60BC86B575E5AB4844BF972 ).
There had been a 3 week period for research for Crypter choice. 300+ Crypter tools are gathered. Only 94 of them are tested, the others are eliminated based on the tools’ oldness and well known scale, inefficiency, even though they have different names but produce the same output and last but not least, the tools which ruins the malware software.
" }-

Malwaretestlab.com (http://malwaretestlab.com/)

[Inspector Clouseau]

Testing unpacking abilities / generic detection with one sample by repacking it with different packer/crypter is USELESS and doesn't verify that a antivirus product would have detected a specific sample that's packed with a runtime packer/crypter.

Here's the fact: If a specific AV vendor doesn't detect the specific sample you packed with a runtime packer/crypter ABC then this doesn't mean that the vendor is unable to deal with the packer / crypter! What you completely forgot is that some AV companies using heuristic / generic detections AFTER they unpacked something. Now depending on the packer / crypter they MIGHT be able to trigger that generic detection ( more to this later how this can work ) but they don't reach the point where they have the 1:1 binary for scanning for a specific signature ( what you assume with this testing method )

So that said everyone who added this sample via signature in a range that the unpacker or the emulation doesn't reach (even if they unpack / emulate to some extend successfully!) is screwed.

Why i bring this up is simple... Some packers you can fully emulate and some you can emulate parts of it (where you see what's API-wise going on) and some you don't.

So if you pick a proper sample that has been detected by other heuristics as well ( maybe based on the imports that it uses ) you'd be surprised how many companies would flag that even if they can't emulate the full file and don't find the specific code-signature for your sample.

Example: If something is packed with Packer "ABC" and you emulate to this extend that you see the API's (for example you see that its trying to load winsock functions and urlmon functions) you could flag that at least as suspicious because you know it's strange runtime packed and makes use of such functions and you could prolly check a couple of other things as well. (You have to assume that it does that since your emulation will most likely not run to the end where you can VERIFY it without DCT because your emulation will time out because you have only a specific amount of time that is available for emulation before your kernel mode driver goes into a unstable mode or starts BSOD'ed.

And the sample that was used will most likely not trigger many heuristics even if it's detected by all via signature.

[maymoons]

Hi, we release only one malware samples test.
We made the confirmation tests with other sample malware. We get same result with this result.
This test is not useles.


Many software use their own special technology. They can blocked many viruses or trojan with their proactive technology.

For example, Gdata's behavior blocker can dedect Filecoder.
Classic Crypter tools is not effective for Behavior blocker or Proactive technology.

But some Crypter Tools are effective for behavior blocker too
İt can kill Av, Av cant dedect it.
Some Tools can baypass Kaspersky Proactive protection for example.
Virtual machine not problem for the Crypter Tools


Anyway, This test is first test, if i am not wrong for the crypter category


Every test has some limitation. Real world is a bit different.
But, this test clean, open, repeatable.
Anybody can repeat it. He will see same results.

you can catch some virus with proactive technology. But its limited. this tools can bypass Av's dedection ability. And they are very dangerous. You can watch video, see yourself

[funkydude]

Hello, it's malware, not malwares. Malware is both singular and plural.

If you didn't write it, nevermind. :)

[maymoons]

-{ Quote: "Hello, it's malware, not malwares. Malware is both singular and plural.

If you didn't write it, nevermind. :)" }-

:) thanks. Wilders is a school for me. My english is bad, i am learning :) help me

[trjam]

;) dont worry maymoons. Some of us Englishders have very poor English. You are fine.

[Inspector Clouseau]

-{ Quote: "You can watch video, see yourself" }-

Are you kidding me? Don't get me wrong but please don't assume that i write here because i have nothing better to do.

May i ask how many years you spend fulltime in AV business or Antivirus Research?

[trjam]

-{ Quote: "Are you kidding me? Don't get me wrong but please don't assume that i write here because i have nothing better to do.

May i ask how many years you spend fulltime in AV business or Antivirus Research?" }-
So, IC, I guess a good set of chaps and a vest are next.8)
Also a Scorpian helmet will fit the bill.

[maymoons]

-{ Quote: "May i ask how many years you spend fulltime in AV business or Antivirus Research?" }-

:)
this is classical...
Maybe you spend 50 year?
It is not important for me.
i dont interest.

you say this test useless because blablabla
i with your some idea. i agree but there is a some problem.

i ask myself,
if i get crypted filecoder malware with e-mail
AV software can protect me?

i get my answer, NO.

Crypted Filecoder can encrypt my files. Many times. AV software cant stop it. it is important for me, and many other AV buyer.

maybe i am only curious customer and i want to share my experience?
why you take offence offense?

[kareldjag]

Hi,
I am quite agree with Inspecteur Clouseau first post.
The second is highly contestable: av are designed by virus experts with years of experience, and easily defeated by kids of 12 years old...so sorry but in any Security world, the only unit measure and key word is simple EFFICIENCY...all the rest is litterature.
And an av dev. must be evaluated by the effectiveness of the product he codes, not by the n years of experience he has.
With objectivity and neutrality, Sunbelt vipre is a very good product, sold under the respect of end customer and that tends to be popular in the underground community if we consider anti sandbox evaders tools as the image example...
I ve begin to travel in reversing land only 2 years ago and have not the experience of InspecteurClouseau (on the other hand i plan to pass the GIAC forensic analyst certification, not to have a career in the av industry).
I've build a collection of av evasion tools for research purpose, and a set of 3 samples malwares , each one with about n600 variations (600 packers/crypters/protectors/wrappers/binders/polymorphic engine).
All this for unpacking training(good hobby in the train) and private test only.

A few remarks if it is permitted
-Eicar string test file to test the efficiency of an av is not serious if Manymoons wishes to be considered as serious: simply uses a little zoo of superstars malwares, logically known from any serious av editor.
-All av evasion tools used are not crypters, there is also packers and protectors like Molebox: a cat is not lynx which is not a lion which is not a tiger...
-The tester must be sure that the original malware sample is not already protected/crypted/packed.
It was for instance the case with speculation on a thread in the Privacy software area about Ultrasurf proxy tool.
In an ideal way, it is necessary to be sure that these tools are from original source, in a few words that they re not backdoored as it is often the case in offensive/hack/undergroud boards.
Ideally each stub/obfuscator engine must be unique, not a tool with the stub of crypter omega with the name of alpha
-The result must be statically reliable, and only one malware with n variations is not enough (of course each test is limited by time and number of testers resource, this explains that).
More over, a safe pe file should be incorporated in the test samples list for a more accurate verdict(for my concern for instance i take the example of srip32.exe as safe file).
Ideally a good heuristic engine can detect the signature of the srip32 safe file without classifying it as a malware, and be able to detect the packed malware by its orginal name, not by its packer signature (filecode instead of Pohernah or Troj.Crypt.Gen.
Off course, more the av engine is able to be as close as possible to the original entry point, and more the verdict is accurate.
This will helps to distinguish av with simple packers/crypters signatures from those which integrated more complex engines like VM/emulation and dynamic analysis.
-A test must be as close as possible to what happens in the wild/real life situations.
How many average user run under VMWare ?
VMWare is used in testing only by tradition, not by real technical need if we consider existing alternatives like instant image recovery.
More over anti virtual machine/emulation are used by in the wild malwares, but also by protectors or antipyracy solutions like Themida or VMProtect, and also by heuristic engine...isn't i too much ?
Are cars crash test done in Second Life?....
There is product like RollbackRx and co that can make the job easier and limit risk of errors.
-Euh there is not 10 000 000 malwares...where does this number comes from...in the biographee of General jaruselski?
etc...
But there is also some good points in manymons test

-Static and dynamic ( by running the malware) testing gives an idea of the product potential resistance against this kind (there is other ones of course) of evasion methods.
-efforts done for proof testing with screeshots and videos are highly appreciated
It is not the case of well known organisations, totally obscure and reputation based only.
As far as i know this test is fuly independent.
And this is not the case of well known organisations :since av editors need to pay in order to be tested, these tests can not be considered as independent.
And like the implication of rating agencies like Moodys, Standard and Pool and Fitch in the financial crisis, i consider this kind of testing model (the client pay for geting the right to be tested) as fully incestious.

As i said it somehere else, i am convinced of the emptiness of security soft comparative testing.
1/ it is impossible to prove with no contest that av/firewall/hips a,b, or c is the best or the most effective.
2/ having a high rated or best av/firewall/hips in your line defense does not mean that your system is immune from malware or intrusions
All desktop av/firewall/hips can be technically defeated
If some editor like Prevx tries to fake the competitive editors with corrupted tests and pure lies, i suggest to this kind of editor to launch a defeating challenge with a reward of 5000 Euros. i' ll be glad to participate and give the money to a french charity organization.
At last resort, it has been demonstrated that there is no unbreakable system, from first computer buyers host to US Defense ones.
3/ A test does not tell you which product is the most suitable for you, in relation to your native language, level of experience, budget, habits, kind of pc use etc...
A comparative test only gives a snapshot of result according to a methodology M at instant T.
Nothing else. The result can t be considered as an absolute or Evangelic mantra.
More over, i guess that those who give too much importance to av tests are those who have not circumscribe what is Insecurity by the practice.
If it is was the case, there will be no need to consider Security as a variable of product...
As a conclusion there is no perfect av test, and on the other hand there is no perfect av. product.
Then seriously where is the problem Kamrades....typo mistakes that i have no time to correct...
Rgds

[bellgamin]

-{ Quote: "maybe i am only curious customer and i want to share my experience? why you take offence offense?" }- The problem is that you come across as trying to sound like an expert. But you are not one.

On the other hand, Inspector Clouseau IS an expert -- a world-class professional in the field of anti-malware. Thus, your situation is analogous to that of a high school math student trying to argue quantum physics with Einstein.

I welcome your comments, but suggest that you not be so confrontational. It is possible to disagree without being disagreeable.

Aloha... bellgamin

[Stefan Kurtzhals]

And even better - did you submit the samples and the packers to any of the AV companies? It's easy to raise a stink - how about helping to solve the problem?

[Inspector Clouseau]

-{ Quote: "And even better - did you submit the samples and the packers to any of the AV companies? It's easy to raise a stink - how about helping to solve the problem?" }-

Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it".

The fact that executables can be runtime compressed or encrypted is really nothing new. That exists since earlier MS-DOS times, even earlier than that on other non-IBM systems.

At the end of the day it really counts how much REAL malware a product detects. Real malware means malware that you can find on a users machine.

There is no point in concentrating on "what if we pack that with that and how do we detect it then" when in the same time you think about that for a single minute 360 new samples come in were you DO KNOW that they represent a serious risk because some customers submitted them.

It's also a speed/resource usage versus overall protection. Nobody would use a AV product that takes 7 sec to scan every file. The AV industry has currently to deal with a lot of other stuff than "only" runtime compressed executables. Lots of the recent malware is server-side polymorphic and that stuff IS in massive amounts out at much more user machines than a dedicated packed backdoor for example. So you have to put your priorities there. And not only priorities you have to put the CORRECT priorities there.

[risl]

The problem with protectors, packers and similar tools still exists and some usually well performing avs failed this time. It doesn't suprise though that everyone then attacks the test and the tester. Ofcourse the test isn't perfect and is very narrow but makes a valid point/suggestion.

But.. if I understood correctly about Inspector's first post, failing to detect a virus that has been packed with several different tools is lack of advanced technology?

[Inspector Clouseau]

-{ Quote: "But.. if I understood correctly about Inspector's first post, failing to detect a virus that has been packed with several different tools is lack of advanced technology?" }-

Yes and no. Keep in mind that some AV companies have limited resources and that they have to deal with lots of other stuff. (See my previous post)

So even if you COULD provide a solution for a specific packer (by updating/extending your emulation or whatever) that still doesn't make sense for some of the companies because the amount of time you have spend into this is HUGE. And after you worked for lets say 2 months with a couple of guys on something then the packer will not be used anymore they rather pick a new one or create a custom one. So usually av companies start to support unpacking were they have a serious amount of samples packed with and resources allow to finish this task.

[Kees1958]

Wow heavy gunnery.

Besides the traditional Business to business (B2B) and Business to consumer (B2C), the internet with its (consumer) forums opened the Consumer to consumer (C2C) communication to a level of transparancy never encountered before.

It is the faith of respected companies to deal with C2C: either totally deny/silence it out or deal with it. A funny video posted on you tube can damage your reputation. For a company it is a hard to draw the line when involving with C2C, do I involve with comparatives test, do I involve in hobby forums, what criteria do I maintain, what is my communicated policy.

It is a compliment to IC and Stephan that they took the time to respond.

For independant specialists like Kareldjag there is a new opportunity to handle this. In a different Industry I have adviced a few reputable companies to agree on a "fee per incident" with some independant experts. Those experts respond in forums to seamingly expert statements in hobby / test / comparatives websites. The respectable companies have added this as their public policy statement in regard to these matters and have no influence on the respond of the independant expert (transparacy is the key). They get a copy of the experts reply and the subject responding to. This to provide the company a trigger to post their own response might they disagree with the experts opinion or when they feel the need to provide additional explanation.

Off course the financial reward with the "fee per incident" somehow questions the independance of the third part expert. In this particular industry this is handled by not paying the expert directly, but the (educational) institute they are working for.

Regards Kees

By the way: Comodo earns a compliment: they use "beleivers" to deal with this, this approach was only used in fanatic relegious / political movements before (by the way: I am not saying this is the way the go, but it is a cheap and powerfull policy to deal with C2C attacks).

[Inspector Clouseau]

Well usually i stay out of that because it doesn't make any sense to argue with guys that give a "****" about how detailed you try to explain something based on facts. I usually enjoy discussions but the latest trend in "antivirus and security" is really worrying. People open somewhere a website and try to educate people about computer security. Nothing wrong with that BUT most of the guys just blow you directly in the face "and i don't care about the other guys that have been working in this field for quite some time now". Here's the advise: Give some respect to the guys that have been working for some time in this field and you have a chance end up getting some respect from them. But all this takes some time. You don't gain respect because you think you DESERVE it you get that when OTHERS think you deserve it.

[EraserHW]

-{ Quote: "
If some editor like Prevx tries to fake the competitive editors with corrupted tests and pure lies, i suggest to this kind of editor to launch a defeating challenge with a reward of 5000 Euros. i' ll be glad to participate and give the money to a french charity organization. " }-

I'm off topic, but it's not the first time I read such statements by you. If you don't like Prevx, then don't use it. But if you don't know at all how does it work, I suppose you should first ask to the company about your doubts before writing such sentences.

Thank you

Sorry for the OT

[Inspector Clouseau]

Btw good morning to Italy ;D

[EraserHW]

-{ Quote: "Btw good morning to Italy ;D" }-

To you too ;D

I'm a bit far from the epicentre, ~150km, yet the earthquake has reached my city too :-\

[Cretemonster]

Well here is a way to bring the heurineers out of the basement and into the sunlight, too bad its just another useless test that has no viable results.

You 3 guys really should try getting out more and mingle with us little people or can you even see us from up there? :P

This same discussion will come up again 1000 more times and the heurineers will come out and play again....SSDD.

The dude with the wood for prevx definitly has a fitting picture for an avatar I must say. ;D

[maymoons]

-{ Quote: "Well, the test do has flaws. There are several cryptors/packers in the test which comes up as "not supported" which we do unpack. It always depends on the samples you pack. You selected a very unsuspicious hack tool which will not trigger any heuristic or generic detection anyway. And using just 1 sample is far too less. We get gigabytes of new malware per day!

Not detected if packed does not equate to product cannot unpack the cryptor/packer." }-

Actually i tested 3 different malware but i didnt release result.
Anyway.

i selected very unsuspicious malware espacially.
Because, i dont want to see proactive dedection rate. i want to see anti-crypter performance.

i dont say, AV never cant catch crypted malware. Their proactive technology can be stopped many malware.

For example, Panda has very good proactive dedection rate , Gdata can block Filecoder without malware fingerprints, Rising Hips can block filecoder, Comodo Defense+ too. And agnitum can block ...

But we already know this information.

if AV's proactive protection skipped file, if cant catch it... We know it is not perfect yet.

This is only a test, surely it will have restrictions. All of the other tests also have restrictions.

Our aim is attract attention to this subject and measure the existing performance. You can say what do you want. Internet is full of Crypter tools and you can see them all of the hack forums.

Everbody who wants to have them can get them. And it can knock into a cocked any virus which is known by everbody. You know that it is not a new tool. But now we can see improved samples of them and it is increasing by the day. A lot of turkojen, poison and similar rat tools are used repeatedly after crypt. This test is done with virtual machine. But if the machine is real nothing will be change. We respect to your technology which is improved by you. But this is real that technology isn’t improved only by you. There is a other black side working. This black side is improving its technology. We only make them collide. This is a important subject. Bitdefender and the other AV firms are interested in this subject very much. But i don’t understant your stand out.

"Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it"."

I don’t do this. I sent 250.000+ malware samples to the many AV firms. A lot of them get communication with me and take me a special statute.

And i don’t give them that answer. "Why should i if AV doesn't detect it"

Stephan did right. He wants answer when he doesn’t ascertain and always assesses.

Also Comodo is like it. They want help from curious people like me and now they get million of malware sample. Maybe they have fastly developing database.

Anyway, i respect who are you and your info. But theoretical info doesn’t interest me, i interest in impact. I did and saw.

[andyman35]

-{ Quote: "Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it".

The fact that executables can be runtime compressed or encrypted is really nothing new. That exists since earlier MS-DOS times, even earlier than that on other non-IBM systems.

At the end of the day it really counts how much REAL malware a product detects. Real malware means malware that you can find on a users machine.

There is no point in concentrating on "what if we pack that with that and how do we detect it then" when in the same time you think about that for a single minute 360 new samples come in were you DO KNOW that they represent a serious risk because some customers submitted them.

It's also a speed/resource usage versus overall protection. Nobody would use a AV product that takes 7 sec to scan every file. The AV industry has currently to deal with a lot of other stuff than "only" runtime compressed executables. Lots of the recent malware is server-side polymorphic and that stuff IS in massive amounts out at much more user machines than a dedicated packed backdoor for example. So you have to put your priorities there. And not only priorities you have to put the CORRECT priorities there." }-
Difficult to argue with such logic.Since resources aren't infinite AV companies have to target the widest spread of malware possible and leave conceptual,POC stuff in the background.As always though there's a very simple remedy for this...disk imaging ;)

[Inspector Clouseau]

-{ Quote: "Also Comodo is like it. They want help from curious people like me and now they get million of malware sample. Maybe they have fastly developing database.
" }-

LOL. That's EXACTLY what i mean with concentrating on stuff that hasn't a high priority. The malware what you submit is basically a new variant that you created yourself for such tests. That takes time and resources away that could be used to implement detection for the stuff that is REALLY hitting users.

nvm, that is exactly what i mean: You don't even EVALUATE what i say you go straight to your "i know better" story. So be it then.

No point in trying to explain something to somebody who isn't even interested in listening and valuables other input.

[Miyagi]

It basically boils down to business/personal reputation when such a test is performed. From a consumer point of view, I've seen countless tests that vary and never is a 100% satisfied testing. :wacko:

One needs to be educated to see the real picture. Seeing only one side of a figure is not really a successful marriage. 8)

BTW - It's good to learn from these posts. Thanks for your input!

[maymoons]

-{ Quote: "LOL. That's EXACTLY what i mean with concentrating on stuff that hasn't a high priority. The malware what you submit is basically a new variant that you created yourself for such tests. That takes time and resources away that could be used to implement detection for the stuff that is REALLY hitting users." }-

you havent got any idea about my samples.
you cant know.
how you can say "The malware what you submit is basically a new variant that you created yourself for such tests."
but i dont want to debate any more
you are true, perfect, right, great and more...and vipre too.WoW

[m00nbl00d]

-{ Quote: "The problem is that you come across as trying to sound like an expert. But you are not one.

On the other hand, Inspector Clouseau IS an expert -- a world-class professional in the field of anti-malware. Thus, your situation is analogous to that of a high school math student trying to argue quantum physics with Einstein.

I welcome your comments, but suggest that you not be so confrontational. It is possible to disagree without being disagreeable.

Aloha... bellgamin" }-

I hope I am not mistaken, but, if memory still serves me, not so long ago - perhaps 1 or 2 years ago - a Portuguese science student managed to prove that one of Einstein's theories wasn't right or totally right.

So, anything is possible.

[bellgamin]

-{ Quote: "I hope I am not mistaken, but, if memory still serves me, not so long ago - perhaps 1 or 2 years ago - a Portuguese science student managed to prove that one of Einstein's theories wasn't right or totally right." }-It's an urban legend. Try Ginkgo Biloba for your memory. ;)

[subset]

Hi,

what about the results of the test?
I mean apart from - this is questionable and that is wrong - quarreling. :P

Kaspersky and Bitdefender have really outstanding unpack engines? Yes/No

What about this "business class detection" of packed files?
Is this just a "I am the sheriff's deputy" ambition, like Ikarus or Sophos show.
An AV should detect malicious files and not tag a non threatening keygen as most.evil.malware.ever :argh:

If it comes to PE Packers, most often you can learn more about the AV industry than about malware. :blink:

Cheers

[vijayind]

Just came back from a trip and saw this thread.

The tester here has done both a on-demand test and then also executed his own packed version in realtime.

So AVs with good unpacking skills would/should pick the malware in the on-demand scan itself.
Others will catch the malware on execution, when the sample unpacks itself.

-{ Quote: "Here's the fact: If a specific AV vendor doesn't detect the specific sample you packed with a runtime packer/crypter ABC then this doesn't mean that the vendor is unable to deal with the packer / crypter! What you completely forgot is that some AV companies using heuristic / generic detections AFTER they unpacked something" }-

So whats the argument about ??
Any decent AV should pick the malware using heuristics, sandboxing, Behavioral Blocking, etc. during execution of malware. The really good one would probably detect the malware in both on-demand also. In all the test seems very reasonable.

[Inspector Clouseau]

Ok so you also don't understand what i'm trying to say...

Is it really so difficult to understand that some AV's can emulate some packers to some extend were they CAN GET ENOUGH INFO to trigger a heuristic ( i don't mean here just simple packer detecting !!!! ) but don't reach the 1:1 unpacking binary hence don't detect the signature they created for?

That means if you pick another "origin" sample and not a "virtool" (because usually av's don't have generic rules for that) then quite a lot of av's would have flagged packed samples at least as "suspicious" were they simple detect nothing in the way it was done here.

Once again: You have to know how AV companies add malware. And i give here now a really last fact and if someone doesn't understand that please don't reply here and try to educate other people. As rude as this sounds but this has to be said in this way now.

Some of the malware can be added (for example) as a full-file-crc. That means a over the file crc gets calculated. Now lets think about that:

The AV is able to partly unpack crypter named "WHATEVER". But it doesn't restore all the stuff eg section structure, header's etc. (If you do not know what a section structure or header is please stop reading here and do something else!)

So remember: You added a full file CRC over a raw, unpacked malicious file. AV usually does this when they consider a file "not to be widely used", in most cases its automated done with mass-adding. The really important files getting added in a completely different way!

So what happens now is that the file has a completely different CRC when runtime packed because the binary stream is completely different.

Now, if u unpack the file (EVEN IF YOU MANAGE TO FULLY UNPACK IT!) You won't find the full file CRC! Because a memory mapped file looks completely different than a file on disk. That has to do with different facts: There is alignment issues and some packers simply merge sections together. So even if you unpack that, the over all CRC WILL FAIL. There are a couple of tricks that you can use ( i don't reveal that now ) but still it's almost impossible to ensure a 100% detection with a full file crc and runtime packed stuff.

Now if u take ANOTHER raw sample and the AV has a let's say 64 byte signature for some specific malicious code in their database IT WILL DETECT THIS SAMPLE if it can emulate the packer to at least some content. Because this signature will be found in the unpacked data in contrast to the full file crc that will NOT be found.

That's the PROOF that you cannot write "Oh they don't support this packer" when you don't know how they added the sample you repacked in the first instance!

And the same issue applies for the generic detection! If you pick a sample that you repack and that sample isn't detected via heuristic/generic and doesn't have any valuable heuristic trigger info how do you expect the AV to say later if it's runtime compressed that it's suspicious?

Believe me, AV is not so simple... You really have to understand what's going on before you can argue. Most of the hobby-testers don't understand that.

Serious Testers are aware of THIS issue and that is the reason why nobody really tests that because the results would be wishy-washy.

You would have to sit down for every AV INDIVIDUALLY to test that. A "i just repack some samples and let all scanners test it" DOESN'T WORK IN THIS CASE.

Mike

[EraserHW]

Just to make things a bit easier:

some antivirus engines use full body crc for detection. What does it mean? There are several ways an antivirus can recognize a malicious software. Often, when you know that the malware body won't change (simple not polymorphic trojans, etc...etc...) a full body crc (i.e. MD5) would be enough (not the best way, but this is another topic) to detect that malware.

Yes, there are a lot of packers out there which are able to pack the malware (UPX,PECompact, ASPack, Themida, Armadillo, blablablabla). By doing so, the full body crc totally change and the signature is bypassed.

Now, antivirus engines have unpacking routines so that they are able to handle those packers.

You can think that, after the unpacking step, antivirus engine is able to check the full body crc and detect the file.

The problem is how the unpacking step is handled. Some antivirus engines are able to unpack the packed malware without rebuilding the full original file with its structure. What does it mean? Basically that the antivirus doesn't rebuild totally the original file, but it unpacks the packed file until it gets to a stage that is enough for analyzing the malware.

But, if the original malware was detected by full body crc, the signature won't be anyway recognized, because the unpacked malware has not been totally unpacked until the original form.

The fact that the malware has not been recognized if you've packed it with a specific packer, this doesn't mean that the unpacker used is not handled by the antivirus engine.

This is why doing these kind of tests isn't easy at all.

[risl]

So, basically automatically mass-adding some simple full body hash(crc32/md5/etc.)detections for all previous test sets and virus collections is one way to get good test results and percentages in av-comparatives and similar tests that only do on-demand scanning of a large malware set?

Is it so that good detection rates in these tests are quite easy to achieve without any elegant high tech? ;D

[Fly]

All very technical.

Just a question about the unpacking abilities of the AVs: I've read that kind of information about the different AVs on av-comparatives, but I can't seem to locate it. A nice long horizontal graph that sort of splits out the performance in unpacking and other aspects. Maybe it's the change of the forum ? If anyone can point out where I can locate it, thanks.

[Eice]

-{ Quote: "So, basically automatically mass-adding some simple full body hash(crc32/md5/etc.)detections for all previous test sets and virus collections is one way to get good test results and percentages in av-comparatives and similar tests that only do on-demand scanning of a large malware set?

Is it so that good detection rates in these tests are quite easy to achieve without any elegant high tech? ;D" }-
Essentially yes. But the long-term drawbacks are serious enough that I doubt any vendor with sane people staffing their viruslabs would even consider doing this en masse.

[EraserHW]

-{ Quote: "So, basically automatically mass-adding some simple full body hash(crc32/md5/etc.)detections for all previous test sets and virus collections is one way to get good test results and percentages in av-comparatives and similar tests that only do on-demand scanning of a large malware set?

Is it so that good detection rates in these tests are quite easy to achieve without any elegant high tech? ;D" }-

Not really.

Using full body crc can be useful as fast and effective way to block simple trojans which are detected in the wild. There are a lot of simple trojans that have always the same body.

You have to fight against real threats, not against malwares created ad-hoc for av testing (repacking, rebasing, etc..etc..)

[vijayind]

Thank You, Inspector and EraserHW for the inside information.

So basically the unpacking to memory and AV unpack engine hinder 1:1 detection of fragment/hash of code. And to avoid FPs most AV would probably use multiple vectors to ascertain that a file is malicious. Hence when packers hide some of those vectors, some pragmatic suites will not detect them. That in no way means that they are incapable of dealing with real threats.

[A_Shabanov]

-{ Quote: "
Anyway, This test is first test, if i am not wrong for the crypter category." }-

It isn`t true, we completed the similar test in 2006.
For your information - Testing of antivirus software for packers support (http://www.anti-malware-test.com/?q=taxonomy/term/13)

But we didn`t do this test again, because it included in the proactive test by default.

[maymoons]

-{ Quote: "It isn`t true, we completed the similar test in 2006." }-

so i am wrong :)
thanks for test, actually same result.