Malwaretestlab Crypter vs Antivirus Test

Malwaretestlab.com

 

Testing unpacking abilities / generic detection with one sample by repacking it with different packer/crypter is USELESS and doesn't verify that a antivirus product would have detected a specific sample that's packed with a runtime packer/crypter.

Here's the fact: If a specific AV vendor doesn't detect the specific sample you packed with a runtime packer/crypter ABC then this doesn't mean that the vendor is unable to deal with the packer / crypter! What you completely forgot is that some AV companies using heuristic / generic detections AFTER they unpacked something. Now depending on the packer / crypter they MIGHT be able to trigger that generic detection ( more to this later how this can work ) but they don't reach the point where they have the 1:1 binary for scanning for a specific signature ( what you assume with this testing method )

So that said everyone who added this sample via signature in a range that the unpacker or the emulation doesn't reach (even if they unpack / emulate to some extend successfully!) is screwed.

Why i bring this up is simple... Some packers you can fully emulate and some you can emulate parts of it (where you see what's API-wise going on) and some you don't.

So if you pick a proper sample that has been detected by other heuristics as well ( maybe based on the imports that it uses ) you'd be surprised how many companies would flag that even if they can't emulate the full file and don't find the specific code-signature for your sample.

Example: If something is packed with Packer "ABC" and you emulate to this extend that you see the API's (for example you see that its trying to load winsock functions and urlmon functions) you could flag that at least as suspicious because you know it's strange runtime packed and makes use of such functions and you could prolly check a couple of other things as well. (You have to assume that it does that since your emulation will most likely not run to the end where you can VERIFY it without DCT because your emulation will time out because you have only a specific amount of time that is available for emulation before your kernel mode driver goes into a unstable mode or starts BSOD'ed.

And the sample that was used will most likely not trigger many heuristics even if it's detected by all via signature.

 

Hi, we release only one malware samples test.
We made the confirmation tests with other sample malware. We get same result with this result.
This test is not useles.


Many software use their own special technology. They can blocked many viruses or trojan with their proactive technology.

For example, Gdata's behavior blocker can dedect Filecoder.
Classic Crypter tools is not effective for Behavior blocker or Proactive technology.

But some Crypter Tools are effective for behavior blocker too
İt can kill Av, Av cant dedect it.
Some Tools can baypass Kaspersky Proactive protection for example.
Virtual machine not problem for the Crypter Tools


Anyway, This test is first test, if i am not wrong for the crypter category


Every test has some limitation. Real world is a bit different.
But, this test clean, open, repeatable.
Anybody can repeat it. He will see same results.

you can catch some virus with proactive technology. But its limited. this tools can bypass Av's dedection ability. And they are very dangerous. You can watch video, see yourself

 

Hello, it's malware, not malwares. Malware is both singular and plural.

If you didn't write it, nevermind.

 

thanks. Wilders is a school for me. My english is bad, i am learning help me

 

dont worry guest. Some of us Englishders have very poor English. You are fine.

 

Are you kidding me? Don't get me wrong but please don't assume that i write here because i have nothing better to do.

May i ask how many years you spend fulltime in AV business or Antivirus Research?

 

So, IC, I guess a good set of chaps and a vest are next.
Also a Scorpian helmet will fit the bill.

 

this is classical...
Maybe you spend 50 year?
It is not important for me.
i dont interest.

you say this test useless because blablabla
i with your some idea. i agree but there is a some problem.

i ask myself,
if i get crypted filecoder malware with e-mail
AV software can protect me?

i get my answer, NO.

Crypted Filecoder can encrypt my files. Many times. AV software cant stop it. it is important for me, and many other AV buyer.

maybe i am only curious customer and i want to share my experience?
why you take offence offense?

 

Hi,
I am quite agree with Inspecteur Clouseau first post.
The second is highly contestable: av are designed by virus experts with years of experience, and easily defeated by kids of 12 years old...so sorry but in any Security world, the only unit measure and key word is simple EFFICIENCY...all the rest is litterature.
And an av dev. must be evaluated by the effectiveness of the product he codes, not by the n years of experience he has.
With objectivity and neutrality, Sunbelt vipre is a very good product, sold under the respect of end customer and that tends to be popular in the underground community if we consider anti sandbox evaders tools as the image example...
I ve begin to travel in reversing land only 2 years ago and have not the experience of InspecteurClouseau (on the other hand i plan to pass the GIAC forensic analyst certification, not to have a career in the av industry).
I've build a collection of av evasion tools for research purpose, and a set of 3 samples malwares , each one with about n600 variations (600 packers/crypters/protectors/wrappers/binders/polymorphic engine).
All this for unpacking training(good hobby in the train) and private test only.

A few remarks if it is permitted
-Eicar string test file to test the efficiency of an av is not serious if Manymoons wishes to be considered as serious: simply uses a little zoo of superstars malwares, logically known from any serious av editor.
-All av evasion tools used are not crypters, there is also packers and protectors like Molebox: a cat is not lynx which is not a lion which is not a tiger...
-The tester must be sure that the original malware sample is not already protected/crypted/packed.
It was for instance the case with speculation on a thread in the Privacy software area about Ultrasurf proxy tool.
In an ideal way, it is necessary to be sure that these tools are from original source, in a few words that they re not backdoored as it is often the case in offensive/hack/undergroud boards.
Ideally each stub/obfuscator engine must be unique, not a tool with the stub of crypter omega with the name of alpha
-The result must be statically reliable, and only one malware with n variations is not enough (of course each test is limited by time and number of testers resource, this explains that).
More over, a safe pe file should be incorporated in the test samples list for a more accurate verdict(for my concern for instance i take the example of srip32.exe as safe file).
Ideally a good heuristic engine can detect the signature of the srip32 safe file without classifying it as a malware, and be able to detect the packed malware by its orginal name, not by its packer signature (filecode instead of Pohernah or Troj.Crypt.Gen.
Off course, more the av engine is able to be as close as possible to the original entry point, and more the verdict is accurate.
This will helps to distinguish av with simple packers/crypters signatures from those which integrated more complex engines like VM/emulation and dynamic analysis.
-A test must be as close as possible to what happens in the wild/real life situations.
How many average user run under VMWare ?
VMWare is used in testing only by tradition, not by real technical need if we consider existing alternatives like instant image recovery.
More over anti virtual machine/emulation are used by in the wild malwares, but also by protectors or antipyracy solutions like Themida or VMProtect, and also by heuristic engine...isn't i too much ?
Are cars crash test done in Second Life?....
There is product like RollbackRx and co that can make the job easier and limit risk of errors.
-Euh there is not 10 000 000 malwares...where does this number comes from...in the biographee of General jaruselski?
etc...
But there is also some good points in manymons test

-Static and dynamic ( by running the malware) testing gives an idea of the product potential resistance against this kind (there is other ones of course) of evasion methods.
-efforts done for proof testing with screeshots and videos are highly appreciated
It is not the case of well known organisations, totally obscure and reputation based only.
As far as i know this test is fuly independent.
And this is not the case of well known organisations :since av editors need to pay in order to be tested, these tests can not be considered as independent.
And like the implication of rating agencies like Moodys, Standard and Pool and Fitch in the financial crisis, i consider this kind of testing model (the client pay for geting the right to be tested) as fully incestious.

As i said it somehere else, i am convinced of the emptiness of security soft comparative testing.
1/ it is impossible to prove with no contest that av/firewall/hips a,b, or c is the best or the most effective.
2/ having a high rated or best av/firewall/hips in your line defense does not mean that your system is immune from malware or intrusions
All desktop av/firewall/hips can be technically defeated
If some editor like Prevx tries to fake the competitive editors with corrupted tests and pure lies, i suggest to this kind of editor to launch a defeating challenge with a reward of 5000 Euros. i' ll be glad to participate and give the money to a french charity organization.
At last resort, it has been demonstrated that there is no unbreakable system, from first computer buyers host to US Defense ones.
3/ A test does not tell you which product is the most suitable for you, in relation to your native language, level of experience, budget, habits, kind of pc use etc...
A comparative test only gives a snapshot of result according to a methodology M at instant T.
Nothing else. The result can t be considered as an absolute or Evangelic mantra.
More over, i guess that those who give too much importance to av tests are those who have not circumscribe what is Insecurity by the practice.
If it is was the case, there will be no need to consider Security as a variable of product...
As a conclusion there is no perfect av test, and on the other hand there is no perfect av. product.
Then seriously where is the problem Kamrades....typo mistakes that i have no time to correct...
Rgds

 

The problem is that you come across as trying to sound like an expert. But you are not one.

On the other hand, Inspector Clouseau IS an expert -- a world-class professional in the field of anti-malware. Thus, your situation is analogous to that of a high school math student trying to argue quantum physics with Einstein.

I welcome your comments, but suggest that you not be so confrontational. It is possible to disagree without being disagreeable.

Aloha... bellgamin

 

And even better - did you submit the samples and the packers to any of the AV companies? It's easy to raise a stink - how about helping to solve the problem?

 

Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it".

The fact that executables can be runtime compressed or encrypted is really nothing new. That exists since earlier MS-DOS times, even earlier than that on other non-IBM systems.

At the end of the day it really counts how much REAL malware a product detects. Real malware means malware that you can find on a users machine.

There is no point in concentrating on "what if we pack that with that and how do we detect it then" when in the same time you think about that for a single minute 360 new samples come in were you DO KNOW that they represent a serious risk because some customers submitted them.

It's also a speed/resource usage versus overall protection. Nobody would use a AV product that takes 7 sec to scan every file. The AV industry has currently to deal with a lot of other stuff than "only" runtime compressed executables. Lots of the recent malware is server-side polymorphic and that stuff IS in massive amounts out at much more user machines than a dedicated packed backdoor for example. So you have to put your priorities there. And not only priorities you have to put the CORRECT priorities there.

 

The problem with protectors, packers and similar tools still exists and some usually well performing avs failed this time. It doesn't suprise though that everyone then attacks the test and the tester. Ofcourse the test isn't perfect and is very narrow but makes a valid point/suggestion.

But.. if I understood correctly about Inspector's first post, failing to detect a virus that has been packed with several different tools is lack of advanced technology?

 

Yes and no. Keep in mind that some AV companies have limited resources and that they have to deal with lots of other stuff. (See my previous post)

So even if you COULD provide a solution for a specific packer (by updating/extending your emulation or whatever) that still doesn't make sense for some of the companies because the amount of time you have spend into this is HUGE. And after you worked for lets say 2 months with a couple of guys on something then the packer will not be used anymore they rather pick a new one or create a custom one. So usually av companies start to support unpacking were they have a serious amount of samples packed with and resources allow to finish this task.

 

Wow heavy gunnery.

Besides the traditional Business to business (B2B) and Business to consumer (B2C), the internet with its (consumer) forums opened the Consumer to consumer (C2C) communication to a level of transparancy never encountered before.

It is the faith of respected companies to deal with C2C: either totally deny/silence it out or deal with it. A funny video posted on you tube can damage your reputation. For a company it is a hard to draw the line when involving with C2C, do I involve with comparatives test, do I involve in hobby forums, what criteria do I maintain, what is my communicated policy.

It is a compliment to IC and Stephan that they took the time to respond.

For independant specialists like Kareldjag there is a new opportunity to handle this. In a different Industry I have adviced a few reputable companies to agree on a "fee per incident" with some independant experts. Those experts respond in forums to seamingly expert statements in hobby / test / comparatives websites. The respectable companies have added this as their public policy statement in regard to these matters and have no influence on the respond of the independant expert (transparacy is the key). They get a copy of the experts reply and the subject responding to. This to provide the company a trigger to post their own response might they disagree with the experts opinion or when they feel the need to provide additional explanation.

Off course the financial reward with the "fee per incident" somehow questions the independance of the third part expert. In this particular industry this is handled by not paying the expert directly, but the (educational) institute they are working for.

Regards Kees

By the way: Comodo earns a compliment: they use "beleivers" to deal with this, this approach was only used in fanatic relegious / political movements before (by the way: I am not saying this is the way the go, but it is a cheap and powerfull policy to deal with C2C attacks).

 

Well usually i stay out of that because it doesn't make any sense to argue with guys that give a "****" about how detailed you try to explain something based on facts. I usually enjoy discussions but the latest trend in "antivirus and security" is really worrying. People open somewhere a website and try to educate people about computer security. Nothing wrong with that BUT most of the guys just blow you directly in the face "and i don't care about the other guys that have been working in this field for quite some time now". Here's the advise: Give some respect to the guys that have been working for some time in this field and you have a chance end up getting some respect from them. But all this takes some time. You don't gain respect because you think you DESERVE it you get that when OTHERS think you deserve it.

 

I'm off topic, but it's not the first time I read such statements by you. If you don't like Prevx, then don't use it. But if you don't know at all how does it work, I suppose you should first ask to the company about your doubts before writing such sentences.

Thank you

Sorry for the OT

 

Btw good morning to Italy

 

To you too

I'm a bit far from the epicentre, ~150km, yet the earthquake has reached my city too

 

Well here is a way to bring the heurineers out of the basement and into the sunlight, too bad its just another useless test that has no viable results.

You 3 guys really should try getting out more and mingle with us little people or can you even see us from up there?

This same discussion will come up again 1000 more times and the heurineers will come out and play again....SSDD.

The dude with the wood for prevx definitly has a fitting picture for an avatar I must say.

 

Actually i tested 3 different malware but i didnt release result.
Anyway.

i selected very unsuspicious malware espacially.
Because, i dont want to see proactive dedection rate. i want to see anti-crypter performance.

i dont say, AV never cant catch crypted malware. Their proactive technology can be stopped many malware.

For example, Panda has very good proactive dedection rate , Gdata can block Filecoder without malware fingerprints, Rising Hips can block filecoder, Comodo Defense+ too. And agnitum can block ...

But we already know this information.

if AV's proactive protection skipped file, if cant catch it... We know it is not perfect yet.

This is only a test, surely it will have restrictions. All of the other tests also have restrictions.

Our aim is attract attention to this subject and measure the existing performance. You can say what do you want. Internet is full of Crypter tools and you can see them all of the hack forums.

Everbody who wants to have them can get them. And it can knock into a cocked any virus which is known by everbody. You know that it is not a new tool. But now we can see improved samples of them and it is increasing by the day. A lot of turkojen, poison and similar rat tools are used repeatedly after crypt. This test is done with virtual machine. But if the machine is real nothing will be change. We respect to your technology which is improved by you. But this is real that technology isn’t improved only by you. There is a other black side working. This black side is improving its technology. We only make them collide. This is a important subject. Bitdefender and the other AV firms are interested in this subject very much. But i don’t understant your stand out.

"Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it"."

I don’t do this. I sent 250.000+ malware samples to the many AV firms. A lot of them get communication with me and take me a special statute.

And i don’t give them that answer. "Why should i if AV doesn't detect it"

Stephan did right. He wants answer when he doesn’t ascertain and always assesses.

Also Comodo is like it. They want help from curious people like me and now they get million of malware sample. Maybe they have fastly developing database.

Anyway, i respect who are you and your info. But theoretical info doesn’t interest me, i interest in impact. I did and saw.

 

Difficult to argue with such logic.Since resources aren't infinite AV companies have to target the widest spread of malware possible and leave conceptual,POC stuff in the background.As always though there's a very simple remedy for this...disk imaging

 

LOL. That's EXACTLY what i mean with concentrating on stuff that hasn't a high priority. The malware what you submit is basically a new variant that you created yourself for such tests. That takes time and resources away that could be used to implement detection for the stuff that is REALLY hitting users.

nvm, that is exactly what i mean: You don't even EVALUATE what i say you go straight to your "i know better" story. So be it then.

No point in trying to explain something to somebody who isn't even interested in listening and valuables other input.

 

It basically boils down to business/personal reputation when such a test is performed. From a consumer point of view, I've seen countless tests that vary and never is a 100% satisfied testing.

One needs to be educated to see the real picture. Seeing only one side of a figure is not really a successful marriage.

BTW - It's good to learn from these posts. Thanks for your input!