Malwaretestlab Crypter vs Antivirus Test

Discussion in 'other anti-virus software' started by guest, Apr 6, 2009.

Thread Status:
Not open for further replies.
  1. guest

    guest Guest

    Malwaretestlab.com
     
  2. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Testing unpacking abilities / generic detection with one sample by repacking it with different packer/crypter is USELESS and doesn't verify that a antivirus product would have detected a specific sample that's packed with a runtime packer/crypter.

    Here's the fact: If a specific AV vendor doesn't detect the specific sample you packed with a runtime packer/crypter ABC then this doesn't mean that the vendor is unable to deal with the packer / crypter! What you completely forgot is that some AV companies using heuristic / generic detections AFTER they unpacked something. Now depending on the packer / crypter they MIGHT be able to trigger that generic detection ( more to this later how this can work ) but they don't reach the point where they have the 1:1 binary for scanning for a specific signature ( what you assume with this testing method )

    So that said everyone who added this sample via signature in a range that the unpacker or the emulation doesn't reach (even if they unpack / emulate to some extend successfully!) is screwed.

    Why i bring this up is simple... Some packers you can fully emulate and some you can emulate parts of it (where you see what's API-wise going on) and some you don't.

    So if you pick a proper sample that has been detected by other heuristics as well ( maybe based on the imports that it uses ) you'd be surprised how many companies would flag that even if they can't emulate the full file and don't find the specific code-signature for your sample.

    Example: If something is packed with Packer "ABC" and you emulate to this extend that you see the API's (for example you see that its trying to load winsock functions and urlmon functions) you could flag that at least as suspicious because you know it's strange runtime packed and makes use of such functions and you could prolly check a couple of other things as well. (You have to assume that it does that since your emulation will most likely not run to the end where you can VERIFY it without DCT because your emulation will time out because you have only a specific amount of time that is available for emulation before your kernel mode driver goes into a unstable mode or starts BSOD'ed.

    And the sample that was used will most likely not trigger many heuristics even if it's detected by all via signature.
     
  3. guest

    guest Guest

    Hi, we release only one malware samples test.
    We made the confirmation tests with other sample malware. We get same result with this result.
    This test is not useles.


    Many software use their own special technology. They can blocked many viruses or trojan with their proactive technology.

    For example, Gdata's behavior blocker can dedect Filecoder.
    Classic Crypter tools is not effective for Behavior blocker or Proactive technology.

    But some Crypter Tools are effective for behavior blocker too
    İt can kill Av, Av cant dedect it.
    Some Tools can baypass Kaspersky Proactive protection for example.
    Virtual machine not problem for the Crypter Tools


    Anyway, This test is first test, if i am not wrong for the crypter category


    Every test has some limitation. Real world is a bit different.
    But, this test clean, open, repeatable.
    Anybody can repeat it. He will see same results.

    you can catch some virus with proactive technology. But its limited. this tools can bypass Av's dedection ability. And they are very dangerous. You can watch video, see yourself
     
    Last edited by a moderator: Apr 6, 2009
  4. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,852
    Hello, it's malware, not malwares. Malware is both singular and plural.

    If you didn't write it, nevermind. :)
     
  5. guest

    guest Guest

    :) thanks. Wilders is a school for me. My english is bad, i am learning :) help me
     
  6. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    ;) dont worry guest. Some of us Englishders have very poor English. You are fine.
     
  7. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Are you kidding me? Don't get me wrong but please don't assume that i write here because i have nothing better to do.

    May i ask how many years you spend fulltime in AV business or Antivirus Research?
     
  8. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    So, IC, I guess a good set of chaps and a vest are next.:cool:
    Also a Scorpian helmet will fit the bill.
     
  9. guest

    guest Guest

    :)
    this is classical...
    Maybe you spend 50 year?
    It is not important for me.
    i dont interest.

    you say this test useless because blablabla
    i with your some idea. i agree but there is a some problem.

    i ask myself,
    if i get crypted filecoder malware with e-mail
    AV software can protect me?

    i get my answer, NO.

    Crypted Filecoder can encrypt my files. Many times. AV software cant stop it. it is important for me, and many other AV buyer.

    maybe i am only curious customer and i want to share my experience?
    why you take offence offense?
     
  10. kareldjag

    kareldjag Registered Member

    Joined:
    Nov 13, 2004
    Posts:
    622
    Location:
    PARIS AND ITS SUBURBS
    Hi,
    I am quite agree with Inspecteur Clouseau first post.
    The second is highly contestable: av are designed by virus experts with years of experience, and easily defeated by kids of 12 years old...so sorry but in any Security world, the only unit measure and key word is simple EFFICIENCY...all the rest is litterature.
    And an av dev. must be evaluated by the effectiveness of the product he codes, not by the n years of experience he has.
    With objectivity and neutrality, Sunbelt vipre is a very good product, sold under the respect of end customer and that tends to be popular in the underground community if we consider anti sandbox evaders tools as the image example...
    I ve begin to travel in reversing land only 2 years ago and have not the experience of InspecteurClouseau (on the other hand i plan to pass the GIAC forensic analyst certification, not to have a career in the av industry).
    I've build a collection of av evasion tools for research purpose, and a set of 3 samples malwares , each one with about n600 variations (600 packers/crypters/protectors/wrappers/binders/polymorphic engine).
    All this for unpacking training(good hobby in the train) and private test only.

    A few remarks if it is permitted
    -Eicar string test file to test the efficiency of an av is not serious if Manymoons wishes to be considered as serious: simply uses a little zoo of superstars malwares, logically known from any serious av editor.
    -All av evasion tools used are not crypters, there is also packers and protectors like Molebox: a cat is not lynx which is not a lion which is not a tiger...
    -The tester must be sure that the original malware sample is not already protected/crypted/packed.
    It was for instance the case with speculation on a thread in the Privacy software area about Ultrasurf proxy tool.
    In an ideal way, it is necessary to be sure that these tools are from original source, in a few words that they re not backdoored as it is often the case in offensive/hack/undergroud boards.
    Ideally each stub/obfuscator engine must be unique, not a tool with the stub of crypter omega with the name of alpha
    -The result must be statically reliable, and only one malware with n variations is not enough (of course each test is limited by time and number of testers resource, this explains that).
    More over, a safe pe file should be incorporated in the test samples list for a more accurate verdict(for my concern for instance i take the example of srip32.exe as safe file).
    Ideally a good heuristic engine can detect the signature of the srip32 safe file without classifying it as a malware, and be able to detect the packed malware by its orginal name, not by its packer signature (filecode instead of Pohernah or Troj.Crypt.Gen.
    Off course, more the av engine is able to be as close as possible to the original entry point, and more the verdict is accurate.
    This will helps to distinguish av with simple packers/crypters signatures from those which integrated more complex engines like VM/emulation and dynamic analysis.
    -A test must be as close as possible to what happens in the wild/real life situations.
    How many average user run under VMWare ?
    VMWare is used in testing only by tradition, not by real technical need if we consider existing alternatives like instant image recovery.
    More over anti virtual machine/emulation are used by in the wild malwares, but also by protectors or antipyracy solutions like Themida or VMProtect, and also by heuristic engine...isn't i too much ?
    Are cars crash test done in Second Life?....
    There is product like RollbackRx and co that can make the job easier and limit risk of errors.
    -Euh there is not 10 000 000 malwares...where does this number comes from...in the biographee of General jaruselski?
    etc...
    But there is also some good points in manymons test

    -Static and dynamic ( by running the malware) testing gives an idea of the product potential resistance against this kind (there is other ones of course) of evasion methods.
    -efforts done for proof testing with screeshots and videos are highly appreciated
    It is not the case of well known organisations, totally obscure and reputation based only.
    As far as i know this test is fuly independent.
    And this is not the case of well known organisations :since av editors need to pay in order to be tested, these tests can not be considered as independent.
    And like the implication of rating agencies like Moodys, Standard and Pool and Fitch in the financial crisis, i consider this kind of testing model (the client pay for geting the right to be tested) as fully incestious.

    As i said it somehere else, i am convinced of the emptiness of security soft comparative testing.
    1/ it is impossible to prove with no contest that av/firewall/hips a,b, or c is the best or the most effective.
    2/ having a high rated or best av/firewall/hips in your line defense does not mean that your system is immune from malware or intrusions
    All desktop av/firewall/hips can be technically defeated
    If some editor like Prevx tries to fake the competitive editors with corrupted tests and pure lies, i suggest to this kind of editor to launch a defeating challenge with a reward of 5000 Euros. i' ll be glad to participate and give the money to a french charity organization.
    At last resort, it has been demonstrated that there is no unbreakable system, from first computer buyers host to US Defense ones.
    3/ A test does not tell you which product is the most suitable for you, in relation to your native language, level of experience, budget, habits, kind of pc use etc...
    A comparative test only gives a snapshot of result according to a methodology M at instant T.
    Nothing else. The result can t be considered as an absolute or Evangelic mantra.
    More over, i guess that those who give too much importance to av tests are those who have not circumscribe what is Insecurity by the practice.
    If it is was the case, there will be no need to consider Security as a variable of product...
    As a conclusion there is no perfect av test, and on the other hand there is no perfect av. product.
    Then seriously where is the problem Kamrades....typo mistakes that i have no time to correct...
    Rgds
     

    Attached Files:

    Last edited: Apr 6, 2009
  11. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,648
    Location:
    Hawaii
    The problem is that you come across as trying to sound like an expert. But you are not one.

    On the other hand, Inspector Clouseau IS an expert -- a world-class professional in the field of anti-malware. Thus, your situation is analogous to that of a high school math student trying to argue quantum physics with Einstein.

    I welcome your comments, but suggest that you not be so confrontational. It is possible to disagree without being disagreeable.

    Aloha... bellgamin
     
  12. Stefan Kurtzhals

    Stefan Kurtzhals AV Expert

    Joined:
    Sep 30, 2003
    Posts:
    701
    And even better - did you submit the samples and the packers to any of the AV companies? It's easy to raise a stink - how about helping to solve the problem?
     
  13. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it".

    The fact that executables can be runtime compressed or encrypted is really nothing new. That exists since earlier MS-DOS times, even earlier than that on other non-IBM systems.

    At the end of the day it really counts how much REAL malware a product detects. Real malware means malware that you can find on a users machine.

    There is no point in concentrating on "what if we pack that with that and how do we detect it then" when in the same time you think about that for a single minute 360 new samples come in were you DO KNOW that they represent a serious risk because some customers submitted them.

    It's also a speed/resource usage versus overall protection. Nobody would use a AV product that takes 7 sec to scan every file. The AV industry has currently to deal with a lot of other stuff than "only" runtime compressed executables. Lots of the recent malware is server-side polymorphic and that stuff IS in massive amounts out at much more user machines than a dedicated packed backdoor for example. So you have to put your priorities there. And not only priorities you have to put the CORRECT priorities there.
     
  14. risl

    risl Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    581
    The problem with protectors, packers and similar tools still exists and some usually well performing avs failed this time. It doesn't suprise though that everyone then attacks the test and the tester. Ofcourse the test isn't perfect and is very narrow but makes a valid point/suggestion.

    But.. if I understood correctly about Inspector's first post, failing to detect a virus that has been packed with several different tools is lack of advanced technology?
     
  15. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Yes and no. Keep in mind that some AV companies have limited resources and that they have to deal with lots of other stuff. (See my previous post)

    So even if you COULD provide a solution for a specific packer (by updating/extending your emulation or whatever) that still doesn't make sense for some of the companies because the amount of time you have spend into this is HUGE. And after you worked for lets say 2 months with a couple of guys on something then the packer will not be used anymore they rather pick a new one or create a custom one. So usually av companies start to support unpacking were they have a serious amount of samples packed with and resources allow to finish this task.
     
  16. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Wow heavy gunnery.

    Besides the traditional Business to business (B2B) and Business to consumer (B2C), the internet with its (consumer) forums opened the Consumer to consumer (C2C) communication to a level of transparancy never encountered before.

    It is the faith of respected companies to deal with C2C: either totally deny/silence it out or deal with it. A funny video posted on you tube can damage your reputation. For a company it is a hard to draw the line when involving with C2C, do I involve with comparatives test, do I involve in hobby forums, what criteria do I maintain, what is my communicated policy.

    It is a compliment to IC and Stephan that they took the time to respond.

    For independant specialists like Kareldjag there is a new opportunity to handle this. In a different Industry I have adviced a few reputable companies to agree on a "fee per incident" with some independant experts. Those experts respond in forums to seamingly expert statements in hobby / test / comparatives websites. The respectable companies have added this as their public policy statement in regard to these matters and have no influence on the respond of the independant expert (transparacy is the key). They get a copy of the experts reply and the subject responding to. This to provide the company a trigger to post their own response might they disagree with the experts opinion or when they feel the need to provide additional explanation.

    Off course the financial reward with the "fee per incident" somehow questions the independance of the third part expert. In this particular industry this is handled by not paying the expert directly, but the (educational) institute they are working for.

    Regards Kees

    By the way: Comodo earns a compliment: they use "beleivers" to deal with this, this approach was only used in fanatic relegious / political movements before (by the way: I am not saying this is the way the go, but it is a cheap and powerfull policy to deal with C2C attacks).
     
    Last edited: Apr 7, 2009
  17. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Well usually i stay out of that because it doesn't make any sense to argue with guys that give a "****" about how detailed you try to explain something based on facts. I usually enjoy discussions but the latest trend in "antivirus and security" is really worrying. People open somewhere a website and try to educate people about computer security. Nothing wrong with that BUT most of the guys just blow you directly in the face "and i don't care about the other guys that have been working in this field for quite some time now". Here's the advise: Give some respect to the guys that have been working for some time in this field and you have a chance end up getting some respect from them. But all this takes some time. You don't gain respect because you think you DESERVE it you get that when OTHERS think you deserve it.
     
  18. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    I'm off topic, but it's not the first time I read such statements by you. If you don't like Prevx, then don't use it. But if you don't know at all how does it work, I suppose you should first ask to the company about your doubts before writing such sentences.

    Thank you

    Sorry for the OT
     
  19. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Btw good morning to Italy :D
     
  20. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    To you too :D

    I'm a bit far from the epicentre, ~150km, yet the earthquake has reached my city too :doubt:
     
  21. Cretemonster

    Cretemonster Registered Member

    Joined:
    Mar 31, 2005
    Posts:
    79
    Well here is a way to bring the heurineers out of the basement and into the sunlight, too bad its just another useless test that has no viable results.

    You 3 guys really should try getting out more and mingle with us little people or can you even see us from up there? :p

    This same discussion will come up again 1000 more times and the heurineers will come out and play again....SSDD.

    The dude with the wood for prevx definitly has a fitting picture for an avatar I must say. :D
     
  22. guest

    guest Guest

    Actually i tested 3 different malware but i didnt release result.
    Anyway.

    i selected very unsuspicious malware espacially.
    Because, i dont want to see proactive dedection rate. i want to see anti-crypter performance.

    i dont say, AV never cant catch crypted malware. Their proactive technology can be stopped many malware.

    For example, Panda has very good proactive dedection rate , Gdata can block Filecoder without malware fingerprints, Rising Hips can block filecoder, Comodo Defense+ too. And agnitum can block ...

    But we already know this information.

    if AV's proactive protection skipped file, if cant catch it... We know it is not perfect yet.

    This is only a test, surely it will have restrictions. All of the other tests also have restrictions.

    Our aim is attract attention to this subject and measure the existing performance. You can say what do you want. Internet is full of Crypter tools and you can see them all of the hack forums.

    Everbody who wants to have them can get them. And it can knock into a cocked any virus which is known by everbody. You know that it is not a new tool. But now we can see improved samples of them and it is increasing by the day. A lot of turkojen, poison and similar rat tools are used repeatedly after crypt. This test is done with virtual machine. But if the machine is real nothing will be change. We respect to your technology which is improved by you. But this is real that technology isn’t improved only by you. There is a other black side working. This black side is improving its technology. We only make them collide. This is a important subject. Bitdefender and the other AV firms are interested in this subject very much. But i don’t understant your stand out.

    "Yub, that's the next problem with these guys. I don't even ask for that anymore since you only get the standard answer "Why should i if AV doesn't detect it"."

    I don’t do this. I sent 250.000+ malware samples to the many AV firms. A lot of them get communication with me and take me a special statute.

    And i don’t give them that answer. "Why should i if AV doesn't detect it"

    Stephan did right. He wants answer when he doesn’t ascertain and always assesses.

    Also Comodo is like it. They want help from curious people like me and now they get million of malware sample. Maybe they have fastly developing database.

    Anyway, i respect who are you and your info. But theoretical info doesn’t interest me, i interest in impact. I did and saw.
     
  23. andyman35

    andyman35 Registered Member

    Joined:
    Nov 2, 2007
    Posts:
    2,336
    Difficult to argue with such logic.Since resources aren't infinite AV companies have to target the widest spread of malware possible and leave conceptual,POC stuff in the background.As always though there's a very simple remedy for this...disk imaging ;)
     
  24. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    LOL. That's EXACTLY what i mean with concentrating on stuff that hasn't a high priority. The malware what you submit is basically a new variant that you created yourself for such tests. That takes time and resources away that could be used to implement detection for the stuff that is REALLY hitting users.

    nvm, that is exactly what i mean: You don't even EVALUATE what i say you go straight to your "i know better" story. So be it then.

    No point in trying to explain something to somebody who isn't even interested in listening and valuables other input.
     
  25. Miyagi

    Miyagi Registered Member

    Joined:
    Mar 12, 2005
    Posts:
    420
    Location:
    Honolulu, Hawaii
    It basically boils down to business/personal reputation when such a test is performed. From a consumer point of view, I've seen countless tests that vary and never is a 100% satisfied testing. o_O

    One needs to be educated to see the real picture. Seeing only one side of a figure is not really a successful marriage. :cool:

    BTW - It's good to learn from these posts. Thanks for your input!
     
Loading...
Thread Status:
Not open for further replies.