[Thread split]MRG Flash Tests 2012

Discussion in 'other anti-virus software' started by LoneWolf, Jun 30, 2012.

Thread Status:
Not open for further replies.
  1. nosirrah

    nosirrah Malware Fighter

    Joined:
    Aug 25, 2006
    Posts:
    561
    Location:
    Cummington MA USA
    Yes but it is not easy. You have no choice but to look at past generations to predict what they are likely to do to break free from the security collective and then work around these changes ahead of time. If you are dealing with a particularly nasty infection you may even need to make more than 1 prediction just to be sure. Even then this is sometimes not enough, sometimes you have to start over because the changes are just too great. In these cases being fast is the only option (or to block the actual source).
     
  2. Tarnak

    Tarnak Registered Member

    Joined:
    Feb 5, 2007
    Posts:
    4,764
    Me too! As long as I don't get bitten ...by the malware bug :D
     
  3. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
  4. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,490
    I'm really loving those small test threads. :D
    Kinda interesting. :cool:

    Thanks for the update malexous!
     
  5. Esse

    Esse Registered Member

    Joined:
    May 26, 2011
    Posts:
    414
    Interesting that Panda failed the same 3 after 6 hours, cloud an all I mean.

    I think I read in the Prevx (WSA) (cloud and all) forum that they did react minutes after, when missed a detection.
    I really miss WSA in these tests.


    /E
     
  6. JoeBlack40

    JoeBlack40 Registered Member

    Joined:
    Apr 1, 2009
    Posts:
    1,579
    Location:
    Romania
    All i can say about Avast is well done and congratulations to the team.:thumb:
     
  7. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
  8. m0use0ver

    m0use0ver Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    81
    Is it just me or does MRG samples 0 hour samples always seem well detected(even if missed by some) yet what Nosirrah is tracking is the exact opposite.

    Regardless of whoever the vendor is they all seem to be struggling so would MRG not provide a more accurate picture if they used the same source ?

    Unfortunetly and excuse the cynic in me but have already come to the conclusion that MRG will not publish data that would be consistently damning of any of their paying clients.They soon would have no clients afterall:blink:

    That said it would still be interesting to see one of their detection tables based around Nosirrahs test findings..predominently fails across the board and that includes vendors that are paying clients to MRG services with near perfect scores todate in their flash tests...
     
  9. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
    Last edited: Jul 24, 2012
  10. nosirrah

    nosirrah Malware Fighter

    Joined:
    Aug 25, 2006
    Posts:
    561
    Location:
    Cummington MA USA
    @m0use0ver

    You have to keep in mind that me and MRG are trying to demonstrate two totally different points. MRG is picking a handful of nasty but very common infections from different sources every test and I have cherry picked 2 sources I know that everyone will be terrible against.

    MRG is trying to show that when you remove the element of being able to prepare for an attack it is much more difficult to pass their test. Keep in mind that MRG is also including blocking the source itself, I am not at all taking this into consideration.

    My goal is a lot more simple. I am simply trying to demonstrate that "current malware" needs to be redefined to a much smaller window of time. I list the full information for a reason as well, I want the vendors to get these samples and then attempt to prepare for the coming mutations. The sources for these infections are also known. I have posted them directly here and on our forums and anyone paying attention should already have them. Allowing everyone to prepare is by design emphasizing the terribly small window of relevance when it comes to "current malware".
     
  11. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    If the sources are known, why aren't the vendors who have missed detections (in the short timeframe as per your test) not adding signatures much quicker than it would seem?
     
  12. nosirrah

    nosirrah Malware Fighter

    Joined:
    Aug 25, 2006
    Posts:
    561
    Location:
    Cummington MA USA
    I cannot answer that. I would like to reiterate that I selected 2 sources that I knew ahead of time would have terrible detection. One is an exploit drop and the other is a fake scanner drop. These types of infections are well known to have poor detection and a very rapid morph rate.
     
  13. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
    Comment by Sveta.
     
  14. cruelsister

    cruelsister Registered Member

    Joined:
    Nov 6, 2007
    Posts:
    1,431
    Location:
    Paris
    To MRG- You also may want to (for your amusement) note the time of day as well as the day of the week that the malware was initially released. The number of people and the amount of resources allocated to the emergency response teams vary depending on the company's size and true focus.

    It can be seen that response will differ with respect to the time parameters listed above- basically great if zero hour is 0900 local time on a weekday, but not so good at 0100 local on a Sunday.
     
  15. Miyagi

    Miyagi Registered Member

    Joined:
    Mar 12, 2005
    Posts:
    424
    Location:
    Honolulu, Hawaii
    :thumb:
     
  16. Solarlynx

    Solarlynx Registered Member

    Joined:
    Jun 25, 2011
    Posts:
    2,006
    That's great that DefenseWall is there! I love to see how it beats all malware.
    Why no Comodo?
    Why no AppGuard?
    They definitely must be there, please.
     
  17. Morphiusz1

    Morphiusz1 Registered Member

    Joined:
    Jul 25, 2012
    Posts:
    1
    Location:
    Poland
    MRG and Comodo don't like each other.
     
  18. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    Nonsense. Hashes are for transparency not for VT-rechecking.
    f.e.: They claim x misses y and call y "Zeus-variant". Wow, a "believe it or not" game...

    And because they use only a few samples posting hashes will not be the big problem.
     
  19. markusg

    markusg Registered Member

    Joined:
    Jun 10, 2009
    Posts:
    248
    do you get also hashes from av-comp and others, do not think so, can not see the problem.
     
  20. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    First: This are often large tests with more samples.
    Second: Because others are not transparent it doesn't mean that all have to do that way.
    Third: Your example AV-C doesn't have such a strange "history" like MRG (SSUpdater "we not" lies, different names here on wilders, Comodo story...). So transparency is a must for MRG.
     
  21. nosirrah

    nosirrah Malware Fighter

    Joined:
    Aug 25, 2006
    Posts:
    561
    Location:
    Cummington MA USA
    That makes their case worse, not better. No matter what its 'generate hash, add to list next.....'. They don't disclose the information that would make it obviously why so many vendors did better than 85%.

    In Sveta's case live sources can be blocked at the source so the actual samples are meaningless. Sources die far too quickly to do a test and report them in a way that can be verified. He could make it sample based only but then it becomes a fake test and completely invalid.

    There are list of live sources all over the web and very simple ways to setup your own test. Anyone that wants to verify this new way of testing can any time of any day. If anything this is a lot MORE disclosure as anyone can replicate the test with a little work.

    You should in fact NEVER trust a test that you cannot replicate in any meaningful way.

    For the record, I have received every sample we have ever missed in his testing without issue. Also, if you were to give 100 samples to Sveta that were collected 3 days ago he would NOT put them into a test because he knows that they are already obsolete.
     
  22. Solarlynx

    Solarlynx Registered Member

    Joined:
    Jun 25, 2011
    Posts:
    2,006
    Really? That's a pity.
    Could you give a link to the source of this info, please.
     
  23. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,490
    They had some kind of misunderstanding in the past. I forgot why. :D
     
  24. Solarlynx

    Solarlynx Registered Member

    Joined:
    Jun 25, 2011
    Posts:
    2,006
    Hopefully they will get along.
    Imho UppGuard deserves MRG attention as well.
     
  25. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    I never said they are "better".

    His test is not only sources based, also family based cause he reports malware sample/family names.

    And - at last for me - such type of tests would be more interesting if there would be
    a.) details, what blocked the file (signature, proactive detection etc.), even if it is not in all cases clear to find out.
    b.) transparency, which is for me more than posting a few tables ;)
     
Loading...
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.