[Thread split]MRG Flash Tests 2012

Discussion in 'other anti-virus software' started by LoneWolf, Jun 30, 2012.

Thread Status:
Not open for further replies.
  1. CloneRanger

    CloneRanger Registered Member

    Joined:
    Jan 4, 2006
    Posts:
    4,979
    Of course no Anti can be expected to get 100% on ALL malware every second of every day. But constantly achieving very high pass rates in ongoing tests such as MRG, clearly distinguishes the top performers, & as such are extremely valid & worthwhile, IMO.
     
  2. syk69

    syk69 Registered Member

    Joined:
    Feb 7, 2010
    Posts:
    183
    How is it extremely valid? If they don't provide insight on the tests. Are you just taking their word for it? Or did you read the details on all these tests performed some place on their website? I looked and couldn't find such info.

    I like the format they are using in showing which AV vendors are fast at adding detection for these malware samples. But without more info than just saying PASSED or FAILED it doesn't really make me trust them. Other than just taking it with a grain of salt.
     
  3. qakbot

    qakbot Registered Member

    Joined:
    Aug 25, 2010
    Posts:
    380
    As a number of folks on this thread have indicated, this test like many others should be taken with a grain of salt as there is no details provided on what constitutes a pass or a fail. No hashes provided and no URLs.
     
  4. Amin

    Amin Registered Member

    Joined:
    May 16, 2012
    Posts:
    437
    Location:
    UK
    yea i saw & that's exactly why i say there is something wrong with it :thumb:
     
  5. LoneWolf

    LoneWolf Registered Member

    Joined:
    Jan 2, 2006
    Posts:
    3,688
  6. Rompin Raider

    Rompin Raider Registered Member

    Joined:
    May 6, 2010
    Posts:
    1,253
    Location:
    North Texas
    Pretty similar to the last few.:(
     
  7. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    If you don't trust these tests then just move on but there are people that are still interested in these tests. ;)
     
  8. ams963

    ams963 Registered Member

    Joined:
    May 3, 2011
    Posts:
    5,983
    Location:
    Parallel Universe
    Yeah like me.:)
     
  9. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    4,602
    Location:
    Outer space
    Bruce from Malwarebytes has done a little test to see how 0day samples are detected on Virustotal, and rechecking the samples at later times like MRG does. SAS does not detect any of these not even the first sample which is retested 2 weeks later. Bruce's test is small scale with few samples so it can't really say much about the detection of a particular product, but it does support MRG's findings about SAS. You can find the test in the Malwarebytes' forum general chat section.
     
  10. Page42

    Page42 Registered Member

    Joined:
    Jun 18, 2007
    Posts:
    6,736
    Location:
    USA
    Thanks once again for the heads up, LW. :thumb:
     
  11. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,899
    Location:
    localhost
    One of the conclusion I can draw form this is that MRG and some AV producers are using the same honeypots/source. It seems highly unlikely that with thousands of new variants a day some products have consistently got 100% detection on a sample of them over several months (from June). Of course if we knew the exact detection type (signature, heuristics, etc.) we could make a better guess if this is really the case. But we don't, right? :rolleyes:
     
    Last edited: Aug 29, 2012
  12. Amin

    Amin Registered Member

    Joined:
    May 16, 2012
    Posts:
    437
    Location:
    UK
    yeah, i saw Bruce's tests and i liked it.
     
  13. CloneRanger

    CloneRanger Registered Member

    Joined:
    Jan 4, 2006
    Posts:
    4,979
    They do & have for some time ;)

    Info in these PDF's about the methodology etc used it their tests http://www.mrg-effitas.com/test-archive & http://www.mrg-effitas.com/current-tests

    Also Sveta has posted on here a number of times about them. Try searching via his username.

    Plus, PrevxHelp has posted on here previously that he appreciates their work etc.
     
  14. Amin

    Amin Registered Member

    Joined:
    May 16, 2012
    Posts:
    437
    Location:
    UK
  15. qakbot

    qakbot Registered Member

    Joined:
    Aug 25, 2010
    Posts:
    380
    Short tip not good enough. Need full detailed methodology and links to URLs, samples, or artifacts in general so we can do the test ourselves
     
  16. Amin

    Amin Registered Member

    Joined:
    May 16, 2012
    Posts:
    437
    Location:
    UK
    none of existing companies do the job which u expect.. but av-c always lists all malicious domains which are included in their test.
     
  17. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,875
    Location:
    Innsbruck (Austria)
    No, most testers (all I know of) provide only to AV vendors their misses (URLs, files, hashes) after the test for verification.
    It would be irresponsible to provide malware / malicious URLs to users, as well as being pointless (what would you do with hash? just search it on VT? make an own rogue AV which detects only those hashes?)

     
  18. nosirrah

    nosirrah Malware Fighter

    Joined:
    Aug 25, 2006
    Posts:
    561
    Location:
    Cummington MA USA
    There is nothing stopping anyone from collecting URLs from all of the public lists and replicating what MRG is doing.

    Think for a second what you need to replicate these tests:

    1. A safe execution environment that can quickly be reset - clone VM, launch, test, close, delete and repeat.

    2. A list of fresh malware URLs - these are posted on multiple sites and updated every day.


    You also have to keep in mind that handing these sources and samples out after the fact goes directly against what they have set out to prove. They are attempting to show that samples even half a day old are completely irrelevant.
     
  19. arsenaloyal

    arsenaloyal Registered Member

    Joined:
    Nov 1, 2009
    Posts:
    513
    defensewall is included in these tests,but its not an Antivirus/Antimalware,so what I am interested to know is are these tests about blocking or detection.
    Because I cant see defensewall detecting anything,but still says passed.
    If blocking then i believe sandboxie should be tested as well,and I am certain that it will block all the malwares thrown at it.
     
  20. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    The general advice being given by some AV experts is that this is not a good idea. However, say they did do this; by the time one wants to do the same test, it's likely the samples are pulled from the URLs listed, or the URLs themselves become "dead". I'm sure it's been said 0-day malware and the URLs they come from have a short time span.

    As Bruce has said, the only real way to replicate this type of test is to do it yourself using the known lists, providing you have the means to refresh the system to restore it back to its condition before being "infected".
     
  21. qakbot

    qakbot Registered Member

    Joined:
    Aug 25, 2010
    Posts:
    380
    AV-C is living in the last decade. You think the malware authors aren't smart enough that they dont know what are the holes in various products. Sorry, but a poor excuse.

    Besides there are lots of sites all over the web where you can download malware. Heck, some of them even allow you to request a specific piece of malware by hash, any people will oblige.
     
  22. qakbot

    qakbot Registered Member

    Joined:
    Aug 25, 2010
    Posts:
    380
    There are many tools available to capture the HTTP traffic, via a proxy. For example, Fiddler. You are correct, URLs are somewhat useless. But with the Fiddler logs I can completely re-create the attack any time I want. Post those. Or post all the artifacts that the tester gives to the AV vendors after each test. They do give them that don't they ?
     
  23. Or they have insiders on 0-day Malware boards. Don't be shocked this would happen, I'm sure it does and has been for some time.

    I think more tests the better, it's valuable information for us end users. IF a product stinks like SAS then we know not to fork over our hard earned money for it. MRG doesn't look to be a sponsored test so I would take them at their word.
     
  24. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    5,989
    Location:
    New York City
  25. Rompin Raider

    Rompin Raider Registered Member

    Joined:
    May 6, 2010
    Posts:
    1,253
    Location:
    North Texas
    Thanks Thankful...man, it's becoming the "witching hour" for some but as you said, Avast is stepping up!:thumb:
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.