MRG Team performes their first Real Time test!

Discussion in 'other anti-malware software' started by Astech, Aug 4, 2009.

Thread Status:
Not open for further replies.
  1. Astech

    Astech Registered Member

    Joined:
    Jan 17, 2009
    Posts:
    67
    Malware Research Group just published their first real time test, they used "only" 22 samples but that proved too much for some!

    They also tested only ten programs, but in my opinion almost all of my favorites are there.

    Enjoy!

    http://malwareresearchgroup.com/forum/viewtopic.php?f=20&t=47

    p.s. I'm glad that I use what I use.....
     
  2. the Tester

    the Tester Registered Member

    Joined:
    Jul 28, 2002
    Posts:
    2,854
    Location:
    The Gateway to the Blue Hills,WI.
    That's interesting.
    Small malware database, but at least the malware that is missed by a program in the test is listed.

    Astech,
    They did also later test Prevx and MSE according to that page. That makes an even dozen programs tested.
     
  3. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    Neither the full Panda security product was included, nor PCAV (Cloud Antivirus) - I would be most interested to see how beta 2 of PCAV does on this test.
     
  4. Astech

    Astech Registered Member

    Joined:
    Jan 17, 2009
    Posts:
    67
    Looks like I missed Prevx and MSE, wasn't posted at time I was there. Excellent result for Prevx I might add:thumb:
     
  5. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,731
    Location:
    New York City
    MSE missed six malware.
     
  6. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    Yes, we can see that. :D :p
     
  7. The fox

    The fox Registered Member

    Joined:
    Jan 25, 2007
    Posts:
    28
    Quite intresting result, Altho I have seen reviews where NOD32 manages to catch all malicious programs, and also managed to clean a sandbox almost completely from viruses, rootkits etc, so I wonder how they got to the results they did.
     
  8. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,852
    I'm a bit skeptical they used only 22 malware samples some of which are 2 months old, in 2 months you could collect about 10k samples, I wouldn't be skeptical at all if these samples we're "high hitters" popular infections, but for all we know, they could be 1-hit extinct files, which brings us back to http://blogs.technet.com/mmpc/archi...your-guide-a-proposal-for-security-tests.aspx

    Can someone prove me wrong?
     
  9. Astech

    Astech Registered Member

    Joined:
    Jan 17, 2009
    Posts:
    67
    Humm one would expect that all should be capable of detecting "high hitters", they also used some very fresh samples, but then again you can never predict which will become "high hitters" and when;)

    p.s. Conficker was discovered about 2 months before people took it seriously, and we all know how that story ended.
     
  10. LoneWolf

    LoneWolf Registered Member

    Joined:
    Jan 2, 2006
    Posts:
    3,408
    Interesting indeed.
     
  11. StevieO

    StevieO Registered Member

    Joined:
    Feb 2, 2006
    Posts:
    1,067
    Yes yes yes, very nice and i'm sure they are viable results. But i thought tests with limited samples wern't allowed on here ? I know my recent test on several Anti's got locked.

    I'm NOT saying this thread, or similar, should be pulled/locked etc, as i feel they are very good " indicators " as to how well products are ahead, or not, at that particular time.

    And further, you can definately piece together a consistent pattern of effectiveness, or not, of products the more that tests such as this are done. Same thing goes for online testing of samples, with the likes of VT/Jotti etc, and of course now the Cloud based approach too.
     
  12. eXPerience

    eXPerience Registered Member

    Joined:
    Apr 28, 2009
    Posts:
    98
    Hi,

    it could be true that they have 10.000 samples, but for the moment, there is no system of testing them all realtime. Remember that it all takes a long time to test, it's not a simple on-demand scan.

    What I did find funny was this :
    Later followed by this :
    But I guess we all know where the strong and weak part of CIS is ?


    Also nice to finally see Prevx tested ! It's seems like a good application, keep up the good works dev team :thumb:

    eXPerience
     
  13. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    Prevx was already tested by PCMag, which Prevx themselves said did them justice in terms of methodology. This is quite some time ago.
     
  14. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    They would do justice testing Norton 2010 as it's beta just like for example MSE - there's a huge difference between 2009 and 2010.
     
  15. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,852

    Um, if the samples are from 2 months ago, sounds like an on-demand scan to me my friend. The only true real time tests I've seen are the ones on youtube with people downloading FRESH files from malicious URL's.
     
  16. Retadpuss

    Retadpuss Suspended Member

    Joined:
    Apr 4, 2009
    Posts:
    226
    Not sure which test you are talking about? The test we are all talking about is the one in the first hyperlink!

    The test clearly says:

    "Samples used in this test are a mix of new (few days old) and a bit older (about one month old)"

    The methodology is also detailed and states:

    8. All 22 samples of malware are packed inside exe files (executable files).
    9. The test is conducted by trying to run (execute) the malicious applications (samples).

    Puss
     
  17. funkydude

    funkydude Registered Member

    Joined:
    Apr 5, 2004
    Posts:
    6,852
    Ok, now that sounds like real-time scanning, but it also sounds like a "what decompression methods you support" test, thank you.
     
  18. subset

    subset Registered Member

    Joined:
    Nov 17, 2007
    Posts:
    825
    Location:
    Austria
    The methodology is soo detailed that you don't even know the settings they have chosen for the programs. :p

    Were all products tested with out-of-the-box settings? Or highest settings?
    Also CIS with Defense+... but the mode was o_O

    Cheers
     
  19. CogitoTesting

    CogitoTesting Registered Member

    Joined:
    Jul 4, 2009
    Posts:
    901
    Location:
    Sea of Tranquility, Luna
    I personally tested NIS 2010 beta and I really was not impressed. Do you believe that NIS 2010 recommended me to install coolwebsearch as a safe product, a known spyware? Needless to say that I was really taken aback. Very soon CogitoTesting will dynamically test a lot of these products and we will be fair to all.

    Tests are being conducted as I'm typing the reply. Our blog will be up and running soon where all of our (testing) videos will be posted as well as on youtube. We are testing the least known products first, no disrespect to any companies, like F-Prot, Arcavir, K7 Computing, QuickHeal etc...
     
  20. jmonge

    jmonge Registered Member

    Joined:
    Mar 20, 2008
    Posts:
    12,883
    Location:
    Canada
    are you testing hips/sandboxes?if you do please test DefenseWall;)
     
  21. CogitoTesting

    CogitoTesting Registered Member

    Joined:
    Jul 4, 2009
    Posts:
    901
    Location:
    Sea of Tranquility, Luna
    Yes we will test not only Defensewall but also GeSwall. For that matter, if you or anyone has a product that they would like us include in our tests please let me know through some posts or private messages. Our testing calendar is pretty much set though. I will soon post our contact information as well as the link to our website a.k.a blog when it is up and running.
     
  22. Pleonasm

    Pleonasm Registered Member

    Joined:
    Apr 9, 2007
    Posts:
    1,201
    Agreed. As suggested by the Microsoft Malware Protection Center, a key improvement in AV-comparatives is to weight the detection of each sample by its prevalence. In this way, the test will more closely mirror the real world experiences of users. Unfortunately, this was not done by the Malware Research Group.
     
  23. jmonge

    jmonge Registered Member

    Joined:
    Mar 20, 2008
    Posts:
    12,883
    Location:
    Canada
    thanks alot:thumb:
     
  24. Pleonasm

    Pleonasm Registered Member

    Joined:
    Apr 9, 2007
    Posts:
    1,201
    A good point.

    As Dan from Symantec said, “inside tip - wait till av-comparatives and other testers of pro-active detection test it - it {Norton Internet Security 2010} will blow away our previous approach” (see this thread).
     
  25. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    Norton recommended you to... what? :blink: Uh... I really need a screenshot of this... o_O
     
Loading...
Thread Status:
Not open for further replies.