Malware Research Group Flash Tests

Discussion in 'other anti-malware software' started by G1111, Dec 24, 2010.

Thread Status:
Not open for further replies.
  1. G1111

    G1111 Registered Member

    Joined:
    May 11, 2005
    Posts:
    2,127
    Location:
    USA
    It looks like MRG completed the tests. Results are posted here: -http://malwareresearchgroup.com/malware-tests/-
    DefenseWall and Emsisoft did not miss any and Hitman Pro only missed one.
     
  2. Franklin

    Franklin Registered Member

    Joined:
    May 12, 2005
    Posts:
    2,517
    Location:
    West Aussie
    How exactly does Defensewall "detect" samples o_O

    Isn't the Emsisoft engine the same as A-squared so why the miss by HMP?
     
  3. Boyfriend

    Boyfriend Registered Member

    Joined:
    Jun 7, 2010
    Posts:
    1,070
    Location:
    Pakistan
    Defensewall prevented system damage during execution tests.

    Emsisoft Anti-Malware also includes Mamutu, which detected malware during execution stage. For further detail, please read the report. They have stated above reasons in the report.
     
  4. Franklin

    Franklin Registered Member

    Joined:
    May 12, 2005
    Posts:
    2,517
    Location:
    West Aussie
    Defensewall - downloaded naturally means child is run as untrusted so maybe that's what they mean as detected?
     
  5. LODBROK

    LODBROK Guest

    And also to note for Hitman Pro is G-Data uses Avast (Engine B) and BitDefender (Engine A).
    Engine A = BitDefender
    Engine B = Avast
    Got it? :D
    And one of the A-Squared engines is Ikarus.
    So lots and lots of engines in HMP!
     
  6. Triple Helix

    Triple Helix Webroot Product Advisor

    Joined:
    Nov 20, 2004
    Posts:
    12,011
    Location:
    Ontario, Canada
    Hitman Pro has the following engines: Dr.Web, GData(2), Emsisoft Anti-Malware(2) and Prevx! That is 6 engines in Total!

    TH

    Capture24-12-2010-2.28.29 AM.jpg
     
  7. PJC

    PJC Very Frequent Poster

    Joined:
    Feb 17, 2010
    Posts:
    2,959
    Location:
    Internet
    EAM, DW, and HMP.
    That's why HMP is so Highly Respected among the Wilders members! :thumb:
    HMP has proved to be one of the Best Tools ever made! :thumb:
     
    Last edited: Dec 24, 2010
  8. Scoobs72

    Scoobs72 Registered Member

    Joined:
    Jul 16, 2007
    Posts:
    1,108
    Location:
    Sofa (left side)
    Yes, but disappointingly they don't state which components got the 'pass' in the each test. e.g. did the Mamutu component of EAM 'detect' everything or were they all signature detections. Great tests, badly presented results unfortunately.
     
  9. LODBROK

    LODBROK Guest

    Good points, Scoob...

    It would have been nice, too, to see reported which engine(s) got the hits in HMP.

    I find the AVG results suspect as 2011 wasn't even out yet when this series of tests was started. They began with AVG AV 9 which has no BB (the ID Protection component) and never announced when they switched to 2011 or how they upgraded - resident updater or clean install. Regardless, the results fly in the face of recent AV-Test and VB100 RAP scores; something was wrong with their AVG test bed.

    They started testing PCT AV, then PCT SD/AV, then back to AV.

    Some apps ran the whole several months, some just a few weeks.

    Sloppy procedures and reportage for sure. But it did hold my attention for a while.
     
  10. Solidify

    Solidify Registered Member

    Joined:
    Jun 26, 2010
    Posts:
    10
    immunet isnt too bad :)
     
  11. jmonge

    jmonge Registered Member

    Joined:
    Mar 20, 2008
    Posts:
    12,883
    Location:
    Canada
    what about SpyShelter?
     
  12. LODBROK

    LODBROK Guest

    It was Immunet Plus which was tested and it adds on the BitDefender engine to their cloud engines. The free version is cloud only. If you followed those near daily MRG tests, when BitDefender failed, Immunet Plus did likewise, with a few exceptions. The exceptions indicate their cloud model is viable, though, and as they evolve could prove to be quite effective.

    SpyShelter and MRG had a falling out a while back apparently over the philosophy of HIPS alerts presentation and the resulting human intervention needed as to what determines a pass or fail. I think there was a posting over at the MRG forum that SpyShelter chose to be excluded from the Flash tests.
     
  13. Gobbler

    Gobbler Registered Member

    Joined:
    Jul 30, 2010
    Posts:
    270
    One thing I didn't understand is that PE Guard and BluePoint went out of the tests at the very same time as HMP did but HMP is well mentioned in the report but is there no mention of PE Guard and BluePoint in the final test report while I believe that PE Guard and BluePoint along with EAM and DW were the only ones to score 100% in all the tests.o_O
     
  14. jmonge

    jmonge Registered Member

    Joined:
    Mar 20, 2008
    Posts:
    12,883
    Location:
    Canada
    thanks for the info
     
  15. lordraiden

    lordraiden Registered Member

    Joined:
    Jan 30, 2006
    Posts:
    3,067
    I dont know what happens with MRG that "nobody" want to be tested by them. It's a pity for us.
     
  16. LODBROK

    LODBROK Guest

    That's so true. But to clarify for readers that might stop in here:

    Hitman Pro is an "open it, run a scan (work the fix if needed), close it" application: on demand. And if I'm not mistarken it needs a live connection to Internet at the time of your scan.

    All the other apps in this test provide some sort of preemptive protection: real time.

    On demand is an emergency room, intensive care and managed care solution which is carried out after your position has been overrun by the enemy.

    Real time is your special forces unit out there on the front line.

    In both cases, how well it works depends largely on funding (suite versions) and training (code).

    Also to note, Malwarebytes free is on demand only (but no Internet needed) while their paid version adds a real time component which is what was tested by MRG. Well, according to their forum - you'd never know by reading their final pdf or the incremental Web reports.
     
  17. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    Nice EAM :D :thumb:
     
  18. G1111

    G1111 Registered Member

    Joined:
    May 11, 2005
    Posts:
    2,127
    Location:
    USA
    Not tested. They did have PE Guard in for awhile and it was doing quite well, but then they dropped it from the tests.
     
  19. 1chaoticadult

    1chaoticadult Registered Member

    Joined:
    Oct 28, 2010
    Posts:
    2,248
    Location:
    Chaotic Land
    Yep Noob. :thumb:
     
  20. jmonge

    jmonge Registered Member

    Joined:
    Mar 20, 2008
    Posts:
    12,883
    Location:
    Canada
    thanks G1111
     
  21. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    207
    Please read the report – and in particular, the methodology. You will see it clearly states “detect or block the function of the sample”. DW blocks the function of the malware by isolating it from the system.

    Regards,
    Sveta
     
  22. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    207
    These were flash tests and as such, were not intended to be as detailed as our standard tests. If you read the report, you will see it is clearly stated that it is not intended to be a rigorous assessment of efficacy. Further to this, you should note that it is not standard practice for testing organizations to detail which component of a security application caught a sample.

    Regards,
    Sveta
     
  23. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    207
    Please read the report. You will see in the introduction it is stated that it is not possible to detail the exact version of each application used. We used the most recent version in every test. Please also read the methodology which clearly states that each applications own update process is used before each test.

    Regards,
    Sveta
     
  24. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    207
    We do not test their products.

    Regards,
    Sveta
     
  25. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    207
    I am not entirely sure what you mean here as there are 22 applications in the test results. The applications included in the test are a mix of the most popular, many of which are our clients. In addition to this, several other vendors contacted us and asked that their products be included.

    If there are any applications excluded in these collated results, it is because we chose to drop them and not as a result of the vendor asking to be removed. I hope this clarifies the situation.

    Regards,
    Sveta
     
Loading...
Thread Status:
Not open for further replies.