[Thread split]MRG Flash Tests 2012

Discussion in 'other anti-virus software' started by LoneWolf, Jun 30, 2012.

Thread Status:
Not open for further replies.
  1. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    People are getting anal about the results because their programs didn't do a good job? :rolleyes:
     
  2. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    It has nothing to do with "wrong" but there are enough issues every time with MRG. This time:
    - report claims vendors got sample of the old simulator. At least not fully true, some vendors got nothing, not even the information that their products are in the test.
    - issue with the date...
    - sandboxie issue (beside the nonsense with default settings in that case), why SB prevents 2 and 3 but not 1 in default settings? Doesn't make sense for me.

    ~~ comment removed ~~
     
    Last edited by a moderator: Nov 18, 2012
  3. Frank the Perv

    Frank the Perv Banned

    Joined:
    Dec 16, 2005
    Posts:
    882
    Location:
    Virginia, USA

    Agree.

    It's like a sport here to criticize all tests.

    As far as I see it, most of the tests are pretty well done. When the test proprietors come and explain the tests -- they seem to have good methodology almost every time.

    And... not every test can be perfect in every way. That's just the nature of testing.

    But all the noobs (no direct reference intended) who have issues with tests when their product does not do well.... whatever.:rolleyes:


    -ftp
     
  4. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    4,867
    Location:
    Outer space
    They have updated the test report and added a footnote that DW is tested on a 32 bit system, they also explained more about the sample submission:
    :thumb:
     
  5. Mman79

    Mman79 Registered Member

    Joined:
    Sep 19, 2012
    Posts:
    2,016
    Location:
    North America
    And on the opposite side you have the ones who will shoot down any and all results that differ from their testing company of choice or their "home tests". It's the Internet, what can I say :D

    @Frank the Perv: Fanboys and girls will be fanboys and girls. The word "noobs" implies they don't know any better, but they almost always do. Let 'em pout and groan, it won't change the test results.

    On the issue of Sandboxie, as I understand it MRG normally tests products with their highest/tightest settings. I'm not certain why this was not the case for Sandboxie? It's possible it still could have failed ((And at this point I should point out that no, I don't want to hear nor care to hear how some may consider that nonsense.)), but perhaps it was not given its best shot at the tests by using a weak default configuration? I'm no Sandboxie fan, I don't even use it. But I figure if you're going to give something a proper test, put it at full power and then see if it holds.
     
    Last edited: Nov 18, 2012
  6. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Now this is the whole point as far as Sandboxie is concerned, I know how it works, had it been tested properly configured and failed, I would have been disappointed but probably wouldn't have posted any comment.

    When tests results are available, some people give them a cursory glance and some want to know exactly how they were carried out. For future tests of Sandboxie, I just wish at this stage for MRG to make sure to configure it properly.
     
  7. Mman79

    Mman79 Registered Member

    Joined:
    Sep 19, 2012
    Posts:
    2,016
    Location:
    North America
    Well, that depends. I'm guessing MRG consults vendors before and after the tests ((I think most vendors request high settings for testing)) so if all this was talked over with Tzuk beforehand, the blame can't necessarily be placed on MRG. Unless MRG or Tzuk speaks up about it specifically, no one is going to have a clue.
     
  8. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    According to Tzuk, he was told by MRG that their official stance is to test applications with default settings. I presume that such a position also makes it easier for them to speed up tests, to configure tightly 32 applications would be time consuming and probably introduce too many variables within the tests.
     
  9. Mman79

    Mman79 Registered Member

    Joined:
    Sep 19, 2012
    Posts:
    2,016
    Location:
    North America
    You're right, I was thinking about AV-C testing. Well, then I'm not surprised at all Sandboxie failed some tests. It won't block anything whatsoever without locking down settings most users would never touch and would annoy them far too much.
     
  10. CoolWebSearch

    CoolWebSearch Registered Member

    Joined:
    Sep 30, 2007
    Posts:
    1,247
    Tzuk specifically said on his own Sandboxie forum that he asked MRG to test Sandboxie with configured maximum protection. They refused, because they say they only default settings.
    Honestly, this is quite dumb why they don't test both default and configured maximum protection in the first place?
    I was rookie in using Sandboxie for example, and had couple of penetrations until I actually learned to use Sandboxie, after that nothing came in or out.
    The other part test what malware or anything else inside the sandbox with start/run restrictions and internet access restrictions, whiy is it so hard to do it?
    Because of these reasons I don't consider MRG very reliable, they seem to put shadow on security programs just because they test it with default configuration.
    Real shame...
     
  11. gerardwil

    gerardwil Registered Member

    Joined:
    Jan 17, 2004
    Posts:
    4,750
    Location:
    EU
    Apart from a handfull more or less expert users I think most (home) pc users worlwide are using security apps. with default configuration.
     
  12. CoolWebSearch

    CoolWebSearch Registered Member

    Joined:
    Sep 30, 2007
    Posts:
    1,247
    People can learn to configure but only if they want to, I did. This is why we have forums like this.
     
  13. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    I have realised the same, but don't find it the right way. Reading some critic points than just secretly updating report without announcements is not professional.
     
  14. skokospa

    skokospa Registered Member

    Joined:
    Apr 1, 2009
    Posts:
    177
    Location:
    Srbija
    from my experience is exactly as you write.users want to install the program and forget about it.
     
  15. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    We all know that if SBIE was configured properly it would have passed all tests so everything is fine anyways.
    As long as the test mentions that it was tested in default settings i see no problem at all with the results. :D
     
  16. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Well, we really don't know, it is an assumption that it would pass all the tests when properly configured, hence I was looking forward to this test. Sandboxie hasn't been tested yet...
     
  17. bo elam

    bo elam Registered Member

    Joined:
    Jun 15, 2010
    Posts:
    6,144
    Location:
    Nicaragua
    ....and it should not be tested against keyloggers. Why?..because SBIE is not an anti keylogger. Can SBIE be expected to detect a keylogger when is downloaded? No, Sandboxie is not designed to detect anything.
    http://www.sandboxie.com/index.php?DetectingKeyLoggers#defend

    Does SBIE protect us against malware running outside the sandbox? like in test 1 that has the simulator running outside the sandbox. No.

    http://www.sandboxie.com/index.php?FrequentlyAskedQuestions#KeyLoggers

    @MRG, next time, if you are going to test SBIE, test it for what its designed to do. Sandboxie is not a detection tool and it wont do nothing when malware is running outside the sandbox like in test 1.

    @Noob, perv, Sandboxie actually did pretty good in the test despite being tested for something that it doesnt do but I guess you guys don't know that since you haven't read much about Sandboxie. Start in page 1.

    http://www.sandboxie.com/index.php?GettingStarted

    Bo
     
  18. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    Hahahahaha well to be honest i'm not the one ranting about SBIE results so . . . :rolleyes:
    I'm fine with the results. (Yeah EAM did excellent but even if it didn't perform as expected i would be okay . . . :D)
     
  19. bo elam

    bo elam Registered Member

    Joined:
    Jun 15, 2010
    Posts:
    6,144
    Location:
    Nicaragua
  20. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,491
    Emsisoft has an awesome support forum. (The best i've ever seen and experienced)
    They always answer me within hours and do a great job sorting out my issues.
     
    Last edited: Nov 20, 2012
  21. a256886572008

    a256886572008 Registered Member

    Joined:
    Oct 26, 2007
    Posts:
    103
    Did they test the X64 version of DefenseWall hips?
     
  22. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    This is probably Sandboxie Achilles' heel in terms of becoming a very popular application among the huge market of AV users. It definitely has a learning curve, and even at default settings it needs some reading in order to understand how to extract things from the sandbox. This is what Tzuk should try to work out in the future if he wants to expand his market, to make things more automated with just a few clicks in very straightforward English.
     
  23. chris1341

    chris1341 Guest

    I find the reaction to these tests very strange. Why the focus on a product that does not claim to prevent keyloggers? Particularly when so many that claim they do failed miserably.

    My main view of the tests was 'so what'. They didn't tell me anything I wouldn't have guessed right anyway.

    If I was surprised at anything its that Defensewall was tested on x32 while the others were x64. Most are aware of the problems vendors have on x64 compared to x32 in implementing kernal hooking/patching. Indeed the reason we don't have a x64 Defensewall is because the developer tells us he won't compromise protection. Others are therefore tested without cognisance of the compromises they've made that Defensewall does not have to make?

    Before I get lambasted I really like Defensewall, its a brilliant product but by any logic all should have been tested on the same platform. Could some that flopped have done better on x32? Who knows?

    In general I don't think MRG have covered themselves in glory on this one. Testing products that could never pass and compromising the tests consistency to include others seems counter productive to me. Unfortunate because I usually like how they approach these things.
     
  24. Solarlynx

    Solarlynx Registered Member

    Joined:
    Jun 25, 2011
    Posts:
    2,015
    No, there is no DWx64 yet.
     
  25. CoolWebSearch

    CoolWebSearch Registered Member

    Joined:
    Sep 30, 2007
    Posts:
    1,247
    Actually, you're wrong here. SBIE does actually protect against keyloggers, but you have to configure it. For example in my computer the download is blocked on D: partition, C: partition is not blocked, but it is completely protected from modifying. Internet access is also blocked for all files, folders, and etc, except for several exe. files, I also blocked majority of exe files to even start/run. So if I have a keylogger inside my sandbox, it cannot access the internet and but it also cannot start/run in the first place, it just stays inside SBIE and cannot do anything unless I change my restriction policy.
    In that way there is no way an keylogger or any other malware can do anything inside SBIE.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.