Strength of sandboxes

Discussion in 'sandboxing & virtualization' started by Kees1958, Apr 19, 2008.

Thread Status:
Not open for further replies.
  1. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Hi,

    In the post https://www.wilderssecurity.com/showthread.php?t=206668 HIPS are tested, but also different approaches

    Policy restriction/virtualisation

    Defense wall scored: protected - prot - error (safe) -prot - prot
    GesWall scored : protected - error - error -prot - prot
    SafeSpace: protected - error - error - error - prot
    OA (run safer): protected - error - error - error - potected
    LUA (vista32) scored: Protected - error - error - error - vulnarable
    Sandboxie: protected - vulnarable - error - prot - vuln


    HIPS
    EQS: vulnarable - protected - error - error - vulnarable
    OA scored: vulnarable - vuln - vuln- protected - prot
    ThreatFire: vulnarable - vuln - vuln - protected - vuln
    Mamuto: vulnarable - vuln - vuln - vuln - vuln


    Conclusions
    Is it safe to say that DefenseWall, GeSWall, SafeSpace are the winners (working with default settings), with DefenseWall a minor advantage on points because it handles the threat in a elegant/controlled way (only one error)

    Policy restriction clearly reduces the attack space as OA prooves, also the LUA protection of Vista out performs classical HIPS.

    Other opnions?

    Off course this is only a small sample but DefenseWall, GeSWall and Sandboxie have a long reputation of with standing intrusions. Newby SafeSpace beats SBIE which is a major feat (I like the GUI over SBIE so this comment is coloured by presonal preference).
     
  2. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,047
    Location:
    Saudi Arabia/ Pakistan
    That,s a gross misinterpretation. Error means full PASS. In no way, Error is less than a full PASS. Showing less errors doesn,t at all means a better result.

    Test shwoed error when it was not able to run properly. It,s common with such tests/ POCs etc. U are not testing a real malware.
     
  3. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    I agree on the fact that with POC errors are common. Reason for this is that POC often address unforseen circumstances in the attacked program.

    But I disagree with the fact that error should be rated the same as a 'protected' answer. Getting a controlled NO means the attacked program had foreseen in this situation. Programming and testing quality standards rate a controlled exception handling as a better result than an uncontrolled error response. A protected answer means the door is closed, the attacker got some sort of negative answer.

    An error often means contol is not handled back in a predicted and repeatable manner. It ususally means inter program communication has crashed some where along the line. After an error you are uncertain to what degree the attack failed.

    I did not say DefenseWall was the clear winner, only a minor advantage on points, so addressing this as a gross misinterpretation is a bit over the top. In programming/testing/IT standards this is the normal quality rating:

    1. A targetted error message (so you get clues of why) with a formatted error response and the attacked program does not dump
    2. A general error message (caused either by the attacked program or by interferance of the OS, Coms or data base manager) and the attacked program does not dump
    3. A general error message cause by OS, communication/DB manager and the attacked program dumps (survived only one attack)
    4. A general error message cause by OS, communication/DB manager and the system software dumps (brings the complete system down)
    5. Instant system crash with no memory dump or what so ever messages


    Windows BSOD is an example of the fourth situation


    Regards Kees
     
    Last edited: Apr 19, 2008
  4. Tidyup

    Tidyup Registered Member

    Joined:
    Aug 6, 2007
    Posts:
    101
    One might argue that the presentation of an error suggested that the application was prevented from doing something, and that a lack of error suggested that the application achieved something. It all comes down to what stage of the test the exception occurred.

    Without understanding what the test samples were trying to do, it's impossible to say whether 'error' is better than 'fail', or vice versa.
     
  5. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,042
    I think thats what Melith said on the Comodo forum, was that an error meant it couldn't do it's task.

    Secondly, Tzuk responded on his forum about the Sandboxie results. His comment in essence was that Sandboxie not being a HIPS, wouldn't stop certain behavior for example setting a Global Hook. What it would do though is only permit setting the Hook on the program running in the sandbox, not system wide. Hence the POC would appear to have succeeded, but only on what's running in the sandbox. Therefore not meaningful test.


    So in this case I question the validity of the test rather than the validity of sandboxes.

    Pete
     
  6. MikeNAS

    MikeNAS Registered Member

    Joined:
    Sep 28, 2006
    Posts:
    697
    Location:
    FiNLAND
    Here are my results with Sandboxie 3.25.02:

    protected
    vulnerable
    testing
    protected
    testing

    Actually I have to disable most of my Testing Sandbox settings so I can even run this test.
     
  7. ErikAlbert

    ErikAlbert Registered Member

    Joined:
    Jun 16, 2005
    Posts:
    9,455
    I can't even run this test. When I unzip CLT.zip, AE removes everything : clt.exe + dll.dll + driver.sys. The unzip folder is empty.
    The strength of sandboxes ? The strength of AE maybe. :)
     
  8. computer geek

    computer geek Registered Member

    Joined:
    Oct 6, 2007
    Posts:
    776
    There's no need for worry then ;)
    I would argue that this was deliberate, and can be biased, since it was made by comodo. Besides, what company would make a test that their product fails? o_O
     
  9. MikeNAS

    MikeNAS Registered Member

    Joined:
    Sep 28, 2006
    Posts:
    697
    Location:
    FiNLAND
    I forgot to say that of course I have to disable DW too :D
     
  10. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,042
    I have to say that thought did cross my mind also.
     
  11. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Mike, release 2.4 of DefenseWall will have simular granular resource control (like GeSWall allready has). The nice thing is that DW out of the box is still as easy to use.

    Peter, Mike
    I do not know what the tests did, so we are left in the dark, but considering that SafeSpace offers simular functionality, it is suspicious that SafeSpace passes. So without knowing what they attacked, you can not say the tests are irrelevant.

    Aigle,
    Without knowing the context of the tests, Tidyup (a software developer) compared error with fail (in stead of pass) in his comment by intuition (without knowing that an error means an unsuccesfull attack). Maybe this put my comment of having three winners, with DW a minor advantage on points.

    Disclaimer
    I am off course also biased: DefenseWall over GeSWall, because DW always keeps untrusted marked as untrusted (unless user removes the file), while GW changes an untrusted file to trusted when you copy it from partition/disk A to B. Same with SafeSpace (nicer more user friendly interface, and it is not a pure shadow sandbox, but offers some additional containment (policy like) on processes and access to resources.

    Tip
    For all of you consideringen a security setup, run LUA. Vista also has a better architecture than XP, which allows for better process containment.

    When running LUA (at least in quiet mode) is impossibe, try at least a policy sandbox (when more users on PC) or a shadow sandbox (SafeSpace, Returnil are completely free). Or use the run safer capabilities of OA (It is a really strong option).

    Therefore on XP I would buy OA, on Vista32 I would use the free Comodo with reduced D+ settings ((https://www.wilderssecurity.com/showthread.php?t=199867)) and buy DefenseWall or use a free copy of SafeSpace. Next version of DW also has configurabe resource protection on processes, files and registry keys. On Vista 64 I would prefer VistaFireWall Control x64 (running LUA in quiet mode and Vista x64 architecture at the moment are so solid, that HIPS support is not really needed yet. and HauteSecure with a global profile enabled (https://www.wilderssecurity.com/showthread.php?t=205330).

    Regards Kees
     
    Last edited: Apr 20, 2008
  12. Huupi

    Huupi Registered Member

    Joined:
    Sep 2, 2006
    Posts:
    2,024
    Hi Kees,in all this you did'nt mention virti's like Powershadow,Shadowdefender and Returnil,they are in their early stages and some work has yet to be done,but IMO these are proponents of how security will evolve into the future. ;)
     
  13. MikeNAS

    MikeNAS Registered Member

    Joined:
    Sep 28, 2006
    Posts:
    697
    Location:
    FiNLAND
    Yeah I know. Using that...
     
  14. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Sorry had not seen your sig (DW = DefenseWall, GW=GeSWall, SB = Sandboxie and SD = ShadowDefender??)

    Could anyone update me on a specific sandbox issue with Digital Right Management (of music, video, especially the wav format is problematic)

    Last time I checked
    - DefenseWall - since version 1.61 no problem (now 2.3, 2.4 beta)
    - GeSWall - workaround available (was Pro version 2.6, so maybe 2.7 has it incorporated)
    - SafeSapace - working on the wav problem, others solved
    - SandBoxie - you can do it yourself, problem is that you have to know where/how data is stored, did not had a SBIE license, did not get a solution for it.

    regards Kees

    Regards Kees
     
  15. Kees1958

    Kees1958 Registered Member

    Joined:
    Jul 8, 2006
    Posts:
    5,857
    Well I tend to disagree about the future:
    A) Testers should go all the way (with VM)
    B) Simple end users should go for policy sandboxes (and maybe SafeSpace becasue it is a hybrid solution, offering DW/GW like protection, SBIE like protection and PowerShadow like protection, depending on the settings, for DW/GW like protection you have to PM TidyUp)

    Policy sandboxes are seamless, meaning that a user does not has to kown in which virtual data pocket his/hers files are in. DW offers a roll back service for power users, so there is really no need for untransparent shadow sandboxes).

    I agree, reason is that I have not seen any test results, maybe some members can post them.

    Regards Kees
     
  16. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,047
    Location:
    Saudi Arabia/ Pakistan
    THe test is to run/ execute and then defence it. It,s not a test of execution.
    That u can do simply by urself with any POC.
     
  17. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,047
    Location:
    Saudi Arabia/ Pakistan
    I agree with this one.
     
  18. MikeNAS

    MikeNAS Registered Member

    Joined:
    Sep 28, 2006
    Posts:
    697
    Location:
    FiNLAND
    GW=GhostWall
     
Loading...
Thread Status:
Not open for further replies.