Strength of sandboxes

Hi,

In the post https://www.wilderssecurity.com/showthread.php?t=206668 HIPS are tested, but also different approaches

Policy restriction/virtualisation

Defense wall scored: protected - prot - error (safe) -prot - prot
GesWall scored : protected - error - error -prot - prot
SafeSpace: protected - error - error - error - prot
OA (run safer): protected - error - error - error - potected
LUA (vista32) scored: Protected - error - error - error - vulnarable
Sandboxie: protected - vulnarable - error - prot - vuln


HIPS
EQS: vulnarable - protected - error - error - vulnarable
OA scored: vulnarable - vuln - vuln- protected - prot
ThreatFire: vulnarable - vuln - vuln - protected - vuln
Mamuto: vulnarable - vuln - vuln - vuln - vuln


Conclusions
Is it safe to say that DefenseWall, GeSWall, SafeSpace are the winners (working with default settings), with DefenseWall a minor advantage on points because it handles the threat in a elegant/controlled way (only one error)

Policy restriction clearly reduces the attack space as OA prooves, also the LUA protection of Vista out performs classical HIPS.

Other opnions?

Off course this is only a small sample but DefenseWall, GeSWall and Sandboxie have a long reputation of with standing intrusions. Newby SafeSpace beats SBIE which is a major feat (I like the GUI over SBIE so this comment is coloured by presonal preference).

 

That,s a gross misinterpretation. Error means full PASS. In no way, Error is less than a full PASS. Showing less errors doesn,t at all means a better result.

Test shwoed error when it was not able to run properly. It,s common with such tests/ POCs etc. U are not testing a real malware.

 

I agree on the fact that with POC errors are common. Reason for this is that POC often address unforseen circumstances in the attacked program.

But I disagree with the fact that error should be rated the same as a 'protected' answer. Getting a controlled NO means the attacked program had foreseen in this situation. Programming and testing quality standards rate a controlled exception handling as a better result than an uncontrolled error response. A protected answer means the door is closed, the attacker got some sort of negative answer.

An error often means contol is not handled back in a predicted and repeatable manner. It ususally means inter program communication has crashed some where along the line. After an error you are uncertain to what degree the attack failed.

I did not say DefenseWall was the clear winner, only a minor advantage on points, so addressing this as a gross misinterpretation is a bit over the top. In programming/testing/IT standards this is the normal quality rating:

1. A targetted error message (so you get clues of why) with a formatted error response and the attacked program does not dump
2. A general error message (caused either by the attacked program or by interferance of the OS, Coms or data base manager) and the attacked program does not dump
3. A general error message cause by OS, communication/DB manager and the attacked program dumps (survived only one attack)
4. A general error message cause by OS, communication/DB manager and the system software dumps (brings the complete system down)
5. Instant system crash with no memory dump or what so ever messages


Windows BSOD is an example of the fourth situation


Regards Kees

 

One might argue that the presentation of an error suggested that the application was prevented from doing something, and that a lack of error suggested that the application achieved something. It all comes down to what stage of the test the exception occurred.

Without understanding what the test samples were trying to do, it's impossible to say whether 'error' is better than 'fail', or vice versa.

 

Here are my results with Sandboxie 3.25.02:

protected
vulnerable
testing
protected
testing

Actually I have to disable most of my Testing Sandbox settings so I can even run this test.

 

I can't even run this test. When I unzip CLT.zip, AE removes everything : clt.exe + dll.dll + driver.sys. The unzip folder is empty.
The strength of sandboxes ? The strength of AE maybe.

 

There's no need for worry then
I would argue that this was deliberate, and can be biased, since it was made by comodo. Besides, what company would make a test that their product fails?

 

I forgot to say that of course I have to disable DW too

 

Mike, release 2.4 of DefenseWall will have simular granular resource control (like GeSWall allready has). The nice thing is that DW out of the box is still as easy to use.

Peter, Mike
I do not know what the tests did, so we are left in the dark, but considering that SafeSpace offers simular functionality, it is suspicious that SafeSpace passes. So without knowing what they attacked, you can not say the tests are irrelevant.

Aigle,
Without knowing the context of the tests, Tidyup (a software developer) compared error with fail (in stead of pass) in his comment by intuition (without knowing that an error means an unsuccesfull attack). Maybe this put my comment of having three winners, with DW a minor advantage on points.

Disclaimer
I am off course also biased: DefenseWall over GeSWall, because DW always keeps untrusted marked as untrusted (unless user removes the file), while GW changes an untrusted file to trusted when you copy it from partition/disk A to B. Same with SafeSpace (nicer more user friendly interface, and it is not a pure shadow sandbox, but offers some additional containment (policy like) on processes and access to resources.

Tip
For all of you consideringen a security setup, run LUA. Vista also has a better architecture than XP, which allows for better process containment.

When running LUA (at least in quiet mode) is impossibe, try at least a policy sandbox (when more users on PC) or a shadow sandbox (SafeSpace, Returnil are completely free). Or use the run safer capabilities of OA (It is a really strong option).

Therefore on XP I would buy OA, on Vista32 I would use the free Comodo with reduced D+ settings ((https://www.wilderssecurity.com/showthread.php?t=199867)) and buy DefenseWall or use a free copy of SafeSpace. Next version of DW also has configurabe resource protection on processes, files and registry keys. On Vista 64 I would prefer VistaFireWall Control x64 (running LUA in quiet mode and Vista x64 architecture at the moment are so solid, that HIPS support is not really needed yet. and HauteSecure with a global profile enabled (https://www.wilderssecurity.com/showthread.php?t=205330).

Regards Kees

 

Hi Kees,in all this you did'nt mention virti's like Powershadow,Shadowdefender and Returnil,they are in their early stages and some work has yet to be done,but IMO these are proponents of how security will evolve into the future.

 

Yeah I know. Using that...

 

Sorry had not seen your sig (DW = DefenseWall, GW=GeSWall, SB = Sandboxie and SD = ShadowDefender??)

Could anyone update me on a specific sandbox issue with Digital Right Management (of music, video, especially the wav format is problematic)

Last time I checked
- DefenseWall - since version 1.61 no problem (now 2.3, 2.4 beta)
- GeSWall - workaround available (was Pro version 2.6, so maybe 2.7 has it incorporated)
- SafeSapace - working on the wav problem, others solved
- SandBoxie - you can do it yourself, problem is that you have to know where/how data is stored, did not had a SBIE license, did not get a solution for it.

regards Kees

Regards Kees

 

Well I tend to disagree about the future:
A) Testers should go all the way (with VM)
B) Simple end users should go for policy sandboxes (and maybe SafeSpace becasue it is a hybrid solution, offering DW/GW like protection, SBIE like protection and PowerShadow like protection, depending on the settings, for DW/GW like protection you have to PM TidyUp)

Policy sandboxes are seamless, meaning that a user does not has to kown in which virtual data pocket his/hers files are in. DW offers a roll back service for power users, so there is really no need for untransparent shadow sandboxes).

I agree, reason is that I have not seen any test results, maybe some members can post them.

Regards Kees

 

THe test is to run/ execute and then defence it. It,s not a test of execution.
That u can do simply by urself with any POC.

 

I agree with this one.

 

GW=GhostWall