New Leaktest / Security Tool Released - System Shutdown Simulator

Has anyone tried Comodo v3 against it?

Thanks

 

Good old Sygate and avira free passed. Threatfire did nonting. Got no other HIPS.

Cheers.

 

Hi MitchE323, and welcome to the forum. Try the "search" function if there's anything particular you're looking for. Quite a few SandboxIE threads (and fans) here

 

Hey right back at ya man, i've seen you over at Sandboxie before and now I run into you again, over here!
Anyways though, Happy Thanksgiving to everyone else as well!

 

Hi Mitch and welcome to Wilders.

Have picked up many good tips on Sandboxie from MitchE323 over at SB's forum.

 

Hello,
Interesting to see all the responses!

Regarding the results of SSS, when I released it I expected most software to pass and only some poorly written software to fail.

However what surprised me was that software like Avast AV, System Safety Monitor, Eqsecure, KIS 7 and Comodo 3 RC1 (Not current release) all FAILED!

I've also released the new version as mentioned (1.0.21) that fixes an issue with the ping results being misleading in some cases.

Regarding Sandboxie, it isn't a firewall so you can't expect it to block the outgoing ping. Sandboxie does isolate the eicar file and auto start key so it passes as a sandbox.

Version 1.1 should hopefully use TCP instead of ping for more accurate results.

 

That's an invalid assumption, as a matter of fact.

 

No, it´s not an invalid assumption since it was me that tested it, and I followed the instructions accordingly. Test it yourself then if you think I´ve done it wrongly.

/C.

 

Actually, I've tested avast in a virtual machine too and it failed.
However I haven't / wont do anymore testing. Other results above supplied kindly by gkweb and others.

 

dmenace, Forgive me as I am slightly late to the party here and I am trying to get a handle on the entire concept of your program. In your opening post you state that "this leaktest highlights a new vulnerability that exists when a user shuts down their computer and a program cancels the shutdown." So if I've got it right, you shut down your computer and all of your running processes begin to close. Some or all of your security programs are shutting down. During this time, another program cancels that shutdown (to Windows only), and one by one all of your running processes completely close. Now you are completely exposed to anything that program wants to do on your computer. Also I imagine that all of this could probably be setup as to happen almost instantly so you would never even notice it. That's a remarkable discovery on your part.

It occurs to me that a completely valid question to any supplier of any security product would be; "Does your program stop monitoring at any time during the shutdown process?" If it does, it is potentially vulnerable. Tzuk has stated over at SandboxIE that his program does not stop monitoring during the shutdown process. It seems an easy enough question for the developer to simply answer.

This could almost become a new 'Standard Question' to ask of any security product you are considering. Then, your program kicks in to verify that all are being correct in their answers.

It seems to me, to be more thorough than one by one actually testing programs because in fact the same program might pass or fail depending on whether or not your test was implemented by the user before or after the security program closed. One user might wait 10 seconds before proceeding to step 3 of your test and another might wait 10 minutes. Also in the test the user is controlling the action (via your test) when in reality, it would be a piece of malware that was determining when to issue the 'Cancel Shutdown' order and when to issue the 'Destroy Computer' order.

You are bound to get a bunch of "it failed", "no it didn't", "well, it failed for me too" responses here. The important thing about all of this is that a user whose products passed would be lulled into a false sense of security when in fact his results were part of a random mix.
mitche323

 

I'm not saying the program says it PASSES, but I'm saying that assuming that this means there's some kind of bypass is incorrect.

Let me explain what's going on. The program (SSS) doesn't actually try to execute the file (eicar), all it does is write the file to the hard drive. Now, it all depends on your avast configuration. By default, when avast finds a virus on-write, it only notifies the user (doesn't delete the file automatically). Since its GUI component is already killed, it doesn't display the popup and nothing really happens. However, no malicious code is executed, of course. You can try running the eicar, you'll see that it will be blocked.

Also, if you're not happy with that behavior, you can change it. In the Standard Shield's settings, just enable Silent Mode (-> with anwer "No") and this will cause all malware samples written to the hard drive to be automatically moved to the quarantine.


Cheers
Vlk

 

Nice to know that Avast doesn,t fail in reality.

dmenace! I think it wil be nice to add execution of Eicar test file as well to the leakests.

 

No worries. A long read I guess...

Not all the processes close. If the process is running at a kernel-level with a driver it should still work even without a GUI.

Yes you could ask that question. However obviously every security product has to shutdown eventually so its incorrect to ask "at any time". Maybe ask "while windows is shutting down / booting".
Software firewall vendors are usually most asked this question.

Yes I understand there is this time issue that may affect results. Usually I just wait till all the tray icons vanish when I run the test.

In reality, the "cancel shutdown" would work straight away when test is run as it hooks the windows api. Then when the user shuts down the computer this would be detected. A malicious payload would be executed when for example, it detects that certain processes have shutdown or only windows processes remain.

Yes I agree there could be a false sense of security. So I would have to point that out more - that you have to wait. I suggest if you are confused simply repeat the test (step 3) again after say a minute.

Phew! No more questions!
BTW welcome to wilders mitch.

EDIT: So it makes sense

 

Easy for you to say but this is a one man operation... with no rewards (ie freeware) so obviously I cant do everything:

1) how to detect if eicar successfully executed

>>> have to scan running processes ughh!

Sorry I shouldn't be complaining and I'll do what I can. But geez I am a bit overwhelmed ~ its a leaktest not a security app.

The concept is there. You can always run your own payload.

Edited

 

Spot on dmenace! Thanx for the totally informative and quick answer - I'm caught up now. lol

 

I agree.

I can understand ur situition. Take it easy. U r not bound to do anything. It,s still a smart leaktest as it is.

 

I agree, and it's still beta! Looks like a good candidate for 'DonationWare'

 

All you have to do is check whether the process creation API succeeded or failed (usually, it fails with Access Denied error code if an AV is blocking the file).

 

@vlk: Running in a LUA I did a new test with Avast (latest version), and followed your instructions accordingly regarding Avast:s settings. But it still writes eicar.com to the harddrive and I can also execute the file, it will not be blocked. Am I missing something here, since it doesn´t correlate with your prediction/test?

/C.

 

Haven´t done any testing myself yet, but just wanted to congratulate dmenace with this nice new testing tool, sure looks interesting.

 

Has anyone notified about it on TF forums and CFP forums( I think both failed to this test)?

Thanks

 

Anyone please!

 

Last time I checked, the "firewall" part of the test passed with flying colors. Maybe there was an issue with the registry part. I don't consider the eicar file a valid test just for CFP alone. Perhaps Dmenace or Endangered Frog knows the answers if there has been a new version of the test released.

 

I've tested it. My AV caught the EICAR,
HIPS and FW caught by V3 ( 288 ).

Previous CFP V3 had a missing protected reg key default that caused it not to pass the test.

Under Automatic Startup Group
*\Software\Microsoft\Windows\CurrentVersion\Run*

If it is not there, then add it.

Al

 

Yahoo has sadly shutdown Zeroday Software's website because apparently any linking to external websites is banned.

As a result if anyone still wants to download System Shutdown Simulator please get it here:

http://www.firewallleaktester.com/mirror/zeroday_software/sss.htm

I've paused development on SSS as I am busy at the moment. May later implement TCP instead of ping. But its pretty much complete and bug free now anyway.

The latest release is 1.0.21 The website issue will be fixed at a later date.