Site to test HIPS ?

[acr1965]

Is there a clean site available to test my HIPS program? I currently have DSA (Dynamic Security Agent) installed and would like to see how it performs. Does anyone know how DSA stacks up against SSM (System Safety Monitor) or Spyware Terminator HIPS program?

[Jarmo P]

You are totally wrong.

HIPS is not something to test about malware.
There are HIPS's like sandboxies, those ones I recommend to you.
Classical HIPS like SSM, they are not either so much testable.

So forget hips testing sites.

[zopzop]

i don't know about any specific sites that test HIPS but you can try the following :

1) advanced process termination test
http://www.diamondcs.com.au/index.php?page=apt

2) the simple process termination test from SSM
http://www.syssafety.com/leaktests.html

3) the keylogger test from SSM
http://www.syssafety.com/leaktests.html

4) morgud's threat simulator
http://www.morgud.com/interests/security/dfk-threat-simulator-v2.asp

5) gentlesecurity's threat test
http://gentlesecurity.com/demo.html

6) spycar's browser hijack tests
http://spycar.org

7) ghostsecurity's registry tests
http://ghostsecurity.com/registrytest/

8 ) martin's keylogger (see if your HIPS program can stop it from recording keystrokes)
http://www.winsite.com/bin/Info?26000000037599

hope those help :D

and if you're feeling super brave, test whatever HIPS you have vs the killdisk virus. but be warned if your HIPS fails, you'll need to reformat your hard drive and fix your MBR.

[acr1965]

-{ Quote: "i don't know about any specific sites that test HIPS but you can try the following :

1) advanced process termination test
http://www.diamondcs.com.au/index.php?page=apt

2) the simple process termination test from SSM
http://www.syssafety.com/leaktests.html

3) the keylogger test from SSM
http://www.syssafety.com/leaktests.html

4) morgud's threat simulator
http://www.morgud.com/interests/security/dfk-threat-simulator-v2.asp

5) gentlesecurity's threat test
http://gentlesecurity.com/demo.html

6) spycar's browser hijack tests
http://spycar.org

7) ghostsecurity's registry tests
http://ghostsecurity.com/registrytest/

8 ) martin's keylogger (see if your HIPS program can stop it from recording keystrokes)
http://www.winsite.com/bin/Info?26000000037599

hope those help :D

and if you're feeling super brave, test whatever HIPS you have vs the killdisk virus. but be warned if your HIPS fails, you'll need to reformat your hard drive and fix your MBR." }-


That should keep me busy for a while. Thanks!!

[Rmus]

Aren't HIPS products supposed to include some type of protection against running executables not already installed?

Since most malware attempts to install a malicious executable, you can test for this protection yourself.

1) go to a download site and see if your program will block downloading an executable

2) try to run an installation CD not previously installed - your program should block the running of the setup.exe on the CD

3) or, have a friend put a program you don't have on a CD, and see if your HIPS will block it's running

These being proven, you can feel secure that unauthorized installation of malware executables would be blocked, and that this part of your HIPS is working OK.

regards,

-rich

[cprtech]

-{ Quote: "1) go to a download site and see if your program will block downloading an executable" }-

Do you mean the antivirus program? AFAIK, a HIPS will not stop the downloading of an executable.

-{ Quote: "Aren't HIPS products supposed to include some type of protection against running executables not already installed?" }-

Otherwise I like the fact you touch on this point, something no one seems to ever mention. We allow the executables of these termination/leak tests to launch after being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so ::)

[Rmus]

-{ Quote: "Otherwise I like the fact you touch on this point, something no one seems to ever mention. We allow the executables of these termination/leak tests to launch after being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so ::)" }-Don't be sorry - it needs to be repeated.

Last year I posted to a firewall leaktest thread that my Kerio 2 passed all of the tests. How can that be, I was asked, because the results on the test site showed differently.

Because the test executable was blocked from running. The response was that I wasn't playing fair.

OK, maybe not from their standpoint. But if the test executable simulated a trojan executable, and your security blocks it, why isn't that valid?

In the case of the firewall leaktest, why should I have to disable my security to let a test run that would prove my firewall couldn't do what it wasn't intended to do in the first place?

OK, interpolate HIPS into this scenario, as you have done, and it is a very valid question to ask.

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[cprtech]

-{ Quote: "Don't be sorry - it needs to be repeated.

Last year I posted to a firewall leaktest thread that my Kerio 2 passed all of the tests. How can that be, I was asked, because the results on the test site showed differently.

Because the test executable was blocked from running. The response was that I wasn't playing fair.

OK, maybe not from their standpoint. But if the test executable simulated a trojan executable, and your security blocks it, why isn't that valid?

In the case of the firewall leaktest, why should I have to disable my security to let a test run that would prove my firewall couldn't do what it wasn't intended to do in the first place?

OK, interpolate HIPS into this scenario, as you have done, and it is a very valid question to ask.

regards,

-rich
" }-

Well said!! :thumb: After all, if someone mistakenly allows a suspicious file to run, then there is no point in them using a HIPS in the first place. Some will argue this but I stand by my convictions and I see you think along the same lines as I :)

[ggf31416]

-{ Quote: " After all, if someone mistakenly allows a suspicious file to run, then there is no point in them using a HIPS in the first place." }-

If someone allows a dangerous file to run, then a good HIPS like SSM or ProSecurity should give additionals prompts if it attemps to do something suspicious.

[cprtech]

-{ Quote: "If someone allows a dangerous file to run, then a good HIPS like SSM or ProSecurity should give additionals prompts if it attemps to do something suspicious." }-

Yes, I agree and would hope that to be the case, and in hindsight my comment in my previous post is probably unfair. Slipping up at the wrong time allowing a dangerous file to run could happen to the best of us, so a HIPS that can catch further actions on such a file is obviously desirable. Still, if it looks suspicious in the first place, then by all means it should not be allowed to run except, of course, for testing purposes as the OP is looking to do. This I am all for because it is a valuable learning experience.

[acr1965]

Well here are some results of the tests. I was unable (or unwilling) to run some of the tests but I ran the Spycar Autostart and IE Config Change tests, the Advanced Process Termination test from diamondcs and the Ghost Security Regtest. I tested both DSA and Spyware Terminator (three settings for the Spycar test with ST: Realtime Shield enabled/HIPS disabled, Realtime Shield disabled/HIPS enabled and Realtime Shield enabled/HIPS enabled). Both DSA and ST were tested seperately.

My computer is a Hewlett-Packard Pavilion, Windows XP w/SP2, P4, 2.4 GHz, 1 GB of RAM.

The results for both DSA and ST for the diamondcs APT test were that they both passed 1-9 and 11 of the APT. Test 10 and 12 were graded as "unable to test." As for the Suspend Test 1 and 2, both passed. Kernel Kill Test 1 and 2 both apparently passed. Crash Test 1 and 2 were not tested.

As for the Spycar, DSA passed every test. ST did not. Here is a list of the six "Autostart Tests":

1. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\Run

2. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce

3. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnceEx

4. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\Run

5. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnce

6. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnceEx

As said above, DSA passed this series of tests. ST tested with Realtime Shield disabled/HIPS enabled passed test #5 but failed all others. With Realtime Shield enabled/HIPS disabled ST passed all six tests. With Realtime Shield and HIPS enabled ST passed all six tests.



The Internet Config Change Tests was performed in a similar fashion. Here is a list of the IECC Tests:

1. change your default home page in IE

2. lockout users from changing the default home page in IE

3. change your default search page in IE

4. remove the Advanced Tab in your IE Internet Options Screen

5. remove the Programs Tab in your IE Internet Options Screen

6. remove the Connections Tab in your IE Internet Options Screen

7. remove the Content Tab in your IE Internet Options Screen

8. remove the Privacy Tab in your IE Internet Options Screen

9. remove the Security Tab in your IE Internet Options Screen

10. remove the General Tab in your IE Internet Options Screen

DSA passed all the IECC tests. ST was tested in the same fashion as with the Autostart Tests: Realtime Shield enabled/HIPS disabled, Realtime Shield disabled/HIPS enabled and Realtime Shield enabled/HIPS enabled. All had similar results in that ST passed test 1 (make Spycar try to change your default home page in IE) but failed all others. In the results section this test was listed as #9 for some reason.

As for the Ghost Security Regtest, neither passed although ST performed better than DSA. ST was tested as above with the various enablings of Realtime Shield and HIPS. But whatever the settings ST had the same results, it allowed REGtest to pass the first attempt but blocked all others. DSA allowed Regtest to do as it wished despite the pop-up warning to block which was clicked. As for test 2, both had "fail" messages after restart.

Well that's it. I think for a limited test sample it would be better to keep ST's Realtime Shield activated and deactivate its HIPS. In place of ST HIPS it appears better to run DSA. Although neither passed the REGtest, it appears ST provides some protection, at least more than DSA.

But these results are strictly on my machine. I feel comfortable with them but anyone else should use their own judgement or run their own tests. Also, there are several HIPS programs one may consider besides DSA.

[Mrkvonic]

-{ Quote: "Do you mean the antivirus program? AFAIK, a HIPS will not stop the downloading of an executable.



Otherwise I like the fact you touch on this point, something no one seems to ever mention. We allow the executables of these termination/leak tests to launch after being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so ::)" }-

Hello,
No one seems to mention!!! HELLOOOOO! I've been repeating it since 1844!
Mrk

[ggf31416]

-{ Quote: "
The results for both DSA and ST for the diamondcs APT test were that they both passed 1-9 and 11 of the APT. Test 10 and 12 were graded as "unable to test." As for the Suspend Test 1 and 2, both passed. Kernel Kill Test 1 and 2 both apparently passed. Crash Test 1 and 2 were not tested.
" }-

What process do you terminate for ST?

My results for ST:

I ran a full scan in order to activate the HIPS. I copied the SpyCar test to a new folder and the APT executable to APT2.exe before starting the test. I allowed APT to run and tried to terminated spywareterminatorshield.exe. If termination appears to be sucessfully I ran the Hosts test and TowTruck.exe to check if the protection was terminated and not only the GUI.
ST version was 1.7.0.899. Expert settings with HIPS and all shield enabled.
Freeze means ST didn't give any prompts but the Hosts test could not run (without any error message).
Yes means the ST process was terminated and the hosts change was allowed
No means the ST process was NOT terminated

Suspend 1 Freeze
Suspend 2 Freeze

Kill 1 No
Kill 2 Yes
Kill 3 Yes
Kill 4 Yes
Kill 5 Yes
Kill 6 Yes
Kill 7 No
Kill 8 No*
Kill 9 Yes
Kill 10 No
Kill 11 Yes
Kill 12 No

Kernel 1 Yes
Kernel 2 No

Crash 1 Yes**
Crash 2 Freeze

* ST asked whether I wanted to allow the APT process to start. I if clicked Yes the ST process would be terminated.
** APT reports that the termination was unsuccefuly but the hosts test runs without any prompt

[djg05]

-{ Quote: "after[/i] being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so ::)" }-

Whilst I agree with you in day to day running, I do feel that when you are trying out these tests you want to know how strong they truly are and what it takes to break through the HIPS, so that if you make a wrong decision you still have some protection. There is no knowing if one of these exploits is buried in some software that you thought was trustworthy or they get on to your m/c through some other means.

[Ice_Czar]

-{ Quote: "

4) morgud's threat simulator
http://www.morgud.com/interests/security/dfk-threat-simulator-v2.asp

" }-
Ive always valued destructive testing :P
-{ Quote: "DO NOT run this simulator if:

1. You think "the registry" refers to bridal gifts.
2. You do not own the computer you are planning to install it on.
3. You do not mind reformatting the machine you are installing it on (in case it crashes).
4. You think "ReadMe" files are pointless and never read them. " }-

the scenario that test proposes is all too plausible (and neatly illustrates the combination of basic social engineering with an advanced level of misdirection\impersonation that could fool even a paranoid with a layered defense.)

How old is that simulator?

[Mrkvonic]

Hello,

A few months. It's not a bad test except ... you need to:

1. download a file.
2. execute it.

=

1. **** the gun.
2. Shoot yourself in the head.

Mrk

[cprtech]

-{ Quote: "Hello,
No one seems to mention!!! HELLOOOOO! I've been repeating it since 1844!
Mrk" }-

LOL! Okay, sorry Mrkvonic, I guess I missed your comments. You have my sincerest apologies ;) BTW, I like your style. You post a lot of cool information and put it across in a very intriguing manner.

[Ice_Czar]

-{ Quote: "Hello,

A few months. It's not a bad test except ... you need to:

1. download a file.
2. execute it.

=

1. c0ck the gun.
2. Shoot yourself in the head.

Mrk" }-

yes but...
the scenario that went with that that tool was highly probable, I'll make one change to it. Rather than it being Judy's friend Carl, its your client Paul, he gets to make a decision about adopting a $35,000 service contract and your kissing his ass to smooth the deal, He thinks its hilarious and expects you to comment. Now you are motivated to c0ck the gun and shoot yourself in the head. And the question becomes not can it subvert your security, but rather will it go unnoticed?

take a chance and reimage just to be sure?
decline and show him your tinfoil underwear?
try it and know youve just been buggered?

[Rmus]

-{ Quote: " Now you are motivated to c0ck the gun and shoot yourself in the head. And the question becomes not can it subvert your security, but rather will it go unnoticed?" }-This is from the v.1 test last year:

dfk test (http://www.urs2.net/rsj/computing/tests/dfk/)

If I noticed anything like that, I would just reboot to good previous state.

Is this what you are referring to?

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[Mrkvonic]

Hello,

Ice, indeed one of the greatest problems I'm facing is .docs and .ppts from various clients and colleagues, who have no clue what they're doing.

I solve the problem the following way:

Open the files in Open Office using Linux hihihihihihi.

Open the files using Open Office / MS Office in VM.

Open the files on the work computer and let the IT worry.

NEVER open files I'm not absolutely sure are ok - which limits it down to about 2-3 people.

BTW, extra sophisticated malware in company docs means clear corporate saborage. Highly unlikely you'll receive something like that in a friendly exchange.

I don't trust scanners to do the work for me - that includes HIPS.

Mrk

[lucas1985]

-{ Quote: "I solve the problem the following way:

Upload the files to VirusTotal or Jotti

Open the files in Open Office using Linux hihihihihihi.

Open the files using Open Office / MS Office in VM.

" }-
:thumb:

[Long View]

-{ Quote: "one of the greatest problems I'm facing is .docs and .ppts from various clients and colleagues, who have no clue what they're doing.


NEVER open files I'm not absolutely sure are ok - which limits it down to about 2-3 people.


I don't trust scanners to do the work for me - that includes HIPS.

Mrk" }-

I have no problem with docs that come from people I don't know - in one way or another they are destroyed. Problem with destroying mail from clients is ..... well they might not understand.

Its the last point that concerns me - "that includes HIPS". My only real experience is with ProSecurity. Having spent a number of weeks working with the program - it appears that I now have a number of rules allowing my programs a certain degree of freedom. If a client now sends me what appears to be a legitimate email with an attachment which I open my hope is that when "it" tries to do something I will receive a warning. I will then have the option to click block bit will more likely just restore the previous days Acronis system image. Am I then simply wasting my time having a HIPS program ?

[Rmus]

-{ Quote: "Upload the files to VirusTotal or Jotti" }-I wouldn't stake my reputation on that:

Zeroday Scan (http://www.urs2.net/rsj/computing/tests/wmf_zeroday)


regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[acr1965]

-{ Quote: "What process do you terminate for ST?

My results for ST:

I ran a full scan in order to activate the HIPS. I copied the SpyCar test to a new folder and the APT executable to APT2.exe before starting the test. I allowed APT to run and tried to terminated spywareterminatorshield.exe. If termination appears to be sucessfully I ran the Hosts test and TowTruck.exe to check if the protection was terminated and not only the GUI.
ST version was 1.7.0.899. Expert settings with HIPS and all shield enabled.
Freeze means ST didn't give any prompts but the Hosts test could not run (without any error message).
Yes means the ST process was terminated and the hosts change was allowed
No means the ST process was NOT terminated

Suspend 1 Freeze
Suspend 2 Freeze

Kill 1 No
Kill 2 Yes
Kill 3 Yes
Kill 4 Yes
Kill 5 Yes
Kill 6 Yes
Kill 7 No
Kill 8 No*
Kill 9 Yes
Kill 10 No
Kill 11 Yes
Kill 12 No

Kernel 1 Yes
Kernel 2 No

Crash 1 Yes**
Crash 2 Freeze

* ST asked whether I wanted to allow the APT process to start. I if clicked Yes the ST process would be terminated.
** APT reports that the termination was unsuccefuly but the hosts test runs without any prompt" }-

I re-ran the APT and trying to terminate STShield.exe and had similar results as your test. I only ran the Kill 1-12 tests and 10, 12 were "unable to test."

Yes means ST was terminated, No means ST was not terminated:

Kill 1- No
Kill 2- Yes
Kill 3- Yes
Kill 4- Yes
Kill 5- Yes
Kill 6- Yes
Kill 7- No
Kill 8- Yes
Kill 9- Yes
Kill 10- unable to test
Kill 11- Yes
Kill 12- unable to test

I received the same results with only Realtime Protection and HIPS disabled.

[Rmus]

-{ Quote: "I have no problem with docs that come from people I don't know - in one way or another they are destroyed. Problem with destroying mail from clients is ..... well they might not understand." }-I don't know why they have to be destroyed.

Students - my 'clients' - at college learn to use MSWord - most will end up using it in business (I'm not saying that's good or bad - it's just the way it is at the moment)

I received 30+ docs per week at home. It's easy to configure MIME types in your email client to open *.doc with another application that is not vulnerable to such exploits.

It's just not a huge problem to solve if approached logically, and there are many solutions.

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[lucas1985]

-{ Quote: "I wouldn't stake my reputation on that:

Zeroday Scan (http://www.urs2.net/rsj/computing/tests/wmf_zeroday)

" }-
If Virus Total or Jotti deliver a clean report about the uploaded file that doesn´t mean I will trust the file.

1-Upload the files to VirusTotal or Jotti.
*If the report says infected/suspicious, the file is rejected.
*If clean, follow the next step.

2-Open the files in Open Office using Linux.
*If strange behaviors are detected, the file is rejected.
*If clean, follow the next step.

3-Open the files using Open Office / MS Office in VM.
*If strange behaviors are detected, the file is rejected.
*If clean, the file is opened/executed in a virtualized enviroment (GeSWall).

-{ Quote: "
It's just not a huge problem to solve if approached logically, and there are many solutions.
" }-
:thumb:

[Rmus]

-{ Quote: "Hello,

Ice, indeed one of the greatest problems I'm facing is .docs and .ppts from various clients and colleagues, who have no clue what they're doing.k" }-Hi Mrk,

-{ Quote: "Non-Affected Software:
• Microsoft PowerPoint 2003 Viewer" }-

See:

Microsoft Security Bulletin MS06-060
http://www.microsoft.com/technet/security/Bulletin/MS06-062.mspx


regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[Rmus]

-{ Quote: "If Virus Total or Jotti deliver a clean report about the uploaded file that doesn´t mean I will trust the file.

3-Open the files using Open Office / MS Office in VM.
*If strange behaviors are detected, the file is rejected.
*If clean, the file is opened/executed in a virtualized enviroment (GeSWall)." }-

Hi lucas,

Why not just save time and go directly to your step 3? :)

It sounds like an ideal solution.

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[lucas1985]

Because step 1 is almost always very fast and step 2 and 3 are quickly made (switch between WMs).
Perhaps I´m a little paranoid ;D

[Rmus]

Well, we've ventured away a little bit from the original topic, and I was going to lead into some other tests and sites, but I see the original poster wants "clean" sites.

One exploit I've searched in vain for is a real MSWord exploit file. Anyone know of one?

All that I've seen is the PoC referenced a while back at sans.org. Has anyone tested that with a HIPS product?

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[acr1965]

Maybe it would be for a new thread, with the DSA and Spyware Terminator results included, but it would be interesting if others who run HIPS programs (SSM, Prosecurity, etc) to test against the Spycar, APT and RegTest to see how they do. So far DSA passes all tests in Spycar and APT but performs poorly in the RegTest. Spyware Terminator with Realtime Shield and HIPS activated performs well in the Spycar Autostart tests but poorly in the IE Config Change. Also ST performs exceptionally poor against the diamondcs.com APT but has some success in the Ghost Security RegTest (although Ghost claims it failed).

[Devil's Advocate]

-{ Quote: "
Otherwise I like the fact you touch on this point, something no one seems to ever mention. We allow the executables of these termination/leak tests to launch after being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so ::)" }-

It's somewhat of a truism that if the exe doesn't start it can't do anything.
The assumption however is that you already know the executable is dangerous, this is not always the case.

What I disagree is people who say "My setup is immune to problem X", and their proof is that they click on the exe that tests problem X, HIPS prompts that X is starting and don't even allow the exe to start.

But this is silly, because since you already know it was dangerous, you might as well not click and start the exe in the first place. What does your "test" tell you except that the HIPS function of alerting on unknown processes is working?

But the test was not a test of its ability to evade what Rmus calls anti-executable , so you learn nothing by not even allowing the test to start.

Also in practise you can never know for sure if the exe you allowed to run is really a trojan horse, that's where other aspects of HIPS comes in.

-{ Quote: " After all, if someone mistakenly allows a suspicious file to run, then there is no point in them using a HIPS in the first place. " }-

I think most people would disagree with this statement. You (and Rmus I think) seem to think HIPS is simply the function of whitelisting processes and blocking unknown. That's an overly limited view.

You might be happy with just that, but if HIPS was simply that it would be of very limited use to the average person, Who needs someway to determine if something they are running or going to run is likely to be malicious.

[cprtech]

-{ Quote: "It's somewhat of a truism that if the exe doesn't start it can't do anything.
The assumption however is that you already know the executable is dangerous, this is not always the case.

What I disagree is people who say "My setup is immune to problem X", and their proof is that they click on the exe that tests problem X, HIPS prompts that X is starting and don't even allow the exe to start.

But this is silly, because since you already know it was dangerous, you might as well not click and start the exe in the first place. What does your "test" tell you except that the HIPS function of alerting on unknown processes is working?

But the test was not a test of its ability to evade what Rmus calls anti-executable , so you learn nothing by not even allowing the test to start.

Also in practise you can never know for sure if the exe you allowed to run is really a trojan horse, that's where other aspects of HIPS comes in.



I think most people would disagree with this statement. You (and Rmus I think) seem to think HIPS is simply the function of whitelisting processes and blocking unknown. That's an overly limited view.

You might be happy with just that, but if HIPS was simply that it would be of very limited use to the average person, Who needs someway to determine if something they are running or going to run is likely to be malicious." }-

Excellent point of view. My post #10 more or less concurs with your statements. However, you can be pretty much 100% sure the executable is trusted if you obtain it from a known, trusted source. I would not download, for example, mp10setup.exe from Warez 'R Us, nor would I accept it as an email attachment from anyone. Rather, I would download it from Microsoft. That goes for any and all software I download. I would say those who open email attachments willy-nilly and routinely download and run cracked software are at greatest risk.

[Mrkvonic]

Hello,

lucas, I see we rather quite agree on methodology ...

Rmus, I'm not worried about exploits in Office per se, because they are many alternatives, I'm talking about general approach to files from people whom you must interact - mainly docs, ppts and pdfs. OO / Linux is a fine approach, especially for one using them already! Skipping over 2 to 3 is viable, but then VM can also be installed on Linux, which makes it even more fun!

Mrk

[Ice_Czar]

A few years ago I adopted employing multiple task specific computers
the workstation wasnt even hooked full time to my LAN and was a www virgin.

I relied on W2K hardening, policies, AV, HIPS, and Filechecker for my lesser boxes (900MHz > 1.2GHz) as surfboards.

But with virtualization\sandboxes Im rethinking the whole game, its a bit much for those old boxes, but I deflowered my workstation about a year ago and in the last few months have started to employ VMware\Sandboxie.
But its pretty obvious Im not thinking in that groove yet. ::)

Ive also acquired an new standalone 3.06GHz surfboard though it could stand having its RAM upped. Looks like its time to find other duties for the old hardware.

[Notok]

1. You download and run the leaktest under the assumption that it simulates malicious actions and therefore premeditate your actions: click "Block" to any and every alert you see. This is not, in any way, representative of a real life situation. There is no guarantee that you'll click "Block" to anything, especially not anything meaningful.

2. Behavior blockers rely entirely on you, your knowledge, and your decisions. This is something that keeps getting overlooked or taken for granted as not applying to anyone here, but is something I think should really be seriously considered. Would you apply for a malware analyst position with your current level of confidence in your ability to pick out malware from legitimate files? Just becuase you know what should be running on your system doesn't mean that a new file isn't legitimate - even if unexpected. With that in mind, the only way to truely "test" the effectiveness of your behavior blocker would be to test your own ability to distinguish legitimate files from malicious ones, and if you can distinguish a malicious action from a legitimate one even when they are exactly the same action being performed by exactly the same process. Also consider the "chicken little effect" - how many false positives do you tolerate before you stop taking those prompts seriously?

3. Such tests perform a single action. An application that takes only one single action that a behavior blocker may or may not monitor is more representative of a legitimate application than a malicious one. These actions are legitimate resources offered by the operating system for legitimate applications. They are mostly documented and supported programming functions available for any and all software to use. Single actions will also give you no way of predicting whether the behavior blocker will block malware, since malware may not utilize monitored actions.

4. Actual malware takes many many many actions to infect a system. Just because a program does not stop a single action does not mean that it's not effective. The software you are using may address the malware problem at a completely different point than what the leaktest (or other test file) is meant to test. In such a case, calling it a "fail" would be not only meaningless but naive to the point of absurdity. An example would be a script filter. If you download pretend malware and say it "fails" [to protect you] because it didn't alert to the intended action, this would be completely missing the point that the program is meant to stop the file from downloading, not what programs do that are already downloaded.

5. Leaktests and test files are mostly made by commercial entities for scare tactic marketing. These tests are specifically made to show off a single (or small number of) feature(s) unique to that vendor's product. Read that again - that file is made specifically to bypass most or all other products as a way of scaring you into thinking that you need to give that vendor your money to be protected. This also goes into number 3. Just because a product doesn't contain a specific feature does not render that product ineffective within it's scope of protection. What is actually more important is to know what that product does, how it does it, how that relates to malware, what kind of malware that relates to, and how prevalent that malware is. Without knowing those things, you will have no way of knowing how that program could protect you, and certainly not how "well" it can protect you in any meaningful way. Also ask yourself whether you really want every product to incorporate every other product's features. Not only would that lead to bloat and a maddening flurry of prompts every time you try to do anything, it would quickly render them all useless as malware writers simply avoid using those techniques, or at least use techniques to evade them.

6. The real tests of those features are not contained in commercially made leaktests and other test files. The real test of those application's functions are performed by security researchers that know programming. I don't recall ever seeing a test file that uses symbolic links, for example, to see if your behavior blocker recognizes them. These are a greater threat, since not everyone on these forums use them as buzzwords. Better to do some reading up.

7. The point of behavior blockers is to block new malware that is unknown to your other security software. How do you calculate the effectiveness of malware using new evasion techniques that haven't been discovered yet? How many people will be infected before the application is updated, or will it even be updated (is the problem within the scope of protection for that application)?

8. Even if you do block each and every action that the behavior blocker monitors, how many other actions would the malware take? No behavior blocker could block 100% of them, so what are you left with? Considering that there's virtually no end to the possibilities, you have to consider that a new technique may be employed to leak your data, so how does the behavior blocker account for that, how much additional damage can be done?

9. Building a defense generally consists of assuming that any given application can and will be bypassed so that you can narrow down the potential damage to an acceptable level, knowing that 100% security is not possible. How will you react if you do end up infected, and what role does the application play?

I've posted most of that before, but there's a few new ones in there :) I'm sure I could think of more, but hopefully that should be a good enough start. Answering those questions for yourself will give you more answers than test files ever will. Yes, it's much easier to just run a file and receive a prompt that says either "Pass" or "Fail", but for those to be meaningful in any way depends on a whole lot of things and be performed in a meaningful context (which I have never seen). If you don't know and understand what those things are, then you're easy prey for marketing goons- any company can come up with something that will tell you that you're insecure unless you are using their products; they're all over the web by both legitimate and illegitimate vendors. As far as I'm concerned they're all the same since they all want you to see the answer in black and white terms: "pass vs fail" or "secure vs insecure". Even if you use live malware to test, most of the above still applies. You're still running it knowing full well what's going to happen, it's still out of context, there's still additional actions that can be taken, there's still the question of how common that malware's techniques are, there's still the question of yet unknown techniques, etc etc etc. The only way to assess the effectiveness of a behavior blocker is to know the facts.

Knowledge is the only true way to secure your system beyond a strong but basic set of applications. Armed with knowledge, you won't need any more.

[Rmus]

-{ Quote: "You (and Rmus I think) seem to think HIPS is simply the function of whitelisting processes and blocking unknown. That's an overly limited view." }-I was careful to state in my post #5,

-{ Quote: " These being proven, you can feel secure that unauthorized installation of malware executables would be blocked, and that this part of your HIPS is working OK." }-

[Rmus]

acr1965: People have tested sites until they are blue in the face, but they still have a lingering feeling that they are missing something.

In your post #31, your results illustrate this. How will you make your decision? I don't envy you.

Notok: very nice write-up.

Regarding 1.and 5. (Leaktests):

Robin Keir, author of Firehole (dll injection using the Win32 function SetWindowsHookEx)
http://keir.net/firehole.html

-{ Quote: " I decided to write a proof of concept tool with the aim of obtaining a covert outbound network connection that would go undetected by the firewall.

I want to reiterate what I said right at the start of this article. This, and all similar techniques rely on a rogue program getting onto your system and executing. If you can stop this then you are safe." }-Two logical questions arise:1) How do you stop this from "getting onto your system and executing?"

2) What if it does get in?

Taking the second question, "What if": I'm reminded of two visitors to a canyon overlook. One observes that the other has climbed over the guard rail to get closer to the edge for a look. Upon climbing back, he asks the first, "Where is your parachute?"

"Why do I need a parachute?"

"What if you fall over the edge."

"I'm not going to go on the other side of the guard rail."

To carry over into the computer world, "Where is your ______ (fill in the blank) to stop the dll injection?"

"I'm not going to let anything install that would do that."

For the first question, How can I prevent that?

There are two common ways firehole (or to update: any current trojan) could install.

1) surreptitiously, via remote code execution (iframe, .wmf file, etc): Easily blocked

Here is a current one (http://isc.sans.org/diary.php?storyid=1983)

2) piggy-back as part of another program. I realize I'm in a minority here, but being aware of what I install has been my guide.

I'm asked, "How can you be sure?"

"I'm just sure."

It's curious, I've gotten a number of PMs during the past year from people telling me how they downloaded/installed/gotten infected by this or that, -- By PM because they didn't want to discuss how it happened on the forum -- and what if that occurred to me? I respond, that it wasn't something I would ever download/install. So why should I worry?

To quote again my favorite quip,"Just because Mr. Smith's shoes are too tight, why should my feet hurt?"


In every case of a reported malware infection at Wilders and DSLR where I've been able to ascertain how it happened, (not always forthcoming from the victim) it's a situation that could never have happened to me or anyone that I've been involved with in computing.

It's been said on these forums that people need a way to determine if something they are going to run is likely to be malicious.

If this refers to some type of program, then it's a lost cause because there has yet to be developed a reliable, trustworthy application to do that in all cases.

I submit that one's own good sense and careful habits are just as reliable, as I have proven for myself and others over many years. And as some here (in the minority) have also stated.

How? It has to do with your comment about knowledge, which is so pertinent. So, How do you make a computer user "knowledgeable?" And what does this "knowledge" consist of?

Big questions, with no simple answers, because you are dealing with states of mind.

When Bruce Schneier's book, "Beyond Fear," was released in 2003, I made a note from a review:

-{ Quote: " Schneier's practical approach to problem-solving is a refreshing antidote to today's doomsday pessimism and anxiety." }-I no longer recommend security setups without seeing first hand the system and how it is used - the user's computing routine. Because I'm convinced that the first step in a "practical approach to problem-solving" is to understand what the problem is - *problem* being, addressing the user's security needs - beginning with the user's state of mind: fears, anxieties - most of which are influenced by the media.

I love working with the complete beginner: She/he has no bad habits, misconceptions (or fears) to unlearn. I've maintained contact over the years with several home users whom I've helped - who started out as beginners. Ask them what these terms mean: HIPS, rootkit, hook. They would respond, "Huh?"

Are they -- just average home users -- not "with it" because they don't know these terms?

Yet they've never had a virus (that term they *would* recognize, since it's so commonly used in the general media).

I realize that it's difficult to imagine that everyone could have "hands on" help. But as I've advocated in the past, why not "adopt" a user? If not a beginner, one who perhaps has had a bad experience with a virus, and who agrees to listen to you.

There will always be those who will climb over the guard rail, but they should be left to their own devices, search for their parachutes, and reap their own rewards.

I'm more interested in those who really would appreciate some help with the basics. Certainly each person here knows someone like that.

Develop a teaching plan: "introduction to computer security." Where would you start? Make a list of products people should install?

Or thinking through how you would teach someone to be a safe, knowledgeable, and secure computer user, and from that basis, developing a layer of applications that meet her/his needs. (Many ways to approach this, but that's for another discussion)

Just think, that if everyone here "adopted" just one person - how many more knowledgeable computer users there would be. Time consuming - yes, but we can always find time to help someone. It's not only been rewarding in knowing I've helped someone, but I've always learned a lot from the experiences.

How much more useful than jumping on the doomsday wagon, which has already started its forward march into 2007 with its lists ad infinitum of predictions of worst case scenarios.

Take a stand: step out from the crowd and be separate. Help do something about it!

regards,

-rich

________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[Mrkvonic]

Hello,
Rmus, nicely said.
Mrk

[Ice_Czar]

elaboration of the points

-{ Quote: "Would you apply for a malware analyst position with your current level of confidence in your ability to pick out malware from legitimate files? Just becuase you know what should be running on your system doesn't mean that a new file isn't legitimate" }-

no, but then as mentioned I do know which are the legitimate aps and anything new is easily researched, its always been a matter of trust and experimentation, which is why there is a box specifically to try things out. What trips most people up is ignorance of an infection vector and experimenting within thier core systems rather than their disposable installs. Even if you don't run multiple boxes, sandboxes or virtualization its what 6GB for a dual boot? Which likely would break most malware attempting to infect the second install.

-{ Quote: "9. Building a defense generally consists of assuming that any given application can and will be bypassed so that you can narrow down the potential damage to an acceptable level, knowing that 100% security is not possible. How will you react if you do end up infected, and what role does the application play?" }-

reimage to a known secure state \ tripwire
(or at least a state with a high confidence level) ;D

[gkweb]

Hello,

-{ Quote: "
Leaktests and test files are mostly made by commercial entities for scare tactic marketing
" }-
(emphasis is mine).

This word "mostly" is the key word here. Personally I'm not talking about leaktests to scare people, as it's written (http://www.firewallleaktester.com/faq.htm#04) in my website FAQ, but instead to help everyone testing and adjust it's security (often "failing" a leaktest is simply a matter of incorrect software or OS configuration).

I try to do my best to highlight the fact that leaktests are meant (from my point of view) to help you securing your computer and see by yourself that none product can protect against everything. Leaktests are often associated to evil marketing scare tactic, I'd like to say otherwise as they are also used with good intent in mind.

That being said, I woud like to recall one page I've done to help people securing their computer, as well as a PDF document to secure Windows :
http://www.firewallleaktester.com/advices.htm
http://www.firewallleaktester.com/docs/Securing%20Windows.pdf

I cannot prevent you to still hate leaktests or people creating them, but at least I can show you good things coming from them :)

While I'm here, happy new year ;)

Regards,
gkweb.

[Paul Wilders]

-{ Quote: "While I'm here, happy new year" }-

Happy New Year to you as well, Guillaume ;) now, back on topic...

regards,

paul

[Ice_Czar]

-{ Quote: "

Develop a teaching plan: "introduction to computer security." Where would you start? Make a list of products people should install?

Or thinking through how you would teach someone to be a safe, knowledgeable, and secure computer user, and from that basis, developing a layer of applications that meet her/his needs. (Many ways to approach this, but that's for another discussion)

" }-

1. I generally supply a list of classes of applications with examples

2. Generally refer them to this tutorial (http://www.microsoft.com/technet/security/guidance/serversecurity/avdind_0.mspx)



-{ Quote: "
________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier" }-

well one way allows you to make a lot of money and manipulate society
the other allows you address the problem. and of course isnt limited (http://www.searchlores.org/realicra/realicra.htm) to computer security

of course when the problem is itself ignorance, it may be the first step towards a solution. It does however require a transition from the sky is falling to rational assessment. But when the sky is falling message is repeated again and again its the boy that cried wolf, and looses all legitimacy

[Mrkvonic]

Hello,

Very well said, gkweb.

... Do not rely exclusively on your software firewall for protection from these kinds of vulnerabilies.. Try to catch the threats before they hurt ...


And one more thing that I personally love:

... There are also software-based NAT routers like Microsoft's Internet Connection Sharing and Sygate's SHN that do the same thing if installed on a home computer that serves as a gateway to a small number of other PCs on a home LAN ...

Cheers,
Mrk

[Notok]

-{ Quote: "This word "mostly" is the key word here." }-Indeed, and your site is actually one that I consider to be in proper context with leaktests, which I had forgotten about while writing my post... that is as long as they read the other stuff, which you have now taken steps to ensure they do :) So no disrespect to your efforts in any way, what I don't like is people judging the effectiveness of behavior blockers with various demo trojans, mostly made by other commercial entities, without consideration for what the program actually does.

Ice_Czar: Indeed, and thanks :)

Rmus: Hehe, the example is a good one. I definitely agree with your points.

[Devil's Advocate]

-{ Quote: "I was careful to state in my post #5-{ Quote: "I was careful to state in my post #5,
Quote:These being proven, you can feel secure that unauthorized installation of malware executables would be blocked, and that this part of your HIPS is working OK.
" }-
" }-

But you don't 'prove' that part of your HIPS is working okay, by clicking on some random exe and clicking no..... :)

If you want to argue that leak tests or test demos are pointless, you could argue as Notok as argued that you wouldn't know it's malicious so you wouldn't block it.

The irony is, this argument is actually the same one that can be used against people who think they are safe because they click no to a program starting when they wouldn't necessarily know it's malicious in real life.


-{ Quote: "If this refers to some type of program, then it's a lost cause because there has yet to be developed a reliable, trustworthy application to do that in all cases." }-

Depends on what you mean by reliable and trustworthy. If you mean 'perfect', no such beast exists. But surely no one is asking for that.

I find it interesting that while Notok and Rmus unite in declaring the uselessness of test demos, leak tests whatever, you two seem to have quite different reasons for doing so.

Rmus I think believes that there is no reliable way to determine if a certain software is malicious if you choose to run it and hence it is a matter of trusting the right sources to download from, common sense etc.

Notok the representative of Prevx1 obviously feels otherwise, because that is after all what Prevx1 is about. Notok is more about failure of knowledge to responding to prompts hence the need for Prevx intelligent design making heuristics or something.

Or have I misrepresente the positions of either of you?

[Devil's Advocate]

-{ Quote: "
I've posted most of that before, but there's a few new ones in there :)
" }-

Where? You have to point them out for me. Seems like the same old arguments. :)

-{ Quote: "
Yes, it's much easier to just run a file and receive a prompt that says either "Pass" or "Fail", but for those to be meaningful in any way depends on a whole lot of things and be performed in a meaningful context (which I have never seen). If you don't know and understand what those things are, then you're easy prey for marketing goons-
" }-

Aren't you arguing above that nobody except for bona-fided malware analysts (which would presumably exclude even you), would have a chance of figuring out things? Does that mean everyone is easy prey?

-{ Quote: "
As far as I'm concerned they're all the same since they all want you to see the answer in black and white terms: "pass vs fail" or "secure vs insecure".
" }-

Definitely the result isn't as important as knowing what is happening.

-{ Quote: "
The only way to assess the effectiveness of a behavior blocker is to know the facts.
" }-

What facts? You already made a big point of showing that probably nobody knows any facts e,g how common the technique is etc See #7

symbolic links? I'm sure i mentioned that before.....

[Mrkvonic]

Hello,

Determining what an application does? It's not difficult but it can take a lot of time. And people do not wish to spend hours trying to reverse engineer an application. They want it plug and play.

In this regard, the best approach is probably combined:

1. If you do not trust the source - don't run it.
2. If you supposedly trust or must use a file - a work colleague, for example - then you can try simulated behavior, like VM or test computer or external forensics via live CD to try to gauge the symptoms.

But if there's a doubt, there's no doubt.

Mrk

[BlueZannetti]

-{ Quote: "Determining what an application does? It's not difficult but it can take a lot of time. And people do not wish to spend hours trying to reverse engineer an application." }-Let's think about these statements for a bit...

Is it realistic to assume that even a small fraction of folks frequenting this site could reverse engineer an application? Let's say they get their hands on a disassembler - will they actually know what to do or look for? Look at the basic unit operations of a good application versus a malicious application - do they really differ to a level that is diagnostic? In some cases, perhaps, but in most instances I'd say no.

The unit operations of malicious applications are not that unique. They differ in objective (measures taken to preserve the infected state via autostart entries and multiple processes monitoring/restarting each other) and degree (dominating CPU and I/O), but not really kind. For most users the only signs of an issue are indirect - say a slowly responding machine or abnormal spikes or levels in traffic. That can be enough to aid in assessment, but at that point it is a post-mortem, not an a priori diagnosis

As for some of the comments above such as:-{ Quote: "Aren't you arguing above that nobody except for bona-fided malware analysts (which would presumably exclude even you), would have a chance of figuring out things?" }-isn't the answer transparently obvious, even for advanced users, at least with respect to a direct assessment?

Blue

[Mrkvonic]

Hello,

Blue, I take my statement back. I should have been more clear. Reverse engineering - behavior like rather than low-level hacking, which could also be quite illegal. Observing an application through a series of nets and filters, including disk operations (read, write), including Windows-specific tasks like autostart entries, registry etc, memory operations (read, write, execute), process injection, and so forth. Then comes the network analysis, using in-system and external diagnosis (from another computer or through a firewalled gateway). This requires quite a bit of tools, including registry, file, disk monitors, packet analyzers, most likely virtual or sandboxed environment.

For the average Joe, even if he were given all the applications needed to do all the forensics, 99.99999999999% would refrain from doing so and simply trash the suspect. It just isn't worth it. Hell, it's dirty and boring and long. No one would do that just to decide if he should run a simple browser.

It's definitely doable but far from practical. Most people will use behavioral, indirect indications - as you mentioned - rather than go by brute force code line-by-line examination. As to the behavior, skill and knowledge really hold sway.

Finally, the issue of probability. For most purposes, good advice and use of trusted source is enough, like DA said.

Mrk

[BlueZannetti]

-{ Quote: "Hello,

Blue, I take my statement back. I should have been more clear." }-OK - that's clearer.

Cheers,

Blue

[Mrkvonic]

Hello,
I'll have to fast for a day because of that noobish mistake. I usually do not make them and I take them very hard. Damn, my day is ruined. I'm off to spill some boiling water on my hand. My first critical spelling mistake for 2007. Damn.
Mrk