Site to test HIPS ?

Discussion in 'other anti-malware software' started by acr1965, Dec 30, 2006.

Thread Status:
Not open for further replies.
  1. acr1965

    acr1965 Registered Member

    Joined:
    Oct 12, 2006
    Posts:
    4,954
    Is there a clean site available to test my HIPS program? I currently have DSA (Dynamic Security Agent) installed and would like to see how it performs. Does anyone know how DSA stacks up against SSM (System Safety Monitor) or Spyware Terminator HIPS program?
     
  2. Jarmo P

    Jarmo P Registered Member

    Joined:
    Aug 27, 2005
    Posts:
    1,188
    You are totally wrong.

    HIPS is not something to test about malware.
    There are HIPS's like sandboxies, those ones I recommend to you.
    Classical HIPS like SSM, they are not either so much testable.

    So forget hips testing sites.
     
  3. zopzop

    zopzop Registered Member

    Joined:
    Apr 6, 2006
    Posts:
    632
    i don't know about any specific sites that test HIPS but you can try the following :

    1) advanced process termination test
    http://www.diamondcs.com.au/index.php?page=apt

    2) the simple process termination test from SSM
    http://www.syssafety.com/leaktests.html

    3) the keylogger test from SSM
    http://www.syssafety.com/leaktests.html

    4) morgud's threat simulator
    http://www.morgud.com/interests/security/dfk-threat-simulator-v2.asp

    5) gentlesecurity's threat test
    http://gentlesecurity.com/demo.html

    6) spycar's browser hijack tests
    http://spycar.org

    7) ghostsecurity's registry tests
    http://ghostsecurity.com/registrytest/

    8 ) martin's keylogger (see if your HIPS program can stop it from recording keystrokes)
    http://www.winsite.com/bin/Info?26000000037599

    hope those help :D

    and if you're feeling super brave, test whatever HIPS you have vs the killdisk virus. but be warned if your HIPS fails, you'll need to reformat your hard drive and fix your MBR.
     
  4. acr1965

    acr1965 Registered Member

    Joined:
    Oct 12, 2006
    Posts:
    4,954

    That should keep me busy for a while. Thanks!!
     
  5. Rmus

    Rmus Exploit Analyst

    Joined:
    Mar 16, 2005
    Posts:
    3,943
    Location:
    California
    Aren't HIPS products supposed to include some type of protection against running executables not already installed?

    Since most malware attempts to install a malicious executable, you can test for this protection yourself.

    1) go to a download site and see if your program will block downloading an executable

    2) try to run an installation CD not previously installed - your program should block the running of the setup.exe on the CD

    3) or, have a friend put a program you don't have on a CD, and see if your HIPS will block it's running

    These being proven, you can feel secure that unauthorized installation of malware executables would be blocked, and that this part of your HIPS is working OK.

    regards,

    -rich
     
  6. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    Do you mean the antivirus program? AFAIK, a HIPS will not stop the downloading of an executable.

    Otherwise I like the fact you touch on this point, something no one seems to ever mention. We allow the executables of these termination/leak tests to launch after being alerted by our HIPS that they are trying to do so (yes, I know we let them run to see if these tests can run the gamut), but this is akin to unlocking our doors and windows and then wait to see if the thief can break into our home. Sorry to repeat this ad-nauseum but I feel obligated to do so :rolleyes:
     
  7. Rmus

    Rmus Exploit Analyst

    Joined:
    Mar 16, 2005
    Posts:
    3,943
    Location:
    California
    Don't be sorry - it needs to be repeated.

    Last year I posted to a firewall leaktest thread that my Kerio 2 passed all of the tests. How can that be, I was asked, because the results on the test site showed differently.

    Because the test executable was blocked from running. The response was that I wasn't playing fair.

    OK, maybe not from their standpoint. But if the test executable simulated a trojan executable, and your security blocks it, why isn't that valid?

    In the case of the firewall leaktest, why should I have to disable my security to let a test run that would prove my firewall couldn't do what it wasn't intended to do in the first place?

    OK, interpolate HIPS into this scenario, as you have done, and it is a very valid question to ask.

    regards,

    -rich

    ________________________________________________________________
    "Talking About Security Can Lead To Anxiety, Panic, And Dread...
    Or Cool Assessments, Common Sense And Practical Planning..."
    --Bruce Schneier​
     
  8. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    Well said!! :thumb: After all, if someone mistakenly allows a suspicious file to run, then there is no point in them using a HIPS in the first place. Some will argue this but I stand by my convictions and I see you think along the same lines as I :)
     
  9. ggf31416

    ggf31416 Registered Member

    Joined:
    Aug 20, 2006
    Posts:
    314
    Location:
    Uruguay
    If someone allows a dangerous file to run, then a good HIPS like SSM or ProSecurity should give additionals prompts if it attemps to do something suspicious.
     
  10. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    Yes, I agree and would hope that to be the case, and in hindsight my comment in my previous post is probably unfair. Slipping up at the wrong time allowing a dangerous file to run could happen to the best of us, so a HIPS that can catch further actions on such a file is obviously desirable. Still, if it looks suspicious in the first place, then by all means it should not be allowed to run except, of course, for testing purposes as the OP is looking to do. This I am all for because it is a valuable learning experience.
     
    Last edited: Dec 30, 2006
  11. acr1965

    acr1965 Registered Member

    Joined:
    Oct 12, 2006
    Posts:
    4,954
    Re: Site to test HIPS ? (some test results)

    Well here are some results of the tests. I was unable (or unwilling) to run some of the tests but I ran the Spycar Autostart and IE Config Change tests, the Advanced Process Termination test from diamondcs and the Ghost Security Regtest. I tested both DSA and Spyware Terminator (three settings for the Spycar test with ST: Realtime Shield enabled/HIPS disabled, Realtime Shield disabled/HIPS enabled and Realtime Shield enabled/HIPS enabled). Both DSA and ST were tested seperately.

    My computer is a Hewlett-Packard Pavilion, Windows XP w/SP2, P4, 2.4 GHz, 1 GB of RAM.

    The results for both DSA and ST for the diamondcs APT test were that they both passed 1-9 and 11 of the APT. Test 10 and 12 were graded as "unable to test." As for the Suspend Test 1 and 2, both passed. Kernel Kill Test 1 and 2 both apparently passed. Crash Test 1 and 2 were not tested.

    As for the Spycar, DSA passed every test. ST did not. Here is a list of the six "Autostart Tests":

    1. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\Run

    2. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce

    3. try to drop a file and install a Registry key to execute it under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnceEx

    4. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\Run

    5. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnce

    6. try to drop a file and install a Registry key to execute it under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnceEx

    As said above, DSA passed this series of tests. ST tested with Realtime Shield disabled/HIPS enabled passed test #5 but failed all others. With Realtime Shield enabled/HIPS disabled ST passed all six tests. With Realtime Shield and HIPS enabled ST passed all six tests.



    The Internet Config Change Tests was performed in a similar fashion. Here is a list of the IECC Tests:

    1. change your default home page in IE

    2. lockout users from changing the default home page in IE

    3. change your default search page in IE

    4. remove the Advanced Tab in your IE Internet Options Screen

    5. remove the Programs Tab in your IE Internet Options Screen

    6. remove the Connections Tab in your IE Internet Options Screen

    7. remove the Content Tab in your IE Internet Options Screen

    8. remove the Privacy Tab in your IE Internet Options Screen

    9. remove the Security Tab in your IE Internet Options Screen

    10. remove the General Tab in your IE Internet Options Screen

    DSA passed all the IECC tests. ST was tested in the same fashion as with the Autostart Tests: Realtime Shield enabled/HIPS disabled, Realtime Shield disabled/HIPS enabled and Realtime Shield enabled/HIPS enabled. All had similar results in that ST passed test 1 (make Spycar try to change your default home page in IE) but failed all others. In the results section this test was listed as #9 for some reason.

    As for the Ghost Security Regtest, neither passed although ST performed better than DSA. ST was tested as above with the various enablings of Realtime Shield and HIPS. But whatever the settings ST had the same results, it allowed REGtest to pass the first attempt but blocked all others. DSA allowed Regtest to do as it wished despite the pop-up warning to block which was clicked. As for test 2, both had "fail" messages after restart.

    Well that's it. I think for a limited test sample it would be better to keep ST's Realtime Shield activated and deactivate its HIPS. In place of ST HIPS it appears better to run DSA. Although neither passed the REGtest, it appears ST provides some protection, at least more than DSA.

    But these results are strictly on my machine. I feel comfortable with them but anyone else should use their own judgement or run their own tests. Also, there are several HIPS programs one may consider besides DSA.
     
  12. Mrkvonic

    Mrkvonic Linux Systems Expert

    Joined:
    May 9, 2005
    Posts:
    8,698
    Hello,
    No one seems to mention!!! HELLOOOOO! I've been repeating it since 1844!
    Mrk
     
  13. ggf31416

    ggf31416 Registered Member

    Joined:
    Aug 20, 2006
    Posts:
    314
    Location:
    Uruguay
    Re: Site to test HIPS ? (some test results)

    What process do you terminate for ST?

    My results for ST:

    I ran a full scan in order to activate the HIPS. I copied the SpyCar test to a new folder and the APT executable to APT2.exe before starting the test. I allowed APT to run and tried to terminated spywareterminatorshield.exe. If termination appears to be sucessfully I ran the Hosts test and TowTruck.exe to check if the protection was terminated and not only the GUI.
    ST version was 1.7.0.899. Expert settings with HIPS and all shield enabled.
    Freeze means ST didn't give any prompts but the Hosts test could not run (without any error message).
    Yes means the ST process was terminated and the hosts change was allowed
    No means the ST process was NOT terminated

    Suspend 1 Freeze
    Suspend 2 Freeze

    Kill 1 No
    Kill 2 Yes
    Kill 3 Yes
    Kill 4 Yes
    Kill 5 Yes
    Kill 6 Yes
    Kill 7 No
    Kill 8 No*
    Kill 9 Yes
    Kill 10 No
    Kill 11 Yes
    Kill 12 No

    Kernel 1 Yes
    Kernel 2 No

    Crash 1 Yes**
    Crash 2 Freeze

    * ST asked whether I wanted to allow the APT process to start. I if clicked Yes the ST process would be terminated.
    ** APT reports that the termination was unsuccefuly but the hosts test runs without any prompt
     
  14. djg05

    djg05 Registered Member

    Joined:
    Apr 6, 2005
    Posts:
    1,504
    Whilst I agree with you in day to day running, I do feel that when you are trying out these tests you want to know how strong they truly are and what it takes to break through the HIPS, so that if you make a wrong decision you still have some protection. There is no knowing if one of these exploits is buried in some software that you thought was trustworthy or they get on to your m/c through some other means.
     
  15. Ice_Czar

    Ice_Czar Registered Member

    Joined:
    May 21, 2002
    Posts:
    696
    Location:
    Boulder Colorado
    Ive always valued destructive testing :p
    the scenario that test proposes is all too plausible (and neatly illustrates the combination of basic social engineering with an advanced level of misdirection\impersonation that could fool even a paranoid with a layered defense.)

    How old is that simulator?
     
  16. Mrkvonic

    Mrkvonic Linux Systems Expert

    Joined:
    May 9, 2005
    Posts:
    8,698
    Hello,

    A few months. It's not a bad test except ... you need to:

    1. download a file.
    2. execute it.

    =

    1. **** the gun.
    2. Shoot yourself in the head.

    Mrk
     
  17. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    LOL! Okay, sorry Mrkvonic, I guess I missed your comments. You have my sincerest apologies ;) BTW, I like your style. You post a lot of cool information and put it across in a very intriguing manner.
     
  18. Ice_Czar

    Ice_Czar Registered Member

    Joined:
    May 21, 2002
    Posts:
    696
    Location:
    Boulder Colorado
    yes but...
    the scenario that went with that that tool was highly probable, I'll make one change to it. Rather than it being Judy's friend Carl, its your client Paul, he gets to make a decision about adopting a $35,000 service contract and your kissing his ass to smooth the deal, He thinks its hilarious and expects you to comment. Now you are motivated to c0ck the gun and shoot yourself in the head. And the question becomes not can it subvert your security, but rather will it go unnoticed?

    take a chance and reimage just to be sure?
    decline and show him your tinfoil underwear?
    try it and know youve just been buggered?
     
  19. Rmus

    Rmus Exploit Analyst

    Joined:
    Mar 16, 2005
    Posts:
    3,943
    Location:
    California
    This is from the v.1 test last year:

    dfk test

    If I noticed anything like that, I would just reboot to good previous state.

    Is this what you are referring to?

    regards,

    -rich

    ________________________________________________________________
    "Talking About Security Can Lead To Anxiety, Panic, And Dread...
    Or Cool Assessments, Common Sense And Practical Planning..."
    --Bruce Schneier​
     
  20. Mrkvonic

    Mrkvonic Linux Systems Expert

    Joined:
    May 9, 2005
    Posts:
    8,698
    Hello,

    Ice, indeed one of the greatest problems I'm facing is .docs and .ppts from various clients and colleagues, who have no clue what they're doing.

    I solve the problem the following way:

    Open the files in Open Office using Linux hihihihihihi.

    Open the files using Open Office / MS Office in VM.

    Open the files on the work computer and let the IT worry.

    NEVER open files I'm not absolutely sure are ok - which limits it down to about 2-3 people.

    BTW, extra sophisticated malware in company docs means clear corporate saborage. Highly unlikely you'll receive something like that in a friendly exchange.

    I don't trust scanners to do the work for me - that includes HIPS.

    Mrk
     
  21. lucas1985

    lucas1985 Retired Moderator

    Joined:
    Nov 9, 2006
    Posts:
    4,047
    Location:
    France, May 1968
    :thumb:
     
  22. Long View

    Long View Registered Member

    Joined:
    Apr 30, 2004
    Posts:
    2,295
    Location:
    Cromwell Country
    I have no problem with docs that come from people I don't know - in one way or another they are destroyed. Problem with destroying mail from clients is ..... well they might not understand.

    Its the last point that concerns me - "that includes HIPS". My only real experience is with ProSecurity. Having spent a number of weeks working with the program - it appears that I now have a number of rules allowing my programs a certain degree of freedom. If a client now sends me what appears to be a legitimate email with an attachment which I open my hope is that when "it" tries to do something I will receive a warning. I will then have the option to click block bit will more likely just restore the previous days Acronis system image. Am I then simply wasting my time having a HIPS program ?
     
  23. Rmus

    Rmus Exploit Analyst

    Joined:
    Mar 16, 2005
    Posts:
    3,943
    Location:
    California
    I wouldn't stake my reputation on that:

    Zeroday Scan


    regards,

    -rich

    ________________________________________________________________
    "Talking About Security Can Lead To Anxiety, Panic, And Dread...
    Or Cool Assessments, Common Sense And Practical Planning..."
    --Bruce Schneier​
     
  24. acr1965

    acr1965 Registered Member

    Joined:
    Oct 12, 2006
    Posts:
    4,954
    Re: Site to test HIPS ? (some test results)

    I re-ran the APT and trying to terminate STShield.exe and had similar results as your test. I only ran the Kill 1-12 tests and 10, 12 were "unable to test."

    Yes means ST was terminated, No means ST was not terminated:

    Kill 1- No
    Kill 2- Yes
    Kill 3- Yes
    Kill 4- Yes
    Kill 5- Yes
    Kill 6- Yes
    Kill 7- No
    Kill 8- Yes
    Kill 9- Yes
    Kill 10- unable to test
    Kill 11- Yes
    Kill 12- unable to test

    I received the same results with only Realtime Protection and HIPS disabled.
     
  25. Rmus

    Rmus Exploit Analyst

    Joined:
    Mar 16, 2005
    Posts:
    3,943
    Location:
    California
    I don't know why they have to be destroyed.

    Students - my 'clients' - at college learn to use MSWord - most will end up using it in business (I'm not saying that's good or bad - it's just the way it is at the moment)

    I received 30+ docs per week at home. It's easy to configure MIME types in your email client to open *.doc with another application that is not vulnerable to such exploits.

    It's just not a huge problem to solve if approached logically, and there are many solutions.

    regards,

    -rich

    ________________________________________________________________
    "Talking About Security Can Lead To Anxiety, Panic, And Dread...
    Or Cool Assessments, Common Sense And Practical Planning..."
    --Bruce Schneier​
     
Loading...
Thread Status:
Not open for further replies.