Test Your HIPS - Comodos 5 New Security Tests

[CogitoErgoSum]

On 4-16-08, Comodo released 5 new security tests.

ht tp://download.comodo.com/securitytests/CLT.zip
http://forums.comodo.com/leak_testingattacksvulnerability_research/comodo_release_5_new_security_tests-t21917.0.html

Under Vista 32 SP1 with DefenseWall v2.40 beta, I got the following results.

Rootkit Installation 1........Protected/Blocked
Rootkit Installation 2........Protected/Blocked
DLL Injection 1.................Tentative Block*(*Note: This test(3 of 5) hangs and apparently does not complete.)
DLL Injection 2.................Protected/Blocked
BITS Hijack.......................Protected/Blocked


Peace & Gratitude,

CogitoErgoSum

[HURST]

EQS Alcyon's ruleset

Rootkit 1----------protected
Rootkit 2----------vulnerable
DLL1--------------error
DLL2--------------error
BITS--------------vulnerable

anyone can confirm this?

win xp sp2

[HURST]

Running sandboxed with SBIE (EQS disabled)

Rootkit1-----protected
Rootkit2-----protected
DLL1--------some kind of loop, frozen in "testing"
DLL2--------protected
BITS--------leaktest crashes...I asume "protected"?


hhhmmm strange...I chose to test again sandboxed, now with "Run all tests" checked, and I got this:

Rootkit1-----protected
Rootkit2-----vulnerable
DLL1--------some kind of loop, frozen in "testing"
DLL2--------protected
BITS--------vulnerable

[CogitoErgoSum]

Here is an alternate link to the Comodo test program.

http://personalfirewall.comodo.com/cltinfo.html


Peace & Gratitude,

CogitoErgoSum

[sukarof]

In Vista 32 Bit without any HIPS (SRP though but I allowed the execution):

Rootkit installation 1 - Protected
Rootkit installation 2- error
DLL injection1 - error
Dll injection 2 - error
BITS Hijack - vulnerable

I guess this BITS Hijack doesnt actually download anything, it looks in the svchost and sees the vulnerability? coz I ran it without any internet connection too and it still says Vulnerable.

[Kees1958]

-{ Quote: "
Under Vista 32 SP1 with DefenseWall v2.40 beta, I got the following results.

Rootkit Installation 1........Protected/Blocked
Rootkit Installation 2........Protected/Blocked
DLL Injection 1.................Tentative Block*(*Note: This test(3 of 5) hangs and apparently does not complete.)
DLL Injection 2.................Protected/Blocked
BITS Hijack.......................Protected/Blocked
" }-

Same with XP and DefenseWall 2.4 beta. Incredible new feature resource protection. GeSWall always was a little less easy to use for noobs, but had more freak options. With 2.4 DefenseWall offers a strong easy HIPS for noobs and an opton where freaks can fine tune isolated environments.

[soccerfan]

-{ Quote: "Running sandboxed with SBIE (EQS disabled)

Rootkit1-----protected
Rootkit2-----protected
DLL1--------some kind of loop, frozen in "testing"
DLL2--------protected
BITS--------leaktest crashes...I asume "protected"?


hhhmmm strange...I chose to test again sandboxed, now with "Run all tests" checked, and I got this:

Rootkit1-----protected
Rootkit2-----vulnerable
DLL1--------some kind of loop, frozen in "testing"
DLL2--------protected
BITS--------vulnerable" }-
Sandboxie 3.25.12, with a lot of bug fixes and some security fixes has just been released.
Perhaps you could try the tests again on this new version and let us know how it fares.

[Page42]

When I run the tests, the first thing that happens is ZoneAlarmPro produces a security alert that clt.exe is trying to load the driver: CLT\driver.sys. Choices are Allow or Deny. At this point I get very different (and unexpected) results depending on what I tell ZAP to do.

If I choose Deny in ZAP, ThreatFire is silent (not a peep) and I get:

Rootkit installation 1 - Protected
Rootkit installation 2- Vulnerable
DLL injection1 - Hangs on "Testing"
Dll injection 2 - Protected
BITS Hijack - Vulnerable

If I choose Allow in ZAP, ThreatFire alerts right away on a HIGH risk and potentially malicious action and offers either Allow or Quarantine.

Choosing to Quarantine, TF places the executable, .dll & .zip file all in quarantine.

If I allow ZAP and have TF protection suspended, a-squared with Malware-IDS enabled just sits there silently.

--> Edit: After seeing aigle's post (http://www.wilderssecurity.com/showpost.php?p=1225204&postcount=20), I realize I made a mistake in running all tests at once, as this made it difficult for me to see that ThreatFire was actually NOT blocking all malicious behavior. TF only blocked one test. I concur with aigle, as below:

Rootkit installation 1 - Vulnerable
Rootkit installation 2- Vulnerable
DLL injection1 - Vulnerable
Dll injection 2 - Protected
BITS Hijack - Vulnerable

[HURST]

I only find SBIE 3.24 (my current version) on the website...

when I click update on SBIE it reports no updates available...

[Henk1956]

GesWall under Windows XP Home:

Rootkit Installation 1.....Protected
Rootkit Installation 2.....Error.........Read only access to SERVICE OBJECT\Beep
DLL Injection 1.............Error.........Access to C:\WINDOWS\system32\dll.dll denied
DLL Injection 2.............Protected
BITS Hijack..................Protected

[hammerman]

EQS (modified Alcyon's ruleset)
=======================
Rootkit Installation 1.....Vulnerable
Rootkit Installation 2.....Protected
DLL Injection 1.............Error
DLL Injection 2.............Error
BITS Hijack..................Vulnerable

OA (Run Safer)
===========
Rootkit Installation 1.....Protected
Rootkit Installation 2.....Error
DLL Injection 1.............Error
DLL Injection 2.............Error
BITS Hijack..................Protected

OA (not Run Safer)
==============
Rootkit Installation 1.....Vulnerable
Rootkit Installation 2.....Vulnerable
DLL Injection 1.............Vulnerable
DLL Injection 2.............Protected
BITS Hijack..................Protected

[soccerfan]

-{ Quote: "I only find SBIE 3.24 (my current version) on the website...

when I click update on SBIE it reports no updates available..." }-
Sorry, I should have provided a link. Here it is
http://sandboxie.com/phpbb/viewtopic.php?t=3178
Thanks for testing this!

[HURST]

SBIE 3.25
XP SP2

Rootkit1: Protected
Rootkit2: Vulnerable
DLL1: Loop
DLL2: Protected
BITS: Vulnerable

[LoneWolf]

GesWall Free......

[curious george]

What i dont get is the difference between the PRO version and the free versions. Anyway, Comodo has come a long way from V2...

[LoneWolf]

-{ Quote: "What i dont get is the difference between the PRO version and the free versions." }-
Well the pro versions usually cost money and the free versions are usually free ::) (Just Kidding) ;D
More features in the pro.

As far as GesWall is concerned........

[curious george]

Haha true. Very true. Is it worth getting the pro version though?

[LoneWolf]

-{ Quote: "Haha true. Very true. Is it worth getting the pro version though?" }-

Absolutly, much more configurable.
When my tax check gets here i'm gettin mine. ;D
But the free works great. Great protection. :thumb:

[aigle]

SafeSpace .....:thumb:

[aigle]

ThreatFire

[Coolio10]

I am guessing error means protected :D.

[aigle]

Yes, ofcourse.

[Franklin]

Can't even unrar the tests here with Sandboxie set to block all except defined.

Under [GlobalSettings]:
ProcessGroup=<restricted>,firefox.exe,Start.exe,SandboxieDcomLaunch.exe,SandboxieRpcSs.exe

Under [DefaultBox]:
ClosedFilePath=!<restricted>,*
ClosedIpcPath=!<restricted>,*

[EASTER]

Today my tests with EQSecure 4.0 Beta on this "PASSED!" all of them except the very last one. Using of course Alcyon's Rulesets.

The BITS didn't make it but i'm sure i could adjust EQS to cover that one too.

[ink]

vista sp1 with DEP enabled on all application, run under user mode
rookit 1 protected
rookit 2 error
dll 1 error
dll 2 error
BITS vulnerable

[InVitroVeritas]

FYI : Dynamic Security Agent passed (or "errored") all except the first one, on a test XP pro, under admin account.

slight and non-constructive disgression : I've to admit that, as usual, I'm a little undecided towards those tests.

[Blackcat]

Just tried the Free GesWall on my system; Win XP Pro.

Different from LoneWolf's results above. Any idea why?

[bellgamin]

-{ Quote: "ThreatFire" }-Dismal results for TF! :-\

ProSecurity passed all tests.

Did anyone test SSM?

[InVitroVeritas]

-{ Quote: "Dismal results for TF! :-\
" }-
I'll surmise that is simply because there is no actual "rootkit installation" or "dll injection" in these leaktests, which are most certainely *not* tailored made to test behavior blockers, but rather classic HIPS or FW with HIPS like features.

[jrmhng]

Are we sure with sandboxie that it actually failed some of them? Could it be that the programs are just running in the sandbox and it thinks it got the computer but it actually dosen't?

[Peter2150]

-{ Quote: "Are we sure with sandboxie that it actually failed some of them? Could it be that the programs are just running in the sandbox and it thinks it got the computer but it actually dosen't?" }-

That is exactly what happens. Sandboxie isn't a HIPS, so the test can do it's thing, but only affects sandboxed programs, not the system.

[erreale]

-{ Quote: "ProSecurity passed all tests." }-

Sure? I tried with prosecurity 1.43 and the second test is vulnerable. I can not finish the fifth because of an error message windows. Some idea of diversity results?

[nick s]

-{ Quote: "Sure? I tried with prosecurity 1.43 and the second test is vulnerable. I can not finish the fifth because of an error message windows. Some idea of diversity results?" }-With its default file rules, PS 1.43 will fail the second rootkit test because there is no rule to filter the *.sys_old file extension.

Nick

[nick s]

You need to add a broader rule to pass the test...

[cheater87]

I can't download it. It does not let me hit OK. Also what do I do after I download this???

[erreale]

-{ Quote: "With its default file rules, PS 1.43 will fail the second rootkit test because there is no rule to filter the *.sys_old file extension.

Nick" }-

thank you very match

[Mele20]

EQSecure 3.41 passes all but the BITS test.

ProcessGuard full version 3.51 passes both rootkit tests and dll injection 2. It fails dll injection 1 and BITS.

The test GUI claims that EQSecure and PG both fail rootkit 2 test. This is incorrect. Both pass the test.

[EASTER]

-{ Quote: "EQSecure 3.41 passes all but the BITS test.

ProcessGuard full version 3.51 passes both rootkit tests and dll injection 2. It fails dll injection 1 and BITS.

The test GUI claims that EQSecure and PG both fail rootkit 2 test. This is incorrect. Both pass the test." }-

http://www.wilderssecurity.com/showpost.php?p=1225400&postcount=24

Now confirmed by another EQS user.

[C.S.J]

defensewall gives me Vulnerable on the last item. :dry:

[Ilya Rabinovich]

-{ Quote: "defensewall gives me Vulnerable on the last item. :dry:" }-
Interesting. Could you send me DW's log file on the last test?

[hammerman]

-{ Quote: "Interesting. Could you send me DW's log file on the last test?" }-

On my system Defensewall 2.30 passes BITS Hijack test and all others except DLL Injection 1. This test seems to hang. See screenshot.

[HURST]

-{ Quote: "
The test GUI claims that EQSecure and PG both fail rootkit 2 test. This is incorrect. Both pass the test." }-

Good to know!

[EASTER]

I,ve tested REAL rootkits and severe malware samples at EQS so these test successes with EQS are not exactly surprising. PE386, Haxdoor, etc now those are real nail biters and what about ADS, they are still very much a threat as they ever were. I even use an ADS on one of my disk for fun that launches an .exe rubberball everytime i access either a notepad or some other %WinDir% system file.

I think this was more of a showcase then any real HIPS test IMO.

[aigle]

-{ Quote: "GesWall Free......" }-

-{ Quote: "Just tried the Free GesWall on my system; Win XP Pro.

Different from LoneWolf's results above. Any idea why?" }-


Hi, make sure that all thre files, clt.exe, dll.dll and driver.sys are marked isolated. I get same results for GW free and Pro, on XP Home.

[Alcyon]

Comodo will always impress me (sarcasm).

[Rasheed187]

SSM Pro:

Rootkit 1----------Protected
Rootkit 2----------Vulnerable
DLL1--------------Testing.....
DLL2--------------Protected
BITS--------------protected

Neoava Guard:

Rootkit 1----------Protected
Rootkit 2----------Error
DLL1--------------Error
DLL2--------------Protected
BITS--------------protected

[EASTER]

-{ Quote: "Comodo will always impress me (sarcasm)." }-

So will EQSecure (COMPLIMENTS!) 8)

[Smokey]

Tested with DefenseWall HIPS v2.30 / Vista-32:

all tests passed.:)

Good job Ilya!

[bellgamin]

If anyone here is running DriveSentry (http://www.drivesentry.com/), will you please test it against Comodo's 5 bagger, and post results?

[Mele20]

-{ Quote: "SSM Pro:

Rootkit 1----------Protected
Rootkit 2----------Vulnerable
DLL1--------------Testing.....
DLL2--------------Protected
BITS--------------protected


Did you just go by what the GUI reports? I would expect SSM to protect on Rootkit 2. ProcessGuard passed but the GUI claimed otherwise. I had a flashing PG icon in the systray after the Rootkit 2 test and when I checked further PG had blocked the rootkit installation. You need to do the tests one at a time. Uncheck the box to do all tests. The results on both PG and EQSecure are even more inaccurate/misleading unless the tests are done individually. SSM might not hang on DLL injection 2 if the test is done individually.

[EASTER]

-{ Quote: "The results on both PG and EQSecure are even more inaccurate/misleading unless the tests are done individually." }-

Is that so?

Then i suggest Comodo fashion it to throw all of them at once. Contrary to the jealousy that critics try to enjoy on EQS, they ARE NOT! MISLEADING.

EQS even in beta 4 stage repels most if not all HIPS tests most formidably and without problems like lock ups and screen freezes that others fall to.

[cheater87]

Passed the first rootkit, failed the rest. This is with Spyware Terminator only. :( With Comodo Defense Plus I got passed first rootkit failed second one, passed 3rd option and failed the rest.

[nick s]

-{ Quote: "...I had a flashing PG icon in the systray after the Rootkit 2 test and when I checked further PG had blocked the rootkit installation." }-How did you check? I suspect that, as with ProSecurity, PG is failing in its handling of the odd file extensions (.sys_old, .sys?, .sys_) in the System32\Drivers folder. PG successfully blocks the first two steps of the attack involving beep.sys...hence the flashing icon. Thereafter, PG fails as does ProSecurity with its default rules. There is at least one method of installing drivers (rootkit or otherwise) to which PG is blind.

In the following sequence, ProSecurity (with my added rule) passes if I allow the first two file writes, but deny everything thereafter. If I deny the first two writes, but allow the others, then ProSecurity fails.

clt.exe
[WRITE FILE] 2008.04.20 22:19:23
[ALLOW] C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: beep.sys

clt.exe
[WRITE FILE] 2008.04.20 22:19:23
[ALLOW] C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: beep.sys

clt.exe
[WRITE FILE] 2008.04.20 22:19:27
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: [B]beep.sys_old?

clt.exe
[WRITE FILE] 2008.04.20 22:19:27
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: [B]beep.sys_old?

clt.exe
[WRITE FILE] 2008.04.20 22:19:28
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: [B]beep.sys_old?

clt.exe
[WRITE FILE] 2008.04.20 22:19:28
[BLOCK] C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: beep.sys

clt.exe
[WRITE FILE] 2008.04.20 22:19:29
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: [B]beep.sys?

clt.exe
[WRITE FILE] 2008.04.20 22:19:29
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File:[B] beep.sys?

clt.exe
[CREATE FILE] 2008.04.20 22:19:30
C:\test\clt.exe
Command Line:"C:\test\clt.exe"
[ACCESS TO] Folder: C:\WINDOWS\system32\drivers\
File: [B]beep.sys_

Nick

[HURST]

-{ Quote: "Passed the first rootkit, failed the rest. This is with Spyware Terminator only. :( With Comodo Defense Plus I got passed first rootkit failed second one, passed 3rd option and failed the rest." }-

Comodo failed it's own test??

[Smokey]

-{ Quote: "ThreatFire" }-
Tested it too with TF / Vista-32, same sad, embarrassing results:

1 test passed, 4 tests failed.

PC Tools, please revamp TF asap!!!

[Pedro]

If these are all POC, then TF has a false positive to fix.
TF was built to detect malicious activity, not activity in general.

[aigle]

I agree but my concern is only about the driver loading( first test).

[Mele20]

-{ Quote: "How did you check? I suspect that, as with ProSecurity, PG is failing in its handling of the odd file extensions (.sys_old, .sys?, .sys_) in the System32\Drivers folder. PG successfully blocks the first two steps of the attack involving beep.sys...hence the flashing icon. Thereafter, PG fails as does ProSecurity with its default rules. There is at least one method of installing drivers (rootkit or otherwise) to which PG is blind.

In the following sequence, ProSecurity (with my added rule) passes if I allow the first two file writes, but deny everything thereafter. If I deny the first two writes, but allow the others, Nick" }-

You are right. PG does FAIL Rootkit 2 test. Sorry, I had that wrong earlier.

[interact]

-{ Quote: "If anyone here is running DriveSentry (http://www.drivesentry.com/), will you please test it against Comodo's 5 bagger, and post results?" }-

Just tested with latest copy of Drivesentry

1, Rootkit 1---------- XP = fail / Vista = protected
2 ,Rootkit 2---------- protected
3, DLL1-------------- protected (block extracted DLL)
4, DLL2-------------- error (method doesn't work)
5, BITS-------------- protected (DS doesn't have network protection / detects file being written to disk).

I run PC Tools Firewall plus with Drivesentry on three of my PCs so test 5 would be trapped by PCT. Test 1 is a not a huge concern as I've now switched over to Vista Pro and this undocumented way of loading a driver is now obsolete.

~interact

[nick s]

-{ Quote: "...Sorry, I had that wrong earlier." }-
Not a problem :). I confirmed the failure with PG 3.410 (full) this morning. I don't have the later versions that followed after Jason left DCS. PG was an awesome app in its day.

Nick

[aigle]

-{ Quote: "Just tested with latest copy of Drivesentry

1, Rootkit 1---------- XP = fail / Vista = protected
2 ,Rootkit 2---------- protected
3, DLL1-------------- protected (block extracted DLL)
4, DLL2-------------- error (method doesn't work)
5, BITS-------------- protected (DS doesn't have network protection / detects file being written to disk).

I run PC Tools Firewall plus with Drivesentry on three of my PCs so test 5 would be trapped by PCT. Test 1 is a not a huge concern as I've now switched over to Vista Pro and this undocumented way of loading a driver is now obsolete.

~interact" }-
Good results for DS.

[alex_s]

-{ Quote: "Good results for DS." }-
In case this is Vista with UAC this is rather Vista's result, that anything else.

[Smokey]

-{ Quote: "In case this is Vista with UAC this is rather Vista's result, that anything else." }-
All credits to DriveSentry: tested DS v3.0.2.16 with Vista-32/deactivated UAC and DS passed all tests.

So a :thumb: for DS Development too.

[aigle]

-{ Quote: "In case this is Vista with UAC this is rather Vista's result, that anything else." }-
From his post it apppears that he did test on both Vista n XP.

Smookey has now cofirmed it.

[interact]

-{ Quote: "In case this is Vista with UAC this is rather Vista's result, that anything else." }-

I don't run UAC on Vista as it's a pain in the ass.

~interact

[InVitroVeritas]

for DriveSentry, under XP pro, I've got different results : vulnerable to the first, third and fourth of those so called test. ::)

[alex_s]

-{ Quote: "I don't run UAC on Vista as it's a pain in the ass.

~interact" }-
Completely agree :)

[Kees1958]

-{ Quote: "Completely agree :)" }-


Why not run LUA in quiet mode with TweakUAC as an alternative? You do noy have the all the elevation pop-ups, still all programs run UAC, registry and file virtualisation work and IE will run in protected mode?????

WHy did you buy Vista in the first place when you are not using its improvements?

[InfinityAz]

-{ Quote: "Why not run LUA in quiet mode with TweakUAC as an alternative? You do noy have the all the elevation pop-ups, still all programs run UAC, registry and file virtualisation work and IE will run in protected mode?????

WHy did you buy Vista in the first place when you are not using its improvements?" }-

I agree and use TweakUAC (it makes a big difference and you get to keep the security UAC provides without the nuisance).

[alex_s]

-{ Quote: "Why not run LUA in quiet mode with TweakUAC as an alternative? You do noy have the all the elevation pop-ups, still all programs run UAC, registry and file virtualisation work and IE will run in protected mode?????

WHy did you buy Vista in the first place when you are not using its improvements?" }-

There is no way not to buy Vista in case you buy new computer or laptop. All of them have Vista preinstalled here.

[bellgamin]

-{ Quote: "There is no way not to buy Vista in case you buy new computer or laptop. All of them have Vista preinstalled here." }-You can special order a virgin computer via internet, or -- better yet -- have a computer built for you, set up for dual-boot (to Linux & XP). Or... buy a MAC.

[Mele20]

-{ Quote: "There is no way not to buy Vista in case you buy new computer or laptop. All of them have Vista preinstalled here." }-

Buy a Dell before June 30. Dell offers a lot of computers with XP Pro. You should check both their Home Division and Small Business Division. You do not have to own a small business to buy from that division. Or have a computer locally built but do it before June 30th which is the date Microsoft will stop OEMs from selling XP on computers and will pull retail XP off the shelves. However, small shops and others who build computers can still obtain XP Pro until next January.

You can also buy a new OEM computer with Vista Business or Ultimate installed and then invoke your downgrade rights. You can downgrade it to XP Pro. If you already have an XP Pro disk you can use that disk to downgrade. The OEM might provide you a downgrade disk for a modest fee but they are not required to do so. You could also buy an XP Pro OEM disk from New Egg if need be. If you want to invoke downgrade rights you must buy the new computer with Business or Ultimate editions of Vista. The other versions do not come with downgrade rights.

[Peter2150]

-{ Quote: "f you want to invoke downgrade rights you must buy the new computer with Business or Ultimate editions of Vista. The other versions do not come with downgrade rights." }-

I saw on the Dell site, you can invoke those download rights, and they will install and ship with XP installed.

[Mele20]

-{ Quote: "I saw on the Dell site, you can invoke those download rights, and they will install and ship with XP installed." }-

"downgrade" not "download" (I know you meant downgrade but someone might get confused so thought I would point it out).

I haven't been there in awhile. I didn't realize they were doing that. Supercool! That saves the customer from having to uninstall Vista and install XP. Did you see this at Small Business? I wonder if they are doing that for Home division buyers too?