Comodo Firewall Test Suite

Thanks for the screenshots, Rickster.

The comparison is interesting but it ends here. It looks like that there's a problem concerning the ActiveDesktop test: even if I block it completely, it'll be shown as vulnerable. There was similar tests problems with earlier versions of CLTs! Are these Comodo tests reliable? You're the judge. I need to investigate but if there really is a coding glitch (which will not surprise me at all) with the first test, there's a high probability that many others will show you as vulnerable even if you're not.

As an EQS user, I will not (and probably never will) rely on Comodo leaktests to make EQS rules.

 

Yep, he answered a question, but he didn't solve the problem. The problem is a user thinks he is safe, while actually he is not.

 

I can say for sure BIS test is also not reliable. For example, even if BITS job was submitted by COM interface, it doesn't mean the JOB was actually executed.

For example, svchost can be restricted with addresses it is allowed to connect to. Then the submitted BITS job will just senslessly try to execute.

Other option is to get and event on new BITS job and enumerate them all and suspend all of them and give a user a chance to select which jobs a good and which are bad. "Bad" jobs can then be cancelled. But the test alsways says "failed". It should surely check was job actually sucessful or not.

Then registry autoruns. You can submit as many autoruns as you wish, but if HIPS prevents unauthorised autoruns fromstart then this is not a leak at all.

And so on ... And I completely agree with Ilya, filedrop "test" is a bullshit.

 

Hi,

what I really do not unterstand is why you can pass some tests with OS only and no security (Admin account; Windows Firewall, Windows Defender and UAC deactivated).

XP SP3 20/340
Protected:
30. Invasion: Runner
34. RootkitInstallation: MissingDriverLoad

Vista SP1 100/340
Protected:
3. Hijacking: ChangeDebuggerPath
6. Hijacking: UIHost
8. Hijacking: WinlogonNotify
13. Impersonation: OLE automation
18. Injection: CreateRemoteThread
21. Injection: ProcessInject
26. Invasion: DebugControl
28. Invasion: PhysicalMemory
32. RootkitInstallation: DriverSupersede
33. RootkitInstallation: LoadAndCallImage

I have posted this at Comodo forums, no serious answer so far. Maybe someone can explain this here.

Cheers

 

Strange how some of us are getting differnet results with the same security setup, os, admin account etc. etc. I'm running vista home (sp1) admin account using CIS (without the AV) on Internet secuity (Safe mode on firewall and D+) and Avast Home (not that it matters in this test) and I get a perfect score. Strange.

Ice

 

Not really strange I must say, considering that it's Comodo leaktests made BY Comodo. They probably haven't included many leaktests with "fail" results as a marketing strategy. That's how i'm seeing it.

To see the true strength of hips and firewalls, what is needed is third-party leaktests... And i'm not talking about Matousec.

 

No upsets.

Sorry about before mate... No hard feelings! Thanks everyone for testing.

 

Nothing doing. Some tests were interesting.

PS. Now you are much better, I'm really glad

 

It's totally out of character i know but i can easily readjust EQS 3.41 rules to successfully snuff off every single one of those CLT leaky tests, although the likelyhood of most of them are obviously biased in Comodos favor as being originair of it, so IMO it;s not to be taken as wide ranging gospel for sure.

EQS 3.41 w/ Alcyon's Super Trapper (Adjustable) Rules = 100% Score in EQS favor. No Sweat,

 

KIS .454 score 240 out of 340 with comodo leaktest tested 3 times

i tested it with outpost pro + KIS firewall uninstalled its shows 300 tested 3 times same result

in administrative mode

4. Hijacking: StartupPrograms
19. Injection: DupHandles
29. Invasion: RawDisk
31. RootkitInstallation: ChangeDrvPath

are Vulnerable

please retest and confirm .......with your results?

can any one confirm the results in usermode?

core 2 duo 2.4
2Gb ram
windows vista home premium
KIS+outpost pro

 

I managed to get 340 with Comodo.
It seems there was something wrong with my initial installation: some missing protected registry keys.

 

Best I can squeeze out of KIS (alone) is 290

 

Newest Malware Defender beta release and Jetico 2 firewall w/Process Attack filter disabled:

330/340 - only the dubious Active Desktop test failed

Great work by Malware Defender's xiaolin to fix the compatibility bug with Jetico 2 so quickly

 

There is only two options. Either the test is inconsistent (which was proven already) or product is inconsistent. Taking in account there are products that shows the same (or very similar) score despite of setup, you are free to make conclusion.

 

Impressive; I got 10/340... Not disappointed by the yellow company performance at all, just what I expected...

 

Norton hasn't got an HIPS, so that's completely normal!

 

I think you hit the nail on the head. What is really needed is a un-biased 3rd party test.

Ice

 

Now that is true progress!

Congrats!

EASTER

 

I'm encouraged by what I'm seeing with this HIPS and especially the tireless efforts of the developer. He's obviously highly skilled in this area of coding, so I hope he does not bail on the product that has such tremendous potential as others have done. I have no problems supporting this kind of effort going into a very promising security application, even if I don't actually need a license.

 

If people are having really weird scores with various products, excluding CIS - Try and set them all up to high on every angle of the Firewall & HIPS/Behavior Blocking your using.

This Testing Suite was designed by the AV Labs, And these tests use REAL malware techniques, including those nasty rookits!! It was designed to test the full strength of a product. That's why if you have "COMODO - Proactive Security" Configuration in CIS, And you block all Defense+ Alerts you will get 340/340, or you should anyway. Proactive Security Configuration activates all of D+ Full power, All D+ Monitor Settings & Image Execution is Enabled, Where in the default installation configuration it was just Internet Security Enabled, And eased off in D+ Places due to Antivirus, etc.

So I advise if you could look for some "high" settings in Kaspersky, and any other product your using - and blocking ALL Alerts they show, to fully pass this test. Just a though on the top of my head. Goodluck!

 

Thank you 3xist. Is CIS achieving this perfect result under an administrator account or limited account? I'm still of the opinion these tests should be run under a limited account, even if it's Power user.

 

Admin Account...

I haven't tested Limited.

 

Hi Easter

Are you saying that with Alcyon's latest rules, a 100% score is achieved against these leak tests? What do you mean by Super Trapper (Adjustable) Rules?

Thanks.

 

I tried Spyware Terminator beta on this. It left an empty frame. So score was 0/0.