PCT firewall + Leaktests

Most on the forum will know that I dont judge a firewall by its leaktest ability, not because I think HIPs are of no use, far from that, I believe a well implemented HIPs is a good layer of protection if it is implemented well and the user can understand the implementation and make correct decisions. I just wonder at times if the coding of certain security applications are purely based around the leaktests and not real world threats

So on to the reason of my post.

I was having a look at PCT firewall (V6.0.0.74).

I have found a number of issues, but for now will just show this example:
The "Leaktest" http://www.grc.com/lt/leaktest.htm I think we all know that test and how to use it. For those who dont, a simple explanation. The test is used to see if the firewall checks the MD5(or sha) of the application, it is to see if the firewall will intercept a changed binary (in case the application is altered). A full explanation can be found at the link above.
So on to the text.
I renamed the leaktest to firefox, then replaced the original firefox with the leaktest, and behold, a popup to show the interception of a changed binary.



But hang on a minute, the popup is a little incorrect, it states the binary of "Firewall Leak Testing Utility" as changed, not firefox as expected. Never mind, lets expand the test.
First I checked the MD5 of firefox, then made sure the firewall allowed that firefox internet access. I then changed firefox by editing its actual file, I then checked the MD5 again, and as you can see from the image, there is a difference. I got no notification from PCT firewall, firefox was simply allowed out with a changed binary.



So, to check further, I again replaced firefox with the original, and deleted the rules for firefox from the firewall, then allowed it access again. This time I replaced firefox with another application, which I had renamed to firefox. Now instead of getting the popup of changed binary, I was just given a popup to see if I want to allow the application.



Now, the first thought could be that this is a bug, however, the firewall always catches the "Firewall Leak Testing Utility" when I replace firefox with it, and the popup of "Binary changed" will show, but no popup of "Binary changed" when I actually change the binary of firefox or replace it with a different application. (Note: testing made several times to confirm behavior)

Something is certainly not right with this, but I will leave it up to the members here to make their own conclusion.


- Stem

 

Hi,

may be notifications are based on product name not file name ?

regards,

MaB

 

If it did, then there would not be the "changed binary" alert for that leaktest, it would be consistent with the other alerts, and just ask permission for the leaktest to connect out.

Regards.,



- Stem

 

You are right, this seems inconsistent.
Sorry did not pay attention to the different notifications

Regards,

MaB

 

I think what PCT checked is the digital signature or the property of a app.While you renamed a app,its property and digital signature are not changed.So the rule of the app was still avail to PCT instead of pop-up.

 

The white listing was disabled, but even so, a binary change even of a trusted signed application should cause an alert when the firewall already has rulesets for the original application.


- Stem

 

Hi Stem,

This is really interesting,and weird. But I think the first popup "leak test binary has changed" is not "a little incorrect",but totally incorrect!

It's just my wild guess. PCtools firewall may be using some tricks to cheat on those suffocating tests?

 

Hi bonedriven,

There is something not quite right. I did change the leaktest file, but it is checked when executed, so could not verify if PCT was simply picking up its product/embedded name.


- Stem



@all, I have split some off-topic posts to a new thread


.

 

I ran this leak test on:

Rising Personal Firewall 2009 Free - Failed
Symantec EndPoint Protection - Failed
PC Tools Firewall Plus 3.0.0.14 - Failed
Comodo Firewall (only) Current - Passed

 

I would suspect blacklisting based on internal name, but yes, GRC test will fail to run (with an informative message!) if name/owner/etc are altered, so unfortunately that can't be checked. A warning for changed binary where there is no actual check for this (with other apps, I have tried a few) is plain lying/misleading.
If anyone is a member at PCT forums, this thread should be linked (if allowed) or question asked there. I would love to see an official reply.

According to Matousec, PCT passed this test (direct link to PDF report) with previous build (LOL) -

Nevertheless, how is this info relevant here? Reread the thread, it is about PCT firewall giving a false warning, not about it's ability to pass the test (or not).

Cheers,

 

Yes Comodo Firewall Enterprise Strength Only nothing else but you still have to load the entire CIS suite though. All the wireless laptop and 1 desktop runs this one. I really like Symantec Endpoint Protection I left that running a my Enterprise Server seems to find all sorts of pest on that on it's own. Never really prompts you about anything.

I found out why Rising Personal Firewall didn't box the Leak Test, so I fixed that and now it blocks those again. So it Passed after the fix. I had to install this one again I can't seem to get the Comodo Firewall to work with BT stuff just can't seem to get the application and gobal rules to allow access keeps on blocking access to the BT client software. On well Rising Personal Firewall once it see the ports I use it configures the access point.

 

I've tried all PCT FWP even the latest one but if the PC Tools experts didn't take into a fact about Leaktest tool then that one could have been missed. Sure any software can give you a FP or False warning they're not 100% shield protection. I am sure some kind of rule at the kernel end would fix this issue.

 

Hi Stem:

Is there anyway you could find time to repeat this exact same test for Outpost Firewall Pro 2009? Version 6.7.1 build 2983.450.0714?

I suppose that to do an apples to apples compare the same rule sets would be needed as with PCT but I don't know how you got the rules for PCT did it just generate it's own over time or did you have to put in your own set?

If you did do OP I guess it would be a new thread?

 

Hi Escalader,

I will try to find time later.

There are no rules to create for this. The firewall should create an MD5/SHA for an application when it is added to the firewalls ruleset, be it on installation of the firewall with some auto feature picking up digital sigs of applications, or when an application is first executed etc. Different security applications/firewall have different options on how it is handled, some give an option to allow based on whitelist and/or digital signatures, some even allowing auto updates of MD5/SHA of those (so for example, there is not a lot of popups for changed checksum after a windows update). But when those options are disabled, then there should be a popup/warning of a changed application. I do want to re-install PCT firewall to make further checks to see if I can get to the bottom of this. I will then also check OP pro.


- Stem

 

Hi Stem:

No rush, my set up is heavily layered and blocked. When you are ready I'll provide the latest version number for you if you need it.

In the OP testing and optimization thread, I'm active testing the features and rules as you may have seen. So I have plenty to keep me busy

 

Hi Escalader,

I installed OP pro 6.7.1. It catches firefox being altered OK.
I see that the "Host protection- warn about starting new or unknown executables" still does not work correctly. Never mind, for another thread


- Stem

 

Okay, when you you are ready with the thread I'll help anyway I can.

 

Re: Windows 7/Compatible Firewalls

EDIT: posts have been split from this thread and added here

Well it makes me laugh, certainly as with PCT firewall as it does not even protect its own system files, which can be renamed and make it useless (just one example of its flaws)


- Stem

 

Re: Windows 7/Compatible Firewalls

Well, before we can judge and laugh you should explain the method with which this flaw can be used. Or at list provide a link...an article...something. Saying there is a flaw...bla bla bla...says nothing to me.

Allow me to say somenthing else too. Your post surprised me since you are a moderator here.

 

Re: Windows 7/Compatible Firewalls

"bla bla bla"? Sounds a bit sheepish to me, a little like users following a security application based on non-malicious leak tests results.

Malware is known to attack security applications in an attempt to kill off or rename its system files.

If you dont like my posts, report it to an Admin.


- Stem

 

Re: Windows 7/Compatible Firewalls

Still you have provided no informations on how this flaw can be used. Have you ever seen a (specific) malware disable PC Tools Firewall Plus? If yes what other protections did your machine have?

You just say: "malware is known to attack security applications", well in this case we talk about PC Tools Firewall Plus.

If you have no personal experience with a malware defeating PC Tools at least provide a link or something that proves it.

Is your understanding of security just a firewall installed all alone on a system?

Strange...I did not know that we can report posts that we don't like. In any case why I should like or dislike your posts. I can agree or disagree. I have only said that I am surprised since you have expressed such a negative opinion about a security product. I always thought this forum would like to keep some equal distances. For sure I cannot and I would never try to stop you saying your thoughts.

I would like to hear more about this flaw and please provide facts. I am a user of Pc Tools Firewall Plus and I care about my security. If possible please provide the combination of security apps that failed against a specific malware...the case you have seen PC Tools Firewall disabled with renamed files.

Thank you.

 

Re: Windows 7/Compatible Firewalls

That is like me putting forward a need to see a specific malware that attempts to kill off a specific security application and if not found then question why that security application needs such protection. I dont test firewalls based on having other 3rd party security applications installed. It is not good practice to install 2 HIPS.

Malware is know to attack security applications, that is why part of the "Pro active" tests is to see if the application defends itself. Do you need to see specific malware that specifically aims at PCT firewall to have that protection in place, or would you be happy if it was not there as "No one as shown this malware"?

For all the types of "Leak tests" available, have you found links or info on specific malware that uses those techniques?

My understanding is that malware can be malicious, it does not conform to simply attempting to "Phone home" It can and does attack the system, it does not care if it destroys the security setup in its path, "leak tests" do.

If you like PCT firewall and rely on its protection, then maybe this thread I made may be of interest.


- Stem

 

Re: Windows 7/Compatible Firewalls

Well, thank you. At least some facts.

Believe me I understand very well what you are saying and it is really common ( things are better lately ) to see security applications that don't protect themselves or doing it not efficiently.
I did not ask for a malware that attacks specifically PCT Firewall. I have asked for a malware, yes a specific malware capable to disable PCT Firewall. My poor english maybe did not allow me to say it better. I'm convinced that security applications should protect themselves but I'm also convinced that there will always exist a malware that can disable a certain version of a security application.

Tests have specific rules. Unfortunately tests don't take in consideration the idea and the strategy of a company about a system's security. But a user can do it.
PC Tools provide a great range of products that can provide good security just following the companies idea and thinking about security. This is why there is Threatfire there too and the PCT Antivirus.
I can accept a test between firewalls...if we consider them firewalls...not a firewall and half hips...not firewall+hips...etc. Firewall has one and only meaning. You did not test it as firewall...you have tested its self protection. There is a difference here.

And then I believe that in the real world( not in your test ) there should be another application there that should have detected a possible malware which could have changed Firefox. There is a gap in the logic of your test and a disconnection with the real world...or your test is very limited...or it's just another one of those tests that want to create impressions without a dose of realism.

You say "I don't test firewall based on having 3rd party security software"...but then you come with the phrase..."It is not good practise to install 2 HIPS"...so you test firewalls or you test firewalls with hips capabilities. Because I understand that PCT Firewall is more a firewall than a hips.

Checking PCT forums about related issues I found a thread which at least reports that some fixing will be done about the popup not reporting the name of the changed application...in your test...Firefox.

I would be happy if the application could protect itself. But then your test says Comodo...passed. Passed what...your simple test? Ok, thank you for letting us know. But this says nothing again. Since there could be a method to bypass comodo's self protection. So I consider misleading your results. I agree that you provide a serious indication..but nothing more. If your test was more complete then I could understand and believe it more. If I doubt about Matousec's tests, I think you can immagine that I doubt about yours.

My understanding is that you go nowhere without layered security. PCTools Firewall could be a layer. Nobody promised total security with just using PCT Firewall. This is why I find misleading your replies and your test.

I would really like to thank you for spending your time letting me know. Although I'm not totally convinced and I find your post here and your test ( the other thread ) misleading.

 

Re: Windows 7/Compatible Firewalls

PCT does not effectively protect itself, simply because it does not protect its own system files.

it would be very simple to add into malware a small routine to search and replace unprotected files. Maybe PCT are awaiting such malware before that add such protection, but that would not be very proactive.

So if you believe "I'm also convinced that there will always exist a malware that can disable a certain version of a security application." then why the discussion?

I agree, but the proactive test are not run against 2 or more products installed, so why should I?

That sounds too much like an advertisement.

The term "Firewall" is now used loosely by vendors. But if you prefer, I will say PCT firewalls ESV is flawed.

That is what could be said about the "proactive"(leaktests) as they have the same logic. It is why basing a choice of firewall on those tests is not realistic.

I stated:-

What do you think the "Proactive" test actually test, the firewall or the HIPs?

The actual test is based on if an applications checksum as been altered. PCT firewall does not check that. You or anyone else can check that.

I have not mentioned Comodo. Please show me the post.

It is why I laugh at users who base their choice of security based on those leak tests, it was my main point, which you are agreeing with.

I agree on a layered protection, that again is why I do not follow the proactive(leak test) results.

I do not attempt to mislead anyone I simply make tests and post the results. The test shown are accurate and can be made by anyone, so would suggest you try it yourself rather than trying to state I am misleading, I take that as an insult.

- Stem

 

Re: Windows 7/Compatible Firewalls

You don't attempt but for me it remains misleading.
A test against a single application is by itself misleading even if the results are accurate. This is how I see it. You have every right to pubblish and post it but for me remains misleading.

The test cannot be made by everyone. So somebody will believe you just by reading. I don't doubt that your test is accurate. Your quoted words are not accurate. You consider certain that everybody can try that test. Well, this is not the case.

I'm really sorry that you feel insulted.

I have tried to not get insulted.

I could agree but this was not my point at all. You have quoted really short phrases of mine...so you have focused where you wanted.

If it is simple then you could tell me about one that you have seen disable PCT firewall. I believe that it is simple but the malware authors know that they have to face other things on protected systems before touching PCT firewall or any simple firewall. This is why they have invented more creative and deep ways to infect systems than your simple method described in the thread with the test.

For sure you cannot consider proactive basing your security on a single firewall.

You have missed my point again or I was not capable passing it to you. Actually this is why I say that your test is misleading = No need to point the finger against PCT firewall when we all know the truth. The truth is that for every version of a security application there might be in the wild a malware that can disable it, bypass it or make it crash.

I have already commented this. Actually you know, I already feel involved in a discussion about a product for which I have a relative interest. I have just installed it on my HTPC. If I try to defend here some things is a matter of principles and not an advertisement or other interests. If you believe that I have other interests defending PCTools ( which probably I do it in a bad way because of my limited technical knowledge ) then you are wrong.

Thank you for the correction.

The point here is what you test and how you call your tests.

I don't understand how this qualify to my reply:
"Checking PCT forums about related issues I found a thread which at least reports that some fixing will be done about the popup not reporting the name of the changed application...in your test...Firefox. "

I have to apologise about this. Another user mentioned it, using your method/test.

Yes I agree.

But still you post proactive tests although you don't follow them. This is a proactive test right, you said it before: "I agree, but the proactive test are not run against 2 or more products installed, so why should I?"

I quote a phrase you wrote on the other thread ( test ):
"Most on the forum will know that I dont judge a firewall by its leaktest ability, not because I think HIPs are of no use, far from that, I believe a well implemented HIPs is a good layer of protection if it is implemented well and the user can understand the implementation and make correct decisions. I just wonder at times if the coding of certain security applications are purely based around the leaktests and not real world threats"

Well, I agree absolutely. But then your test follows those words. Believe me what remains to the user is not your words above but the result of the test.
That test and your initial post in this thread:
1. you mention only PCT firewall
2. you don't tell us how common is for security application to not protect themselves or not doing it correctly.
3. you don't make clear how fragile self protection is.
4. you don't take other parameters in consideration...like OS and added layers of security ( this refered only to your post in this thread..not to the test ).
5. your role of moderator adds to some extend additional credibility to your posts which sometimes could be misleading especially when you are posting so clearly and negatively for a product.

Said all these things, I believe I made a mistake continuing this discussion. Again I am sorry if you feel insulted...not my intention, although your intention was only to insult me with the advertisement thing. Please reply back if you wish but I want to make clear that I'm out of this ( although I am the kind of guy that never leaves the battlefield... I do it because I have a high respect of this forum and the rest of the community members. )