new leaktest available : Ghost v1.0

yes it is.

For proof, i do all my test with full access granted to IE/explorer/Mozilla, and even with that, when leaktests try to use trusted apps (it's the purpose), firewalls sees them and block them.

To quote few, there Tooleaky, FireHole, PcAudit, PCAudit v2, and even Ghost, which are blocked whereas targetted apps are trusted.

It's the purpose of leaktest to target a fully trusted app to see if your firewall
block it or not.

EDIT : my favourite firewall? the one which fits better to my needs

 

Tss tss come now man - he's a FIREWALL TESTER. Objectivity is crucial, so IF he does have a favourite FW, he certainly shouldn't - and won't - name it!

 

So what firewall fits your needs?

 

i can say i have firewall setups of all firewalls i am evaluating

i can say too that my favourite firewall by far is my NetFilter on my Linux box, that i still tweaking sometimes.

About personal firewalls, i have one which fits exactly to my needs, and all i can say, is that me and mvdu haven't at all the same needs

 

Will you let me know in a private message?

 

damn, i don't remember how to use my mouse to click on PM...
Oh no, critical error, "your mouse driver has expired"

 

You can go into my profile and click Send this member a private message if you want.

I just tried OPP 2.0 with PCAudit 2, and although it fails, not everything was transmitted (the screen shot wasn't.) Interesting.

 

Indeed, may be it is _again_ a buggy feature.
Is there a firewall without bugs ? i am wondering.

That a firewall fails a leaktest because it doesn't have features to handle it, ok, but if it fails because of a buggy feature... it's sad.
But it could be just a network error, not necessarely a bug.

Have you tried my small program above with OPP?

 

Yes - ghost goes right past Outpost. Am using ZAP right now, but I remembered to try with Outpost.

 

I was talking mainly about the Unhidden_IE_Launch.exe (small test program).
If i'm right, OPP shouldn't even block it.

 

Hi gkweb,

Another nice tool you have there!

I tried your Ghost against TinyFW and with all modules enabled it says that ghost was trying to spawn IE and prompted allow/deny so I had an avenue of blocking it.

When I tried it with the Windows Security module disabled it went to the website without any popup, etc.

 

What about MBtest - was is successfully tested against all firewalls, or did it fail to run properly with some of 'em?

 

thanks you Dan Perez

@Morgott
no, MBtest was ran successfully, if not i wouldn't write "failed" in the results page
But it doesn't work on your comp, you have to modify the C++ sources to write your MAC adresse in it.

 

O, OK. It's just that in the previous results page (which included the 'Outbound' test results), some firewalls passed Outbound, but now they fail MBtest.
Take ZA: it is supposed to have passed the Outbound test, but now I see it fails MBtest, although Outbound and MBtest are supposed to be equivalent. So I was wondering how it failed the test - does MBtest have 2 different tests or something? Does ZA still have "low level" capabilities??

 

MBtest and Outbound are equivalent.
They both use WinpCap library to send packet to the network interface.

The difference is the OS, apprently it is easier to catch on Win9x than on 2000/XP, i can't tell you why, i'm not a firewall developper

Unfortunaly i don't have any Win9x/Millenium system currently to test Outbound, but if someone has one, go to my website "MBtest" page, download the two WinpCap files, copy them where it is specified, and run Outbound.
At the end, post your results here.
(it's the network filtering which is stressed with this kind of leaktest, not the application filtering directly).

 

I just saw on the website author that outbound doesn't use the "npf.sys" from WinpCap whereas MBtest does, may be it's the difference (both use "packet.dll").

 

actually I tried running Outbound on Win2k (had 'outbound.exe' & 'packet.dll' in the same folder), and the test was able to launch, then ZA popped up a warning about Outbound.exe trying to access the Net, I was able to click 'no' & block it, but then Outbound sort of...froze: it told me to select an adapter & click 'continue', but there was none in the list, and the 'continue' button was grayed out!
But I take it ZA was still able to see Outbound, even in Win2k - I think...

So I ask myself if ZA does have "low-level" capabilities, or instead if is it "stuck at the application filtering" level. Since it passes Outbound but fails MBtest), this seems contradictory. Could it be (yet) another bug in ZA?

-> And I'm just curious to know how it failed MBtest - are several different stages in this test, and ZA passed some but failed others? Or did it failed all steps?

 

all steps (8 different packets).

All firewalls failing MBtests were blocking at the best case two packet with protocols 47 and 50 (one is GRE protocol), or nothing
(all was checked with a Sniffer, not on the same computer of course).

Outbound, from the author, can't works on XP because something is different, and apparently has troubles in 2000 too (as you noticed), so i won't do a test on a missworking leaktest and said firewalls pass it or not (but Outbound works fine on Win9x OS).

In the past, Tom Liston, Outbound author, highligted that ZA was simply blocking all not Winsock traffic instead of investigating the issue.
So, IF it is still the case that ZA didn't really handle low level but put again a makeshift (i said IF, i don't know), it wouldn't be surprising that it fail MBtest but not Outbound.

In addition Windows TCP/IP stack is different from Win9x to NT OSs, so i am not sure that you can say that a firewall having a low level network control on Win9x has still one in NT OSs, or atleast not necessarely as good.
Just see as how many firewall has DLL injection protection feature on NT OSs but not on Win9x ones, it's two different kind of OS.

It's again something interesting to investigate further, but i can't do all test in same time, i have to work with OPP for now...

EDIT : about your test, outbound needs _two_ files and put in system32 and system32/driver i think.

 

Hey, I read that article from Tom Liston - what a SHAME!

But at least they disclosed their means of "blocking" such tests with their beta-patch, by blocking all non-winsock traffic or something... But still, this is cheating!
First BID, then OPP, now ZA.
Now I wonder if there are any "honest" FWs out there, and who to trust...

But the article was written in 2001. I don't know if things are still the same today (I hope not, for if so I'll just stop using ZA once & for all).

Anyways I tested again Outbound against ZA: before Outbound malfunctionned, ZA warned me that Outbound.exe was trying to access the Net (destination: IP 127.0.0.1? That's me!?!), so I guess maybethis time it could really see the source of the leak, even in Win2k.

So I changed the MD5 signature of Outbound.exe with a hexeditor (changed some text within the file), just in case the test was blacklisted. Also changed the filename. Then ran the new, modified file ('Outbound2.exe) and again ZA warned that Outbound2.exe was trying to reach the Net.

Perhaps MBtest is trickier...

So now I'm confused. Very confused...

 

Don't try to know how firewalls works internally, no firewall vendors will tell you.
You can just regarding leaktests results do hypothesis.

Secondly, how to define this "features" ? cheating ? usefull ?
All depend of the point of view, i prefer to call them "shortcuts", which is really of help in real environnement for the end user, but in a way it's too a "cheat", because the problem isn't really adressed.

A lot to say just about that, i'm sure JV Morris could write you a book only on this question.

About BID, it was definitly cheating and unfair at the time.
About OPP, it's a makeshift enabled by default and can't be disabled in the GUI (you have to dig into ini files...)
About ZA, i don't know, their "OpenProcess" feature is a makeshift too, what doesn't mean it isn't usefull, and about their network filtering, the article is old now, who knows if they really adressed the issue ?
Only leaktests can tell us, but as you can see all results has to be verified
often to ensure their rightness.

The problem in fact, as Tom Liston highlited, is a marketting problem.
Let's say users rely on leaktests to rate firewalls, wouldn't it be really attracting to add one or two makeshift for make users happy to see their firewall "passing" leaktests ?
This is exactly why i go hunt makeshift or "shortcuts" to see the _real_
filtering strength.
From vendors point of view, if their firewall can pass many tests (leaktests, networks tests, scanners, etc...) then it is for them a lot of money earned.
And even if people as all leaktests authors tries to demonstrate that many firewall vendors are cheatting, do you seriously think that we can explain all of that, all thread about the subject at wilder forum, all my site, to an avereage user or a security beginner ?
It's too much to explain to convinced a beginner, so firewall vendors are safe from this point of view.
As i said my main purpose isn't to blame any firewall vendors, i just want to test firewalls and to report results, but when i see _how_ many leaktests are tried to be "passed", in my head i'm between :
- it's usefull features added for the security of the end user
- it's a cheat to pass more leaktests

How to know ? we can't.

If it's usefull features, why to waste time to add makeshift whereas they could investigate seriously the problem and make something really better than a shortcut ? May be because the majority of users just want that nothing can pass throught firewall overall feature without taking care of the network application filtering weaks ?

So many questions, i understand why you are more confused than before, and i will try to shed some light on the subject, the adventure continue, and it's difficult without having firewalls source code that i will never have

 

About Look ‘n’ Stop v2.05b1 and pcAudit v4.0.0.0, indeed it fails but the older versions of pcAudit are capable of being seen and denied.

 

Yes indeed, someone here has said the opposite ?

I'm continuing my thought about "cheat or feature?" :

This makeshifts are cheating i think only if the firewall vendors claims to pass leaktests using these "shortcuts", if not, may be they can just be seen as usefull features, that i use after all with SSM (a separate dedicated software).

 

pcAudit v4.0.0.0 uses different design to bypass Application Filtering Layer than the previous versions….

 

yea i know, all results are available on my website.
For now only ZA and NPF pass PCAudit v2 (4.0.0.0).

 

Perhaps we can...by asking the vendors for the source code - politely ?

Well, I'm am a newbie myself, but I ain't that easily fooled, No Sir!!

Are you SURE about that? Especially ZA, for if it does, it must certainly be a coincidence - I doubt they would have gone at great lengths to block such sophisticated leaks as the PcAudit2 type, yet apparently let it fail MBtest, which itself is a variant of Outbound, which ZA passes...man, really confusing...

As for NPF, it's an even deeper mystery - it passes PcAudit2 yet fails the more "primitive" PcAudit? Yeah, after all, why not...


Phantom:

Indeed, and Ghost as well. But according to what I read on the LnS features list, one might have thought it should have at least been able to pass the PcAudit2 test. So the question is, is this perhaps a bug of some sort, and if so - leaving the Copycat issue aside - is there a (permanent, not BID-fashion ) fix scheduled for PcAudit2-type leaks (and possibly Ghost-type leaks as well) in the immediate future?