new leaktest available : Ghost v1.0

[gkweb]

Hi there,

I release in a kind of "beta test" my new leaktest "Ghost".
In fact, my website ( http://firewallleaktester.webhop.net ) had a major update which is waiting your feedback first about "Ghost" leaktest before i put it online.
When all will be ok with "Ghost", i will add it to my site in same time of all others update (including new leaktests results).

Ghost beta test version available here :
http://perso.wanadoo.fr/jugesoftware/Ghost.exe
MD5 : D5F8069EEDC4AA75EE0F001D517DE972

I would want your input about it, if you think something need to be improved or explained, as well as of course results you can see with your firewall.

How it works ?
In a very simple way.
First step, It just call directly Internet Explorer, in the same way that the "test 2" from Wallbreaker or Tooleaky does, and which is seen by firewalls i evaluated.
Second Step, the "secret" :) , is that since firewall reacts to events, if you are quicker and can "disapears" you can put them in trouble to locate the source.
This is why "Ghost" rerun itself, modifying by the way his PID.

Ghost just reach one page, but by doing multiple time this trick, i was able to transmit data through firewalls whereas they were able to block it if done normally.

So i would want to know what do you think about it, and if it works on any OS (was tested on win98/Millenium/Win2000/XP) and if you think it's ready to add it to the site and release it officialy (not as beta test version).

Thanks you :)

EDIT : to test it, you have to give full access to IE and see what happens.

[mraka]

It hasn't leaked on Win2k with Sygate 5.5 or Outpost 2. If you put IE in trusted applications it will leak in both. Component control made no difference.

[gkweb]

-{ Quote: "If you put IE in trusted applications it will leak in both." }-

you have too :)

For understand why, you just have to know that when IE is fully trusted, direct access are nevertheless blocked (ex : "Tooleaky" leaktest).

So apparently Sygate Pro 5.5 and Outpost Pro v2 fails "Ghost", that was i found too, but let's wait more results.

thanks you for your testing.

[mvdu]

I tried it with ZAP 4.5, with Internet Explorer given access to the internet and network. ZAP sees internet explorer trying to use internet explorer to connect to the network, and the page doesn't load. It does not see the application Ghost, though.

[gkweb]

Hmm, strange, on my comp ZA PRo 4.5 see ghost.exe trying to launch
iexplore.exe, that made me thinking that ZA passes it.

I didn't well understand your sentence, even with IE having full access ZA block without seeing Ghost so without reason ?

[mvdu]

No it did not see Ghost - maybe our settings are different in some way.

[gkweb]

can you recall me your OS please ?
And btw your CPU speed.

thanks you :)

(i attached what i see on my comp)

[mvdu]

Never mind, when I saved the file on my hard drive, ZAP sees Ghost and stops it. Our results are the same, then. :)

[gkweb]

So if i understood right, ZA saw it the first time and then you blocked it definitly, and now ZA just block IE to open without saying it is coming from Ghost ?

Sorry i have difficulties to understand it's pretty late where i live :)

[mvdu]

When I just opened the file from your link, ZAP stops the page from loading, but it doesn't see Ghost.

When I save the file on my hard drive and run it, ZAP stops the page from loading AND sees Ghost.

[gkweb]

ok so ZA passes Ghost, thanks you ;)

[mraka]

It is good to standardize testing, especially when detecting certain features, but how many actually run IE as trusted? :D Now that itself is a risk ;D

[mvdu]

You're welcome. Have you tested this with Norton Personal Firewall? If you haven't, I have that on the downstairs computer, and can later give you results. Do you want results with automatic configuration of Internet Explorer?

[gkweb]

Of course i would be intesrested too.
As for any firewall i am evaluating i have already results on my side but they need to be checked by other people just in case :)

I think that the result with automatic configuration or full access for IE will lead to the same result for NPF _2004_ :)

[gkweb]

@mraka

of course in normal circumstances you should restrict your applications ;)

[gkweb]

to sume up :

no stability issue or any other pb on Win 98 / Millenium / Win 2000 / Win XP.

from here :
passed by : ZA Pro 4.5
failed by : Sygate Pro 5.5, Outpost Pro 2.0

from my tests :
passed by : ZA Pro 4.5, NPF 2004
failed by : Sygate Pro 5.5, Outpost Pro 2, Look'n'Stop 2.05b1, Kerio 4.0.8

Is anyone else can check results ?
(mvdu for example about NPF)

About how "Ghost" works, i will just add that in addition of the two steps i gave, when you click on "try" it creates an internal thread giving him the highest priority possible in windows (critical priority), and push code into it, to ensure to be the fastest possible.
"Ghost" leaktest is in fact more a "timing attack" than a complex trick.


If you think that all results are good, that there is no pb, and you have or haven't any comments, please tell me.

If no one see any pb with Ghost, i could update my site with Ghost and all the stuff currently waiting, i'm just waiting for you, i can't update it by adding a leaktest that makes computer to crash for example ;)

Thanks you :)

[mvdu]

Just tested with NPF, and got the same result - it passes Ghost.

[gkweb]

that was i found too, NPF and ZA passes it and other failes it.

EDIT : Ghost in fact tests where firewall system "hooks" are put,
if they are put too late in the application calling processus, they will missed Ghost.

[gkweb]

With "out of the box" settings, NPF passes it but not ZA, right ?

[gkweb]

ok, website updated, i quote the "news" page here :

-{ Quote: "
December 14 2003 : ** Major site update **

- The results page has been updated
- Now results are explained in an additional page (link on the results page)
- Leaktest "Wallbreaker v3.0" released
- New Leaktest "Ghost v1.0" released
- Categories page updated
- Firms pages updated
- all leaktest information pages updated, added : the leaktest categories, leaktests fingerprint
(MD5/SHA-1/SHA256)
- new "statistics" page with the most loved firewalls from users.

And soon a total rework of the website graphism to make it more user friendly.

if you find any mistakes in the results page, and that in addition you perfectly understood
on which criteria leaktests testing are based on, so that you can be sure of the mistake,
pls write an email to gkweb@wanadoo.fr with a subject like
[FWLT, Results error report] : OS/firewallname/leaktestname.
Then, in your mail, please detail your settings and all your firewall configuration
(why not with screenshots) which will help me to reproduce exactly the result you
found, i don't correct a result only by trust, i must be able to do it myself.

All results has been tested many times each.
However, i apologize in advance if you find any mistake, all that is necessary to be
done will be in the purpose to have right and fair results for all firewall evaluated.
" }-

Of course if you encounter any issue with Ghost, feel free to post it here.

I hope you will like all the new stuff, and that some people will find help on it.

regards,

gkweb.

[mvdu]

Under my test, the latest version of Kerio, 4.0.10, passes Ghost. Internet Explorer has access to the internet and network.

[gkweb]

email me for i can keep record and redo test, thx ;)

EDIT : only enable the "network security" under kerio and disable all other, including "system security" which is application monitoring.

[controler]

Look & Stop didn't stop it with default settings

con

[mvdu]

I don't think that should be disabled. Because it just watches for modifications like ZAP does with component control.

[gkweb]

i just redo test with 4.0.10 and it fails it.

The reason for what it should be disable is that intrusion detection and web filtering feature has nothing to do with leaktests.
In addition, if you want i even can create an executable for you, with "system security" enabled Kerio just check when an access is done, not an internet access, it has been discussed with you in another thread.

From the Kerio help :
-{ Quote: "
Kerio Personal Firewall controls all applications in the operating system, regardless of whether they are deployed into network communication or not.
" }-

about "system security" of course.

EDIT : And so, if it's only component related like you said, how can it detect Ghost whereas it doens't modify/inject any components ?

[mvdu]

Ok, I misuderstood what the "modify" part in Kerio meant. I agree that for your purposes at least, it fails.

[gkweb]

I know, after i had long discussion with you in an other thread, that you are more interested in the overrall firewall results, which i doesn't do i'm sorry.

For overall results, i like to go on http://www.firewall-net.com/en/ i don't know
if you know this one.

[Morgoth]

Gkweb:

Is it true that for Outpost to get 10/10 in the AWFT, explorer has to be restricted (no access to the Net AND no using other progs to access the Net)? I showed U the link once...

[gkweb]

on my comp, with IE and explorer fully trusted it got this result.
And about your link:


-{ Quote: "you will have to remove explorer.exe from your Partially Allowed Application" }-

I understand that you have to remove it from partially allowed, ... to fully allowed.

EDIT : OPP 2 settings : http://perso.wanadoo.fr/jugesoftware/settings.zip

[Morgoth]

Hey, got a fun about about ZA 4.5.530 ;D

Just tested it with AWFT and sure enought it failed some tests - 4 and 5.

But it's the way it failed that puzzles me.

I had component control set to MAX, Advanced Program Control enabled but OpenProcess control DISABLED.

As required, Explorer was given Net acces right AND the right to use other apps to access the Net.

With these settings, ZA failed 4(1 pt) and 5(3 pts!!!), thus losing 4 points out of 10.

But here's what's strange: upon BOTH tests, ZA popped in to aks whether I wanted to let explorer and components access the Network (the listed components were something like 'awftr1.dll' or something). But before I could reply, or even if I replied 'NO', the tests still leaked through!!!
In other words, ZA did seem to "see" the 4 & 5 tests, but could not stop them, even if I told it to.

How can that be explained? ???

[gkweb]

bug :)

sygate has one too with the "MBtest" leaktest, it sees it, it block it, but the first time all MBtest packets go trought it (verified with a sniffer).

To see something is one task, but when seen to freeze it properly is another.

[Morgoth]

Wow, that wuz a quick answer :)

So in other words, this is a design flaw in ZA, some sort of weakness that prevents it from blocking the leak properly? They don't seem to have corrected it even in v4.5??

If so, SHAME ON THEM!!! >:(
I was thinking of switching (back) to Outpost or LnS, but these 2 on the other hand apparently failed PCaudit2 & Ghost. Back to waiting I guess...

But are U sure that LnS fails PCaudit2? From what I read, v2.05b1 is supposed to be able to handle it. Or perhaps this will only be the case for the final v2.05...


BTW, what exactly are the AWFT tests 4 and 5? Are they similar to Thermite in some way?

[gkweb]

To choose one firewall because it fails this or this leaktest isn't a good idea, indeed since no one passes all leaktest, so you take no one firewalls right ? ;)

I don't think any firewall vendors sould have Shame, i think it's a pretty hard job to do something efficient in _every_ way (web, intrusion detection, network filtering, etc...).

About AWFT, quote from their site :
-{ Quote: "
One: Attempts to load a copy of the default browser and patch it in memory before it executes. Defeats the weakest PFs.
Two: Creates a thread on a loaded copy of the default browser. Old trick, but most firewalls still fail.
Three: Creates a thread on Windows Explorer. Another old trick, but almost every firewall still fail.
Four: Attempts to load a copy of the default browser from within Windows Explorer and patch it in memory before execution. Defeats PFs which require authorization for an application to load another one (succeeding on Technique 1) - Windows Explorer is normally authorized. This test usually succeeds, unless the default browser is blocked from accessing the Internet.
Five: Performs an heuristic search for proxies and other software authorized to access the Internet on port 80, loads a copy and patches it in memory before execution from within a thread on Windows Explorer. Very difficult test for PFWs!
Six: Performs an heuristic search for proxies and other software authorized to access the Internet on port 80, requests the user to select one of them, then creates a thread on the select process. Another difficult nut to crack for PFWs!
" }-

About Look'n'Stop and PCAudit v2, yes, it fails it whereas it has components control, but Outpost too is in the same case.

So i think it's hard to choose a firewall only on his leaktest results, until it really failed too much leaktests.

[Morgoth]

Actually, what I meant was, it's REALLY a shame for Zonelabs, because AWFT are NOT recent tests, and the issue about ZA failing the 4 & 5 test has apparently been known for a while - unless the bug only dates from version 4.5 (because versions 4 and prev. would not run at all on tests 4 & 5 !?!), but in that case, it's even worse since that would mean there's a new bug in version 4.5 that wasn't there in previous versions! >:(

Outpost, LnS & others may fail PCAudit2 & Ghost, but these 2 are recent tests at least, so it's not that "scandalous", I think.

One thing's for sure - as soon as another FW succeed in all the tests that ZA also passes (and hopefully, also on tests that ZA fails), I'll quit ZA, for I don't think they'll correct their AWFT flaw, nor their famous mem problem - at the time I'm writing this, vsmon has already swelled up to 25Mb RAM (it was only 6Mb at startup!)...

Oh, and yes I read the specs about the AWFT tests, but these are a bit hazy to me (I'm but a young Padawan, remember? :)), that's why I was asking about a more 'down-to-earth' explanation about the 4 & 5 tests. From what YOU have read, are they similar to other tests such as Thermite (or another test) in some way?

[gkweb]

First, i don't want to be responsible in public of people switching firewall because of me, i don't want to be in the aim of firewall vendors ;)

About AWFT:
-{ Quote: "
Four: Attempts to load a copy of the default browser from within Windows Explorer and patch it in memory before execution. Defeats PFs which require authorization for an application to load another one (succeeding on Technique 1) - Windows Explorer is normally authorized. This test usually succeeds, unless the default browser is blocked from accessing the Internet.
" }-
I think it's a thread injection (like Thermite) into Explorer.exe (not IE) putting in it iexplore.exe itself, so it bypass firewall which asks for explorer launching IE, because it doesn't launch it, it "hosts" it in his own process area
So this test could be seen as something like Thermite, but different because Thermite inject executable code which access the Internet, but here it's IE itself which is injected, a trusted application.

Notice i can be wrong, it's what i understood.


-{ Quote: "
Five: Performs an heuristic search for proxies and other software authorized to access the Internet on port 80, loads a copy and patches it in memory before execution from within a thread on Windows Explorer. Very difficult test for PFWs!
" }-

This test ask you first to browse on websites, while you do that it tries to see which running processes has access to the Internet on port 80 and is allowed.
Then it does like the other test, it loads a copy of the process inside explorer.exe (as a new thread) and patch it (how ? no information) before accessing the internet.
Which is difficult for firewall, is that it isn't necessarely your browser which could be used (i try with "success" with other processes other than my browser).

Unfortunaly, on both tests, we don't know how the executable is patched in memory.

[Morgoth]

-{ Quote: "First, i don't want to be responsible in public of people switching firewall because of me, i don't want to be in the aim of firewall vendors " }-

Fear not - objectivity is of the essence here, and you're doing a good job. So should you ever be in their line of fire, just stand your ground and fight the danger, soldier! ;D

Second, I don't want to confuse U in any way, but about Outpost (again, I know, but it's among my favourite FWs): the issue with AWFT is rather unclear, as some users setting explorer to 'fully trusted' have failed it, unless they enter 'explorer.exe' into a special .ini file which in turn adds some restriction to it, see this thread and scroll down:
http://www.outpostfirewall.com/forum/showthread.php?s=f30049bf32346712b8fe874b67b81cd5&threadid=8539&highlight=atelier

Makes sense too: if it could truly pass AWFT 10/10, then it would also pass Thermite, which it doesn't.

As for LnS, I guess it (really) passes AWFT 10/10 thanks to its new 'thermite patch'...

[gkweb]

sounds logic, i will investigate this issue, thanks you ;)

EDIT : someone else talked to me about that but i forgot in the meantime, to remember of 108 tests (6 FW, 12 leaktests + 6 tests AWFT) is not easy :)
So Outpost should apparently only pass the 4 tests ? so 4/10 ?
oups, really different than 10/10...
If i had to remove the makeshift protection in the ini file, OPP will it failed too "Tooleaky" ???
Really need investigation.

[Morgoth]

-{ Quote: "sounds logic, i will investigate this issue, thanks you

EDIT : someone else talked to me about that but i forgot in the meantime, to remember of 108 tests (6 FW, 12 leaktests + 6 tests AWFT) is not easy " }-

Reaaly sorry for the extra hassle, soldier ;D
'tis difficult to test all these FWs, I concede.

One way to find out about Outpost would be to set it so it can pass AWFT 10/10, while granting FULL rights to explorer, BUT without any extra "special" rules for explorer, and STILL be able to use:
1) The browser
2) EMULE to connect to edonkey servers.

[TAG97]

-{ Quote: " quoting: gkweb link=board=23;threadid=17802;start=30#msg110270 date=1071421816]
First, i don't want to be responsible in public of people switching firewall because of me, i don't want to be in the aim of firewall vendors ;)

Hi gkweb.
I just have to ask this question. After all the work you put in testing these Firewalls what Firewall do you use and why? Keep up the great work. Once again you proved one should never put Internet Explorer in Trust Mode.
Best Regards
Tim" }-

[gkweb]

To be sure i install OPP on my other family computer, and re do tests.
First, i'm "surprised" (may be the good word would be astonished...) that a so crucial config in Outpost (block hidden processes explorer.exe and iexplore.exe by default) is hidden into ini files of OPP folder and that there isn't any option on the GUI!

Such option is a makeshift that should be disabled to do leaktests testing, someone in the past talked me about it too but i forgot this "small" point which is in fact a vital point.

I installed OPP on a win2k machine, and i got this results :

Leaktest : passed
FireHole : passed
Tooleaky : failed ! (OPP was passing it by blocking a hidden iexplore process)
Yalta : passed
PCAudit : passed

AWFT :
1 - failed
2 - i wasn't able to do it (didn't recognize my default browser loaded)
3 - passed
4 - failed
5 & 6 : i wasn't able to test them, even while opening 10 browser AWFT always said me "browse a little then retry".

Thermite : failed
Copycat : failed
WB : failed
PCAuditv2 : failed
Ghost : failed

After uninstalling blue screen even in safe mode... i'm currently restoring a ghost image and doing windows updates.

Again, the "to block hidden process" is a makeshift, OPP doesn't even see "Tooleaky", damn, since how long i was fooled ?
(file : OPP directory\outpost.ini)

Sorry to not be able to continue tests today, i have to re configure my other computer.
If you can test on your side, please pot your results.

thanks you.

[Morgoth]

-{ Quote: "thanks you." }-

thank you WHO? ;)


Good job, soldier!

But I don't know if I should lament or rejoice. First ZA, now OPP. Man, it's about time woke up!

At least LnS is able to (truly) pass AWFT thanks to its 204b2 patch.

But there's one thing I still don't get:

What the hell is this 'block hidden processes' option - and how is it "vital"? Why does it HAVE to be disabled?
And most importantly, what does it mean??

[mvdu]

Again, all I care about is if the firewall CAN stop an app without being really restrictive. That's what helps me decide whether to use a firewall or not. Still, it looks like ZAP and Outpost have bugs, and that bothers me. It's hard to decide on a firewall to use.

[gkweb]

@Morgott

-{ Quote: "
What the hell is this 'block hidden processes' option - and how is it "vital"? Why does it HAVE to be disabled?
And most importantly, what does it mean??
" }-

this feature block any processes attempting to access the Internet if they are hidden, OPP doesn't see if a malicious software launch it, it doesn't see the leaktest, it doesn't even pass the leaktest idea.
Tooleaky idea has never be to launch IE in hidden mode, it could as well do his job with IE not hidden, and in this case OPP would fail it because IE isn't hidden (in both case it doesn't see Tooleaky, it just see an hidden process).

It's like the "OpenProcess" of ZA which monitore something without passing the leaktest, here, to block unconditionaly any hidden process to access the Internet _is not_ to pass the leaktest.

Sorry to can't explain more, i return to my other computer.

[Morgoth]

OK, but take Explorer 4 example.

What is a "hidden" Explorer process?
What's the difference with a normal process which we see in the taskbar?
Can I manually laucnh ANY application as 'hidden', say by typing something like the following command line:
'explorer.exe /hide' or 'emule.exe /hide'? (just to illustrate) ???

[mvdu]

Well, it all depends on how someone defines "pass the leaktest." My ZAP asks me if I want to allow an app to use a process.

[Morgoth]

OK, but take Explorer 4 example.

What is a "hidden" Explorer process?
What's the difference with a normal process which we see in the taskbar?
Can I manually laucnh ANY application as 'hidden', say by typing something like the following command line:
'explorer.exe /hide' or 'emule.exe /hide'? (just to illustrate) ???

[gkweb]

A hidden process is decided in the code, when you launch it.
It's a process which runs normally, but all his windows (his GUI) are hidden to the user (but appears in the task manager).

example in "purebasic" :
RunProgram(IE,"","", #PROG_HIDDEN)

"hidden" is a process property, it's an information which can be retrieve by another program, in our case OPP.

OPP blocks any hidden process to access the internet.
The fact is that OPP hasn't at all any program launch monitoring, i mean that if i launch IE normally, as any legitimate program would do, OPP doesn't see it, it blocks _indirectly_ Tooleaky by blocking directly a hidden process : IE.

Try this test program (quicly written):
http://perso.wanadoo.fr/jugesoftware/Unhidden_IE_Launch.exe

it launch IE _normally_ on http://www.apache.org

_Any_ firewall with application launch monitoring (xx.exe launches yy.exe to access the internet, do you want to allow it?" will easily and without any pb block this simple executable (it could be seen as Tooleaky but not hidden).

If someone want to try it with OPP, feel free to post your result here.

[Morgoth]

-{ Quote: "OPP blocks any hidden process to access the internet" }-

so it would seem it is "cheating" more like - as ZA with 'OpenProcess' against Copycat.

U once said that an OpenProcess can be legitimate even when used to inject code into a process.

So the question is, can hidden processes also be legittimate?

[gkweb]

OPP isn't cheating really, this feature could be interesting for the end user, DLL injection and thread injection could be legitimate, and a hidden process too, but i haven't any example of software doing it (hidden process).

In the reality, this feature could be usefull, i don't say Agnitum is cheating, i just say that this feature block many leaktests, not by seeing them, just by seeing at the end a hidden process, and by blocking it.
OPP doesn't even popup you, it just block it silently.

an example for make you understand : if i run in hidden mode all Wallbreaker test, OPP would pass Wallbreaker without seeing at all Wallbreaker, just because at the end it would be a hidden iexplore.exe or explorer.exe

[mvdu]

So, do you think it is possible for a firewall to pass all these apps while having both Windows Explorer and Internet Explorer allowed? Is it just laziness on the firewall makers' part?

I'd also like to know if you have a favorite firewall, gkweb. :)

[gkweb]

yes it is.

For proof, i do all my test with full access granted to IE/explorer/Mozilla, and even with that, when leaktests try to use trusted apps (it's the purpose), firewalls sees them and block them.

To quote few, there Tooleaky, FireHole, PcAudit, PCAudit v2, and even Ghost, which are blocked whereas targetted apps are trusted.

It's the purpose of leaktest to target a fully trusted app to see if your firewall
block it or not.

EDIT : my favourite firewall? the one which fits better to my needs :)

[Morgoth]

-{ Quote: "I'd also like to know if you have a favorite firewall, gkweb. " }-

Tss tss come now man - he's a FIREWALL TESTER. Objectivity is crucial, so IF he does have a favourite FW, he certainly shouldn't - and won't - name it! :o

[mvdu]

So what firewall fits your needs? ;)

[gkweb]

i can say i have firewall setups of all firewalls i am evaluating 8)

i can say too that my favourite firewall by far is my NetFilter on my Linux box, that i still tweaking sometimes.

About personal firewalls, i have one which fits exactly to my needs, and all i can say, is that me and mvdu haven't at all the same needs ;)

[mvdu]

Will you let me know in a private message? ;)

[gkweb]

damn, i don't remember how to use my mouse to click on PM... ???
Oh no, critical error, "your mouse driver has expired" :'(

[mvdu]

You can go into my profile and click Send this member a private message if you want.

I just tried OPP 2.0 with PCAudit 2, and although it fails, not everything was transmitted (the screen shot wasn't.) Interesting.

[gkweb]

Indeed, may be it is _again_ a buggy feature.
Is there a firewall without bugs ? i am wondering.

That a firewall fails a leaktest because it doesn't have features to handle it, ok, but if it fails because of a buggy feature... it's sad.
But it could be just a network error, not necessarely a bug.

Have you tried my small program above with OPP?

[mvdu]

Yes - ghost goes right past Outpost. Am using ZAP right now, but I remembered to try with Outpost.

[gkweb]

I was talking mainly about the Unhidden_IE_Launch.exe (small test program).
If i'm right, OPP shouldn't even block it.

[Dan Perez]

Hi gkweb,

Another nice tool you have there! ;)

I tried your Ghost against TinyFW and with all modules enabled it says that ghost was trying to spawn IE and prompted allow/deny so I had an avenue of blocking it.

When I tried it with the Windows Security module disabled it went to the website without any popup, etc.

[Morgoth]

What about MBtest - was is successfully tested against all firewalls, or did it fail to run properly with some of 'em?

[gkweb]

thanks you Dan Perez ;)

@Morgott
no, MBtest was ran successfully, if not i wouldn't write "failed" in the results page ;)
But it doesn't work on your comp, you have to modify the C++ sources to write your MAC adresse in it.

[Morgoth]

O, OK. It's just that in the previous results page (which included the 'Outbound' test results), some firewalls passed Outbound, but now they fail MBtest.
Take ZA: it is supposed to have passed the Outbound test, but now I see it fails MBtest, although Outbound and MBtest are supposed to be equivalent. So I was wondering how it failed the test - does MBtest have 2 different tests or something? Does ZA still have "low level" capabilities??

[gkweb]

MBtest and Outbound are equivalent.
They both use WinpCap library to send packet to the network interface.

The difference is the OS, apprently it is easier to catch on Win9x than on 2000/XP, i can't tell you why, i'm not a firewall developper :)

Unfortunaly i don't have any Win9x/Millenium system currently to test Outbound, but if someone has one, go to my website "MBtest" page, download the two WinpCap files, copy them where it is specified, and run Outbound.
At the end, post your results here.
(it's the network filtering which is stressed with this kind of leaktest, not the application filtering directly).

[gkweb]

I just saw on the website author that outbound doesn't use the "npf.sys" from WinpCap whereas MBtest does, may be it's the difference (both use "packet.dll").

[Morgoth]

actually I tried running Outbound on Win2k (had 'outbound.exe' & 'packet.dll' in the same folder), and the test was able to launch, then ZA popped up a warning about Outbound.exe trying to access the Net, I was able to click 'no' & block it, but then Outbound sort of...froze: it told me to select an adapter & click 'continue', but there was none in the list, and the 'continue' button was grayed out!
But I take it ZA was still able to see Outbound, even in Win2k - I think...

So I ask myself if ZA does have "low-level" capabilities, or instead if is it "stuck at the application filtering" level. Since it passes Outbound but fails MBtest), this seems contradictory. Could it be (yet) another bug in ZA?

-> And I'm just curious to know how it failed MBtest - are several different stages in this test, and ZA passed some but failed others? Or did it failed all steps?

[gkweb]

all steps (8 different packets).

All firewalls failing MBtests were blocking at the best case two packet with protocols 47 and 50 (one is GRE protocol), or nothing
(all was checked with a Sniffer, not on the same computer of course).

Outbound, from the author, can't works on XP because something is different, and apparently has troubles in 2000 too (as you noticed), so i won't do a test on a missworking leaktest and said firewalls pass it or not (but Outbound works fine on Win9x OS).

In the past, Tom Liston, Outbound author, highligted that ZA was simply blocking all not Winsock traffic instead of investigating the issue.
So, IF it is still the case that ZA didn't really handle low level but put again a makeshift (i said IF, i don't know), it wouldn't be surprising that it fail MBtest but not Outbound.

In addition Windows TCP/IP stack is different from Win9x to NT OSs, so i am not sure that you can say that a firewall having a low level network control on Win9x has still one in NT OSs, or atleast not necessarely as good.
Just see as how many firewall has DLL injection protection feature on NT OSs but not on Win9x ones, it's two different kind of OS.

It's again something interesting to investigate further, but i can't do all test in same time, i have to work with OPP for now...

EDIT : about your test, outbound needs _two_ files and put in system32 and system32/driver i think.

[Morgoth]

Hey, I read that article from Tom Liston - what a SHAME! >:(

But at least they disclosed their means of "blocking" such tests with their beta-patch, by blocking all non-winsock traffic or something... But still, this is cheating!
First BID, then OPP, now ZA.
Now I wonder if there are any "honest" FWs out there, and who to trust...

But the article was written in 2001. I don't know if things are still the same today (I hope not, for if so I'll just stop using ZA once & for all).

Anyways I tested again Outbound against ZA: before Outbound malfunctionned, ZA warned me that Outbound.exe was trying to access the Net (destination: IP 127.0.0.1? That's me!?!), so I guess maybethis time it could really see the source of the leak, even in Win2k.

So I changed the MD5 signature of Outbound.exe with a hexeditor (changed some text within the file), just in case the test was blacklisted. Also changed the filename. Then ran the new, modified file ('Outbound2.exe) and again ZA warned that Outbound2.exe was trying to reach the Net.

Perhaps MBtest is trickier...

So now I'm confused. Very confused...

[gkweb]

Don't try to know how firewalls works internally, no firewall vendors will tell you.
You can just regarding leaktests results do hypothesis.

Secondly, how to define this "features" ? cheating ? usefull ?
All depend of the point of view, i prefer to call them "shortcuts", which is really of help in real environnement for the end user, but in a way it's too a "cheat", because the problem isn't really adressed.

-{ Quote: "
First BID, then OPP, now ZA.
Now I wonder if there are any "honest" FWs out there, and who to trust...
" }-

A lot to say just about that, i'm sure JV Morris could write you a book only on this question.

About BID, it was definitly cheating and unfair at the time.
About OPP, it's a makeshift enabled by default and can't be disabled in the GUI (you have to dig into ini files...)
About ZA, i don't know, their "OpenProcess" feature is a makeshift too, what doesn't mean it isn't usefull, and about their network filtering, the article is old now, who knows if they really adressed the issue ?
Only leaktests can tell us, but as you can see all results has to be verified
often to ensure their rightness.

The problem in fact, as Tom Liston highlited, is a marketting problem.
Let's say users rely on leaktests to rate firewalls, wouldn't it be really attracting to add one or two makeshift for make users happy to see their firewall "passing" leaktests ?
This is exactly why i go hunt makeshift or "shortcuts" to see the _real_
filtering strength.
From vendors point of view, if their firewall can pass many tests (leaktests, networks tests, scanners, etc...) then it is for them a lot of money earned.
And even if people as all leaktests authors tries to demonstrate that many firewall vendors are cheatting, do you seriously think that we can explain all of that, all thread about the subject at wilder forum, all my site, to an avereage user or a security beginner ?
It's too much to explain to convinced a beginner, so firewall vendors are safe from this point of view.
As i said my main purpose isn't to blame any firewall vendors, i just want to test firewalls and to report results, but when i see _how_ many leaktests are tried to be "passed", in my head i'm between :
- it's usefull features added for the security of the end user
- it's a cheat to pass more leaktests

How to know ? we can't.

If it's usefull features, why to waste time to add makeshift whereas they could investigate seriously the problem and make something really better than a shortcut ? May be because the majority of users just want that nothing can pass throught firewall overall feature without taking care of the network application filtering weaks ?

So many questions, i understand why you are more confused than before, and i will try to shed some light on the subject, the adventure continue, and it's difficult without having firewalls source code that i will never have :)

[Phant0m]

About Look ‘n’ Stop v2.05b1 and pcAudit v4.0.0.0, indeed it fails but the older versions of pcAudit are capable of being seen and denied.

[gkweb]

Yes indeed, someone here has said the opposite ? :)

I'm continuing my thought about "cheat or feature?" :

This makeshifts are cheating i think only if the firewall vendors claims to pass leaktests using these "shortcuts", if not, may be they can just be seen as usefull features, that i use after all with SSM (a separate dedicated software).

[Phant0m]

pcAudit v4.0.0.0 uses different design to bypass Application Filtering Layer than the previous versions….

[gkweb]

yea i know, all results are available on my website.
For now only ZA and NPF pass PCAudit v2 (4.0.0.0).

[Morgoth]

-{ Quote: "How to know ? we can't." }-

Perhaps we can...by asking the vendors for the source code - politely ? ;D

-{ Quote: "And even if people as all leaktests authors tries to demonstrate that many firewall vendors are cheatting, do you seriously think that we can explain all of that, all thread about the subject at wilder forum, all my site, to an avereage user or a security beginner ?" }-

Well, I'm am a newbie myself, but I ain't that easily fooled, No Sir!!

-{ Quote: "For now only ZA and NPF pass PCAudit v2 (4.0.0.0). " }-

Are you SURE about that? Especially ZA, for if it does, it must certainly be a coincidence - I doubt they would have gone at great lengths to block such sophisticated leaks as the PcAudit2 type, yet apparently let it fail MBtest, which itself is a variant of Outbound, which ZA passes...man, really confusing...

As for NPF, it's an even deeper mystery - it passes PcAudit2 yet fails the more "primitive" PcAudit? Yeah, after all, why not... :-\


Phantom:
-{ Quote: "About Look ‘n’ Stop v2.05b1 and pcAudit v4.0.0.0, indeed it fails but the older versions of pcAudit are capable of being seen and denied. " }-

Indeed, and Ghost as well. But according to what I read on the LnS features list, one might have thought it should have at least been able to pass the PcAudit2 test. So the question is, is this perhaps a bug of some sort, and if so - leaving the Copycat issue aside - is there a (permanent, not BID-fashion >:() fix scheduled for PcAudit2-type leaks (and possibly Ghost-type leaks as well) in the immediate future?

[Phant0m]

Look ‘n’ Stop v2.05b1 Feature-List didn’t cover pcAudit v4.0.0.0 or Copycat, so no bug. And as for the most commonly asked question, fix when? I’m just as clueless as the next fella! ;)

Regards,

[Morgoth]

Then a dark day for firewalls it is comrades, for so far none can avert all existing threats...

Leaktests win once again.

So for now:
Leaktests 1 - 0 Firewalls . :'( :'(

[gkweb]

-{ Quote: "
Quote:
For now only ZA and NPF pass PCAudit v2 (4.0.0.0).


Are you SURE about that? Especially ZA, for if it does, it must certainly be a coincidence - I doubt they would have gone at great lengths to block such sophisticated leaks as the PcAudit2 type, yet apparently let it fail MBtest, which itself is a variant of Outbound, which ZA passes...man, really confusing...
" }-

Outbound/MBtest, ran (on my tests) two different kind of OS.
Outbound uses a ".dll" ans a ".vxd" in Win9x, MBtest use a ".dll" and a ".sys". As i said, results need to be checked.
When i tested ZA, it passed PCAv2 without anything strange or weird, for each apps trying to access the Internet ZA saw a new component which was the PCAudit dll.

-{ Quote: "
As for NPF, it's an even deeper mystery - it passes PcAudit2 yet fails the more "primitive" PcAudit? Yeah, after all, why not...
" }-

if you go on my site, in the results page there is a link to a "detail/explanation of results" on which i have written exactly what you say.
Even after many many try... i wasn't able to pass PCAudit v1.
For PCAudit v2, NPF like ZA saw each time that there was a new component, the PCAudit dll.

-{ Quote: "
Then a dark day for firewalls it is comrades, for so far none can avert all existing threats...

Leaktests win once again.

So for now:
Leaktests 1 - 0 Firewalls .
" }-

There is _so much_ ways to escape from a computer that firewall vendors
will always have a hard work to do.
Will ever firewall wins against leaktests ? I think it's a never ending cycle where leaktests are released first, and firewall tries to block them after.

But i keep the faith, may be, one day... ;)

[Morgoth]

Yeah, FW vendors are working hard, no doubt.

May the Force be with them.

But as U said, many are tempted by the Dark Side and and least one (BID) has succumbed to it. ;D

[gkweb]

I hope to never be tempted by the dark side too, imagine, i would just plug off from the internet and suddently all leaktests passed ;D

[Morgoth]

-{ Quote: "[...] imagine, i would just plug off from the internet and suddently all leaktests passed " }-

That's it! You have found the ULTIMATE SOLUTION!

Why did FW vendors make it so complicated when the solution was under their noses all this time?

On the other hand, imagine a trojan that could leak out even with the Net connection severed - LOLOL that would surely be a trojan spawned in Hell itself ;D

[gkweb]

Leaktests solution, copyrighted under gkweb licence.

[Morgoth]

"Beta" version?

[gkweb]

Of course, it may have bugs, like bad cutting the wire and let you a half of your bandwidth, in very rare cases.

Fixed in the next version.

[Morgoth]

Now that's what I call "sharp" wit 8)

[gkweb]

what ?

[Morgoth]

Let's just say we're talking about "cutting-edge" technology...

[gkweb]

;D

[mvdu]

Hi again, gkweb - KAV detects ghost as a virus/trojan. Since it's obviously not one, maybe you should straighten this out with Kaspersky.

[gkweb]

:o

virus ? trojan ?

Ghost is just packed with UPX, yes, but this doesn't turn it into a threat.
Thanks you for the info, i will try to find an email adress to write to Kaspersky.

[mvdu]

It does the same thing with the Firehole leak test. It's annoying, because you have to pause the AV to test. It detects Ghost as Exploit.Win32.Firehost. Glad you are writing them about it.

[Phant0m]

It is an Exploit, and since it’s labelled an Exploit and not a virus/Trojan/worm, you shouldn’t worry about it.

[gkweb]

I just asked them what does it means exactly, just in case, and if it's a mistake, i request them to remove it from their database.

If i have an answer i will post it here.

[mvdu]

Yes, if you have an answer, I'd like to see it.

Personally, I'd have a hard time using a firewall that can't pass both PCAudits. Those are important tests. Ghost seems to exploit real vulnerabilities in the basic protection, so failing that test also bothers me a lot.

[gkweb]

my email to Kaspersky :
-{ Quote: "
Hi,
>
> I am the author of a website about Windows personal firewalls and
> leaktests program (small demonstration program trying to bypass firewall
> to demonstrate firewall vulnerabilties).
>
> I am too the author of two leaktests program, one of them is named Ghost
> (ghost.exe) available on my website : http://firewallleaktester.webhop.net
>
> A Kaspersky user warn me that my leaktest was labelled as
> "Exploit.Win32.Firehost" whereas it has nothing malicious, it is just a
> test that a user runs willingly.
> This same use warn me too that another leaktest "FireHole"
> was see as an exploit too.
>
> Is this a mistake ? if yes could you remove it from your threats database ?
>
> Thanks to have read me, i'm looking forward to you.
>
> Best regards,
>
> Guillaume.K (gkweb)
" }-


answer from Kaspersky :

-{ Quote: "
The programs you have describes as 'leaktests' do actually contain exploit parts that try to breach the computer's security. It is not a false positive detection that we identify them as malicious ones. With a little modification these programs may posess a real threat to our users.

We are going to keep the detection routines for both 'ghost' and 'firehole'.


Sincerely yours,
Igor Soumenkov
Virus analyst

Kaspersky Lab Ltd
Moscow, Russia
Tel/Fax: +7 (095) 797-8700
E-mail: newvirus@kaspersky.com
Internet: http://www.kaspersky.com, http://www.viruslist.com
" }-

what do you think about it ?

[gkweb]

Outpost Pro v2 problem :
--------------------------------

Outpost has a outpost.ini file in his directory with inside by default explorer.exe and iexplore.exe in the "block hidden process" area.

With this default settings (that you can't modify in the GUI) it has 10/10 AWFT score and passes Tooleaky.

Without this settings, OPP has 5/10 AWFT score and fails Tooleaky.


From my point of view the fact to block hidden processes to access the Internet is again a makeshift since the threat isn't identitied, the process isn't block while accessing the network because it is "remotely" maliciously used but just because it is hidden.
Of course, as for all other firewall makeshift protection this feature is really interesting and usefull in a real environnement, but what is your opinion about it ?

- it's a makeshift, it doesn't pass leaktests, results are 5/10 and Tooleaky failed
- it's a legit protection, results are 10/10 and Tooleaky passed.

thanks you for your input, it will directly affect the official results page.

EDIT : without this options enabled, OPP seems to not have application launching monitoring at all (trojan.exe -> trusted.exe -> internet)

[Morgoth]

My jury finds Outpost guilty of failing the AWFT test at 5/10 by a unanimous 1/1 vote ;D

[gkweb]

Anyone else? mvdu ?

[Morgoth II]

I agree with Morgoth ;D ;D

[mvdu]

My vote - failing AWFT because Windows Explorer has to be off of the partially allowed applications list.

[gkweb]

it would lead to very weird following results :

"out of the box" settings : AWFT 10/10
"highest settings" : AWFT 5/10 !

Very funy case...

[gkweb]

so it will be :

Outpost PRo results "out of the box"/"highest settings":
Tooleaky : failed
AWFT : 5/10

If someone disagree, tell it know or keep the silence for ever lol
I find amazing that nowadays a firewall without application launching monitoring can exists :o
Every application/trojan launching IE in a visible manner won't be blocked by OPP.

[Aggressor]

-{ Quote: "If someone disagree, tell it know or keep the silence for ever" }-

hehe - you make it sound like a wedding or something ;D

5/10 it shall be then - a sad day for OPP - I used to like it, and still do somewhat, but what has to be done has to be done.

Better include a note under the test results to explain the reason, though...

[gkweb]

-{ Quote: "
Quote:
If someone disagree, tell it know or keep the silence for ever

hehe - you make it sound like a wedding or something
" }-

that was my purpose ;)

the fact to have 5/10 or bad leaktest score doesn't mean that the overall firewall is bad, it is just a result on a particular feature.
And of course i will add details in the page made for that.

But i'm still waiting, with the chrismast not everyone are browsing wilders forum but addicted persons that we are :)