Leaktests

Hi
Tried some leaktests. These are the results:

DNStest FAIL
Leaktest 1.2 PASS
OUtbound PASS
PC Flankleaktest FAIL
Surfer FAIL
Tooleaky PASS
Yalta PASS
Awft FAIL (4,5,6) PASS (1,2,3)

 

Was that with it set to automatic or interactive filtering? Does it make the firewall more/less effective depending on which setting you have? Just curious.

 

It was all based on interactive. Did not test on automatic. I'll try later

 

Yes, I can confirm it fails those tests also.
The GRC test everyone loves so much, is actually a pretty poor indicator of the firewall, if you want the Real test, go to Matousec.com and pick up some of the self test modules.
That's probably where you got those, isn't it "The One"?

It would be very nice to hear from any ESET personnel on these results. The only firewall to really pass the tests was Comodo, which isn't Vista ready yet (Alpha stage still), and Jetico.
I had to laugh tho...the Windows Firewall scored a BIG FAT 0 out of a possible 9625 points!!!!!!
LOLOLOLOLOLOLOLOLOLOL!!!!!!!!!!!!!!!!!

For ESS to score well you pretty much have to deny everything both directions, or it will fail some of the tests. I've been considering going with NOD32 for AV and Comodo for firewall when it's released, unless ESS can pick it up on the firewall side of things.

 

Eset is well aware of leaktest results in automatic and interactive modes since the release of beta 1 including tests on PCFlank and from Matousec as well as results from AWFT a firewall testing program
I assume that future releases will cure the test results over several more beta releases

 

Hi nellie

Yes, I hope that Eset developers have been made aware of these results awhile back, but "assuming" that they'll be addressed in future releases and actually hearing from Eset people that they're working to address those results, are 2 totally different things.
I...like you, certainly hope that Eset will include full HIPS (Host Intrusion Prevention System) in future releases, and seriously look at Matousec testing for some advice on their product. It's a thorough test setup and will be helpful in guiding them to a better firewall product.

One thing tho, and not meant to be at all condescending, nor is it a Flame of any kind, but being a member for a month and with 3 total posts, that's quite the statement to make. It makes you sound like an "insider" of some sort, or have access to info the rest of us don't have.

 

I took them from firewallleaktester.com but those are the same.
When using a IDS ESS will detect them.

 

If you're behind a NAT router, surely all these leak test are pointless?

 

Hi,

Er, no. Leaktests are used to test how the firewall protects against bypassing methods, that can send data out. Being behind a NAT helps preventing inbound attacks, but not outbound data, in this case, the data the leaktests (or any malware using the same kind of mechanism - which is the point of leaktests, see how a firewall can resist to techniques malware could use) send.

So being behind a NAT doesn't protect against them.

 

Hi IcePanther

Yes, you're 100% correct there. Unfortunately, too many people think that very same thing about NAT routers, or Router firewalls.
I tested new Malware varients for 3 years for Adaware, Spybot, and HijackThis. At times I had over 400 infected objects to test on a sandboxed, isolated system, and every single month these things got more clever and far more complex in their methods. Gone are the days of simple Home Page hijackers, and simple dll's that deliver a few popup advertisements.
These Malware writers (includes virus & trojans too) are now highly educated in the lowest level of OS workings, and their products these days show an extremely high level of obfuscation and complexity. The major varients are comparable to highly polished and advanced codeing practices of major legit software companies, and it's getting harder to keep them out every day. Inbound is not enough anymore and anyone who thinks so is going to be surprised someday very soon.

I was always one who thought that I knew enough about security to never fall for these things so I'd never get infected. I've stopped thinking like that nowadays, it matters not how much you think you know anymore...these things are so clever and complex, they can get past anything for awhile. All it takes is a second someday, and anyone can fall victim now.

 

Leaktests are the biggest waste of time - running a completely non-malicious exe on your company by choice, that does something non-malicious, and you expect the product to alert ? and worst of all ask the user "what would you like to do (with this non-malicious program) - Allow/Block ?". ALLOW. If you look at the comparatives, you will notice that the big boys Symantec, McAfee all at the bottom, since they realized long ago that for the large number of users they have, popping up such an alert is a colossal waste of time.

 

IMO sending informations from my PC without my knowledge is far from being non-malicious. So I am very interested in leaktests and I think those tests are very important. I'll never use a firewall with a "very poor" level of protection.

 

I didn't know the leaktests send information without user's knowledge , when users have just run them to see how these tests will send traffic/info. Wow , what a malicious test ?!

 

I am sorry. You misunderstand.
I didn't say " leaktests send information " , just " sending information from my PC" if you can feel the difference.
Of course leaktests are run with user's intention but that is done to test a real threat behavior.
Hope I make myself clear now.

 

No , I didn't. I just pointed you out how it looks like.

Well , it's much better now

 

The leaktest apps dont send out any private information. Do they steal keystrokes ? No. Do they steal game Serials ? No. Do they still Quicken Info ? No. Then they are non-malicious.

Like I said, those leaktest apps are a waste of time.

 

Zombini,

You've got the right not to find leaktests useful, but many people will differ, and if you still don't understand how leaktests can prove useful despite them not being malicious, okay, but don't repeatedly hammer into the forum that they're useless without proving your point furthermore.
Leaktests do not, as you said "steal quicken info, game serials [...]". Some, however, DO log keystrokes and send them. All of them DO send "fake" information to a remote computer via hidden methods, and it's the duty of a firewall to detect these because some malware could have sent private data the exact same way. They're like POCs, or vulnerability tests, aimed at firewalls.
It is obvious a leaktest shouldn't be detected by AV signature, since it's not technically malware. BUT, a firewall (or a behavior blocker, or an HIPS) sould detect these tests, which often use methods like DLL injection, memory modification, and so on, to try and use legitimate processes to connect and transmit data to their servers. What malware can do, and does. So the point is to test a firewall against known penetration methods, and see how well it reacts, which is a good indicator of how good it will be in a "real world" situation.

In a large network, there are other filtering methods, like snort lists, gateway level filtering... But on a home network, there isn't very often such a kind of protection, so a firewall should protect against unwanted data transmission (what a NAT doesn't prevent) and ways to cirumvent it are tested by those leaktests (which are only part of the "security tests" software category, there are also kill tests, vulnerability tests...) to see how well a firewall protects against that.

The file itself is harmless, but that doesn't mean it's useless...