testing and "Matousec's Latest" merged

Is this a well respected testing site?

http://www.matousec.com/projects/wi...ysis/leak-tests-results.php#firewalls-ratings

The leak-test results shown are startling. Are they valid?

 

Re: testing

Using the "search" function here will yield many, many existing threads pertaining to matousec leaktests where you can read and get a feel for forum members opinions on the subject.

 

Matousec's Latest

As of today's (10/07/07) date. Online Armor is now tied with Jetico for 2nd place with a score of 9375. Most impressive. Yes, Comodo 2.4.18.184 still stands alone as top dog.
Scope it: http://www.matousec.com/projects/windows-personal-firewall-analysis/leak-tests-results.php

 

Re: Matousec's Latest

I was hoping OA would overtake Comodo but excellent job nonetheless.

 

Re: Matousec's Latest

It will, I have no doubt.

 

Re: Matousec's Latest

The site is down.

 

Re: Matousec's Latest

We failed one test, and received a perfect score far as I can tell on the others. The annoying thing is that the test we failed, we'd previously passed Seems a bug crept in - we'll fix that today.

Leak testing takes a lot of time, so we might need to be satisfied with the #3 position for the moment. Still - we're not just a firewall, so to be rated EXCELLENT on just one part of OA is very satisfying to say the least.

 

Re: Matousec's Latest

Don't sweat it Mike. I think OA is an excellent program regardless of its leaktest stopping power.

 

Re: Matousec's Latest

Ohh Mike keep sweating it you are doing so good

 

Re: Matousec's Latest

very nice to see improvements on many products ...

i also hope that future version of Sunbelt Firewall 'fix' the 'bad' rating
at least Sunbelt CEO posted they working hard on improving the firewall/Vista version etc.

 

Re: Matousec's Latest

I would not choose or reject any security product with leak test performance as a "litmus test". Its only one of many factors, especially when looking at suites.

The appeal of leak testing is that a numerical ranking may easily be assigned to the products tested. How about whether you have to be fooling with the rule editor every 30 minutes?

There seems to be a near inverse relationship between ease of use and leak test performance, and the same could be said of HIPS. I know how to run all of this stuff, but if Mrs. Diver can not use it without bothering me every time there is a pop up, I don't want it on her machine or mine either. I am too old to worry about my mother or grandmother using a computer...

 

Re: Matousec's Latest

Hello,
Diver, the best solution for you would be a Mac, or failing that, a Linux machine. No hassle, no worries, wife-proof.
Mrk

 

Re: Matousec's Latest

Hehe - you don't know my wife

 

Despite of the respect there is one very good thing about Matousec ratings. All the tests are accessible and you can check every figure of those tests.

 

Re: Matousec's Latest

I do like the Mac, but then I have to get another copy of MS Office to keep her happy. Linux is hard to get running on a notebook, unless purchasing a Dell with Ubuntu on it. But 6 months later the next version of Ubuntu is out and its hell to pay to get it running.

Naw, I just think anything that requires too much user intervention is useless. If the user can get it right, then he/she is savvy enough to not get hit in the first place. Otherwise, the user does not understand the prompt, says yes to everything and kaboom.

Just ask yourself, could this product go out to 5000 desktops without phones ringing off the hook at the help desk? However, its fun to have a hobby and play with these security tools. Many of them are not finished. Some of the ideas do eventually wind up in sophisticated useful products.

Often I wonder if any of these leak techniques are being used by real world malware.

 

Post removed.

Let's do stay somewhat directed toward the thread topic and save our non Microsoft operating system\program posts for a more appropriate thread Please.

Thanks,
Bubba

 

since I use Norton IS, I'm a bit upset, but my other fav Look n Stop didn't do any better. How reliable are these?

 

NIS I don't know but I would never use their products. L 'n S is a very good firewall. It is Matousec’s opinion that software firewalls should pass all these leaktest criteria before they can be considered a serious product. Personally, I would be more concerned about the products stability; does it cause freezing, general system instability, conflicts with other utilities that should not occur, or BSOD's?

If you keep malware off your machine then leak control plays no factor in the equation at all. If malware gets in, then it is probably time to restore an image or format and re-install Windoze.

 

If you keep malware off your machine you don't need security software. OS is quite enough.

 

I meant his ratings a test is only good as the person testing it. Who made the decision his opinion was worth dropping your current firewall for?, I mean if someone can show me where he's right fine I'll buy in but I was just wondering how reliable his testing procedures are...

 

Sure, okay

Has anyone been able to prove his testing is unreliable? Not that I'm aware of so I would say his testing is very reliable.

 

Hello Forum,
Online-Armor FW just keeps getting better & better.Well done Mike and Tallemu Team.
Badcompany.

 

Perhaps Matousec's model of computing is everyone is sitting at a desktop behind a NAT or Router with SPI. Inbound protection is assured so the only thing left for the software firewall to do is block unauthorized outbound, plus a bit of application management on the inbound side. That is not my model of the world, because I have a notebook and use public wifi a lot. It may be orders of magnitude safer thana direct connection to the internet, but you would be amazed how many people can be online in a place like O'hare.

The question I have, is one of relevance. Are the techniques used by the leak tests being used by actual malware, and I don't mean one time targetedck targeted attack. Today's malware must communicate witha command and control center. If there is a dedicated address they will be caught so there would need to be a server listening for for a change in orders otherwise. The other popular technique is to communicate with a public IRC server. One famous virus changed its command address periodically and used encryption to hide the future addresses, but this encryption was soon broken. My understanding is most of the leak tests simply load IE which on nearly all systems is authorized for outbound connections. Is actual malware loading IE using techniques in the leak tests, and getting it to work on, say, IRC, or to send spam using an active-x plugin, or are these techniques incompatible with the objectives of spam, DDOS and the like.

 

anyone with common sense takes his tests are just part of mosaic ...
it's just up to You how important part of mosaic You consider it ...

if each piece of puzzle helps me to complete the mosaic together and reach optimal (ideal) security then why 'deny' it