Outpost 4 analysis by Matousec

i was waiting on this result for some time and
i was in hope this time outcome will be better than ZA
but ... i guess not ... hopefully all these bugs gets fixed one day ...

yet at least result is quite good compared e.g. to SKPF ...


http://www.matousec.com/projects/wi...lysis/Outpost-Firewall-PRO-4.0-(964.582.059)/

http://www.matousec.com/projects/windows-personal-firewall-analysis/results.php

sorry if this was discussed already but it should be fresh news ...

 

Outpost scored nearly the same as ZA. Not a big difference. It is their rating based on their testing methods and results. How accurate and reliable is Matousec anyways? Does anyone know? Are we to take their results as gospel or with a grain (maybe a large grain) of salt? The one thing mentioned about OP ver 4.0 is the likely instability issues it will have with other security software. Based on my brief experience with it and all the problem reports in the Outpost forum from a number of members, I would tend to agree with that assessment. To be honest, it actually worked quite well for me, but I found it was slowing system and browser performance a bit as compared to ver 3.51.

It also appears to me that quite a few of the failures listed for the products tested are security measures that would be handled by a HIPS application.

 

"...Very unstable application that is likely to have compatability problems with common security software..."

 

It's beginning to look like my initial opinion of matousec was not only premature, but may end up being flat out wrong. I must admit that I initially did not think much of matousec's claims; however, I just took a quick look at his website and it looks like he is beginning to quantify his claims and add substance to his allegations. I am now withholding judgment.

 

I haven't found enough info on these guys to form any kind of concrete opinion on them. Where exactly do they quantify their claims? Has anyone seen how much they charge for their firewall bug analysis reports?

http://www.matousec.com/purchase.php

I'm not saying their testing results are flawed. It's just that they don't provide details of their analysis unless you pay big $$ Not only that, who are these guys, really? Their writeup on "About Us" is very brief and lacking substantive information.

 

report was quite interesting.

 

Update for new visitors to this thread: After further review, this is not a cool site.
It is simply a store that has BUGS FOR SALE that is disguised as a review site. This is the WRONG way to go.

I think that is a cool site.
You have to read all the reviews completely even the ones you aren't interested in.
Don't just look at the chart.
These guys/gals mean business!
They don't mince words and they basically don't recommend any among the ones tested.
I love the bug pricing model, genius.
All right developer, our security professionals will show you how to improve your product.
For a developer, this kind of security analysis is dirt cheap.
If the developer wants to be a cheapskate, they can try to figure it out on their own. Or if they are smart, they will use the beneficial security analysis available to them.
There is an incentive for them to find fault, but if it is a scam, it will be discovered and exposed soon enough.

This type of security research firm is exactly what we the consumers need more of. This can only be good for us, it's a win win situation.

I would like to know what firewall they actually like!

 

well what about his partnership with zone-alarm

He was suppose to be entering into a deal with them quite some time back..
Anyhow another one of his great reviews,which i am not all interested in reading...

 

Hi Legendkiller,

Where can I learn more about this supposed partnership?

Such a statement in a review does not support the notion of a partnership.

I found the reviews to be very detailed and informative.
What is it that you don't like about the reviews?
Have you observed bias?

 

well i think it was around in august or september 1st week when in one of the post matousec himself said that they are close to making a deal with zonealarm to help make their products better...
People like Dallen who were also their at that time will remember the same...

To be honest,around that time only i read a couple of his reviews including one of Norton IS 2006,in which the only bugs he got were "cosmetic" bugs or defects in GUI of norton...and concluded that norton wasn't a good product...

From what i read from his review of outpost,where are the specifications from which he declares that outpost isn't secure??

All that info regarding installation/common program control etc etc are very very basic things which anyone who downloads outpost,installs it,runs it for a 1-2 days can find out,even a novice can..

My only problem is that when you are reviewing a security product you should provide stats like no of leak-test failed and which one's,how he managed to by-pass a malicious program by renaming it(which was fixed way back in version:3 or 3.5) through auto-rules of outpost..

you cannot just say that it is vulnerable...and moreso where has mentioned the list of applications which have compatibilty problems with outpost?

And moreso he expects us to buy the private info...my god!

 

It looks like he is looking for advantage and trying to make money out of the bugs, instead of trying to help the firewall developers to fix the bugs and make a better product, if the bugs are real.

 

any time cost money , do serious security tests cost money ... take look at what's fee for some AV "lab tests"

reports of Matousec team were already bought by Sunbelt and Zone Labs

Agnitum is aware of O4 report and more to come ...

 

Information is the most valuable object these days. You can hire a botnet for any purpose you want, eg to shutdown an opponent's webpape or just to buy some software bugs to be able to get more bots. Bots are just like dividend paying stock, more bots mean more profit in return.

The vulnerabilities are also available in specialized "stores" over the net and as you can see, they are not quite that expensive, especially for crackers, who profit big money from malware. I am sure, that Agnitum will buy it soon (silently), they would be fools, if they would not do it.

 

Bugs for sale?

Are we heading for Double-sided fear: on one side the software vendor of a security product (firewall ) will shout loud and offer proof that we cannot do without a software firewall and on the other side a security expert is selling the vulnerabilities/bugs of software firewalls?

 

such groups and firms exist for quite long time ...

some even offer huge rewards on certain software flaws ...

after You sell details to them, they re-sell that informations to security firms (for ten times higher etc) ...

 

I read the whole thread Re: 1st july....bugs in zonealarm as well as the BBR thread In depth technical review of Kerio and ZoneAlarm Pro !.
They are not negotiating a partnership with ZoneLabs. They are negotiating the sale of the vulnerability/exploit report. There is no bias indicated other than to find more bugs.

Maybe things have changed since you looked at it last.
A design bug is not a "cosmetic" bug. It is a flaw in the software design allowing exploitation.
Is the Norton Open Bug confirmed?

The only review that shows open public bugs is the Norton review.
Only private bugs are shown in the Outpost review which don't give any details.
Why should he give away his research for free? That would kind of defeat the purpose of the business model.
Maybe he could show us a few more open public bugs.

Agreed, but it is still useful info in a complete review.

Leak test results would be a good addition to the review/ratings.
But I don't know what kind of deal would have to be made with gkweb (firewallleaktester.com) in order to use his tests or the results.

Some credibility has been already established with the Norton Open Bugs.
From everything I've read so far, they are very professional.
If they don't deliver the goods, their credibility will be destroyed and their name forgotten quickly.
Scam businesses don't have a long life span.
This doesn't look like a scam to me.

The $2000 reports are not for consumers to buy.
The vulnerability/bug reports are for the firewall vendors first and then security firms.
They provide the info first (and only) to the vendor. That sounds right to me.
I think too big a deal is being made of this.
This is not so different from other Vulnerability Research Services:
QUARTERLY VULNERABILITY CHALLENGE. That's $10k for ONE bug!
What do you think they do with the info gleaned from such a contest?
They turn around and sell it for a profit. The difference is they are a big corporation and Matousec is a small independent vulnerability research contractor.

Secunia has for pay services about managing all the vulnerabilities.
Securityspace offers vulnerability tests on your system for pay.

If a company went looking for a security professional to provide an outside security audit of their security program, that would cost a LOT of money.

There are dozens of others all about finding bug/flaws/vulnerabilities for money.

So I don't see a problem with the business model.

What I do wonder about is that they consider that a firewall should include a full fledged HIPS.
I don't know if it should.

 

I knew there was a big profit motive in here somewhere.

BUGS FOR SALE.....

 

The purpose of the business model is what again, blackmail?

 

contributing vulnerability report, that these guys are doing, seems to me, can help firewall companies make their products better. in this sense, these guys are kind of part time employees/consultants of the firewall companies, and i see no reason why they shouldn't get paid. but what matters here is, that these bug reports can only be sold to respective firewall companies and must not be disclosed publicly whether bought or not.

ps: these guys haven't been around enough to know that their website favicon is not a japanese kanji originally.

 

From what I can see they say this firewall or that firewall has bugs, publish this statement but not what the bugs are and know fine well that the people who have bought the firewall will be worried people. They will know that on the back of this that the company will therefore then be worried too.
So, in order to see what these bugs are the company hands over the cash, what if it then finds that the bugs are either not there or so insignificant as to be of no real security concern? What if the firewall company disputes the tester/bug finders ascertions?
By publishing their findings publicly in such a limited way it's putting pressure on companies to pay the money to see the evidence.
There is another issue with this business model, they say they have found 'x' amount of bugs in a firewall, the firewall company buys the info on the bugs only to discover that they knew about them already and were addressing them or have addressed some or all of them for the next build. Result, your paying for a bag, the contents of which are a mystery until you splash the cash.

What if I published on a web site that a model of car (for example) had safety flaws but wouldn't name them. If the car company handed over 'x' amount of money they could see what my findings were with a condition written in they do not publish or discuss these findings publicly.
This is virtual blackmail as your letting it be public that in your opinion a make of car has serious safety faults generating public fears. This fear will only be allayed if the company hands over the money. What then if the company refutes these claims? The moneys been made and I move on to another target. Easy cash as all the car company are buying is my findings, even if they're wrong.

This type of business model could be open to abuse especially if it's located in a country where the laws are not strong in cases of misrepresentation/fabrication.
Of course the way to remove any doubt about virtual blackmail would be not to publish your findings openly and instead sending your findings to the company concerned but then this doesn't generate the public concern needed to force companies into parting with cash.

'Cynical mode off'

 

Well said Joliet Jake.

Funny, at the top of there site it says "transparent security".

 

somehow Your example got flaws ...

vendor is always notified

as time progress there are public posted advisories (after vendor warning safe time expired) for ZoneAlarmIS(1), SunbeltKerioPF(2), NortonIS(3 and BlackICE(3)

also no company will go and buy nothing, they will for sure demand at least 1 of claimed bugs to be shown before buy such report (link that to public advisories again ...

chance that some of these bugs are already know to company exist , that's why they can buy each standalone or whole report or nothing ... noone forces them ... now you ask then why it's there to buy ... well i guess You don't know there exist security firms whose PATCH holes of other security firms products ... no matter how strange is sounds it's true ... and they can "buy" this report after "vendor protected period expired" ...

somehow i don't see anything shady on this ... it will be shady if someone demand something or threat by something ...

 

1. How come you know?

2. If there is nothing shady, why don't they develop a perfect firewall and make money?

Simple, they cannot do it. These are the very people who possibly come from the school of thought that all software firewalls are rubbish and offer people a sense of false security.

Some of the reviews are a tad short of public mockery of security vendors.

 

what's point lf Your comment about developing own firewall ? ...
You gunna argue against VirusBulletin Ltd. or WebCoastLabs (CheckMark) tests in way why they don't made own AntiVirus etc. ?

and there is lot of similar issues with firewalls and antiviruses for example ... with single product there is no 100% safety ...

and why they should bother to made own firewall ... they good on finding attack vectors and breaking down code of others ... so they decided to made valid biz out of it ... white crackers

but maybe You want them to sell security holes to every cracking lama around so we get even more botnets going ...

 

The AWARDS from VirusBulletin Ltd. or even WebCoastLabs (CheckMark) are dearly sort after by a good number of AV vendors. I think that is different from BUGS for sale. Again what secunia is doing is less shady to me.

Having said that, I really do not swear by security products. There are many issues with bugs, incompatiblities, resource usage, installation problems, vulnerabilities and what-have-you but the 'tone' of the Matousec review was boastful in my opinion.