Test My PC Security .com Independent, uptodate security testing site

Finally some uptodate firewall results! And finally some real comodo v3 testing.

The testing is run by users at the comodo forums who volunteer. It would be nice if some wilders users could join and test many firewalls, especially vista users because no vista results are available yet.

http://www.testmypcsecurity.com

Here are the results of firewalls tested so far:

http://www.testmypcsecurity.com/view_results_xp.html

Individual Results for each firewall are also availlable for download for those who want to keep records or for easier viewing of results.

http://www.testmypcsecurity.com/view_results_individual.html

 

You must be joking. Testing Comodo firewall with Defense+ with firewalls without hips is totally wrong and unreliable. Why not testing hips with hips and firewalls with firewalls?

 

Why its the developers faults for putting hips into firewalls?
You can't not count a firewall because it has a hips. The firewalls are tested with default settings which means OA and Comodo keeps their HIPS ON.

 

Association with Comodo is reason enough to wonder about biased results.

 

So how about matousec? Is he disabling the HIPS part of a firewall before computing his ranking, or are his test also "totally wrong and unreliable"

 

I'm wordering about the same thing...
The next Matousec test will be REALLY Interesting!

 

Looks like a cool site, but I will tell you one thing, those tables are completely useless! How the heck am I supposed to get a clear overview of the results? You get the same problem with Matousec´s site, and the famous HIPS comparison made by LUSHER.

http://www.matousec.com/projects/windows-personal-firewall-analysis/leak-tests-results.php
http://wiki.castlecops.com/HIPS/IDP_programs/services

 

By comparing firewalls with hips and firewalls without hips, you mislead the user into thinking that his firewall, alone, is less efficient than others while it could be as good even if it doesn't block rootkits, keyloggers, trojans or others. Hips and firewalls are two different things.

 

http://www.matousec.com/projects/firewall-challenge/results.php

 

I doubt the testing and results are embellished to favor Comodo or anyone else. If you look at the ratings, they compare to those of Matousec's, and his are no doubt reliable. In fairness, Matousec also tests products with or without HIPS.

It would sure be nice, however, to see testing results on pc firewalls "pure" packet filtering ability, such as SPI level.

 

Why is everybody talking about Matousec when the main topic is about Comodo?

Comparing hips with non-hips isn't fair. For me, it's just another lame promotion trick from Comodo.

 

Well, my congratulations to Comodo for getting a good result - as well as their beta test team.

If you plan to decide which firewall to use just based on these tests then I think you better get an imaging solution - because with at least the three top vendors wanting to be the best here - you'll be changing firewalls frequently

Mike

PS.. If you get me the minidump (you can add atatchment now at our forum) then we can fix your BSOD But as for winning next challenge - who can guarantee ?

 

Hello,

Congrats on the site...

Now, a few words of criticism:

You ask a few very critical questions, like will transmission of data to a hacker be prevented or will a virus be stopped from destroying files ...

But, nowhere do you explain the meaning and implication of these threats or the infection vectors.

Just a few crude examples:
- You can spill coffee over your computer and destroy your data.
- Your hard disk can die and destroy your data.
- And what hacker are we talking about? Someone with vendetta? A botnet? A file you downloaded off the net and it's a crack for some app?

And ALL of the above have nothing to do with packets of information that firewalls have to filter in and out.

Finally, since all firewalls are a part of the kernel or depend on kernel info - if the kernel gets infected and subverted, firewalls become useless. Not getting infected is the key. Because then, you don't need to worry about keyloggers or hackers or anything else.

And data integrity is a story unto itself.

Mrk

 

Just out of interest I have checked some results. Especially, I'm concerned about OA. The two tests (spt and ProcX) that are listed failed for OA are actually passed. And which is more, spt listed as passed at Matousec even for OA version 19, while there v 95 was tested. Then I gave up checking. My conclusion: those results are biased and incorrect. This is very sad the fair competition turns into the dirty tricks

 

It is not biased because of human mistake. I will ask the tester to retry. If you tested and made a mistake would it also be bias?
We cannot mimic the testing environment of matousec which affects results.

 

If there was no bias, then OA free would be tested. Idea is plain obviouse: "WE ARE THE ONLY FREE AND COOL FIREWALL". But this is not what disturbs me much. The "mistakes" disturb me really. When mistake is yourth and in your favour this always has a bad smell.

 

What part did you not understand about users running the tests?
A user picks a firewall they want to test and tests it. Online Armor free is not tested, be my guest and test it if you want.

 

I have already done it, with the both versions. They run all the same against all the tests. But my results differ from those that are published THERE. How can I publish my alternative results THERE ? BTW, what to do in case I retest Comodo and make some "mistakes" that bring it down ?

PS. I'm kidding. I will not do any "mistake", just accurate testing. Mistakes are unacceptale in the tests that may affect reputation or public opinion. I just cannot trust THOSE results because I see they are inaccurate at least for one tested product. And I regard this VERY UNFAIR and COMPLETELY UNACCEPTABLE.

 

Well then you can't win.

It was either comodo staff run the test or the users do? There are cons either way but their is a higher trust in users then comodo.

If tallemu ran the website it would be the same situation, either the staff or users do it. If comodo wants to run the site its up to them but its either user error or staff bias.

Its better for mistakes from users.

And public opinion is always better here at wilders then bad results, take the av forum for example. As soon as some website puts up bizarre results most go against it depending on their product of use. Its always been that way.

If you really want your own results then pm melih in the comodo forums for access.

 

Oh, no. Incorrect or cheated information cannot be fair. This is just impossible.

As for the results I have calmed down. I talked to Mike about these damned tests and he said they were concentrated on Vista to pay too much attention to the tests. But now there is a light in the end of the tube and so the next beta build will be 100% Matousec-compatible again

 

That is very true. I have not tried I but I have heard god things about OA.
I might give it a run(free version). I am sure OA will scoring 10 again soon also I have Heard it is slightly easier to use although possibly slightly less configure-able.

Thanks for the link to the new Matousec resultsI didknow they were out yet

OD

 

cant understand why all the firewall that those company are testing is with old version of Bitdefender v10 firewall and not ver11 as 2008 ?
can anyone explain it? '
the same with maoustec or what it calls.

 

Hi,

such sites are just a waste of time for readers and testers and a waste of money for hosters, a complete joke.

Imagine any AV or FW or whatever software producer would host a site,
sort out a matching test suite, choose handy testers and publish a result
with his own product as one and only Excellent in Protection Rating.

The biggest problem with such sites is:
Even if everything is 100 % true and correct, nobody will believe it.

Even in Comodo forums the resonance was none,
only about AV's FP when downloading the test suite.

Cheers

 

One thing you might take note of is Testmysecurity.com is running some tests that Matousec is not. In the first round of testing Keylogger and DeleteVolume which CFPv3 is failing, however they(Testmysecurity.com) are not testing the one test that CFP is failing in Matousec.

Possibly BSODhook could be included In future testing by Testmysecurity.com as well as some other tests not included this will be decided by the testers. If some one would like to contest the result of any tests. Please register for the Comodo Forums and request to join the testing group.

I believe there are still plans to test CFP2.4, but I do not think the tests have been completed yet for that version

as to the the argument for Separating Pure packet filters from Firewalls with HIPS this could also be discussed by the testers. I will say there should be some sort of notations on the site. All this is open to discussions by the testers

Probably the best is to take result from all sources and try to make a discussion based on these as well as comments on some of the forums in regards to user friendliness of the particular product.

OD