Test My PC Security .com Independent, uptodate security testing site

Discussion in 'other firewalls' started by Coolio10, Mar 18, 2008.

Thread Status:
Not open for further replies.
  1. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Finally some uptodate firewall results! And finally some real comodo v3 testing.

    The testing is run by users at the comodo forums who volunteer. It would be nice if some wilders users could join and test many firewalls, especially vista users because no vista results are available yet.

    http://www.testmypcsecurity.com

    Here are the results of firewalls tested so far:

    http://www.testmypcsecurity.com/view_results_xp.html

    Individual Results for each firewall are also availlable for download for those who want to keep records or for easier viewing of results.

    http://www.testmypcsecurity.com/view_results_individual.html

     
    Last edited: Mar 18, 2008
  2. Alcyon

    Alcyon Registered Member

    Joined:
    Jan 16, 2008
    Posts:
    438
    Location:
    Montr?al, Canada
    You must be joking. Testing Comodo firewall with Defense+ with firewalls without hips is totally wrong and unreliable. Why not testing hips with hips and firewalls with firewalls?
     
  3. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Why its the developers faults for putting hips into firewalls?
    You can't not count a firewall because it has a hips. The firewalls are tested with default settings which means OA and Comodo keeps their HIPS ON.
     
  4. Nebulus

    Nebulus Registered Member

    Joined:
    Jan 20, 2007
    Posts:
    1,582
    Location:
    European Union
    Association with Comodo is reason enough to wonder about biased results.
     
  5. pitzelberger

    pitzelberger Registered Member

    Joined:
    Jan 10, 2008
    Posts:
    56
    So how about matousec? Is he disabling the HIPS part of a firewall before computing his ranking, or are his test also "totally wrong and unreliable" o_O
     
  6. SecOmnius

    SecOmnius Registered Member

    Joined:
    Mar 2, 2008
    Posts:
    70
    Location:
    In the Light of PARTHENON
    I'm wordering about the same thing...
    The next Matousec test will be REALLY Interesting!
    :)
     
  7. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    8,055
    Location:
    The Netherlands
    Last edited: Mar 18, 2008
  8. Alcyon

    Alcyon Registered Member

    Joined:
    Jan 16, 2008
    Posts:
    438
    Location:
    Montr?al, Canada
    By comparing firewalls with hips and firewalls without hips, you mislead the user into thinking that his firewall, alone, is less efficient than others while it could be as good even if it doesn't block rootkits, keyloggers, trojans or others. Hips and firewalls are two different things.
     
    Last edited: Mar 18, 2008
  9. MikeNash

    MikeNash Security Expert

    Joined:
    Jun 9, 2005
    Posts:
    1,654
    Location:
    Sydney, Australia
  10. wat0114

    wat0114 Guest

    I doubt the testing and results are embellished to favor Comodo or anyone else. If you look at the ratings, they compare to those of Matousec's, and his are no doubt reliable. In fairness, Matousec also tests products with or without HIPS.

    It would sure be nice, however, to see testing results on pc firewalls "pure" packet filtering ability, such as SPI level.
     
  11. Alcyon

    Alcyon Registered Member

    Joined:
    Jan 16, 2008
    Posts:
    438
    Location:
    Montr?al, Canada
    Why is everybody talking about Matousec when the main topic is about Comodo?

    Comparing hips with non-hips isn't fair. For me, it's just another lame promotion trick from Comodo.
     
  12. MikeNash

    MikeNash Security Expert

    Joined:
    Jun 9, 2005
    Posts:
    1,654
    Location:
    Sydney, Australia
    Well, my congratulations to Comodo for getting a good result - as well as their beta test team.

    If you plan to decide which firewall to use just based on these tests then I think you better get an imaging solution - because with at least the three top vendors wanting to be the best here - you'll be changing firewalls frequently :)

    Mike

    PS.. If you get me the minidump (you can add atatchment now at our forum) then we can fix your BSOD :) But as for winning next challenge - who can guarantee ?
     
  13. Mrkvonic

    Mrkvonic Linux Systems Expert

    Joined:
    May 9, 2005
    Posts:
    8,702
    Hello,

    Congrats on the site...

    Now, a few words of criticism:

    You ask a few very critical questions, like will transmission of data to a hacker be prevented or will a virus be stopped from destroying files ...

    But, nowhere do you explain the meaning and implication of these threats or the infection vectors.

    Just a few crude examples:
    - You can spill coffee over your computer and destroy your data.
    - Your hard disk can die and destroy your data.
    - And what hacker are we talking about? Someone with vendetta? A botnet? A file you downloaded off the net and it's a crack for some app?

    And ALL of the above have nothing to do with packets of information that firewalls have to filter in and out.

    Finally, since all firewalls are a part of the kernel or depend on kernel info - if the kernel gets infected and subverted, firewalls become useless. Not getting infected is the key. Because then, you don't need to worry about keyloggers or hackers or anything else.

    And data integrity is a story unto itself.

    Mrk
     
  14. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Just out of interest I have checked some results. Especially, I'm concerned about OA. The two tests (spt and ProcX) that are listed failed for OA are actually passed. And which is more, spt listed as passed at Matousec even for OA version 19, while there v 95 was tested. Then I gave up checking. My conclusion: those results are biased and incorrect. This is very sad the fair competition turns into the dirty tricks :(
     
    Last edited: Mar 19, 2008
  15. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    It is not biased because of human mistake. I will ask the tester to retry. If you tested and made a mistake would it also be bias?
    We cannot mimic the testing environment of matousec which affects results.
     
  16. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    If there was no bias, then OA free would be tested. Idea is plain obviouse: "WE ARE THE ONLY FREE AND COOL FIREWALL". But this is not what disturbs me much. The "mistakes" disturb me really. When mistake is yourth and in your favour this always has a bad smell.
     
  17. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    What part did you not understand about users running the tests?
    A user picks a firewall they want to test and tests it. Online Armor free is not tested, be my guest and test it if you want.
     
  18. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    I have already done it, with the both versions. They run all the same against all the tests. But my results differ from those that are published THERE. How can I publish my alternative results THERE ? BTW, what to do in case I retest Comodo and make some "mistakes" that bring it down ?

    PS. I'm kidding. I will not do any "mistake", just accurate testing. Mistakes are unacceptale in the tests that may affect reputation or public opinion. I just cannot trust THOSE results because I see they are inaccurate at least for one tested product. And I regard this VERY UNFAIR and COMPLETELY UNACCEPTABLE.
     
    Last edited: Mar 19, 2008
  19. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Well then you can't win.

    It was either comodo staff run the test or the users do? There are cons either way but their is a higher trust in users then comodo.

    If tallemu ran the website it would be the same situation, either the staff or users do it. If comodo wants to run the site its up to them but its either user error or staff bias.

    Its better for mistakes from users.

    And public opinion is always better here at wilders then bad results, take the av forum for example. As soon as some website puts up bizarre results most go against it depending on their product of use. Its always been that way.

    If you really want your own results then pm melih in the comodo forums for access.
     
  20. alex_s

    alex_s Registered Member

    Joined:
    Aug 13, 2007
    Posts:
    1,251
    Oh, no. Incorrect or cheated information cannot be fair. This is just impossible.

    As for the results I have calmed down. I talked to Mike about these damned tests and he said they were concentrated on Vista to pay too much attention to the tests. But now there is a light in the end of the tube and so the next beta build will be 100% Matousec-compatible again :)
     
  21. opus dei

    opus dei Registered Member

    Joined:
    Jun 26, 2007
    Posts:
    8
    That is very true. I have not tried I but I have heard god things about OA.
    I might give it a run(free version). I am sure OA will scoring 10 again soon also I have Heard it is slightly easier to use although possibly slightly less configure-able.

    Thanks for the link to the new Matousec resultsI didknow they were out yet

    OD

     
  22. Dorn

    Dorn Registered Member

    Joined:
    Mar 12, 2008
    Posts:
    34
    cant understand why all the firewall that those company are testing is with old version of Bitdefender v10 firewall and not ver11 as 2008 ?
    can anyone explain it? '
    the same with maoustec or what it calls.
     
  23. subset

    subset Registered Member

    Joined:
    Nov 17, 2007
    Posts:
    825
    Location:
    Austria
    Hi,

    such sites are just a waste of time for readers and testers and a waste of money for hosters, a complete joke. :p

    Imagine any AV or FW or whatever software producer would host a site,
    sort out a matching test suite, choose handy testers and publish a result
    with his own product as one and only Excellent in Protection Rating. :argh:

    The biggest problem with such sites is:
    Even if everything is 100 % true and correct, nobody will believe it.

    Even in Comodo forums the resonance was none,
    only about AV's FP when downloading the test suite. :rolleyes:

    Cheers
     
  24. opus dei

    opus dei Registered Member

    Joined:
    Jun 26, 2007
    Posts:
    8
    One thing you might take note of is Testmysecurity.com is running some tests that Matousec is not. In the first round of testing Keylogger and DeleteVolume which CFPv3 is failing, however they(Testmysecurity.com) are not testing the one test that CFP is failing in Matousec.

    Possibly BSODhook could be included In future testing by Testmysecurity.com as well as some other tests not included this will be decided by the testers. If some one would like to contest the result of any tests. Please register for the Comodo Forums and request to join the testing group.

    I believe there are still plans to test CFP2.4, but I do not think the tests have been completed yet for that version

    as to the the argument for Separating Pure packet filters from Firewalls with HIPS this could also be discussed by the testers. I will say there should be some sort of notations on the site. All this is open to discussions by the testers

    Probably the best is to take result from all sources and try to make a discussion based on these as well as comments on some of the forums in regards to user friendliness of the particular product.

    OD
     
Loading...
Thread Status:
Not open for further replies.