Firewall leaktests

Discussion in 'other firewalls' started by Peter2150, Mar 20, 2007.

Thread Status:
Not open for further replies.
  1. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    Hello all

    I am some what skeptical about all this leak testing stuff, but want to learn if I am wrong. With that in mind I have a few questions about the subject.


    1. How many of the leaktests are based on real life documented attacks as opposed to theoretical attack vectors.

    2. This is the tough one. Lets assume I really want to evalute the firewall in the context of my system. So I use the OA Firewall to test. I also run KAV with it's PDM module, and System Safey Monitor. To truly evaluate the significance of the firewall passing leak tests, they should run without anything warning from the other programs. So question two is for me to evaluate OA's leaktest ability how many of the tests would run on my system, and not be detected by KAV or SSM. I ask this as if they are detected, and blocked it's not a test, and if I were to allow them, then I am the problem

    3. In the real world which of the two following options would provide me more safety

    a) Running the OA firewall without the HIPS features, and any one of the quality AV's discussed here on Wilders, and then being a safe internet user. No dodgy sites, no dodgy downloads, and no opening email attachments.

    b) Running the full featured firewall passing all the leak tests, and a variety of other security programs, and then doing all the dumb things online that any one person could do.

    4. This is something that has just occured to me. When the malware folks find something they thing might be a threat to their nefarious schemes they react to counteract it, and can probably react faster then the the white hat guys. So if these leak tests were so important, wouldn't it be probable the the black hat crowd would counter them, making for a vicious circle. If this is going on, it's going to be a never ending circle. If it isn't happening, what does that tell us. Thoughts??

    Lay it on me guys.

    Pete
     
    Last edited by a moderator: Mar 22, 2007
  2. dallen

    dallen Registered Member

    Joined:
    May 11, 2003
    Posts:
    824
    Location:
    United States
    Pete,
    I don't know if your questions are so good that no one feels that they are worthy of providing an answer, or if they are so challenging to current paradigms that people are afraid to try.

    One thing I did not like about your questions is that you stipulated "no dodgy sites, no dodgy downloads" in almost the same breath as you talked about "the real world." Is there anyone in this real world that abstains from these necessities?

    On a more serious note, why doesn't a Comodo loyalist respond?
     
  3. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    Ok, i waited for others to comment, but as only Dallen replied, i'll share my thoughts too.
    I don't know, but i assume they are all theoretical. They represent ways a program could use to bypass a firewall's applications rules to connect. Using an authorized program like IE or something else.
    One of them, tooleaky, was designed specially to demonstrate how easy it is to bypass a FW's outbound control.
    If i understand you correctly, my answer is turn off the HIPS, or allow the pop-ups, except the firewall's interception of the leaktest. Yes, you are the one who has to understand the pop-up, or your rules are already so mature that the fact that a pop-up shows up, raises your attention.
    Using SSM for "leaktest protection" (there has to be a better concept, because this is not about leaktests per se) is more than appropriate, because it gets more, and in real time.
    Who knows, but in a) you have to know what you're doing, and constrain yourself of certain things to be sure. Or simply there's nothing on your computer that exposes you personally, so you don't care. IMO Mrk, IMO!
    In b), if he's dumb enough, he's lost anyway. But if he's doing those things aware of the potential risks, and how to avoid them, i think his programs will help him a lot.
    But not the best scenario comparison.
    Well, i'll give the example of Comodo. Comodo's leaktest interception is a good one. It's always good to try and find new ways to control outbound better, even if there are other ways to continue to bypass Comodo, way too many i've heard.

    But not only there are too many ways to bypass it, when Comodo detects a new dll in a program (IE for example, previously allowed in the rules), or something else, some users report that by blocking it, they lose connection for the program (IE) altogether. It must be tricky to unload the dll once it's already there, for a FW. It's working like an IDS it seems, it detected a potentially unwanted item in a trusted connection, but it cannot block IT alone. Only a reboot can solve the issue for IE. This is good and bad. Good because it was detected, bad because it's not ideal, and not in real time (the big damage is done).

    This is where i think Comodo shifted. They will incorporate HIPS, to detect and block in real time those changes in programs, block intrusions/infections altogether, and control efectively outbound.

    There must be other ways to bypass this solution too (SSM is a present example), i hope some folks here point me how so i can learn. But this is no surprise since no software is perfect.

    Is it better than Windows Firewall or Kerio 2.1.5? IMHO, hell yes, a whole lot.
    Is it usable by everyone? That's the big question, answering pop-ups...
    For me it will be ok, more so to the average Wilders member, but for the others, Comodo has been working on a safelist as big as they can so you can choose to leave safe programs alone, and reduce pop-ups dramaticaly.

    Will it have Locate32 or Faststone Capture? I don't know, but if you consider that the average user doesn't know what these are, and only use IE7 etc., or even FF, and Thunderbird, and so on, i think it's reasonable to expect not many pop-ups will show, or when they do, attention will be given. Time will tell, and only seing it for ourselves will we be able to comment.

    PS: Dallen what do you mean Comodo loyalists?:ninja: :D

    :p
     
  4. Paranoid2000

    Paranoid2000 Registered Member

    Joined:
    May 2, 2004
    Posts:
    2,839
    Location:
    North West, United Kingdom
    Leaktests tend to be theoretical with malware using similar techniques following later. See FirewallLeaktester: In the Wild for some examples of malware using leaktest techniques.
    This is simple enough - either disable all other software aside from your firewall during your test, or allow any action reported by other software (more applicable with SSM which prompts by default).
    Well, if you're going to "act dumb" then that would presumably include saying "YES! ALLOW!" to every firewall/SSM prompt. Security software should be viewed as a seatbelt - it can stop you flying through the windscreen in a crash but it won't help your driving skills for avoiding an accident in the first place.
    I presume here that you mean the black hats would be trying to counter security software rather than leaktests? The answer is they do - both by stealth (leaktest techniques) and brute force (firewall termination). However security software has a "home ground" advantage (assuming that it was installed on a clean system) of being able to control initially what is run and what it does - only when malware gains privileged access (by accessing physical memory or loading a driver) can it gain total control of a system and disable any security software.
     
  5. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    This is where I have a problem. The mechanics as you say are simple. It's the logic of the action that I have a problem with. The several programs that I run are part of a security system, and it's the how the system performs that matters. If I am running along and SSM alerts one to something that you recognize as bad, would you allow it thinking, no problem I know my firewall won't let it out so I don't have to worry. No one in their right mind would. By the same token it doesn't make sense doing a test, and deliberately allowing something to run.

    My point is if I am going to test, why would I act differently with the test then with actual malware.


    Are we saying these tests are that important, so if we pass them, we don't need any other security software. I don't think so, since by the time the leak test comes into play, something is already running. Prevention being better then a cure, doesn't a test that checks whether the crude can run would be more important then once it's running can it "leak"


    BTW I am posing this challenge because of my background as an engineer. I worked on part of the space program, and we constantly challenged what we were doing. I do the same thing with my business today. Thats why I also challenge some of the things here that people take for granted.

    Pete
     
  6. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    Exactly! I agree 100%. However, not everyone will agree with you. We all have our opinions. If you are using a HIPS, or whatever software it may be, when you are running the tests, and that software alerts you to something regarding the test you recognize as suspicious and you are able to stop the progress of the test by answering deny/block on the alert, then you asre successful against the test.

    We do have to remember one thing about running tests: they are far easier to stop because we are prepared for and expecting the alerts. When we are not running tests, we aren't expecting these "test" alerts, so when they happen, we have to carefully observe and scrutinize them, otherwise there is greater chance of inadvertantly allowing something malicious because we were thinking of other things or outside distarctions have numbed our vigilance.

    Well, you wouldn't. But, alas, some people feel you do have to act differently with a test. Go figure o_O

    The point is if you want to see how only your firewall performs against leaktests and you are running other security software such as HIPS, then you probably want to disable the HIPS during the tests. If you are intent on augmenting your firewall with HIPS, or whatever, then you should run the tests with those apps running as well, and if it alerts you before the firewall does, then you are successful at stopping the test even if, just for fun, you do allow it past the HIPS to see how the firewall fairs and find that the test breeches its security, you can still feel content that the test did not make it past the first line of defense.

    IMO, yes.

    Nice approach :)
     
  7. gkweb

    gkweb Expert Firewall Tester

    Joined:
    Aug 29, 2003
    Posts:
    1,932
    Location:
    FRANCE, Rouen (76)
    Hello Peter,

    I will only answer to this question, as this is the most confusing point about the leaktests.

    The problem comes from the fact that there is two angles to look at the leaktests : the test/theoretical angle, and the real life/world angle. When you mix them, some contradictions occurs.

    The test angle, where you disable any other security software (including HIPS) is simply to check how your firewall would behave if all of your other security layers were bypassed. I explain this point on the following PDF document http://www.firewallleaktester.com/docs/leaktest.pdf (see chapter "II.2.b : how to seek leaks ?").

    It is also a point of view to demonstrate that your firewall alone cannot block everything, and that consequently in a real life scenario, you must use HIPS software and other security layers, point explained there : http://www.firewallleaktester.com/advices.htm

    These are two different contexts : the theoretical study, and the practical consequences.

    That's always good to challenge what people takes for granted. That was one reason why I started in the leaktest world. Few years ago, the firewalls were too much promoted as 100% secure and too much people thought that having a firewall was a panacea, that it was making them invulnerable. I (and others) demonstrated that it was not the case.

    Regards,
    gkweb.
     
  8. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    Hi gkweb

    Now that makes sense. I guess the problem is the when firewall authors, pick up on the leak tests, and use them for marketing, and then suddenly firewalls are rated by their ability on leak tests, we lose sight of what you said above.

    This affirms two things in my mind. First there is value to the leak tests, and/but Second, we have to put them in perspective as opposed to making them a "standard"

    Thanks,

    Pete
     
  9. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    As the others said, the leaktest is a test, if you want to test the firewall, of course you must allow in the HIPS or disable it, in order to test the FW, if you block it, the FW won't be tested. That's what i meant above, sorry for not being clear Peter.

    If you want to test your setup, sure, block with whatever you got. That way you test the whole setup.

    The response to leaktests can be interpreted in two ways: either the FW developers take it seriously and try to block not only that specific leaktest, but also what it represents (class of leaks if i can put it that way.. or method), then it's a good way. If they only try to get that leaktest and they didn't really do anything to improve the FW, then that's bad, and false advertisement.
     
  10. cprtech

    cprtech Registered Member

    Joined:
    Feb 26, 2006
    Posts:
    335
    Location:
    Canada
    Nicely put gkweb.

    I've come to and settled on the conclusion that there are many of us who expect too much from the firewall; we want it to be the "Holy Grail" of security software, with the ability to stop everything but the proverbial kitchen sink, so when our present "flavour of the day" doesn't pass all the leak tests, we are overcome with disappointment and seek something else to replace it. "What is the the new Binford2000", we muse. We anxiously seek out a product that can do better on the leaktests in an effort to appease our anguish, all the while forgetting that we can cover all the bases with a layered approach to security - HIPS, sandbox, antivirus, router, alternate browser...etc - enhanced with a little common sense and some quality research.
     
  11. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    Yes, excellent points here by both experts. Selected sentences would make a useful sticky. A lesson for everyone. How nice that Pete has brought up this question. I tried something similar before, but unsuccesfully. :rolleyes:

    Cheers
     
  12. ErikAlbert

    ErikAlbert Registered Member

    Joined:
    Jun 16, 2005
    Posts:
    9,455
    Since I've read the article "Firewalls are made of straw", I don't consider firewalls as the "Holy Grail" anymore. Fire and straw don't go together, any fireman will tell you that.

    If something passes through my firewall, it will cause a change on my harddisk, am I right about this ?
     
  13. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    Thanks Seer.

    I thought this thread was going to die, but Dallen gave it a kick start boost.


    Pete
     
  14. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    Yes, I find it interesting how people ignore the obvious, and tend to blindly follow the trend of tests and the idea of "absolute security".
     
  15. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    Hi Erik.

    I know what you are going to say, but I wouldn't be relying on FD-ISR too much. It is a process as any other, and the archived files are relatively easily accessible. As for physical image (ATI), I agree that this is untouchable, but ATI is really not an anti-malware application. It is the last solution, when all else fails. I believe the trick is to use anti-malware apps to avoid the last solution. You are taking a shortcut with imaging, but you miss the middle part which is the real security.
     
  16. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    HI Seer

    I agree with you up to a point about FDISR. Particularily archives are off disk as are disk images. Also one would have to first detect FDISR as being on the machine, and then do the programing to get to the archive. Given the small number of FDISR users relative to the mass of machines, why would any one bother. Just to many easier pickings.

    Pete
     
  17. zcv

    zcv Registered Member

    Joined:
    Dec 11, 2002
    Posts:
    355
    Hi Seer,

    Its not the last solution or the middle solution or the first solution. Its a solution used depending what the situation is.

    Real security is first prevention, and second, using the tools and skills one has in solving the problem if one is compromised. That could be the use of images, or cleanup tools, or even a reinstall.

    There isn't "real" security actually in the sense you're using that term, only real recovery.
     
  18. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    I guess that under "real security" I assumed "prevention". Anyhow, you are right, from a practical point of view. But I wouldn't qualify reinstall as a security tool or skill.

    Cheers.
     
  19. zcv

    zcv Registered Member

    Joined:
    Dec 11, 2002
    Posts:
    355
    Sure it is, and it can be done well or badly.
     
  20. Seer

    Seer Registered Member

    Joined:
    Feb 12, 2007
    Posts:
    1,596
    Location:
    Singidunum
    :D How would you know that the installation package you just downloaded isn't malicious? You can cross the street today well or badly. There's some security involved there too. That being from a philosophical point of view.
     
  21. Pedro

    Pedro Registered Member

    Joined:
    Nov 2, 2006
    Posts:
    3,502
    For what is worth (...) ErikAlbert's strategy is one of the few that makes sense. He's only worried about what happens until he loads the offline snapshot (external HD i presume).
     
  22. ErikAlbert

    ErikAlbert Registered Member

    Joined:
    Jun 16, 2005
    Posts:
    9,455
    My archived snapshots are stored on my external harddisk. So what's the problem ? Isn't that as safe as images on an off-line external harddisk ?
    Do you really think that scanners and all the other INCOMPLETE and OVERLAPPING security softwares will do a better and FASTER job.
    In your dreams man, not according my readings and the better skills of the bad guys. You have to come up with something better to beat my approach. :)

    I know already I'm doing the right thing for the future.
    I'm still polishing it but the basic idea and how to do it is there and I stick to my plan, no matter what people say, UNLESS they come up with better ideas.
    My biggest problem is to find the RIGHT softwares, because the security industry created such a BIG MESS and that's the cause of all discussions at Wilders. Most people don't even know what to choose anymore, including me :)

    And my simple question in this thread is still not answered : if something passes through my firewall, will it cause a change on my harddisk or not ?
     
    Last edited: Mar 27, 2007
  23. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,049
    I think the simple answer is if the something is inbound, it well could modify your hard drive, just depends on what it is and what you have that might stop it. If it is outbound then more than likely it already has.

    Pete
     
  24. ErikAlbert

    ErikAlbert Registered Member

    Joined:
    Jun 16, 2005
    Posts:
    9,455
    Thanks.. I will take care of that too.
     
  25. herbalist

    herbalist Guest

    I don't agree with the turning off all other security apps to test a firewall with leaktests. How well the firewall or any other single app protects you doesn't matter. Why do people feel that leaktests have to be passed by the firewall alone? It makes no difference if the warning you get about an attempt to launch a malicious process comes from a firewall suite or a free standing HIPS. It doesn't matter which one detects and defeats a leaktests hook, as long as it's done. What's important is how well your whole security package does. I don't expect Kerio 2.1.5 to detect browser hooks. That's SSM's task. I don't ask SSM to block popups. That's Proxomitron's job. Neither SSM-free or Proxomitron will stop incoming traffic to Windows Explorer, but Kerio will.

    Testing individual apps tells you little in regards to how secure your system is. Test the system you use daily, the whole system, using the software you normally run. That's what will be running if you encounter a malicious website or an infected e-mail. Their combined performance (and your responses to prompts) will decide whether your system gets compromised.
    Rick
     
Loading...
Thread Status:
Not open for further replies.