Why Leaktests Are Mostly Irrelevent

[dmenace]

If UltraVNC is already installed on a machine:

Installed FortKnox Firewall Trial and Webroot Desktop Firewall Free.

After Restart I can still connect to these machines remotely with UltraVNC. In theory firewall's should block this inbound connection and display a prompt (allow/deny). In firewall configuration, UltraVNC is not even mentioned or mentioned as "listening" only.

My guess is that as UltraVNC was installed before as a driver, it could bypass the newly installed firewall's protection easily and accept incoming connections without interference from firewall.

This shows that software firewall's are ineffective at protecting an already compromised pc. Outbound filtering / leaktest performance becomes irrelevent when the pc allows inbound connections due to poor implementation of inbound filtering and focus on outbound leak test performance.

Just interesting to discover how stupid some leaktests are in reality. System Shutdown Simulator though, is an exception.. ;D

This observation makes leaktests a marketing tool only (FUD), and shows that HIPS is the way to go in reality ~ to prevent a pc from being compromised in the first place...

Edit:
Alternatively use a brand-name firewall with better inbound filtering. Relatively new / unknown firewalls may be buggy and not match the performance of established firewalls like Outpost or Look'n'stop

[CoolWebSearch]

{QUOTE-> If UltraVNC is already installed on a machine:

Installed FortKnox Firewall Trial and Webroot Desktop Firewall Free.

After Restart I can still connect to these machines remotely with UltraVNC. In theory firewall's should block this inbound connection and display a prompt (allow/deny). In firewall configuration, UltraVNC is not even mentioned or mentioned as "listening" only.

My guess is that as UltraVNC was installed before as a driver, it could bypass the newly installed firewall's protection easily and accept incoming connections without interference from firewall.

This shows that software firewall's are ineffective at protecting an already compromised pc. Outbound filtering / leaktest performance becomes irrelevent when the pc allows inbound connections due to poor implementation of inbound filtering and focus on outbound leak test performance.

Just interesting to discover how stupid some leaktests are in reality. System Shutdown Simulator though, is an exception.. ;D

This observation makes leaktests a marketing tool only (FUD), and shows that HIPS is the way to go in reality ~ to prevent a pc from being compromised in the first place...

Edit:
Alternatively use a brand-name firewall with better inbound filtering. Relatively new / unknown firewalls may be buggy and not match the performance of established firewalls like Outpost or Look'n'stop <-QUOTE}

Agreed 100% personally I think both HIPS, NIPS and firewalls should protect from inbound malware installations and attacks, if your computer is already infected what's the point of your HIPS BLOCKING ALREADY INSTALLED malware if HIPS can't clean it at all?

[huangker]

I don't know what this really shows. Maybe the firewalls already have pre existing rules to VNC and are letting it in and out. Maybe the default rule is not to block all incoming connections. Can you give more details?

[Hugger]

What is NIPS?
Thanks in advance.
Hugger

[lucas1985]

{QUOTE-> What is NIPS?
Thanks in advance. <-QUOTE}
Network intrusion detection/prevention systems (http://en.wikipedia.org/wiki/Intrusion-prevention_system#Network) :)
Snort (http://www.snort.org/)

[alex_s]

{QUOTE-> Agreed 100% personally I think both HIPS, NIPS and firewalls should protect from inbound malware installations and attacks, if your computer is already infected what's the point of your HIPS BLOCKING ALREADY INSTALLED malware if HIPS can't clean it at all? <-QUOTE}
Imagine you start new program you downloaded from Internet, just published and claims to be something useful, for example registry cleaner. No inbound protection (signature based) can detect it. But then you see that this utility tries to install driver, for example, or inject dll, or tampers other processes or just tries to connect in unusual way. This is where HIPS are useful. Then and only then it goes to antimalware bases and becomes known. BTW, to start a program doesn't mean to get infected. I started dozens of malwaredroppers and was not infected. It was stopped by my HIPS when trying to do something suspiciouse. No inbound protection can be complete by definition, it needs signatures, it needs that somebody determined a beast is a beast, and only after this it can protect from this beast. Outbound protection + behaviour based HIPS can theoretically be complete.

[Kees1958]

{QUOTE-> Outbound protection + behaviour based HIPS can theoretically be complete. <-QUOTE}

You mean like http://www.wilderssecurity.com/showthread.php?t=207773 ?

8)

[alex_s]

{QUOTE-> You mean like http://www.wilderssecurity.com/showthread.php?t=207773 ?

8) <-QUOTE}
I mean "theoretically", and as for me personally I use different setup. What HIPS does better than signature based security is a way of prevention. Yes, I allow that there can be found new ways to bypass any HIPS, but once a hole is discoved next HIPS version then prevents the whole concept, while signature-based system prevents only one particular example. So theoretically HIPS can cover all the cases in the end, while signatures will alway be at least a step late (in the very best case) :)

There is one more way - heuristic, but as for me this approach has too many FPs (by definition). It should predict what a code can do, even in case code actually does not. While HIPS awakes when a real action happens.

[Espresso]

The flawed port control of these particular firewalls (your fault or theirs) doesn't negate the importance of blocking programs from opening unsolicited outgoing connections. It's not a zero sum game.

I'm behind a router so incoming protection is a secondary concern for me.

[Hugger]

{QUOTE-> Network intrusion detection/prevention systems (http://en.wikipedia.org/wiki/Intrusion-prevention_system#Network) :)
Snort (http://www.snort.org/) <-QUOTE}

Lucas,
Thank you.
Hugger

[CoolWebSearch]

{QUOTE-> Imagine you start new program you downloaded from Internet, just published and claims to be something useful, for example registry cleaner. No inbound protection (signature based) can detect it. But then you see that this utility tries to install driver, for example, or inject dll, or tampers other processes or just tries to connect in unusual way. This is where HIPS are useful. Then and only then it goes to antimalware bases and becomes known. BTW, to start a program doesn't mean to get infected. I started dozens of malwaredroppers and was not infected. It was stopped by my HIPS when trying to do something suspiciouse. No inbound protection can be complete by definition, it needs signatures, it needs that somebody determined a beast is a beast, and only after this it can protect from this beast. Outbound protection + behaviour based HIPS can theoretically be complete. <-QUOTE}

But do you think that leak-tests can match the power of real malware?
I'll be honest to you as poster to poster, here.
The only reason why I don't need HIPS is because I have downloaded all kinds of files. Right now, I know which file might be infected, which might not.
For example, CFP 3.0 Defense+ has detected possible malware in keygen.exe (mostly for games) in my USB stick, but guess what?
There was no malware at all. Basically, I had completely harmless keygens, but one of them was detected by CFP 3.0 as possible malware-which it wasn't.
Is this a false positive or not?
It wasn't malware in there at all.
The reason why I know this because I opened this same exe file about 50 times from now and nothing happened, however, it's very hard to know if an keygen contains malware or not. 25-30% of all keygens contain malwares. I know this from personal experiences.
So the question is next:
Can heuristics + behaviour based HIPS recognize real malware in keygen. exe or not (or in any other file) without false positives?

One more question: Did you perhaps all of those malwares tested you mentioned above tested against CFP 3.0 defense+?
And do you perhaps know if CFP 3.0 is compatible with Windows XP Service Pack 3?

[alex_s]

{QUOTE-> But do you think that leak-tests can match the power of real malware?
<-QUOTE}
I don't think. I know, it can. I have just analized an example of real malware Trojan-Dropper.Win32.Pincher.bk. It is relatively new, I think, because Nod32 database misses it. OA treated it without much trouble. First it created svchost.exe (I allowed it using my OA), then it tried to start it (I allowed it with OA), then it tried to change memory protection in real svchost (I blocked it with OA). So this relatively new malware example didn't pass classical leaktests functionality.
{QUOTE->
I'll be honest to you as poster to poster, here.
The only reason why I don't need HIPS is because I have downloaded all kinds of files. Right now, I know which file might be infected, which might not.
For example, CFP 3.0 Defense+ has detected possible malware in keygen.exe (mostly for games) in my USB stick, but guess what?
There was no malware at all.
<-QUOTE}
I agree, D+ is extremly paranoid and confusing, this is why I use more quiete and intelligent OA. And taking in account RunSafer feature (it's very like Vista UAC, but not that noisy) + KAV AV database, it turned to be a winner here.
{QUOTE->
Basically, I had completely harmless keygens, but one of them was detected by CFP 3.0 as possible malware-which it wasn't.
[skip]
And do you perhaps know if CFP 3.0 is compatible with Windows XP Service Pack 3? <-QUOTE}
Do not estimate all the HIPS by just D+. D+ is not the best HIPS implementation. My criteria of HIPS is not a lot of alerts on every move, but intelligence, usability and safety, and I think I have found what I looked for. But there are other HIPS systems out there and you can find a one that suites you. I just want to say that I think HIPS approach is the most powerful and promising way today. And yes, every particular implementation is not ideal, but they don't stay, they move ahead with every version.

[huangker]

{QUOTE-> But do you think that leak-tests can match the power of real malware?
I'll be honest to you as poster to poster, here.
The only reason why I don't need HIPS is because I have downloaded all kinds of files. Right now, I know which file might be infected, which might not.
For example, CFP 3.0 Defense+ has detected possible malware in keygen.exe (mostly for games) in my USB stick, but guess what?
There was no malware at all. Basically, I had completely harmless keygens, but one of them was detected by CFP 3.0 as possible malware-which it wasn't.
Is this a false positive or not?
It wasn't malware in there at all.
The reason why I know this because I opened this same exe file about 50 times from now and nothing happened, however, it's very hard to know if an keygen contains malware or not. 25-30% of all keygens contain malwares. I know this from personal experiences.
So the question is next:
Can heuristics + behaviour based HIPS recognize real malware in keygen. exe or not (or in any other file) without false positives?
<-QUOTE}

Couple of things here:
1) D+ is a classical HIPS so its focus is to give you tight control over your computer rather than differentiate between legit software and malware.
2) Heuristics in AV scanning on the other hand tries to differentiate between legit software and malware. However it's implementation varies greatly across different AVs.
3) Just because you have opened a keygen many times doesn't mean it is clean. It may in fact just be resident on your computer logging keystrokes or sending out spam. A better way to check is to upload it to an online sandbox like the Sunbelt one and check out what it actually does on a system.
4) Most leak tests is POC and if you have a look at statistics at shadowserver, you will find that most malware (99%+) just use UDP to connect out. While most try to evade AV detection, 99%+ will stop at that and won't try to evade HIPS, virtualization etc.

{QUOTE-> Do not estimate all the HIPS by just D+. D+ is not the best HIPS implementation. My criteria of HIPS is not a lot of alerts on every move, but intelligence, usability and safety, and I think I have found what I looked for. But there are other HIPS systems out there and you can find a one that suites you. I just want to say that I think HIPS approach is the most powerful and promising way today. And yes, every particular implementation is not ideal, but they don't stay, they move ahead with every version. <-QUOTE}

Sounds like you are talking about smart HIPS. What are you using?

[ErikAlbert]

Smart HIPS ? I never saw one in my lifetime. I have to do all the work, instead of the HIPS software.
I know what I want, I just don't get it. :dry:

[Seer]

{QUOTE-> CFP 3.0 Defense+ has detected possible malware in keygen.exe...

... Is this a false positive or not?
It wasn't malware in there at all. <-QUOTE}

It depends on what you consider to be a malware. For you it is a false positive, but for a vendor whose product is being cracked by it, keygen.exe indeed is a malware.
Such software (cracks & keygens) may or may not be malware, depending on your perspective. It is a grey area.

[huangker]

{QUOTE-> Smart HIPS ? I never saw one in my lifetime. I have to do all the work, instead of the HIPS software.
I know what I want, I just don't get it. :dry: <-QUOTE}

Haha well I guess the better term is 'smarter than classical' HIPS or behavior blockers.

{QUOTE-> It depends on what you consider to be a malware. For you it is a false positive, but for a vendor whose product is being cracked by it, keygen.exe indeed is a malware.
Such software (cracks & keygens) may or may not be malware, depending on your perspective. It is a grey area. <-QUOTE}

Most of us here would be more interested in the 'user' perspective.

[alex_s]

{QUOTE-> Sounds like you are talking about smart HIPS. What are you using? <-QUOTE}

Yes, I really like it :)

[Seer]

{QUOTE-> Most of us here would be more interested in the 'user' perspective. <-QUOTE}

Sure we would. But call it a 'malware' or not, the keygen is being properly flagged imo, whatever the perspective.

{QUOTE-> D+ is not the best HIPS implementation. <-QUOTE}

How opinions differ. I would say just the opposite.

[alex_s]

{QUOTE-> How opinions differ. I would say just the opposite. <-QUOTE}
You just never digged deep enough. A lot of useless alerts, still inability to analize such basic things as command line parameters or entry point infection. They say you can turn off this or that, but then D+ loses its sence as long as it just stops to catch important events. With D+ fully activated it goes completely unusable and at some point you just give up to read those numerous alerts which does mean a poor usability.

[Seer]

{QUOTE-> You just never digged deep enough. <-QUOTE}

It is never "good/deep enough" for me, so I'm always digging something.

As to the 'classical' over 'smart' HIPS topic, it is a matter of taste, or better yet, level of control you want to achieve. I tend to prefer classical HIPS as these will give granular control over processes while remaining utterly indifferent. Or stupid, if you will. I just like to make my own decisions (not every time I click on something, of course) instead of trusting an 'intelligent' software with preconfigured patterns. I am not sure what are the exact statistics, but 'artificial inteligence' (behavioral/heuristics) still plays a minor role in overall security, while black/white listing is implemented far and wide with great success (sandboxing i.e.).
I personally do not expect 'thinking' and 'intelligence' from a machine. It is (still) just a tool to provide horsepower. I will do the thinking part.

[dmenace]

Just to clarify some points:

a) webroot firewall has a mandatory learning mode hence it probably allowed ultravnc without asking.

But it has no options available for individual port control...

b) fortknox firewall is buggy... I don't know why but it seems not properly implemented.

If you use a popular / trusted firewall such a worst case scenario breach like i mentioned above wouldn't happen.

Just shows you have to be careful when choosing a firewall...

[huangker]

{QUOTE-> It is never "good/deep enough" for me, so I'm always digging something. <-QUOTE}

It becomes really tiresome for me. But I can see why someone would want to use it to have good control over their system.

{QUOTE-> Sure we would. But call it a 'malware' or not, the keygen is being properly flagged imo, whatever the perspective.
<-QUOTE}

If the crack doesn't actually harm the users' system, the AV/AS shouldn't flag it. It is not security software's role to police software piracy.

[Seer]

{QUOTE-> It is not security software's role to police software piracy. <-QUOTE}

You've made a clear point here :)

[CoolWebSearch]

{QUOTE-> I don't think. I know, it can. I have just analized an example of real malware Trojan-Dropper.Win32.Pincher.bk. It is relatively new, I think, because Nod32 database misses it. OA treated it without much trouble. First it created svchost.exe (I allowed it using my OA), then it tried to start it (I allowed it with OA), then it tried to change memory protection in real svchost (I blocked it with OA). So this relatively new malware example didn't pass classical leaktests functionality.
<-QUOTE}

Here is the main problem with this. What you have shown is the CLASSICAL leak-test functionality. Over 95% of leak-tests are useless because they don't match the behaviour and effectiveness of real malware. I would strongly
support leak-tests that are using REAL malware methods of how to gain into your system, memory modifications and etc.. These are all proof of concept simulator tests-but when CFP 3.0 with Defense+ was tested against real Trojans in PC Welt it failed 2 of 10. There is no reason why would PC Welt lie about since they are well respected german PC Magazine who tests various products like firewalls, AVs ASs and etc... (Outpost Pro was awarded several times). All the methods are in that magazine the only problem is that you have to buy this magazine and read the methods and against what kinds of Trojans CFP 3.0 failed to block from phoning home.
CFP 3.0 also failed to block programs with stolen rights to connect to the internet and phone home.
These tests were done by Arne Arnold AV expert from av-test.de.
I will do my best to find these articles in pdf format on the internet.
Cheers!


I agree, D+ is extremly paranoid and confusing, this is why I use more quiete and intelligent OA. And taking in account RunSafer feature (it's very like Vista UAC, but not that noisy) + KAV AV database, it turned to be a winner here.

Do not estimate all the HIPS by just D+. D+ is not the best HIPS implementation. My criteria of HIPS is not a lot of alerts on every move, but intelligence, usability and safety, and I think I have found what I looked for. But there are other HIPS systems out there and you can find a one that suites you. I just want to say that I think HIPS approach is the most powerful and promising way today. And yes, every particular implementation is not ideal, but they don't stay, they move ahead with every version. <-QUOTE}

[alex_s]

{QUOTE-> Here is the main problem with this. What you have shown is the CLASSICAL leak-test functionality. Over 95% of leak-tests are useless because they don't match the behaviour and effectiveness of real malware. I would strongly
support leak-tests that are using REAL malware methods of how to gain into your system, memory modifications and etc.. <-QUOTE}
Are you coder ? Do you know any special "malware" win API ? No doubt any win32 application uses the same win 32 API, any native application uses the same native API. And the methods to tamper the memory are all known. If you can be specific and point out what malware uses what method and what security fails it, we can continue. Otherwise your claim is groundless, sorry. And all the dozens of real malware I studied used the same API leaktests use. And in addition I think people often overestimate malware makers. They mostly are young and ambitiouse people who copy-paste the same code pieces. 99.99% of real malware is nothing but leaktests. Very random there new threats apper (like Rustock.C), but once it was revealed the way to change partition is closed, so all the power of Rustock is in vain.

Edit. Just yesterday OA catched another "leaktest-like" beast on my wife's laptop. She brought it home with her flash disk (her student's diploma). The beast was catched trying to create .sys file. Unfortunately, KAV failed to recognize it.

[CoolWebSearch]

{QUOTE-> Are you coder ? Do you know any special "malware" win API ? No doubt any win32 application uses the same win 32 API, any native application uses the same native API. And the methods to tamper the memory are all known. If you can be specific and point out what malware uses what method and what security fails it, we can continue. Otherwise your claim is groundless, sorry. And all the dozens of real malware I studied used the same API leaktests use. And in addition I think people often overestimate malware makers. They mostly are young and ambitiouse people who copy-paste the same code pieces. 99.99% of real malware is nothing but leaktests. Very random there new threats apper (like Rustock.C), but once it was revealed the way to change partition is closed, so all the power of Rustock is in vain.

Edit. Just yesterday OA catched another "leaktest-like" beast on my wife's laptop. She brought it home with her flash disk (her student's diploma). The beast was catched trying to create .sys file. Unfortunately, KAV failed to recognize it. <-QUOTE}

Well, it's nice you gave me examples from the real life.
However, I thought you were using CFP 3.0 for HIPS protection!?
What do you use for protection?

Do you recommend me to have CFP 3.0 with HIPS or Online Armor (shareware)?
Because my USB memory stick is frequently infected with malwares (especially when I'm writing my diploma)-so far I was lucky, I've been able to catch them all, but for how long?
Also, to catch malware with CFP 3.0 should it be in Custom Policy mode, Paranoid mode, Very High frequency alert level?
Thanks for the informations and advices you casn give me!

[alex_s]

{QUOTE-> Well, it's nice you gave me examples from the real life.
However, I thought you were using CFP 3.0 for HIPS protection!?
What do you use for protection?

Do you recommend me to have CFP 3.0 with HIPS or Online Armor (shareware)?
Because my USB memory stick is frequently infected with malwares (especially when I'm writing my diploma)-so far I was lucky, I've been able to catch them all, but for how long?
Also, to catch malware with CFP 3.0 should it be in Custom Policy mode, Paranoid mode, Very High frequency alert level?
Thanks for the informations and advices you casn give me! <-QUOTE}
I think the both are good. But they are differently designed, so I don't know what approach you would like more. As far as I know CPF allows a user to configure every security aspect. But here can be a trap. Until you fully understand what danger every aspect brings you'd better use paranoid mode, but this mode is almost unusable. I use OA AV+ and it doesn't matter what mode to use, because OA modes and settings affect mostly interface way and security level remains the same. So I'd recommend CPF to a user with a good knowledges of system internals and OA to an average user.

[CoolWebSearch]

{QUOTE-> I think the both are good. But they are differently designed, so I don't know what approach you would like more. As far as I know CPF allows a user to configure every security aspect. But here can be a trap. Until you fully understand what danger every aspect brings you'd better use paranoid mode, but this mode is almost unusable. I use OA AV+ and it doesn't matter what mode to use, because OA modes and settings affect mostly interface way and security level remains the same. So I'd recommend CPF to a user with a good knowledges of system internals and OA to an average user. <-QUOTE}


One more thing. I'm sure ZoneAlarm's application level firewall and OSFirewall would catch this beast.