Community based detection and prevention of malware in behavioral hips software...

[denniz]

Most users here at Wilders know what behavioral hips are and which software uses these techniques. Some behavioral hips use detection patterns purely based on what the programmer put into it. While other behavioral hips use additional rules driven by the community that are using these programs.

It's this community based prevention I'm interested in. The idea sounds cool, but are there any downsides to letting a community decide which programs are and aren't "clean"? A specific behavioral hips I tested allowed me to put in a certain percentage to determine if a program was automatically allowed or not. The default value was at 90%, meaning that if 90% of the community used/encountered this program, then is was automatically allowed.

But how easy is it to manipulate or circumvent these community based contributions? What about really rare programs that not many people use? Or custom made malware that only infect highly specific computers? What happens when large parts of the community allows/trusts the malware to be run?

So some questions would be:

1) What preventive measures do programmers of behavioral hips take to avoid polluting general detection by wrongly made decisions that happen in the community?

2) What if large parts of the community just click "trust" or "allow" when they encounter unknown malware in what they think is a safe program?

3) What are the chances that the community could be wrong or right when judging to allow or block a program/malware?

4) How large must the community be to receive and provide trustful contributions to the other community members?

5) If some programs/malware are rarely encountered in a community, what are the chances that the few members that encounter it will accurately judge it?

6) In the case of a worldwide massive malware outbreak, what are the chances that the community will provide accurate contributions to the community database?

Does anyone have any useful thoughts about this?

[bellgamin]

-{ Quote: "What about really rare programs that not many people use?" }-In Mamutu's case, they have a fairly sizeable minimum number of users of a given app that must be met before community-based factors have any effect on Mamutu's operations.

[denniz]

Maybe someone from Prevx or Emsisoft could clarify some things?

[alex_s]

-{ Quote: " While other behavioral hips use additional rules driven by the community that are using these programs." }-

I believe community-based security decision is a great security hole, taking in account that average community user is not experienced enough to perform true investigation. Community info can only serve as a starting point in a real investigation.

[C.S.J]

Your missing the point, especially if taking about a prevx edge type of product.

The community is only one part of the process, there are other types of detection added into the package that watch it's behaviour on your machine, regardless if many People say an infected file is clean.

[BlueZannetti]

-{ Quote: "I believe community-based security decision is a great security hole, taking in account that average community user is not experienced enough to perform true investigation...." }-The community isn't about, and really doesn't need to do, investigation at all. At least to me, they are primarily a broadly based active conduit of new content that's out and working 24/7 as new material appears. About the only additional comment that the community really need provide is one of content sourcing (open/commercial download, box, unknown, etc.)

Blue

[djohn]

-{ Quote: "Your missing the point, especially if taking about a prevx edge type of product.

The community is only one part of the process, there are other types of detection added into the package that watch it's behaviour on your machine, regardless if many People say an infected file is clean." }-
Exactly.

[Kees1958]

Denniz,

There are several forms of community based detection

a) white - blacklist sampling
When a user makes a decision of an unnown program, than this programs is sent to central intelligence, where the programs is analysed initially automatically (same sort of analysis as Twister AV does realtime in a simplified form). Some companies like PrevX claim they do 95% of their analysis fully automated. That is why some rootkits are such a night mere for AV companies. After code automated code analysis and human expert analysis thsi program is added to white/blacklist and/or behavior patterns are updated.

b) community voting
This is Mamutu, A2's implementation. I am not rather fond on these type of implementations when they are a feature of intelligent systems (like A2 and Mamutu). Most Mamutu/A2 buyers bought these (good IMO) products becase they have a hard time deciding themselves. I hope EMSI checks the community decisions themselves. I have both lisences of Mamutu and A2 and despite my personal dislike of this type of protection, I have to say in all honesty that both A2 and Mamutu always gave the correct out come. So it works better in practise than I has expected.

c) flock protection
I think Drive Sentry has a nice implementation of this. The first victim sends an alarm signal, which is stored at the central data base, so when next customers are faced with this threat the program will say NAY!. It is based on the fact that when a group of tourist is chased by a lion, you can survive by outrunning just one member of the group (hence the name flock protection).

We will see more (and combo's) in future. As a matter of fact I think that PrevX will add a temporary blacklist until they have incorporated lasting protection in there heuristics/behavior ananlysis (so is a combo of all three). PrevX is the company with the highest communicated ambition level. Future versions evn will have some client side virtualisation (which TF already has), but wPrevX has said they will also provide automated (server side triggered) rolll back of wrong user decisions (I think they will provide an option, like Windows update reminds you of updates missed/skipped).

Cheers Kees

[djohn]

Nicely put kees.I like Example C) with the lion Analogy.;D

[alex_s]

-{ Quote: "The community isn't about, and really doesn't need to do, investigation at all. At least to me, they are primarily a broadly based active conduit of new content that's out and working 24/7 as new material appears. About the only additional comment that the community really need provide is one of content sourcing (open/commercial download, box, unknown, etc.)

Blue" }-

What I did mean is something like:

XX users allowed this application
YY users blocked this application

and if XX >> YY than a user likely clicks "Allow".

But what those figures have to do about security ?

[denniz]

-{ Quote: "Denniz,

There are several forms of community based detection

c) flock protection
I think Drive Sentry has a nice implementation of this. The first victim sends an alarm signal, which is stored at the central data base, so when next customers are faced with this threat the program will say NAY!. It is based on the fact that when a group of tourist is chased by a lion, you can survive by outrunning just one member of the (hence teh name flock protection).

Cheers Kees
" }-

I'm not familiar with the workings of Drive Sentry. Is the first victim actually the very first victim or could there be another first victim who actually passed on a wrong decision to the central database by clicking "allow/trust", while he should have clicked "block"?

[Kees1958]

-{ Quote: "I'm not familiar with the workings of Drive Sentry. Is the first victim actually the very first victim or could there be another first victim who actually passed on a wrong decision to the central database by clicking "allow/trust", while he should have clicked "block"?" }-

Denniz,

This mechanism should work the other way around. I am no developer of DS. So I can not tell you in detail. Looking at their marketing material and having played with it a little, they are developing their community feature along those lines (is my guess).

Hope this answer does not disappoint you to much

[BlueZannetti]

-{ Quote: "What I did mean is something like:

XX users allowed this application
YY users blocked this application

and if XX >> YY than a user likely clicks "Allow".

But what those figures have to do about security ?" }-and my point is that this isn't what it's about, nor should users necessarily be approaching it in that fashion.

A key feature of any listing based approach - black, white, grey, it doesn't really matter - is rapid access to the actual content that needs to be listed. Once that content is accessible, then there needs to be a response to it. However, how well the second step is handled is moot without the first one occurring. The community comes into play on the first step, and for the most part (aside from something like "came from http://www...") they should not be relied upon for the second step. At least that's my take on it.

Blue

[alex_s]

-{ Quote: "and my point is that this isn't what it's about, nor should users necessarily be approaching it in that fashion.

A key feature of any listing based approach - black, white, grey, it doesn't really matter - is rapid access to the actual content that needs to be listed. Once that content is accessible, then there needs to be a response to it. However, how well the second step is handled is moot without the first one occurring. The community comes into play on the first step, and for the most part (aside from something like "came from http://www...") they should not be relied upon for the second step. At least that's my take on it.

Blue" }-

OK, I see your point. But this is rather community based collecting, that community based detection :)

Detection, as I see it, is an action like "alarm ! malware detected" :)

[andyman35]

-{ Quote: "and my point is that this isn't what it's about, nor should users necessarily be approaching it in that fashion.

A key feature of any listing based approach - black, white, grey, it doesn't really matter - is rapid access to the actual content that needs to be listed. Once that content is accessible, then there needs to be a response to it. However, how well the second step is handled is moot without the first one occurring. The community comes into play on the first step, and for the most part (aside from something like "came from http://www...") they should not be relied upon for the second step. At least that's my take on it.

Blue" }-

I agree totally,the community is a very efficient way of collecting large numbers of programs/files for analysis,that is where their input should end IMO.

[denniz]

I agree with the statement that community members are an excellent way to gather statistics on what programs people encounter in everyday computing. And I also agree that it should stop there, I don't think it's a wise idea to let community members decide which programs are safe and which are malicious. Most people are simply not qualified enough to make those kind of decisions, because many times they will either allow everything or block everything.

I'm just curious where security companies draw the limit when they incorporate community based protection in there security programs. What weight do community decisions have on the final outcome that the security software will give to the end-user.

[bellgamin]

-{ Quote: "Nicely put kees.I like Example C) with the lion Analogy.;D" }-Ah yes, the ever popular lion & flock analogy. Just do not be the slowest runner because -- if the lion gets you -- you are surely flocked. :argh:

But seriously- a superb explanations, Kees-sensei. I learn a lot from your posts. 10 Q