Regarding the test of Kaspersky's behaviour blocker

I have the following comments regarding the test of Kaspersky's behaviour blocker:

1.
It would be great if another behaviour blocker/IDS was tested. This would allow for a comparison. IMHO such comparison would show that (i) Kasperskys' solution is NOT sophisticated (i.e., it does not use highly intelligent rules) and (ii) there are much better solutions on the market. If you only look at the detection rate (99.x%) you may come to the conclusion that the Kaspersky solution is very good (although it is below average).

2.
It would be very helpful to explain the test procedure. This would allow people to understand how basic the Kaspersky rules are and what rules are still missing.

3.
It would be very helpful to also comment on the number of false alerts that are triggered by the Kaspersky behaviour blocker. For instance, you could delete any files being subject to false alerts and determine the impact on the computer. What kind of useful programs/applications will be destroyed?

4.
I would be interesting to know whether Kaspersky requested this test (e.g., did they pay for it)?

EDITED: Did I really create a separate topic for this post or is this the ordinary Wilders magic? I tried to reply to another thread.

 

no, not paid for the tests. It was done for the users as they asked me too much about how the PDM / v6 scores (and I asked KasperskyLab if they allow me to do it).
As I do not paid for the work (at least so far), asking for working more by adding/testing more products is not very appropriate [I do not get like you 400$ at hour ]. So, no, Kaspersky did not pay anything for this test (but I would welcome anything from anyone, btw ).
It is also not good to ask which rules are currently not implemented in the PDM, as some bad peoples could take advantage of it in their new creations/ideas (it is of interest to know for KasperskyLab, but not for users). Of course the PDM is tuned in a way that it does not flag on every file, as this would require too much user intervention and annoy peoples (like some other HIPS) - but due that it can happen that not everything is blocked.
IMO, the PDM is just the last instance if all other things (in KAV e.g. the signature based detection and his heuristics/"generics") were not able to identify/block the malware.

 

@IBK

I believe my post was moved by a moderator to a separate thread. You may want to copy & paste your response. I am glad to hear that you still do not accept any money for your tests which are, in general, highly appreciated.

"As not paid for the work, asking for working more by adding more products is not very appropriate"

I think DEMANDING would not be appropriate. But asking?? Did you automatically execute the samples (probably yes)?

"[I do not get like you 400$ at hour"

Unfortunately, this also applies to me since I am currently employed (i.e., the money ends somewhere else ;-) Btw. ... I think I have been banned from the forum to which you refer *g*

" It is also not good to ask which rules are currently not implemented in the PDM, as some bad peoples could take advantage of it in their new creations/ideas. "

I consider it more likely that people will choose an IDS using more sophisticated rules that cannot be easily bypassed. The problem is that 9x% of the people here at Wilders do not know how a behaviour blocker works and how to distinguish a basic IDS from a sophisticated IDS.

"IMO, the PDM is just the last instance if all other things (in KAV e.g. the signature based detection and his heuristics/"generics") were not able to identify/block the malware."

100% agreed.

 

Well, saying it detected 99% out of ~6000 samples is impressive already.
Thats ~5940 samples. So i'd call it extremelly sophisticated. I don't know where you got that "below average" thing... So combine 99% signature detection and 99% proactive detection and you should get the picture.
Not to mention they can update PDM rules via update just like ESET can update their Advanced Heuristics (on the fly like regular signature updates).

All stuff detected by PDM can be moved to Quarantine so even if there is a false positive you can restore the file back when FP is fixed. Even if you delete it, it should be in the Backup folder. So no worris about this.

I was also kinda sceptical about efficiency of PDM at first, but now i see it's very effective.

 

{If you only look at the detection rate (99.x%) you may come to the conclusion that the Kaspersky solution is very good (although it is below average).}

I have no reason to defend KAV. As far as I am concerned any software stands or falls on its own.
However, since you said that 99.x is below average, upon what do you base that statement? It can't get much better than 99.4%. Which AVs do better than that?
Surely you are not just pulling that out of the air so please provide back-up for that statement.

My conclusion based upon the AV Comparatives results is that it is very good, and so far am persuaded that it is the best out there overall. I am willing to learn of better ones.

Jerry

 

np, I found it now here .

well, do not misunderstand me, if someone wants to give me something for the work done in past, I will accept it - somehow I have to try to cover expenses, esp. internet connection and hardware/software (and of course also something to give to my 2 friends*, as currently due the lack of money only I and 2 friends are currently still involved activly in av-comparatives, the rest is currently gone).
*EDIT: I forgot one: of course also myself .

sorry, my english is not so good to know the difference between asking and demanding . No, they were not executed automatically, thats why it took so long (I think it was 2 weeks? not sure) and was quite much work for all of us (and the PC's ).

did not know.

 

@RejZoR

Did you have a look at the KIS rules editor relating to the behaviour blocker or, respectively, the alert window (including the "details" link)? This will allow you to see all possible rules (which is another reason why it makes no sense not to talk about the rules).

The rules are really super-basic. By contrast, other developers have spent much time on the development of sophisticated rules. KAV does not use ANY sophisticated rules. There is nothing about opened ports in connection with hidden or transparent windows, connections established by dlls, and so on. The KAV behaviour blocker will already fail to detect a trojan if such trojan is installed with the help of a installer package so that it does not copy itself to Windir or register itself (autostart entry).

 

"No, they were not executed automatically, thats why it took so long (I think it was 2 weeks? not sure) and was quite much work for all of us (and the PC's )."

Kewl! That's an awful lot of work. I once tested a memory scanner with a few hundred samples and this was already quite burdensome.

 

Hello,

By the integration of a pro-active defense, Kaspersky labs show one of the probable and reliable (more than heuristic engines) way of antivirus evolution.

I've not taken a look at its HIPS features, but it is planed to be tested in my new hips tests.
The firs part of the methodology (behaviour) is already published here:
http://security.over-blog.com/article-2915915.html

This methodology is more exhaustive and vicious than the KAV's one, and hope that it will show some of its weaknesses (its seems that's it is the first experience of Kaspersky labs in behavioural blockers).

If some members already use this product, they're welcome to join our team of testers.

Regards

 

@kareldjag

Very interesting. Looking forward to your results.

One suggestion: you could/(should?) try to clearly distinguish between the tests relating to system firewalls/HIPS etc. (i.e., behaviour that is potentially dangerous but not typical for malware) and tests relating to "behaviour blockers"/IDSs (i.e., suspicious behaviour that is typical for malware).

Example: create remote thread, installation of services & api hooks etc. would belong to category 1. By contrast, registering autostart entries and copying itself to windir, invisible application opens a port etc. would belong to category 2.

 

Well something has to open connection. And that is usually an executable. And as such they don't start from nowhere but they require some form of auto start mechanism. And thats where KAV can step in even if it can't detect remote/outbound connections. Considering it has and uses network drivers i wouldn't be suprised if PDM can check that too...
On the other hand i+m pretty sure KL won't give entire PDM documentation to bad guys. Thats kinda logical i think... The more secret you keep about your behaviour engine, the more effective it can do it's job.

 

Security by obscurity never worked and it won't work for behaviour blocker rules either ...

 

Where can we get a look at A2 squared rules?

 

,.-
{If you only look at the detection rate (99.x%) you may come to the conclusion that the Kaspersky solution is very good (although it is below average).}

I am honestly interested in which AVs do better, and especially so much better that 99.4% is considered below average. Can you provide that information?

Thanks,
Jerry

 

I think the original poster is trying to compare KAV's proactive detection module with other such programs like PrevX, rather than other AV programs.

 

I can confirm that a pdmkl.dat file is downloaded along with other updates periodically.

I don't know about that, but even if they were, the fact they scored a little over 99% in the proactive test says a lot.

 

There is no established test procedure for behaviour blockers or the like. More importantly, it is relatively unclear what a behaviour blocker should do and what it should not do.

In consequence, you have to firstly determine what makes sense. Then you have to design a test. And finally, you can evaluate a behaviour blocker. (This is similar to firewall leaktests.)

For instance, it does not make much sense to block the execution of every program. While such "execution protection" feature is not entirely useless (that's why, for example, ProcessGuard provides for execution protection) it does not help you to distinguish harmless software from potential malware. I call programs that allow you to control the execution of software or the like "system firewalls".

Certain other functions like CreateRemoteThread, the installation of services etc. are semi-suspicious (although many legit programs use them). Such functions may be controlled by a system firewall. Maybe also a behaviour blocker will inform you about such behaviour. In other words, there is no clear borderline between system firewalls and behaviour blockers.

Moreover, there are functions which are clearly suspicious (e.g., an invisible program estalishes an internet connection). Such behaviour will generally not be blocked by a system firewall. It can be blocked by a personal (internet) firewall. Unfortunately, such internet firewall will not tell you whether an internet connection is suspicious or not (unless it has additional components like an IDS or the like). This is where a behaviour blocker should come into play. A good behaviour blocker should also inform you if, for example, a rootkit tries to hide files or the like.

To sum it up: I expect a behaviour blocker NOT to bug me (like a system firewall does) but to inform me about highly suspicious activities. It should not allow me to control the entire system (like a system firewall does) but help me to decide whether certain behaviour is suspicious or not. This requires a behaviour blocker to include a detailed help file and good explanatory remarks with respect to alert windows.

If you go to our small forum you can download a very simple test that will help you to determine whether a behaviour blocker can distinguish between harmless internet connections and suspicious "stealthy" connections. Due to the TOS of this forum I am unable to provide you with a link.

 

"If you go to our small forum you can download a very simple test that will help you to determine whether a behaviour blocker can distinguish between harmless internet connections and suspicious "stealthy" connections. Due to the TOS of this forum I am unable to provide you with a link."

Now I understand!
I am happy with the 99.4%.

Jerry

 

You can still send me a PM ;-)

 

I don't understand. Why can't the link be posted? I just read something totally ridiculous about how you can't copy and paste an email here but you can retype it and post it that way. Absurd.
(It makes absolutely no sense to not be able to quote an email from a vendor's support. I'm sure a vendor would far prefer to be QUOTED than to have the poster be forced to paraphrase which might really cause headaches for the vendor. The vendor should not be sending ultra secret information to a user asking for support anyhow).

Now, I read that I can't test KAV's ProActive Defense because you can't post a link to a website What the ....? And why is this prohibited? I don't understand why either of these is prohibited.

 

DesktopArmor has a small "Scoundrel" simulator. If you go to their site, scroll to the bottom, you will see it. Harmless, but tests a few BASIC things.

http://www.desktoparmor.com/index.shtml

 

i think it is probably because it is a dark or grey site, whcih contains links to live malware.

About email privacy: afaik in most european and other countries the law "DL 196/2003" applies, which says that the mails are intended only for the receiver and can not be e.g. posted somewhere without permission of the sender.
I am not sure, but at least in Italy, Austria and Germany it is so I think.

It is also not allowed to post direct links to subpages if the author of the site explicitly prohibits it or uses some basic methods to avoid that.

 

Among other considerations it is just common courtesy
not to post someone's e-mail on a public message board
without their consent.

 

To all,

Let's get back to the topic at hand, shall we?

Thanks.

Blue

 

I like the guard IDS of A2.
Would you consider this to be a sophisticated behavior blocker, or suggest one better?

I have no interest in installing any KAV products.
>_<!!!!