Types of HIPS

Isn't it so, that the tester decided to install these 5 misses, although Prevx1 gave him the choice not to install these 5 misses ?

 

I think it can imply to KAV PDM only. No other AV has such a thing so far or I am not aware.

 

To my understanding that is indeed the case.

 

do not know the test does not mention.

 

I hope you have the frozen snapshot in a different partition, Even so First Defense is not a security program. Still if you use the off-line frozen snapshoy you should be safe. First Defense uses the NTFS file security and permissions, which itself is breakable. But I am not trying to get 100% security, so your security setup seems safe enough to me.

One point I disagree with you. Your remark that it only was a small test set, would frighten me more than 5 out of large test set.

Regards Kees

 

You cought me there. I know one of the paid has a strong defense, two other fairly well heuristics. Since I am using Antivir free, I might have messed up (KAV, AVK)

 

That's what I thought too.
If a program doesn't exist in the Community Database, Prevx1 considers the program as "unknown".
If the setting for "unknown" programs in Prevx1 is
1. Query (= default value), Prevx1 will ask the user to install it or not. So the tester had the choice.
2. Block, Prevx1 will BLOCK it, which means NO installation
3. Allow, Prevx1 will install it.

 

Erik,

I do not think they allowed it. I looked up the maning of missed

"The program was not able to recognise the malicious activity and/or to protect the system by blocking the malware"

I conclude out of this description that PrevX completely missed the malware.

There also was a Blocked meaning that the software was blocked in interactive mode, but not able to recognise it in auto mode.

In the scenario you are describing the test result should have been Blocked in stead of missed

Considering that CB also was tested (a behaviour blocker) it gives further reason to believe that PrevX behavior did not recognise any malicious behavior.

 

I think they must have allowed but not sure.

 

Hi there. My first post. I had to ask:
What's the link to that test? And if he doesn't say if the unknown query poped, how can we tell then? I'd like to check that for myself. Very strange if it didn't...
Anyway , best to complete the sec. suite with GeSWall and SSM (for kicks)

 

Well, I don't know much about malwares. I don't even know how malwares look like.
I don't know how to do these tests. I don't even know where I can get these malwares.
As long I don't know all that, I'm not going to discuss this.

 

AV-COMPARATIVES :
http://www.av-comparatives.org/

 

test of prevx was done online. prevx asks on every application that is unknown/new to prevx db if you really want to run it. this has nothing to do with security, as this will happen with any new legitimate software you wan to install and that is not between the 18millions files in prevx whitelist db.
the 5 samples were missed, because prevx a) did not have them in their database and therefore did not recognize them as malicious files and b) because prevx did not block the samples (the actions done by the samples, so, neither by their behavior). prevx has now added some rules in order to cover malware like the 5 missed ones.
in general to all users who see their favorite product not scoring best: yes, the sample set is limited, but anyway it shows even in the limited sample set (which is even worser depending on how you see it) it missed some. anyway the vendors got the samples they missed and added more rules or fixed bugs in order to protect you in future against this and similar malware.
remember that the results/misses were confirmed by the company/ies before the report was uploaded, so i do not understand why users still have problems to accept that also their favorite product is not absolute perfect at any time and in all cases. (same also to those which ask if the new version was tested: yes, the vendors submit us their very latest versions prior the tests, sometimes even before it gets uplaoded to their websites).
other thing: the programs like KIS PDMn, SnS etc. were run without their signature databases etc., in order to keep the results comparable and to test only the feature that was wanted to test.


EDIT: "There also was a Blocked meaning that the software was blocked in interactive mode, but not able to recognise it in auto mode."
yes, if you read the report, you will notice that one product had this serious bug, which got fixed thanks to this test. in order to inform you on which samples this bug would have occured, i added that category.

 

On that basis, even a whitelist/blacklist HIPS with only 2 processes in its database would never be given a low score. One could even establish a security program that ALWAYS responds "Query" to any process that it encounters for the first time on a given user's computer. BINGO! A perfect scored every time.

As I read it, AV-C's test aimed to determine if the various programs did the job in their present state. Under those conditions, a miss is a miss is a miss.

Disputing the tester on behalf of a favorite program shouldn't be done -- unless, of course, the tester is a gross incompetent. But AV-Comparatives is quite competent.

 

"a security program that ALWAYS responds "Query" to any process that it encounters for the first time on a given user's computer. BINGO! A perfect scored every time."
Yes, correct, but they don't.
Overall i got it IBK, but "the sample set is limited, but anyway it shows even in the limited sample set (which is even worser depending on how you see it)", no, in a small sample results tend to be more erratic. A trend is shown as the sample gets larger. Statistics.
But i understand what you said. Still one question: was it in Pro or ABC? Or both?

 

no, it was not just abc

 

ok then. A good site i think. Never came across with it. Seems very objective and reports the method well enough. Very good!

 

When applying test theory like TMAP (test management approach):
- you only have to test one sample to know whether a specific situation is covered or not (being a program decision).
- for every decision you use a test, hence a test set is made which reflects the decision tree of a program.
- knowing IBK's reputation he will have put together a sample reflecting the purpose of the test (that is why he tested KIS without the black list data base)

Security programs use parts of malicious codes (called sniplets) to identify whole families or variations of virusses/malware et cetera. With sniplets it is possible to have a small black list data base. The same applies to program behavior (e.g. dll or data injection, adding an hidden registry key, etc.). To get a statistic relevant idea of the abilities of a specific security feature, you therefore do not have to put together a large test.

With this in mind, missing out 5 of relative small test set of 40 TYPES of malware by PrevX, is worse than missing out 20 of a test set of 100.000 variations.

I am glad that people like IBK do this testing (for us)

 

I know already that none of the security softwares are foolproof. I knew this already, when I ran my second scanner 3 years back.
I only wanted to know why Prevx1 failed and I was looking for an explanation. I got that explanation now and thanks for that.
I just don't understand why people take it always personal. I'm talking about SOFTWARES, not PEOPLE. If somebody says to me my software is crap, I simply don't care, that doesn't hurt me. If he is right I replace my software immediately with his suggestion, otherwise I keep it.

This isn't really about Prevx1 you know, replace Prevx1 with any other security software name and the result is the same : they fail or they don't fail, all security softwares have that in common. That's why I don't have any favorite security software, because I can't trust any of them.
That softwares fail is NOT really a problem, because they can fix it, once it is reported and that is certainly not a reason for me to ditch a software.
Prevx1 failed 5 times and Prevx1 fixed it, case closed. I find all that very normal.

Since all security softwares have their own failures, I need something that removes those failures as well.
That something is my frozen snapshot. A frozen snapshot acts the same way like malwares do.
Malwares change your computer by adding, deleting and replacing objects in order to damage your system
A frozen snapshot changes your computer by adding, deleting and replacing objects in order to put it back in a healthy state.

Prevx1 missed 5 malwares, so these 5 malwares were installed on my frozen snapshot.
The question is : "Will my frozen snapshot UNDO any change, caused by these 5 malwares, during reboot ?"
I don't know the answer for sure and frozen snapshots aren't included in the comparative tests either.

So the only way to test my frozen snapshot is to infect my computer on purpose with more sophisticated malwares and see what my frozen snapshot has done after reboot.
That's why I'm asking how I can infect my computer with malwares on purpose and under control.
Each time I ask that question, I don't get any answer, while several members are infecting their computer on purpose to see what happens.
I can visit dangerous websites of course, but I wouldn't be in control.
Once I know that my frozen snapshot removes these bad changes, I can do other more wild tests.

I can't work with a frozen snapshot alone, that's why I still need security softwares to stop the installation of malwares or to stop the execution of installed malwares.
My frozen snapshot has only one task : remove the malwares, that bypassed my security softwares.

 

Erik,

I started a discussion "malware, highly overrated", because I discovered I was downloading malware, driving by malicious sites just for the fun to see how strong security is. This because on demand scannerrs had not picked up any malware during a long time. Then I realised that in real life I did not go outside my house, pick up stone and threw it against my windows, just to test how strong the hardened safety glass of my windows are. So I promised myself to stop testing security aps.

In other words I agree with you. No reason for you to test your security when the idea (you call it R.I.P for Rollback Intrusion Protection ) behind your set up is smart.

Regards

 

Actually, it's a little more complicated than that. As a specific example, if you used a product such as AntiExecutable, you would receive a perfect score since none of these samples would be allowed to execute, pure and simple. No user intervention required. The only thing required would be for a user to close down the notification dialog stating the program was denied.

There is also a certain amount of user configuration at work here. Default configurations were tested, as they should be. However, if a user is really concerned regarding the results of Prevx as a specific example, a configuration such as Block all unknown programs (Query is the default action) would have yielded a rather different result. Depending on the circumstance, this may or may not be a viable approach.

Once again, yes and no. Within the test, you're correct - a miss is a miss. However, I've used a number of the products tested. Pop-up fatigue is a real issue with any HIPS type product in the hands of a real user. For some of the products a pop-up will be an exceptional event, for others it will be a routine occurrence (I'm implicitly assuming a steady stream of install/uninstall activity, otherwise a total lockdown solution such as AE is probably more appropriate). How a user views and responds to these situations will generally be different.

I really don't seen anyone disputing the results. I do see folks trying to understand what they mean for their own circumstances, and that should be encouraged, not discouraged.

My own take on the test is uncertain, and I'm not talking solely about the misses, I'm also focusing on the products that had no misses. An obvious novice user question could be whether they should immediately install one of these products. IBK has explicitly noted that these are not a replacement for AV's, but should be used in conjuction with an AV, so hopefully nobody will be asking if they should dump their AV for one of these products alone. But the broader question is, in my opinion, still open as is the best option if one decides to install one of these products. In dealing with a product that requires direct user input of Allow/Block to deal with ambiguous circumstances, any test result is of that pairing, not of the individual components (i.e. not of the user or the application alone). It is important not to lose sight of that.

Blue

 

I'm not taking anything personal, geez! I asked a question. Period. I needed to know why the program i have! failed. If i care about having this in the 1st place, i need to know. I know that none is fullproof. That's why i use layers like most of you.
Kees1958: yes, but how many specific situations are there? That's the population. This set is a sample. As the sample gets larger, the results in percentage wise are more accurate. For comparison purpuses. And i understood IBK. For these situations it would have failed. End of story. Now it's fixed, that's all that i want to know for now, i agree Erik.

Cheers

 

@E-A you want to throw some stones:
BIG ROCKS here:
http://forum.sysinternals.com/forum_posts.asp?TID=3446&PN=1
Several "harmless" samples here down the page http://illusivesecurity.pytalhost.com/viewtopic.php?t=56
and here:
http://illusivesecurity.pytalhost.com/viewtopic.php?t=52
If they are still active links.
You may find loys of nasty little test there

 

Damn this thread is moving fast:

IBK: many thanks , again.

I am paid up member of PrevX community and have followed this thread with interest.

There have been a number of threads here and at Castlecops prevx forum re:
"does it work?"

There have been many lengthy responses of it is "the new paradigm " type, and the "you dont understand" type.

Various claims have been made about speed of response to new malware detections etc. The position has been put forward that there is no actual test that can adequately assess PX and what it does.

There has been a thread here about executing simple test script files that PX did not block.

Missing such a high percentage of the small sample is not necessarily extrapolatable but is a concern. Particularly in a comparison: very stark.

Hhmm.
I'm glad that you have been able to do a test of simple known malwares that has been able to show some holes: will has lead to some obviously necessary updating. Good for me and others.

I dont doubt that there are good people working on PX and the utility will continue to evolve. Perhaps not quite there yet? They may need to do a little more rigorous in house testing. I assume they would have the same access as you to the malware samples?. They could tone down the hyperbole a bit.
I also like their idea. I'm not good enough to know whether the internet based screening will eventually have an advantage over other utilities; hopefully so.

Heh: Frank the perv and I will have to do lots of personal research so the database can grow.
Maybe PrevX can give a prize to the user who turns in the most malware/month. LOL.

I hope the PX team are hitting every crack/warez/blackhat/rightwingnutters/grom web sites they can find ASAP

As 'twas pointed out above PX can be set to query/block every running app: that would be disastrous !!

A very loud warning to anyone who relies solely on PX as protection (as has been posted elsewhere).

Echo that.

@Blue:

No doubt, but defeats at least one purpose of PX?

Regards

 

I wouldn't say it defeats one purpose of PX. My take is that this would make it a somewhat broader version of something akin to AntiExecutable - in other words, instead of all applications currently resident on a given PC being assumed good and therefore executable (AE), all applications characterized as good in the PX database (whether resident on the PC or not) will be executable (PrevX). New applications will not be covered on release, but that's simply a time lag issue.

Blue