Personal HIPS Tests

Evening/Hello all,


If proactive security softs are not absolutely necessary, they can prevent many problems and infections in a risky using environment (P2P, no hardened machine, porn and similar web sites and so on).
Many products for home users are available on the market (paid and free); but it's quite difficult for the potential consumer to find exhaustive evaluations of these proactive/personal HIPS security solutions.

That's why i've decide to start my tests at the beginning of this year: it's just a potential consumer's work for potential consumers.
The first goal is to provide free, exhaustive and independant informations about products capabilities, and to give a beginning of answer to some frequent questions like this one ( https://www.wilderssecurity.com/showthread.php?t=90295 ) or that one ( https://www.wilderssecurity.com/showthread.php?t=89829 ).

12 very good products had been submitted to various situations and attacks; and a rating is applied for each one of them in relation to results and package's quality.


*The Methodology and the disclaimer:
http://kareldjag.over-blog.com/article-232553.html

*Abtrusion Protector: http://kareldjag.over-blog.com/article-243262.html

http://kareldjag.over-blog.com/article-246879.html

http://kareldjag.over-blog.com/article-248306.html

*AntiHook V2: http://kareldjag.over-blog.com/article-268184.html

AntiHook V2.5: http://kareldjag.over-blog.com/article-553678.html

*OSsurance Desktop: http://kareldjag.over-blog.com/article-498061.html

*Paradore (Security File Protection): http://kareldjag.over-blog.com/article-568094.html

*PrevX Pro: http://kareldjag.over-blog.com/article-311069.html

*ProcessGuard: http://kareldjag.over-blog.com/article-382176.html

*Safe'n'Sec: http://kareldjag.over-blog.com/article-259190.html

*SafePC: http://kareldjag.over-blog.com/article-276082.html

*SecuriTask2005: http://kareldjag.over-blog.com/article-345034.html

*SoftClan Integrity 2005: http://kareldjag.over-blog.com/article-533352.html

*System Safety Monitor: http://kareldjag.over-blog.com/article-343483.html

*Viguard Pro: http://kareldjag.over-blog.com/article-447716.html

*Podium and Overall: http://kareldjag.over-blog.com/article-447659.html


NB. I'm currently using none of them and don't recommend specifically one of them: as usual, it's better to try products by yourself.


Best regards

 

Hi kareldjag,

I wanted to thank you for sharing all of your findings with this group. It is immensely helpful.

Regards,
Rich

 

kareldjag,

Thank you for sharing those with us. And for the time and effort you put in to those tests. Can i ask if you will consider testing one of Finjan's Proactive protection softwares as they have been around a long time and have some very impressive results with their prevention.
http://www.finjan.com/

muf

 

Many thanks for this kareldjag.

I now have my reading set for the next few hours

 

Yes many thanks kareldjag,

I think these tests are really useful, at the moment I´m checking out the different products and this helps a lot of course.
Only, it would be nice if the site looked a bit nicer, the font is very big in IE for some reason.

 

try control+mouse wheel up ;-)

 

In stated order:

TrojanDemo seems to be overstating what is possible - more information is needed on how this operates. Hence, I would have left it off the test. I also think the Finjan vbs tests should not have been included, nor the open/close the CDRom drive test.

For Abtrusion Protector, the program seems to be essentially a checksum-database checker with little thought to security of the program itself. I find it strange that it was so easily terminated, allowed dll-injecting into itself and I don't understand why they bothered with a 'registry-monitoring' feature. Worse, what's with the listening ports?!

For Antihook, why can the process terminate only after multiple attempts? This may a flaw in your testing results or an indication of poor programming.

Again it is interesting to note how weak Prevx was to termination attempts and dll-injection. It would seem that easy termination of security programs is more a problem than I originally thought - perhaps self-protection is still an afterthought for most vendors. And no buffer overflow protection?? (Wasn't that on the promotional material?)

Actually, as far as termination protection goes, I think most vendors are putting critical functionality into load-on-demand drivers. Hence when forceful termination occurs, their products usually fail closed.

On the memory manipulation issue: I think this is really not a big deal. I don't think it's realistic or necessary to prevent analysis from 'physical' disk tools.

My thoughts:
Most disappointed in Prevx's performance, considering the ranting and raving going around about this product. I would have thought it to be an anomaly-based behaviour HIPS going by their claims of 'zero-day' protection. But don't be fooled. If this review is anything to go by, Prevx is no such thing. Avoid such FUD-producing vendors at all costs.
I previously thought ProcessGuard was becoming an bloated and irrelevant product. But this review changed my mind. Perhaps it is only the name of the product which is misleading. I like the potential of SSM as a complete sandbox. However, perhaps unlike the conclusion of this review, I continue to find it too unstable to use in a production environment.

 

I know I posted this somewhere before, but I can't find it now, so I will say it again.

While I think that Kareldjag's tests are very useful info, I do have two concerns with them.

The first one is that some of them are very misleading. Prevx is a case in point. Prevx's strength has always been the prevention of installation of malware. Kareldjag's tests appear to rely on the programs doing the testing being already installed...thereby defeating the purpose of most of the tests conducted this way (majority of them).

But as a criticism of Prevx, their PAWS data showed 50% of people where choosing (obviously unknowingly) to let malware through. This last problem appears to have been overcome in Prevx1 Beta.

The second, semi-criticism, is that Kareldjag tests for things that programs don't claim to protect against...however as I said, that's only a semi criticism. On the whole his info is quite helpful (and many thanks to him for the time put into the testing).

 

Interesting, that my thoughts on the products are very much aligned with yours (including the name "ProcessGuard" ).

SSM appears to me to have the most security "potential", but something tells me that the underlying architecture that allows for this overall security is also leading to overall instabilities. That is, it SSM is "intruding" in spaces and times that make "predictability" of operation difficult. Sometimes designers overlook "unintended consequences" when designing a system.

Which leads me to ProcessGuard, which when combined with RegDefend (as kareldjag suggests), provides very solid, configurable, and transparent (i.e. what you see is what you get) security. I think DiamondCS has a wonderful group of designers who know how to provide deep security while avoid de-stabilizing activities.

Online Armor, from my brief experiences so far, also seems to have not only excellent technical design, but also excellent usability design. While it does not yet have the configurability and transparency of PG+RegDefend, it appears to me to be a heck of a good HIPS with lots of possibilities of greatly expanding the marketplace for HIPS type systems. I am looking forward to kareldjag's review of this product to see what he comes up.

I have similar concerns as you have about Prevx. I think there are reasons that they are going in the direction that they are going, and I think it lies in their business/management model. We'll see where it goes, but they will certainly be pressed for market position, if Online Armor is successful with their launch.

The other product I tried out was Safe "N' Secure. I think it plays in the same space as Online Armor, but for a variety of reasons, I think OA presents itself better to the user. Maybe Aussies are just better at this HIPS game. Anyway, I think this new type of protection is fantastic. When running TrojanHunter's simulator on my system, the simulated trojan was caught in the following order (Online Armor was installed after ProcessGuard):

1) Ewido
2) Online Armor
3) ProcessGuard
4) BOClean

Looking forward to hearing other comments.

Cya,
Rich

 

Just asking but is there any real importance in the order in which security software catches a potential threat? I think some people would assume that one app is better than the other depending on the order in which it is caught. Do you believe this is true?

 

Richrf,

It seems that these days, instead of merely discussing the technical merits of software, we must now also be able to discuss the business/management/revenue model of a security software computer and its products.

A technically superior software is still not worth investing in , if the business management model is inferior and hence doomed to failure.

I'm convinced by arguments (yours among other people's) that any security product line which relies on signatures that has a freebie line and/or a fixed one off charge is doomed to extinction and worse yet it will drag down all the other products in the lineup.

I encourage people using AVG, Antivir, Ewido Security suite
to either demand that the freebie line be discontinued AND/OR
force the payment of a yearly or perhaps monthly subscription fees (if that didn't exist before).

SIGH, and poor me though it was hard enough just figuring out which product was technically superior!!!

Could you explain to me the significance of this test? Does it mean that anything that caught it last is inferior to the thing that caught it first?

---

 

Hi Question,

I don't think the order is relevant. More important is how much "work" the simulator was able to perform on the system before it was caught (the idea being, the earlier it is caught, the better).

From this perspective, it is somewhat interesting to me that Ewido (a scanner) caught the simulator before the HIPS programs and was able to clean the simulator from the system, and BOClean caught it after the HIPS programs (and also successfully cleaned the simulator). I think that is somewhat relevant. It would be interesting to hear comments on this.

Rich

 

I do think long-term viability and technical competence, of any product, in a highly competitive market, does to a certain extent depend upon the revenue model. I do not think this is an absolute, but I think long-term this does hold true. Technical and marketing personal, who are not paid for product development, can get "bored" and leave - and then there is no revenue to attract replacement personel.

When I use products that help secure my system, I do pay for them or contribute. I do not think this is a good model (i.e. relying on charity), but it what I do. I do think that Ewido, considering the capabilities of its product, should end the "free product" (which essentially I am helping pay for) and change the product to a 30 day trial for its next major release.

Some of my thoughts on this matter is in my prior posting. Hopefully, there will be other comments. Thanks for your comments.

Cya,
Rich

 

Hello Richrf

What you are saying is that these tests don't tell us how early the simulator was caught. For all instances and purposes all the security software might even be considered working simultaneous say hooking at the same 'place' (shall we say), but one of them had to be first.

So what is the point of the test again? Assuming there was one.

I don't think it is wise to assume that a scanner should always be slower than HIPS. It's depends entirely on how the real time scanner is setup (scan on - on file read/creation/modifiction) , the technique used (almost always some kind of hook) , the order in which the hook is created , quirks in the boot order, to name just four.

HIPS are even a greater mystery to me, and no two are alike, so I won't even try speculating on that without research.

 

I fully agree. In fact moving towards a BOclean model of no trials backed with no questions money back refunds within a reasonable period might even be worth a look.

 

I disagree. If you don't like that there are free products available don't use them or ignore them. Many people rely on at least some free software, or they would be far less protected, and they would be spreading malware to others, and becoming zombies etc..., without these necessary freebies. Maybe your rich, well good for you, but not everyone is so fortunate as yourself, and need free security software to stay protected.

 

The test was really to check for compatibility. But I do think it is interesting that Ewido catches it first - for whatever reason.

Rich

 

The issue is, for those who are looking to pay for software, is the long-term viability of a product - as witnessed by the latest TDS-3 events. Customers who are not interested in this aspect, can ignore the "pay vs. free" issue entirely. As a paying customer, I look at both the technical and revenue model when deciding which product to purchase, since it does affect me.

Cya,
Rich

 

The order is a concern to me. It concerns me that Ewido catches it before ProcessGuard. This would indicate that Ewido is scanning the read/write of a file which means it's treading in the AV's territory. AT's usually patrol the memory and leave the read/write to the AV or in your case ProcessGuard. Looking at your results, i would expect Ewido to be vying with your AV for access.
Have you tried the test with your AV running?

I'm really surprised Ewido is in front of ProcessGuard. Isn't that the whole point of ProcessGuard? To tell you about the file launching before it gets into your system enough that an AV/AT can butt in? Obviously i don't know enough about this sort of thing as i find it disturbing that Ewido beats ProcessGuard to the punch. Couple that with it patrolling the same area as your AV and it could be risky.

muf

 

Hi muf,

I have been running Ewido alongside KAV for over a year now with absolutely no conflicts. KAV has caught all malware first (except for the tracking cookies that Ewido may find during an on-demand scan). and there has never been any problems between the two.

I have no idea what is happening underneath the covers. Apparently, Ewido is able to catch the malware very early in the game without conflicts. I am not sure how it compares to BOClean in this respect, since BOClean is catching after the HIPS products in the situations that I have tested Trojan Simulater.

Cya,
Rich

 

Hi muf,

Just as an addendum, for testing purposes, I currently have ZoneAlarm Pro, Kaspersky 5 Pro, Ewido, ProcessGuard, RegDefend, BOClean (off and on) and now Online Armor running currently. So far, no conflicts during regular operation nor when I ran the Trojan Simulator. I am on a 512mb, 2.5 Ghz SP2 machine.

Rich

 

I think only someone that really understands what is going on "under the hood" of these software applications and is able to research it can tell whether the order has "meaning" or lack of "meaning".

All else is speculation. I think most of the people on this board (with the exception of a few) are simply operators of software.

Just like on a ship.....I am a engineer. I know for the most part HOW the engines work. The Mates control the speed of the ship and the direction of the ship using the rudder but a MATE has not a clue as to how the engine operates or how the rudder actually moves.

A mate does not understand the difference between a Steam engine, a Diesel engine or a Gas Turbine....they only know the engine turns and moves the propellor which moves the ship.

Likewise, I am a operator of software. I don't know the difference between the KAV engine, the NOD engine, the Ewido Engine or the BoClean Engine, so it is hard for me to comment on it like I am a expert.

I have crazy opinions.....crazy theories but I take none of them seriously because that is not my specialty. My specialty is simply driving my computer to where I want it to go........



Starrob

 

My specialty is observing and asking questions to try to find answers to interesting situations. As you may observe from many discussions, even the actual coders of software are often surprised by results (it is called beta testing), so even if one has access to system code, it doesn't mean that they have the actual understanding of how the software is behaving under all circumstances. And even as a "passenger", I can figure out a lot about how a boat is doing - especially if it is going down, in the wrong direction, or just milling around doing nothing. Something I learned while being a systems engineer.

Rich

 

Yes....direction of the boat yes......That is easy to tell. The actual operation of the engine is harder.
That is why we always joke with the Mates because it does not take as much intelligence to simply drive the boat.

The real intelligence comes in taking apart a engine (especially when it is not working right), troubleshooting it and coming up with solutions in order to make it work properly then putting it all back together again and most importantly have the reconstituted machine WORK.

I can do this with something like a Lube Oil Purifier but I confess I can be at a loss at this when it concerns software. I have not studied software much so it is outside my field but I am entertained by it....so I like to theorize too.

One can only observe interesting effects and make theories but theories can be correct, incorrect, neither correct or incorrect or both correct and incorrect.

In short.....your guess is as good as mine and probably no one should take guesses as gospel. I certainly don't take my own theories as gospel. Indeed, I am often incorrect.



Starrob

 

Yes, I may be in a better position than you since I have worked with computer software for over 30 years, designed many large systems for mainframes, minis, and micros, and consulted with most of the major computer software companies during that period of time.

From this experience, I have learned that no one has all of the answers (therefore I am quite willing to ask questions), including the developers themselves. In my own experiences, users/customers were as an important source (if not more important source) of information as the testing/development department. Everyone is observing something different under different conditions, and all information is equally relevant.

I couldn't imagine going back to a user/customer and telling them that they were "wrong", simply because I couldn't immediately replicate a condition in my own lab. Sometimes, I would just "observe" and work backwards to the problem, using "what if" scenarios. This was often the only way to debug critical issues. "Top-down" is a valid as "bottom-up". It is matter of what is more comfortable for the developer.