Put your Anti-Spyware Apps to the Test!

Put Your Antispyware Apps to the Test
http://www.pcworld.com/news/article/0,aid,125138,00.asp

SpyCar is an AS testing effort being undertaken by Tom Liston, a senior security consultant with Intelguardians, based in Washington, DC.

Liston is developing the software with Ed Skoudis, also an Intelguardians security consultant.

Spycar will be available free of charge in May. More information will be made available on the company's Web site at that time.

See: http://www.intelguardians.com/spycar on May 1st.

Confessions of a Spyware Author:
http://isc.sans.org/diary.php?storyid=1295

25 mini spyware-like applications to test the effectiveness of your anti-spyware software that detects and alerts you to behaviors that can indicate that your AS software may not be on the up-and-up. Behavior based detection and blocking are a must.

SPYCAR -- an homage to the European Institute for Computer Antivirus Research (EICAR) antivirus test file.

-- Tom

 

I think this is a nice move, it´s obvious that we need more and better testing tools, lately there have been introduced a lot of new HIPS and Sandbox applications, but how to test them? I mean at the moment the only thing I can do is installing apps (on my virtual machine) and see if I get any alerts or not.

But what about Remote Code Execution attacks (on browsers like IE, FF)? AFAIK there are still no good tests available to test if a HIPS will stop these attacks. So far the only interesting test (besides the firewall leaktests) that I know of is the GeSWall Demo test, but I´m not sure yet how to interpret the results of the test. I do know that KIS and Prevx1 performed very poorly, while ZA Pro and Neoava Guard performed a whole lot better.

http://www.gentlesecurity.com/demo.html
https://www.wilderssecurity.com/showthread.php?t=129234

 

Hi Guys

I was reviewing this thread and realised how little you guys understood about Prevx1. BTW that's our fault not yours!

First some background. Prevx began developing HIPS products in 2001. Prevx launched Prevx Enterprise in Q2 2003 and this was followed with Home and Pro (May 2004/October 2004 respectively). Prevx Home was the first volume consumer HIPS product with around 1.2Million active users. As you will probably already know Prevx Home and Prevx Pro included a 'call home' feature which allowed us to collect anonymous threat information across the web. In 18 months we had built a data mountain of some 3+ terrabytes.

After much analysis of this info it became patently clear that the mass consumer market just cannot deal with the technically oriented popups. In fact most users are more afraid of stopping their system working by choosing to stop an app (or indeed an attack) than they are about the risk of being infected. Simple put, a user has a greater than 70% probability of allowing an event than stopping it. This made a complete mockery of the protection we were trying to provide. True for a technically advanced user PrevxHome/Pro had a lot going for them. But few users want to step through an app one potentially malicious step at a time. The vast majority > 99% of users want a security application to just protect them, if possible zero pop ups, but above all else easy to use. Enter Prevx1.

Prevx1 monitors more than 120 different system behaviours. It anonymously reports 'unique' application behaviour back to our community database which then monitors this feed in real time constantly assessing and re-assessing an application's behaviour. Also this process is not just associated with looking for malicious code it is also looking to identify benign code too.

To give you some perspective on this. We are currently seeing more than 50,000 unique new executables each and every day (actually closer to 100,000 in the last few weeks). Around 2.5% are found to be malicious!

Our community database gives us an ability to determine malicious code more accurately and with fewer false positives. It also has a wide range of information at its disposal which HIPS would never have. Such as knowing that a piece of code never uses the same name twice, or rarely. Such as knowing that a specific file has many executable forms. Such as knowing this piece of code is only ever created by known malware. In total the database has more than 200 datapoints to determine the ancestry, genetics, behaviour and propagation of an entity. The community database is getting stronger and stronger every day.

In the last month we have noticed, based on comments in forums like this, that we are spotting new malware faster and faster. Just take a look at the number of first, and often only entries we are getting under google for new mailicious file names. It speaks volumes about the effectiveness of our technology to detect and determine new malware first.

Claim 1: We are beginning to see more malware than others and we are seeing it faster. This advantage is growing every day. We may not win every battle or test but we are winning more and more each day. We see our technology is on the ascent while others are struggling to keep pace.

Beyond spotting new malware we are also seeing mutations of existing malware almost instantly. This last week we saw, and immediately protected against a new agent of Spwyware Quake and Spy Falcon. See google: http://www.google.com/search?q=atmclk.exe.

Claim 2: we are tracking and protecting against more variants and mutations than others and we are doing this faster every day.

We recently added the first generation of our clean up technology into Prevx1. Because of our community database we can also see the success or failure of our clean up technology. The next two releases will see this important aspect of the product improved further to address some of the really tricky clean up operations that we know are defating all other security products.

Now to the crux of this thread. How to test Prevx1. If you want to test Prevx1 as a pure behaviour based app, then let us configure it that way for you, that is really easy for us to to do. You will get more alerts, it will win tests. But get this, the mainstream user doesn't want it and as our technology gathers momentum we don't think even techies need it.

Remember, Prevx1 will NOT allow any unknown code execution on a user's PC without a prompt. Therefore testing Prevx1 with a piece of code which you choose to run has already bypassed Gate 1 of our protection. Once you have chosen to allow an app to run, we will monitor its behaviour centrally. We have made matters worse for ourselves by marking many of these tools as safe. Safe apps are immune from our behavioural checking. Consequently, we will always fail these tests. Maybe we should just mark all of these as caution. The user will be prompted and the behaviour will then be checked.

We speak to large enterprises all the time who are trying to deploy and manage HIPS and other behavioural products. These are simply not working on scale. They still has too many false positives for widespread adoption. One false positive in a commercial environment can stop thousands of users from working at a cost running into hundreds of thousands an hour. Most of the time these products are detuned to provide minimal protection in return for less interruption.

Remember also that Behavioural products are also not immune from Zero Day. They only trap certain behaviours or patterns of behaviours. These patterns must be updated as malware evolves. They are not a solid state defence or panacea!

Each week we take the malware samples harvested in the wild and fire them at more than 10 of the top security products. The average detection score is around 50% and declining. Even running all apps together the detection is only around 95%.

We are building products geared for real world conditions. Prevx1 is protecting thousands of new users each day, detecting and removing infections that their existing security product did not know about. It is well worth noting that all of our traffic is search engine generated, which typically means that these users have an infection and found us looking for the cure.

I am not trying to score points here against other products. We know we have created a different approach with Prevx1. It is slowly but surely overtaking many other products and we are confident this trend will continue. After all we do have more information at our disposal about the make up and distribution of malware than any other company. So please don't think of Prevx1 as HIPS, it is a very different model.

I welcome your thoughts.

Regards

Prevx

 

Spycar will be released the week of May 1, 2006
We are sorry for the delay.
- The Management

Hmmm, wonder when the actual release date?

 

They sent me an email today acknowledging that they were putting me on the email list, and that they expected it to be released in two days or less!

-- Tom

 

Pls inform su as well when it is released.
Thansk

 

Hi again Peter !
I must agree with you , AGAIN . Sheesh ( This has got to stop )
The post is a very good one in my opinion . But , I am a little confused to as to why it seemed a bit defensive . However , he makes extremely valid points . Prevx can protect you from almost anything out there . It all has to do with how it is configured . That is one of things that makes Prevx as good as it is . It is good for ANYBODY wanting protection . Out of the box , it is fine . The more advanced you are , the further you can take it . Very sweet in my opinion . Anyhoooo , keep on testing things out Peter and remember to tell me about an app that you are using that I am not . we just cannot have that now can we ? lol
See ya

 

Hello,
Most HIPS are as good as their user.
Let's say I want to install this game a friend gave me.
And this friend gave me a crack. Of course ...
So I start to run the crack.
The HIPS asks me is this or that, and I allow it of course, I want this game.
The HIPS warns me that the application is trying to insert itself into HKLM/RunTwice/BS/{0334EE24DA56A33C32786BB0F843}. Duuh? What does that mean? Ah yes, the crack needs to under-shadow the patch so it can run ... Of course, I click yes.
The HIPS warns me about the buffer overflow at memory allocation 8xEF0000E. Holy bananas? Of course I allow it, this is the game we are talking about - from a friend - what can be wrong about it.
Three more clicks and the things works magically. The game works!
It tries to connect to the Internet. It's ok, I wanna play online and showoff my new aimbot from another cracksite, which accidentally plants a very benign trojan onto the pc. But I let that one too. Besides, anti-virus slows down my pc (lol), so I don't need it.
Typical scenario that happens 100 times an hour on average everywhere around the world.
Result - HIPS is as good as its user. And that's the Catch 22.
You want to use HIPS to keep malware away. But you need to be proficient enough to use HIPS properly. But if you can handle HIPS messages - you don't need HIPS! As simple as that.
HIPS are mostly used by people like here at Wilders, who want control of many aspects of their pc, this is a hobby and they like to feel complex. But for those who really need help against malware, HIPS are useless.
I bet my left kidney that 99% of people submitting their hijackthis logs in various forums:
Run only anti-virus if anything at all, out of date.
Have preinstalled Windows, have no clue about it and use IE.
Click on anything and everything that pops in front of their eyes.
Believe ads that claim 500% increase in Internet speed and such.
Sadly, 99% of infected people do not even realize they're infected and have never heard the word forum in their life, unless they studied something roman-related in schools.
Mrk

 

If this is part of your computing life, you need more than a HIPS for protection.

Aren't you posting in the wrong forum?

 

Hello,
That's called AN EXAMPLE.
I was not talking about myself ...
Mrk

 

Understood.

But maybe an example that would be more likely to be encountered
by the typical home user?

 

Hello,
I think this is the most typical.
Many people get infected by using cracked software or by downloading who knows what, thinking they are safe. Many times they do what they think is right - and this one defeats all and any security.
Mrk

 

This gentleman is absolutely correct. You wouldn't believe how many people I know (both professionally and personally) who infect themselves by means of installing something that a buddy told them was super cool. Heck, one of my network security professors at the university infected himself with malware during a lecture by executing what he thought was a safe powerpoint presentation, which was given to him by another professor. This, I believe, is a huge part of the problem. Security experts everywhere admonish home users to not play around with files or programs that come from untrusted or unknown sources...and maybe they go ahead and do that. However, nobody makes the effort to tell them that trusted sources can be entry points for disaster, as well. Honestly, how many home users out there would suspect an e-card from grandma or a floppy from a hubby to be malicious? Probably very few. And as long as this remains the case, no amount of security is going to help.

Unless, of course, machines learn to think for themselves, at which point they'll be able to prevent you from causing harm to the system, even if the source of the harm was originally trusted beyond reproach.

 

It’s hard for me to believe that this could happen. Anyone who takes no precautions with something from his buddy deserves the consequences. I certainly don’t know anyone as foolish as that. And even so, there should be protection in place to catch something like that anyway. Those people need some basic security training.

This is an absurd situation. And this is a network security professor? Something wrong with procedures. Does faculty/staff have access to virus scanners, especially for taking home files received from others on campus? Do classroom and laboratory computers have a program such as Deep Freeze installed? At the college where I teach, those are in place and the situation you describe could not have happened. Someone is asleep at the wheel.

There is certainly a need for help/teaching for the home user. None of my users would "go ahead and do that." In-place instruction is the best solution. A daunting task, for sure, but if forum members could help even one person, think of how many more knowledgeable people there would be.

There is really no reason for this to happen if people follow careful procedures in dealing with such situations.



________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."

--Bruce Schneier​

 

I don´t see why this thread has become another discussion about the pro´s and cons of HIPS, I think we all know them by now. People who don´t care or don´t want to learn about security will always be comprised sooner or later.

@ PrevxCares

Nice to know that Prevx 1 is doing so well but I assume that you didn´t like my comments about Prevx1. So I guess what you´re trying to say is that since the test was not considered to be malicious, you won´t see any alerts from Prevx1. I still think it´s a bit strange, because shouldn´t you be alerted about all suspicious behaviour when running in expertmode?

But back to the topic, I think it would be a nice idea if security companies would build sites with actual exploits on them, so that we can all test the strenght of our pro active defense (non signature based). Strange that no one has done this yet. What do you think about it PrevxCares?

 

still no sign of this, just says it will be available the week of May 1, 2006 which is fast coming to a close!

 

so it was, thanks Ron

 

Done!

Spycar test with DW 1.55 with my docs in secured files – my standard setup.

When the result is “not performed” – I don’t understand – I did the test twice with the same result and all the time when I tested I got the answer after each test that “test is complete” or similar – so I guess “Spycar change not performed” is a good result also.

Ran the test directly under IE DW untrusted (didn’t download and save to disk first) as Admin.

Rolled Back all test entries afterwards without problems.

Didnt answer any popups from Antivir or anything during the test so I hope its my main man and silent protector Defensewall that did the job!

I have Active-X blocked generally in OP Active Content Plug In - dont know if tht matters.

Test results;

Autostart Tests

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\Run
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnceEx
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\Run
Spycar change blocked

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnce
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnceEx
Spycar change not performed

Internet Explorer Config Change Tests

Click here to make Spycar try to change your default home page in IE
Spycar change blocked

Click here to make Spycar try to lockout users from changing the default home page in IE
Spycar change blocked

Click here to make Spycar try to change your default search page in IE
Spycar change blocked

Click here to make Spycar try to remove the Advanced Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Programs Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Connections Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Content Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Privacy Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Security Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the General Tab in your IE Internet Options Screen
Spycar change blocked

Network Config Change Tests

Click here to make Spycar try to add an entry to your hosts file (typically c:\windows\system32\drivers\etc\hosts)
Spycar change blocked

Best Regards

 

Re: Done!

That seem to be very nice results!

 

Re: Done!

What does it mean? RollbackRx or any thing else.

 

Re: Done!

I am waiting for the results from other users here with different appliances. It will be interesting to warch!