Put your Anti-Spyware Apps to the Test!

[lotuseclat79]

Put Your Antispyware Apps to the Test
http://www.pcworld.com/news/article/0,aid,125138,00.asp

SpyCar is an AS testing effort being undertaken by Tom Liston, a senior security consultant with Intelguardians, based in Washington, DC.

Liston is developing the software with Ed Skoudis, also an Intelguardians security consultant.

Spycar will be available free of charge in May. More information will be made available on the company's Web site at that time.

See: http://www.intelguardians.com/spycar on May 1st.

Confessions of a Spyware Author:
http://isc.sans.org/diary.php?storyid=1295

25 mini spyware-like applications to test the effectiveness of your anti-spyware software that detects and alerts you to behaviors that can indicate that your AS software may not be on the up-and-up. Behavior based detection and blocking are a must.

SPYCAR -- an homage to the European Institute for Computer Antivirus Research (EICAR) antivirus test file.

-- Tom

[Rasheed187]

I think this is a nice move, it´s obvious that we need more and better testing tools, lately there have been introduced a lot of new HIPS and Sandbox applications, but how to test them? I mean at the moment the only thing I can do is installing apps (on my virtual machine) and see if I get any alerts or not.

But what about Remote Code Execution attacks (on browsers like IE, FF)? AFAIK there are still no good tests available to test if a HIPS will stop these attacks. So far the only interesting test (besides the firewall leaktests) that I know of is the GeSWall Demo test, but I´m not sure yet how to interpret the results of the test. I do know that KIS and Prevx1 performed very poorly, while ZA Pro and Neoava Guard performed a whole lot better. :)

http://www.gentlesecurity.com/demo.html
http://www.wilderssecurity.com/showthread.php?t=129234

[PrevxCares]

Hi Guys

I was reviewing this thread and realised how little you guys understood about Prevx1. BTW that's our fault not yours!

First some background. Prevx began developing HIPS products in 2001. Prevx launched Prevx Enterprise in Q2 2003 and this was followed with Home and Pro (May 2004/October 2004 respectively). Prevx Home was the first volume consumer HIPS product with around 1.2Million active users. As you will probably already know Prevx Home and Prevx Pro included a 'call home' feature which allowed us to collect anonymous threat information across the web. In 18 months we had built a data mountain of some 3+ terrabytes.

After much analysis of this info it became patently clear that the mass consumer market just cannot deal with the technically oriented popups. In fact most users are more afraid of stopping their system working by choosing to stop an app (or indeed an attack) than they are about the risk of being infected. Simple put, a user has a greater than 70% probability of allowing an event than stopping it. This made a complete mockery of the protection we were trying to provide. True for a technically advanced user PrevxHome/Pro had a lot going for them. But few users want to step through an app one potentially malicious step at a time. The vast majority > 99% of users want a security application to just protect them, if possible zero pop ups, but above all else easy to use. Enter Prevx1.

Prevx1 monitors more than 120 different system behaviours. It anonymously reports 'unique' application behaviour back to our community database which then monitors this feed in real time constantly assessing and re-assessing an application's behaviour. Also this process is not just associated with looking for malicious code it is also looking to identify benign code too.

To give you some perspective on this. We are currently seeing more than 50,000 unique new executables each and every day (actually closer to 100,000 in the last few weeks). Around 2.5% are found to be malicious!

Our community database gives us an ability to determine malicious code more accurately and with fewer false positives. It also has a wide range of information at its disposal which HIPS would never have. Such as knowing that a piece of code never uses the same name twice, or rarely. Such as knowing that a specific file has many executable forms. Such as knowing this piece of code is only ever created by known malware. In total the database has more than 200 datapoints to determine the ancestry, genetics, behaviour and propagation of an entity. The community database is getting stronger and stronger every day.

In the last month we have noticed, based on comments in forums like this, that we are spotting new malware faster and faster. Just take a look at the number of first, and often only entries we are getting under google for new mailicious file names. It speaks volumes about the effectiveness of our technology to detect and determine new malware first.

Claim 1: We are beginning to see more malware than others and we are seeing it faster. This advantage is growing every day. We may not win every battle or test but we are winning more and more each day. We see our technology is on the ascent while others are struggling to keep pace.

Beyond spotting new malware we are also seeing mutations of existing malware almost instantly. This last week we saw, and immediately protected against a new agent of Spwyware Quake and Spy Falcon. See google: http://www.google.com/search?q=atmclk.exe.

Claim 2: we are tracking and protecting against more variants and mutations than others and we are doing this faster every day.

We recently added the first generation of our clean up technology into Prevx1. Because of our community database we can also see the success or failure of our clean up technology. The next two releases will see this important aspect of the product improved further to address some of the really tricky clean up operations that we know are defating all other security products.

Now to the crux of this thread. How to test Prevx1. If you want to test Prevx1 as a pure behaviour based app, then let us configure it that way for you, that is really easy for us to to do. You will get more alerts, it will win tests. But get this, the mainstream user doesn't want it and as our technology gathers momentum we don't think even techies need it.

Remember, Prevx1 will NOT allow any unknown code execution on a user's PC without a prompt. Therefore testing Prevx1 with a piece of code which you choose to run has already bypassed Gate 1 of our protection. Once you have chosen to allow an app to run, we will monitor its behaviour centrally. We have made matters worse for ourselves by marking many of these tools as safe. Safe apps are immune from our behavioural checking. Consequently, we will always fail these tests. Maybe we should just mark all of these as caution. The user will be prompted and the behaviour will then be checked.

We speak to large enterprises all the time who are trying to deploy and manage HIPS and other behavioural products. These are simply not working on scale. They still has too many false positives for widespread adoption. One false positive in a commercial environment can stop thousands of users from working at a cost running into hundreds of thousands an hour. Most of the time these products are detuned to provide minimal protection in return for less interruption.

Remember also that Behavioural products are also not immune from Zero Day. They only trap certain behaviours or patterns of behaviours. These patterns must be updated as malware evolves. They are not a solid state defence or panacea!

Each week we take the malware samples harvested in the wild and fire them at more than 10 of the top security products. The average detection score is around 50% and declining. Even running all apps together the detection is only around 95%.

We are building products geared for real world conditions. Prevx1 is protecting thousands of new users each day, detecting and removing infections that their existing security product did not know about. It is well worth noting that all of our traffic is search engine generated, which typically means that these users have an infection and found us looking for the cure.

I am not trying to score points here against other products. We know we have created a different approach with Prevx1. It is slowly but surely overtaking many other products and we are confident this trend will continue. After all we do have more information at our disposal about the make up and distribution of malware than any other company. So please don't think of Prevx1 as HIPS, it is a very different model.

I welcome your thoughts.

Regards

Prevx

[EASTER.2010]

Spycar will be released the week of May 1, 2006
We are sorry for the delay.
- The Management

Hmmm, wonder when the actual release date?

[lotuseclat79]

They sent me an email today acknowledging that they were putting me on the email list, and that they expected it to be released in two days or less!

-- Tom

[aigle]

Pls inform su as well when it is released.
Thansk

[Peter2150]

Hmm

Not sure how PrevxCares got the idea that people were confused about how Prevx1 works from the start of this thread. Sort of had a faint smell of SPAM to it.

This comment is not a reflection on the product, just the post.

[hollywoodpc]

Hi again Peter !
I must agree with you , AGAIN . Sheesh ( This has got to stop )
The post is a very good one in my opinion . But , I am a little confused to as to why it seemed a bit defensive . However , he makes extremely valid points . Prevx can protect you from almost anything out there . It all has to do with how it is configured . That is one of things that makes Prevx as good as it is . It is good for ANYBODY wanting protection . Out of the box , it is fine . The more advanced you are , the further you can take it . Very sweet in my opinion . Anyhoooo , keep on testing things out Peter and remember to tell me about an app that you are using that I am not . we just cannot have that now can we ? lol
See ya

[Mrkvonic]

Hello,
Most HIPS are as good as their user.
Let's say I want to install this game a friend gave me.
And this friend gave me a crack. Of course ...
So I start to run the crack.
The HIPS asks me is this or that, and I allow it of course, I want this game.
The HIPS warns me that the application is trying to insert itself into HKLM/RunTwice/BS/{0334EE24DA56A33C32786BB0F843}. Duuh? What does that mean? Ah yes, the crack needs to under-shadow the patch so it can run ... Of course, I click yes.
The HIPS warns me about the buffer overflow at memory allocation 8xEF0000E. Holy bananas? Of course I allow it, this is the game we are talking about - from a friend - what can be wrong about it.
Three more clicks and the things works magically. The game works!
It tries to connect to the Internet. It's ok, I wanna play online and showoff my new aimbot from another cracksite, which accidentally plants a very benign trojan onto the pc. But I let that one too. Besides, anti-virus slows down my pc (lol), so I don't need it.
Typical scenario that happens 100 times an hour on average everywhere around the world.
Result - HIPS is as good as its user. And that's the Catch 22.
You want to use HIPS to keep malware away. But you need to be proficient enough to use HIPS properly. But if you can handle HIPS messages - you don't need HIPS! As simple as that.
HIPS are mostly used by people like here at Wilders, who want control of many aspects of their pc, this is a hobby and they like to feel complex. But for those who really need help against malware, HIPS are useless.
I bet my left kidney that 99% of people submitting their hijackthis logs in various forums:
Run only anti-virus if anything at all, out of date.
Have preinstalled Windows, have no clue about it and use IE.
Click on anything and everything that pops in front of their eyes.
Believe ads that claim 500% increase in Internet speed and such.
Sadly, 99% of infected people do not even realize they're infected and have never heard the word forum in their life, unless they studied something roman-related in schools.
Mrk

[Peter2150]

-{ Quote: "Hello,
Most HIPS are as good as their user.
Let's say I want to install this game a friend gave me.
And this friend gave me a crack. Of course ...
So I start to run the crack.
The HIPS asks me is this or that, and I allow it of course, I want this game.
The HIPS warns me that the application is trying to insert itself into HKLM/RunTwice/BS/{0334EE24DA56A33C32786BB0F843}. Duuh? What does that mean? Ah yes, the crack needs to under-shadow the patch so it can run ... Of course, I click yes.
The HIPS warns me about the buffer overflow at memory allocation 8xEF0000E. Holy bananas? Of course I allow it, this is the game we are talking about - from a friend - what can be wrong about it.
Three more clicks and the things works magically. The game works!
It tries to connect to the Internet. It's ok, I wanna play online and showoff my new aimbot from another cracksite, which accidentally plants a very benign trojan onto the pc. But I let that one too. Besides, anti-virus slows down my pc (lol), so I don't need it.
Typical scenario that happens 100 times an hour on average everywhere around the world.
Result - HIPS is as good as its user. And that's the Catch 22.
You want to use HIPS to keep malware away. But you need to be proficient enough to use HIPS properly. But if you can handle HIPS messages - you don't need HIPS! As simple as that.
HIPS are mostly used by people like here at Wilders, who want control of many aspects of their pc, this is a hobby and they like to feel complex. But for those who really need help against malware, HIPS are useless.
I bet my left kidney that 99% of people submitting their hijackthis logs in various forums:
Run only anti-virus if anything at all, out of date.
Have preinstalled Windows, have no clue about it and use IE.
Click on anything and everything that pops in front of their eyes.
Believe ads that claim 500% increase in Internet speed and such.
Sadly, 99% of infected people do not even realize they're infected and have never heard the word forum in their life, unless they studied something roman-related in schools.
Mrk" }-

Hi Mrk

Good post.

Pete

[Rmus]

-{ Quote: "Hello,
Most HIPS are as good as their user.
Let's say I want to install this game a friend gave me.
And this friend gave me a crack..." }-If this is part of your computing life, you need more than a HIPS for protection.

Aren't you posting in the wrong forum?

[Mrkvonic]

Hello,
That's called AN EXAMPLE.
I was not talking about myself ...
Mrk

[Rmus]

Understood.

But maybe an example that would be more likely to be encountered
by the typical home user? :o

[Mrkvonic]

Hello,
I think this is the most typical.
Many people get infected by using cracked software or by downloading who knows what, thinking they are safe. Many times they do what they think is right - and this one defeats all and any security.
Mrk

[Maji]

-{ Quote: "Hello,
I think this is the most typical.
Many people get infected by using cracked software or by downloading who knows what, thinking they are safe. Many times they do what they think is right - and this one defeats all and any security.
Mrk" }-

This gentleman is absolutely correct. You wouldn't believe how many people I know (both professionally and personally) who infect themselves by means of installing something that a buddy told them was super cool. Heck, one of my network security professors at the university infected himself with malware during a lecture by executing what he thought was a safe powerpoint presentation, which was given to him by another professor. This, I believe, is a huge part of the problem. Security experts everywhere admonish home users to not play around with files or programs that come from untrusted or unknown sources...and maybe they go ahead and do that. However, nobody makes the effort to tell them that trusted sources can be entry points for disaster, as well. Honestly, how many home users out there would suspect an e-card from grandma or a floppy from a hubby to be malicious? Probably very few. And as long as this remains the case, no amount of security is going to help.

Unless, of course, machines learn to think for themselves, at which point they'll be able to prevent you from causing harm to the system, even if the source of the harm was originally trusted beyond reproach.

[Rmus]

-{ Quote: "This gentleman is absolutely correct. You wouldn't believe how many people I know (both professionally and personally) who infect themselves by means of installing something that a buddy told them was super cool. " }-It’s hard for me to believe that this could happen. Anyone who takes no precautions with something from his buddy deserves the consequences. I certainly don’t know anyone as foolish as that. And even so, there should be protection in place to catch something like that anyway. Those people need some basic security training.

-{ Quote: "Heck, one of my network security professors at the university infected himself with malware during a lecture by executing what he thought was a safe powerpoint presentation, which was given to him by another professor. " }-This is an absurd situation. And this is a network security professor? Something wrong with procedures. Does faculty/staff have access to virus scanners, especially for taking home files received from others on campus? Do classroom and laboratory computers have a program such as Deep Freeze installed? At the college where I teach, those are in place and the situation you describe could not have happened. Someone is asleep at the wheel.

-{ Quote: "This, I believe, is a huge part of the problem. Security experts everywhere admonish home users to not play around with files or programs that come from untrusted or unknown sources...and maybe they go ahead and do that. " }- There is certainly a need for help/teaching for the home user. None of my users would "go ahead and do that." In-place instruction is the best solution. A daunting task, for sure, but if forum members could help even one person, think of how many more knowledgeable people there would be.

-{ Quote: "However, nobody makes the effort to tell them that trusted sources can be entry points for disaster, as well. Honestly, how many home users out there would suspect an e-card from grandma or a floppy from a hubby to be malicious? Probably very few." }- There is really no reason for this to happen if people follow careful procedures in dealing with such situations.



________________________________________________________________
"Talking About Security Can Lead To Anxiety, Panic, And Dread...
Or Cool Assessments, Common Sense And Practical Planning..."
--Bruce Schneier

[Rasheed187]

I don´t see why this thread has become another discussion about the pro´s and cons of HIPS, I think we all know them by now. People who don´t care or don´t want to learn about security will always be comprised sooner or later. ::)

@ PrevxCares

Nice to know that Prevx 1 is doing so well but I assume that you didn´t like my comments about Prevx1. So I guess what you´re trying to say is that since the test was not considered to be malicious, you won´t see any alerts from Prevx1. I still think it´s a bit strange, because shouldn´t you be alerted about all suspicious behaviour when running in expertmode? :-\

But back to the topic, I think it would be a nice idea if security companies would build sites with actual exploits on them, so that we can all test the strenght of our pro active defense (non signature based). Strange that no one has done this yet. What do you think about it PrevxCares? :)

[ronjor]

Spycar has been released. spycar.org (http://www.spycar.org/Welcome%20to%20Spycar.html)

[rothko]

-{ Quote: "Put Your Antispyware Apps to the Test
http://www.pcworld.com/news/article/0,aid,125138,00.asp

SpyCar is an AS testing effort being undertaken by Tom Liston, a senior security consultant with Intelguardians, based in Washington, DC.

Liston is developing the software with Ed Skoudis, also an Intelguardians security consultant.

Spycar will be available free of charge in May. More information will be made available on the company's Web site at that time.

See: http://www.intelguardians.com/spycar on May 1st." }-
still no sign of this, just says it will be available the week of May 1, 2006 which is fast coming to a close!

[ronjor]

It was released this morning. See link in post above yours.

[rothko]

-{ Quote: "It was released this morning. See link in post above yours." }-so it was, thanks Ron

[Rivalen]

Spycar test with DW 1.55 with my docs in secured files – my standard setup.

When the result is “not performed” – I don’t understand – I did the test twice with the same result and all the time when I tested I got the answer after each test that “test is complete” or similar – so I guess “Spycar change not performed” is a good result also.

Ran the test directly under IE DW untrusted (didn’t download and save to disk first) as Admin.

Rolled Back all test entries afterwards without problems.

Didnt answer any popups from Antivir or anything during the test so I hope its my main man and silent protector Defensewall that did the job!

I have Active-X blocked generally in OP Active Content Plug In - dont know if tht matters.

Test results;

Autostart Tests

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\Run
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnce
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKLM\Software\Microsoft\Windows\CurrentVersion\RunOnceEx
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\Run
Spycar change blocked

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnce
Spycar change not performed

Click here to make Spycar try to install a Registry key under HKCU\Software\Microsoft\Windows\CurrentVersion\RunOnceEx
Spycar change not performed

Internet Explorer Config Change Tests

Click here to make Spycar try to change your default home page in IE
Spycar change blocked

Click here to make Spycar try to lockout users from changing the default home page in IE
Spycar change blocked

Click here to make Spycar try to change your default search page in IE
Spycar change blocked

Click here to make Spycar try to remove the Advanced Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Programs Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Connections Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Content Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Privacy Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the Security Tab in your IE Internet Options Screen
Spycar change blocked

Click here to make Spycar try to remove the General Tab in your IE Internet Options Screen
Spycar change blocked

Network Config Change Tests

Click here to make Spycar try to add an entry to your hosts file (typically c:\windows\system32\drivers\etc\hosts)
Spycar change blocked

Best Regards

[aigle]

That seem to be very nice results!

[aigle]

-{ Quote: "Rolled Back all test entries afterwards without problems.
" }-

What does it mean? RollbackRx or any thing else.

[aigle]

I am waiting for the results from other users here with different appliances. It will be interesting to warch!

[Bubba]

-{ Quote: "What does it mean? RollbackRx or any thing else." }-It means when you run any of those tests you then run their cleanup tool called TowTruck which then reverts your system to its pre-test state....Rollback. Meaning it
removes any of the executable files it was able to drop on your hard drive or any of the registry entries it was able to set.

As a side note....the Network Config Change Test is only applicable to XP or greater.

[Rivalen]

Edit; Sorry Bubba - but your to quick now and not familiar with DW.

The new DW has a Rollback function that means you can erase also the eventual traces of malware that has been deactivated by DW - without using DW Rollback you have to use an AV/AS/AT to remove those harmless deactivated remains of the malware - or you can leave them be because they dont damage your computer - only take up a tiny little space on your hard disk.

This is how I have understood it - cant explain it in tech terms - so I Rolledback afterwards to test that function. I dont think it really means a lot for this instance - since after the test Spycar had a removal function to reverse what it tried to do.

Sorry I cant explain any better.

Best Regards

[Rasheed187]

Well, actually I am very disappointed with these spycar tests, these people are completely missing the point, I mean I can test these things myself! I don´t need any apps for that. I think a test like for example the DFK Threat Simulator is more exciting than Spycar. And Spycar doesn´t even seem to give the correct results. :wacko:

I do not understand why security companies are not coming up with more advanced/smarter tests, maybe because most security tools will perform poorly? This gives me stuff to think about. :shifty:

And btw, I do know about sites like malware.com, but the problem is that most of the exploits do not work anymore, and it´s sometimes difficult to find out if the exploits worked or not. But IMO this is the best way to find out if your tools can protect against remote code execution attacks, the ones that we all fear.

[Bubba]

-{ Quote: "Sorry Bubba - but your to quick now and not familiar with DW" }-The program DW never crossed my mind until now that you make reference to it but being unfamilar with DW is not quite true ;)

My mistake was not seeing correctly what aigle was asking about and my explanation was more to do with how SpyCar does rollback or remove if you will what they placed on your hard drive\registry. Sorry for the oversite :-\

[aigle]

-{ Quote: "The new DW has a Rollback function that means you can erase also the eventual traces of malware that has been deactivated by DW - without using DW Rollback you have to use an AV/AS/AT to remove those harmless deactivated remains of the malware - or you can leave them be because they dont damage your computer - only take up a tiny little space on your hard disk.
" }-

Does this Rollback can be used as recovey sustem as well if system becomes corrupt due to any reason.
Thanks.

[Franklin]

Just an observation.

Ran the spycar tests inside Sandboxie with only ZAP running realtime.Each test stated it was succesful but no changes or warnings could be seen.

I couldn't work out what was happening until I checked out ZAP's logs.

Seems I had inadvertantly restricted "generic host process for windows32 services" the process the spycar tests are using to execute.

There are many other warnings in the log stating that generic host process was denied access communictaing with other programs.

No expert here but I'm fairly sure Generic host process is a needed services but I haven't had any probs with it being restricted.

OH well,seems I will be "googling" the rest of the night trying to work this out.

[Rasheed187]

I still think that these guys are a bunch of amateurs, I mean after all this hype they come up with these simple apps? The only thing these spycar apps do is trying to modify certain registry settings, you can do the same with a lot of other apps, you don´t need spycar for this. Or is it just me? :blink:

[Maji]

-{ Quote: "I still think that these guys are a bunch of amateurs, I mean after all this hype they come up with these simple apps? The only thing these spycar apps do is trying to modify certain registry settings, you can do the same with a lot of other apps, you don´t need spycar for this. Or is it just me? :blink:" }-

It's not just you. These tests are a joke...and to prove it, I executed them on one of the OLD Compaqs we use in my university's network testing laboratory to see how a computer protected ONLY by an anti-virus would fare against these tests. Needless to say I was shocked to discover that not only did some of the tests fail to execute, but that even when they did execute properly, the changes all failed. I made sure to check for things like software restriction policies and other security policies which might be preventing these programs from making their changes, but I could not find anything of the kind. After numerous trials, it became quite clear to me how ineffective these security tests really were. :P

If you want to test youre security, by all means don't use these lame programs. Go with tried and true security testing software or, preferably, get a security expert to perform an audit of your system. In my case, I know several individuals who would be willing to do it for FREE. 8)

[TNT]

-{ Quote: "Well, actually I am very disappointed with these spycar tests, these people are completely missing the point, I mean I can test these things myself! I don´t need any apps for that. I think a test like for example the DFK Threat Simulator is more exciting than Spycar. And Spycar doesn´t even seem to give the correct results." }-I agree. The DFK Threat Simulator might not be perfect, but it sure as hell gives a lot more useful indications than these completely ridiculous tests. I ran some of them in Sandboxie and not only they give very poor indication (for a regular user) of what the threat is, but they don't even report the results correctly.

[EASTER.2010]

-{ Quote: " I think a test like for example the DFK Threat Simulator is more exciting than Spycar." }-

I'm right there on the same page with you guys over SpyCar.

Doesn't come close to Threat Simulator and some others i used in the past. In fact it reminds me more on the order of a RegTick Pro for you fellows familiar with that settings modifier.

[Rivalen]

aigle!

The Rollback in DW is - as far as I understand - there to Rollback entries in the untrusted zone.

So if I act less wisely and installs - as trusted - from a corrupt CD something that also contains malware I will not be able to Rollback that since its not in the untrusted zone/sandbox. Had I installed as untrusted under DW from the same CD I would have been able to Rollback.
How this works is dependant of wether you run DW in expert mode or ordinary mode and if you have added E: (CD)to untrusted or not - so their are some setup options.

I run DW in expert mode - hehe.

With "your" RollbacK you would be able to reverse your PC to any choosen previous Rollback copy - right. So even if you make a mistake for whatever reason - you can Rollback to a clean version.

If a make - in expert mode - a mistake that lets malware into the trusted zone - I cannot DW Rollback that. If I run in ordinary mode and have say A: and D: as untrusted any installed file should also be untrusted and be able to be DW Rollbacked.

Sorry I cant explain it better - this is how I understand DW (until corrected) maybe if you read at their site you get better answers.

Edit; my explanation sounds like Defensewall is a complicated software - its not - its so easy to use - trying to understand it might not be necessary?

Said if Spycar is a poor test - I ll test that other one that was said to be harder and see if there is a thread for exchange of test results from that test.

Best Regards

[aigle]

thanks!

[Rivalen]

Where can I find a working DFK Threat Simulator download link?

Best Regards

[Franklin]

http://www.morgud.com/interests/security/dfk-threat-simulator.asp

[Rivalen]

What am I doing wrong. Dont get adownload at that link. Just send me round in circles. Have you tried it?

Best Regards

[Franklin]

At the bottom of the page.

DFK-Threat-Simulator.zip (zip password: morgud.com)

[Rivalen]

I will have to wait until I get that DFK-link to work. I have heard that DefenseWall passes the test but I want to try myself.

Anybody knows of any other such malware test that is considered to be worth the effort.

Best Regards

[edskoudis]

Ed Skoudis here...

Thank you for your provocative comments. To help clarify the motivation of Spycar and its value, I’ve prepared the following responses to particular issues described in this thread. I'll preface each point made earlier in the thread with a *, followed by my response.

* But, Spycar Only Changes Registry Keys

As you know, Windows is controlled to a massive extent by the Registry. To plant itself on a system, a lot of spyware diddles with various Registry keys, including some of the ones we modeled in Spycar. Some anti-spyware tools try to prevent changes to these keys with their behavior-based defenses. Spycar tries to verify this protection by changing the same Registry keys as the spyware.

Furthermore, not all of Spycar focuses on Registry keys. The alter hosts file element appends an entry to the hosts file itself.

And, finally, please note (as we say at the Spycar website) that we released only the first batch of Spycar modules on Friday, May 5. Call it Spycar 1.0 if you’d like. We've got several other modules up our sleeve, and we have implemented them. The harder part of a tool like Spycar is to roll back the changes in a consistent and comprehensive manner. We're working on implementing those clean-ups in TowTruck and releasing the new modules in the coming weeks. Some of the new modules we're working on include:
- A simple keystroke logger, which will gather just 3 keystrokes (that would not be a mere registry change)
- Importing a code-signing cert into IE
- Importing an SSL cert into IE
- Firefox behavior alteration tools, akin to our current IE suite
- Many others...

* Spycar is Simplistic

It has been pointed out that larger, more complicated applications can test more functionality and model more behavior. But, with the goals of the Spycar project, small and simple beautiful, for several reasons.

First, we wanted anyone (not just technical specialists) to be able to evaluate their anti-spyware tool. Technical experts are welcomed to use the tool. Many have, and have provided highly useful input. But, we also wanted non-experts to be able to give it a spin and evaluate their protection.

Second, in the case of what Spycar is trying to measure, technically speaking, small and simple are desirable. If Spycar were a big, monolithic application testing a whole bunch of items in a single executable, an anti-spyware tool might detect it early in its testing cycle and shut down the testing process. Then, all tests after that would not be accurate. Serious anti-spyware heuristic testing must be atomic if it is to get results from which conclusions can be drawn. Do you let me do this? No... Well, do you let another form of me do that? Yes...

It is important to note that an all-in-one application can test whether a given application is ranking up a score of maliciousness (assigning points to each behavior before deciding to pull the trigger on an application), and shut it down when its score exceeds a threshold. Spycar does not perform that sort of testing, focusing instead on each behavior with a simple question: do you warn me about a process making this change, do you block it, or do you just let it slide by?

And finally, when considering the simplicity of Spycar, consider the EICAR anti-virus test file. Now, there is simplicity for you, and it has provided significant value in verifying anti-virus programs. Spycar is not an exhaustive test (although it has found some interesting results… see below for descriptions of some interesting findings with some vendors), but focuses on modeling certain aspects of spyware behavior.

* The Guys Who Created Spycare Are Amateurs

I cannot comment authoritatively on who is a pro and who is an amateur. Such a conversation would spread more heat than light. But, just to kick in a few thoughts: I've been doing information security product testing for large-scale organizations for over ten years, including crypto products, anti-virus tools, firewalls, IPS products, and anti-spyware tools, for organizations including telcos, banks, government agencies, energy companies, etc. Some of my public test results are located at the following places:

- Anti-virus product testing (June 2004): http://infosecuritymag.techtarget.com/ss/0,295796,sid6_iss407_art803,00.html
- Anti-virus support testing (October 2004): http://infosecuritymag.techtarget.com/ss/0,295796,sid6_iss486_art1005,00.html
- Network-based IPS testing (November 2005): http://informationsecurity.techtarget.com/magIssue/0,291265,sid42_iss1137922,00.html
- Anti-spyware testing, using Spycar, as well as several other methods for evaluation (May 2006): http://informationsecurity.techtarget.com/magLogin/1,291245,sid42_gci1184258,00.html

Putting that aside, consider some of these results we learned with Spycar testing:

One of the major anti-spyware vendors (McAfee) offered no protection for Run, RunOnce, and RunOnceEx reg keys if the process that is changing them has a name greater than 15 characters in length. Their behavior-based protection worked just great unless the process doing the attack had such a name, when such protection would vanish. We discovered this using Spycar, informed the vendor responsibly, and they released a fix within 48 hours.

Another anti-spyware vendor, Webroot, protects Run and RunOnce, but does not properly protect the RunOnceEx registry keys. That's because the structure of successfully written RunOnceEx keys differ from their brethren, a fact not widely known. Again, we discovered this using Spycar, and informed the vendor.

* I Can Do Those Tests By Hand

Sure, you can, for the most part. Of course. But, few people choose to do so (see the findings for the various run registry keys above.) We wanted a test suite that was accessible to lots of testers. The pros can do their own thing. Have at it. By the way, for those who say they can do the tests by hand… have you published your results yet? Please let us know where we can see your hand-based testing results. We’d love to learn from you, and incorporate some of your testing concepts into Spycar.

The reason I said, “for the most part” above is that there are some changes you cannot really test by hand in the same way as Spycar. For example, note the process name greater than 15 character issue mentioned above. If you were to try to make that change by hand, the McAfee tool would block it, because the process making the change would be interpreted as explorer.exe, the Windows GUI. That name is less than 15 characters, so you appear to have protection when you do it by hand. Only with a separate application could you make such a change and verify the protection of the product. Yes, it is something we stumbled upon accidentally in our testing (that happens often, in the testing business). But, it is a significant result, and something that testing by hand would not have uncovered.

* Spycar Didn’t Make the Changes to My Unprotected System

One poster here mentioned that Spycar didn’t make any changes to a system that was unprotected (an old Compaq system). That’s a fascinating finding. Any idea why? Can you either send us a description of the build so we can figure out why, or run RegMon and see where it is getting hung up? As many have pointed out, these tests are very straightforward, so their failure on your box is an interesting outcome. I’d love to know why, but cannot discern from the sketchy details in your post. We’ve had many hundreds of people run Spycar successfully, so your results are a fascinating outlier.

* Spycar was Overhyped

Spycar does no more and no less than we promised up front. In all of our interactions with people, we explained as clearly as we could what Spycar would do. We got a tremendous amount of positive feedback up front, from very large software companies that I cannot name here, about the idea. Since its release, we've gotten a lot of enthusiastic e-mails from both individual consumers and IT professionals who have said they were shocked at the lack of protection they have on their machines. In the end, that's why we released it... so people could test their protection and see if it matched their assumptions.

If you have other questions or comments about Spycar, please do let us know.

Thanks for the input and challenging points—
--Ed Skoudis
Senior Security Analyst
Intelguardians

[ronjor]

Welcome to Wilders Ed and thank you for your post.

[edskoudis]

Thank you, Ron. It's good to be here.

[Rivalen]

Thanks Ed Skoudis - interesting reading - any eta on these new expanded tests?

Franklin! - can you download from that link you gave me? I ran iexplore as trusted by Defensewall but I simply cant download.

Best Regards

[beetlejuice69]

Good read Ed and thanks...oh and welcome. :)

[Devil's Advocate]

-{ Quote: "I still think that these guys are a bunch of amateurs, I mean after all this hype they come up with these simple apps? " }-

-{ Quote: "

I cannot comment authoritatively on who is a pro and who is an amateur.
" }-

Heh so modest.

-{ Quote: "
Such a conversation would spread more heat than light. But, just to kick in a few thoughts: I've been doing information security product testing for large-scale organizations for over ten years, including crypto products, anti-virus tools, firewalls, IPS products, and anti-spyware tools, for organizations including telcos, banks, government agencies, energy companies, etc.
" }-

Hi Ed, no need to list your list of credentials, you are well known or should be, for people who really are in the know. Considering the known abilites of the poster who called you an ameteur, it's pretty hilarous I think. :)

PS I enjoyed reading your 'Counterhack' books .*Back to lurk mode.*

[JimIT]

Hi Ed! Cool to see you in these parts!!

;)

[Rasheed187]

OK thanks for the feedback, I understand it all better now. My comments were based upon the facts that I had expected a bit more advanced tests as you have noticed. That´s why I was not impressed at all, and when I called you guys amateurs, I meant that guys with your background should have come up with something better. But I see that you´re coming up with more interesting stuff, nice to know. Also nice to see that you´ve actually discovered flaws in certain products. :)

But yes it´s true, a lot of anti spyware apps do not offer strong real time protection and even more advanced HIPS can not always correctly detect certain (possible malicious) changes made to a system. I´ve tested this with all kind of applications (including registry tweakers, startup control, process/service/driver tools etc.).

@ DA

Nice to see you back, I´m surprised that you don´t have anything negative to say about these tests, after all you wasn´t too impressed with other more advanced tests. Please post more often we really need more posts from "experts" like you. And thanks for providing me with so much fun during our little private conversation via PM´s. But I see you have finally decided to take my advice, I hope you sleep better now, kuddos! ;D

[edskoudis]

Rasheed187,

Thank you for your input. And, I agree with you... there's a lot of great ways to analyze and test things, with reg tweakers, DLL injection tools, and all kinds of fun stuff. Testing can be complex or simple, and each measures it's own arena with thought-provoking results.

As for your comment on HIPS stuff... that's really cool. I think the ground is very fertile for testing of the burgeoning HIPS realm. I look forward to others kicking the tires on some of those product in the near future.

I usually do one big round of testing in one space per year. 2 years ago was anti-virus. Last year was network-based IPS. This year is anti-spyware. I'm thinking in 2007 of focusing on HIPS, but would love to see others embark on this realm sooner. I typically spend a few months getting my mind around what the vendors are trying to accomplish, and then thinking about assumptions they may have made. I devise a test regimen that can be consistently applied across various vendors. Then, we roll into testing. The 2007 HIPS analysis isn't a commitment, by the way... just something I'm thinking about.

But, back to Spycar... The next round of Spycar will include some more interesting stuff (at least I think it's more interesting... the import certificate stuff promises to be). The first round of Spycar modules was to get things rolling, and focused on straight-forward tests that we honestly expected all of the anti-spyware tools to handle. We were surprised by some of their gaps. We also wanted our first release to be limited just to make sure TowTruck backed out changes ok. It seems to have done reasonably well (with a few exceptions that we're following up on).

We've started refining the GUI to make the results more understandable, while adding more test modules.

To answer Rivalen about ETA for new modules, I just spoke with Tom Liston, my colleague at Intelguardians, and we expect more tests to be released in 2 weeks or so. I'll put a post in this thread when they come out.

And, thanks for your well-wishes, JimIT and Devils Advocate. Lurk no more, DA! :)

Thanks again, guys!
--Ed.

[edskoudis]

BTW, I should say that the DFK Threat Simulator is a solid piece of work. Very interesting tests, in a comprehensive package. They chose a different approach to our more atomic testing in Spycar. It is also interesting their reference back to the truly pioneering work of EICAR. We do stand on the shoulders of giants, or a least really, really tall people. ;)

--Ed.

[Inspector Clouseau]

I didn't take a look at the test procedure from spycar, but just blocking "suspicious" registry entries i consider myself as unreliable. Why? Assuming you test the adding of so called browser helper objects. Not every browser helper object is malicious/spyware - there are several clean and useful plugins. So judging this based on registry blocking is not a reasonable way to test it. Instead of this it's more important to know what the binary actually does. I mean of course you can block a lot from adding to the registry, but who from the normal users knows what is really a "bad" entry? You have to take a look for this into the binaries of the corrospondending DLL or OCX files, before making a final conclusion what is bad and what not (expect the already wellknown registry hacks without binaries) because a binary could be completely different even if it would have the same class id than a malicious one. If you rename notepad.exe into svchost.exe and put this into the windows folder instead of the system folder does this make the innocent application notepad malicious just because it uses a technique which is widely used by trojans? No! It's still a innocent notepad file which just has another name (it's a god given right to rename files with administrator rights into whatever) And if i decide now to add this into autorun then it's also my right without getting bothered that it's malware. So basically there's absolutely no way around except to know the programs which are behind these keys.

[dja2k]

Well don't know if it was mentioned before, but ewido 4 beta blocked all the spycar test and wouldn't even let me run them. Now its time to run them without ewido to see what other defense I have against them :P

dja2k

[notageek]

Spycar Scoring
HKCU_Run : Spycar test not performed
HKCU_RunOnce : Spycar test not performed
HKCU_RunOnceEx : Spycar test not performed
HKLM_Run : Spycar change allowed
HKLM_RunOnce : Spycar test not performed
HKLM_RunOnceEx : Spycar test not performed
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar change blocked

My test results using MS defender. Unless I have MS defender setup wrong, it's not blocking some stuff.

[lotuseclat79]

Have you reported the results back to the Windows Defender group at MS?

-- Tom

[notageek]

Not yet, I'm on my way over to report it in a few minutes.

I'm going to try the test with RegDefend enabled to see the results.

[Bubba]

-{ Quote: "My test results using MS defender. Unless I have MS defender setup wrong, it's not blocking some stuff." }-
-{ Quote: "IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed" }-I do not know of any anti-spyware programs that monitor the Policies\Microsoft\Internet explorer\Control panel registry keys given the fact the fast buck for the malware authors is to alter a users Home page\Search page\Hosts file....which MS defender did block.

-{ Quote: "I'm going to try the test with RegDefend enabled to see the results." }-Your results will be remarkebly different especially if you use puff-m-d's (http://www.wilderssecurity.com/member.php?u=42402) and\or TonyKlein's Ghost files (http://www.wilderssecurity.com/showthread.php?p=676747#post676747) ;)

[notageek]

Thanks bubba.

[Bubba]

-{ Quote: "Thanks bubba." }-My plaeasure ;)

[duke1959]

Not familiar with this test other than what I read on their website, and here, but has anyone tried this with Cyberhawk? Or how about with Spyware Terminator with HIPS turned on, and all Guards set to stop threats and unknown access? I have AVG ISS which has AVG Antispyware, but I'm not sure I want to try this test just yet.

[Blitzen]

Just signed on to your site and I'm finding it quite interesting. Got some rather distrubing results with that test, as I am paying for Spyware Doctor and here's what I got:

Spycar Scoring
HKCU_Run : Spycar change allowed
HKCU_RunOnce : Spycar change allowed
HKCU_RunOnceEx : Spycar change allowed
HKLM_Run : Spycar change allowed
HKLM_RunOnce : Spycar change allowed
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change allowed
IE-SetSearchPage : Spycar change allowed
AlterHostsFile : Spycar change allowed

I may not be renewing in 20 days...:blink:

[duke1959]

Hey Blitzen. Other than the unfavorable results, I guess you didn't have any trouble with you're PC after trying this test then, correct?

[Blitzen]

-{ Quote: "Hey Blitzen. Other than the unfavorable results, I guess you didn't have any trouble with you're PC after trying this test then, correct?" }-

Not sure what kind of anormal behavior I should be looking for. Seems OK but I just did the test. I am now dloading spyware terminator and will retry the test using it and spyware blaster. I am a noob at this so here's hoping this improves!

[duke1959]

I was just wondering how safe this test was Blitzen, as some of other types of tests have caused PC problems from what I've heard. When trying it with Spyware Terminator, if you want to, do it with no HIPS enabled and then see if it makes a difference with it enabled. I think you have to do a full scan first though, before you can enable the HIPS if my memory serves me. There are also different settings for it's Guards too. Just asking, as you certainly don't have to listen to me. I'm just curious as to any results, because I'm probably going to try this test with AVG ISS soon. Take care, and hope to hear back from you.

[Blitzen]

-{ Quote: "I was just wondering how safe this test was Blitzen, as some of other types of tests have caused PC problems from what I've heard. When trying it with Spyware Terminator, if you want to, do it with no HIPS enabled and then see if it makes a difference with it enabled. I think you have to do a full scan first though, before you can enable the HIPS if my memory serves me. There are also different settings for it's Guards too. Just asking, as you certainly don't have to listen to me. I'm just curious as to any results, because I'm probably going to try this test with AVG ISS soon. Take care, and hope to hear back from you." }-

Just doing the full scan now. Almost done and then I'll test with HIPS on.

[Blitzen]

So here's what I got with Spyware Terminator (w/Resident Shield and HIPS up) and spyware blaster:

Spycar Scoring
HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar test not performed

I very rarely use IE so I guess this is much better than what I had before. Forgot to add that I also run antivir. I only have a hardware firewall so I'll try to add a software add-on as well and rerun.

[Tommy]

Only four words:
Boclean stopped them all :thumb:

[Blitzen]

So now I've also added Regdefend with TK's rules and everything was stopped.

[Hyperion]

WinPooch stops the registry and host file change,allows the IE tests.

[duke1959]

Anyone willing to try Cyberhawk?

[MaB69]

Hi all,

Tried Spycar against Arovax Shield 2.0.70 and here the excellent result :

Spycar Scoring
HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change blocked
IE-KillAdvancedTab : Spycar change blocked
IE-KillConnectionsTab : Spycar change blocked
IE-KillContentTab : Spycar change blocked
IE-KillGeneralTab : Spycar change blocked
IE-KillPrivacyTab : Spycar change blocked
IE-KillProgramsTab : Spycar change blocked
IE-KillSecurityTab : Spycar change blocked
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar test not performed ( because i deleted it (the host file) )

[duke1959]

Wow, good for Arovax Shield. I always thought AS was more practical than Cyberhawk for the average safe surfer, and I think this test shows this may be true. Of course there is a possible Zero Day Threat that CH would hopefully handle, but for daily safe surfing it looks like AS is a better choice to add as a lightweight HIPS, don't you agree? It even bested ST. Are these reliable tests though, would be my only other question.

[duke1959]

Just want to add that read a Washington Post article by someone named Brian Krebs (Brians Blo) who used Spycar to test Windows Defender. It only blocked one thing, and failed to block IE changes. You would think this should be the very thing it should protect against. Not sure how long ago the test was done, but it looks like it may have been around Spring of 2006, so I don't know how WD would fair now.

[ggf31416]

-{ Quote: "Anyone willing to try Cyberhawk?" }-

CyberHawk detects the Spycar tests by signature (red popup). However if the files are modified, for example using an exe packer, CyberHawk shows an yellow popup (suspicious) for all tests, so CyberHawk monitor the files and keys.

-{ Quote: "I guess you didn't have any trouble with you're PC after trying this test then, correct?" }-

Some antivirus detects the files that the RunOnceEx tests add to startup. If System restore makes a backup copy you will need to exclude the files if you don't want to lose the restore points.

[Blitzen]

-{ Quote: "Just want to add that read a Washington Post article by someone named Brian Krebs (Brians Blo) who used Spycar to test Windows Defender. It only blocked one thing, and failed to block IE changes. You would think this should be the very thing it should protect against. Not sure how long ago the test was done, but it looks like it may have been around Spring of 2006, so I don't know how WD would fair now." }-

I think someone posted similar results here for WD...in this same thread I think.

[ejr]

Spyware Doctor 4.0:

HKLM_RunOnceEx : Spycar change blocked

Allowed all others.

I don't know whether to question the validity of the test or my AS program.

[Blitzen]

-{ Quote: "Spyware Doctor 4.0:

HKLM_RunOnceEx : Spycar change blocked

Allowed all others.

I don't know whether to question the validity of the test or my AS program." }-

So I guess I'm not hallucinating then. Just sent the following e-mail to tech service at SD:

My license expires in under 20 days and I just thought I'd look around to kick the tires of other apps like Spyware Doctor. In doing so, I ran into a security test called Spycar and tried it on SD. To my dismay, SD did not block a single thing. Unless some clear explanation of this can be brought forth, I'm thinking I won't be renewing, especially since free software that I installed afterwards managed to block everything thrown at it.

Thoughts?

Thanks!

[Firefighter]

-{ Quote: "Anyone willing to try Cyberhawk?" }-Hmm! I'm gonna switch to ArovaxShield right now. Here are my results with Cyberhawk when the resident shields in AVG AntiVirus and AntiSpyware were disabled. Damn! ???

Best regards,
Firefighter!

[duke1959]

Firefighter, did you try it with AVG Spyware Shield enabled? I was going to, but if decide you want to try it I would appreciate it. I didn't even download the Spycar Test yet and won't be able to try it until tomorrow. It would be nice to see your results with AVG AS enabled. I have AVG ISS, but have added Arovax Shield today. Take care.

[Firefighter]

-{ Quote: "Firefighter, did you try it with AVG Spyware Shield enabled? I was going to, but if decide you want to try it I would appreciate it. I didn't even download the Spycar Test yet and won't be able to try it until tomorrow. It would be nice to see your results with AVG AS enabled. I have AVG ISS, but have added Arovax Shield today. Take care." }-AVG AntiSpyware 7.5 failed in all tests.

ArowaxShield, although the notifier jumped up in every test and I made a Block rule against all of them, scored only a bit better than Cyberhawk. ???

Best regards,
Firefighter!

[Firefighter]

-{ Quote: "Only four words:
Boclean stopped them all :thumb:" }-My BOClean 4.22.002 was almost as good as yours. The only failed one was deleted after the second try. :) I'm still confused. Because BOClean deleted all my test files except the report/clean one, was that only because it has signatures to them? :-\

But if BOClean really is capable to protect all these kind of attacks, should I throw AVG Anti-Spyware away and all my HIPS too, so that my only security applications should be COMODO Free Firewall, SpywareBlaster, BOClean and AVG Antivirus 7.5? :'(

Best regards,
Firefighter!

[Tommy]

Lets say it this way for my case.
What SSM does not catch, catches BoClean and reverse, as a lot of registry entries are observed with SSM.

In your case i think it is not nececarry to run two background watcher like BoClean and AVG simultaniasly. I would use AVG for on demmand scan as this feature lacks in BoClean.

[MaB69]

-{ Quote: "AVG AntiSpyware 7.5 failed in all tests.

ArowaxShield, although the notifier jumped up in every test and I made a Block rule against all of them, scored only a bit better than Cyberhawk. ???

Best regards,
Firefighter!" }-

Hi everybody,

Firefighter, i think you unchecked " Guard Windows Policies " or there is something wrong with your install

MaB

[Firefighter]

-{ Quote: "Hi everybody,

...or there is something wrong with your install

MaB" }-There may be something wrong in my just full patched/reinstalled WinXP Home. I can't make a full C:\ scan with DrWeb, F-Prot and some other av:s for instance. Sometimes they'll shut down this laptop when the scan reaches HP PSC 1510 system files and sometimes in Windows/System32 folder files. I only have 448 MB RAM and everything possible installed in this d...ed laptop. ???

-{ Quote: "

Firefighter, i think you unchecked " Guard Windows Policies " " }-No, the only what was unchecked was the Opera Browser, because I haven't that one. Look at the setup.

Best regards,
Firefighter!

[ejr]

-{ Quote: "So I guess I'm not hallucinating then. Just sent the following e-mail to tech service at SD:

My license expires in under 20 days and I just thought I'd look around to kick the tires of other apps like Spyware Doctor. In doing so, I ran into a security test called Spycar and tried it on SD. To my dismay, SD did not block a single thing. Unless some clear explanation of this can be brought forth, I'm thinking I won't be renewing, especially since free software that I installed afterwards managed to block everything thrown at it.

Thoughts?

Thanks!" }-


What freeware did you install that blocked all?

[ejr]

QUESTION: Howa re you all copying and pasting your results here? What key sequence do you use?

I was not able to highlight the results inside TOW TRUCK and copy them.

[duke1959]

Just want to say thanks to you Firefighter for the results. Also, how do like Arovax Shield so far?

[ggf31416]

My Results:

Arovax Shield

HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change blocked
IE-KillAdvancedTab : Spycar change blocked
IE-KillConnectionsTab : Spycar change blocked
IE-KillContentTab : Spycar change blocked
IE-KillGeneralTab : Spycar change blocked
IE-KillPrivacyTab : Spycar change blocked
IE-KillProgramsTab : Spycar change blocked
IE-KillSecurityTab : Spycar change blocked
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar change blocked

CyberHawk (blocked by signatures)

HKCU_Run : Spycar test not performed
HKCU_RunOnce : Spycar test not performed
HKCU_RunOnceEx : Spycar test not performed
HKLM_Run : Spycar test not performed
HKLM_RunOnce : Spycar test not performed
HKLM_RunOnceEx : Spycar test not performed
IE-HomePageLock : Spycar test not performed
IE-KillAdvancedTab : Spycar test not performed
IE-KillConnectionsTab : Spycar test not performed
IE-KillContentTab : Spycar test not performed
IE-KillGeneralTab : Spycar test not performed
IE-KillPrivacyTab : Spycar test not performed
IE-KillProgramsTab : Spycar test not performed
IE-KillSecurityTab : Spycar test not performed
IE-SetHomePage : Spycar test not performed
IE-SetSearchPage : Spycar test not performed
AlterHostsFile : Spycar test not performed

Spyware Terminator
HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar test not performed

WinPatrol Free

HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar change blocked

[dah145]

KIS passes them all. :)

[Firefighter]

-{ Quote: "QUESTION: Howa re you all copying and pasting your results here? What key sequence do you use?

I was not able to highlight the results inside TOW TRUCK and copy them." }-Just capture 3 images from TowTruck by Gadwin PrintScreen 3.5 and link them together to one picture in M$ Powerpoint. ;)

Best regards,
Firefighter!

[Firefighter]

-{ Quote: "My Results:

Arovax Shield" }-Have you turned IE7 on during the test, I didn't?

-{ Quote: "CyberHawk (blocked by signatures)" }-BOClean too. Is this kind of blocking a valid result in this kind of test? Even my AVG Antivirus 7.5 alerted when I tried to execute the TowTruck.exe file during my first scans. ;D

Best regards,
Firefighter!

[Firefighter]

-{ Quote: "Hi everybody,

Firefighter, i think ... or there is something wrong with your install

MaB" }-I have this problem too when I used Cyberhawk. First of all I made a new system restore point in my WiXP Home system before this test. Then I disabled resident shields in AVG Antivirus and Antispyware. After that I activated Trojan-Downloader.Win32.Zlob.asf as you can see in my VirusTotal scan, but Cyberhawk was silent. :-\ ???

Best regards,
FF again

[ejr]

Based on countless reviews I have seen on my antispyware tool, I know that I have chosen an effective product. But it fails almost all of the Spycar tests.

This makes me think that there may be something inherently wrong with the test. I am not certain that I trust the results.

[Blitzen]

-{ Quote: "What freeware did you install that blocked all?" }-

I installed Spyware Terminator and RegDefend (I know it is technically shareware). Still waiting for the answer from SD tech service. They sent me a first response saying that SD is compatible with Windows XP. ??? :wacko: So I asked them to try again.

[Blitzen]

-{ Quote: "Based on countless reviews I have seen on my antispyware tool, I know that I have chosen an effective product. But it fails almost all of the Spycar tests.

This makes me think that there may be something inherently wrong with the test. I am not certain that I trust the results." }-

What do you use? Spyware Doctor kept getting great reviews and that's what I've had for a year now. Just want to know what their tech folks have to say about failing tests. I read in one place one guy saying that it's still the best at removing malware but isn't it better to prevent it in the first place?

[tobacco]

-{ Quote: "I am not certain that I trust the results." }-

And never trust reviews!.

[lucas1985]

Folks, you are misunderstanding these tests. Most (if not all) antispyware are mainly signature-based scanners. Spycar should be tested against behaviour blockers, HIPS and antispyware with HIPS functions.

[Blitzen]

-{ Quote: "Folks, you are misunderstanding these tests. Most (if not all) antispyware are mainly signature-based scanners. Spycar should be tested against behaviour blockers, HIPS and antispyware with HIPS functions." }-

That's fine but I just want to see if SD will fess up to the fact that their software doesn't cover that.

[Firefighter]

-{ Quote: "Just want to say thanks to you Firefighter for the results." }-Nice to see that! -{ Quote: "Also, how do like Arovax Shield so far?" }- Arovax has so far behaved smoothly, even lower CPU impact than Cyberhawk had, at least the feeling in surfing. Yet I want to resolve one problem and it was the test issue. Has anyone else got the same results? :-\

Best regards,
Firefighter!

[Hyperion]

-{ Quote: "Nice to see that! Arovax has so far behaved smoothly, even lower CPU impact than Cyberhawk had, at least the feeling in surfing. Yet I want to resolve one problem and it was the test issue. Has anyone else got the same results? :-\

Best regards,
Firefighter!" }-

Arovax is IMHO as light as it can get for a program of its class.Even Winpatrol that is a poller eats more resources.

The screenshot is CPU/PID/CPU Time/running since time/RAM.

[Firefighter]

-{ Quote: "QUESTION: Howa re you all copying and pasting your results here? What key sequence do you use?

I was not able to highlight the results inside TOW TRUCK and copy them." }-OK! Here it is.

1. In "Preferences" Tab, take the Hotkey = "PrintScreen", "Hide icon when capturing", "Preview captured image" and "Run at Windows startup" options

2. In "Source" Tab, take the "Rectangular area" option + "Capture Mouse cursor"

3. In "Destnation" Tab, take at least the "Copy captured area to a File" and "Ask for the name after capturing" options plus in "Open with", I have "C:\Program Files\IrfanView\i_view32.exe", because it's just my favourite viewer

4. In "Image" Tab, take the "JPEG Bitmap" format

After this you can split the picture in the parts like this and combine these together afterwards in M$ PowerPoint.

Best regards,
Firefighter

[Firefighter]

Is it possible to run SpywareTerminator with HIPS enabled and ArovaxShield at the same time when surfing? Just because SpywareTerminator is much CPU friendly than AVG Anti-Spyware 7.5 with resident shield enabled. >:(

Best regards,
Firefighter!

[Ice_Czar]

-{ Quote: "
Result - HIPS is as good as its user. And that's the Catch 22.
You want to use HIPS to keep malware away. But you need to be proficient enough to use HIPS properly. But if you can handle HIPS messages - you don't need HIPS! As simple as that.
HIPS are mostly used by people like here at Wilders, who want control of many aspects of their pc, this is a hobby and they like to feel complex. But for those who really need help against malware, HIPS are useless." }-

which overlooks the excellent aspect of HIPS in teaching the clueless about security. Most any user can disable their security, but the real value of a HIPS is to just one more time warn someone that maybe, just maybe they shouldnt just click allow.

I configure a computer or two a month for folks, I generally do it from the ground up, install and run over 90% of the software it will ever see, show them what it takes for them to install software past the HIPS and basically tell them if they aren't installing a completely trusted ap, and this window ever pops up (see this screen here? very bad juju) they need to think really hard about if they give a damn. Because if they screw it up, the only option I'll provide is to re-image it to the day I gave it to them. I can't after the fact fix it and guarantee its 100% clean these days.
(their data is on a separate partition, but all aps, rules customizations ect will be lost)

And of course the clueless's little rugrats generally aren't even told about the HIPS so it simply autoblocks with no prompting for a rule. They'd need to master cracking the admin account before they figure out why,
at which point I generally make then the admin. :p
Its pretty gratifying to see a clueless n00b running a hash on a bit of freeware and thinking about it a little more before hitting that button ;)

paranoia is a contagion, just call me Typhoid Mary :-*


spycar\towcar doesnt seem to get very far on my boxes
"You need to actually run some tests before attempting to cleanup"


it does seem rather simplistic was expecting something more along the lines of ATK (http://www.computec.ch/projekte/atk/) but for spyware, with various new plugins available

[duke1959]

I just wonder why Cyberhawk doesn't do better with this test? It did so well in that other test that a thread was started about here not too lonng ago.

[Rasheed187]

This Spycar thing is just a simple registry modifier, only apps that are protecting the registry keys will pass them all, but what I´m trying to say is that you guys need a good registry protector, like for example KAV/KIS, ProSecurity or SSM Pro. It´s cool that Arovax Shield passed the test but what if spyware will try to modify other registry keys that Arovax does not cover?

Also, most AS tools do not protect the registry for some reason, so that´s why it´s no surprise that most tools fail. You got to put this test in perspective, I really don´t know why some of you are getting all excited by this simple and IMO quite silly test. ::)