Put your Anti-Spyware Apps to the Test!

Rasheed187,

Thank you for your input. And, I agree with you... there's a lot of great ways to analyze and test things, with reg tweakers, DLL injection tools, and all kinds of fun stuff. Testing can be complex or simple, and each measures it's own arena with thought-provoking results.

As for your comment on HIPS stuff... that's really cool. I think the ground is very fertile for testing of the burgeoning HIPS realm. I look forward to others kicking the tires on some of those product in the near future.

I usually do one big round of testing in one space per year. 2 years ago was anti-virus. Last year was network-based IPS. This year is anti-spyware. I'm thinking in 2007 of focusing on HIPS, but would love to see others embark on this realm sooner. I typically spend a few months getting my mind around what the vendors are trying to accomplish, and then thinking about assumptions they may have made. I devise a test regimen that can be consistently applied across various vendors. Then, we roll into testing. The 2007 HIPS analysis isn't a commitment, by the way... just something I'm thinking about.

But, back to Spycar... The next round of Spycar will include some more interesting stuff (at least I think it's more interesting... the import certificate stuff promises to be). The first round of Spycar modules was to get things rolling, and focused on straight-forward tests that we honestly expected all of the anti-spyware tools to handle. We were surprised by some of their gaps. We also wanted our first release to be limited just to make sure TowTruck backed out changes ok. It seems to have done reasonably well (with a few exceptions that we're following up on).

We've started refining the GUI to make the results more understandable, while adding more test modules.

To answer Rivalen about ETA for new modules, I just spoke with Tom Liston, my colleague at Intelguardians, and we expect more tests to be released in 2 weeks or so. I'll put a post in this thread when they come out.

And, thanks for your well-wishes, JimIT and Devils Advocate. Lurk no more, DA!

Thanks again, guys!
--Ed.

 

BTW, I should say that the DFK Threat Simulator is a solid piece of work. Very interesting tests, in a comprehensive package. They chose a different approach to our more atomic testing in Spycar. It is also interesting their reference back to the truly pioneering work of EICAR. We do stand on the shoulders of giants, or a least really, really tall people.

--Ed.

 

I didn't take a look at the test procedure from spycar, but just blocking "suspicious" registry entries i consider myself as unreliable. Why? Assuming you test the adding of so called browser helper objects. Not every browser helper object is malicious/spyware - there are several clean and useful plugins. So judging this based on registry blocking is not a reasonable way to test it. Instead of this it's more important to know what the binary actually does. I mean of course you can block a lot from adding to the registry, but who from the normal users knows what is really a "bad" entry? You have to take a look for this into the binaries of the corrospondending DLL or OCX files, before making a final conclusion what is bad and what not (expect the already wellknown registry hacks without binaries) because a binary could be completely different even if it would have the same class id than a malicious one. If you rename notepad.exe into svchost.exe and put this into the windows folder instead of the system folder does this make the innocent application notepad malicious just because it uses a technique which is widely used by trojans? No! It's still a innocent notepad file which just has another name (it's a god given right to rename files with administrator rights into whatever) And if i decide now to add this into autorun then it's also my right without getting bothered that it's malware. So basically there's absolutely no way around except to know the programs which are behind these keys.

 

Well don't know if it was mentioned before, but ewido 4 beta blocked all the spycar test and wouldn't even let me run them. Now its time to run them without ewido to see what other defense I have against them

dja2k

 

Spycar Scoring
HKCU_Run : Spycar test not performed
HKCU_RunOnce : Spycar test not performed
HKCU_RunOnceEx : Spycar test not performed
HKLM_Run : Spycar change allowed
HKLM_RunOnce : Spycar test not performed
HKLM_RunOnceEx : Spycar test not performed
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar change blocked

My test results using MS defender. Unless I have MS defender setup wrong, it's not blocking some stuff.

 

Have you reported the results back to the Windows Defender group at MS?

-- Tom

 

Not yet, I'm on my way over to report it in a few minutes.

I'm going to try the test with RegDefend enabled to see the results.

 

I do not know of any anti-spyware programs that monitor the Policies\Microsoft\Internet explorer\Control panel registry keys given the fact the fast buck for the malware authors is to alter a users Home page\Search page\Hosts file....which MS defender did block.

Your results will be remarkebly different especially if you use puff-m-d's and\or TonyKlein's Ghost files

 

Thanks bubba.

 

My plaeasure

 

Not familiar with this test other than what I read on their website, and here, but has anyone tried this with Cyberhawk? Or how about with Spyware Terminator with HIPS turned on, and all Guards set to stop threats and unknown access? I have AVG ISS which has AVG Antispyware, but I'm not sure I want to try this test just yet.

 

Just signed on to your site and I'm finding it quite interesting. Got some rather distrubing results with that test, as I am paying for Spyware Doctor and here's what I got:

Spycar Scoring
HKCU_Run : Spycar change allowed
HKCU_RunOnce : Spycar change allowed
HKCU_RunOnceEx : Spycar change allowed
HKLM_Run : Spycar change allowed
HKLM_RunOnce : Spycar change allowed
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change allowed
IE-SetSearchPage : Spycar change allowed
AlterHostsFile : Spycar change allowed

I may not be renewing in 20 days...

 

Hey Blitzen. Other than the unfavorable results, I guess you didn't have any trouble with you're PC after trying this test then, correct?

 

Not sure what kind of anormal behavior I should be looking for. Seems OK but I just did the test. I am now dloading spyware terminator and will retry the test using it and spyware blaster. I am a noob at this so here's hoping this improves!

 

I was just wondering how safe this test was Blitzen, as some of other types of tests have caused PC problems from what I've heard. When trying it with Spyware Terminator, if you want to, do it with no HIPS enabled and then see if it makes a difference with it enabled. I think you have to do a full scan first though, before you can enable the HIPS if my memory serves me. There are also different settings for it's Guards too. Just asking, as you certainly don't have to listen to me. I'm just curious as to any results, because I'm probably going to try this test with AVG ISS soon. Take care, and hope to hear back from you.

 

Just doing the full scan now. Almost done and then I'll test with HIPS on.

 

So here's what I got with Spyware Terminator (w/Resident Shield and HIPS up) and spyware blaster:

Spycar Scoring
HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change allowed
IE-KillAdvancedTab : Spycar change allowed
IE-KillConnectionsTab : Spycar change allowed
IE-KillContentTab : Spycar change allowed
IE-KillGeneralTab : Spycar change allowed
IE-KillPrivacyTab : Spycar change allowed
IE-KillProgramsTab : Spycar change allowed
IE-KillSecurityTab : Spycar change allowed
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar test not performed

I very rarely use IE so I guess this is much better than what I had before. Forgot to add that I also run antivir. I only have a hardware firewall so I'll try to add a software add-on as well and rerun.

 

Only four words:
Boclean stopped them all

 

So now I've also added Regdefend with TK's rules and everything was stopped.

 

WinPooch stops the registry and host file change,allows the IE tests.

 

Anyone willing to try Cyberhawk?

 

Hi all,

Tried Spycar against Arovax Shield 2.0.70 and here the excellent result :

Spycar Scoring
HKCU_Run : Spycar change blocked
HKCU_RunOnce : Spycar change blocked
HKCU_RunOnceEx : Spycar change blocked
HKLM_Run : Spycar change blocked
HKLM_RunOnce : Spycar change blocked
HKLM_RunOnceEx : Spycar change blocked
IE-HomePageLock : Spycar change blocked
IE-KillAdvancedTab : Spycar change blocked
IE-KillConnectionsTab : Spycar change blocked
IE-KillContentTab : Spycar change blocked
IE-KillGeneralTab : Spycar change blocked
IE-KillPrivacyTab : Spycar change blocked
IE-KillProgramsTab : Spycar change blocked
IE-KillSecurityTab : Spycar change blocked
IE-SetHomePage : Spycar change blocked
IE-SetSearchPage : Spycar change blocked
AlterHostsFile : Spycar test not performed ( because i deleted it (the host file) )

 

Wow, good for Arovax Shield. I always thought AS was more practical than Cyberhawk for the average safe surfer, and I think this test shows this may be true. Of course there is a possible Zero Day Threat that CH would hopefully handle, but for daily safe surfing it looks like AS is a better choice to add as a lightweight HIPS, don't you agree? It even bested ST. Are these reliable tests though, would be my only other question.

 

Just want to add that read a Washington Post article by someone named Brian Krebs (Brians Blo) who used Spycar to test Windows Defender. It only blocked one thing, and failed to block IE changes. You would think this should be the very thing it should protect against. Not sure how long ago the test was done, but it looks like it may have been around Spring of 2006, so I don't know how WD would fair now.

 

CyberHawk detects the Spycar tests by signature (red popup). However if the files are modified, for example using an exe packer, CyberHawk shows an yellow popup (suspicious) for all tests, so CyberHawk monitor the files and keys.

Some antivirus detects the files that the RunOnceEx tests add to startup. If System restore makes a backup copy you will need to exclude the files if you don't want to lose the restore points.