Anti-Malware Testing and Ratings

I am wondering if there is any third party testing and rating of anti-spyware apps which is similar to AV-Comparative.

 

http://spywarewarrior.com/asw-test-results-1.htm
http://www.mylovelyapps.com/comp/antispyware-test2009.htm

found this

 

Jmonge,

Thanks for the reference. This is from 2004. I am looking for something that is more current.

 

sorry man let me find more in google

 

Thanks for the additional link.

 

your welcome,did you find more info?

 

"Sticky: Thoughts on Anti-Spyware Testing" (stickied since 2006)
http://www.spywarewarrior.com/viewtopic.php?t=22210

So now, late 2009, the independent 'ntinternals' guy tests, publishes his results
see: (anti-rootkit) Hidden Process Detection (50+ products compared
https://www.wilderssecurity.com/showthread.php?t=259876

...and his audience splits hairs, caught up in debating the merit in the tester's methodology and in discounting the relevance of the results. With an audience so hellbent on dismissing any (painfully) significant results, it seems reasonable to expect publication of comprehensive 'independent' test results to remain scarce.

"The emperor has no clothes!"

 

Inka,

Well said.

If the consumer is deprived of legitimate testing results, then the antispyware companies can remain unaccountable and enjoy a marketing approach to business rather than a results approach.

 

Hi Inka,

Boy, you nailed that. Every test that shows an application in less that tops is labeled as bad methodology or a result of paid ads in a magazine or bribes.

Regards,
Jerry

 

JerryM ans Inka this is very true in reality when it comes to money people even still or even kill for money my 2 pesos

 

Yes, and I'm glad you did not say your 2 million pesos or you might be in mortal danger.

Best,
Jerry

 

andale orale yes man thanks again

 

Unfortunately this seems to be more common than extracting and discussing the useful data. There are no "perfect" tests but most at least have some useful information and the one you were referring to I found interesting.

 

IMO a test should go like this .

Start with a list of infected sites and direct download URLs , no sample packs allowed . Next set up the test machine(s) with web facing software that is reasonably old (so exploits can take) . Setting the OS to its current SP with no further updates is a good way to set it a reasonable amount back (again to make exploits kick in) . Update and activate protection just before the test starts .

Stage one . Check web protection first . Any sample that can be created by an exploit or downloaded through web protection should be collected for stage two .

Stage two . Make a copy of the files that made it through stage one , they are needed twice . Execute each sample to determine if it is blocked by signatures/heuristics/whatever . All samples that able to execute will move on to stage three .

Stage three . Retest the samples that are not blocked at primary execution and check for secondary detection . The majority of malware does nothing if everything from secondary execution on is blocked . Obviously this is not as good as killing it at step one or two but if an infection had no ability to ever get to its core malicious behavior then you were protected and this should on some level count .

Stage four . For all infections that were able to completely take a cleanup should also be attempted . Success at this step should count the least but still should in a small way count . Good examples would be malware that are of the annoying type (adverts/search redirectors/settings modifiers.....) .

An arbitrary scoring system could be 1 point for cleanup , 2 for secondary defense , 4 for primary defense and 8 for web defense for example . This would make each step twice as valuable as the step below it but the tester could come up with any fair system for step weights .

What I like about this test model is that no app that is awful against current malware can pass and no app that is great against current malware can fail (obviously given that care was taken to gather a reasonably diverse and large list of links/sites) .


Now compare the amount of work and skill required to do all of that to asking 50 security guys you know for samples , bunch them up in one folder and then doing a right click scan test against that folder , it is obvious why the latter is the default "test" .

 

Ouch!
Nooooo... you would need to have the set of (discrete/fixed) test samples in hand!

Many operators are clever enough to NOT serve the malware to your same IP more than once.

Many operators are clever enough to serve the malware intermittently, rather than every visit.
(foils blacklist maintainers followup checks to verify reported URLs)

Many malware campaigns target specific demographics (user-agent, IP netblock, etc)
(if you pass out a list of URLs & send various people out to test, and invite people to re-test... you should fully expect differing outcomes)

 

There are ways around this obviously . Even using test machine in different regions would work .

 

sigh
{speechless}
aghast