malware scanner test

Nick as much as do respect your good work your starting to sounds like a broken record. Everytime SAS fails something you start about those hidden rootkits everyones having. You really should lay of the scare and buy aproach.

 

Folks,

There really is no reason that the discussion cannot remain civil, even when you're expressing wildly divergent opinions. Thanks in advance.

Blue

 

Yes, FPs are really annoying and, like I already said, sometimes can be potentially as devastating as missing a threat, although that's on rare occasions.
As far as Ad-Aware is concerned, it is also a very poorly performing piece of software. In all my years of experience Ad-Aware has never found anything but tracking cookies on my PC or other PCs that I'd tried cleaning with it, whereas programs like Spybot and SAS had found other threats. What's worse is that although Ad-Aware seems to specialize in detecting tracking cookies, it had still failed to remove them all and had left some for the second scanners. At least the 1.06 Personal version was light and swift. The 2007 eliminated that small sort of redeeming feature.
BTW maybe Ad-Aware should be renamed to TrackingCookie-Aware.

 

In my scanner-period, more than 2 years ago. I ran Spybot always first, then Ad-Aware and it always detected 2 things : tracking cookies + MRU's.
I could solve the tracking cookies by running a history/cookie cleaner first, but never got rid of the MRU's and MRU Blaster didn't solve the problem either.
Nowadays and after running Ad-Aware again as a test, I finally got rid of MRU's as well.
Spybot and Ad-Aware are not that good anymore as in the past, both find nothing but easy-to-remove malware, not the real nasty ones, like SAS and MBAM do. The days of simple malware are over.

 

There is no "scare and buy approach" - our product is 100% free and will remove the rootkits and other items I mention - there is nothing to buy.

The reason I mention these rootkits and other items is because we see over 30,000 computer diagnostics a DAY through our research center and we see EXACTLY what is running on the infected systems - and they are certainly not the rogue products - the rogue products represent a small percent of the infections. I would rather remove less old rogue products then miss critical rootkits that are actually causing harm and stealing data.

I am a believer in education - when I see these types of tests - it's important to let users know there is more out there than "rogues" and some of the basic threats.

 

Which such a huge database, you may have interest in doing some statistics to share with the community. I mean: OS version, language, patching level, prevalent infections (adware, trojan, rootkit, etc), top 20 infections (Zlob, Vundo, WinFixer, etc), security settings, browser and such. What do you think?

 

We are actually looking into ways to provide that information as an education resource to the community. We currently use the data to locate zero-day threats as they appear.

 

I think to some extent he is right. Rouges are installed intentionally by the user and are far less important than rootkits, drive by downloads, trojans etc.

 

Great. I can't wait to see some numbers

 

Brings up an interesting point. What exactly is rogue software and how is it installed? You say it is installed by the user, but others disagree. I posted here a list of criteria that Eric Howes uses for rogue programs. The list includes, "3. Being installed by adware, spyware, or malware" and, "5. Being aggressively or stealthily installed without users' full, knowing consent".

 

I will clear that up a little .

Current rogues are installed by trojans , trojans that SAS and MBAM got to great lengths to stay on top of .

Old and non critical rogues can only be installed by intentional user action .

 

Yes, that clears that up. Thank you.

If I may put you on the spot, Bruce... the tests referenced in this thread produced a pass/fail ratio of 46/24 for MBAM. Regarding the 24 fails, did you subsequently add detections for those rogues?

I'd like to hear the same from SAS. The tests produced a pass/fail ratio for SUPERAntiSpware of 38/32. Regarding the 32 fails, did Nick subsequently add detections for those rogues?

 

We added definitions for all rogues on the list that we were able to track down the installers for - some of them we already removed where the test showed we didn't, so I am not confident that they were using the latest definition set when scanning.

 

This is how it works . There are several trojan families and for the most part they advertise/install rogues unique to their family but there is some crossover . As they add new rogues the advertisment/installation of the old ones is dropped . The only time a user has any chance of comming into contact with them is when they are current .

We did not add detection for any dropped rogues because their time has passed . I do not have the time to make MBAM pass pointless tests . If I had I would have been being dishonest to our users making them think I "added protection" when all I had done was pad definitions .

To be honest I did not even look at any of the results other than the current ones , the old ones are that irrelevent .

I went back and checked the current ones , we were 100% on all but 1 , that 1 is now covered .

The only current rogue they show us missing is antispyspider and this is not correct , it is in defs .

 

Many of the "older" rogue sites are still active (user's can still download) so in reality, they can still end up on user's systems - although they are not "rampantly" installed, they are still there - so it's not being dishonest to add those to the definitions.

 

We actually cross posted , that was not at you although the timing sure made it look that way , sorry .

Rogue Remove will likely pick up the other rogues .

 

I didn't think it was targeted at me at all - I was just making the point that user's still can install those, so you would not be "dishonest" to add those definitions.

 

I understand that the tester kept logs in case vendors wanted a copy for review. The logs should help with the installers, yes? And they should show which definition set was used.

 

I am thinking that I may add a few of them to test something out . We have the option for our users to log what was detected .

It will be interesting to see if any of these actually show up as detected .

I agree with Nic on the defs thing , there were a few odd results that could only be explained by old def versions .

 

Excellent point.

The tester said it this way:

http://www.dozleng.com/updates/index.php?showtopic=18279

 

One has to agree, when putting together a test of any sort, one of the most basic fundamentals would be the use of the most current definitions set. To do otherwise would be quite deficient on the tester's part. I am thinking that her logs would include this information.

 

Yes, testing is hard, but the methodology has to be solid - I had written a blog on the subject a while ago, but it may provide some good information for the readers here:
http://www.superantispyware.com/blog/2006/10/12/the-importance-of-testing-methodology/

 

My belief in the detection abilities of SAS and MBAM are quite strong. However, my belief in a-squared AM has been proven. Hopefully this test will be a barometer for SAS and MBAM to improve even more.








ThePheonix Always Rises

 

Yes, I guess SAS and MBAM can increase our abilities to detect harmless/annoying rogues that are easy to create definitions for I would be curious to see A-Squared's abilities against Rustock and other rootkits

 

It never hurts to pay attention to the so-called little things. Users want you to knock out the big stuff and the small stuff. A tradesman who takes the time and prides himself in doing the little things gets commended for his attention to detail. He doesn't get referrals and kudos for doing only part of the job. I would like to see MBAM, SAS or any of the others take the same approach.