Interesting Test -- Norton Impresses Me

I always take the reviews on malware products with a grain of salt most times. It seems like ever review comes out with different results. I thought it would be interesting to conduct my own tests. While i admit my tests weren't scientific in any stretch of the imagination they did enough they did enough to convince me what i will be using on a ongoing basis. I've been hearing some of the free alternatives getting good praises (Avira, Avast, MSE,ect....) In the past i always considered Norton & Kaspersky to be the cream of the crop. I wanted to see how each one stacked up against each other.

How i tested:
First i chose to use windows xp w\ SP2. Pretty sure some products won't even install without SP2 so thats where i started. Some of you might not think this is a fair test since a lot of the new technologies in newer SP or newer OS adds to the picture but i wanted a pure test against the AV product itself. I didn't want to test the security features of new Operating Systems. I used IE6 for everything. So with a fresh 0 day list of malicious URLs everyday i went to conduct my tests. After running the AV through a slew of test then i would run malwarebytes (the quitisential standard to pick up things that most AV software misses).

Threatfire:
I've tested this before and it did pretty poorly. I just wanted to give it another go. I know its not AV per se but i wanted to see if it would block *most* of what i would be throwing at it. Again it failed miserably. I think within 10 links into the test and i couldn't use the internet and the machine had been infect by multiple things. Like with a lot of the other products i couldn't go anywhere in ie. I couldn't open IE. It would close immediately. Trying to open malwarebytes would immediately close the program

MSE
I really had high hopes for MSE. I've heard such good things about it. Before you ask, no i wasn't using the beta. When they beta is released maybe i will try the test again. Anyways, MSE did pretty good to about halfway through the test. Then i got pretty badly infected with a trojan. MSE was still running but when you tried to update the virus definitions you got the following error:
"Virus & spyware definitions update fialed

Microsoft Security Essentials wasn't able to check for virus & spyware definition updates
Make sure your computer is connected to the internet

Error Code: 0x80070422
Error description: Microsoft Security Essentials can't start the update service because it's been disabled by the local administrator or as a result of a problem in the registry data."

Trying to open malwarebytes would immediately close the program.


Panda Cloud 1.1
Panda also got owned within 20 links. Again malwarbytes immediately would close and i was getting all kinds of popups.

Avira: Again i had high hopes for Avira. I have heard it had one of the better detection rates but was kind of bad on false negatives. Well halfway into the 1st page i got infected pretty badly with something called "Antivir Solutions Pro" believe it or not. I thought it was kind of ironic. Anways this nasty wouldn't allow me to navigate away from the page where they are asking you to buy it and it would close anything i tried to open (task manager, malwarebytes, ect). Go here to find out what antivir solution pro infection looks like:
http://www.2-spyware.com/remove-antivir-solution-pro.html


Avast: It got infected with the same nasty as above but it was able to close Avast so about half way into the first page Avast also got owned.

NIS 2010: I didn't have actually have that high of hopes for Norton. Why? Well because i would assume that is what a lot of these malware authors test against because of the number game. There are a lot of computers out there protected with Norton. I started throwing the slew of 0 day links against it. Norton was just catching EVERYTHING. The browser toolbar itself was catch a lot of attacks. I could tell norton didn't have definitions for some of the links i had but it did a tremendous job of blocking them. It would throw up a big block strongly recommending that you block whatever it couldn't recognize. I went through 4 pages of 0 day links and after everything the machine looked clean. I did a scan with malwarebytes and there was nothing to be found. WOW!!!!


KIS 2010: Kaspersky did quite well but it wasn't perfect. It blocked most things unfortunately it got infected about 3/4 of the way down the first page even though the nasty it got infected with (antimalware doctor) wasn't as malicious as some of the other things i got infected with above that made me stop my best because i couldn't get any further. I could get to any page i tried to go to. Here is the final scan from malwarebytes after the finalization of my tests against kaspersky:

Malwarebytes' Anti-Malware 1.46
www.malwarebytes.org

Database version: 4364

Windows 5.1.2600 Service Pack 2
Internet Explorer 6.0.2900.2180

7/28/2010 8:46:09 PM
mbam-log-2010-07-28 (20-46-09).txt

Scan type: Full scan (C:\|)
Objects scanned: 147146
Time elapsed: 16 minute(s), 41 second(s)

Memory Processes Infected: 1
Memory Modules Infected: 0
Registry Keys Infected: 2
Registry Values Infected: 2
Registry Data Items Infected: 0
Folders Infected: 0
Files Infected: 4

Memory Processes Infected:
C:\Documents and Settings\Administrator\Application Data\7AC33FB59C8C0E33900A9F03A345A378\KB1098894.exe (Trojan.Agent.Gen) -> No action taken.

Memory Modules Infected:
(No malicious items detected)

Registry Keys Infected:
HKEY_CURRENT_USER\Software\Antimalware Doctor Inc (Rogue.AntimalwareDoctor) -> No action taken.
HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Uninstall\Antimalware Doctor (Rogue.AntimalwareDoctor) -> No action taken.

Registry Values Infected:
HKEY_CURRENT_USER\SOFTWARE\Microsoft\Windows\CurrentVersion\Run\kb1098894.exe (Trojan.Agent.Gen) -> No action taken.
HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Run\registrymonitor1 (Trojan.Agent) -> No action taken.

Registry Data Items Infected:
(No malicious items detected)

Folders Infected:
(No malicious items detected)

Files Infected:
C:\Documents and Settings\Administrator\Application Data\7AC33FB59C8C0E33900A9F03A345A378\KB1098894.exe (Trojan.Agent.Gen) -> No action taken.
C:\Documents and Settings\Administrator\Local Settings\Temporary Internet Files\Content.IE5\2T2UM00Z\fix714upload[1].exe (Malware.Packer.Gen) -> No action taken.
C:\Documents and Settings\Administrator\Local Settings\Temporary Internet Files\Content.IE5\PEN6HE6R\ff-update[1].exe (Rogue.Installer) -> No action taken.
C:\Documents and Settings\Administrator\file.exe (Trojan.Dropper) -> No action taken.

I know my test wasn't perfect and i didn't test everything. The only product i really wanted to test but couldn't was PrevX. Unfortuantely we know how Prevx is with its licences. They won't even give you a trial. Kind of hard to test something when it will telll you something is infected but not do anything to stop / prevent the infection.

 

I have tested Norton AV 2011 a number of times, and have never gotten anything past it. I think it is one of the, if not the best.

 

Ya i have a new found respect for them. I really didn't expect the results i got. I currently run NIS on my machines and my liscence will run out soon. Fortunately being a comcast customer i can download norton 360 4.0 for free which is almost the same thing as NIS (use same engine) with a couple extra features added on. I will definitely stick with norton after what i found. I'm kind of curious though how their enterprise solution would stack up (Symantec Endpoint Protection) because it has seemed to lag behind their consumer version for some time. Maybe i'll have to test that at some point.

 

One more thought. The people over at malwarebytes really should make their own AV product. Its really what i compare to everything else.

 

Can u pm me the zero day links u used

 

Within the last 4-6 months I have had two friends who used Norton that got infected with Antivir rogue. Norton did not detect and prevent or find it on a scan. Malwarebytes on a quick scan detected the rogue and removed it.
I would expect Norton at this time to have put it in their data base.

My own conclusion is that no AV will detect and remove or prevent some malware. I notice those that clean computers as a regular thing always use MBAM.
Accordingly, I run MBAM real time alongside my AV or suite.

There is little doubt in my mind that Norton is one of the very best, but in the cases I mentioned even it did not prevent or remove infection of that particular rogue.
I think a person leaves himself without at least one important layer of protection if he does not use something like MBAM, which in my mind is the single all around best.

Regards,
Jerry

 

I'm sure every piece of security software is vulnerable to some piece of malware in the wild but i don't think that discredits my findings. There were some cases with norton couldn't recognize the threat from definitions but it thought there was a threat. Sure you could click through the warning and get infected but i'm not sure because i didn't try. I thought the AV product at least giving you a warning "....don't do it!!!!!" was enough. Every time i got infected...i wasn't warned. For example most of the AV products didn't even give me any warning and going to a web page got me infected without clicking anything at all. At least with kaspersky i had to open an executable which it didn't detect as malicous. I think the big thing that norton had going for it was "norton insight". It has a big pool of stuff it knows is installed on peoples computers and is safe. If it doesn't recongnize something it definitely lets you know.

 

Dont Forget SONAR and Insight

 

Hey Champ I guess its against the rules to ask for links that possibly direct to malware and I guess ncage1974 had definetly taken links from MDL

 

Got It

 

try testing KIS 2010 with a slight tweak in its settings .......

 

MalwareBytes does have a paid version.

Norton does a good job at catching, but has a very hefty footprint and a great penchant for FPs. Heck, even malware removal tools (GMER, MBR.exe, OSAM, etc) get caught as malware by Norton. So when somebody does get infected while running Norton (Which I see all the time), I end up needing to neuter or remove Norton in order to successfully remove the malware. Also quite interesting to see that, for example, startup time of the system more than doubles with Norton installed in many cases and on a test machine, wiped clean and re-imaged between tests, the same MBAM scan without N360 installed took 13 minutes, and with N360 installed took 47 minutes... *twitch* *Sigh*

 

That is not true. Have no idea what OSAM is but for Gmer , MBR and others , Norton doesn't detect the files:

Right clicking the icon , then choose Disable Auto-Protect and you are ready.

Symantec has made Norton Power Eraser and Norton Bootable recovery tools (both free) to use in cases the other product can't beat/find some malware

 

What's so special about NIS2010? It blocks everything that's unknown, off course it will have a higher success rate that way. You can have the same effect by turning on SRP Nonetheless, it does a better job than the rest because it will prevent users getting infected.

 

You are right and it's funny because in all the latest test Norton is scoring badly, but in dynamic test got good results because Norton alert you that the app is not very common bla bla bla... and this seems to be enought to accept the alert as a real detection. The problem is that I have seen many times the same alert with safe and not very common apps.

 

Or another simple tweak is to turn on KIS sandbox

 

you test Threatfire, then why not Prevx. Seems only logical and fair.

 

And turn on Interactive Mode

Nice test though, Norton seems to be very good at default settings, especially for normal home users.

I think the AVs (Avira, Avast) could have done better if you would've used their paid versions.

Also, Malwarebytes isn't the best way of determining how infected a machine is (although you said the test isn't perfect so it's ok).

 

Can't see the point in marking all the new and unrecognized files as dangerous.

You can't recommend the product to unexperienced users...they won't be able to point which alert is false-positive.

 

It seems eminently sensible to me to classify all unknown files as potentially malicious/risky.

 

I would have thought it would classifying as "unknown" or "suspicious". Cant say it is dangerous without knowing... that's more of a false-positive.

Inexperienced users would seldom go anywhere near unrecognised files anyway though, they stick to the popular main-stream software. Chances are, if they are near a new, unrecognised file, it should alert the user, as it's got a good chance of being malware.

 

Especially since Norton's "known good files" now number... what is it up to? Many tens of million, anyway. The average user might never come into contact with any other, at least not before they are added to this number.

 

Not all the time coz what can you say about BETA applications that are just temporary they are in most cases unknown

 

However, I could say it could be an indication.

Thanks.

 

That's why I said they should be classified as potentially unsafe. What's the alternative,just allow all unknown stuff free reign and hope for the best?