Interesting Antivirus Results..

I came across a thread on another message board where someone got some interesting results.... http://www.planetamd64.com/index.php?showtopic=10788

Check it out and I was really surprised to see that he reports that nod32 didn't detect any of them.

 

F-Prot, BitDefender, McAfee VirusScan Enterprise v8.0i, Trend Micro PC-Cillin 2005 detected all nine "trojans/viruses".

Very weird...Kaspersky missed 3 and NOD32 missed all...

Surprised that F-Prot would detect all of them.

 

LOL, yet another home test.. sigh

 

I dunno, but I think that poster states a valid concern. Granted, the "test bed" is comprised of "non-threats", but that's not the point. Shouldn't a highly touted AV such as Nod still be able to find anachronistic malware since its advanced heuristics is supposedly so stellar? Just wondering b/c I am trialing Nod right now and this is at least giving me food for thought.

 

By topic title its easy to understand that positive result could make him feel bad

 

Turbinehead posted this reply in that topic:

BitDefender's results and names of viruses/trojans:

 

Something I just noticed:

0 second scanning time? That's not right...

Notice that it started and completed at the same time.

 

Well actually it is. It's quite fast

Edit: Added a screenshot

 

Well, then I gotta get me one of those

I currently have F-Prot as my AV, thinking of switching to NOD32 or Kaspersky. But a bit hesitant now, seeing that F-Prot actually detected all 9 files in MFM's test...

 

IMO you should not be swayed by one test of older samples. In my experience with NOD32 it has always averaged above 93% with variable of 2%. I am not stating my simple tests are enough to justify buying it, but by doing more research you will see that NOD32 always seems to have detection rates in the mid to upper 90th percentile. HTH

 

Let's think about this for half a second folks. A reasonable testbed, not a complete one mind you, a reasonable one such as used at www.av-comparatives.org contains upwards of 350,000 - 400,000 samples and people are using electrons to discuss results reflective of a testbed of 9(!)

Do any of you see the statistical problem here? Anyone? To tell you the truth, watching this is absolutely depressing. This type of stuff is completely and utterly unsound, heck, I don't know if it even rises to the level of unsound, it's probably something less than unsound, whatever that is.

Cheers,

Blue

 

I agree, see my first post But currently this is the only active thread that I can join in on

 

"Ridiculous"? "Embarrassingly Nonsensical"? "Statistically Worthless"? "Not Representative Of Sanity"? "Pathetic Flummery"?

 

older samples or not, shouldn't heuristics pick up at least something?

 

If it was trained to do so then yes.
I think someone wrote a good article on what Heuristics were actually supposed to do, I will try to find it to help clear up this misconception.

 

OK, here we go with some math. The last www.av-comparatives.org on-demand test used a testbed of 386,104 samples. NOD32 detected 368,746. Let's subtract: 386,104-368,746 = 17,358 undetected. Let's make it harder. Let's take the top of the heap KAV, it detected 384,743, yielding an undetected population of 1,361. There's plenty of room for a handful of samples.

Given these numbers why would any result under the sun be unexpected? The answer is that it's not. You'll get results with any AV spanning none detected to flagging all and both extremes and the middle are all equally meaningless. Let me repeat - meaningless!

Blue

 

From what I understand heuristics brings some intelligence into detection rather than relying on signatures - perhaps recognizing behavior or signs of. BlueZannetti, how many of those detections can be detected purely by heuristics?

 

bbb,

I have no idea for the on-demand test. Look at the proactive test to get a sense of that. Same site.

My question to you - why would you maintain that programatic behavioral characteristics of the DOS world are necessarily relevent today?

Blue

 

This is not an unbiased test. He specifically targeted NOD.

He posted at the beginning "There seems to be a religious cult around NOD32, so I decided to do a little test with nine viruses and Trojans, of which NOD32 finds _NONE_".

If your purpose is to discredit a specific AV, out of all the samples out there, going back to the 90's, one could pick and choose 9 samples that any specific AV would miss all.

However, what I found interesting, in his endeavor to discredit an AV, is that he could only come up 9 old "non-threat" samples out of all the hundreds of thousands of samples available.

It is sad that a few folks will place some type of value on this type of test.

 

No one with a modicum of understanding of AVs and malware will give such a "test" the time of day, Stan.

 

- Not to mention the FREE updates you get. You don't have to buy a 2004,2005,2006 version - As long as your license is valid & running you get it all for free

This was to Kye-U

 

But he put the time of day on the tests







Hehe

Nick

 

Why should a heuristic be designed to target old viruses? Instead of wasting time with detection of old dos viruses via heuristics we concentrate at real worlds problems: Worms, Trojans and the like on Windows Platforms.

Besides, the first sample seems to be a dead sample - corrupted. Meaning some AV's might pick it up via Signature Match, others might not. The next sample is a Joke Virus - it does flip the screen for ONCE and never ever again - until you start it manual the next time. Great Virus Samples - really Crying and blaming AV products ( yes he blames also other av products for not detecting it ) and does not even understand WHAT he has tested.

 

PlanetAMD64 isn't exactly a "Bastion" of intelligence. One read of the admins posts and you'll figure this out.

Best avoid it, and any posts there. This test is useless garbage from someone with a vendetta against NOD32 because it slayed his dog.