AV Performance Statistics

Folks, I think that the tree is stopping you to view the forest. Discard engines known to generate many FP and known to flag any runtime packed executable (Fortinet, Panda, eSafe, Sophos, CAT-QuickHeal). Discard little known/new engines (Ikarus, Sunbelt, UNA). Now read the statistics again:
-Antivir.
-BitDefender.
-PrevX.
-Kaspersky, NOD 32.
-VBA32.
-Dr. Web.
-Norman.
-McAfee.
-AVG, Avast.
Doesn´t that have a certain sense?
Now, my doubts:
-why is NOD 32 behind BD and in the same level as Kaspersky? It is in the level of Antivir.
-why is Norman behind VBA32 and Dr. Web? I think that it is between both.
-why Kaspersky scores so high? They are quick at adding signatures but they heuristics are not so good.
-where is F-Prot?

 

that does have some sence.
but bitdefender does have hourly updates like kaspersky does and very good heristics so that could give it the edge in this random test.
lodore

 

I´m not questioning the score of BD, they have worked hard enough to occupy that position. But NOD 32 should be in the top-three (Antivir, BD, NOD 32) and not together with Kaspersky.
Also, the PrevX score is nice, the automated research in each customer and the central database seem to produce good results. I´m not sure how corrupted samples affects PrevX, though.

 

im quite surprised kaspersky and nod32 arent higher up in the chart.
lodore

 

I need more information from the maintainer of this page in order to better judge its validity.

Now "all detected malware" - does this mean even packed/crypted files that are erroneously flagged by say eScan are recorded as "malware"?

If so this would be a major flaw in the already inaccurate detection percentages.

 

With reguards packers and malware surely there is one or or 2 vendor employees here that receive detected files from the VirusTotal service and would be privvy to whether these packed files are infact dud or *live* malwares

This would cover some of the bases

 

im not surprised by nods place, but let me state i dont think this test is accurate in any way at all.......

if i were to be surprised, it would be mcafees.

 

I find it interesting that Fortinent is ranked #1 on two different "independent" reviews that obviously have suspicious testing methods w/ skewed results. I doubt AV-Comparitives will even consider it for their next review.

 

Fortinet will be included in the 2007 tests of AV-Comparatives.

 

Excellent. At least we'll finally have a current credible review of their product. Maybe someone can start a poll on which certification level it will reach.

 

No need for that, detection rates are good enough, the results will speak for themselves.

I do, however, suspect that Fortinet will produce lots of FPs.

 

Here is study of false positive detection of runtime packed files. eSafe, Fortinet, Ikarus, Quickhel and Sunbelt say notepad.exe is infected if it is packed with UPX, ASPack or Yoda:
http://blog.hispasec.com/laboratorio/189
(in spanish only)

 

Finally a break from bot collecting,sorry for the delay but here's my sincere reply to all that question the reliability of those results.

Ok we will have to differ on our views on the merits of reported data
This is my defence of how i interpret the results and the debunking in this thread that is evident.

I personally accounted for a pecentage of those files uploaded to VT service,i would estimate well over 4 figures in files since the birth of MIRT in November06.

Do i upload my specialized packed tools....do i upload legitimate packed files as with debunking in this thread ?

No on both accounts,i upload files imported by malware infections fo mal checking and if 10 or more of the DB's are'nt calling them out i'm uploading to MIRT(700+ todate).This results in the loss of about 80% of checked malwares but the upload's still happened@VT.

Sometimes the only detections are by these so called *packer* sensitive heuristic's at that point in time the suspect heur's have checked a malware file for their user's potentially.

Maybe this is not precision science but surely checkpoints for new malware such as this are not a bad thing ?

In finalising i dare suggest that the amount of actual malware files uploaded to VT against legitmate packed files is probaly a massive percentage not acknowledged by anyone here.

So for some carry on debunking like politicians,if the results don't go your way at the polls try spinning your way out of it

I'm back off to continue uploading suspect malware files

 

Again, Again, detection is one thing, detection and removal is another. Until someone can show this for the truth, everything else means crap.

 

3 months ago I sent HijackThis.zip to Fortinet because it was detected as malware. Also 4-5 months ago I did the same for McAfee,,

 

Quick everyone jump ship to PrevX!!

http://xs512.xs.to/xs512/07063/PrevX1.jpg.xs.jpg

Seems the graph is having some issues

 

AntiVirus Graph.

I was wondering about the results of this graph. It seems to show the likes of NOD32 and Kaspersky as being extremely average. Anyone want to comment? I was under the impression that NOD32 and Kaspersky were as good as it gets as far as anti virus is concerned.

 

Re: AntiVirus Graph.

in a word, 'bullshit'

thats my opinion, keep with nod and kaspersky or whatever, it is better than that graph says. w

 

Re: AntiVirus Graph.

...
truth

 

Re: AntiVirus Graph.

i agree. first of all fortinet detects anything packed with certain run time packers as suspicious, panda isn't that vicious but also has some detections like that.
a good look would be at the av-comparatives test.

 

Re: AntiVirus Graph.

sure av-comp is a good test and always has interesting results, but i like those av-test.org ones, they test a hell of a lot av's, when is there next test results published, anyone know?

 

Re: AntiVirus Graph.

The graph is interesting, but the test samples contain lots of false positives and garbage executables. Also, the test doesn't show you the high false positive rate some of the scanners have, they are running in "paranoid" mode on Virus Total. Also, the scan results are incorrectly rated. For example, information messages in the scan log are rated as detection.

 

Re: AntiVirus Graph.

Who said biased graph?

 

Re: AntiVirus Graph.

it is unprofessional and all bs do not take anything from it at face value.

 

Re: AntiVirus Graph.

It doesn't look like it's been updated since February 9 anyway.