Virus Weekly Stats....0Day

[apm]

http://www.shadowserver.org/wiki/pmwiki.php?n=Stats.VirusWeeklyStats

vendor detected total percent

AntiVir 329415 332300 99.13%
Vexira 326631 332300 98.29%
DrWeb 326452 332300 98.24%
VirusBuster 326468 332300 98.24%
AVG7 326360 332300 98.21%
Clam 326195 332300 98.16%
Norman 325852 332300 98.06%
F-Secure 325728 332300 98.02%
Avast 325388 332300 97.92%
F-Prot6 325283 332300 97.89%
McAfee 323491 332300 97.35%
VBA32 322576 332300 97.07%
F-Prot 316545 332300 95.26%
Panda 248000 332300 74.63%
BitDefender 159137 332300 47.89%
Kaspersky 159126 332300 47.89%
NOD32 99250 332300 29.87%

::)

[JasSolo]

How about FP's in this test?

Cheers

[plantextract]

something doesn't compute:
F-Secure 325728 332300 98.02%
Kaspersky 159126 332300 47.89%

i can't belive f-secure's heuristics engines are that performant, no other tests show such a huge gap

BitDefender 159137 332300 47.89%
bit defender has a great heuristic engine and a very good detection rate in general (Av-test, av-comparatives)

[trjam]

looks accurate to me.;)

[trjam]

Monthly are even better
http://www.shadowserver.org/wiki/pmwiki.php?n=Stats.VirusMonthlyStats

[Edwin024]

I'm curious about how they test, these results are so different from other tests... If this is true especially Eset really has a problem. With Bitdefender and Kaspersky also in the relegation zone. Weird.

EDIT: Another strange thing - last edited files are from November 12 2007. And that is for the weekly list....

[Bunkhouse Buck]

As usual, Avira is a wise choice.

[C.S.J]

I too would like to know how they test

[Bunkhouse Buck]

{QUOTE-> I too would like to know how they test <-QUOTE}

However they test, it makes Dr. Web look a whole lot more effective than the AV-Comparatives routine. I might even give the good Dr. another try on my boxes myself! :D

[C.S.J]

That doesn't matter, what does if its credible or not.

I leave it up to anti-malware to give drweb some light ;)

[Bunkhouse Buck]

{QUOTE-> That doesn't matter, what does if its credible or not.

I leave it up to anti-malware to give drweb some light ;) <-QUOTE}

How do we know if any of them are credible? The empirical tests and resultant data also presuppose that the rigors of the test were/are as stated by those performing the test. The reality is, the trust in the objective validity of various AV tests might beg the question. Not to argue from skepticism (which is a philosophic contradiction), but there must be necessarily a lot of "faith" in the integrity of the testers, and from my long years in the computer industry-I would advise caution.

[MalwareDie]

absolute bs just by looking at clamav's score.

[rogervernon]

Who was it who said "Never trust ANYTHING posted on Wiki" ?

[dNor]

Those are certainly some...unique...results. :wacko:

[larryb52]

they look a bit off espeically Kaspersky...

[Blackcat]

Some discussion on previous results from this site here (http://www.wilderssecurity.com/showthread.php?t=179475&page=2&highlight=ShadowServer).

[C.S.J]

results on zero-day threats are intersting, but we need more information about how they test them and where the samples are from.

results over the whole year, 2007.

it puts clamav rightly so in its place for 2007.


F-Secure 98.89%
F-Prot6 97.58%
DrWeb 96.31%
AntiVir 94.11%
Norman 90.12%
Kaspersky 83.19%
McAfee 81.80%
NOD32 81.54%
Vexira 81.19%
AVG7 78.94%
F-Prot 77.03%
BitDefender 74.16%
VirusBuster 71.91%
VBA32 67.62%
Avast 63.59%
Panda 44.75%
Clam 11.18%

good results for drweb over the last year for zero-day, but like i said... we need to dig deeper to see how the tests are done :)


-------------------------------------------------

if you look at the daily results, for today.

antivir scores well as always, but drweb is fantastic too.

we know antivir has the detection-overall, so how can today.. on the daily tests, antivir detect 99%

surely, these are not all FP's.

also avg and avast scored 99% for today, and they are not known for fps.

nod32 is at the bottom of the weekly test, but today they got 98% while kaspersky remains at 0.07%

so, are these tests legitmate for zero-day testing

but...

maybe someone can tell me why the results and the charts do not match?

[patrikr]

{QUOTE->
results over the whole year, 2007.

F-Secure 98.89%
<-QUOTE}

Looks good to me ;)

But yeah, it's strange that our products detection of Worm.Win32.VB.es (311390 last week) would be better than Kaspersky's detection of the same malware (153978 in the same period)

I don't have insight how these numbers are calculated but I know who to ask, will check it out.

--
Patrik
F-Secure Security Labs

[trjam]

{QUOTE-> Looks good to me ;)

But yeah, it's strange that our products detection of Worm.Win32.VB.es (311390 last week) would be better than Kaspersky's detection of the same malware (153978 in the same period)

I don't have insight how these numbers are calculated but I know who to ask, will check it out.

--
Patrik
F-Secure Security Labs <-QUOTE}
with all due respect, you dont know your software to well. This is the 2nd test in 2 weeks that talks about F-Secures ability to handle zero day threats better then all the rest. It is because of Deep Guard, which Kaspersky does not have. The results are accurate based on a AV really being good at detecting zero day threats, which F-Secure is because of its internal HIPS, and others because they are designed to detect anything, FPs and all.

[IBK]

doesn't kav have the pdm?
anyway the samples were not executed in this test, so Deepguard and PDM were not in action.

[Edwin024]

Guys...what 0days? It says that all tests are from November 12 2007...

[C.S.J]

let us know what you find out about it patrik ;)

[C.S.J]

kaspersky is fighting back with #1 for today, so it can do it..

but over the past year, drweb has been flying high, so ... So much for the really low detection rate, apparently ;)

[Bunkhouse Buck]

{QUOTE-> kaspersky is fighting back with #1 for today, so it can do it..

but over the past year, drweb has been flying high, so ... So much for the really low detection rate, apparently ;) <-QUOTE}

And you relied to me from above:

"That doesn't matter, what does if its credible or not."

So now you think the test is credible?

[patrikr]

{QUOTE-> with all due respect, you dont know your software to well. This is the 2nd test in 2 weeks that talks about F-Secures ability to handle zero day threats better then all the rest. It is because of Deep Guard, which Kaspersky does not have. The results are accurate based on a AV really being good at detecting zero day threats, which F-Secure is because of its internal HIPS, and others because they are designed to detect anything, FPs and all. <-QUOTE}

Of course I know our software and I also know Shadowserver and which version of our scanner they're using. The one they're using doesn't have DeepGuard, simple as that.

The difference is this case seem to be for one single detection (Worm.Win32.VB.es) which is a detection by the Kaspersky engine in our product. It should therefore be detected by both products but it isn't for some reason.

The c't test that showed that DeepGuard performed really well was conducted in a totally different manner than just simply scanning a file (in which case DeepGuard doesn't do anything). DeepGuard is behaviour based and only triggers when a process is started as IBK correctly pointed out.

--
Patrik

[C.S.J]

bunk, I still ain't sure

Waiting for someone to find out if it is ;)

By either way, alot of malware is tested and drweb detects alot of it, so, so far I'm happy

I never rely on tests but some ppl do, to me ... I know from personal experience that drweb is better than most ppl think here on wilders.

[Blackcat]

{QUOTE-> bunk, I still ain't sure

I know from personal experience that drweb is better than most ppl think here on wilders. <-QUOTE}
Most of these critics have never even used Dr Web!

[Bubba]

numerous posts totally un-related to the thread topic at hand moved to a thread of their own for further discussion.

Moved here---> http://www.wilderssecurity.com/showthread.php?t=197113

[patrikr]

{QUOTE-> let us know what you find out about it patrik ;) <-QUOTE}

Got a reply and they're using the publicly available signatures for all of the scanners and are testing all products against the same set of binaries. So the numbers are true.

Patrik

[freed0]

Evening all,

Richard here from Shadowserver. You can blame me for any mistakes on our web pages. In the future if anyone has a question about any of the statistics, our methods on generating them, or anything, please drop me an email.

Now on to the more specific questions.

All the results that you see on our pages are really controlled by several factors. The primary being what types of binaries we actually gathered in the last day, and which ones were queued up for the testing. Some of the AV vendors do very well against certain types of malware and not as well against others.

We get in between 50k to 200k new unique binaries each day. We are only able to test a portion of those because of resource constraints. We generate the test list and each vendor is tested against the exact same list. Not all vendors were created equal, and some can take a large amount of time to test the same set of binaries. So, this limits us on how many we can test. Unless of course someone wants to donate more dual quad-core machines for us to use... :)

The specific results are those that each vendor spits out. We did our best to write a generic parser to catch each of the 'real' names that the vendors uses, but even that can never be perfect.

We treat each of the vendors equally and are not sponsored by any of them. Several have donated licenses for us to us, but we have paid for the majority of them ourselves. Any vendor that donates a license will be sent up to 10k binaries each day that they do not detect. Assuming that do not detect that many.

Each vendor is updated once an hour from the normal public repositories that any normal user would be using. So, depending on your version and options, YMWV. You can see the specifics on our command usage here:

http://www.shadowserver.org/wiki/pmwiki.php?n=Stats.Viruses

We have tried to show the data as impartially as possible, so, if anyone has any suggestions on how to improve upon what we are showing, I am happy to take in any suggestions that you may have. I just cannot promise to try to reply to all the posts, unless someone warns me that I should.. :)

Hopefully this helped clarify some of how we are generating the statistics that seemed to have created some confusion.


Richard

[FRug]

The problem is, people have a tendency to see numbers as an absolute, while often (as in this case) they do not mean exactly what they think they do.
Admittedly that's not really the fault of Shadowserver, but you may want to add something about the numbers explaining there is a heavy bias towards samples of a malware variant that you receive very often.

Taking todays stats as an example:
Total number of samples: 67400
Brontok.Z.1: 63334

So, ~93% of the sample set are actually one and the same malware strain. Binary-different maybe, but that's not really relevant here now is it?

Dear Forum Folks, you should know by now that you can't take such values as absolute indicators of detection rate.

@freed0: You really should add something mentioning that bias to your stats pages, it might even prevent some misunderstandings of what you guys do and keep random accusations off your backs. In very large red letters ;)

[C.S.J]

{QUOTE-> Got a reply and they're using the publicly available signatures for all of the scanners and are testing all products against the same set of binaries. So the numbers are true.

Patrik <-QUOTE}
well done for drweb then ;)

[Mele20]

Does that web site not like Firefox? The 0Day Summary and ReTry Summary (whatever that is) is not there. Two large blank spots.

[Gizzy]

{QUOTE-> Does that web site not like Firefox? The 0Day Summary and ReTry Summary (whatever that is) is not there. Two large blank spots. <-QUOTE}

works fine with Firefox for me,
perhaps an add-on is causing the problem?

[huangker]

Yes it works fine for me too. Maybe just refer to the screenies posted before.

[Mele20]

I don't know what it could be. I only have 7 extensions and none of them should affect the display of the page like that. I tried on both Fx 1.5.12 and Fx 2.0.

I then tried on Opera 9.24 and I can see it there. Fx displays EVERYTHING on that website in a weird manner. I didn't realize how strange the display is of the part of the page I can see on Fx until I tried Opera there.

[Bunkhouse Buck]

{QUOTE-> I don't know what it could be. I only have 7 extensions and none of them should affect the display of the page like that. I tried on both Fx 1.5.12 and Fx 2.0.

I then tried on Opera 9.24 and I can see it there. Fx displays EVERYTHING on that website in a weird manner. I didn't realize how strange the display is of the part of the page I can see on Fx until I tried Opera there. <-QUOTE}

I have Fx and no problems at all.

[flyrfan111]

Same here, Firefox and Opera work perfectly. It must a plug in or something Mele:o

[BlueZannetti]

{QUOTE-> It must a plug in or something Mele:o <-QUOTE}or something with how Proxomitron is handling things....

Blue

[Sjoeii]

Norman is doing great here
http://www.shadowserver.org/wiki/pmwiki.php?n=Stats.VirusWeeklyStats

[C.S.J]

there are many 99% avs this week for zero day

[Sjoeii]

{QUOTE-> there are many 99% avs this week for zero day <-QUOTE}
Indeed
They did well