http://virusscan.jotti.org/

How come if you watch the statistics on http://virusscan.jotti.org/ NOD32 only finds like 1 of 10. If you follow Kaspersky it finds 9 of 10. I dont understand..

[RejZoR]

You better not to. Forget about statistics,because you don't see everything. Besides,everything is running on Linux machine so results will be different as on Windows machines (not much,but there are differences).

[izi]

KAV is the best!!!

[richrf]

Hi,

It also appears to me that KAV is catching much more than NOD32 in these online samples. Of course, eyes can be deceiving. But, assuming that KAV is doing much better than expected, it may be that KAV's on-demand scanner is better than NOD32's (especially with packed files), while their real-time packers have greater parity.

I would like to note that recently, while cleaning a machine, KAV's on-demand scanner missed malware that was hidden in ADS files, but was picked up by their real-time scanner. So there are differences in scan detection, even within Kaspersky's own products.

Rich

[pykko]

Look at this guys! KAV or NOD ??

[StU]

Let 10 people take a look at different times and you may get 10 different results.

[Honyak]

I could have posted 10 or more screenshots of NOD missing in the last day or so. You can't base a decision on a single scan from Jotti's.

[jlo]

I Agree,

I just clicked on Jotti scanner and saw

AntiVir X
Avast X
AVG Antivirus Win32/Small.A
BitDefender X
ClamAV X
Dr.Web X
F-Prot Antivirus X
Fortinet X
Kaspersky Anti-Virus X
mks_vir X
NOD32 probably unknown WIN32
Norman Virus Control X
VBA32 X


May be AVG is the best LOL! What KAV did not detect this

The lesson to be learn't is that no AV is 100% perfect.

What I like about KAV is they activivlty get all failed detections sent to them and you can bet a couple of updates later they will be detecting it.

Again Nod32 excellent AH caught this one with out defs!

I have licences for Nod and KAV and like both very much for different reasons.

Cheers

Jlo

[RejZoR]

I hope you don't expect 100% detection from KAV...
And this detection is by normal heuristics,not AH...

[Hyperion]

Hi.Since from 21 May,i do exactly that,i mean keeping random statistics of some avs that interest me from the results of Jotti,here's what i have till now (although the last few days i ve been busy).I don't keep the absolute number of scans,so i don't know how many samples i ve kept.So these are relative differences.Nothing sicentic,just for my curiocity.NOD 32 is impressive for the fact that uses very often the heuristics (i could almost say that from the times i remember,half of them were caught by heuristics).

AntiVir 23
Avast 18
AVG 11
BitDefender 29
ClamAV 15
Kaspersky 45
NOD32 25

Last piece of malware found was Trojan-Clicker.Win32.Agent.db in qwinnta.exe, detected by:

Scanner Malware name
AntiVir TR/Click.Agent.DB
Avast Win32:Adan-032
AVG Antivirus Clicker.9.V
BitDefender Trojan.Clicker.Agent.DB
ClamAV Trojan.Clicker.Agent-31
Dr.Web Trojan.Click.357
F-Prot Antivirus W32/Agent.NL
Fortinet W32/Agent.DB-tr
Kaspersky Anti-Virus Trojan-Clicker.Win32.Agent.db
mks_vir Trojan.Clicker.Agent.Db
NOD32 X
Norman Virus Control W32/Agent.CTA
VBA32 Trojan-Clicker.Win32.Agent.db



Hmm even Fortinet finds this one and NOD32 doesnt.....how come the heuretics doesnt find this one?

[Honyak]

Originally Posted by Hyperion

Hi.Since from 21 May,i do exactly that,i mean keeping random statistics of some avs that interest me from the results of Jotti,here's what i have till now (although the last few days i ve been busy).I don't keep the absolute number of scans,so i don't know how many samples i ve kept.So these are relative differences.Nothing sicentic,just for my curiocity.NOD 32 is impressive for the fact that uses very often the heuristics (i could almost say that from the times i remember,half of them were caught by heuristics).

AntiVir 23
Avast 18
AVG 11
BitDefender 29
ClamAV 15
Kaspersky 45
NOD32 25

Interesting to not see DrWeb and MKS stats, from my observations both find more trojans than NOD32.

[richrf]

Hi Hyperion,

Do you know how many actual observations you made? In other words, how many KAV missed? Thanks.

Rich

[RejZoR]

I don't get it why people expect that NOD32 heuristics should pick everything!? C'mon,they are best on market,but you cannot expect them to be almighty
Also there is no point of calculating anything based only on random visits of page and reading results. You can view page at wrong times and you'll miss potential detections of specific AV,thus resulting in lower "score".

[richrf]

Hi RejZor,

I agree. Heuristics are an "extra" level of protection. A good implementation will not give too many FPs while still getting those nasties that the signatures are picking up.

The results that Hyperion reveals is essentially my own non-scientific experience. I visit Jotti several times in the week just to check on what is going on, and I would rank the top AVs very similarly in terms of "Jotti detection rate". However, it is tough to say what this means, especially since this is on-demand scanning as opposed to real-time scanning, and for me real-time scanning is by far an away more important. My guess is, based upon what I have seen on Wilders over the past two years, that KAV (with its very frequent and comprehensive signature updates) and NOD32 (with its heuristics) are in rough parity nowadays.

Certainly I would have no problems recommending either of them, but I am more likely to recommend an AT (like Ewido or BOClean), if someone is running NOD32. For some reason, I feel that KAV probably is more of a stand-alone product. Maybe I am misreading the situation a bit.

Rich

[Hyperion]

Interesting to not see DrWeb and MKS stats, from my observations both find more trojans than NOD32.

I don't keep stats for them too,because they don't interest me,since i will definitely not be running them on my PC anytime soon (never saw them on sale in Italy and i avoid internet sales when i can.).That's why i said that i keep for those that "interest me".I had all of the above except NOD32,which i might try though if it is as light as they say.Right now i have AVG resident.

Hi Hyperion,

Do you know how many actual observations you made? In other words, how many KAV missed? Thanks.

No,as i said,i don't keep the absolute number of scans.I ve simply started an xls and each time i add a point to every AV that has catched the malware.Since it's not scientific observation,i m more interested on relative performance.Actually i included KAV as a point of reference ,for obvious reasons.I d say,that more or less i ve logged about 55 scans.
You know what?Even if i m late i ll add yet another line in the Excel file and start noting the bumber of samples too.I ll be off by about 55-60 that are the ones i havent logged till now,but more or less,when the number grows,it ll become negligible.I ll note 55 (say KAV lost 10) and continue counting from there.

[Hyperion]

Originally Posted by Rejzor

Also there is no point of calculating anything based only on random visits of page and reading results. You can view page at wrong times and you'll miss potential detections of specific AV,thus resulting in lower "score".

Of course you can miss potential detections.I don't pass all my day at Jotti's.When i remember it ,i go and pick the malware i find and as i said,i wanted to have relative results,not absolute.I started this,because i have my doubts of what is "ITW" for the pro testers and what is "ITW" for the simple PC user and wanted so see in real-life (as that can be) conditions,what the tendencies would be.
I consider this as a poll.Just like when you go at the street and ask people randonly.Of course you ll miss many,you might meet more of the same opinion in a quarter than another because of different social-economical level,but at the end,as the sample becomes bigger,the error should decrease and at least,the tendencies should become quite stable.


For example,this is what i had posted 2 days after i had started this thing:

AntiVir 11
Avast 9
AVG 5
BitDefender 14
ClamAV 7
Kaspersky 22
NOD32 12

http://www.wilderssecurity.com/showthread.php?t=79135

The relative order,although the sample was small,has continued immutated untill now.What happened is that as the sample became larger,some of the differences were made more clear,for example AVG has lagged behind even more with AntiVir,Avast and secondarily Clam getting clearer distance from it.

I quite happy about it actually and even if it's not scientific,i know it's free of tester bias,since i m not in any way related to internet security,i m a univ student studying completely different thing than informatics.This is only a hobby for me.

[Honyak]

Certainly I would have no problems recommending either of them, but I am more likely to recommend an AT (like Ewido or BOClean), if someone is running NOD32. For some reason, I feel that KAV probably is more of a stand-alone product. Maybe I am misreading the situation a bit.

I agree with your assessment as when I ran KAV or KAV clone, I was not concerned with a B/U scanner. But with other AV's I will use a back-up scanner (usually KAV engine) and AT.
I do not decide my AV choice based on Jotti's, but do observe it often when in the office during the weekdays. I like the fact that it is a more real world test versus a lab test.

[vlk]

Just FYI, the linux version of avast (used by Jotti) does not currently support any Win32 unpackers (not even UPX) -- and since many scum is now constantly being (re)packed, I reckon this is making avast perform considerably worse than would its Windows counterpart.

We will provide Jotti with a version that supports exe unpackers soon - I'm curious if there will be any visible detection boost then...

[RejZoR]

Vlk,you want to say that avast! detected so much stuff without ANY unpacking?
Well,then i have no doubt that avast! will score better.
How can AV work without any unpacking anyway (unless you make hundred signatures for just one sample)?

[vlk]

Well,

1. we do have not hundreds, but tens of thousands of signatures
2. even though I THOUGHT the linux version of avast has at least some limited number of exe unpackers (at least upx, aspack etc - i.e. the basic set from avast 4.1). But - I was told that I was wrong.

[rdsu]

If the Jotti can make a service based for Windows, was better to evaluate the AV's detections rate...

[rdsu]

Another thing:

What is the interest to use Linux to test AV's detections rate!?

[Firecat]

Originally Posted by VaMPiRiC_CRoW

Another thing:

What is the interest to use Linux to test AV's detections rate!?

I guess Jotti's probably uses Linux to reduce costs.

[IBK]

Originally Posted by VaMPiRiC_CRoW

If the Jotti can make a service based for Windows, was better to evaluate the AV's detections rate...

http://virusscan.jotti.org/ is not designed to evaluate AV detection rates.