could you please correct your data about Threats missed by other security vendors

[vtol]

shown here

http://www.prevx.com/avgraph/12/Eset.html

as according to Eset Moderator:

-{ Quote: "Please avoid referring to Prevx statistics which are ridicular." }-

in this post # 68

http://www.wilderssecurity.com/showthread.php?t=270288&page=3

trusting publishing ridiculous numbers does not elevate Prevx's reputation... ...but wait, maybe it is because it does not make them look so good

[Konata Izumi]

There's nothing to correct here.

[vtol]

-{ Quote: "There's nothing to correct here." }-

maybe it is a bit cheeky to ask why not - care to elaborate, it is your statement vs. Eset's moderator? latter working for a AV vendor

[funkydude]

-{ Quote: "maybe it is a bit cheeky to ask why not - care to elaborate, it is your statement vs. Eset's moderator? latter working for a AV vendor" }-

If you look at the real word situation, this information is pretty accurate. Maybe 80-90%. Like it or not, I personally feel ESET (and a lot of other AV products) have slipped immensely over the years.

Thats my opinion, I won't add anything further to prevent starting an argument.

[vtol]

-{ Quote: "If you look at the real word situation, this information is pretty accurate. Maybe 80-90%. Like it or not, I personally feel ESET (and a lot of other AV products) have slipped immensely over the years.

Thats my opinion, I won't add anything further to prevent starting an argument." }-

no argument here, is already going on in the eset forum, but seems eset is not so much concurring with you and users of their product. might be the reason their forum moderator call the Prevx data ridiculous, just guessing

[SweX]

@Vtol

Not trying to be rude here. But you joined Wilders in April 2010, so you probably don't know that this has been discussed here many many times before.

And if you search you will probably find more threads discussing about these Prevx statistics, and some answers too ;)

But i'm sure Joe is around the corner and will soon give you some answers anyway. :)

[PC__Gamer]

-{ Quote: "shown here

http://www.prevx.com/avgraph/12/Eset.html

as according to Eset Moderator:



in this post # 68

http://www.wilderssecurity.com/showthread.php?t=270288&page=3

trusting publishing ridiculous numbers does not elevate Prevx's reputation... ...but wait, maybe it is because it does not make them look so good" }-
there is nothing wrong with the graph, people just mis-read it and compare each vendor to each other in the graph, this is a false mis-interpretation.

if only 1 pc is checked with MSE and 5,000 with NOD32, NOD32 would be very likely to have many more infections on their customers computers.


the problem that people see in the graph is people compare each vendor to each other:

Nod32 yesterday leaked in 6408 according to Prevx, while customers using Panda only leaked in 497, this is a false comparision & does not make Panda (although it could still be) better.

The graph DOES NOT tell people how many files/computers/customers were checked to each vendor, so there is zero comparision between each vendor. (this is exactly the thing alot of people do, and then complain about)




--- the graph shows simply how many infections have been caught on X-amount of computers using that particular antivirus.

--- but what the graph DOES show, is that prevx is finding infections on all computers regardless of what Antivirus they are using.




Hope that helps. ;)

[trjam]

I think it is safe to say that, some of those AVs may have also detected something Prevx may not have. That is why Joe talks so much about how they compliment each other. The reality is, a good AV and Prevx will give you just about all the horsepower you need out of a engine.

[vtol]

-{ Quote: "there is nothing wrong with the graph, people just mis-read it and compare each vendor to each other in the graph, this is a false mis-interpretation.

if only 1 pc is checked with MSE and 5,000 with NOD32, NOD32 would be very likely to have many more infections on their customers computers.


the problem that people see in the graph is people compare each vendor to each other:

Nod32 yesterday leaked in 6408 according to Prevx, while customers using Panda only leaked in 497, this is a false comparision & does not make Panda (although it could still be) better.

The graph DOES NOT tell people how many files/computers/customers were checked to each vendor, so there is zero comparision between each vendor. (this is exactly the thing alot of people do, and then complain about)




--- the graph shows simply how many infections have been caught on X-amount of computers using that particular antivirus.

--- but what the graph DOES show, is that prevx is finding infections on all computers regardless of what Antivirus they are using.




Hope that helps. ;)" }-

let me try to digest it with my simple brain of the number's causality.

the lesser of number of users of any particular av solution the less protection is assumed? or 5 computers getting infected with 10 viruses is less good then 5,000 computers getting infected with the same 10 viruses?

if understood right the Prevx data does not count the number of computers but the number of things getting by each listed AV. Certainly the kind of infections will differ but the data gives an abstract of the number of infection passing by, which I see as comparative, since no AV shall judged by the number of user but by the number of infections getting by.

appreciating clarification if wrong

[vtol]

-{ Quote: "I think it is safe to say that, some of those AVs may have also detected something Prevx may not have. That is why Joe talks so much about how they compliment each other. The reality is, a good AV and Prevx will give you just about all the horsepower you need out of a engine." }-

that is well is understood and appreciated, but is not the point of the Prevx data published at their website

[vtol]

-{ Quote: "@Vtol

Not trying to be rude here. But you joined Wilders in April 2010, so you probably don't know that this has been discussed here many many times before.

And if you search you will probably find more threads discussing about these Prevx statistics, and some answers too ;)

But i'm sure Joe is around the corner and will soon give you some answers anyway. :)" }-

no offence taken, like any sort of input/reply.

though admitting being a newbie on this forum, which might not be the way one should be judged by, I am not a newbie to forums at all, neither to computers nor to AV.

I did a bit digging prior posting, just did not help me with the comment of the Eset moderator, calling that Prevx data ridiculous. And since Eset believes in the stupid user I thought to give them support.

Perhaps I should have made the ironic tone a bit more clear.

[Cudni]

If it is hard data and it looks like it then no need to change anything. Only shows that not 1 AV, nor 1 security software, can be the only defence (for majority of users as there are some who don't need an AV).
It would be interesting see the breakdown by AV/version but that is just me :)

[vtol]

-{ Quote: "If it is hard data and it looks like it then no need to change anything. Only shows that not 1 AV, nor 1 security software, can be the only defence (for majority of users as there are some who don't need an AV).
It would be interesting see the breakdown by AV/version but that is just me :)" }-

now I understand the goal of the data, to make people aware that there is no such thing as 100% protection, but that might not be achieved even with a layered approach, except for users being vigilant on top of anything else.

having said that, those numbers could be randomly chosen by Prevx, which though I would like to believe are real world statistics. And thus providing a comparison to the curious eye.

in which case showing Eset, not only there, say here too http://www.pcworld.com/reviews/product/413777/review/smart_security_4.html, performing badly, something disliked by the staff over at Eset and therefore dismissing any of those, calling it ridiculous...

[Cudni]

-{ Quote: "
having said that, those numbers could be randomly chosen by Prevx, which though I would like to believe are real world statistics. And thus providing a comparison to the curious eye.
" }-

it could be this and could be that but it needs to bear the scrutiny if challenged. Prevx has everything to lose and nothing to gain from not being able to support the publicised data. To me, it is what it is and all AV are in each others good company.

[vtol]

-{ Quote: "To me, it is what it is and all AV are in each others good company." }-

for me it actually matters how good an AV performs, not whether the company or its employees are good.

[TonyW]

-{ Quote: "the problem that people see in the graph is people compare each vendor to each other:

Nod32 yesterday leaked in 6408 according to Prevx, while customers using Panda only leaked in 497, this is a false comparision & does not make Panda (although it could still be) better." }-However, that's not what the current debate is about. The comparison is on Eset alone in this instance.

I agree people shouldn't compare vendor by vendor as the detected samples in each graph may not be present on the system protected by an alternative product.

Considering the current discussion, to a casual observer, the statistics for X vendor implies that yesterday computers with X vendor installed collectively missed (n) infections that were detected by Prevx.

If this is the wrong way to look at it, the phraseology needs to be changed, but that is how it comes across when first looking at the information.

[PC__Gamer]

Yes tony, that's what the graph is saying, yesterday prevx found that amount of infections from users using nod32.

It does not however, tell you how many computers using prevx and nod32 there are.

[Triple Helix]

-{ Quote: "shown here

http://www.prevx.com/avgraph/12/Eset.html

as according to Eset Moderator:



in this post # 68

http://www.wilderssecurity.com/showthread.php?t=270288&page=3

trusting publishing ridiculous numbers does not elevate Prevx's reputation... ...but wait, maybe it is because it does not make them look so good" }-

It won't happen as every AV or AM company has there own way of marketing on there website! ;) As said above by SweX this subject has been mentioned so many times before! It's best to see it on there main page here: http://www.prevx.com/ Just scroll down!

TH

[TonyW]

-{ Quote: "It does not however, tell you how many computers using prevx and nod32 there are." }-This is why I said "collectively". We don't know how many NOD32+Prevx installations there were yesterday, but the graph implies NOD32 missed over 6,000 infections on all those machines it was on, bearing in mind each machine won't have the same detections as another. It is obviously quite complex.

[IBK]

below only an opinion/guess:
example:
product X is listed as having 341 missed infections
eset is listed as having 6408 missed infections
you do not know how many users/PC's are behind the numbers. Maybe Prevx found the 341 infection on 300 different PC's, and e.g. 290 is the number of users which have Prevx and product X running (which would mean product X failed for almost every user). While e.g. the 6408 infections were found e.g. on 4000 different PC's, while the number of users running ESET and prevx may be much higher, e.g. 50000 (esp. considering that ESET and prevx are hosted on same forum - wilders, ESET may be more represented, as well as the free AV's avast and AVG). even if prevx says that the numbers should not be compared and are meaningsless, users see a graph which suggest a comparison and indirectly also that prevx detects more than the other products, although it does not say that other products may detect much more than prevx if tested the way around (as everyone is detecting something that some other one may miss) and also does not say anything about false alarms (which also prevx has and affect the results) or "disarmed" samples (renamed/quarantined files and leftovers).
it would not hurt to give the information about how many users are behind each number (should be not difficult to provide and also give more details to the readers). while its true that all vendors miss threats, maybe the additional information would give an insight which would no longer fit with the intended scope when displaying this kind of chart.

[Konata Izumi]

-{ Quote: "maybe it is a bit cheeky to ask why not - care to elaborate, it is your statement vs. Eset's moderator? latter working for a AV vendor" }-

Sorry, it's hard to type and I'm bad at english. Just read what people will say here.
If there's something that needs to be changed, it is that ESET's moderator comment. :D
P.S. Sorry again for giving another statement without elaborating. :)
Please wait for Joe and see how a good moderator should answer.


@IBK
+1

Konata Izumi

[vtol]

-{ Quote: "below only an opinion/guess:
example:
product X is listed as having 341 missed infections
eset is listed as having 6408 missed infections
you do not know how many users/PC's are behind the numbers. Maybe Prevx found the 341 infection on 300 different PC's, and e.g. 290 is the number of users which have Prevx and product X running (which would mean product X failed for almost every user). While e.g. the 6408 infections were found e.g. on 4000 different PC's, while the number of users running ESET and prevx may be much higher, e.g. 50000 (esp. considering that ESET and prevx are hosted on same forum - wilders, ESET may be more represented, as well as the free AV's avast and AVG). even if prevx says that the numbers should not be compared and are meaningsless, users see a graph which suggest a comparison and indirectly also that prevx detects more than the other products, although it does not say that other products may detect much more than prevx if tested the way around (as everyone is detecting something that some other one may miss) and also does not say anything about false alarms (which also prevx has and affect the results) or "disarmed" samples (renamed/quarantined files and leftovers).
it would not hurt to give the information about how many users are behind each number (should be not difficult to provide and also give more details to the readers). while its true that all vendors miss threats, maybe the additional information would give an insight which would no longer fit with the intended scope when displaying this kind of chart." }-

if that true than the numbers are indeed misguiding, in particular as the table just below the chart refers to 'the top files from all groups that bypassed Eset security products', singled out by specific filenames. Which sorts of leads to the impression that numbers shown are related to a specific threat but not to the number of systems, whereas also duplicate detections would count and thus blowing up the numbers unrelated to the number of threat specific infections.

Yep, so in that case the chart might be altered accordingly, which though may not have the desired marketing effect anymore, or if the numbers are thread specific Eset is just a bad performer and its moderator somebody in denial.

[PrevxHelp]

-{ Quote: "While e.g. the 6408 infections were found e.g. on 4000 different PC's, while the number of users running ESET and prevx may be much higher, e.g. 50000 (esp. considering that ESET and prevx are hosted on same forum - wilders, ESET may be more represented, as well as the free AV's avast and AVG)." }-

This is exactly correct and we post this on the chart itself saying that more popular vendors will logically miss more detections.

-{ Quote: "even if prevx says that the numbers should not be compared and are meaningsless, users see a graph which suggest a comparison and indirectly also that prevx detects more than the other products, although it does not say that other products may detect much more than prevx if tested the way around (as everyone is detecting something that some other one may miss)" }-

We're gladly open to seeing the charts from other vendors as well :)

-{ Quote: "and also does not say anything about false alarms (which also prevx has and affect the results)" }-

Assuming a catastrophically high false positive rate of 1%, the numbers don't really change at all.

-{ Quote: "or "disarmed" samples (renamed/quarantined files and leftovers)." }-

Prevx only focuses on active infections so these types of quarantined or disabled samples won't be detected.

-{ Quote: "it would not hurt to give the information about how many users are behind each number (should be not difficult to provide and also give more details to the readers)." }-

We have considered this in the past but have no intentions of doing so because it would turn us into a testing organization. We are merely trying to prove the point that no products detects 100% of threats - while many users consider that to be obvious, misleading product names like "Total Protection" hide the point and back users into a corner making them think that their single layered solution is sufficient when no vendor is perfect.

-{ Quote: "while its true that all vendors miss threats, maybe the additional information would give an insight which would no longer fit with the intended scope when displaying this kind of chart." }-

Again, we aren't trying to compare vendors here - the chart is raw data and explains exactly what it should. From our "Explain this Chart" area:

-{ Quote: "The Security Vendor chart displayed above shows, in simple terms, a total count of malicious programs found yesterday by Prevx products on PCs protected by security products supplied by each of the vendors shown.

You should expect to see a higher number against the more popular security vendors because we see more of these users and consequently a higher number of malware infections.

These statistics are provided to show that all vendors miss threats and cannot be interpreted to compare the effectiveness of one product to another." }-

[Konata Izumi]

Well said. I'm lovin' Joe :-*
...I mean PrevX ...okay, Joe too ;D

[BoerenkoolMetWorst]

Its just a graph which can be easily be misinterpreted. And even if it wasn't, Eset would be a hypocrite, look what's on their site:

http://www.eset.com/home/compare-eset-to-competition

[acr1965]

Not to fuel the fire any, but I was under the assumption that Norton, McAfee and Trend made up about 60%-70% of the consumer market. But they seem to make up only about 17% of infections found. It's hard to believe that the disparity between the big three is that substantial over the other vendors.

Also, of the infections found: how many are later determined to be false positives by Prevx and how many infections found are on computers using out of date software (e.g.- license expired and no updates)? Also, as mentioned earlier, how many machines were infected as opposed to number of total infections?

[BoerenkoolMetWorst]

Prevx probably has a much higher market share among the computer-savvy persons, and Norton and McAfee probably have a higher market share among the computernoobs as on a lot of PC's they are already preinstalled by the manufacturer, plus a lot of people dislike them, so that could have a great influence on the samples missed by Norton and McAfee, but I'm not sure if TM is also preinstalled.

[Konata Izumi]

PrevX is not a testing organization. Okay?

@acr1965
That was already answered by PrevXHelp (Joe).

[Pleonasm]

If a user is currently running Eset on their PC, installs the paid version of Prevx today while maintaining the installation of Eset, and Prevx detects malware missed by Eset, then the count of “threats missed by other security vendors” for Eset is incremented -- which makes sense.

Now, what if the same user who is running both Eset and Prevx is infected with malware tomorrow morning (i.e., the threat was missed by both Eset and Prevx) and then later in the afternoon that same threat was detected by Prevx -- will the count of “threats missed by other security vendors” for Eset also be incremented despite the fact that the PC was protected by Prevx at the time of the attack?

Perhaps a simpler way of probing the issue: is every instance of a “threat missed by other security vendors” based only upon the detection scans by the free version of Prevx (which has no prevention capabilities)?

[Konata Izumi]

-{ Quote: "
Perhaps a simpler way of probing the issue: is every instance of a “threat missed by other security vendors” based only upon the detection scans by the free version of Prevx (which has no prevention capabilities)?" }-

The chart might be for both free and paid version of PrevX.

[Pleonasm]

-{ Quote: "The chart might be for both free and paid version of PrevX." }-
And, if the “threats missed by other security vendors” chart is based upon data from both the paid and the free version of Prevx, then that is potentially very troublesome. Malware that is missed by both the paid version of Prevx and Eset (for example) and is downloaded/executed/installed, and is then later discovered by Prevx shouldn’t increment the count of “threats missed by other security vendors” for Eset unless it also increments a corresponding count for “threats missed by Prevx.”

So, the question remains: is every instance of a “threat missed by other security vendors” based only upon the detection scans by the free version of Prevx? The fact that the chart is described as representing “a total count of malicious programs found yesterday by Prevx products” (plural “products”) suggests that the data may in fact be based upon a combination of PCs running either the free or the paid version of Prevx.

[PrevxHelp]

"Threats missed by other security vendors" includes data on threats that are Active on the PC when Prevx is first installed - not files that also got past Prevx. It is irrelevant if Prevx is registered or not because the threats would already be active - we only count data from a first installation of a new user as that is when we'd be seeing the pre-existing infections, and we measure if the AV is reporting to the security center and enabled + up to date.

[Pleonasm]

-{ Quote: ""Threats missed by other security vendors" includes data on threats that are Active on the PC when Prevx is first installed - not files that also got past Prevx. It is irrelevant if Prevx is registered or not because the threats would already be active - we only count data from a first installation of a new user as that is when we'd be seeing the pre-existing infections, and we measure if the AV is reporting to the security center and enabled + up to date." }-
I was hopeful this would be the answer! :)

As a corollary, this implies that the total daily count of the “threats missed by other security vendors” across all vendors is an estimate of the total number of new Prevx users per day -- correct? (The estimate may be somewhat high, because more than one threat may be detected per PC upon the initial installation of Prevx.)

Additionally, when you say “we measure if the AV is reporting to the security center and enabled + up to date,” how do you define “up-to-date”?

[PrevxHelp]

-{ Quote: "
As a corollary, this implies that the total daily count of the “threats missed by other security vendors” across all vendors is an estimate of the total number of new Prevx users per day -- correct? (The estimate may be somewhat high, because more than one threat may be detected per PC upon the initial installation of Prevx.)" }-

It is a subset of all of the users of Prevx, not counting SafeOnline or any of our OEM deals or partners distributing Prevx.

-{ Quote: "Additionally, when you say “we measure if the AV is reporting to the security center and enabled + up to date,” how do you define “up-to-date”?" }-

There is a flag set in the security center information which AV products transmit that says "productUpToDate". It is maintained by the AV software itself but if the AV product is lying to the user, I suspect there are more issues involved than just detection :)

[redwolfe_98]

i think prevx takes the position that if something is flagged by prevx and there is another av installed, the other av missed it, and that that is how they get their statistics, prevx flagged something and so it must have been missed by the other av that was installed..

i think the vast majority of instances where prevx is supposedly flagging something that was missed by another av is when people use the eicar test file, to test prevx, and prevx counts that as prevx catching something that some other av missed..

[PrevxHelp]

-{ Quote: "
i think the vast majority of instances where prevx is supposedly flagging something that was missed by another av is when people use the eicar test file, to test prevx, and prevx counts that as prevx catching something that some other av missed.." }-

This is incorrect - if you look at the pages under the charts, you can see the most popular filenames used: http://www.prevx.com/avgraph/2/Avast.html

I did a search through our database and eicar is not within the top 100,000 threats seen on a daily basis.

[Pleonasm]

-{ Quote: ""Threats missed by other security vendors" includes data on threats that are Active on the PC when Prevx is first installed" }-
To clarify, does “active on the PC” mean that Prevx has detected installed malware -- or, that it has a detected the presence of a malicious file that has yet to be executed? (Thank you.)

[PrevxHelp]

-{ Quote: "To clarify, does “active on the PC” mean that Prevx has detected installed malware -- or, that it has a detected the presence of a malicious file that has yet to be executed? (Thank you.)" }-

It counts malware which is currently active (i.e. running in memory) or will become active (pointed to by a bootup registry entry, shortcut in the startup folder, etc.)

[Matthijs5nl]

All companies display information to persuade people to buy their product. I think PrevX is better known by expert users than average users. Therefor expert users (we) will absolutely interpret the information right: the chart shows no product is perfect and you need a second-opinion/layer. But I agree that average users might interpret it wrong.
But like I said all companies use these types of charts, and PrevX does it in this way. It is a little questionable because PrevX also knows people might misinterpret it. But it is completely fair because PrevX explains the charts: if people don't read it it is not their mistake. Also the data will be true.
It will be fair to average consumers add the number of pc's (7000 missed samples on 1 million ESET pc's or on ten thousand?). But this is won't be fair since PrevX will be COMPARING the products of their concurrents then. Particularly because it are self-collected data which is can never be completely free of errors.

If all visitors of the PrevX site would click on the 'Explain this chart' button there is nothing PrevX is doing wrong.

But PrevXHelp: when I click on ESET I see you detect a lot of (missed) Cloaked Malware with PrevX. I don't really understand the category cloaked malware. What kind of malware does it contain?