FPs=In the Cloud?

[trjam]

Ok, Joe, putting you on the spot here but I have to. My biggest argument has been and still is FPs. I know you say this has been drastically reduced but I have a question. It would seem to me that with In the cloud reporting back to your servers that at some point this would rectify this issue. The FPs I see here are fairly common so they have had to been checked by the cloud technology. I guess I still dont understand.

I do understand that any vendor can max their hueristics settings out and detect everything. And if it is a test checking for only 10 pieces of malware, well you come out looking good because it caught them all. But in tests like AV-Comparitives it looks at not only detection but at the number of FPs. And that is the kind of test that I personally feel will tell the truth about Prevx.

Do we just max the settings and fix the ones reported, or at some point are the settings adjusted to balance out the good from the bad. Cause I have to be honest, I love Prevx, but to me the FP rate is still to high which explains why everything is detected.

[PrevxHelp]

I will say again that the FPs are near zero compared to the volumes detected If you look at the FP thread going here, there is just a small trickle and the volume reported to us internally is even less. We receive about 2 reports per week to report@prevxresearch.com.

We don't "max out" our heuristics and we also have a unique view from any other vendor as to what FPs are real threats to many users. Most of the FPs we get reported to us are seen by less than 5 users - any FP which would affect a large number of users is automatically caught and dropped so we rarely have true "critical" FP issues like some other AVs have historically had.

The largest FP I've seen in ages affected about 100 users and was ironically caused because of one of our human researchers marking that single file incorrectly

I still believe our FP rate is well within the normal range and probably outside of it (to the low end).

[trjam]

Ok, well then, I guess that answers my question. Prevx is good. Prevx is very good and for the most part, I never know what I am talking about anyway. But I have my eye on you Joe.

[TonyW]

Originally Posted by trjam

Cause I have to be honest, I love Prevx, but to me the FP rate is still to high which explains why everything is detected.

I would ask to whom is the FP rate high? It does, I think, depend on the user and what they're installing on their systems. Like with any conventional AV, you'll hear of some users reporting FPs and others who say they haven't had any, but that's likely because they don't have the application that may flag up a FP on their system. It's all very relative.

As long as people know how to report a FP should they encounter it, there shouldn't be a problem, but obviously the less FPs the better; however, it isn't an exact science and as such the coding will always need to be tweaked whichever anti-malware program you use.

[Pleonasm]

Originally Posted by trjam

But in tests like AV-Comparitives it looks at not only detection but at the number of FPs. And that is the kind of test that I personally feel will tell the truth about Prevx.

Not many folk appear to realize that there exists an entire ‘discipline’ on this subject, built upon signal detection theory. Anti-malware vendors should consider calculating and reporting the d’ statistic, but I have never seen any that do so.

Originally Posted by PrevxHelp

We receive about 2 reports per week to report@prevxresearch.com.

PrevxHelp, to what extent is the status of samples that were initially classified as malware changed? Isn’t that the accurate way to measure false positives -- rather than user complaints?

For example, if Prevx initially classifies 1,000 new samples as malware, but within two hours updates their designation to “safe,” then the software has 1,000 false positives over that time period.

[PrevxHelp]

Originally Posted by Pleonasm

PrevxHelp, to what extent is the status of samples that were initially classified as malware changed? Isn’t that the accurate way to measure false positives -- rather than user complaints?

For example, if Prevx initially classifies 1,000 new samples as malware, but within two hours updates their designation to “safe,” then the software has 1,000 false positives over that time period.

When we create a new rule or signature, we are automatically warned if it could potentially create FPs (because we're able to compare it against the historical view of file data) so we don't have to go back and correct them because we know the exact effect of every change we make.

Unless a user reports a FP we don't have to fix anything and FP reports generally come from wide-reaching signatures which detect tens/hundreds of thousands of individual threats but may generate 1 stray FP.

[Defenestration]

If a user adds a flagged file to their exclusion list, is this sent to PrevX automatically so the researchers can look at this file/signature and possibly fix the FP, if necessary ?

I thought it did, and so didn't always report FP's via e-mail/forum.

[PrevxHelp]

Originally Posted by Defenestration

If a user adds a flagged file to their exclusion list, is this sent to PrevX automatically so the researchers can look at this file/signature and possibly fix the FP, if necessary ?

I thought it did, and so didn't always report FP's via e-mail/forum.

Yes it does, but sometimes for low-volume FPs they get lost in the mess of malware authors trying to game the system but we still do go through each of the reported FPs. If you do see a FP not being fixed swiftly, feel free to post here/email us/PM me and I'll see why it wasn't caught quickly