Problems caused by false positives

"SecureAnywhere's database is indeed tens of terabytes in size and hosted in the cloud."

Hi,

Would be nice an expert opinion here...

While the methodology of testing may have on impact on ,so called, "detection rate", however, I am surprised that, with "tens of terabytes of database", false positive is so high.

This concerns me more that detection rate; if WSA is configured to immediately delete or quarantine infected files, a false positive in an essential file can render the operating system or some applications unusable or your pc can become unbootable.

So, any idea why, in both consecutive tests on AVComparatives, WSA had so many FP's?


Thanks,
Claudiu

 

Folks that have any security software configured in such manner soon learn the errors of the their ways. And stop doing it, if at all

 

Well, the question was how is possible that , with a database of "databasetens of tens of terabytes" still to have such high # of FP's?

Teoretically, a huge databse and a cloud technology should give almost zero FP's.

 

I agree 100% and that's why WSA by default doesn't Delete or Quarantine any files without user intervention but if they want to brick there systems to let it do so well I couldn't of said it better Cudni!

Cheers,

TH

 

I never experience a false positives with WSA since its release. So, the problem here is negligible. Once again, another reason to leave the test(s) where they are and base your assessment on your experience and actual use. Nice try. Next?

 

@claudiu - This topic has a thread already so why do you continue to start new useless threads with the same Questions? https://www.wilderssecurity.com/showthread.php?t=333740 & https://www.wilderssecurity.com/showthread.php?t=333713

TH

 

The topic indicated was about poor detection rate of WSA, not about FP's.

My question is a technical one, that's why I was expected an expert opinion rather that an enthusiastic user opinion.

So, once again, how is possible that , with a database of "databasetens of tens of terabytes" and instant access to it , to have such ammount of FP's?



Thanks,
Claudiu

 

So enthusiastic experts don't count? Good luck then.

You can safely completely ignore AV-C's FP test. It's already been observed on the community that they turn everything up to maximum for those tests, which is what causes the FPs. At that point, Cudni's observation is most valid.

 

Because he/she is desperate to discredit WSA in any way possible. Somehow sick/disturbing/alarming behaviour at this point. Isn't it time to stop this, please? And to your technical question: It has already been said main times: Those false positives are due to rare or uncommon software been used to test FP.

 

"Enthusiastic experts" have the tendency to be blindsided by their enthusiasm, so their opinions are, most of the time , flowed .

 

"It has already been said main times: Those false positives are due to rare or uncommon software been used to test FP. "

I believe that, the same set of files were used to test ALL Av's.
So, how is possible that MSE will have ZERO FP's for the same set of "rare or uncommon software been used to test FP."

This is another example of being blindsided from an enthusiast WSA user.

Let' wait for an expert, please.

Thanks,
Claudiu

 

As usual your sick attitude keep forgetting that WSA is a behaviour based AV not a signature based AV. MSE does NOT have a signature for those files and ... "magic" they don't get detected. Rare or unknown files to the community/cloud can trigger warnings! Please stop this, its an insult to the intelligence of wilders members and readers. You keep flooding wilders, dlsreport, etc with "I need an expert..." and always, surprise surprise, its about WSA. You don't need an expert...you need some fresh air and another target, please.

 

I feel this thread is going no where so at this time it will be closed and it will be up to PrevxHelp if he wants to reopen it!

TH

 

*
I am no "expert", but I am answering @claudiu anyways:

1.) It doesn't matter how many terabytes the database is big. Because every day new software (so far unknown to the cloud) is released, good and bad. That software has to be rated first for example by watching it in action. Of course there is always a possibility having a false positive as with any given AV solution.

2.) Of course WRSA is (like previous versions of Prevx) having false positives. Anyone stating otherwise is probably a fanboy or has only standard software installed that is used by millions. - It got better over the time, at least I have this feeling and remember it being much worse in the past on my system.

But the thing is: I don't care anymore about false positives! In most cases I know that a program is trustworthy (not doing shady things like using keygens anymore) and I stop monitoring it right away. Therefore I want a more accessible link to "control active processes", because I don't want useless monitoring on my SSD.

What I care about very much is the LIGHTNESS of my not so new system. It's like having a new computer. That's mostly thanks to WRSA, because of very good Firefox development (not the ressource hog it used to be) and on my system thanks to ProcessLasso.

So yes, there ARE false positives with WRSA! And yes, Avira for example has them too (and I am sure every other AV-product that has decent detection). But it is only a problem for people that don't know what they are doing. I wouldn't rely just on what WRSA says, never! I always (!) check for example at VirusTotals. Of course, if you always believe WRSA when it says it is a virus you can easily kill legit software. Having the great feeling that WRSA saved your ass but infect didn't.

I could not let my relatives handle detections of WRSA/Prevx on their own. Because they don't know what is real malware and what is just a false positive again. But if they are past the first scan of their SSD/HDD it isn't such a big problem anymore. It used to be in the past (Prevx), yes (even if Joe said otherwise! ) but not now with WRSA. Else I would get more calls for help.

That is my experience with the product. Yours may differ. But I am being honest here as always. - And I feel secure enough with WRSA so that there is no need for anything else (like Avira before). That changed too. Now I scan from time to time additionaly with Malwarebytes free, but that's it. No problems, no infections and a very fast old system that isn't dragged down by my AV solution.

I am very happy with it and even if tests like the ones you mentioned show some unusual ammount of fp's or not so good detections I will stay with it. Killer feature is it's unique lightness.

That is what matters to me, only that! So not detection rate (if it's not abysmal) or how much false positives it has. - I get never infected anyway and know my stuff.

But there is always room for improvement of course! I would prefer any time a product that has near 0 fp's even if that means it has only 90 % detection rate and not 99. - Because it can be a real hassle to deal with fp's, sometimes that is the real harm done, not the malware you always fear but never experience! I remember a time when I called Prevx "scareware" because of that and I meant it (having to deal with relatives who thought they were infected all the time but weren't). But not anymore with WRSA. So good job and well done, Joe and others.

*Just saying that in my experience that old FP problem isn't such a problem anymore.

But to be sure I will delete now all my detection overrides (there are a lot, long list!) and see how many FP's there are back after doing a new scan and let you of course know if it is real bad.

 

I've reopened this thread but I'm largely just echoing the comments already made by users here. I would consider many users on this forum experts not only in WSA but in security software in general, and their viewpoints are completely accurate. However, I'll reiterate them myself just to ensure you're getting the "official" word.

The impact and true false positive rate of security software is very hard to measure. Yes, every AV has the occasional false positive, and in some AV tests, they may detect a handful of files as malicious which aren't, but it is the scope of these files which is very relevant. WSA intentionally closely scrutinizes little-known software or software which has recently changed or was recently released. We've looked at the data and virtually every file which we've FP'd on in the last many AV tests was seen for the first time in the WSA community by the testing firm. That just shows how obscure some of these programs are. As an interesting observation, in some tests, vendors are sent the files they didn't detect as malicious.

WSA has significant measures in place to prevent FPs on more common applications/system components so the end result is a very low false positive rate with very high protection.

In the case of many AV tests, the files are all static (not executed). This takes them completely out of context so we can't apply the logic that users normally see in the actual use of WSA. This unfortunately results in lower detection and higher false positives, and is just a factor of how our product works. We could probably change it to work differently in these scenarios but we have many more important things to focus on which provide value to real users rather than perform better in tests that don't replicate what real users experience.

 

I had this file come up as a detection recently, and I posted about it in the forums. Not a false positive, but I chose to ignore the detection, because I will never activate it. It is an information only file.



P.S. I will try and find my other post about the file.

 

Hi PrevxHelp,

Thank you for your answer!

The fact that "...virtually every file which we've FP'd on in the last many AV tests was seen for the first time in the WSA community by the testing firm..." should't be used as an excuse for the high # of FP.

If Webroot would have had those files in the "tens of terabytes" of database in the cloud of course those files wouldn't have been anymore FP'S.

All players tested were exposed to the same set of files; I doubt that any other vendors have a significantly better database that Webroot /Sophos and Prevx cumulated. In fact traditional AV's like MSE will rely on a significantly smaller database localy stored.

So ,is not a matter of files seen or not seen before, is how a product can handle a file notseen before, or so called zeroday.

And WSA has to drastically improve this area!



Claudiu

 

Thankfully you couldn't be more off the mark and incorrect.

"Not seen before" means that it wouldn't be in those terabytes of information. That much is obvious to anybody with half a wit.

Traditional DBs don't know anything about the files either is precisely why they don't detect them as FPs, but also wouldn't be detecting them as true threats if they didn't have defs for them. The AV DBs only know "Bad" or "Unknown", and everything good is unknown. So they just ignore those along with everything good. The Webroot DB has information on millions if not billions of Known Good things, so it will -never- FP on these, ever, unlike other products which know a tiny handful of critical whitelisted items and the local DB can define anything else as bad.

And the community thread points out that the AV-C test puts all the settings to maximum for the FP test. If you knew anything at all about the product you are trying to hate, you'd know this means that popularity heuristics are at maximum then, which means that first-seen things would be warned about, but just WARNED about it being new to the Webroot Community with "Allow" as the default. Unfortunately, AV-C considers this warning a fail for the FP test because the FP test is not as smart as most humans. Of course it's always possible for some humans to make the wrong decision or show bad judgement, as this thread shows.

 

Once again,all players tested were exposed to the same set of files;when you see WSA getting the last place and AV Comparative saying "crazy many FP", well ....something is not right!

 

Or it just means that WSA has a different userbase than the other products and our users see different files than users of other AVs. Despite being an interconnected planet, there are massive differences in geographies with what software is used.

 

Once again, the other AVs may not know about the files either, and since they don't warn people about files they've never seen before, they don't fail the test as an FP for that file for saying "This has never been seen before".

 

I think what he is saying is, why do other AV programs with 100MB databases have no FP's and when we talk about WSA's Terabyte hive, how can other AV software with small DB's out_perform WSA and what exactly is WSA using all that data space for.

 

Because signature AV only store malware fingerprints and thus are silent to good or unknown files (good or not). WSA works differently, it store behaviour (endpoints) of all software. A software unknown to the cloud can raise suspiciousness.

Once you understand how WSA works (not signature based!) you will also understand why you have certain effects. Effects that you will likely only see in tests by running uncommon files.

 

To me, the only fair way for a testing organization to test WSA would be to scan a set of malware, then the ones so-called not detected, run them and then see if WSA detects them. By adding these two totals together, do we then see a accurate assessment of its ability.

 

Exactly

I completely agree. I know some people don't follow their tests, but this is exactly what PC Magazine does when testing security software and most closely mimics what a real user encounters.