WSA Poor Detection Result

A poor virus detection result for WSA (latest AV Test Org report).
In past detection tests WSA used to score Five and a Half, now it is only Two and a Half.
Either WSA is now completely missing more zero day threats, or these threats are being allowed to run in monitor mode for longer before being fully dealt with, and AV Test Org is by its own testing methodology giving WSA more fails. Whatever WSA seems to be much less efficient these days at (completely / quickly) eradicating zero day malware.

 

Got a link to that latest report?

 

Same discussion but different months here:
https://www.wilderssecurity.com/showthread.php?t=351147

 

Still too many FPs.
Webroot placing its faith in its rollback feature.

 

You might be also interested in reading this thread from the Webroot community ... quite long but giving explanations what lies behind Webroot's results in tests.

 

I don't know why WSA still participates in the AV-Test, unless this decision is out of their hands. The results don't make WSA look good.

 

Hopefully it sparkles for better times, have a look in this thread, especially post made by Grayson (the first on the third page).

 

We're working very closely with AV-Test/AV-C on new testing methodology which better reflects the many aspects of WSA which work very differently from other products (not just rollback).

I believe we'll be posting a more detailed response shortly with more specifics - as soon as I see it posted (or if someone else catches it before me), I'll link it in here.

 

Sounds great Joe, looking forward to it, thanks for letting us know in advance

 

honestly Joe, I don't buy that idea of detecting by a different way. You either detect or you don't ~Phrase removed~

Bring Prevx back.

 

That's actually not the case - there are many cases where a threat does nothing or is prevented from doing anything, which is not taken into account in these tests.

As you know, WSA is effectively Prevx 4 (and considerably more), and 100% of Prevx 3 is included in it. Prevx 3 would score abysmally in the recent AV-Test/AV-Comparatives tests despite actually blocking threats and actually protecting the user. This is exactly why we're working with the testing firms on assessing WSA as accurately as possible. The reason for the declining scores over time is because of a changing malware landscape, moving from being blocked by signatures to being blocked by other methods.

 

we really need something concrete as to why the results are so poor, this has been rumbling on now for a while

 

Well from what I've seen, this same query/discussion keeps getting regurgitated and analysed, and yes it appears WSA gets mixed test results in recent times, but the proof of the pudding is that in real-world everyday usage, people are overall remaining infection free if using WSA.
https://www.wilderssecurity.com/showpost.php?p=2261339&postcount=10

 

But at the same time, Dermot, many people out there are still not grasping the concept of how WSA works, including some of the testing organisations whom the company is now actively working with.

A lot of users understand the traditional blacklist/heuristic approach because that's been accepted for so long, and have yet to fully appreciate "the many aspects of WSA which work very differently from other products (not just rollback)." Webroot certainly have a hard task in convincing some that their new method works.

As to real-world usage, I agree with what you say, but then you could say that about other products. I've remained infection free with WSA, but that's also true with others I've tested. Admittedly, testing organisations do test against samples you or I are hopefully never likely to come across in everyday usage so that's why that point stands.

 

I get occasionally zerodays of a friend's (malware reverse engineer for a bank) honeypot. With the few tweaks I have applied I allways get a 100% detection score.

My Tweaks
This is based on increased whitelisting for risky sources, e.g. internet (set popularity heuristics to high), setting internet facing programs manually to untrusted (keeps them in LUA type of sandbox), increasing identity protection to denying change of browser for normal HTTP (raising default). Last tweak (firewall warning for untrusted programs) does not make a differences in these tests.

 

Kees what are you using for the heuristics rating, before or after age/Popularity?

 

Also, when you mean keeping internet facing programs set to untrusted, do you mean set to "monitor" in Control Active Processes?

 

Just curious, mostly these tests use Windows 7 machines, Windows 8 is supposedly more secure. I use Windows 8 and WSA so I have to assume I'm better protected than Windows 7 users, correct ?

 

The big problem with the tests (and the folks who scream that they are right) is that they still run based on the premise of "If it's not detected immediately, it won't be detected in a useful time frame" and "If the threat code runs, it's game over".

With other AV, this is absolutely the case. If you scan and it doesn't detect, it will take an average of two weeks to be detected. If the malware code runs, the computer is considered 100% compromised. All because the other AV only knows "Bad" and "Everything else", and "Everything else" is Good or Unknown (bad) or Unknown (good), but gets free reign, all its stuff done, and no further inspection except to check it against the bad list on occasion and do a generic cleanup in the event that it's detected finally.

With Webroot, sandboxing, shielding (Especially ID), ongoing cloud inspection, live heuristics*, and journaling mean that neither of those assumptions is correct. If it's not detected immediately, it can definitely be detected, for example, 30 seconds or fifteen minutes or even two hours later based on actions taken while running. Even if it runs, it is known to be Unknown (as opposed to known Good), so it's poked, prodded, watched, segregated, and prevented from doing a lot of things, and its (sometimes only attempted) actions are sent to the cloud for examination. So it does not have free reign and in real life cases, it does not get its stuff done at all and ends up removed 100% cleanly due to journaling.

For brevity and time efficiency, the tests make some assumptions of fact that hold true for other AV, but simply do not for Webroot SecureAnywhere.

(* Standard heuristics involve looking at the code or semi-executing it to see what certain portions do. This is all based in a fragment of a second during a scan. Live Heuristics involve looking at what the Unknown is doing when it is actually running fully. Oh, it waited fifteen minutes, connected to a site in another country, and downloaded another chunk of code to run into memorry without making a file for it? The heuristics in normal AV would never catch it because the site was stored as a little-endian 32-bit integer? We know that's bad to connect there. We know it's bad to download machine code into RAM (where it would evade other AV, not being a file), and we know that machine code has just tried to get to your stored passwords in Firefox. We blocked it from getting the stored passwords, and now we whack the whole program as Bad. For another AV, this would require a threat researcher to look at the program for over fifteen minutes and see what it does, then make a definition, and finally get that definition to you. Your Firefox passwords are long-gone by then and the master key for them has been rainbow-tabled perhaps, assuming you even protected them with a master key.)

 

As usual, excellent explanation!
Yes, its hard to get this to the users. They are simply not prepared for this, especially the Gurus that are used to the "business as usual" security landscape.

 

How does this differ from KAV's System Watcher and Bit Defender's Active Virus Control? Both of which claim to monitor the behavior of all active processes on your PC in real time.

 

Neither can revert back all actions done by the process, right?

 

I believe that Kaspersky claims that it does.

 

Are you referring to something similar as to allowing the code to run sandboxed while it is being analyzed to determine whether it is malicious, and then removal if determined to be a threat? Since the code was not allowed to modify any important system components WSA should get a pass or given credit for blocking the threat even if it did not initially detect it?

 

so the malware is sandboxed or isolated and allowed to do its thing until it is determined to be a threat. So it still gets to do the nasty, until at some point off in the future it is determined to be a threat. Then it is deleted. But it has already done its damage.

Sorry, but this sounds rather weird or yet better, an excuse, for not determining it was malware to begin with.