WSA Shares PCMAG Editors' Choice Award

http://www.pcmag.com/article2/0,2817,2372364,00.asp

Congratulations, and keep up the great work!

 

The scores seem a little wrong, as every other testing organization since the conception of WSA has placed BitDefender higher in detection score, Infact this is one of the reasons people complain about WSA, WSA staff say WSA "Works in a different way" this is why its static detection of Malware is poor but it can "Roll Back" changes. I also find it suspect that Norton is high up also due to the way it detects threats, and having to enable advanced set of Malware detection rules to get good static detection results. The fact that both products beat BitDefender in static Malware detection is suspect at best. Webroot 100% all Malware detected, is like the most fishy statement of the year compared to what we have seen in every other test in the past year.

Also don't delete my post, its a valid question or statement.

 

Read the testing process for each one.

Unlike testing organizations, this reviewer can spend several weeks testing one product by itself and acting like a real user rather than a machine doing testing.

On the lines of "the special way WSA works", though it's been said plenty of times before, it's basic:
For other AV, there is only "Known Bad" and "Not Known Bad", and anything that is the latter has 100% free reign to do stuff, hide things, and get away with anything it wants to. Then even if it becomes "Known Bad" in a day or two with a def update, there is a good chance some of the stuff it hid and did would be passed over. That is the problem there. Also, heuristics still look at things that are actually good because it has no idea, which has performance impact.

For WSA, it's "Bad", "Unknown", and "Good". Good things are left alone completely as long as they stay good, so no performance impact. That's -all- good things, not just the good things that are commonly used to judge performance impact in testing. Bad things are whacked of course. Unknown things - which is what all completely new threats come in as -are watched and logged and that means if they bring in other unknown things before they are detected as bad, those other things are removed as well when the determination is made. This is completely unlike other AV, who have no idea what the threat poked at while it was not detected as bad.

It's sad to a degree. The testing organizations have fallen behind because they are testing test scenarios. All it takes is seeing the reality and testing the reality to know what's up, and this reviewer does that.

 

Just to aid my understanding are you saying that these products were not tested at 'Default'/out of the box settings, but rather that settings were modified for each?

 

I think you are wrong in your interpretation of what 'roll-back' does/has an impact on. It has no impact on detection or prevention. Webroot have never denied that a malicious threat it classifies as 'Good' or 'Unknown' could theoretically capture, and send your banking details externally. The fact that if the malicious file that was classed as 'Good' or 'Unknown' has it's status changed by WSA to 'Bad', then WSA attempts to 'roll-back' the file's actions, i think is fairly irrelevant when thinking about prevention and detection. The or some damage (data capture) *may* already have been done.

Why there is so much voodoo talk about WSA as if it is performing some kind of magic i don't know. WSA is a decent AV, i have recommended it to others for it's lightness, speed and decent performance. But I have no illusions that if it's detection and prevention test performance slips, 'roll-back' aint going to make a difference. We all know what 'system compromised' means, and if the test methodology explains what the definition of system compromise is for a particular test and WSA gets 'compromised', why do we make excuses and hold up 'roll-back' as if it mysteriously unlocks time travel and puts us back to before the infection took place? It won't. I don't think WSA claim it will - they can't if the AV has been beaten by a new banking trojan which sent your login detail externally during a system compromise - WSA cannot time travel that data leak back into the box.

That said, i acknowledge that a roll-back feature could be great from a repair viewpoint, not that i think this feature has ever been tested by a 3rd party - how successful is it in doing what it is claimed to do?

 

I just find it odd that WSA in static detection in PCMAG's eyes detected 100%, an in the same article they post independent tests where it only got 66%. I also find it interesting they only show the 100% result and count only the 100% result and hide the 66% result in the page where people would not look.

 

100% is broadly consistent with the results here:
http://www.av-test.org/no_cache/en/tests/test-reports/?tx_avtestreports_pi1%5Breport_no%5D=130566

Why are you surprised about the results?

 

Roll back data alone will be useless against malware, you are right. However, Roll back is combined with the identity shield in all WSA products. Unknown/malware/known good will simply fail to grab data and leak it. There is no known malware able to escape to it.

 

Seriously? Come on faxy, play fair! Maybe except the three malware simulators in this MRG test from last month that WSA failed to stop "the simulator from capturing and sending the logon data to the MRG results page or local store location." (Read the test methodology, in particular the definition of a 'test failure'. Data was grabbed and leaked in this test. By test system running WSA).

You (fax) even started a thread on the issue at the time!

WSA subject to three new browser extension attacks
https://www.wilderssecurity.com/showthread.php?t=345421

MRG Effitas Project 32 March 2013
http://www.mrg-effitas.com/wp-content/uploads/2013/04/Comparative-Efficacy-Assessment-of-Wontok-SafeCentral.pdf

I like WSA. It is a very decent AV. Your statement "There is no known malware able to escape to it." though is untrue.

 

Detection of widespread and prevalent malware discovered in the last 4 weeks (the AV-TEST reference set) 100%
Protection against 0-day malware attacks, inclusive of web and e-mail threats (Real-World Testing) 95%

This means it takes 4 weeks in this test for Webroot to score 100%. Compair the score to Bitdefender in the same test.

Protection against 0-day malware attacks, inclusive of web and e-mail threats (Real-World Testing) 100% *BitDefender*

It does not make sense and in no test other then this one Webroot scored 100% over everything else.

I have seen so much Malware get past its identity shields. I was actually surprised how much Malware does get past.

 

Please mind reading what I post? There is no known malware capable of grabbing data. Those are simulators, that will be added to the protection as soon they will give them to WSA developers.

You arguments are simply flawed as you seem not understanding how WSA works (or pretend to).
WSA is designed to prevent the data leak otherwise the roll back for certain malware (financial related) is pretty useless

 

Interesting selective quotation! LoL
Protection against 0-day malware attacks, inclusive of web and e-mail threats (Real-World Testing) 97% (February)

And no it does not take 4 weeks to detect 100% as already it detects the 97% of 0-day.

I don't know another reputable testing organisation other than AV-C and AV-TEST ORG. Sorry

 

I was using the lowest score for both, Bitdefenders lowest was 100%. In my own testing and from what has been done by others Webroot is one of the slowest in 0 day Malware detection. Even if you use the highest 97% for Webroot its still lower then Bitdefender by 3% and thus the whole point of my argument that the test from PCMAG saying 100% on Webroot and nothing else was skewed.

 

Difference is minimal in %... and them main point here was to mention that its not unusual for WSA to have very high detection rate. I guess that the sample used was not favourable to Bitdefender in this specific case. Just bad luck.

You have to judge software no just on one test but on several ones and even better over time (e.g. 6 months), just review the results for Bitdefender over time at AV-test org, it is not always 100%. This would help to draw more solid conclusions.

...and small % difference from one product to another do not make one product better than another. Big variations of course do. But its not the case here as both products provides excellent protection and both were chosen as editor's choice.

 

If we all use the search function on this forum i think we will discover there have been several threads discussed on the pros and cons of WSA.

Is there really any point in harping on about it again.
Ok the loyal illuminati may enjoy this waltz down memory lane but im finding it repetitive and downright boring.

 

(or the Shadow Government) It's a slow security day!!!

 

For the naysayers how come we don't see any infection complaints in the forums? Do you guys even look at other AV forums, have a look and learn.

TH

 

TH, very good point. I don't see posts saying that WSA missed xxx Trojan and my computer is screwed. I have seen posts like that for other famous products here. So, he has a point. If it didn't protect, then who would know better then their customers. WSA works just fine. I don't care for PCMag but the true reality is WSA, not the magazine.

WSA works, plain and simple.

 

TH: Remember that somehow WSA -hides- the infections so people can't find out they have them. ;P

No, but seriously, in personal work, I have removed infections from computers with everything (even BitDefender's 100% coverage) -except- Webroot systems. I'll admit, I've removed fewer from some things. More removed from Norton than from BD for example. Very few from Vipre. None from Webroot. I do get other business from Webroot customers and users, so I know they exist.

 

Very true Techfox there hiding under the covers.

TH

 

the only thing important is :

"Have been infected while using WSA during a safe usage of your computer, Yes or No?"

- If No, no need to look for something else, keep and enjoy it.
- If Yes, How? do you feel unsafe with WSA? if yes again, you should maybe use something else.

 

I was not saying the product does not work. I am saying that PCMAG saying it got a 100% in signature file detection is wrong. I have never seen anywhere where it got 100% in signatures other then this test. WSA's strengths are not in its signatures which is why its weird that's the result it got, over other "Heavily" signature based programs.

 

You cannot say its wrong, it depends on the type and number of files been detected. If you don't know how the test was performed you can't say its wrong. Sorry

 

If you get infected during use of your computer, it wasn't safe usage by definition.

Technically, "Safe" use is a subjective term with no agreed-upon bright line. That "feeling safe" factor is completely true though. That's why all those statistics are on the front page of the UI. To help people see that it was doing something and help them feel safe. For the average customer, it worked too. Kind of freaky, innit?