AV-Comparatives detection test 2012

[darts]

Originally Posted by PrevxHelp

I actually just asked the same of our internal research team as I was surprised to see us scoring low on there as well. They're expecting this to improve quickly also and come more in line with what we've seen from other tests.

Ok i hope also because i got a 1 year license so i hope that soon it will be solved and all the test will be positive with a good result.
But it is strange that in 1 test Webroot scores 98.2% but with lots of false positives and in the monthly result from the same test it scores at about 90% .

Greats,

darts

[STV0726]

So without the false positives, they would have scored Advanced+?

I would normally say "Tough. You scored what you scored. Fix it." Except for the fact that I use Maximized heuristics across the board on all my machines and could count the number of FPs I've had probably on one hand for the last 2.5 years.

This is why AV-C makes that statement that if you are an advanced user you can ignore the FP considerations.

Still, I'm not thrilled...

We're rapidly approaching mid-2012 and if by then the scores aren't flying colors, then that cute, energetic lady named Cat (who I'll admit is very attractive...who knew...security companies have women!) who says Webroot is "revolutionizing the security world" is going to seem a bit like a joke.

[kdcdq]

Originally Posted by PrevxHelp

I'll pose a question to everyone here: have you been experiencing vast quantities of false positives? ]

Great question. I have been running WSA Essentials on three systems for about a month now, and I have had ONE FP: from MBAM Pro. NO FPs from WSA; zero. There; I said it....

[TonyW]

Originally Posted by PrevxHelp

none of these were on popular programs or bound to affect any large number of users as they were all relatively obscure from what I've seen.

This can be said for the other vendors where FPs were reported. I've hardly heard of many of the packages mentioned let alone have them installed. And all the vendors will have those FPs fixed already anyway.

[TonyW]

Originally Posted by PrevxHelp

IBK, I saw in the report that Avast, Avira, and Kaspersky had their heuristics set to High for the test. Could you let me know if Webroot's were also elevated?

It'd be good to know IBK's answer to this.

[Triple Helix]

Originally Posted by TonyW

It'd be good to know IBK's answer to this.

He said yes in post #20!

TH

[TonyW]

I didn't read that as yes to the elevation question; I read it as an affirmation IBK expects the FPs rate to be fewer next time.

[Triple Helix]

Originally Posted by TonyW

I didn't read that as yes to the elevation question; I read it as an affirmation IBK expects the FPs rate to be fewer next time.

I didn't say it was clear maybe IBK can expand on that?

TH

[IBK]

not elevated.
@PrevxHelp: if you have further questions, please ask your colleagues at Webroot, as it was already all communicated/discussed weeks ago.

[Legendkiller]

See my POV is pretty clear:
1.WSA is revolutionary approach to a Security Suite and many may follow suite.
2.As official prevx support guy has said not all FP maybe of good reputed companies and not everyone gets affected.

But, bottom line is that ir-respective of build version , this was a officially released product and stability/maturity of a product could be used only if it depends on its impact on system w.r.t cpu usage/compaitibilty with other programs, not with detection of FP's.

[Techfox1976]

(Just as one of those *cough cough* moments... The "support guy" here (PrevxHelp) just happens to be a Vice President at Webroot... if his colleagues didn't forward what was apparently discussed weeks ago, somebody's probably gonna be in trouble. )

[TonyW]

Originally Posted by Techfox1976

The "support guy" here (PrevxHelp) just happens to be a Vice President at Webroot...

Might be wrong here, but I don't think he is a Vice President at the company; he is, however, most certainly the Senior Software Engineer for the product.

[PrevxHelp]

Originally Posted by TonyW

Might be wrong here, but I don't think he is a Vice President at the company; he is, however, most certainly the Senior Software Engineer for the product.

Yes I am - VP Endpoint Solutions Engineering

[TonyW]

Originally Posted by PrevxHelp

Yes I am - VP Endpoint Solutions Engineering

I do apologise. Couldn't see that listed on the website, but knew you were lead software engineer though.

[justenough]

I'm more an average user than knowledgeable like a computer tech, so I look at false positive rates first and then detection rates second in order to avoid sorting out problems in other programs caused by fps. Maybe this AV-C test is unfair somehow, but I'm removing WSA-E for now. I like the program and will keep track of this thread and other tests, and will reinstall WSA-E when results look better.

edit: After doing some reading and thinking, changed my mind about removing WSA-E, see post #51.

[Triple Helix]

Originally Posted by PrevxHelp

Yes I am - VP Endpoint Solutions Engineering

Congratulations Joe!

Daniel

[Triple Helix]

Originally Posted by TonyW

I do apologise. Couldn't see that listed on the website, but knew you were lead software engineer though.

Have a look in the Webroot section "Webroot: Beware Cross-OS Infection": http://securitywatch.pcmag.com/none/...on-experts-say

TH

[STV0726]

At least Webroot treats a less than favorable test score as an opportunity to improve...

Some companies *cough* MSE take it as...well not sure why they even participate in tests when they have MCCs on their forums making claims that AV-Comparatives and AV-TEST are not effective ways to review AVs and that MSE is inevitably the best. Give me a break.

[TonyW]

Originally Posted by justenough

I'm more an average user than knowledgeable like a computer tech, so I look at false positive rates first and then detection rates second in order to avoid sorting out problems in other programs caused by fps.Maybe this AV-C test is unfair somehow, but I'm removing WSA-E for now.

If the FP rate is an issue for you, I suggest what you really should be asking is "has this product produced FPs for me?". If you've experienced a copious number of FPs as apparently seen in this test, then I agree, there is a problem which needs looking at, but if you've only seen few FPs or none at all then it's no reason to ditch WSA-E just because of one apparent poor FP test result. Remember that test was done with build 8.0.1.95 and we're told the issues have long since been corrected. We're now at v8.0.1.165 and the product continues to evolve.

[Techfox1976]

Originally Posted by TonyW

If the FP rate is an issue for you, I suggest what you really should be asking is "has this product produced FPs for me?".

I saw a weird set of FPs around the time that test was run. Support indicated it was a hiccup in the system that was corrected really fast. Sure enough, in the time it took me to reach support and be told to restore everything that was detected that day (nothing was removed, since I knew they were FPs), then rescan, the issue was fixed. No waiting for the next day or even a dozen hours for a definition fix. I haven't seen any similar problems since.

[Thankful]

A problem for Webroot is the coming Retrospective Test will use the March 2012 False Positive Test results and will again be penalized for the high number of FPs.

[Technical]

Originally Posted by IBK

for Webroot quite some were not "obscure", listed here:
http://www.av-comparatives.org/image...fp_mar2012.pdf

Thanks IBK.

[justenough]

Originally Posted by TonyW

If the FP rate is an issue for you, I suggest what you really should be asking is "has this product produced FPs for me?". If you've experienced a copious number of FPs as apparently seen in this test, then I agree, there is a problem which needs looking at, but if you've only seen few FPs or none at all then it's no reason to ditch WSA-E just because of one apparent poor FP test result. Remember that test was done with build 8.0.1.95 and we're told the issues have long since been corrected. We're now at v8.0.1.165 and the product continues to evolve.

TonyW, your question might be 'Has this product produced FPs for me?', but my question is 'How does this product compare to others as to the likelihood of getting a FP?'

edit: TonyW, you made a good point, sorry if my reply seemed curt. WSA-E is back on my computer.

[IBK]

@Thankful: Webroot will not be included in that test, as we offered an opt-out at begin of the year. It would anyway not make sense to test a pure-cloud product like Webroot in that test, as it would not detect any of the samples (and also zero FPs) if there is no connection to the cloud. This year that test includes also a behavioral part where remaining samples are executed (to evaluate e.g. the behavior-blocker etc.).

[STV0726]

WSA isn't really a pure cloud product, not at least by my understanding.

It is also an advanced behavior blocker, especially in offline mode.

As long as samples are executed and we're not just talking about right-click scanning a folder full of dormant malware, WSA should catch it fine.