WSA Poor Detection Result

Discussion in 'Prevx Releases' started by james246, Sep 18, 2013.

Thread Status:
Not open for further replies.
  1. james246

    james246 Registered Member

    Joined:
    Nov 5, 2005
    Posts:
    80
    A poor virus detection result for WSA (latest AV Test Org report).
    In past detection tests WSA used to score Five and a Half, now it is only Two and a Half.
    Either WSA is now completely missing more zero day threats, or these threats are being allowed to run in monitor mode for longer before being fully dealt with, and AV Test Org is by its own testing methodology giving WSA more fails. Whatever WSA seems to be much less efficient these days at (completely / quickly) eradicating zero day malware.
     
  2. MarcP

    MarcP Registered Member

    Joined:
    Jun 9, 2009
    Posts:
    708
    Got a link to that latest report?
     
  3. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,727
    Location:
    localhost
  4. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,731
    Location:
    New York City
    Still too many FPs.
    Webroot placing its faith in its rollback feature.
     
  5. pegas

    pegas Registered Member

    Joined:
    May 22, 2008
    Posts:
    2,016
    You might be also interested in reading this thread from the Webroot community ... quite long but giving explanations what lies behind Webroot's results in tests.
     
  6. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,731
    Location:
    New York City
    I don't know why WSA still participates in the AV-Test, unless this decision is out of their hands. The results don't make WSA look good.
     
  7. pegas

    pegas Registered Member

    Joined:
    May 22, 2008
    Posts:
    2,016
    Hopefully it sparkles for better times, have a look in this thread, especially post made by Grayson (the first on the third page).
     
  8. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    We're working very closely with AV-Test/AV-C on new testing methodology which better reflects the many aspects of WSA which work very differently from other products (not just rollback).

    I believe we'll be posting a more detailed response shortly with more specifics - as soon as I see it posted (or if someone else catches it before me), I'll link it in here.
     
  9. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    Sounds great Joe, looking forward to it, thanks for letting us know in advance :thumb:
     
  10. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    honestly Joe, I don't buy that idea of detecting by a different way. You either detect or you don't ~Phrase removed~

    Bring Prevx back.
     
    Last edited by a moderator: Sep 19, 2013
  11. PrevxHelp

    PrevxHelp Former Prevx Moderator

    Joined:
    Sep 14, 2008
    Posts:
    8,242
    Location:
    USA/UK
    That's actually not the case - there are many cases where a threat does nothing or is prevented from doing anything, which is not taken into account in these tests.

    As you know, WSA is effectively Prevx 4 (and considerably more), and 100% of Prevx 3 is included in it. Prevx 3 would score abysmally in the recent AV-Test/AV-Comparatives tests despite actually blocking threats and actually protecting the user. This is exactly why we're working with the testing firms on assessing WSA as accurately as possible. The reason for the declining scores over time is because of a changing malware landscape, moving from being blocked by signatures to being blocked by other methods.
     
  12. webbit

    webbit Registered Member

    Joined:
    Nov 2, 2008
    Posts:
    222
    we really need something concrete as to why the results are so poor, this has been rumbling on now for a while
     
  13. Dermot7

    Dermot7 Registered Member

    Joined:
    Dec 20, 2009
    Posts:
    3,196
    Location:
    Surrey, England.
    Well from what I've seen, this same query/discussion keeps getting regurgitated and analysed, and yes it appears WSA gets mixed test results in recent times, but the proof of the pudding is that in real-world everyday usage, people are overall remaining infection free if using WSA.
    https://www.wilderssecurity.com/showpost.php?p=2261339&postcount=10
     
  14. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,634
    Location:
    UK
    But at the same time, Dermot, many people out there are still not grasping the concept of how WSA works, including some of the testing organisations whom the company is now actively working with.

    A lot of users understand the traditional blacklist/heuristic approach because that's been accepted for so long, and have yet to fully appreciate "the many aspects of WSA which work very differently from other products (not just rollback)." Webroot certainly have a hard task in convincing some that their new method works.

    As to real-world usage, I agree with what you say, but then you could say that about other products. I've remained infection free with WSA, but that's also true with others I've tested. Admittedly, testing organisations do test against samples you or I are hopefully never likely to come across in everyday usage so that's why that point stands.
     
  15. Windows_Security

    Windows_Security Registered Member

    Joined:
    Mar 2, 2013
    Posts:
    3,068
    Location:
    Netherlands
    I get occasionally zerodays of a friend's (malware reverse engineer for a bank) honeypot. With the few tweaks I have applied I allways get a 100% detection score.

    My Tweaks
    This is based on increased whitelisting for risky sources, e.g. internet (set popularity heuristics to high), setting internet facing programs manually to untrusted (keeps them in LUA type of sandbox), increasing identity protection to denying change of browser for normal HTTP (raising default). Last tweak (firewall warning for untrusted programs) does not make a differences in these tests.
     
  16. kjdemuth

    kjdemuth Registered Member

    Joined:
    Jul 29, 2005
    Posts:
    2,960
    Location:
    Boston, MA
    Kees what are you using for the heuristics rating, before or after age/Popularity?
     
  17. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,363
    Location:
    Sweden
    Also, when you mean keeping internet facing programs set to untrusted, do you mean set to "monitor" in Control Active Processes?
     
  18. sturgess

    sturgess Registered Member

    Joined:
    Aug 24, 2011
    Posts:
    158
    Just curious, mostly these tests use Windows 7 machines, Windows 8 is supposedly more secure. I use Windows 8 and WSA so I have to assume I'm better protected than Windows 7 users, correct ?
     
  19. Techfox1976

    Techfox1976 Registered Member

    Joined:
    Jul 22, 2010
    Posts:
    749
    The big problem with the tests (and the folks who scream that they are right) is that they still run based on the premise of "If it's not detected immediately, it won't be detected in a useful time frame" and "If the threat code runs, it's game over".

    With other AV, this is absolutely the case. If you scan and it doesn't detect, it will take an average of two weeks to be detected. If the malware code runs, the computer is considered 100% compromised. All because the other AV only knows "Bad" and "Everything else", and "Everything else" is Good or Unknown (bad) or Unknown (good), but gets free reign, all its stuff done, and no further inspection except to check it against the bad list on occasion and do a generic cleanup in the event that it's detected finally.

    With Webroot, sandboxing, shielding (Especially ID), ongoing cloud inspection, live heuristics*, and journaling mean that neither of those assumptions is correct. If it's not detected immediately, it can definitely be detected, for example, 30 seconds or fifteen minutes or even two hours later based on actions taken while running. Even if it runs, it is known to be Unknown (as opposed to known Good), so it's poked, prodded, watched, segregated, and prevented from doing a lot of things, and its (sometimes only attempted) actions are sent to the cloud for examination. So it does not have free reign and in real life cases, it does not get its stuff done at all and ends up removed 100% cleanly due to journaling.

    For brevity and time efficiency, the tests make some assumptions of fact that hold true for other AV, but simply do not for Webroot SecureAnywhere.

    (* Standard heuristics involve looking at the code or semi-executing it to see what certain portions do. This is all based in a fragment of a second during a scan. Live Heuristics involve looking at what the Unknown is doing when it is actually running fully. Oh, it waited fifteen minutes, connected to a site in another country, and downloaded another chunk of code to run into memorry without making a file for it? The heuristics in normal AV would never catch it because the site was stored as a little-endian 32-bit integer? We know that's bad to connect there. We know it's bad to download machine code into RAM (where it would evade other AV, not being a file), and we know that machine code has just tried to get to your stored passwords in Firefox. We blocked it from getting the stored passwords, and now we whack the whole program as Bad. For another AV, this would require a threat researcher to look at the program for over fifteen minutes and see what it does, then make a definition, and finally get that definition to you. Your Firefox passwords are long-gone by then and the master key for them has been rainbow-tabled perhaps, assuming you even protected them with a master key.)
     
  20. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,727
    Location:
    localhost
    As usual, excellent explanation! :thumb:
    Yes, its hard to get this to the users. They are simply not prepared for this, especially the Gurus that are used to the "business as usual" security landscape. ;)
     
  21. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    How does this differ from KAV's System Watcher and Bit Defender's Active Virus Control? Both of which claim to monitor the behavior of all active processes on your PC in real time.
     
  22. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,363
    Location:
    Sweden
    Neither can revert back all actions done by the process, right?
     
  23. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    1,955
    Location:
    DC Metro Area
    I believe that Kaspersky claims that it does.
     
  24. Cutting_Edgetech

    Cutting_Edgetech Registered Member

    Joined:
    Mar 30, 2006
    Posts:
    4,950
    Location:
    USA
    Are you referring to something similar as to allowing the code to run sandboxed while it is being analyzed to determine whether it is malicious, and then removal if determined to be a threat? Since the code was not allowed to modify any important system components WSA should get a pass or given credit for blocking the threat even if it did not initially detect it?
     
  25. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    so the malware is sandboxed or isolated and allowed to do its thing until it is determined to be a threat. So it still gets to do the nasty, until at some point off in the future it is determined to be a threat. Then it is deleted. But it has already done its damage.

    Sorry, but this sounds rather weird or yet better, an excuse, for not determining it was malware to begin with.
     
Thread Status:
Not open for further replies.