Webroot SecureAnywhere 8.0.0.7 AV on-demand testing

Discussion in 'Prevx Betas' started by lordraiden, Sep 4, 2011.

Thread Status:
Not open for further replies.
  1. lordraiden
    Online

    lordraiden Registered Member

    After reading some people commenting the low performance in detection of Webroot SecureAnywhere have decide to test it myself.

    I have tested a total of 3038 (0day and recent malware) files and the results are.
    Webroot SecureAnywhere 1599 52,6%
    Emsisoft Free 2807 92,4%

    Also Webroot SecureAnywhere has failed to delete any of the files (must be a bug)

    There is any reason for this low performance? it's related with the beta?
    Even if the cloud is not 100% ready we should expect a better protection, if not I don't even want to imagine what would happens with the computer offline.
  2. andyman35
    Offline

    andyman35 Registered Member

    According to PrevX Help in another thread,when asked about it's current strength:

    I'd expect it to be much better once it's final and of course it's traditionally been a lot stronger real-time than on-demand.
  3. lordraiden
    Online

    lordraiden Registered Member

    Ok, I will check again with the final version.
    Maybe it's time to include Webroot in AVC
  4. Rivalen
    Offline

    Rivalen Registered Member

    In my thread Joe said it had all the protection of Prevx + all new stuff in WSA. Was Prevx ever tested anywhere? Confusing - 52% is so poor if Prevx is the base.

    Best Regards
  5. PrevxHelp
    Offline

    PrevxHelp Former Prevx Moderator

    Could you please send me a log after running the scan over the folder to report@prevxresearch.com so that I can take a look? That is definitely far lower than what we would expect and it's far lower than private testing being done by 3rd party testers has shown as well.

    Thank you!
  6. shadek
    Offline

    shadek Registered Member

    My tests show something different... this is what I wrote a few days ago;

    "I used to be able to copy ~12.000 fresh malware per day and around 2.000 would remain after WSA did its job. Now a lot more samples are detected by Malware.Generic definitions...leaving around 500 samples left for each batch... is this a co-incident or is it great engineering?"

    Note that most days after that has shown the same result meaning the detection rate for on-demand is around 96%, and a lot of my samples are 0-day. So my tests show something different, I'm not sure why. If I try to run the remaining 4% of the samples, WCA usually detect the file as malicious within seconds after execution via the suspicious behavior. Only a few, most of them rouge software, are passing through protection and are left running wild in OS.
  7. Triple Helix
    Online

    Triple Helix Webroot Product Advisor

    @ PrevxHelp In this case would going in the System Tools Tab>System Control>Control Active Processes can you kill the Rogue's processes?

    TH

    04-09-2011 1-31-17 PM.png
  8. shadek
    Offline

    shadek Registered Member

    Yes, I can easily kill them! :) No worries there. I'm just talking about the detection rates/prevention rates! The intervention rate is probably close to 100% with WCA.

    EDIT: By the way, killing the processes doesn't mean all the files/registry keys the rouge software installed are removed completely!
  9. Triple Helix
    Online

    Triple Helix Webroot Product Advisor

    No I understand as most AV's have trouble with Rogeware and this is where I would like to see WSA improve upon as then we don't have to rely on other scanners to clean up the leftovers! ;)

    TH
  10. PrevxHelp
    Offline

    PrevxHelp Former Prevx Moderator

    Adding the file with Manual Threat Cleanup should remove any registry keys/files created as well if WSA was installed before the infection :)
  11. Triple Helix
    Online

    Triple Helix Webroot Product Advisor

    WoW great to know! Your going to have a big list to give us for us to know all the capabilities of WSA! :D

    TH
  12. shadek
    Offline

    shadek Registered Member

    Thanks for sharing the information, Joe. Based on this new knowledge, I have something on my mind. I'm thinking of doing an extensive on-execution test and share the numbers with devs at Webroot (aka PrevxHelp) and then after that to all here at Wilders. Now that I know how to clean-up missed detections properly it's going to be a lot of fun!

    The test will consist of 0-day malware, rouges, rootkits, etc. I will include detection rates, prevention rates, clean-up rates. MD5s will be provided to the public users here at Wilders while missed samples and all tested samples with MD5 will be provided to the staff behind Webroot. Around 100 samples will suffice for an on-execution test, don't you guys think?

    P.S. We're not talking samples off MDL or malc0de. I have an entirely different malware pool. :)
  13. PrevxHelp
    Offline

    PrevxHelp Former Prevx Moderator

    That sounds fantastic :thumb: It's very similar to what we do internally on a day-to-day basis so it will be great to see what your samples return to get a picture of the malware you're seeing.

    Let me know your results or if you want anything different from my end to help :)
  14. lordraiden
    Online

    lordraiden Registered Member

    I have sent you the log, if you need something else let me know.

    About the testing I must to say that I was testing a full package with script, dlls.... testing just the exe files the result is
    Scan Results: Files Scanned: 2039, Duration: 1m 0s, Malicious Files: 2019

    So pretty good :thumb:

    Anyway Emsisoft was able to detect as malicious most of the dlls, scrips, binary files...

    Tomorrow I will repeat it again with new exe files.

    Capture.PNG
  15. PrevxHelp
    Offline

    PrevxHelp Former Prevx Moderator

    Thanks for the logs - we received them here. I haven't heard of that website but it certainly is interesting :) The on-demand/right click scanner only uses a small local database against scripts/non-executable files because of the possible privacy issues associated with sending documents/PDFs/etc. to the cloud. If a file actually tried to threaten the PC from a script, it would be blocked but we don't try to scan these when out of context.

    DLLs are handled like exes, however, so you should see good detection for those as well.

    Thanks for the testing!

    (A note regarding the "removal not completed" - there are a few cases where the engine will detect if the system is so bad off that it needs a support engineer to assist. When finding 2000+ infections, I'd think the user would be in pretty bad trouble :D This might need to be changed to handle people intentionally doing on-demand tests like this but for the average user, we're trying to make the process as easy as possible for them without potentially breaking applications on their PC)
  16. shadek
    Offline

    shadek Registered Member

    I am pleased that you're delighted with this test and that you support it. I will begin gather varied samples tomorrow! I will also describe the methodology and inclusion/exclusion criteria! So if there are any doubts, one can know for sure how the test was conducted and people will be able to criticize the methods.
  17. Triple Helix
    Online

    Triple Helix Webroot Product Advisor

    Just a heads up! Lets not get into this versus that or the Wilders staff will close the thread as it's not aloud in the AV section! But it will be nice to see a few results! ;)

    TH
  18. shadek
    Offline

    shadek Registered Member

    Of course. The test I'm going to conduct is just to evaluate a beta-product's performance! :)
  19. lordraiden
    Online

    lordraiden Registered Member

    But it would be nice to have a point of reference like Norton or Fsecure
  20. shadek
    Offline

    shadek Registered Member

    I will not do that. Firstly, I am not really interested in testing other products. Secondly, the test is there to test WCA beta's detection/prevention/clean-up capabilities. :)
  21. lordraiden
    Online

    lordraiden Registered Member

    Yes, but if it scores a 60% or 89% or a 95% will be a good result, a bad, normal? what is the point of the test if you can't qualify the result?
  22. andyman35
    Offline

    andyman35 Registered Member

    While comparitive results can be informative,it's my understanding they won't be allowed on Wilders due to A v B unless from a professional organisation.
  23. lordraiden
    Online

    lordraiden Registered Member

    It's not allowed to discuss about which one is better, something totally different from posting the results of 2 AV's or even better, 3.
  24. kero68
    Offline

    kero68 Registered Member

    "Note that most days after that has shown the same result meaning the detection rate for on-demand is around 96%, and a lot of my samples are 0-day. So my tests show something different, I'm not sure why. If I try to run the remaining 4% of the samples, WCA usually detect the file as malicious within seconds after execution via the suspicious behavior. Only a few, most of them rouge software, are passing through protection and are left running wild in OS."
    Sadek, what heurestic settings have you configured to get good results like this?
  25. pykko
    Offline

    pykko Registered Member

Thread Status:
Not open for further replies.