MRG Effitas 360 Assessment & Certification - Q2 2016

Discussion in 'other anti-virus software' started by LagerX, Sep 9, 2016.

  1. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    Yes, I read that.

    On the note on Webroot. The fanboyism is strong in this one. Note that I use Webroot myself on three of my machines.

    Even if Webroot DOES 'monitor' the threat which it didn't detect in the first place, that is no guarantee Webroot is totally isolating the malicious process. It might very well leak out sensitive information before it gets detected (in this test within 24 hours) and the changes it have done are 'journaled' back. I'm an avid Webroot user myself, but that is the weak part about WSA. It's always better to detect and block without letting the process run. If I had known I had a malicious file running for 24 hours, even if Webroot would eventually detect it, the harm would be done. I'd have to change all passwords, contact with bank etc because I just don't know what Webroot has let the malicious file do (even if the devs behind WSA claim no harm was done).

    On top of that, even WD did better, and that is unacceptable to me as a paying Webroot customer. I expect top notch protection and not just 'words'. Webroot should always perform better than the baseline AV, Windows Defender. It didn't do that now but I hope it will someday.
     
  2. LagerX

    LagerX Registered Member

    Joined:
    Apr 16, 2008
    Posts:
    565
    I agree with Muddy3 and ProTruckDriver.

    I've also set up
    Hi!

    I agree with you. But, monitored file should be kept away from private information as far as I'm aware. Not 100% sure tho.
    Anyways, Identity Shield has been flawless in many many financial malware tests (if not all?). That means even if your PC is infected, protected applications' info is not leaked. At least I hope that's the case.
     
  3. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    On your first point: (“fanboyism”) I personally think it’s unhelpful to make ad hominem remarks which I question whether are ever helpful in advancing the developing argument. I think it’s very important to deal with the arguments and address those only. Having said that, I have all too often seen myself falling into the same trap and so I have to always watch out for that one.

    My experience, as I have often said, is that I used to have a lot of frustration with AVs that were household names and seemed to excel in all the tests but which frankly dismally failed to protect me. I started using Prevx in 2006 and transitioned to Webroot after it acquired the company Prevx and replaced its own cybersecurity technology with Prevx’s. Since 2006, I can honestly say I have not once knowingly been infected.

    That experience obviously colours my views.

    Having said that, I’m not sure I agree with you that “it’s always better to detect and block without letting the process run”. I personally don’t want my AV to delete a process before it’s sure that it really is malware. It may be a perfectly valid process. Your counter-argument is that you can’t necessarily trust the devs that say your data is fully protected so better to go ahead and delete it immediately. As an aside, if that is your view, I would strongly advise you to ditch Webroot from the three computers you have it on. However my argument is that because, unlike other reputable AVs, Prevx/Webroot has fully protected me in all the areas that I HAVE been able to observe (and that is now 10 years and counting), I have decided that I am prepared to give them my trust in the areas I CAN’T observe. However I admit I can’t in any way prove a negative, i.e. that sensitive data hasn’t slipped out, any more than you can prove that because Antivirus XYZ performs best in a given test that shows you can trust that antivirus to protect you more.

    What I can also say is that I AM aware of leaks of my personal data having taken place in other areas (in the cloud) with Dropbox (very recent news item, and my account was unfortunately one of those 8 million) and with another well-known internet firm; however I am NOT aware to the best of my knowledge of any leaks having taken place from my local disk security information (passwords, credit card details) since 2006.

    I’m also curious that you mention bank details, and also that Windows Defender did better than Webroot. I’m surprised. If you look at the financial malware test, WD failed after 24 hours. Over the same period, Webroot did not have one single failure.

    I expect this post will get some reaction. I just very much hope the reactions will address the arguments advanced (ad rem) rather than the person making them (ad hominem). Particularly as I find this an interesting discussion :) and would not want to feel the need to withdraw due to any lowering of the tone of argument.

    And I also hope this post is not going too much off topic :(?
     
  4. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    Certainly not a reply on my post with the colors of 'fanboyism'. I thank you for your objective point of view!

    If you combine all artificial tests, Kaspersky has close to 0.01% (if not less) false positives (I take that vendor as an example since it was the only vendor passing with flying colors). If you combine that fact with the fact that it blocks 99.7-100% of threats immediately it sure is a good probability your data will never leak since the malicious file isn't allowed to run in the first place. That is not the case with Webroot. (these numbers are not on-topic since they are also based on other test labs' numbers). Your argument that you don't want your AV to delete a suspicious file falls because there are AVs today that can distinguish extremely well between good or bad, at least statistically, and block the threat accordingly, without causing problems with false positives. This is what I pay for! A top-of-the-line protection.

    Webroot has done well historically, and to be honest, it's still my top choice. I have 5-6 years of licenses left in my vault that I'm just waiting to use. It's light (I play a lot of PC games), it DOES have a good sandbox (journaling/monitoring) and it's relatively cheap when on sale. But compared to the free, baseline WD it's not THAT much better in this test. I seriously think Webroot (and a few other vendors, but I'm not their customer) need to step up their game. I'm a paying customer of Webroot and I expect them to protect me the best, not any other vendor. On the other hand, there is a thread here that state the fact that WD is becoming too strong and the other vendors have problems keeping up. I suppose this is good for everyone!

    On a side note, I've been a customer of Prevx since 2007 and bought licenses from them and Webroot ever since! :)
     
  5. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    Just a quick reaction:
    Well, yes and no. In terms of user experience, from anecdotal evidence and personal experience yes it has done outstandingly well. In terms of tests, on the whole (there are notable exceptions) it has done pretty appallingly.
    I'm no cybersecurity specialist but I find that difficult to believe. More instant detection must surely make that detection more prone to potential FPs? I'll leave those who are more competent than me to comment on that one.

    Coming back to my first point, what has intrigued me down the years is the often very wide discrepancy between AV tests and people's real life experience of those AVs. For example, in the past I was greatly disappointed by an AV of great repute and that had always received top-notch test rankings. Equally, I have been continually intrigued by the total lack of correlation between users' experience* (including mine of course) of Webroot SecureAnywhere and the pretty dismal way it has fared in most tests.

    Such are my Saturday evening reflections...

    *Compare for example the number of reports of infections coming into the Webroot forum with some (many) other AV forums
     
  6. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    6,061
    Location:
    DC Metro Area
    FWIW: PC Mag only uses malicious URLs provided by MRG Effitas. The malware samples used is the Review Editor's own collection

    I don't know how he collects them, but security products that do well in AV Comparatives, and tests by other organizations, often do not perform so well in blocking The Editor's sample collection..

    http://www.pcmag.com/article2/0,2817,2382605,00.asp
     
  7. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    5,614
    Location:
    Milan and Seoul
    Not to be infected since 2006 should be a good reason to stick with Webroot. Although that doesn't mean that Webroot is better or worse than other products. I have used 4 different companies since 2005, and the reason I switched had nothing to do with infections: One was too heavy, the second one black listed my license as I moved to a different country, the third stopped having excellent results with AV Comparatives, and my current choice so far hasn't given me a reason to switch.

    One of the reasons I have never even trialed Webroot, is the lack of consistency in tests and the fact that certain aspects of its capabilities have to be somewhat tested under specific environments that most test companies are unwilling to provide. I have no doubt it is a top application, too many savvy Wilders members can vouch for it, but lack of infections is not the only criterion for judgement. I believe I have never really been infected not only because of the AV, but the positive complicity of my layered defense and my judgement might have all contributed to avoid infection from any potential malware.
     
    Last edited: Sep 10, 2016
  8. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    What stands out for me is the contrast between regular-ish infections in the past (admittedly this was only a few AVs, I didn't try every AV on the market!) and no infections since moving to Prevx/Webroot.
     
  9. Baldrick

    Baldrick Registered Member

    Joined:
    May 11, 2002
    Posts:
    2,674
    Location:
    South Wales, UK
    Hi shadek / Hi Muddy3

    Have read your reparti with interest and congratulate you both on a useful & dare I say it 'proefessional' discussion on the topic...nice to see here as WIlders can be somewhat unruly at times re. these sorts of debates.

    I have to say that having tried most of the big names over the years (and which I regularly try again occassionally at the time of a new release) I am on Muddy3's side of the discussion (not that I wish to establish 'sides' per se) based on anecdotal & personal experience of WSA. Having said that I believe that the rise of WD should not be used as a benchmark as to whether WSA or any other AV/IS/AM application provides value or not.

    Granted WD was something of a joke with even MS stating that it was only a 'temporary' solution until the user supplanted it with a 3rd party applicaiton...but to MS's credit they have continued to develop it and whilst it may be, in mu opinion, 'clunky' when compared to WSA, etc., it is great to see that like the Windows Firewall before it, it has been developing into a serious piece of 'kit'. Not that I would use it, as it still does not give the all round protection that WSA does, but it is definitively a move in the right direction.

    As for WSA and the recent MRG test I feel that there is a flaw in their testing methodology; I noted "In the 360 Assessments, trojans, backdoors, ransomware, PUAs, financial malware and “other” malware are used".

    In my view MRG should be aware of the positioning of PUAs/PUPs and of the contention that resides generally as to what can/should be classified as one or not...the debate has raged in forums at points in the past, and therefore to truly represent the position properly they should have more clearly segregated PUAs/PUPs from the 'real' malware categories and therefore allowed the 'punters' to form their own opinion based on that separation.

    We all know that Webroot take a more conservative & cautious view as to what is a PUP/PUA then say MBAM (and this is not a criticism leveled at Malwarebytes, I hasten to add) given the potential legal & commercial ramifications of 'mis-labelling', etc.

    So I feel it is almost disingenuous of MRG to have approached the test in the way they have, i.e., lumpy PUPs/PUAs in with malware, and that is what lowers them & it in my estimation at present...I hope that they rectify this quickly (and I may well try getting into contact with them directly to point this 'flaw' out them).

    Just my thoughts on the topic...for what they are worth.

    Regards, Baldrick
     
  10. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    Yes, this was a point that Cache raised earlier in this thread (and that, until he mentioned it, I had competely failed to notice):
    In reply, I had the same reaction as you, i.e. that they should show these PUAs as a separate category. However it's a very good thing that you brought this up again as this point got rather lost, indeed practically forgotten, in the ensuing discussion.

    The other point is this: if any of the samples missed by Webroot were not PUAs (but maybe they were all indeed PUAs - edit: or maybe none were), then it would be very useful to know whether the remainder were being monitored (knowing Webroot, they almost certainly were). If so, they were effectively:
    That is, until such time as they were judged to be malware, the malware then deleted and the machine journalled back to the state previous to infection, all sensitive data having been meanwhile protected by Identity Shield (as I've said in earlier posts, I do tend to trust Webroot in this, and as Lagerx has pointed out in a previous post in this thread—I had also forgotten this :(—many or maybe even all financial malware tests have shown this function to work "flawlessly". Can anybody elaborate on this: is it "many" or "all"? Or maybe even contradict this? I should also hasten to add that, just because a test shows a function to work flawlessly, doesn't necessarily mean that it actually does work flawlessly. All tests should be taken with a pinch of salt.).

    If you do decide to get in contact with them, it would be good (if it proves practicable) to raise these different points. It would also be very, very good if you could report back here to Wilders with the answers provided (if they are prepared to give them).
     
    Last edited: Sep 11, 2016
  11. Triple Helix

    Triple Helix Specialist

    Joined:
    Nov 20, 2004
    Posts:
    13,262
    Location:
    Ontario, Canada
    Here is what Joe replied to over 4 years ago in a Webroot Threat Blog: https://www.webroot.com/blog/2012/0...garding-av-comparatives-results/#comment-5446
    So this is the best understanding and can post and I would assume WSA has improved upon it over the years.

    Daniel
     
    Last edited: Sep 11, 2016
  12. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    and
    I'd like to discuss this part of the debate and not just in relation to Webroot. Time and again we see people say that while using vendor X they've not been infected. Is this because

    (a) they've not seen any alerts so believe their system has not been infected

    or

    (b) they do see alerts and are able to prevent the infection

    I imagine the "safe surfer" probably would not see many alerts but one who is click-happy might. It's interesting to note with the latter type of user that situation only seems to come to light when running a scan with another product. "Product X didn't detect such and such but oh look product Y did during an on-demand scan." That sort of thing.

    Just bringing to the table the oft-commented statement when a user says he/she has not been infected with product X and trying to understand what that actually means in reality. :)
     
  13. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    Tony:
    A combination of both for me (though probably more of the former)! And a feeling that the machine is clean and responsive as it should be, and is showing none of the typical evidences of infection. Note that in some of my statements "not been infected", I have prudently added the word "knowingly" ;)
    I should also add that for quite some time I used to follow the advice of using from time to time scans with different "Product Y"s to double-check the efficacy of "Product X" (Prevx/Webroot). Because the "Product Y"s never came up with anything, eventually out of laziness I gave up.

    TH:
    Thanks for the post from Joe, explaining neatly and concisely the method Webroot uses. Joe, of course, was the chief architect both of Prevx and of Webroot SecureAnywhere. And the article to which he has added his post is also very interesting and informative. Nothing new of course but it helps me to see things more clearly. :)
     
  14. Triple Helix

    Triple Helix Specialist

    Joined:
    Nov 20, 2004
    Posts:
    13,262
    Location:
    Ontario, Canada
    Your very Welcome: Joe stills does some work for Webroot when they really need him, and the reason I posted the above from Joe is to provide clarity, much has changed in 4 years and we have Ransomware now so Webroot continues to add generic detection and continue to update the Monitoring and Rollback feature in these cases every day! The reason why Webroot releases so many program updates (in a blink of an eye) to keep on top of new emerging threats. Who knows what will be coming next?
     
  15. cavehomme

    cavehomme Registered Member

    Joined:
    May 19, 2010
    Posts:
    137
    Location:
    Alps
    Yes indeed, MS have really been massively improving in the past 6-12 months. During July and August they were ahead of even Bitdefender in my tests of Trojan downloaders embedded as macros in doc and xls files. However, in the past week or two in my tests they've been delayed in updating their signatures and with a negative knock-on impact upon detection.
     
  16. cavehomme

    cavehomme Registered Member

    Joined:
    May 19, 2010
    Posts:
    137
    Location:
    Alps
    As you say Triple Helix the roll-back feature is not perfect and that's my issue with it, in the context of "wait and see for 24-72 hours" for unknown files. What I mean is that of course nothing can be perfect, but when malware is given free reign to do what it likes and then the rollback may fail, then it places the user in a bad situation.

    Possibly related to this is that we have an enhancement request on the Webroot forum concerning blocking the webcam because the user was compromised and blackmailed.

    In my own tests of trojan downloaders in the past few months, Webroot failed to detect them, as you may recall from my posts on the forum. As I have suggested, a simple virustotal look up (not requiring signatures and big exe files) that flags such trojans much earlier then quarantine them would be a better way to deal with suspect malware. If that's not feasible then Webroot could go the way more towards Comodo of "default deny". At least not give suspect malware restricted freedoms and then attempt to rollback, better to block fully and quickly, err on the side of caution, period.

    I've been using Prevx / Webroot on some PCs since at least 2010 and I'd like to use this solution on all my PCs, but this is one of a handful of issues that I struggle with; I'm clearly not alone.
     
  17. Triple Helix

    Triple Helix Specialist

    Joined:
    Nov 20, 2004
    Posts:
    13,262
    Location:
    Ontario, Canada
    The Feature request about the WebCam is just that a Feature request and I know of no issues with WebCam but I can ask. Now with your Malware testing IMO is not real world and most users would not even come across such infections. And the greater Security Communities frown upon Malware testing by users. Webroot and other Security vendors would deconstruct malware in there Labs so I would call them the experts so I would not recommend anyone testing malware it's reckles IMO.
     
  18. Muddy3

    Muddy3 Registered Member

    Joined:
    May 31, 2010
    Posts:
    415
    Location:
    Belgium
    After writing this, I realised that those were the days I was still running another AV (Nod32) alongside Prevx/Webroot*. So I thought it only fair to do a fresh Product Y scan to verify Webroot's efficacy at protecting my machines as a standalone AV. I chose Kaspersky "Virus Removal Tool" as my Product Y and scanned what has been my main machine until just a few weeks ago. As expected, it found absolutely zilch :D.

    *Many of us original subscribers to Prevx had found it difficult to shed our habits from the earliest and youngest days of Prevx when they were suggesting that one run it alongside one's main AV with the aim of using it to catch those malwares that were missed by the primary AV.
     
    Last edited: Sep 12, 2016
  19. Mover

    Mover Registered Member

    Joined:
    Oct 1, 2005
    Posts:
    180
    Notice the yellow 'block in 24 hr' in all results on page 9 of the paper.

    I'll use Kaspersky as an example (2%); if an infection wasn't 'auto blocked' or 'behavior blocked' prior to the 24 scan, what was it doing on the system ?
     
  20. cavehomme

    cavehomme Registered Member

    Joined:
    May 19, 2010
    Posts:
    137
    Location:
    Alps
    Many, even most of you, are far more experienced in IT security matters than I am, at least from an understanding as to how exactly malware works and what mitigates it.
    But ultimately whom can i really trust, apart from my own experiences?

    Context: I've been using PCs since year dot and have tried every AV solution bar the Chinese "copy cats" and "innovations", no offence intended whatsoever to them.

    So I used to utilise Prevx as a bolt-on extra layer, but eventually with Windows 10 it was forced to becoming the ONLY layer. So then I took more notice of Webroot lab tests...

    These MRG results look good, but only if you fully trust that there is NO leak within those 24 hours, AND assuming the PC is rebooted - how may of us leave PCs on for days or weeks? The argument begins to crumble, no?

    So for one moment let me try and trust 100% the Webroot technology that allows all unknown files to run and which promises me that once it's known bad it rolls-back all changes, no screenshots etc are leaked, they promise me....

    So if I accept that at "face value", how can I see today on the Webroot forums that i) a cryptolocker type malware actually ran and damage could not be reversed and b) someone wants a feature request to block webcams because they were "caught with their pants down" WTF o_O

    Seriously guys, there are some major failures here. Open your eyes.

    As a Prevx/WR user for 8-9 years, I want to really "love" this product and shout for it from the rooftops, but these documented issues, plus my own experiences of WR completely ignoring detection of known Trojan Downloaders, unlike many others, who jump on them within minutes, whereas WR lets them roam for weeks and months....come one WR guys, how can I trust WR on their words rather than actions and real-world evidence?!

    Bottom line is that these REAL examples essentially kill your argument, sorry to say...and yes I know that no AV is 100% perfect, far from it, but those strengths that you are selling me are not strong enough. We all know there are REAL holes reported by REAL users.
     
    Last edited: Oct 30, 2016
  21. Triple Helix

    Triple Helix Specialist

    Joined:
    Nov 20, 2004
    Posts:
    13,262
    Location:
    Ontario, Canada
    We already tried to explain to you over a the Webroot Community but sorry you don't understand. Attachments are not malicious Java Script/Macro files again are not bad and many companies use them. Now if you happen to get a Bad one that tries to download a payload then Webroot will react to the Payload as the said Script/Macro itself it not malicious. Now if Webroot Detected any Scripts and Macros and a company or a user wants to run them well that would cause more issue so WSA looks for the ones that try to download a Malicious Payload then reacts. It's nothing new as Joe said many years ago so I can't explain it any better.

    "July 20, 2012 at 11:55 am
    The vast majority of threats are still blocked automatically – just in the case where we don’t block a threat, we have the backup measures in place to continue protecting the system, unlike other security products which do nothing and allow threats to roam free. WSA applies a transparent sandbox which will limit the infections ability to affect the system and will automatically undo its changes even if they are complex and wide reaching. The generic information stealing trojan protection will prevent it from stealing user data or logging keystrokes. So, while our goal is to certainly block everything before it runs, you’re still safe even if we miss it initially.

    Hope that helps!

    Regards,
    -Joe Jaroch"

    And no one product is 100% or everyone would be using it. In my case I would not need any AV as I'm smart enough not to open attachments from people I don't know and a safe surfer and know what I'm doing! I have Office 2016 security to deny unsigned Macro's so again it's security built into Office 2016 and as far as I hear they just added it to Office 2013. https://support.office.com/en-us/ar...ce-files-12b036fd-d140-4e74-b45e-16fed1a7e5c6

    Education is 99.9% of all security in life.

    Daniel
     
  22. cavehomme

    cavehomme Registered Member

    Joined:
    May 19, 2010
    Posts:
    137
    Location:
    Alps
    Daniel, I do understand the Webroot approach, it has been well-explained by you, Joe, Baldrick and others, but I personally don't agree with it enough to to give me as much confidence as I would like. I remain a user though. If the approach was truly effective then we would not see webcam spying and ransomware slipping through since ANYTHING unknown is meant to be isolated and changes rolled back. This means that malware is getting in under the radar screen, or are not being properly isolated, or roll-back not fully effective. Two of those three scenarios can be effectively eliminated by removing suspect malware as soon as is possible; it's this unnecessary risk which some of us don't agree with. Some people advocate using Voodoo Shield in combination with Webroot; this confirms that Webroot is not as effective as it could be and we'd like.
     
  23. Mover

    Mover Registered Member

    Joined:
    Oct 1, 2005
    Posts:
    180
    Would be nice to have the details (ie. PUA, trojans, etc) of what the yellow 'block in 24 hr' consisted of for each security suite understanding that 1% is equivalent to 4 samples.

    I might reach out to MRG to suggest this for their next test.
     
  24. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    I agree!

    Kaspersky and Bitdefender have again and again prevented trojans and other malware to run in the first place. Having 100% scores and 0 false positives over a period of nearly a decade is possible, just look at Kaspersky and Bitdefender. How can I trust Webroot to protect my sensitive data when I can't trust Webroot to stop the malware from running in the first place like the top antimalware-solutions do? How am I supposed to know the malware that was allowed by Webroot to run for a week haven't stolen any data or modified my computer? The only solution is instant detection of the malware to prevent it from running in the first place.

    I do understand the mechanics of Webroot, I've been running it since PrevX 3... but Webroot simply cannot compete anymore when it comes to instant detection of 0-day malware. I'm currently running Webroot on 3 machines but will eventually replace it as the bugs I'm experiencing haven't been fixed by support (tickets been sent and STILL waiting for a release they said would be available in the future) - I'm confident doing this because I know there are other really good antimalware solutions available.
     
  25. Gein

    Gein Registered Member

    Joined:
    Dec 8, 2013
    Posts:
    219
    I think you're all missing the point with Webroot. It's very popular with sysadmins for a reason. It's gotten to the point with many of the av systems I use that I can't really install them on a novice user's pc without worrying about them allowing something through when the av asks them, or trusting them to deal with false positives. It's also decent at remediation which means I don't have to take a trip out to clean their PC as often when their anti-malware system misses something. And yes - they will get infected, no matter what you put on their PC. Afaik they're not doing mitm against a user's https connections, which ESET, Kasperksy, and I believe Bitdefender all do. Avira is bundled with crapware, and Avast/Comodo all pester users with insecure bloatware chrome spinoffs masquerading as "secure" browsers. I think EMSIsoft had an article about that -- something like has the AV industry gone mad?

    The only thing I do not like is the browser extension installation. I'd much prefer webroot find another way to do what it needs to do.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.