AV-Comparatives - Microsoft-prevalence-based analysis of the File Detection Tests

Discussion in 'other anti-virus software' started by SweX, Aug 10, 2014.

Thread Status:
Not open for further replies.
  1. ance

    ance formerly: fmon

    Joined:
    May 5, 2013
    Posts:
    1,360
    Well said. :thumb:
     
  2. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    And that tells it's better, how? Of course it doesn't have ASLR "problems if it's writen by the same company which makes the OS and doesn't offer a single advanced function like behavior analyzer that might have to work some other way or ASLR doesn't matter for that component.
     
  3. vlk

    vlk AV Expert

    Joined:
    Dec 26, 2002
    Posts:
    621
    The article on insanitybits is completely bogus. The author obviously has no idea how these things work. The problem is that he used the tool on kernel-mode drivers, while neither ASLR nor DEP is supported for kernel-mode drivers in Windows.
     
  4. guest

    guest Guest

    To make it less exploitable. Too bad there isn't much to read about SSL updating so I can't draw a full conclusion out of this topic.

    Aah, finally one statement I expect after so many no-AV discussions to balance things out. Thanks, I'll bring this one up when there's the chance.
     
  5. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    *Sigh*

    This conversation feels dated, doesn't it? Like, why are people comparing which stick will hurt the mammoth the most when some guy over there has invented the rifle?

    Anyways, obviously ASLR and DEP are not the point on the kernel drivers. They don't exist on Windows. My article makes no mention of this because it is a naive basic look into what is packaged in modern "security" software. I removed only one type of false positive, kernel drivers are not one of them.

    That said, let's look at Avast! and AVG using nothing but the basic information in that article. Oh, look, multiple files that *aren't kernel* drivers don't use ASLR.

    OR, my god, do I hope that 'AVG Secure Search_toolbar.dll' injected into the browser isn't a kernel driver :)

    I also do find it a bit funny that "Oh, we use ASLR, that' sjust a whole bunch of kernel drivers we load up" is the response to questions of security. But, you know, that's what all of those complex-but-ultimately-useless features get you, I guess.

    Now, as for ASLR 'not being needed for some components' or some components breaking it. No. Not how it works. There should be no issues with any program using ASLR if it is written by anyone not-completely-incompetent. Basically unless you're hardcoding addresses you're fine, and I think the windows linker even patches binaries at runtime to get around that.

    Kernel drivers can not use ASLR, the concept can not exist in the Windows kernel a this point. The kernel is itself a static image loaded into a special segment of memory.

    Now, it's been a while, but wasn't Avast that one AV company that got fuzzed and then bitched about it and cried foul? I could swear I was laughing about that a little while back... Boy, I sure would trust that company with my systems' security! Just screams Secure Development Lifecycle.

    HMMMMMMMM.

    So, the question, is the stick known as MSE better than the sticks known as Avast and/or AVG?

    I don't think it matters. I think they're fun to look at as ways into a system, because if you look at them for protection purposes it's quite dull. I think that says it all. And I think anyone who's taken a deeper look into antivirus as attack surface will agree.
     
    Last edited: Aug 18, 2014
  6. zfactor

    zfactor Registered Member

    Joined:
    Mar 10, 2005
    Posts:
    6,102
    Location:
    on my zx10-r
    imo with mse you are just getting lipstick on a pig. i have to repair and remove infections from more systems running mse than any other av no matter if its one we deal with or not. mse just simply does not do a good enough job for the average user who has no idea that the page asking you to update flash player is fake, or that the java update they say okay to infects the system. regardless of who argues it with me its the fact of the matter. i keep track of which av systems were running when i repair the systems. and at least for me mse is not worth the time it takes to install it as far as im concerned.

    a good example is one lady who would call me and i would go out and remove the infection and i would ask her to let me switch her av even to something else that would be free if not paid. her husband said no. okay, a couple weeks later i get another call this time she is upset and says i think its infected again. why didn't you do something to prevent this? i explained i asked her to allow me to switch her av and she said no. so back out i go, this time for a nasty infection (first time was a bunch of fake av's). i clean it all up and this time she is able to go about a month before yet again getting infected. she keeps telling me i have no idea how this is happening. i go over (once again) how not to get infected in the first place as best is possible. this time her husband is there ~ Snipped as per TOS ~. he tells me he is sick of this. i finally get them to switch to a av we deal with. she has yet to be infected in almost a year now.

    but i see this happen all the time with mse. again this is my opinion and im not going to argue with anyone. but going by my statistics plain and simple mse sucks.
     
    Last edited by a moderator: Aug 18, 2014
  7. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    The reason why MSE doesn't need special drivers is maybe because it has like zero features? avast!'s virtualization and behavior analysis, HTTP scanning and browser script scanning simply requires layers that allow that functionality. Of course it will have stuff, but that doesn't mean it's any easier to exploit. If that was the case, we'd have a constant stream of such problems, yet other than few very conceptual methods, we haven't seen any.

    Also the fact that the person in that article included antivirus definition files into the list tells me he has pretty much no knowledge about anything what so ever, he just found out one program that gave him info on what is ASLR enabled and what isn't and he made some wild conclusions out of it without knowing no kernel driver can ever be ASLR enabled. Hey, i didn't know that either, but then again i also didn't write an article stating something i had no clue about. Vlk wasn't defending their product (avast!), he simply stated universal "truth" that applies to all AV's and if they use kernel drivers, this means the entire list in that article is pointless. See how easy it is when someone who knows his trade explains things. Still wana rely on "truth" that person is posting in his articles?

    After all, he could make the list and then ask developers about it. And they'd explain to him that stuff listed there are kernel drivers that can't be ASLR enabled and that would clear up his curiosity. Instead, he posted an unverified information.
     
  8. Hungry Man

    Hungry Man Registered Member

    Joined:
    May 11, 2011
    Posts:
    9,146
    edit: lol you know what? Converstaions like this, where it's now my job to educate someone on how computers work, that's why I don't bother posting here.
     
    Last edited: Aug 18, 2014
  9. anon

    anon Registered Member

    Joined:
    Dec 27, 2012
    Posts:
    7,980
    ....... find here the results as interactive map:
    http://impact.av-comparatives.org/
     
  10. JohnBurns

    JohnBurns Registered Member

    Joined:
    Jul 4, 2004
    Posts:
    778
    Location:
    Oklahoma City
  11. Rompin Raider

    Rompin Raider Registered Member

    Joined:
    May 6, 2010
    Posts:
    1,254
    Location:
    Texas
    Thanks for the post. That should chap some trolls! Standby....let me get my ear plugs and crash helmet on..:eek:
     
  12. Nekomaou

    Nekomaou Registered Member

    Joined:
    Aug 19, 2014
    Posts:
    5
    MSE is still good enough, for free its very good.
     
  13. Securon

    Securon Registered Member

    Joined:
    Jan 11, 2009
    Posts:
    1,960
    Location:
    London On
    Good Evening! Does Statistical Analysis Focus on Vendors Combating Malware in their Specific Countries...and indicate which Malware might be more Prevalent...in A country versus B country? So from a Marketing Demographic...would this sway an individual's decision...as too what Product...might supposedly provide Superior Protection? I've suspected that based on Regional Malware activity...there might be a Real Trend Emerging. Especially Banking and Government and Financial Institutions. Interesting...but something to which the majority of Wilders Members are Already Aware of...in most instances. Sincerely...Securon
     
  14. TheWindBringeth

    TheWindBringeth Registered Member

    Joined:
    Feb 29, 2012
    Posts:
    2,171
    My feeling at this point is that these top-down statistical views aren't *necessarily* relevant to individual decisions. An individual would be concerned about statistics *that are adjusted to match their specific context*.

    An individual's context might be such that they have a low probability of encountering and being burned by the most prevalent threat in the world or the most prevalent threat in the country where they are currently located. On the other hand, an individual's context might be such that they have a high probability of encountering and being burned by the least prevalent threat in the world or the least prevalent threat in the country where they are.

    IOW, I feel like there is potential value in prevalence-weighted test results *as a supplement to non-prevalence-weighted test results*, at least if/when individuals are very careful about interpreting things.

    I would also point out that telemetry busting is SOP in secure environments, so that has to be figured in.
     
  15. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    Sorry but that's rubbish. MS with its crappy MSE on 6th place. Only in their dreams. So, everyone else spending massive amounts of money on advanced cloud systems and behavior blockers and other kinds of protection systems are just throwing money out the window and then MSE comes with absolutely no such features and takes the crown. It's just doesn't compute. It just doesn't. Period. And with Bitdefender that has constantly and consistently beat the crap out of everyone is now 15th? C'mon, am i the only one who thinks it's BS?

    https://www.microsoft.com/security/portal/mmpc/shared/protection.aspx

    Also this one is funny. If MSE systems aren't reporting any infections, it may just as well mean you're not detecting any and doesn't mean that there weren't any. And the performance nonsense. Just open a folder with more than one EXE file in it and you'll get in a world of waiting for things to happen. I haven't seen a single other AV that is so bloody slow at scanning EXE files. And every time i reinstall my Windows 8.1 system where MSE comes pre-enabled since no AV is installed, i can feel it drag my system on its knees. After i install avast! (or any other AV for that matter) it seems to feel like i'm on a different computer. And i have a bloody quad core i7 CPU at 4GHz, 18GB of RAM and hybrid HDD+SSD storage. They can show their graphs all they want, MSE is not fast by any definition. Only thing saving it in the long run is daily grinding of drives it does, that is populating their scan cache and makes things tiny bit faster. avast! (or again any other AV) doesn't have to grind my HDD's daily, yet it's still faster.
     
  16. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Doesn't MSRT include only small set of malware that is detected and removed each Patch Tuesday? Can this small sample really be used to measure prevalence of specific malware family?
     
  17. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    Of course it can by Microsoft's methodology. They detect and clean 10 types of threats and if they encounter those 10, it means they have a 100% detection rate. Time to bang on the chest.
     
  18. TheWindBringeth

    TheWindBringeth Registered Member

    Joined:
    Feb 29, 2012
    Posts:
    2,171
    Interesting question, and one that makes me curious. FWIW, at http://support.microsoft.com/?kbid=890830, More information for advanced users, Usage and release information, there is release information and this note:
    There are 242 entries in the "Malicious software family" table, and given the wording it would seem that those families and variants are what the latest version detects.
     
  19. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Thank you for link.
    Some families are really old and are probably not prevalent any more. Either way, MSE has advantage over other AVs in this test. Detection for "most prevalent" malware was probably added to MSE detections before MS released MSRT. What is prevalent shouldn't be decided by the same company that has AV participating in test.
    As you said in previous post, other variables could be included when weighting each malware family: location, computer usage and sites regularly visited...
     
  20. TheWindBringeth

    TheWindBringeth Registered Member

    Joined:
    Feb 29, 2012
    Posts:
    2,171
    Is there a neutral source for threat prevalence? Perhaps one that gathers data from numerous sources? Seems to me at least some would be AV companies, or am I missing an option?

    Perhaps there are ways to create prevalence numbers for rough categories of users. I better say it now though... I'd really hate to see "we need more data collection and information about you in order to personalize your AV profile and provide you with better protection" nonsense.
     
  21. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    I remember there was some test published here a while ago where Avs were tested and results were weighted using different user types (gamers, P2P users, social media users...). I can't find that thread though...
     
  22. xxJackxx

    xxJackxx Registered Member

    Joined:
    Oct 23, 2008
    Posts:
    8,623
    Location:
    USA
    Agreed. I had a machine here that was still running MSE because I was out of KIS licenses. One of my developers was having to wait 10 minutes for our source control software to launch. Couldn't see an obvious reason for it. Disabled MSE and it launches instantly. So for some reason it was slowing an application launch for literal minutes without any indication that it was doing something. Worse than no AV at all.
     
  23. zfactor

    zfactor Registered Member

    Joined:
    Mar 10, 2005
    Posts:
    6,102
    Location:
    on my zx10-r
    laptop came in yesterday running mse and malwarebytes free (which had not been updated in over 700 days and was set to only scan manually). she had a keylogger, a trojan (real one not a fp), and a fake av running i had to clean off. of course the tech who worked on it last put mse and malwarebytes on there and never told her she had to run malwarebytes. i assume he used it as a tool to fix the system last time. its a shame more tech do not take the time to set someone up the RIGHT way and explain to the client more how not to get infected. we set her up with a good av and did explain to her some of the ways to not get infected.. but i see this almost daily. have a desktop running mse coming in later today she says is infected.
     
  24. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    No need for MSE bashing, I think you should read the AV-C report carefully!

    Then you understand the different ranking. It simply doesn't mean MSE is now the 6th best. It's a complete different question that this test focuses.

    The regular file-detection test goes for detection rates. (On-demand, with all it's disadvantages, but that should not be topic here.)

    This retest looks (only) at the missings and how widespread they are - according to M$ prevalence data! Thats the point, nothing about detection. And of the missings from M$ many are not so widespread, for the fewer missings from f.e. Avast in the pure "detected/not detected sheme" some are more widespread and thatswhy higher weighted.

    So complete different question. Detection for other AVs are still much better. (for this testset)
    This analysis is nevertheless a first interesting approach, and if you want to critize valid starting points could be:
    - only one source of prevalence data (different vendors have different data...who has the "best"? ;) )
    - counting and weighting points and formulas
    - the whole testset from AV-C is already selected according to different prevalence data, so a re-weighting in analysis is statistically wrong (beside the fact, that only one data source is used for that)

    But: It's all written down clearly and transparent in the reports of AV-C! The widespread misinterpretation is not the fault of AV-C and not of M$.
     
  25. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    It's not bashing, but things just don't add up to anything they are saying. And MSE is as much hit/miss as avast!. Either it detects something or it doesn't. There is no middle way. But avast! updates its definitions every 3 minutes using Streaming Updates and heavily relies on extended cloud detection (FileRep), something MSE doesn't even have unless if you're using Internet Explorer or Win8.1 with enabled SmartScreen. If you're on WIn7 or Win8 with disabled SS or you happen to use Firefox, Opera or Chrome, scores dive like crazy. And 24 hour update cycles don't help either. In 24 hour frametime, a lot of threats begin their life and disappear before MSE would even update itself. Would you trust that? I know i wouldn't and i don't.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.