AV-Comparatives - Microsoft-prevalence-based analysis of the File Detection Tests

Discussion in 'other anti-virus software' started by SweX, Aug 10, 2014.

Thread Status:
Not open for further replies.
  1. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    [from the AV-C blog]
    http://www.av-comparatives.org/microsoft-prevalence-based-analysis-file-detection-tests/
     
  2. phyniks

    phyniks Registered Member

    Joined:
    Jun 3, 2011
    Posts:
    258
    Can anyone tell me where exactly MSE is?!!!

    Capture11.PNG

    Between the colors....OK....after Panda(as missed 3.369 out of 1000) but,
    Missed 10% of samples,missed sample percentage:90% >>>>>so it must be just above Ahnlab :confused:


    Does it want to say this?
    although MSE missed lots of samples,the impact of the "nonblocked" malwares were not that much....
     
  3. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    Yes. Why detect things that don't affect users? But when files are missed that affect many users, then this get's a hight weight. Just look at the first 2 columns in the table.

    Interesting test-methodology, a step in the right direction IMO.
     
  4. phyniks

    phyniks Registered Member

    Joined:
    Jun 3, 2011
    Posts:
    258
    Thanks SLE
    That s odd I think....It says your system is more vulnurable to ruin when you are using most of the famous third-party AVs....
    Among them are Bitdefender,Eset,F-Secure,Avast and AVG !!!!!
     
  5. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    These reports indicates what Microsoft (and Webroot also, I believe) have explained before.
    Although they may miss some samples, they aren't finding their users becoming infected.
     
  6. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    No. For the specific testet samples, weighted by prevalence. Beside that we still talking about on-demand scanning, what says nothing about ruining systems.

    With the difference that M$ has some proofs in form of telemetry data and are actively giving advices for testing, provides this data etc.
    Webroot...only has their marketing sayings.
     
  7. Syobon

    Syobon Registered Member

    Joined:
    Dec 27, 2009
    Posts:
    469
    avira strong
     
  8. Windows_Security

    Windows_Security Registered Member

    Joined:
    Mar 2, 2013
    Posts:
    3,081
    Location:
    Netherlands
    Interesting big data based stats when I understand it correctly. Every month Microsoft runs the Malicious Software Removal Tool. This tool specifically checks on infections for the dominant malware infections of that month. Microsoft probably determines this selection of dominant malware this from its own source (IE URL/reputation filter) plus third party data (shared malware sources, business partners, etc). MSRT simply check what malware is found and what AV was installed to known what slipped through on that 'unique' machine.

    So what bugs me is why this "big data" selection of malware differs so much from what testing companies like AV-Test and AV-Comparatives consider to be real world tests. When you expand for instance the AV-Test over a few months (and asume 50% of the user dependant decisions are done/guessed right), increasing the samples and applying some hocus pocus, these test are starting to show a pattern

    Top freebie group: Avira, Panda, Qihoo360 and Fortinet (all have low 99%), while AVG and Avast are behind (both low 97%). It is only 2% less, but when you take the missed percentage perspective it is relatively a lot more (worse).

    Relating different test sets is always arbitrary, but marketeers and policy makers often use second hand data to obtain circumstantial evidence to support decision making. So wondering whether this ranking confirms the opinion of forum members (based on all the info you read in all your different sources).
     
    Last edited: Aug 10, 2014
  9. Impet

    Impet Registered Member

    Joined:
    May 5, 2013
    Posts:
    895
    So would you recommend MSE instead of Avast or AVG? o_O
     
  10. guest

    guest Guest

    As for AVG I would say it deserves to be thrown at the trash bin. :D
     
  11. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    Kudos to McAfee, it is outstanding what they have done in the last 2 years.
     
  12. henryg

    henryg Registered Member

    Joined:
    Dec 13, 2005
    Posts:
    293

    It think it has something to do with becoming a wholly owned subsidiary of Intel....
     
  13. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    The trend over the six month period is a greater normalized customer impact and poorer actual protection for the majority of vendors. Microsoft has remained relatively constant with a normalized customer impact of approximately 0.003.
     
  14. Inside Out

    Inside Out Registered Member

    Joined:
    Sep 17, 2013
    Posts:
    421
    Location:
    Pangea
    I wonder how the two-faced cowards Norton and G-Fail would fare according to the data from Sept 2013.
     
    Last edited: Aug 11, 2014
  15. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,071
    Location:
    Germany
    Please note that this is a prevalence based analysis of the static file detection test, not the real-world protection test. It does not tell you which product protects you better. In the former, G-Data had scored well and Norton bad. I assume the results would look considerably better for Norton (in the real-world test Norton would look great btw).
     
  16. Inside Out

    Inside Out Registered Member

    Joined:
    Sep 17, 2013
    Posts:
    421
    Location:
    Pangea
    Thanks for the reasoned response unlike those of some others I could mention. Sure, it's based on "just" a file detection test, but IIRC Symantec's one of the main counter-arguments was the prevalence argument, which is why they're so quick to scrap the signatures they consider "irrelevant". Besides, I'm sure real-world results will also be analyzed the same way from now on. Isn't it hypocritical of them to boycott this static file detection test while letting those done by far inferior testers like AVT (static or dynamic) off the hook?

    Also, Bullguard, another BD-based product, got a horrible result here, so G-Data scoring well in Sep 13 doesn't guarantee anything. Same goes for Norton if MSE can do well here despite missing so much.
     
    Last edited: Aug 11, 2014
  17. Impet

    Impet Registered Member

    Joined:
    May 5, 2013
    Posts:
    895
    I see, another useless test. :(
     
  18. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,071
    Location:
    Germany
    No, it's not another useless test, it's a reevaluation of a debatable test from the past ( = static file detection) and puts things in to perspective.
     
  19. oliverjia

    oliverjia Registered Member

    Joined:
    Jul 21, 2005
    Posts:
    1,517
    I agree with you. PPL should wonder WHY Norton opted out the AVC tests while still participating tests held by some other organizations? As to the theory of static file detection being irrelevant, I call that BS. If virus signatures are really irrelevant in today's AV technology as Norton and some of the Wilders members claim, why Norton still uses signatures which are the basis of static file detection? Why not completely get rid of all the old-school virus signature from your database? why bother updating virus definitions?

    Can someone answer my questions?
     
    Last edited: Aug 11, 2014
  20. Windows_Security

    Windows_Security Registered Member

    Joined:
    Mar 2, 2013
    Posts:
    3,081
    Location:
    Netherlands
    I am not recommending anything, just noting that the classic top three of the freebies has changed.

    MS claiming MSE does better, is like a butcher inspecting his own meat, I would be inclined to look more at the results of AV-Test and AV-Comparatives , so have more faith in AVG and AVAST.
     
  21. JRViejo

    JRViejo Global Moderator

    Joined:
    Jul 9, 2008
    Posts:
    20,954
    Location:
    U.S.A.
    Removed Off Topic Posts. Let's Focus On The Subject, and Not Each Other. Thank You!
     
  22. RejZoR

    RejZoR Registered Member

    Joined:
    May 31, 2004
    Posts:
    6,426
    Well, technology itself speaks for itself as well. There is just NO way MSE can be better than avast! (or AVG) with 12 hour update cycles, lack of any kind of cloud functionality (the stuff they say it's cloud does nothing to protect the user right now), lacks behavior analysis entirely and there are several other things that are missing and give other AV's a huge edge. MSE is basically what other AV's were some 10 years ago...
     
  23. true indian

    true indian Registered Member

    Joined:
    Sep 24, 2012
    Posts:
    755
    Location:
    india
    Hmm....This is kind of interesting stats.Keeping in mind avast,qihoo did worser than expected.I get a feeling if MSE is that good why it only has 70-80% block rate in AV-C real world charts whereas all freebies block above 98% these days in av-c charts.

    Here,it doesnt seem like there is something black in the curry but the whole curry seem black to me ;)
     
  24. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    ALL AV vendors must have their own telemetry data. This analysis seems quite suspect to me.
     
  25. guest

    guest Guest

    Yes there is.
     
Loading...
Thread Status:
Not open for further replies.