AV-Comparatives: Real-World Protection Test – May 2018

Discussion in 'other anti-virus software' started by anon, Jun 15, 2018.

  1. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,399
    Location:
    U.S.A.
    It is only as good as the malware samples employed. Appears DNS-changer and scareware samples were not employed against Avira. Even if they were, the variants that you were hit with obviously were not.

    Also of note by this statement:
    You are running a non-default setting configuration.

    Bottom line - AV Lab tests are "best guess" rough approximations on how AV software will perform under default settings for the average user. Note the like worded AV-C disclaimer I posted in reply #5. Also of note is "the default setting" rule is no longer an absolute in AV lab testing as noted in AV-C's recent test of Enterprise products. So these reports need to closely examined for like particulars; not just a review of any graphical results chart.
     
    Last edited: Jun 17, 2018
  2. FanJ

    FanJ Updates Team

    Joined:
    Feb 9, 2002
    Posts:
    3,964
    Thanks Andreas.

    But I still don't understand this about PUA settings:

    You wrote:
    According to that, PUA detection is on.
    That is not the default setting of Eset.
    But your own site tells at https://www.av-comparatives.org/real-world-protection-test-methodology/ :
     
  3. guest

    guest Guest

    exactly, the public doesn't read the methodology, just those shiny bars and numbers then go into fistfights when their beloved AV score "low".
    not saying the samples used are not even mentioned.
     
  4. cruelsister

    cruelsister Registered Member

    Joined:
    Nov 6, 2007
    Posts:
    1,519
    Location:
    Paris
    Very well put. Both the age and the particular type of malware are rarely (if ever) stated and sadly no one ever seems to call them on this omission. Further, it always seemed to me that the Pro testing organizations were (are) being overly kind to the products being tested by their concentration on riff-raffy crap like ransomware instead of more insidious malware such as scriptors which are both harder to detect and more prevalent.
     
  5. guest

    guest Guest

    If serious 0-days and scriptors were tested they all will score like what? 20% (and i'm kind),it would bad advertisement for vendors (and they wont participate anymore), so it will never happen.
    Test labs are AV just hidden marketing teams for vendors, and they all laugh when they see fanboys fighting over results.
     
  6. cruelsister

    cruelsister Registered Member

    Joined:
    Nov 6, 2007
    Posts:
    1,519
    Location:
    Paris
    I would just love to see Ophelia run the tests for just 1 month. There would be much Moaning and Gnashing of Teeth...
     
  7. whitestar_999

    whitestar_999 Registered Member

    Joined:
    Apr 1, 2010
    Posts:
    145
    If you don't mind may I know who Ophelia is,you frequently mention her.:)
     
  8. roger_m

    roger_m Registered Member

    Joined:
    Jan 25, 2009
    Posts:
    8,070
    It's her (obviously very talented) cat.
     
  9. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,399
    Location:
    U.S.A.
    One also has to perform "due diligence" when employing AV lab tests as evaluation criteria and put in some effort to seek out as much data as possible. Not all AV lab testing is the same. For example, SE Labs last consumer AV product Realtime quarterly test: https://selabs.uk/en/reports/consumers included 75% URL based malware samples and 25% targeted attack malware samples.

    Some AV labs also performed AV product testing against specific malware categories. Malware Research Group that I previously reference is known for its quarterly banking and payment protection tests where financial malware samples are used. AV Lab - Poland did AV product testing against fileless based malware last fall: https://avlab.pl/sites/default/files/68files/Malware_Fileless_Protection_Test_EN.pdf and just recently a specialized comparative using ransomware, coinminer, and "bashware" samples: https://avlab.pl/en/best-antivirus-software-2018-based-three-security-tests .

    The main point in these tests and any malware testing for that matter is the test scope is of a limited malware nature. Both AV-Test and AV-C perform "full spectrum" dynamic tests that are overall a better evaluation tool to a given AV product effectiveness; again test malware scope is limited but broader than the realtime tests.
     
    Last edited by a moderator: Jun 18, 2018
  10. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,399
    Location:
    U.S.A.
    An "illuminating" read about AV lab testing is given in this 2016 Virus Bulletin conference paper:

    ANTI-MALWARE TESTING UNDERCOVER
    https://www.virusbulletin.com/uploads/pdf/magazine/2016/VB2016-Corrons-Zwienenberg.pdf

    Unfortunately, things have gotten worse since this paper was written as evidenced by the "shenanigans" last year to "accommodate" Next Gen vendors.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.