AV-Test.org: June 2018 test results (Windows 10)

Discussion in 'other anti-virus software' started by anon, Jul 30, 2018.

  1. mekelek

    mekelek Registered Member

    Joined:
    May 5, 2017
    Posts:
    518
    Location:
    Hungary
    yea Ophelia is a very good coder idk what you on about people
    /s
     
  2. roger_m

    roger_m Registered Member

    Joined:
    Jan 25, 2009
    Posts:
    8,627
    I was actually talking about Meghan herself. But her cat, is very talented :)
     
  3. xxJackxx

    xxJackxx Registered Member

    Joined:
    Oct 23, 2008
    Posts:
    8,624
    Location:
    USA
    Yes, very skilled and credible. More so than most folks.
     
  4. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Exactly. When AV-Test started testing PC Matic a while back, their FP rate was in the 1000's.

    If you look at their scoring in the Usability section for this test in the malware test of 1,666,880 samples, PC Matic only had 29 for May and 33 for June false detections. So something has definitely changed in their detection capability. I suspect they are now also employing signatures along with whitelisting.

    It can also be attributed to only 41 samples were used in the flash warnings/blocking categories. Assume a retest here was in play since PC Matic has spent a lot of money at AV-Test in the past on commissioned tests.
     
    Last edited: Aug 1, 2018
  5. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    The problem is AV-Test is the only AV Lab that has consistently scored Eset low in the performance category. Every other AV lab rates it at the top in this category.

    Personally, I don't want anything to AV-Test after the Cylance "synthetic" malware charade last year.
     
  6. m0unds

    m0unds Registered Member

    Joined:
    Nov 12, 2015
    Posts:
    219
    as i said, all other products are tested using the same methodology. no amount of saying "but they do well in other tests" makes the results of /this particular test/ different.

    then just say that outright. no point in arguing or hemming and hawing over data from tests that every product was subjected to. :thumb:
     
  7. m0unds

    m0unds Registered Member

    Joined:
    Nov 12, 2015
    Posts:
    219
    she also routinely declines to provide samples to vendors who are curious how their products were bypassed.

    she ignores that most modern products use systems that determine the reputation of files based on where they come from (is this a new domain? is it from a known infected host? has the product seen bad files from this host or ip range in the past? etc etc etc), or community intelligence (has this file been seen before? has it done bad stuff before? how many users have encountered it? where did they encounter it?) and just deploys the files locally and executes them. go figure stuff fails when the test case is outside the realm of the product's protection mechanisms and completely ignores all the systems that work in tandem to try to protect a computer.

    imo, that sort of testing is as useless as most other youtubers' tests.
     
  8. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
  9. Azure Phoenix

    Azure Phoenix Registered Member

    Joined:
    Nov 22, 2014
    Posts:
    1,556
    @m0unds

    > or community intelligence (has this file been seen before? has it done bad stuff before? how many users have encountered it? where did they encounter it?)

    Isn't the point to test them against zero-days? How can that info be available if that were the case?
     
  10. m0unds

    m0unds Registered Member

    Joined:
    Nov 12, 2015
    Posts:
    219
    the lack of info makes it more suspicious and subject to additional scrutiny, if it were downloaded and not just placed or written on the system being tested.
     
  11. m0unds

    m0unds Registered Member

    Joined:
    Nov 12, 2015
    Posts:
    219
    fwiw, wd only scored 5.5 in performance. the only performance section where it scored poorly in with av-test was installation of new software. that's the biggest user complaint i've seen everywhere wrt defender. also the "i have a folder full of exes and it's slow to scan them" (go figure). if you don't like their methodology, ignore the results, but don't imply there's some nefarious plot afoot :)
     
  12. mekelek

    mekelek Registered Member

    Joined:
    May 5, 2017
    Posts:
    518
    Location:
    Hungary
    all i want to know from you, randomly appearing everything knowing person, is that what affiliation do you have with said review site? i mean it's pretty obvious at this point that you're just a shill.
     
  13. m0unds

    m0unds Registered Member

    Joined:
    Nov 12, 2015
    Posts:
    219
    i'm a gainfully employed security researcher and network engineer with 16 years of experience, and no affiliation with any product or testing organization. i'm a fan of logic and taking stuff at face value unless there's a reason not to. so, maybe, just maybe, lay off the personal attacks, mmkay?
     
  14. EASTER

    EASTER Registered Member

    Joined:
    Jul 28, 2007
    Posts:
    11,126
    Location:
    U.S.A. (South)
    This user over time is come to not so much rely on comparisons-scores although they are/can be good indicators of progress on many levels. Yet some contributing AV's will pocket reports to their marketing advantage or lack thereof?

    Am at a loss on PC Matic though. Never did expect much if anything from them due to the fact they've hit my TV with tons of commercials/promises etc over the past couple of years or more I took as routine marketing hype.
     
  15. PocketAppZ

    PocketAppZ Registered Member

    Joined:
    Sep 10, 2014
    Posts:
    16
    Why is Webroot not on the list?
     
  16. guest

    guest Guest

    because Webroot works in a mystic way that can't be evaluated/compared with other products :p
     
  17. Gein

    Gein Registered Member

    Joined:
    Dec 8, 2013
    Posts:
    219
    I hope you realize that's tongue-in-cheek.

    What is going on at AV-Test?
     
    Last edited: Aug 2, 2018
  18. bellgamin

    bellgamin Registered Member

    Joined:
    Aug 1, 2002
    Posts:
    8,102
    Location:
    Hawaii
    I get your inference.

    For myself, I don't think that AV-Test jiggled any of the test results' numbers. Also, I am fairly certain that the test methodology applied to WD was the exact same methodology applied to ALL of the AVs tested.

    After re-reading the thread at HERE, I think perhaps the primary fogginess concerning AV tests stems from the fact that they exude a false aura of precision. How so? Let's say that a hypothetical test report showed that AV "A" scored 99.8% whereas AV "B" scored 99.6%, & they therefore rank AV "A" higher than AV "B".

    So, silly bloke that I am, I promptly uninstall AV "B" & rush to buy/install AV "A" whereas the significant difference between AVs "A" & "B" doesn't amount to a pool of warm spit.

    (Hmmm... I wonder whether it would help dispel the fog if test reports somehow eliminated the false precision caused by their reporting of numeric percentage data, especially when carried to 1 place after the decimal?)

    Pragmatically, all we actually know from this hypothetical test report is that"A" & "B" both did pretty okay, but only with respect to:
    • that SPECIFIC set of test data
    • that SPECIFIC test methodology
    • using that SPECIFIC number & quality of technicians that that SPECIFIC test lab could afford to employ
    • given that SPECIFIC amount of time that the lab had for doing the tests so as to meet a publication deadline.
    As I understand it, the cost of designing & executing a near-perfect AV test would be huge. According to Inspector Clouseau's post, a more or less "fully competent" AV test would mandate the following:
    Of course, all of that time, equipment & high-quality personnel would cost so much money that, perhaps, only a nation's government could afford to do it.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.