AV-Comparatives: Real-World Protection Test – May 2018

AV-Comparatives: Real-World Protection Test – May 2018

https://www.av-comparatives.org/com...=2018&chart_month=5&chart_sort=1&chart_zoom=3

https://www.av-comparatives.org/tests/real-world-protection-test-may-2018-factsheet/

 

Avira with perfect score

 

very surprised by how well Microsoft has been doing lately

ESET never does well in these tests, Bitdefender, Avira, Kaspersky always take the cake and now even Microsoft is better.


I won't renew my license with ESET in 2020 when it expires even though I am an old 10+ years customer, these test kinda disappoint me in how ESET is performing.

 

Way to go Avira! Perfect score.

 

Per AV-C Test Methodolgy:

Eset scored 99.1% w/3 FPs(almost unheard of previously). The test included 227 samples and only included web URL delivered malware. Of note is Eset's PUA protection is disabled by default during installation.

"Each to their own" as the saying goes.

 

Looks like Emsisoft has got its game back on

 

@itman we do not include any PUA samples in the tests - and just to be on the safe side and avoid claims, PUA detection is activated for all.

 

Just curious::

From the AV-C website: "Our Real-World Protection Test is currently the most comprehensive and complex test available";

then, "The results are based on the test set of 227 live test cases (malicious URLs found in the field), consisting of working exploits (i.e. drive-by downloads) and URLs pointing directly to malware. Thus exactly the same infection vectors are used as a typical user would experience in everyday life".
Should one infer that Computer Users only need to worry about web-based attacks and not to be concerned about infections spread via Networks (worms), USB, Email, FTP, etc? Or is it that the initial statement may be a tad aggressive and may lead a reader to a false conclusion?

 

MS Defender just keeps hanging in there. At least I don't have to turn it off when downloading Windows Updates. That adds to my consideration in using it. I'm paranoid about Kaspersky/Russia connection, and don't like to spend money for any of them, so I'll continue using Defender I guess.

 

What happened to Panda? MS definitely doing very well...

 

https://www.av-comparatives.org/tests/real-world-protection-test-may-2018-factsheet/

 

It does seem Microsoft is getting better and better.

 

Good for you.

 

i always like a good laugh next to my morning coffee
tho good to see panda being in its well deserved place for once, even if this test is a joke overall

 

Have you checked the impact score?

 

Personally, these periodic real time tests are becoming irrelevant to me.

In this test 14/18 products scored 99+% in detection. Then we have the distorted graphical presentation showing the 1% misses. A corresponding analogy would be I am at the race track. All the horses appear to cross the finish line together. We then have to wait for the enhanced photo finish results to determine which horse actually won.

At least the like MRG tests will sub-divide by category what malware samples were used and what the product detection score was by each category. Finally, none of these tests show detailed statistical data on test malware prevalence; i.e. in-the-wild occurrence, or geographical distribution; i.e. worldwide or restricted to select countries.

 

Default "zoom" level on their site is correct ( 0-100% ), which shows minimal difference in detection without "distortion". Just links on this forum are modified and try to amplify difference between protection rate. Don't know why...

 

The default "zoom" level on their site is "by vendor" and "0-100%"
With above settings t's easier to read the test results, ie: detection rates, which product is better, which one is the worst?
I don't think so.

The 1st, 2nd, etc are always the same, whatever "zoom" level you choose.
Just look the graph in "by value" mode, at any "zoom" level.......

 

I'm now somewhat confused about PUA detection being activated for the test.

On 15 Nov 2017 (in the thread about the Real-World Protection Test - October 2017) Marcos posted here:

So, Marcos posted that AV-C performs tests with default settings, and in the case of ESET that means that PUA detection is off.
And Andreas posted now that PUA detection is activated for all; so that must mean it is also activated for ESET and that is not the default setting of ESET.

Looks to me a bit contradicting.
Has there something been changed in the test methods in that respect between October 2017 and May 2018?
Or is it just me who is confused now?

 

@FanJ: I complained already last year to ESET about the potentially confusing statement done in a forum (and it did not match the feedback we received).

 

Just to clarify.

@IBK

You enabled both potentially unwanted application And potentially unsafe application detection for ESET? Correct?

 

A few other comments about AV lab testing of Eset.

It must be verified that LiveGrid, Eset's reputational scanner, is fully functional. The procedure for doing so is here: https://support.eset.com/kb5552/?intcmp="KBCJ_3418_LiveGrid" . This is especially critical in the latest 11.1.54 version since there have been major changes and enhancements to its processing methods.

In regards to PUA detection, note that Eset not only uses it to detect like executable download, installation, or startup. PUA detection is also used to detect suspicious code in scripts, etc. in regards to abuse of trusted system processes increasingly being used by malware developers.

 

IMO ESET Must enable PUP and potentially unsafe application detection by default. A lot of rookie users just blindly hit NEXT when installing anything and would miss out on the extra protection that ESET *can* provide. Very bad idea to keep it sa disabled by default. Just my 2 cents

 

As an IT Manager I know that I fear email more than anything else. I have users that will open anything because they think it might be from a customer. So yes, I feel that the focus on malicious URLs is misleading.

 

Serious question:
What is such a test good for?

Yesterday two customer machines came in for cleaning.
One was running WIN7, the other Win10.
Both had Avira installed.
Both where unusable.
A DNS-changer and more on the the WIN7, two scarewares on the WIN10.

On the WIN7 machine I run ZAM, and the DNS-changer was gone.
On the WIN10 machine I uninstalled Avira, activated Defender and applied the PUP.reg, and the scareware was gone.