AV-Comparatives: Real-World Protection Test Feb-Mar 2019

AV-Comparatives: Real-World Protection Test Feb-Mar 2019

Chart:
https://www.av-comparatives.org/com...chart_month=Feb-Mar&chart_sort=1&chart_zoom=2

PDF:
https://www.av-comparatives.org/tests/real-world-protection-test-february-march-2019-factsheet/

 

Microsoft Windows Defender: False Positives = 36

 

Congrats to Avira, Kaspersky,and VIPRE.

 

It's easy to get 100% detection when you detect everything.

I was surprised to see ESET get 5 false positives.

 

Reminds me of Webroot SecureAnywhere, let's detect everything and let users report FPs so we whitelist them!

that's a first

 

LOL, with so many false positives, it really can't be recommended. On the other hand, false negatives are even worse.

 

Sometimes. When the false positive is a system file it can trash your entire system and a re-image is necessary. This has happened to me more than once.

 

Do you allow your AV to automatically repair or kill its detections? Not me. I only allow my AV to "report & quarantine" its detections.

 

If it quarantines it too will trash the system if it's a system file. Any other file just won't run. That's my understanding.

 

If critical system files get quarantined, then Windows won't be able to boot. While the damage should be fixable, it won't be as simple as restoring the filse from quarantine, as you won't be able use your antivirus if Windows can't boot.

Although it's not recent, here is an example of a such a situation.
https://arstechnica.com/information...-update-breaks-64-bit-windows-pcs/?comments=1

 

@ act8192 & Roger -- you're right. My concept is FAR from bullet proof. One of these days I might have to restore an image to recover.

Gadzooks Roger! That Bit Defender incident, as reported by the link you gave, is a real horror story!

In the past, I have had 2 instances where a system file got sent to quarantine but -- luckily -- nothing precluded restoring the system file from quarantine when the AV popped the notice. Having said that, luck often runs out just when you need it the most so I still image at least 2X/week -- just in case.

 

I noticed BullGuard is no longer part of any of these tests. What gives?

 

i noticed that too and since now that i am using it this is not such a good news

 

That's great when you make that choice and the product actually respects it. Ever used Norton? Or Trend Micro? Or Windows Defender (at least in the past, not sure if it still does this). It's difficult to use any of these on a software development machine.

 

Bummers! I'm sorry that happened to you.

No, I have never used those 3 AVs. I do not usually run any real-time/patrolling AV. Instead, I depend on HitmanPro (an on-demand AV), OSArmor, & R-drive imager. Nowadays, when I recommend a real-time AV to friends or family, it's ESET.

 

Good call, I am currently using ESET myself. Unfortunately I have relatives on Norton. I hate to make it sound that bad but you know what I mean.

 

Yes, Emsisoft pulled out also

 

If you can't stand the heat, stay out of the kitchen... maybe?

 

If AV-C/T or whatever labs cared to use real 0-days (aja less than a couple of hours) and scriptors, the results would drop from 90+% to 40+% at best. Of course, they won't, because if they do, no one will dare to participate then no more incomes lol.

 

Thnx for those links that explain their absence, stapp

 

As I see it, the core problem with emsisoft & guest's critiques is this: they are saying, in effect, that because the tests are not perfect, the buyer should be left with NOTHING upon which to base a selection of an AV. It's something like saying: (1) "Take a blind shot & hope for the best." or (2) "My product declines to be tested but never mind that -- my product is the best & all the other AVs are cheaters. You know that's true because I say so."

If someone wants to critique test labs by saying that other AVs are gaming the system (thereby inferring that they are ethically inferior to emsisoft) then they should carefully design & institute an economically feasible test that meets their specifications.

There are constructive criticisms, and there are carping criticisms. The main difference between the 2 is that constructive criticisms actively offer to assist in making meaningful, economically feasible improvements whereas the carping criticisms merely carp and carp and carp. It's the same-old same-old, 1-sided "because I say so" criticisms in every thread that discusses the results of AV tests.

 

Well, I have to remind myself these antivirus firms are worth billion/s. Webroot was reportedly sold for well over half a billion USD. If you participate in these comparatives, you stand to gain (or lose) millions in revenue from advertising your brand in that context. This must have culminated in a no-brainer for Emsisoft to pull out, no need to explain. Why are we making this so complicated? It boils down to money and how to minimize losing it while keeping the image intact via community outreach.

 

Incidences of "gaming" tests have been found and the testing organizations have taken some strict actions in the past. Most of the major testing corporations take such issues very seriously and I think those common methods described in Emsisoft's forums are at least well known to AV-C and/or AV-Test. I do not believe that this can be a reason to not participate.

The main question for any security firm is whether getting tested helps them to improve their sales, and for Emsisoft, their management decided that it did not - and previous comments here and on Emsi's forums have indicated they are not where they want to be financially. Spending so much money for a testing certification and not helping their sales isn't a good investment in the end. This is also the reason why eScan switched from Home User Testing to Corporate for AV-C, because their focus market is SOHO and Enterprise and that is where the testing helped them to gain contracts/licenses.

For BullGuard the reason is less clear - I'd imagine they feel secure with BitDefender SDK and they're still on AV-Test meaning they can continue to advertise. What I know is that BullGuard ramped up it's partner program to offer more incentives and profit to the distributors and they're likely adjusting that investment elsewhere - like not participating in AV-C tests.

Personally, I'd not go with a product that is not on either of AV-C or AV-Test, except if it uses reasonable technologies from well-known vendors. For example, Emsisoft is most likely fine to use as it has BitDefender at the very least.

 

@bellgamin ask test labs why they never give publicly the name of the samples they used nor their age... I asked, they refused... Guess why...

There is a huge difference between a 0-hour malware than one having already be in the wild for several hours...

now they may say "in real world situation, being hit by 0-hours/days malware is minimal", so I will say then testing AVs versus known malwares is pointless...

Im not interested by knowing an AV has 95+% detection, I want real stress test, I want test labs to do everything they can to show failures, not success. Then I can see which AV is truly reliable.
I'm a tester, I push products to fail, not to behave properly, and so do all tests in the world but not AVs? Come on...