AV-Comparatives Real-World Protection Test - July-August 2022

Article = https://www.av-comparatives.org/tests/real-world-protection-test-july-august-2022-factsheet/

Chart = https://www.av-comparatives.org/comparison/?usertype=consumer&chart_chart=chart2&chart_year=2022&chart_month=Jul-Aug&chart_sort=1&chart_zoom=2

 

You know, I really can't stand this stuff. Why not simply tell us how many malware samples they failed to block? And what's up with this user dependent stuff? It's not clear to me what this means, and should also be counted as a fail.

But anyway, I was surprised that only AVG, Avast, Avira, G Data and Trend Micro managed to block all samples. While especially Trend Micro is often crap in other tests. End conclusion is, for good security, don't rely solely on your AV, but use extra protection tools.

 

This +1.

 

Yes I mean, some people keep saying that keeping systems up to date, not being click happy and using an AV should be enough. The problem is, AV's are not bulletproof as seen in many tests. And you never know if you might download malware by mistake, with that I mean sometimes malware might be listed on trusted download sites and in rare cases you might get hit with a supply chain attack, where attackers simply modify (or trojanize) legitimate software.

 

It is a potential fail but it doesn't have to be. I don't think it should be included with the absolute fails but I think they should be taken seriously. It's most likely to be a fail for the people that encountered the item in the first place.

 

As usual, Norton and Trend Micro with the false positives... It's hard to miss a detection when everything is detected.

 

Methodology Click here

RT(F)M =

 

Most AVs include Behavior Blockers (BB). A BB is designed to detect potential malware that is so new that signatures have not yet been developed. A BB detects behavior that is typical of malware -- such as execution of powershell scripts. Those behaviors may found in NON-malware apps as well as malware. Thus, the user must decide.

 

Depends on the BB. Not all of them are user dependent

 

The results show marginal differences amongst the programs and that is assuming 100% objective testing, etc. The tests are basically irrelevant.

 

True. Some AVs are able to generate sigs that that are only minutes behind 0-day malware (et al). Since an AV's Behavior Blocker (BB) mainly deals with those apps that are deemed OKAY by the AV's sig-based engine, there will be fewer BB alerts/FPs by those AVs with quickly updated sigs. Also, some AVs have opted for "less aggressive" BB components -- those AVs are trading-off a bit of security so as to be "friendly" (few alerts) but still stay sufficiently effective for most home users.

 

Thanks, now I understand it better. And to be honest, I didn't know that some AV's apparently give users a choice to block certain behavior. IMO, AV's should not be doing this. With Win Defender I don't get to see such alerts since it has no user controlled behavior blocker.

Totally forgot to check this, good point.

 

Nowadays an AV without a Behavior Blocker (BB) or some other way of detecting 0-days (et al) is basically telling its higher risk users to "bend over and grab your ankles."

I'm fairly certain that Win Defender does have a BB of some sorts. It had 2 FPs and most FPs are generated by BBs or HIPS, not sigs. Sigs are like fingerprints. IMO it would be very rare for an FP to be generated by a legitimate app having an identical fingerprint/sig as a malware.

 

In some cases behaviours will be blocked automatically, in other cases you will be prompted if you want to allow or deny the action. Behavioural protection is a very important part of protection these days, for detecting new threats that are not detected by signatures or heuristics.

 

I didn't know that certain AV's would give BB alerts, giving users a choice to allow or deny. Like I said, that's not the job of an AV. An AV should simply auto-block malware. And Win Defender doesn't have a user controlled BB, that's what I meant. And I also don't think that WD's locally based BB/heuristics is that advanced, behavioral monitoring is mostly done in the cloud from what I understood. That's why I'm using SpyShelter, for more control over app behavior.

 

Me, too. It's an excellent, user-friendly choice for adding a strong supplemental layer of protection, as is needed by higher-risk users. So, also, is OSArmor.

 

Hmm if I were a high risk user, I'd probably just use Qubes OS.

 

Per THIS review, Qube OS isn't quite ready for prime time. I find that its misleading name is also a red flag. Qube OS is NOT an OS. It is a front end -- a distro -- for Linux OS.

IMO, switching to an as-yet buggy Linux distro would be questionable for even higher-risk users when, with relatively little effort, Windows can be made virtually airtight. However, to each his/her own.

 

High risk users don't know WTF QubesOS is. High Risk users are running Windows 7/8/10, with an expired antivirus because McAfee or Norton came preinstalled and only lasted for 3 months, but they never paid attention to it.

 

Why? Without behaviour blockers, protection against very new malware would be reduced a lot, which is a bad thing. By including behaviour blockers, it greatly reduces the need to use additional security software. This is why the only security software I use is an antivirus.

 

I wouldn't want to call myself a high risk user, it's more about having complete control of app behavior, even from trusted apps. And it might alert you about unusual behavior, which has sometimes caused me to block and remove apps.

Yes exactly, that's what I'm trying to explain. You don't need to switch to macOS or whatever Linux flavor to stay safe. I haven't had any malware infection 25 years of computing on Windows. And I do whatever the hell I want, I visit lots of websites and have downloaded hundreds of apps.

No you're misunderstanding. AV's should always auto-block and make the decision about whether some app is malware or not. It shouldn't present users with alerts that ''some app might be malicious or not'', what do these alerts even look like? I have never seen them from well known AV's. That's why I also don't see Win Smartscreen as a true security solution.

 

But that presents a problem. Often they don't know if the behaviour is actually malicious or not, which is why they let the user choose an action to take. So while a certain behaviour could be done by malware, it also could be something that in some cases legitimate software can do.

 

AV's seem (at least for me) have always been the 'easy fill in' to scrape up 'KNOWN IDENTIFIED' malware/viruses leaving a monumental task which in my opinion Third Party security programs do the bulk of the heaviest lifting. Especially today in this era of cleverly crafted AV bypasses which leave AV's with their chins on the floor. Yes they have improved to a degree, but relying just on ANY single AV (no matter the brand) be it commercial or home isn't prudent or reliable 'as it's always been the case'. Even a good top reputable AV is only as good as it's supporting security measures around it for just those types of malware which are specially crafted to circumvent and keep them at bay while the malware circles around them undetected.

These tests are nill IMHO and is only for showcase purposes and bragging rights as always. Not to lessen their importance in any way, AV's after all serve a vital role and while still room for improvement, they do prevent what otherwise would be a computer system's downtime nightmare without one.

Of course my latter statement absolutely doesn't apply to the more savvy experienced users/administrators who over the course of time with Windows experience have learned to use 3rd party and local tools and other preventions that head off intruders or merely stop them dead in their tracks before any disruptions can get started.

 

To add to what @roger_m commented --

Either on customer's computer or in the cloud, an AV with a Behavior Blocker (BB) will have a list of specific malware-type behaviors that it watches for. As Roger noted, many of these same behaviors are also found in "legitimate" (non-malware) apps. There are degrees of "likelihood to generate FPs/alerts" as follows::

1- LOW likelihood of FPs/alerts -- Some malware-type behaviors are seldom found in non-malware apps, hence they will generate very very few (if any) FPs/alerts. Examples:
A-Download of remote URLs by using a command line, and B- Execution of .jar scripts
2- MEDIUM likelihood of FPs/alerts -- There are many malware-type behaviors that lie somewhere between LOW and HIGH likelihood of causing FPs/alerts. Example: Execution of .bat scripts
3- HIGH likelihood of FPs/alerts -- Some malware-type behaviors are much more often found in non-malware apps, hence they may generate a goodly number of FPs/alerts. Examples: A- use of cmd.exe, and B- Execution of .msi installer scripts and C- Execution of any "auto-elevate" system process.

As to HOW an AV can configure its Behavior Blocker (BB) so as to reduce FPs/alerts -- the methods include but are not limited to the following:
A- Eliminate many if not all HIGH likelihood behaviors from its BB's list of malware-type behaviors.
B- Eliminate some of the more bothersome MEDIUM likelihood behaviors from its BB's list of malware-type behaviors.
C- Develop and maintain a long hash-values list of tested &/or "high reputation" apps that may manifest one or more malware-type behaviors but are ~known to be safe. Those apps will not be processed through the BB engine.
D- Develop and maintain a validated list of trusted vendors such that apps "signed" by those vendors will not be processed through the BB engine.

Be aware that the above listed Methods WILL reduce protective scope of the BB to some degree. However, they can be the means for attaining a good, user-friendly BB that will tend to prevent users from too-fast "Allow" or "Ignore" click-throughs caused by frequent alerts.

Achieving an effective balance between *scope of protection* VERSUS *user friendliness* requires a competent staff, good computer algorithms, and hard work. Several AVs (such as Avira and Avast) have achieved a near-perfect balance.

However, higher-risk users may wisely decide to augment their AV's "user-friendly" BB by running a broad scope, stand-alone BB such as OSArmor, or else a smart/learning anti-exe app such as VoodooShield. This is especially true for those higher-risk users whose job security or privacy or business information/dealings would be greatly damaged by a major malware invasion.

P.S. Interestingly enough, a higher-risk user might actually feel that, on AV-Comp's test results (topic of this thread), a HIGH number of FPs is actually a good selling point. Why? Because those FPs most likely ensued from an extra-aggressive Behavior Blocker. Go figure, wot?

 

Don't you just miss the good ole days of Threatfire and Mamutu as standalones. It still might have been worth the effort had they remain such but unfortunately we live in a "Sell it and pocket some bucks from the more affluent". The highly prized almighty paper dollar wins out every time at the expense of once fabulous single purpose security programs such as Behavior Blockers.

One can only evermore in retrospect wonder just how such standalones might have faired