Anti-Malware.ru: (Zero-Day) malware test

Yes it was probably three or four seconds. I'll try it again sometime when I have more time to spare.

 

These are the same people who will click on a popup and bypass security anyway.

 

I'm going to have to side with Ilya on this one. Regardless of how good or bad this particular test is, real world experience shows every day that people are getting infected in spite of having security software like big-name commercial AVs or popular free AVs or "firewalls" installed and running on their systems. Who here has not seen an infected system that was running an up-to-date Norton suite or something? Fact is, against new malware, such as some trojan-of-the-day that was released 45 minutes ago on a drive-by-download site, most security software is of little use. AVs for example will not have the definitions for such a new malware yet, and if the AV happens to have some kind of HIPS or cloud component to protect against new malware, that still won't necessarily stop social engineering attacks where the user has been fooled to want to run the malware and will just click yes when warned that the file may be up to bad things.

As for default settings, sure, it's typically true that with some configuration security is improved. This is true for many things, first of all Windows, browsers, and even security software. The obvious problem is, how well can the average Joe User perform the configuration changes needed to increase security and how likely is he to even know about the possible need of changing the configuration to increase security? Not very well and not very likely is the answer usually. In real life, a vast mass of users are using everything with the default settings as far as security goes. We are dealing here with the kind of people who have not all yet discovered that there are other fonts than the MS Word default Times New Roman. These people rely on the default settings - if the default settings are not so good, then they are the ones who suffer from it. Compare, for example, how many Linux users run as root (root is not the default) with how many Windows users run as admin (admin is the default)...

I certainly believe security software testing in general is not worth much. Most tests are silly one way or another for reasons such as testing against a set of malware samples that is far too small to matter. Typically testing seems to serve only two purposes: 1) getting the testing group some name and 2) getting some folks switching from one security software to another in their endless search for the "best" and possibly shelling out a lot of cash to happy security product vendors in the process.

But I think the idea of only being able to get meaningful results by doing one's own tests is problematic. First, would the kind of users who actually need security software be qualified to test security software in any meaningful way? Can they build a safe testing environment, locate enough samples of malware and exploits, and then actually perform the test? Nah. Which leaves us with the thought that the only people qualified to do any meaningful security software testing wouldn't really need said security software to protect themselves and therefore might not have any reason to even want to test them...

Of course in new malware or exploit testing any kind of security feature not relying on signatures is likely going to be stronger than signature-based products. The problem with these non-signature-based methods is that they either directly require users to make wise decisions ("Unknown process foo.exe wants to perform some fancy technical stuff you don't understand and this may be dangerous, allow or deny?") which tends to be a decently reliable recipe for disaster, or at the least they provide ways for the user to still screw up magnificently unless he knows what he's doing ("Oh, look, this malware I just downloaded in my limited user account is saying that it needs admin rights, so I guess I'll just give the admin password now and see what happens" or "Hey, this program says it doesn't want to run sandboxed and untrusted. I guess I'll just run it in the real system then to get it working for realsies.") which is something that most users don't.

But before I digress further, where does all this leave us?
- The single most important thing in security is knowing what you're doing. If you don't, there's always the pretty real chance that you may get owned by incompetence in spite of having AVs and HIPS software, limited user accounts and what not. Even if you're in a read-only OS environment where you can't execute any new code, it still won't stop you from owning yourself by falling for phishing and such attacks.
- Malware testing? I don't see it as being very useful except for marketing purposes and occasionally revealing a particularly clever rogue security software that is only revealed to be a fake by its utter lack of efficiency instead of any more obvious sign. It seldom tells Joe User anything that matters to them.
- AVs? The weakness of traditional anti-virus products is that they suck against new malware. Their strong side is that they require less interaction and brains from the user.
- HIPS/sandboxing/virtualization/LUA&Applocker etc? The strong side is performance against any malware or exploit regardless of age - no signatures required, so the restrictions they apply to malware apply to even unknown and new malware. The weak side? They more or less require the user to have some understanding of what he's doing and how the security measure works. Limited user accounts or sandboxes, for example, don't do much if the user just always gives anything and everything the admin password when asked and will execute anything outside the sandbox if the program bothers to pop up an error message when running in the sandbox. And there are loads of users who will do exactly this, unless you stop them either by taking away their admin password or threatening them with fire and brimstone.

 

Yes, the people who need protecting the most!

So mainstream security needs to work for them automatically.

 

And that's the problem. The best security solutions require the user to take a few minutes to learn a little bit. Automatic security that doesn't require a few minutes to learn doesn't always work out that well.

And in how long it would take to learn enough to save your bacon? How long does it take to read Blue's post about securing your computer? 15-20 minutes? But that little bit of education would go a long way to keeping you safe.

As for the malware test....the subject at hand....excellent results Defensewall!!!!!!!!

 

yes DefenSeWall the life saving bacon security program

 

Did you enjoy your meal with your friend and, most importantly, did you manage to clean his PC?

 

Nice post and I agree with everything you say. I was not seriously suggesting that users undertake tests of security products although I'm sure that some, many of who are posting regularly on Wilders, are more than capable of safely running their own trials.

My main gripe is that these so called tests rarely give out sufficient information on how testing was performed and people are expected to just blindly 'accept' the results as fact. If you are reasonably familiar with using a particular product and you sat in on these tests I imagine you would be critical of how the test was being conducted and you would maybe feel that the result is flawed due to the methodology being employed. Or maybe you would be impressed with the way the tests are being conducted ... but we have no way of knowing which scenario is nearest to the truth. Therefore, people should be cautious in using these tests as a measure of effectiveness.

I accept that DW, for example, would fare better against zero day malware attacks than a signature based scanner which will have little chance to prevent infection, although heuristics would maybe give some chance to prevent infection. However, I would not want to put all my faith in one product and I still believe that a multi layer defence is more effective than a single line of defence.

Of course, multi layer defence means more complexity and is this appropriate for the average user who may make his system more vulnerable to attack by misconfiguration? There is no easy answer because it depends on each user and his ability and desire to assimilate information and understand the ramifications of actions he takes. So does this mean that security products should be 'fool proof' and take away the complexity, or at least hide it, so that a good 'out of the box' experience will give maximum security for no investment in learning about how a particular product works? For some the answer is yes but for others, like most savvy Wilders posters, they would be unimpressed if the product hid all settings and did not allow any tweaking whatsoever.

The bottom line is that no product will ever satisfy all users and certainly not if it only works on standard settings and has no configurability. For any test to proclaim that product A is better than product B without giving a reasoned explanation of why that is so, and listing the caveats in reaching that conclusion, draws into question the validity of the testing methodology.

It's not good enough to just give a list of winners and losers and award gold, silver and bronze awards. That smacks of marketing hype. I use an AV which rarely gets included in tests and when it does it always gets low scores for detection and maybe I should be scared and change to another AV. I have tested many and keep coming back to the one I still use. I have never had an infection so that tells me that something may be working better than it should according to the tests. Recently I added a FW/HIPS and that gives me another layer. Previously I just relied on my router FW. I still didn't get infected so maybe my AV, which gets poor results in the tests, works better than the tests give it credit for.

If I took the tests as being the definitive means to choose an AV then I would be using a different product. The only problem I have is not speaking Russian and most of the forum activity for my AV is in Russian.

I just urge people not to blindly accept that these tests are meaningful and don't use them to choose your protection. The best way is to try different products until one 'feels' right then stick with it unless there is an overriding reason to change. If it ain't broke don't fix it!

Finally, I'm not trying to spread a doctrine. I do what I do and you are free to take whatever approach works well for you. And, whatever you do, be safe!

 

Windchild,

Nice post, also a self declared nice profile you have got shameless LUA troll

In regard to LUA+SRP
By the way I am running LUA + SRP on XP Pro with only Avast file shield and windows FW, because a LUA setup reduces the attack surface with 80% at least. My Son is a fanatic gamer he is using UAC + Sully's PGS + Vista FW-2 way and uses on demand AV only. For him the need for speed and big game updates (very time consuming to re-install after a serious virus infection), is a reason to accept the (user friendliness) limitations of the OS its security features.

In regard to DW
I tried LUA with SRP on my wife's PC, but the fact that she is not allowed to run a new program on HER pc by some stupid security program or click on time (XP box) to check when she can make an appointment with friends is just to limiting. On top of that she is a click OKAY/YES happy pc user.

Normal PC users (mind you not hobbyists), use a PC for its function (webbrowsing, e-mail, digital media files, office aps), she rarely installs programs.

DW makes it possible to enjoy a stronger than LUA seamless protection (no in or out of the sandbox hassle). DW always was very quiet, with the new whitelisting feature (to facilitate safe installs) of the V3 HIPS/FW it is near silent, at least silent enough to keep a very non-tech and critical PC user happy.

Why don't you try DW, not for yourself, but to understand how close this is to (stronger than) LUA + SRP protection and how its user frienliness is close to the ease of use of an AV. From this point of view it is understandable for me why DW was included in this test (it is as easy to use as an AV)

Regards Kees

 

With regard to the wife's PC, you could simply open up SRP rules a little to allow her to execute new apps from some folders - and this would still prevent the average drive-by download attack. With Win 7 and AppLocker's publisher rules, that kind of stuff is easier still than before.

As for DW, I think I understand well enough the kind of protection, benefits and downsides it has. I certainly see no reason why one could not test DW together with AVs against malware of various kinds, but then one should expect the signature-based products to lose such tests. It's not all sunshine and honey, though, for products such as DW. Like with LUA (unless the user does not know the admin password), there's always the chance that the user will just tell DW to trust some random file they really should not trust, instead of letting it remain untrusted. Hard to do anything against that - but the good side to AVs is that they may just automatically eradicate a known bad file without even giving the user any option to trust it. As far as my personal preferences go, DW and HIPS products in general really aren't for me or something that I feel comfortable recommending to average users. One reason obviously is money: many users are unwilling to shell out the bucks for a security software - or any software, having spent a lot of money on the computer hardware already. But for me, the really big issues are like this:

That is to say, issues like possible slowdown and compatibility and stability problems, minor or major. These things are of course not DefenseWall issues so much as issues with security software in general, which is one huge reason why I prefer my brains and security built into the OS over other options. Right now, in a non-English speaking forum I'm helping a guy who has his AV's driver causing blue screens after yesterday's Windows updates. I've seen countless such cases where security software BSODs on systems with perfectly fine hardware and no malware infections. I choose not to trouble my own systems with that stuff. I prefer fast and stable. And it just so happens that on my systems, slowdowns and crashes are like dogs that speak Norwegian - very rare!

But sure - if some cosmic force made me choose between either using only DefenseWall / some other HIPS type product for protection or only an AV of my choice, I would certainly choose DW/HIPS over the AV, because the AV just doesn't perform as well against malware attacks assuming a user such as myself who isn't completely clueless on how one is supposed to use the security software.

That's really one reason why I find tests like the one this thread is about so... well, pointless. "HIPS beats AV against new malware!" the testers proclaim. That's like saying: "In other news, rain is wet, so use an umbrella."

 

Unfortunately, majority of the computer's users have no idea that rain is wet. Other words, they are aware of AV's only.

 

Why would I go with a solution like that that only stops the 'average' drive-by when DW gives much stronger protection with far less hassle? DW is on my wife's PC and I haven't had to touch it in months - it requires zero management from me.

 

How can you have any confidence in this group? They are testing for zero-day attack effectiveness and for Norton they use NIS 2009 when NIS 2010 has been out for several months with its new Insight protection that has been shown to be awesome in blocking zero-day stuff.

You have to wonder.

 

Well I tried that, just it is not as simple as you say. Some installers unpack into temp others into the parent directory, leaving open the two nastiest hole I can imagine, Temp folders and User space main directory, which practically reduces SRP to zero. My bet is that DW is the most used security application for other family members (like Scoobs, JJMonge, and others).

I think DW is a lot cheaper than a Windows 7 upgrade, on XP Chrome starts within a sec cold, subsequent startups are in a blink of an eye.

I do not need Applocker's publisher rules. Try PGS, a really brilliant freebie to implement SRP on application name and even wildcards in the name like wind*.exe or *child*.* , ten times easier than Applocker's publisher rule's.

 

That is very true. But it's not easy to change this state of affairs, unfortunately. The majority of users pay little attention to anti-malware tests if they even know about them at all. And I really don't think that's a bad thing, considering the quality of the tests in general. Better to educate the users by other means that are more understandable and clear, and less... well, financially motivated.

You wouldn't, if you don't think it fits your preferences and needs. I was simply saying what could be done in a case where someone was using SRP and wanted to be able to execute some new programs. That someone might have chosen SRP because of the price or lack thereof, for example.

Sure, some installers will have trouble. But then, if this is a limited user account, that would be the case without SRP, as well, seeing how most installers don't like LUA. One could always install legit software as admin and continue along happily. Where this is not possible or is too much of a hassle, then there are fortunately other options, like the DW that you're using.

Certainly DW is cheaper than a Windows 7 upgrade, but I wasn't telling anyone to upgrade to Windows 7. Instead, the point was that for those who use Windows 7 there are nice improvements. As for Chrome, not exactly my favourite browser. And even if it was, there surely are other programs one might wish to run like Firefox, some of which will suffer from noticeable slowdown at startup, and other things.

There are two issues here:
1) You can make filename based rules with wildcards without PGS just as easy as you can make other path rules. Unless you're using a Home version of Windows, but I wouldn't use those. The XP Home version for example was notoriously dumbed down to the point of being an absolute pain to use for me.
2) Filename based rules are also about ten times less secure than publisher rules. You can rename ikillyourpc.exe to wind.exe and now it will run, if you use filename rules. But if you use publisher rules that allow only files signed by certain trusted parties, that would not work, because the file still will not have a valid digital signature from one of your trusted parties. So, while AppLocker's publisher rules aren't an essential feature, they can come in very handy in some cases.

But, I digress. To return to the subject of malware testing, I've always found it amusing that a simple LUA setup is almost never included in such tests (and a LUA with SRP setup is never included). That is one more reason why security software tests in general are not worth anything much to me - they don't even bother to make a rough comparison between the security software and the operating system's own security measures. I continue to be amazed by how much some people care about such tests in spite of the fact that the tests are massively unreliable at worst and captain obvious at best.

 

Windchild

Security should be user friendly to the one using the PC, for me policy and access management are the way to go (yes I am deformed by old days of VMS/RACF etc), therefore all users at home have a different setup

a) Every day - average user: XP Home = admin with DefenseWall V3

b) Gamer - media junkie: Vistax64 = UAC + Norton UAC + SRP (through PGS)

c) Elderly IT geek: XP Pro = LUA + SRP and ACL

So from a troll's view of point, we nearly share the same insights, with the difference that my scope is a bit broader than LUA, being policy management, see new signature.

 

Interesting, and I was just thinking.

Regards,
Jerry

 

Yes I made that point myself,the test is certainly skewed in favour of the products tested with the latest versions.

 

It does say that it is a long term test started in July, so they used what was available then.

 

About DefenseWall:

I have never used it.
Does it offer complete protection against spyware/adware/phishing etc. ?
A hypothetical example: you visit an infected website (as intended by the owner, or a hacked website) that contains a malicious script, and that script, possibly with a trojan or rootkit, captures your credit card data or other information. Does DefenseWall protect you against that ?
Generally, an AV would. The AV would detect the malicious script, the payload etc. The AV may have some http:/ or link scanner. The McAfee SiteAdvisor would be a bad example of protection against malicious websites.

Next subject: IMO, the matter of security (viruses, spyware) is primarily the responsibility of the user/admin. The user/admin should know what is safe and what is not, and how to deal with uncertainty. No security software will fully protect you against bad decisions. IMO, security software should complement the decisions of the user/admin.
It's safer to 'surf' the web when you know what you are doing without using security software (with the exception of a firewall), than not knowing what you are doing and just relying on security software.

It's really not that hard to learn, and on the long run one saves time, effort and cost if the user/admin learns about IT/internet security.
Unfortunately, few users learn. Any ideas on how to change that ?
Some trust AVs, some choose a HIPS, some a 'tech' solution like LUA+SRP.
Someone stated that LUA+SRP reduce that 'attack surface' by at least 80%.
That still leaves 20 % ...

IMO, it's all about the mind of the user/admin.

I use an AV, but I could probably do without it since I haven't gotten infected for over at least a year.

 

Spyware/Adware/Keyloggers/Rootkits/Trojans- yes, phishing- no, it's your browser's job.

A script can not be with a trojan or rootkit. It may use some techniques to drop maliciuous executable modules (that may contains trojan, rootkit, you name it) at your computer and execute them.

Yes, sure. That's the main point of the project.

Nobody can change it. It's "by design".