How do security vendors differentiate between various malware?

There are various security vendors that produce anti-spyware products that either does or doesn't include anti-virus protection. For example PC Tools and Webroot both have programs that can combine both anti-spyware and anti-virus technology into one product, but you can also buy only the anti-spyware package without the anti-virus protection.

Since a clear difference between different kinds of malware is becoming blurred more and more, and all kinds of malware have characteristics that can either belong to viruses, spyware or other malware.... how do security vendors like PC Tools and Webroot make the decision in what product/suite they are gonna add a certain malware sample as signature? They can either add it only to there combined anti-spyware with anti-virus package or they can choose to also add it to there anti-spyware only package without anti-virus.

Any ideas?

To clarify my question a bit:

Quote from the PC Tools website:

The items marked bold are the differences in detection signatures.

 

Good question. I asked this myself too and I knew ahead, I would never get a clear answer.

Webroot has an AV-box and AS-box. If they get a new malware, they put it in one of those boxes and the choice is based on someone's opinion at Webroot.
The users of course don't know in which box the malware is, so Webroot recommends users to buy AV and AS in order to get a "complete" removal of malware.

Everything about scanners is blurry, insure, .... and I don't like such situation.
So I fixed it myself and now I have a very clear solution.

 

Incredibly blurry indeed, I guess that's one of the reasons why complete security suites are becoming more and more popular, because a clear definition of what specific malware is, is becoming harder as more malware comes out and becomes more advanced.

 

Security suites are as blurry as the rest. They are usually a collection of Firewall, AV scanner and AS scanner, that doesn't make things better.
I call them Frankenstein suites, because many suites are a combination of components (FW, AV,AS) from different companies.

 

i gues its not easy to make an engine(algorithm) that covers them all,so just a litle respect for these developpers who trying hard to keep you safe !

 

I take the same stand as member herbalist here in that if a malware is fashioned well enough it can bring down the entire Suite of firewalls/AS and all and is why i prefer to individualize security with different apps by separate vendors in a classic layered approach.

Just in my own opinion i have more confidence and have realized better results from depending on the experiences individual developers have put into their respective products since they test test them against even POC's which i'm sure commercial types do too, but there is far less potential for the single developers to get lost in a flood of too much at one time, and with a single product they can super fine tune them to be at their very best with the least amount of issues or confusion that plague the bigger players all the time relentlessly.

One point of view. EASTER

 

I remember a thread at a forum I used to work about a PC that was always opening to a page with porn ads. Scans with AVs, both local and online found nothing. Neither did AAW, Spybot, and several other tools. We finally narrowed it down to an hta file, which they send me. At the time, my test PC had every scanner I could get on it. Only one app alerted to this hta. That was Script Sentry, which only warned that the file would open web pages. I checked with different vendors. The AV vendors called it adware or a trojan, depending on which one you asked. Said it was outside the scope of their detections. AAW considered it a trojan, which they didn't detect back then. An anti-trojan vendor called it adware. The end result was it went past all of them.

The pri/sec community has been fighting this classification problem forever. The reality is that malicious code hasn't fit into single categories for years. IMO, classifying malware is an obsolete idea that should have been stopped years ago. Commercial interests have kept this outdated idea alive, primarily to sell users another anti-whatever product, the same interests that still push file scanning and detection by signatures or only detect certain types of threats unless you buy their "pro" version.

The different vendors have never agreed on any standardized definitions for even the simplest of categories such as worms, viruses, trojans, etc. This is easily seen by submitting a malicious file to VirusTotal and looking at the results. It's bad enough that they don't use the same names, but they can't agree on whether it's a trojan, worm, or something else. This has caused nothing but problems for users, especially when the malware requires a specific tool to remove it. If vendor "A" says it takes a special tool to remove a specific piece of malware and the user gets the tool from vendor "B", it's entirely possible that it could be for a completely different piece of malware.

This is just one of the many problems with AVs, ATs, AS, and other "anti's" that use definitions, reference files, etc. Their biggest problem is that they're reactive instead of proactive. If they don't recognize it and don't see any obvious malicious intent in its design, they allow it to run. IMO, that makes them too undependable for front-line defense. All it takes is a new piece of encrypted malware (one that they don't have a signature for) that attacks AVs and security suites to take one down. On mine and the PCs I maintain for my clients, HIPS and a separate rule based firewall are the frontline software defenses, and they're configured to defend each other.
Rick

 

I disagree. AV vendors may be stumbling a bit introducing the needed changes to again fill the gap, but I think as time goes by, you will see they are just as capable of keeping you safe as using multiple products from different vendors.

 

In the last several years, AVs have changed. Whether it's for the better depends on your point of view.
The installers are 3-5 times larger than they were a couple of years ago.
The present installer for AntiVir is over 21MB. Last year it was 11mb. In 2004, the whole package was 3.7mb. AVGs present free version is 36.6MB. In 2006, it was 16MB. In 2004, it was under 10MB. Bloated is an understatement.

They've gone fron using 1 or 2 running processes to 4 or more. They comsume more disk space and resources than they ever have. The performance loss they cause is noticable on most systems. On older ones, it's severe.

Updates had to be made incremental because the reference files are so big they takes hours to download on slow connections. The present detection files for AVG are almost 22MB. For AntiVir, they're just under 20MB. Two or more hours just to download an incomplete list of what you don't want on your PC! With malware kits being sold, this will only get worse.

Many of the AV vendors have started updating several times a day. Some update almost every hour. That does not mean that they can detect new malware within an hour or 2 after its release. I submit the malware that I encounter on the web or that turns up in my webmail box to VirusTotal. The usual results are detections well under 50%. I'm not out hunting for the newest malicious code. Much of it I encounter while searching for something completely different or it just shows up in my mailbox. I've received malware from people I know who don't realize they're infected and don't believe it when I tell them. If AVs were as effective as they'd like us to believe, we wouldn't be contending with huge botnets, created by malware the AVs didn't detect.

IMO, the biggest reason for not relying on AVs for a primary defense is the cost of failure. Malware used to be annoying, intrusive, and very hard on a PCs performance. A lot of it was very "in your face", almost challenging you to get rid of it. Remember CoolWebSearch? Present day malware is careful to hide its existence. The average user would never know it's there. Its purpose is far more costly. Financial theft. Turning your PC into a porn server. Sending spam. Launching attacks on legitimate sites.

It's much harder to detect and remove modern malware. The average AV isn't going to remove a rootkit or one that infects the BIOS. No security software is going to remove malicious code if it infects the firmware. The malware writers maintain and update their code. They test it against the different AVs and security apps. The result is PCs that get infected will most likely remain infected, even with up to date security software. Think about it. If the BIOS or firmware is compromised, even a live CD isn't secure.

The only truly secure OS is one whose components, whitelisted processes, and security policies are strictly "read only", unchangable. Unfortunately, Windows is the exact opposite. Anything goes. A lot of malware can already tell when it's being run in a sandbox or virtual operating system. They will figure out how to break out and infect the underlying system, with the user completely unaware of it. IMO, the only way to keep Windows secure is a strictly enforced default-deny policy for the users, the operating system, and every installed application. The cost of failure is too high to trust your security to a concept that has been obsolete for years.
Rick

 

herbalist, I am disappointed that someone as knowledgeable as you would resort to misinformation simply for the sake of championing your personal ideologies. BIOS and firmware can be corrupted or otherwise rendered inoperable by malware, but they cannot be "infected" in the manner you suggest, and with the newer versions, resetting them to their default states is often a trivial task. I also want to address some other (in my opinion, misguided) points you've made:

1. AVs are bloated. The top vendors detect literally terabytes' worth of malware with only several MBs worth of signatures. I don't think that's bloated; in fact, I think that's amazingly efficient. Their increase in size and resource usage is far, far outstripped by the growth in hardware capabilities, and with several notable exceptions, the majority of them perform well even on low-end systems, since low resource consumption is a desirable product feature that vendors strive for. In fact, I'd have more problems naming products that DON'T perform well on low-resource systems, than naming products that do.

2. If AVs did as well as their creators claim, then we wouldn't see botnets or other malware. This is incredibly short-sighted and ignorant of the dynamics of malware propagation. herbalist, I see you are extolling the virtues of whitelisting in your post to the extent of claiming that the only "truly secure" OS is one that employs whitelisting. Do you honestly think that the malware problem would end once and for all if you took whitelisting to the masses and made it the only option available to them?

Security "experts" have been predicting the death of AVs for years now, and their justifications all read like they're taken from the very same script: that malware writers are evolving to combat traditional AVs and vendors can't keep up with the sheer volume. What many people don't seem to realize is that the battle of attrition is a smaller part of the picture than expected. The antivirus industry thrives on its diversity. Every vendor chooses different parts of a file to fingerprint and add to their signature databases, and every vendor uses different algorithms and techniques for their generic and heuristic detections. Simply put: you can fool some of them all the time, but fooling all of them all the time is a feat achieved only by a very rare few. The antivirus industry itself acts almost like an ecosystem by shoring up each other's weaknesses, with any imbalances quickly rectified by their creators (the antivirus vendors). And unless a virus can defeat this entire ecosystem as a whole, or at least a major part of it, it tends to be quite short-lived indeed.

That, in my opinion, is one of the major reasons antivirus software have performed well, and will continue to do so. Of course, let's not forget that, while they have their own individual weaknesses, the logistics abilities of antivirus vendors far exceed what the general public can conceive. That is also one of the reasons why there's no such thing as a "best" antivirus, for if one would ever emerge to claim such a title and command the market share it would immediately find itself in a precariously besieged position. This was what I think happened to Symantec during 2003-ish, and it's happening to Rising in China now.

 

Infected firmware is a reality. There's already several instances of products being shipped with infected firmware. Pre-infected firmware identified as additional IT security risk factor. This could just as easily be done on PCs. Like any other system update, the BIOS and firmware are updated with downloadable utilities. These could be replaced with infected ones if someone wanted to compromise the server they're stored on. Trusted sites and servers getting hacked is becoming commonplace.

Yes, the BIOS can be reset on the newer hardware, if the user has some reason to think it's been compromised. Modern malware gives very few if any indications that it's there. Just how would the average user determine whether his BIOS or firmware is compromised? I wholly expect that malware will continue to dig deeper into PCs, deeper than the OS itself, into the BIOS, firmware, chipset drivers, etc. This is one instance about which I truly hope I'm wrong.

Efficient?? What is efficient about checking every file and process against against a list that's several megabytes long just to make sure that it's not something that's malicious? It would be hard to come up with a more inefficient method. It's far more efficient to make a list of the desired processes that are part of your system. It's also much safer as the unknown malicious file does not run. Just the fact that malicious code has been created in terabyte quantities tells me just how proactive AVs are not. I'd hate to guess how many infected PCs that translated into before each one was added to their detections.

I am very much aware of how malware is propagated and how that has evolved over the years, enough so to recognize that AV technology has not evolved nearly as fast and has not taken advantage of the advances in technology. Malware writers have made full use of todays high speed internet service to distribute malware and maintain the botnets. Malicious code can be spread almost instantly. Unlike the virus and worms of a few years ago, it doesn't take days or weeks to make their code widespread. What used to takes days can be done in less than an hour now. By the time AVs add a piece of malware to their detections, it's already infected plenty of PCs.

Actually, I said that the only truly secure OS is one where everything is "read only" and unchangable. That's not windows or an installed OS. That's a live CD. Whitelisting the allowed processes is a step in that direction but it's a far cry from a read only system. Yes, I do recommend and use whitelisting and a default-deny security policy. It works. It is so much simpler and easier to keep tract of and allow the hundred or so executables that run on your system than it is to maintain and check against a detection list that contains hundreds of thousands or millions of entries, one that is never complete or completely up to date. As for the masses, as long as users, operating systems, and security packages allow the unknown to install and run, a large percentage of the masses will have infected PCs. Educating users with bits and pieces of advice accomplishes very little. If educating users was effective, we wouldn't be seeing phishing e-mails, infected attachments, etc. Users wouldn't be opening them. The fact that this garbage is in circulation only proves that the users haven't learned the simplest steps to protect themselves, starting with "Don't open that unsolicited junk".

Why would any malware writer need to fool all of them all of the time? Users don't have all of them installed. Most PCs run one AV. A malware writer only needs to beat that one AV one time to own that system. Whether that infection is short term or lasts the life of the PC isn't that important. If it lasts long enough to gain access to your accounts or to harvest a fresh list of potential victims, that's plenty long enough to be very costly.

I want to make one thing completely clear. I have not said that AVs are useless or that the average users shouldn't have one. My position is that they're not dependable enough to be the front line defense. Their weakness is that they allow code that's unknown to them to run. When this is combined with users and an operating system that allows the unknown to run, PCs will be infected, identities stolen, accounts accessed, etc.

A simple policy change regarding how unknown code is handled would prevent a very large percentage of infections if not the majority of them. Setting up a security package that will enforce such a policy is not difficult. Getting the average user to stop opening or installing junk is the hard part.
Rick

 

One need not take on the burden of the masses. Suffice it to work with those who will listen. Whitelisting of all executable files on the user's computer will prevent any other executable from installing without the user's permission.

This takes care of one of the two principal methods by which malware installs: remote code execution.

In another forum there was a heated argument as to the difference between a virus and a trojan. Does it really matter? Whitelisting will block any malicious executable, no matter what its name.

Computer security doesn't have to be complicated.

The second method by which malware installs is when the user is tricked into installing a program which is really malware.

I am not in the camp which believes that educating the users is not effective. It's true, that you can be overwhelmed by the statistics of the misfortunes of the "masses," but why should that influence your own work with people who will listen?


----
rich

_________________________________________________________________________
Just because someone else's shoes are too tight, why should my feet hurt?

 

Reality? What reality?

herbalist, have you taken the time to read that article before quoting it as proof of your so-called reality? I did, and it was an absolute waste of my time. I see nothing but an alarmist "security firm" warning the public about the "reality" of infected firmware from communist China. The basis behind their claim? Some instances of USB removable media being shipped infected with autorun worms in the past. herbalist, please stop wasting my time with ridiculous jokes like these.

In other words, more speculated misinformation. I will have to reiterate my utmost disappointment that one as knowledgeable as you chooses to resort to such methods, herbalist. There are legitimate means to advertise your personal ideologies, and this is not one of them.

You claimed that AVs were bloated, and I was simply refuting that misguided claim. There's no need to shift the goalposts in hindsight by pretending you were talking about something else. There's also nothing "inefficient" about checking every file and process, since efficiency is a measure of input effort against obtained results. The user gets a quite a good measure of protection with minimal (or zero) interaction on their part. Several megabytes of signatures are used to detect terabytes of malware. If those were the other way round, then it would be inefficient; but it's not.

Wrong. Obviously you didn't understand my previous post, if at all. Like all the other paranoid alarmists, you're trying to simplify the situation to virus beats AV = virus owns computer, which is a popular but utterly misguided conception. To make it to that computer, a virus often has to make it through layers of AVs at different checkpoints through the internet, not only the one on the end user's computer that it can beat. A virus has to be unnoticed by the general populace at large if it wants to be successful, only that the problem is that everyone uses different AVs. If a site gets hacked, visitors and providers will find out because they all use different AVs. If a virus spreads via USB drives, it has to get through every single other AV on every single computer to finally reach the computer that uses the AV it can beat. Not to mention that the successful propagation of malware are also dependent on a host of other factors, such as demographics, the presence of exploitable loopholes, how fast can you spread before the AVs update and catch you, etc etc etc. This is why malware writers need to fool all of them all of the time, because writing a virus that beats only one specific AV and then gets wiped out by the collective antivirus ecosystem mere seconds after it is released is so stupid it's funny. What's even funnier is that the pundits who groan and moan about the "death" of AVs actually seem to believe this crap, and the ones they spew.

You also claim that user education is useless. Here's a quick reality check for you, herbalist. ANYTHING without user education is useless, including whatever security methods that you care to praise all the way to the heavens. It all starts and ends with user education, unless you're the owner of their computers and have the legal right to dictate, to the very letter, what they can or cannot do with them.

 

Whitelisting is a nice strategy for those willing to abide by it, but I have found it lacking for my own use. I use a HIPS with a vendor-supplied whitelist. Unfortunately, when I install anything relatively new, it is often not found on the whitelist. So what do I do then - not use it, or find an older version somewhere that perhaps might be on the whitelist? Vendors and download sites often have only the most recent version of a given program. And what if the primary reason I wish to use the new version is that it fixes a security problem?

 

It's clear that you will only see what you want to. It's also clear that you have a full set of derogatory labels for those who don't agree. Malware has managed to make identity theft into big business, making their owners huge amounts of money. They control thousands upon thousands of other peoples PCs and use them to spam and attack whoever they choose, and you call that failure. By your standards, if they don't control the world, they've failed. That's a very twisted definition of failure. The botnets don't get taken down when AVs start detecting the malware that was used to build them. The AVs aren't capable of removing that malware, but you call that failure for the malware and success for the AV vendors? Take off your rose tinted glasses and stop sounding like an industry spokesman claiming that everything will be fixed with the next update. As for infected firmware and BIOS infections, There were a few thousand Google entries for each. Read them yourself.

When I respond to your statements, you accuse me of "shifting the goalposts". You changed the subject each time. I'm not going to waste my time answering each point or addressing each statement of mine that you've twisted.
Rick

 

Don't look at me, herbalist. You're the one presenting vague, unsubstantiated misinformation of infected BIOSes and firmware, presenting redoubtable media sources as "evidence" of your misguided conceptions. I think you're in a poor position indeed to accuse the people who debunk your myths of seeing only what they want to see. Has it ever struck you that you may be guilty of the very shortcoming you so vehemently accuse others of?

And you accuse me of twisting your statements. Well done.

The very simple fact remains that users, as always, play a vital role when it comes to the effectiveness of any software. Whether deliberately or out of genuine ignorance, you're omitting this very important factor and placing the blame squarely on the software, and what's more, you seem more interested in repeating your misguided drivel over and over instead of presenting any scientifically valid statistics that would actually get you somewhere. Of the people who use antivirus software, what are the ratio of infected users to uninfected ones? Of the infected users, how many have kept their software updated, correctly configured, and used other protection in tandem to cover themselves (firewall, antispyware if their antivirus doesn't detect spyware, etc)? The list goes on.

I know it's tempting to simply repeat popular rhetoric, herbalist. It's easy, and it sounds convincing to the unsuspecting, uneducated masses. Best of all, it's hard to debunk because it does contain some truth to it, no matter how twisted to fit your purposes and how irrelevent they are to the situation. Unfortunately, it also gets you nowhere. The Earth isn't flat, herbalist. It's round.

Here's another reality check, herbalist: There's a few thousand Google entries for just about everything under (and above, for that matter) the sun. At this point, I'm no longer surprised that you had to resort to Google for an emergency search of "evidence" to back up your claims, as opposed to you knowing the subject matter before you opened your mouth. And since there's so many; surely it would be a trivial matter to find a credible one that backs up your claims, instead of some vague FUD about communist China developing infected firmware in secret to dominate the world?

 

I specified that I refer to Whitelisting of executables already installed on the computer, which prevents any remote code execution exploit from succeeding - the first method I described for getting malware. For example, this would have prevented being exploited by the 2007 Super Bowl web site hack, or a current banner ad exploit, both of which install a malicious trojan

As far as installing new programs - the second way I described by which malware can install: either you trust your source, or trust what you use to check/scan. I see no difference.


----
rich

 

Very good exchanges gentlemen and good information, i'll back off NOW and let this MEETING continue but i will say this, this Topic is making for a great book of articles i'm personally taking serious note of and it's one of those that will prove very good reading material chalked full of differing and some like comparisons that better brings things to grips on this ordeal we all must deal with.

EASTER

 

In a period of two months I ran most AVs and certainly all the big ones. They didn't detect anything, not even a MRU, except false positives.
That's because I use two whitelists : Anti-Executable (= only executable objects) and FDISR-archives (= all objects). The second whitelist also cleans my system partition in a way I've never seen before.

No AntiVirus scanner can beat an AntiChange scanner and my computer is the living proof of it. Give me any AV/AS/AT/AK/AR/... scanner and I will run it.
I don't only remove known malware, I also remove undiscovered, new malware and unborn malware, including zero-day threats.

No malware can survive in my system partition Isn't that the final goal of security, having a clean system ?
And the beauty of all this, that I don't have to do anything to accomplish this. I only have to reboot my computer, like everyone else does.

 

(partial quote above)

Whitelisting as the solution ?
What if some piece of malware became whitelisted, or a whitelisted program were altered ?

For example, by an exploit. There are plenty of those. See secunia.com. From what I've read, criminals can reverse-engineer a patch to take advantage of a vulnerability in minutes or seconds.

There is social engineering. One way to add a potentially dangerous program to get whitelisted. Or by flash, scripting, etc.

Or perhaps the whitelisting software is bugged. What program isn't ?

Whitelisting can be useful, but it does not provide perfect protection.

 

To kill them all in one sweep,maybe Erik Alberts approach to reboot to a clean slate ?? Its certainly the most hard on any changes including the things you don't like(malware).

 

Ok, noted.

Do you make exceptions for auto-updating programs (including Microsoft Automatic Updates and anti-malware program auto-updates) or simply not use auto-updating? When you install a new program, do you have an exception for a certain download folder, or do you turn off the whitelisting protection altogether temporarily?

Fly's post is correct that a whitelisted program can behave maliciously in the presence of malicious content (whether it's a malicious script exploiting a security vulnerability, a buffer overflow exploit active within a whitelisted process, or perhaps in the future null+offset pointer dereference exploits - see https://www.wilderssecurity.com/showthread.php?t=207023).

 

Please keep in mind the two methods of infection I described. White Listing is the solution for the first method.

In setting up a White List, you assume a clean computer. Then, all executable files are White Listed (there are many besides .exe -- .dll, .ocx, .sys).

(The same applies to creating a clean image, or installing a program such as Deep Freeze: you assume a clean computer.)

At this point after the White List is created, no executable file can install as long as the protection is enabled. This takes care of any remote code execution vulnerability which attempts to install a malicious executable, meaning that even if the system is not patched (a MSWord exploit which has an embedded trojan, for example) that trojan cannot install if the user is running as LUA, or with SRP enabled, or with a 3rd party program such as Process Guard or Anti-Executable.

Not in the case of social engineering, of course, because the user disables the protection to install what is considered to be a legitimate program. This falls into "user education" which I covered in my post #12.

Yes, that is very interesting code manipulation. Will a real-world exploit fall into a remote code execution exploit, or depend on social engineering? The preventative measures are different for each case.

White Listing protection is turned off, the new program is installed, protection turned back on and the program is added to the White List.

This falls into the second method by which malware can get installed: user gets tricked into installing a malicious program. Its evident that White Listing is not intended to protect in this situation.

It seems to me that there are two alternatives here:

1) you trust your source

2) you trust a way of verifying (checking/scanning).

I don't see any difference between the two.

Each gives the user some degree of confidence in allaying the fear of installing something malicious. Each user has her/his own levels of uncertainty and fear about such things.


----
rich

 

Exploits are also objects. You can't exploit a whitelisted object without installing something else in a system partition, so exploits are removed just like any other malware.

 

All you need to do to expose yourself potentially to malware is merely surf the web. From http://www.computerworld.com/action/article.do?command=viewArticleBasic&articleId=9058599, we learn that "the majority of Web sites serving up attack code are legitimate domains that have been hacked by criminals." An example cited by http://windowssecrets.com/2008/04/17/02-Flash-ads-bearing-malware-plague-popular-sites is that merely visiting the USA Today website recently exposed you potentially to malware, as a malicious Flash ad appeared on the site.

Unfortunately, buffer overflow exploits are all too common. And when they happen, the exploit code executes within the whitelisted process. Whitelisting will not stop this. However, whitelisting will stop downstream effects, such as if the buffer overflow exploit code attempts to download and run a malicious .exe to continue the attack. (The space available within the buffer is often limited.)

Using Returnil or similar will indeed restore the system partition upon reboot. However, changes to other partitions are not covered by Returnil. Also remember that stolen data cannot be rolled back by a reboot in Returnil.