Testing methodologies concerning scan engines

Well, technically thats actually true. behavior blockers don't care about cryptors, packers and other anti detection mechanisms. They simply act based on actions caused by the malware.

 

yes, finally someone who uses his mind in the right way without flames and prejudices.

Great another one.

 

Well, we should evaluate separately pure antivirus engine from behaviour blockers. It does not matter if both pieces are integrated in the same software.
On access: antivirus engine + behaviour blocker
On demand : antivirus engine only.
EDIT: To clarify

Thanks IBK.

 

Thanks, it is good to hear that.

I wanted to ask a few questions of you, if you don't mind

1) You said that it was easier to circumvent NOD32's heuristic compared to Kaspersky. Does that mean that NOD32 fails to unpack some of the crypted/packed files in your testing?

2) I am interested to know why you say BitDefender is the best scanning engine. Is your decision based on proactive protection/heuristics alone or have you also taken other characteristics of the engines in account (such as unpack engine, emulator support etc.)?

3) Can you explain your opinion on Dr.Web in a bit more detail?

 

Re: NOD32 2.7 Vs Symantec AV Corp 10 Vs Kaspersky 6

Could you please elaborate how they circumvented NOD32's heuristics in the case of this mass-spammed downloader? As I said, I need a proof and not just trolling.

~ snipped unneeded Virus Total results ~ LWM

 

I know moderators are allowed to post virustotal reports and members aren't but does this still not apply

P.S. I do not agree with Systemjunkie's 'theory' at all, but posting a banned vt screenshot doesn't disprove what he says. He needs to supply proof first (which I don't think will happen).

Regards,
Londonbeat

 

Well, I suggest that we wait for his reply before making conclusions.

 

You guys should keep in mind that a heuristic's primary job is to detect NEW unknown malware. That a heuristic can support the signature scan in overall detection of already known stuff is a NICE SIDE-EFFECT but not the main task of an heuristic! Why? The answer is simple: Because you could have added a signature for the already known and old stuff. Period.
For instance i would rate a AV Heuristic "better" when it's doing well with IMPORTANT new stuff rather than having a moderate overall detection. The magic word is again PRIORITIES. You have to adjust heuristic almost weekly/daily to cover new mailware trends / methods. This task is by far more "difficult" than trimming a heuristic on an already existing testbed of "old" files.

And just for the note, NOD32 is doing there well. Example:
http://blog.washingtonpost.com/securityfix/2006/08/spammers_exploiting_latest_mic.html

 

Well it's all based on nature or shall we say method of detection. You have far more options if you can pack the stuff with some obscured packer and crypter and simply choke up the emulator, rather than circumvent the behavior blocker which actually works on higher level than regular heuristics. Anti emulation, packers and cryptors won't fool the behavior blocker, but there is much higher chance to do so with non behavior blocker mechanism. I'm not saying it's not possible, but you certanly have far less options...

 

That is out of question. Full ACK. HOWEVER ( there's always a however ) Behavior blockers need user interaction. That said a behavior blocker might be annoying if not developed in a proper way. And there we encounter the next "problem" you cannot list every suspicious action since other legal programs might have to use such things as well. I think you see already the duck running... You have to exclude a few things otherwise this behavior blocker will drive you nuts. Now if you exclude something... You get the idea i think. The current problem is that you rarely get any MEANINGFUL text from this blockers WHY a program is considered to be suspicious. And please keep in mind that not everyone is an compter expert... Something like "tries to inject code and creates remote thread" is useless for a normal skilled computer user. He just might click on "go ahead" and starts the malware...

 

Hello,

Very well said, Inspector!

On a side note, this, in part, is the reason why I think that if a person is NOT an expert yet has to deal with security, you might as well land him with the best available least screwable option - Linux.

Once configured, it's the best choice for non-knowledgeable users. They would not know where to start screwing things up. Today, anyone can just about any file in the system32 folder and bye bye operating system.

Mrk

 

Some may call it advanced democracy.

Best regards,
Firefighter!

 

Here is a word of advice folks, forum discussion of moderator actions are not open to public debate, period. If you need any further expansion on that, PM to myself or an Admin.

As to the posting of virustotal and similar screenshots, I suggest you perform a forum search to assess what is and is not banned. They are not tolerated as a part of pointless threads endlessly discussing why or why not Product A does or does not cover a specific file - which may or may not be malicious - which is flagged by other products. They may be appropriate in some very limited circumstances, which may include (note - I've said may, depending on the specific thread context, may not could also apply) attempting to diagnose the circumstances involving a specific event, condition, and/or file or to focus a discussion. As for having an all-inclusive enumeration of all possibilities in which these screenshot are allowed and disallowed, that is not about to occur since these eventualities can be strongly context dependent.

Let me close by repeating, forum discussion of moderator actions are not open to public debate.

Regards,

Blue

 

There are several ways to satisfy the customer's will, open mind policy or something else.

Best regards,
Firefighter!

 

Thanks IC, your response may be understand how things work, a lot better. Again, thanks.

 

As always IC enlightening the masses. Thanks.

~O/T comments removed...."forum discussion of moderator actions are not open to public debate"....Bubba~

 

As IC said, behaviour blockers are too noisy. This is the reason why we have smart code analyzers = AV engines.

 

Well KAV in Normal mode is far from "noisy". Switching it to Advanced however makes it "noisy"...

 

There are so many reactions and questions that it may be a bit difficult to keep the red line..

I tested mainly the pure scan engine power in terms of packed, unpacked and crypted malware. The Malware I used in my tests was mainly focussed on trojans and rats, few other malware. It was a very specific crypt/pack test. Also other simple methods of circumventing AV engines, most of these methods are totally out of date and oldschool, but many AV still aren´t immunized against them to my surprise.

Not directly, it has some lacks on very simple oldschool methods. But in some cases it is also the problem you mentioned above.

The big advantage of Nod is the advanced heuristic detection, less false positives if you compare it with dr.web 4.33, nods heuristic is lots more reliable.

I was very impressed about dr.webs detection capability of sql slammer worm. This little nasty survived for a long time on a old system and only dr.web could detect this bodyless malware.

I tested most av products for the last 3-5 years regularly all 3-6 months simply because I am very curious, so the most interesting step was realized by bit defender in the last 5 years (related to extremely deep heuristic detection and unpacking engine). AntiVir actually realizes huge steps with some remaining bugs. Kaspersky was once the top but lost lots of position during the last 3 years in my tests. Nevertheless Nod32 made also big steps and ranged mainly at top3 of the list in the last 1-2 years. The russian engines are very agressive in detecting wellknown modified servers, such as kaspersky.

 

Hi are you actually technically qualified to know how to test scan engine methodologies(ie the actual methods used)and understand their workings or are you just some"normal" user throwing malware(could be real,corrupted or not malware at all)at various AV's and listing results in a kind of table?
For all we know,because you will not disclose your methods,you could be misinterpreting the results due to the samples used:-if your so far ahead of the game why not make yourself some money from your "knowledge":-you should be able to design a better scan engine than all current ones as you so fully understand the weaknesses of all the ones that are currently available
No offence intended but getting a bit fed up of sudo experts making wild claims!

 

I'd forgotten about this till reading this thread:

Interesting and maybe relevant to some of the discussion here:

http://www.morgud.com/interests/security/faceoff-2006.asp

 

I would spent two words about this thread, if possible.

As Inspector Closeau said, it's not really THAT important that heuristic doesn't find re-packed, crypted executables. As everyone knows, this way to do is well known since AGES and you'll find tons of ways to bypass almost every heuristic engine and even - of course - signature scanning engine.

But this is quite normal, it's easy to bypass what you can see and study, is more difficult to prevent what you can't see - and the best you can do is adapt heuristic everyday to new threats.

The thing that's really important is that heuristic can find REAL THREATS, that are widespread, and block them immediately. And you can see that Nod32 blocked lots of new bot and mm worms during this year and past years.

Then, of course, blocking targetted attacks is a bit different thing, more difficult. As said before, you can find tons of targetted attacks that bypass every antivirus solution, just because attackers and virus writer can study defense software, while antivirus developers can only think HOW attackers COULD attack.

About HIPS: HIPS are, for nature, noisy software. If you don't find any solution to well develop this, you'll have a noisy software. It's almost the so-old history of "war" between security and performance (or user friendly sw). You can "easily" make less noisy a HIPS software, so user will be happier and less noisy, but of course you'll penalize a hips software and its nature. On the other hand, you can have a noisy sw that alert about all suspect actions but ANY user will use it.

Behaviour blocker is a powerful technique but need to be well developed, otherwise there's only high risk to add confusion to the danger of malware.

 

Yep, you are very right, probably too much knowledge can freeze your motivation of doing so.. or the reason might be modesty and a kind of unexplainably satisfaction not striving for high aims. I strived and flew very high in the past, but in other fields then it security, probably burn out symptoms..

But to calm your mind I already work on such a solution , probably I am not able to stand still for a very long time. Work-aholic may come back

yes, yes, true, true

 

dont be shy:-let us know what and who you worked with/for!!!
Sorry to be blunt but think you are a bull******,you didn't answer my question re qualifications,just claimed to have too much knowledge!!you can never have too much knowledge!:-Bored with this now!

 

Naturally you can have too much knowledge in case you don´t empty the cup.

It´s a high art to empty the cup. If the cup isn´t empty you will stuck.
That means too much knowledge.

I have no problem with aggressivity I like aggression, me to could react aggressive but my wisdom does not allow me to use your primitive method actually. ~removed inappropriate personal remark IAW our TOS....Bubba~