NEW! Rootkit 'detection' test

ou could look at it that way, but if a product struggles to get the small sample set, what chances has it for the world wide web amount?

That's why percentages are used.

 

Like how well DrWeb performs in AV-C?

The smaller the sample set, the more inaccurate the overall results. I'm sure you know perfectly what cruelsister is talking about, Chris. No need to play dumb.

 

I'm not playing du b solcrift and I do know what he is talking about....

but sample sets sooo large offer absolutely Zero! Information to know if product A will protect a user or not, yes they are credible tests I'm not disputing that,

why do tests on smaller tests offering real threats give differnt scores, not just for drweb but for the majority?

You think its just luck, or pick of the draw?

Sorry for typos, not
on my computer.

 

And you think a test with 6 virii will?

Since small tests are statistically meaningless, both, probably.

 

you could say the same for both, this is why I never rely on tests.

But you and me both know, many ppl on here do.

 

Hello,

it depends of different things, for example the unhooking test of nicm with 7 samples was really good, even with just 7 samples...

 

An entirely different matter altogether. nicM was testing the ability of HIPS to defend against specific intrusion techniques.

The reason why this is not even remotely applicable to AV testing should be obvious.

 

re,

you miss the point; the quantity does not make the quality. You don't need 1xxx rootkit to know the ability of an av to detect them.

 

Assuming those 1xxx rootkits behave exactly the same and have the exact same code, then no, I guess not.

 

But don't they come one at a time? If so, that one is what the AV has to combat. The other 100K are not of interest at that particular time.

I do realize that a large sample size is more reliable over all, but if the one it misses is the one that attacks me, what do I care if it would have blocked 50K others?

As for the test, I'll just limp along and hope my applications keep me safe.

Regards,
Jerry

 

The problem is: how do you know which one.

Actually, that's exactly why you want a large sample set. The larger the set, the more accurately the test reflects how likely a product is going to block any one sample that attacks you.

 

I agree with that, but would not ignore the one that it missed if that were the one that attacked me.

It is true that if an AV detects 99% I would rather have it than one that detects 89%. But if I have the 99% it can miss 1% of the malware. Have we not said that no AV can detect 100%?

My only point is that although the test samples were few, that does not disqualify the test. It just makes it less important due to that small sample size. But even then there were those that detected more than others.

There are those who say that the test is not any good due to the small number of samples, but there is still a ranking according to the detection rate.

Regards,
Jerry

 

That's why I believe that a good security policy should implement, in order of importance:

1. A sound backup strategy
2. Separate operating environments for different tasks.
3. Antivirus, antispyware, firewall and other "security" software.

 

Just because you were attacked by that 1% of the malware that antivirus X does not detect, doesn't mean it's test results are inaccurate.

You need to distinguish between what a test is and is not telling you. It's telling you the percentage of malware can detect. It is NOT telling you which malware you will get attacked by; this is absolutely nothing at all to do with tests, they were never designed to do this right from the very beginning, and I'm not sure why you're even bringing it up in a discussion about tests.

Actually, yes, it does.

Claiming anything about the performance of a solution due to its detection rate in a test of six samples is nothing short of ludicrious. For all we know, a solution that scored 100% might be able to detect only those six samples and nothing else, while a solution that scored 0% might be able to detect everything else save for those six. Such a test is not less important - it's entirely worthless, since nothing remotely useful can be inferred from its results one way or another.

 

LOL, I remember the first posts I read here: some members listed all the "security" software they were currently using in their signature, even distinguishing between realtime and on-demand apps.

It reminded me of those military dictators with all those medals on their uniforms.

Somehow, it seemed that the focus for some shifted from the quest for true security to how clever they are to be able to make all these different apps from different vendors work together seamlessly.

 

Thinking about all these security tests, a similarity came to me.
They remind me of IQ tests.
IQ tests indicate potential, but provide no guarantee of anything whatsoever!

 

Reminds me of my old habits.

Pile them all in one big heap and if theres no BSOD or performance bite, hey must be good to go. Then let the malware try to navigate thru all that, which of course without HIPS they probably could slip a sliver or two through. Did seem the Logical approach at one time but there is a point where so much is just simply too much, but today with so much better advancements in security technology in the form of Virtuals, Sandboxes, ISR's, HIPS, and so forth, it only takes a choice few to sit secure behind REAL SHIELDING anymore.

 

Yeah, wow.
Thanks for the insight.

EDIT: Yeah, but how necessary is "REAL SHIELDING" for the average user?

 

Exactly

That sounds nice but not if your whole system was already blue-pilled or vbootkit´ed before. (remember you can never be sure, there are millions of possibilities) Then all shields are in vain. Naturally you can reset to zero after reboot but the things you do in the net will be caught no matter if virtualized or not.

 

EXTREMELY IMPORTANT!!!​

It's the so-called "average user" who is by nature very vulnerable to the state of mind that they have an AV + Windows Firewall and so everything will be just fine. Push and press then carry on mentality while the missed intrusive files begin their mischief on disrupting and/or disabling the normal components of the operating system.

But of course you already know that, right?

 

There are users who do perfectly fine with just an AV and Windows Firewall, and there are those who don't.

Doesn't this already give you a hint that user education is what really counts, not how many security apps one can squeeze into their forum sig?

 

What good is user education AFTER THE FACT?

I'm speaking of newer exploits, not to mention Windows own limitations. Surely not even you are so naive to believe that just those basic coverages are secure.

I will tip a nod in favor if they run LIMITED, (LUA), XP speaking of course.

 

The same good it is for everything else. To prevent it from happening again. Of course, educating the user before they get infected is also a desirable option.

Too many people confuse security programs for security. You can give fifty of the best guns in the world to a shitty marksman, and he'll be able to do jack squat with any and all, other than feel good with the fact that he's armed to the teeth.

Actually, I do. So do plenty of other people, and they have no problems remaining clean.

 

Indeed.

Focus has shifted? As far as I know it as always being like that.

The only thing is that in the past, people differentitated between products classes less, so people only tried to get say firewall + AV working.

Then people discovered Antispyware, classic hips, sandbox, behavior blocker, virtualization etc....

So now most people try to juggle all this together.....

 

Statistically you are correct, but if you get infected by one of the six your AV is 0 protection to you. The odds of a meteorite falling on me is 1 in many millions, but if one does it is 100% for me.

I am happy to know what mine won't detect. However, I would not use a certain AV or change as a result of such a test. I would not rank AVs based on such a test, but if I get infected by the one it missed I don't care about the 100K it would have caught.
That is the reason I only depend upon IBK and AVC.

Anyway I'll leave it at that.

Regards,
Jerry