Testing of Infected computers

[C.S.J]

..... for 'that most complicated' malware.

Test (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dnews%26newsid%3d328%26arc%3d0)

Results (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dactive_infection1)

Methodology (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dpackers_methodology2)

Discuss (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2fphpbb%2fviewtopic.php%3ft%3d3474%26sid%3d7972f71823bcd12d280b8c97c2112ca3)

your thoughts?

{QUOTE-> The results of the carried out test showed that of 15 thoroughly tested antiviruses only 6 they possess the effective means of the treatment of the active infection of system, i.e., in the majority of the cases they are capable of successfully liquidating infection.

During the treatment of the infected system with the most effective antivirus it proved to be Dr.Web Anti- virus 4.44, which deservedly received high award Gold Malware Treatment Award

It is desirable to note the difference in the results of the test between Dr.Web Anti- virus 4.44 and previous version Dr.Web Anti- virus 4.33. The latter failed test, after knowing how to cure only 2 infections of 17.

Products Kaspersky Anti- virus 7.0 and Norton AntiVirus 2007 also showed high results (71%) and was obtained reward Silver Malware Treatment Award.

Other three antiviruses, among which Panda Antivirus 2008, Avast! Professional Edition 4.7 and AVG Anti- virus 7.5 showed average results in the treatment of active infection (from 59 to 47%) that it corresponds to level Bronze Malware Treatment Award.

The remaining thoroughly tested antiviruses, alas, appear sufficiently palely and it is not possible to consider them effective in the fight with the contemporary virus threats. <-QUOTE}

[fax]

{QUOTE-> ..... for 'that most complicated' malware.

Test (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dnews%26newsid%3d328%26arc%3d0)

Results (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dactive_infection1)

Methodology (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dpackers_methodology2)

Discuss (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2fphpbb%2fviewtopic.php%3ft%3d3474%26sid%3d7972f71823bcd12d280b8c97c2112ca3)

your thoughts? <-QUOTE}

Very interesting.... but is it enough a test with just 17 sample?
Were sample choosen randomly? It does not look like they were....

At least they should have tried a random selection of sample in more rounds following the criteria they describe just to be a bit more robust...

Cheers,
Fax

[C.S.J]

hi fax,

read this (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dviruses_choice2), it explains why each threat was selected.
im happy to see a massive improvement on removal for drweb, between the versions. :)

[fax]

{QUOTE-> hi fax,

read this (http://babelfish.altavista.com/babelfish/trurl_pagecontent?lp=ru_en&trurl=http%3a%2f%2fwww.anti-malware.ru%2findex.phtml%3fpart%3dtests%26test%3dviruses_choice2), it explains why each threat was selected.
im happy to see a massive improvement on removal for drweb, between the versions. :) <-QUOTE}

Hi C.S.J.
Indeed I have read it... of course its an interesting test however I doubt that it is really rappresentative. We have seen many examples in the past about tests with limited samples . But this is more for virus experts to judge...

Regardless the number and kind of malware choosen, an antivirus main function is to prevent infection not to clean it afterwards. Of course, users that cleans PC for work, for hobby or for friends will find this test most useful.

Most interesting information I can extract from this test is that generic antivirus products have evolved very much recently and are able to deal with spyware and adware as good as mainstream dedicated spyware scanners (though they were not included in the test).

Cheers,
Fax

[kinwolf]

I concur that 17 samples is far from enough. It is too easy to pick up 17 samples that would make antivirus X looks good.

Kasperskogo.. lol, funny translation

[C.S.J]

{QUOTE-> I concur that 17 samples is far from enough. It is too easy to pick up 17 samples that would make antivirus X looks good.

Kasperskogo.. lol, funny translation <-QUOTE}
but this is removal, not detection, its alot harder to test against,

detection is simple numbers, and work out a percentage, removal is much harder to do, which is why there is not many removal tests around.

{QUOTE-> "Kasperskogo.. lol, funny translation" <-QUOTE}

I know, and 'doctor the web' i thought was quite good aswell, never heard of it like that before :)

[the Tester]

{QUOTE-> I concur that 17 samples is far from enough. It is too easy to pick up 17 samples that would make antivirus X looks good. <-QUOTE}

I agree. 17 samples is not enough.

[EliteKiller]

For some reason I feel like I am looking at a malware-test review. :-\

[C.S.J]

{QUOTE-> I agree. 17 samples is not enough. <-QUOTE}
i remember when IBK once posted (http://www.wilderssecurity.com/showpost.php?p=1069349&postcount=185) a link to an interesting self protection test, with 'not many checks' or whatever, and he is a professional :)

and yes, this removal test is from the same people.

[Peter2150]

What a shocker. Dr. Web came out on top. Honestly test like these are almost meaningless.

First is the problem of the samples, then the test methodology, etc. What does it really mean for the average user. Probably not much. My hunch is that the safe surfer won't get in trouble with the worst AV, and the person who does all the foolish things with the computer, probably wouldn't be protected by the best AV, which ever that might actually be.

[Technodrome]

{QUOTE-> For some reason I feel like I am looking at a malware-test review. :-\ <-QUOTE}

But you are not. ;) These guys are well respected testers (in Russia at least).


tD

[C.S.J]

{QUOTE-> What a shocker. Dr. Web came out on top. Honestly test like these are almost meaningless. <-QUOTE}
yeah, i must have wrote it all then. :thumbd:

thanks TD.

[Technodrome]

{QUOTE-> Honestly test like these are almost meaningless.
<-QUOTE}
Every single test is meaningless! Period. But once a while we get a chance to look at these tests from a different angle with different prospective. 8)


tD

[Firecat]

It would be interesting to see Dr.Web 4.44 tested in one of AV-test's removal efficiency tests.

This one is definitely interesting to look at though! Indeed, Dr.Web 4.44 is a significant step forward for Dr.Web in some respects. :)

Maybe I'll post some more thoughts later, currently I am in the middle of studying for an important exam! :)

[C.S.J]

The result is meaningless to me, but the differences from 4.33 and 4.44 is quite amazing from a removal point-of-view.

{QUOTE-> It would be interesting to see Dr.Web 4.44 tested in one of AV-test's removal efficiency tests <-QUOTE}

i know, but they seem to only do these removal tests for the big players like norton etc etc.

[the Tester]

It's good PR for Dr.Web.
Dr.Web is the only program that had two versions tested.
It reads like an advertisement for Dr.Web.The headline for the test should read "Look how much Dr.Web has improved!" And I'm sure that they have had some improvement between versions.
But this test has little value unless you like Dr.Web.

[C.S.J]

{QUOTE-> It's good PR for Dr.Web.
<-QUOTE}
good, cos drweb never gets good PR, especially not in this place. ;)

others did well aswell like the nortons and kasperskys of the world, they got the awards, but loads failed, and its an interesting read into removal.

since when does removal ever get tested, not v.often.

[Firecat]

I am wondering why F-Secure is so much worse than Kaspersky. Are the differences between KAV 6.0 and 7.0 that significant (F-Secure uses KAV 6 engine)?

[lodore]

{QUOTE-> i remember when IBK once posted (http://www.wilderssecurity.com/showpost.php?p=1069349&postcount=185) a link to an interesting self protection test, with only 7 samples, and he is a professional :)

and yes, this removal test is from the same people. <-QUOTE}
Hi chris,
the link IBK posted to was showing how good the self protection of the antiviruses are. not 7 samples.
lodore

[IBK]

I think this test is not meaningless. It shows something.
Esp. the difference between two versions of the same product.
And most cleaning tests are done on a very limited number of samples. The selection of samples seems to make sense.

Greetz from Vienna!

[C.S.J]

edit:

yeah lodore, i just meant it doesnt test much but is still a very good test.

this time, its removal not self protection.

[trjam]

{QUOTE-> I think this test is not meaningless. It shows something.
Esp. the difference between two versions of the same product.
And most cleaning tests are done on a very limited number of samples. The selection of samples seems to make sense.

Greetz from Vienna! <-QUOTE}
Then by all means, this is interesting. I have been preaching a long time that detection is only part of the equation. That if a AV cant clean, it is crap. Eset, Avira and a hosts of others fall into the crap of cleaning catagory. Dr. Web has always been know for its cleaning ability as Kaspersky, Norton.

Now on the flip side Chris is, you also have to be able to detect it, before you can clean it.:dry:

[trjam]

Norton and Kaspersky still rule the cleaning and detecting group. But I think Eset, maybe Avira and others are going to start closing the gap. Dr.Web just needs to go in the other direction, detection and it willb e fine to.

[C.S.J]

{QUOTE->
Now on the flip side Chris is, you also have to be able to detect it, before you can clean it.:dry: <-QUOTE}
i think ive made my arguments on that already and we aint going down this road again trjam.

i hope bitdefender 2008 improved its cleaning though on v.10

its also shocking that nobody recieved the platinum award, it just shows that all talk of detections and features dont mean everything.

i knew 4.44 improved its removal compared to 4.33, but seeing it in this test has eased my mind a little bit.
its also nice to see 4.44 being the only one to remove the rootkit, i was never sure about the rootkit thing, but again... eased my mind.

-------------
@jeff

sure removal means nothing if it isnt detected ;)

but, the same applies to detected threats that cant be removed, right? :D

it is a shame it doesnt mention what these threats actually did, would have been nice to know, and what the product tried to do against it, even if not completing cleaning it.

[EliteKiller]

{QUOTE-> I am wondering why F-Secure is so much worse than Kaspersky. Are the differences between KAV 6.0 and 7.0 that significant (F-Secure uses KAV 6 engine)? <-QUOTE}
That is one of the reasons behind my previous post. ;)

[Diver]

To understand what is going on here, you have to attempt to cleanup a badly infected machine using a series of different AV's. Pretty much installing one trial after another. Its not the best way, you should really get the install disks for the machine and start from scratch, but they are not always available. (Those who are careless enough to mess up their computer have usually lost the install media.)

As you go through this process it is very revealing. AV #1 catches bunches if stuff, but cant remove it. AV # 2 gets some more. AV # 3 says everything is OK. AV # 4 finds some more stuff, but it looks to be inactive remnants, possibly from the quarantine areas of AV #'s 1 & 2. Then you notice Windows Update and the firewall are broken.... and so it goes.

My point is, it is not theoretical. You really have to get your hands dirty to get a feel for what is going on.

[btman]

Thanks for posting this C.S.J. I didn't read too much but the results... But were these done in safe mode? Or would that have caused all of them to be cleaned by the AV's. Just wondering as whenever something can't get removed the first word of advice is usually "Run scan X again in safe mode"

[Zombini]

Go NAV.. and its not even the 2008 version.

[IlyaOS]

The previous active infections treatment test by Anti-Malware.ru was translated and published on website http://www.anti-malware-test.com/?q=node/3
As I know the latest one will be published there after translation.

[fax]

{QUOTE-> The previous active infections treatment test by Anti-Malware.ru was translated and published on website http://www.anti-malware-test.com/?q=node/3
As I know the latest one will be published there after translation. <-QUOTE}

Very nice test on termination and tampering Sergey...
Not sure it was discussed here but interesting!

http://www.anti-malware-test.com/?q=taxonomy/term/16

However, I personally disagree on the the ranking....
I couldn't care less that the junk mail filter in ZASS 7 can be disabled by malware, its not part of the ZASS main defence mechanisms.

If secondary modules like spam, parental control, etc... need to be included in the termination test, I would expected a weighting system that would give less relevance to these elements while firewall, antivirus and antispyware protection should have higher weighting.

The above should reflect the extent of damage that real malware could cause on a system. Disabling spam module has no effect on the protection and integrity of my system if firewall, antivirus and main 'security' related functions remains intact.

Without this weird point system ZASS (and may be other suites?) would have a completely different scoring.

Cheers,
Fax

[C.S.J]

{QUOTE-> Thanks for posting this C.S.J. <-QUOTE}
no problem, :)

i did expect some bashing towards myself, as it was i who posted it, but aint too bothered, most people liked it. :)

[trjam]

I thought, as IBK did, it is a very informative thread and post.

[IlyaOS]

{QUOTE->
However, I personally disagree on the the ranking....
I couldn't care less that the junk mail filter in ZASS 7 can be disabled by malware, its not part of the ZASS main defence mechanisms.
<-QUOTE}

I agree with you. The ranking system is "too hard", but nevertheless the antispam module in ZASS 7 can be disabled. It's weak and not protected part of the product.

{QUOTE->
If secondary modules like spam, parental control, etc... need to be included in the termination test, I would expected a weighting system that would give less relevance to these elements while firewall, antivirus and antispyware protection should have higher weighting.
<-QUOTE}

Nice idea, i hope it will be done in the next text of Anti-Malware Test Lab.

[fax]

{QUOTE-> I agree with you. The ranking system is "too hard", but nevertheless the antispam module in ZASS 7 can be disabled. It's weak and not protected part of the product.
Nice idea, i hope it will be done in the next text of Anti-Malware Test Lab. <-QUOTE}


Yep, but then you should not call it "Antivirus Termination Tests" but "Suite Components Termination test" and have a breakdown by component been tampered.

And for the aggregated scoring you could use a weighting system according to the importance of the tampered component.

In any case, thank you for taking up the idea of a different scoring systems.

Cheers,
Fax

[Escalader]

{QUOTE-> I think this test is not meaningless. It shows something.
Esp. the difference between two versions of the same product.
And most cleaning tests are done on a very limited number of samples. The selection of samples seems to make sense.

Greetz from Vienna! <-QUOTE}

Hi IBK:

Two things:

1) Will av comparatives be doing more in the future on removal/cleaning testing?
2) You mention most cleaning tests, are there any links to other results?


It's interesting to read the circular logic here by the "fans" and others.

1) Can't remove what AV can't detect!
2) Can remove what AV does detect but doesn't!

What about our previous rants about FP's! (or if you like "fictitiousness".)

The weight on selection of tools should IMO be on prevention, eg the FW, the HIPS. Virtual PC.

" an once of prevention is worth a pound of cure" or for the metric guys

an gram of prevention is worth a kilogram of cure.;D

[Escalader]

{QUOTE-> I agree with you. The ranking system is "too hard", but nevertheless the antispam module in ZASS 7 can be disabled. It's weak and not protected part of the product.



Nice idea, i hope it will be done in the next text of Anti-Malware Test Lab. <-QUOTE}

IlyaOS:

You are right to worry about any security product that allows malware to disable any module.

It raises the specter of what other modules in it can be undone. Only the developers can know what is protected and what isn't no one here can tell you the answer to this.

So IMO, the safe bet till it is proven by independent testing is to assume that all modules are vulnerable. It's a matter of confidence.

[Severyanin]

{QUOTE-> Very interesting.... but is it enough a test with just 17 sample?
Were sample choosen randomly? It does not look like they were....

At least they should have tried a random selection of sample in more rounds following the criteria they describe just to be a bit more robust...

Cheers,
Fax <-QUOTE}

Yes, they chose them randomly, it seams.
17 is quite a number for such tests - but you are right, this is a Lab test, not a field test in a real life.

[fax]

{QUOTE-> Yes, they chose them randomly, it seams.
17 is quite a number for such tests - but you are right, this is a Lab test, not a field test in a real life. <-QUOTE}

Thank you Severyanin for the follow up...

{QUOTE-> So IMO, the safe bet till it is proven by independent testing is to assume that all modules are vulnerable. <-QUOTE}

This sounds a bit drastic... IMO :o

Fax

[Escalader]

{QUOTE-> I agree with you. The ranking system is "too hard", but nevertheless the antispam module in ZASS 7 can be disabled. It's weak and not protected part of the product.



Nice idea, i hope it will be done in the next text of Anti-Malware Test Lab. <-QUOTE}

I agree with you IlyaOS.

But again IMO.

The opening of spam and their attachments is not a trivial low weight factor. It is still one of the more common way the public gets infected.

The users of suites hope for one stop security solutions, but are the same users who trust one vendor more than they should and now will get stung by a disabled anti spam module.

Again, parental controls are NOT trivial protections to get low weights in suite evaluations. Why assign a low weight?

Kid's have been lured away by lack of parental control, so to label this feature less important is completely .... what word fits... wrong headed. Safety trumps concern for vendor reputation every time.

Until these features work an ethical vendor would remove it rather than give a false sense of safety.8)

[fax]

{QUOTE-> I agree with you IlyaOS.
The opening of spam and their attachments is not a trivial low weight factor. It is still one of the more common way the public gets infected.

<-QUOTE}

Sorry but known malware will be blocked by your "on access" scanner or e-mail AV scanner regardless if spam module is working or not... the contrary may not be true.

Of course if your AV module is also down then its another story... do you understand what I mean? Its a matter or setting priority and underdstanding the real effect that malware can have on your system.

The same applies to parental control.

But if your base for assessment is not to simulate the damage that malware can do on your system then everything is relative and it would be better not to have aggregated scoring but keep the analysis at component level.

Everybody will have their value about what is important and what is not. All with their merits and limits. You just need to make them clear at the beginning as part of the methodology you adopt for the testing.

Of course, always IMO.... :)

Cheers,
Fax