AV-Comparatives June (May 2007) Results (Retrospective / Proactive Tests)

Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

This may be a silly question, but I'll ask anyway.

Assuming the "new" malware is functional and capable of delivering its malicious payload, why is it unethical and/or unprofessional for an antivirus tester to create such "new" variants for heuristics testing purposes?

 

Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

It's not really a silly question, and there are probably many answers depending on perspective:

• On the ethical front: there's no way anyone (read vendor or fellow tester) can reproduce the results for validation or provide follow-up without getting access to the samples. As access to the samples set is granted, the chance of a release into the wild increases. It simply sets the stage for these samples to be real malware at some future date. Given the alternate approaches available, why risk the downsides of actively contributing to the problem as well? Finally, you set up an ethical dilemma for the vendor - do they spending time developing schemes for the synthetic malware to garner a high test score, or do they serve their customer base by dealing with the real material?
• On the professional front: Does the test reflect anything connected to reality? If I were to whip up 1000 pieces of malware and throw them at a dozen products and record the results, precisely what is known at the end of the day that is germane to real performance? Well, if those pieces are let loose, we know which products will handle them and which won't. If real malware followed the same lead that I did in creating them, they might offer some insight into real performance, but there's no assurance that will be the case. Perhaps I made some poor choices in preparing this synthetic testbed and it reflects the opposite of where real world malware authors are heading. In other words, it's a somewhat sterile intellectual exercise being positioned as providing definitive insight into real performance when that may or may not be true. There are enough unknowns in play without adding to them.

Blue

 

Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

Blue,

Thanks for the very insightful answer.

 

Re: AV-Comparatives June Results (Retrospective / Proactive Tests)

I hope i can keep it daily actual. Over this weekend is actually a funny "game" THERE where u can win a F-Prot Nerd Edition So lot's of fun and joy