VirusP test in PC Utilities

Yup...that is why I tried to chose my words very carefully. being technically qualified does not mean you have the experience. Also I knew many of you wanted to say more and really say it constructively
...with of course your usual summation.

This statement you made...

Not all that essential. Virus Bulletin invites product submissions ... but drive-by tests are OK provided the virus testbed and testing methodology meet the industry standard.


This thread is not really about the sacred cow..but I do not agree with you on that point no matter how you rationalize it.

Also this industry standard thing I see so often come up at these open security forums leaves much to be desired. Some may claim they have been in the business of testing since the Flood and therefore they and a few others have agreed to not disagree on the bed and the method..but where I come from everyone who really cares..signs off on the test standards and documentation.

It needs revisions no matter who claims they have the Holy Grail at this time or 'THE' Test bed.

The year is 2003..time for another conference.




This do it my way or do not do it at all is interesting. But no true professional should really feel threatened.


If you think of your goals as dreams, they will never become real.

 

 
>> Not all that essential. Virus Bulletin invites product submissions ... but drive-by tests are OK provided the virus testbed and testing methodology meet the industry standard.

> This thread is not really about the sacred cow..but I do not agree with you on that point no matter how you rationalize it.

Science, Medicine, Forensics, Engineering, etc ... in fact, just about any profession you can name ... has approved testing methodologies in place. If there isn't a suitable existing methodology available then they create a new one. No professional tester uses "his own" methodology ... at least, not if he wants his findings to be credible.

Academia demands (or should demand) strict adherence to an approved testing methodology if you expect to publish your findings without being ridiculed and ostracized.

By its very definition, an approved testing methodology must be as close to foolproof as possible. Without proper testing methodologies we would see an Apollo 11 disaster (through negligence or oversight) every time the space shuttle was launched, and a new Piltdown Man (through fraud and deception) every week.

Ralph Nader's "Unsafe at any speed" would be laughed out of existence if he published it today. So would Patricia Hoffman's series of "VSum" virus summaries.

Times have changed ... and we must change with them. Viruses are no longer a pimple-on-the-arse-of-your-hard-drive inconvenience ... they're costing the world billions of dollars a year ... and computer users need the best possible information on how to protect themselves..

There is no room for shonky tests!

In antivirus product testing, anything less than perfection is worthless!
 

 

Yes that all makes sense I see many use the words perfect..and I counted the times 100% was used in this thread..and other threads.

Those are pretty powerful words and percentages...Equating to the BEST the one an only

It leaves no room for "others" in the minds of many who use those terms.. even those who might be at 90% or 80%.

They all become shonky tests or products or developments.

When will people wake up ?


Hugs Rod,

Santa is still wrapping gifts..but the name tags have not been applied.

 

 
> Those are pretty powerful words and percentages...Equating to the BEST the one an only

As I said earlier, by its very definition, an approved testing methodology must be as close to foolproof as possible. "100% perfect" is always the strived for ideal. It's seldom easy to achieve ... but it is achievable.

A daunting number of man-hours would be required to properly validate a sample of each of the many thousands of viruses known today from scratch. Virus Bulletin has a head start ... they've been individually verifying viruses as they appeared since 1989, and now need to verify only new viruses as they appear ... perhaps a few hundred a month. (Even this is a fairly big job.)

The fact that 1/2/3/4 (or even 20) scanners identify a file as viral is not a guarantee that it's a real live virus.

The fact that your mate who knows everything about viruses told you a file is viral is not a guarantee that it's a real live virus.

The only way to guarantee that a file is viral is to verify this yourself by actually infecting something under controlled conditions in a sterile environment ... and that's exactly what Virus Bulletin techs do with each and every virus they use in their tests.

> It leaves no room for "others" in the minds of many who use those terms.. even those who might be at 90% or 80%. They all become shonky tests or products or developments.

If an antivirus product test is shonky in itself then all the results are shonky ... whether a particular product detected 100%, 99.99%, or only 2%.

Virus Bulletin puts its reputation ... a reputation they've worked hard and continuously to maintain for fourteen years ... on the line with every test.

You'll always find some loudmouthed pissant "virus expert" slagging VB testing off in Internet forums or in Usenet ... but in its fourteen year history, not one single solitary reputable antivirus professional has slagged it off. Why do you think that is ?

> When will people wake up ?

Some people never wake up!
 

 

Well this thread at our forum here all started with Zander the guest with the little pointy thing to DSRL. So here is another one for you. This time you will have a chance to even talk to VirusP.

http://www.dslreports.com/forum/remark,8685090~mode=flat






And I am sure we would welcome him here

 

Ok, so since i ain't welcomed, i just have a couple of questions for you:

1) I have been annoyed by the fact that noone from this forum or another, who has been an av specialist-security specialist even-has emailed me, suggesting ways to improve the quality of the tests i perform. The only thing i have got till now from certain av related ppl is discredibility, disapprovement and bad rep.

2) I don't think everybody knows if and how much related certain forums are to specific av software companies .. let's say i got an av software, or work at such a company for all that i care, and start up a nice little forum, praising "my own" av software. Would that be just?

3) Since VB is the best and most credible av testing org in the world, how come they never publish the vx list they use, or the procedure they follow

4) Why are certain software ONLY being tested at those tests? I managed to gather-up almost 50 (!!!) antivirus and anti-trojan software, how many of them are included in the VB test? Are the rest of them out of the market? Can't a pc user buy one of them? Why are they excluded afterall?

5) Why do i get the feeling that, like in the av market, people in the av scene do NOT want others to "intrude" and learn the game

6) If i saw some guy trying to learn a job i am pretty good at, i'd try to help him, unless i felt threatend by the fact that one day he could get my job .. i may be considered a "newbie" compared to many of av experts, nevertheless i do the best i can do. What do they do? Sit in front of their screen and start calling guys like me failures. Now, isn't this all a pretty good reason for me to get upset

Best regards to u all

Antony a.k.a. VirusP

 

Thank you for your response. It seems like we see things in another perspective, although i agree with you in some matters. Nevertheless, i must respect this forum's rules and i will not make any more comments on this post. If anyone wants, i can be reached through my website (i suppose most of you know it), or even by IM.

Best regards
Antony a.k.a. VirusP

 

In response to VirusP's questions regarding the VB: I'm no expert, but when cruising the VB archives online some time ago I believe I saw that the list of viruses used (both zoo and ITW) in a particular month's tests was indeed available.

From that I deduced that the detailed information re: the viruses used in the tests and the methodology is indeed available, presumably to subscribers to the publication and perhaps only occasionally or rarely in the archives to non subscribers on the website. Perhaps actual subscribers to the VB can confirm or further illuninate this issue.

As to why doesn't the the VB test ffifty programs or at least more than they do test?

1. The VB tests only AV programs.
2. They only test those programs which AV vendors themselves have submitted to them.

Although some posters in the past have claimed that VB chooses what products to test and thus repeatedly denigrated the VB for not testing their product of choice, this is clearly not the case if one actually investigates based on the info available from the mag itself online.

If one actually reads some of the back issues that are available online it's clear that the vendors choose when they will submit a product for testing and, if they have several products, which one(s) they will submit . So clearly it is the AV vendors that determine what products are submitted for testing. The VB will not test a product that has not been submitted by the vendor.

This info is not hidden from public view, one just has to read the mags available online....sometimes some of the archived mags have more info than others since it appears the whole mag isn't always archived online. But after reading several of those available online, this is the info I gleaned. Obviously, paid subscribers get the mag's full contents and the online archive, as far as I can tell, doesn't appear to be completely comprehensive. But still enough information is there to be able to know when some of the VB's loudest critics on the chatboard circuit haven't bothered to check their "facts."

 

Now, here is some rational, objective thought on the issue. I take full credit for the "tiered results" idea.