About the av-tests; Summary

I believed VB (= VirusBulletin) tests for a long time, until I replied to a post "Snakeoil or not" discussion area.

It seems to be that VB doesn't understand Statistical Process Control at all. That means if you want to have a certain (=same) risk in test results all the time (= risk to be infected with a virus), you must increase the in the Wild viruses amount all the time, so far the total amount of viruses is increasing so rapidly as it does nowadays.

The bad thing is that, AV-test.org seems to make the same mistake, the only good thing is that they take a little bit larger in the Zoo test, what VB does in so called Macro, Polymorphic and Standard tests. The amount of VB:s viruses still remains about the same. It's like we are sitting on a virus mountain which grows higher and higher until we can't breath any more!

In the VB:s in the Wild tests the amount of viruses to be tested has been the same during last 4 years, when home PC user's amount has multiplied. They have lowered the standards all that time and maybe from the beginning. If you were building ships with those new standards, they will sink all.

The whole my trust to those results made by the independent organisations have totally collapsed. The last thing was that VB couldn't even count percents in their August 2002 test reports. Or was it number of misses, who cares, when there are so many mistakes, or is it pure lottery or entertainment?

The test organisations don't understand the phrase "system collapse", so far there are 100% proof results, as there are now. I am thinking all the time that there is something to hide?

Or is the answer so obvious, that the AV-producers don't want to be tested so that everyone collapses, who would buy their products anymore?

None seems to be interested in the limits what different products have anymore! When we are making steel, the specifications are well known, but within av-products, more and more unknown.


"The truth is out there, but it hurts"

Regards,
Firefighter!

 

Firefighter, my friend, having spent some time as a Quality Control Supervisor, I understand what you are saying, but.....
Something you need to take into consideration is that all viruses should not be given the same weight in consideration. At any given point in time, there are maybe 20 or so really prolific viruses loose, and several thousand that it is highly unlikely anyone will catch. I would be more interested in testers giving us such information as what each product missed and what they falsly identified.
Plus, this is a comparison thing were talking about, not an accept/reject based on some number or value.
Like I have said before, all of these comparison tests are of limited value and at best provide just one more piece of information to be used in making our own personal determinations.
As for the number of viruses going from 100 to ten thousand, does it matter? The viruses are like possible defects, if you consider the detection of them is what you are really talking about. If you miss detecting the lovebug virus, that is one defect, no?
Very difficult to apply standard quality assurance concepts to testing AV programs.

 

To Root from Firefighter!

The point was that, the testers should do the same thing that the AV-producers do everyday, they are still smart people, increase the amount of tested viruses test by test.

I appreciatete the AV-producers job very much, but the net environment gets more and more polluted as it is with the real world's environment as well. But maybe it is so deep built system in human nature, none wants to know that the way they are going leads to holocaust.

If the AV-producers should done only the same that the testers have done, within every update of a program there has been removed the same number of viruses from virusbase as updated. Does that happen somewhere?

I think the AV-tester's role has totally chanced, their hobby is to calm people, that they were continuing with their tasks as nothing has happened!

I remember as a kid, when I lived within these forests and tens of thousands lakes, we slept our doors open, none had heard about drugs. Nowadays, when I am living in a village 1/3 of my childhood's hometown, population about 4 000, your cd-radio set is stolen almost every year from your car, doors must be closed also daytime, and so on. It will be happen the same with viruses, as it has happened with drugs. We can't win that war with that weapon arsenal now is used.

That's sure, nothing lasts forever, but Internet as a functioning system, lasts far less time than we ever have thought!

We accept more and more virus infections, until the whole internet system has been regenerated!

I appreciate RAV also very much, I am still using it, but the RAV slogan "Worry less, RAV is watching", should be, "Worry a bit more now, RAV is still watching".

"The truth is out there, but it hurts"

Regards,
Firefighter!

 

Why? Take as an example the tests of VB. Those consider viruses that are reported to be in-the-wild. If there is for a certain period no ITW-being of a virus reported, than this sample will be removed from the testset. Take as an example CodeRed. This one is only ITW if there are servers with old IIS software installed. Once nobody uses such software anymore the basis for such malware is gone. No change to be dangerous anymore. So why this should be tested?

There are companies who remove old malware from there signatures, e.g McAfee or TrendMicro.

I do not agree with this statement.

My last virus infection is more than 14 years ago (was my first and only infection ). I personally think that Spam is more a problem than malware. So I think if we are talking about the death of the internet spam is a more realistic reason.

wizard

 

To Wizard from Firefighter about "VB in the Wild lists"!


Here are a bit more facts that I was picked from VB pdf-files.

July in the Wild lists in VB (1993-2002):

1993   103 viruses
1994   154 viruses
1995   261 viruses
1996   320 viruses
1997   430 viruses
1998   603 viruses
1999   311 viruses
2000   548 viruses
2001   698 viruses
2002   596 viruses

2002    October 572 viruses

So the amount of in the Wild viruses increased all the time until the year 1999 happened something? Was it, that the amount of PC:s suddenly decreased rapidly? If not, what then?

Let's take a little overwiew what really had happened?

Home PC:s had been in real common use, let's estimate since 1998.
Common use of Internet, I don't remember, because I haven't discussed about that with Al Gore.
Plenty new Microsoft's Operating System's have been taken in use.
Allways open ADSL connections have been more and more usual in home PC net use.
People have taken Firewalls and Anti-Trojans in home PC use.

How in the Wild list is then gathered? If I remember right, it is based to the AV-producer's combined feedback about monthly (new?)virus infections, which again is based to the PC users feedback to everyone's own AV-producer.

In the beginning, before Al Gore's marvelous invention, the feedback was based to the motivated IT professionals announcements. Nowadays, when home users are the majority of PC users, it is still IT professionals, who have a big part of all that feedback.

In normal manufacturing environment, it is quite common that an employee gives 0.1-0.5 suggestions for improvement per year, when it in the best manufacturing cultures may be some 30-50 suggestions/employee in a year.

None can force an average citizen to give feedback with foreign language to AV-producers. An average citizen is not the best professional from the best manufacturing culture. They get bored with problems, and don't care any more! How a long time did USSR stayed alive? But was it really functioning?

When I said that no AV-program can get in real life the 100% in the Wild result, that means the all above mentioned issue's have been noticed. Try without Personal Firewall and Anti-Trojans to surf in the net with NOD32, use p2p programs and download things, go to crack sites to download something, and count hours in the real Wild world with 100% proof system!

One of the world's most appraised Economic's Systems thinker, MIT Professor, Peter Senge, has studied the most important issues in economics systems, and he mentioned how for example to find the ultimate truth!
"Use the left hand column!" The most important is what is behind the lines, what has never said by the authorities!

What is the worst thing to happen with internet? If the whole banking industry collapses because of IT crimes? Is it ever possible? Do they ever publish the real amount of IT crimes concerning banks? During last months in Sweden one family got about 20% of the total year budget of the State of Sweden to their personal bank account. Pure casual, so authorities said? If that money could falled into Middle-East or Far-East, what then?

What PC Flank says about PC:s protection!

"We recommend the following:

Minimum (most effective for a small budget):
Purchase a good quality firewall. Follow the security recommendations we provide.
Degree of protection: 60% (if properly configured)

Standard:
Purchase a good firewall and anti-Trojan software. Follow our basic security recommendations.
Degree of protection: 75%

Maximum:
Purchase firewall, anti-Trojan and anti-virus software from trusted manufacturers. Carefully study the User's Guide of your firewall software. Read our basic recommendations on security.
Degree of protection: 99%

Our recommendation to all users is to have at least the Standard set-up. If you're on a tight budget, start with the Minimum and move to the Standard and then to the Maximum as quickly as possible."


Still there seems to be 1 % risk left with best possible recommendations, which in Quality world is totally unacceptable! It means that system is collapsing!

By the way, my DrWeb backup scanner's virusbase has increased 19 % within 8 months. Do you recommend me to remove the oldest 19 % database away? If not, why still VB in the Wild list's virusbase has remaining about the same size? Because they are results of a system, that is based on illusion!

Do you want a free advice? If you have kids, be proud of it, if they will be carpenters or mechanics in the future, beware of IT bussiness!


"The truth is out there, but it hurts"

Regards,
Firefighter!