AVs Test and Competition Abuse

Hi,

A news that i'm aware only by reading a judicial publication in a computer magazine.

The editor and resellers Editions Profil ( http://www.editions-profil.eu/EP/ ) has been ordered to pay (10 000 euros) damages to Doctor Web France for unfair competition for using the test results (2009) of AV TEST in an
unfair and discriminatory way in order to promote its main product BitDefender.
The offical judjement is here (in french only):
http://www.editions-profil.eu/ep/jugement.pdf

I just mean that it is necessary to mitigate AV tests results and logo.
The results should not be taken as an absolute fact, but just as a kind of snapshot of the efficiency of an some AV at date D regarding the methodology M that circumscribes and limites these tests.

% of detection rate does not make sense (99% for BitDefender and 85.04 for Dr Web for the related test), if we consider that the black list approach only is not the right angle for testing the efficiency of antivirus.
More overe it has already been demonstrated years ago that AV detection is an N-P complete and undecidable problem (then 100% results like VB is a non-sense).
The security software industry is higly competitive (some new antivirus appears each year on the market), and these inequal practises are only the result of this competition.
The main victim of this "marketing war" is unfortunately the average user: how can he know which av to choose, certainly not by the tests themselves...
Hope this affair will help in improving equality in the AV competition.

Rgds

 

i haven't seen the links but if they extracted the statistics from real sources then i can't see anything bad here compared to what others do (Ex. Norton)

PS. docs are in french xD

 

well , not first time... every Av vendor has published Av test results showing those only in which they secured well and not complete results

 

Hi kareldjag,

My french is pretty poor or actually, trying to read a french judicial document, non-existant.
Could you summarize/explain what was the verdict?
Did the verdict state that test result scores can only be used in ads containing, like you wrote, a sort of disclaimer regarding time period, test methodology used etc, or was it based on other issue(s) (regarding product comparison)?
I mean, how was the ad unfair and discriminatory? Because of the lack of any disclaimer?
thanks, baserk.

 

hi,

The main BitDefender antivirus resseler in France have used a poster
in all its outlets.
This poster resumed testing of AV test independant organisation in which DR WEB was the last one and BitDefender the number one of the ranking.
Dr Web has contacted Edition Profile to express its displeasure and complaints, and claimed 100 000 euros in damages and interest.

The court has decided that Editions Profile have used clearly attacks and unequal marketing methods towards Dr Web, and therefore conceded that it was factually an unfair cocompetition.
Instead of the 100,000 euros expected by Dr web, the court sentenced Editions Profil:
-10,000 Euros of damages and interest,
-5000 euros for the reimbursements of expenses incurred by Dr Web,
-To the publication of this judjement in 3 computers magazine choosed by Dr Web and on Editions Profil web site.

Comparative advertisement must have some limits, because it is easy to manipulate end potential clients with tests ranking.
I guess that Editions Profil has used an advertising poster like theses ones:
http://www.phil-concept.com/pdf/Flyer_BitDefender_2010.pdf

http://www.av-test.org/reports/2010q2/avtest_report_bitdefender_102274.pdf
Well...business is business as usual...

Rgds

 

^Thanks for the explanation.

 

I wonder if car resellers can be prosecuted under the same legislation,I mean you wouldn't want a Mercedes/BMW/Porsche dealer claiming their cars were better than a KIA would you?

 

I don't know that much about most tests. I can understand any manufacturer wanting his product to look good. For that reason I take a lot of test results with a "grain of salt."

However, I have complete confidence that AV Comparatives is unbiased in their testing.

Regards,
Jerry

 

Website advertising in the UK will be regulated by the Advertising Standards Authority (ASA) from the 1st March, so if you find any sort of claims which look dubious, report it to the ASA and they will investigate.... I'm VERY interested to see if they'll do anything about the advertising habits of certain firms (ala PrevX)

 

"Unbiased" is NOT synonymous with "competent" or "valid" or "statistically significant."

Further, when a testing organization charges a fee to those wishing to be included there is necessarily an **implied bias** against non-participating companies. That is, some folks assume that non-participating organizations have something to hide, whereas a non-participating organization may have chosen NOT to participate for any of a number of other valid reasons such as shortage of advertising funds, or dissatisfaction with the professionalism of the test organization, or .....etcetera.

Ideally, test organizations would include ALL viable/comparable products, and derive their revenues from other sources, such as subscriptions from those wishing to view the test results.

 

Hi,

Of course fully agreed with Bellgamin: independence is not compatible with money: this is true for most av test organizations (av-test, av comparative, VB 100 etc.) as for computer magazines: both need the money of av editors!
The worst example of interest conflicts has been given by financial rating agencies ( http://voices.washingtonpost.com/economy-watch/2010/05/credit_rating_agencies_the_rea.html ).
There are independent tests, but they are very rare.
I can mention a professional one done on engineer security magazine (test of Dr Web) with extraction signature schemes:
http://ed-diamond.com/feuille_misc38/index.html (page 4 to page 17).
But i guess that the independent tests are done by government institutes. This is for instance the case of USA which provides guidelines assessment since years: protection profile for antivirus
http://www.niap-ccevs.org/pp/pp_av_br_v1.2/ (click on protection profile pdf).
A very few years ago, the Russian government needed to choose an AV for its critical and security IT; Kaspersky lab has not accepted the test protocol which requires an inspection of the code source at various time at the office by the military experts.
After a long process, Dr Web was chosen as the AV of the FSB (ex KGB) and ministry of Defense: http://company.drweb.com/licenses_and_certificates/?lng=en
Quite ironical if we consider that kav code source can be easily found on the net now...
As for many other industries, av editors are only interested in things that could help them to sell more and more products!
That's why in the past av contests and challenges that has shown their inefficiency was highly criticized.
It was the case of the "race to zero" (DEFCON) and the "deactivation" challenge of the ESIEA, a french computer engineer school:
http://91.121.184.131/data/pwn2rm_hacklu2009.pdf
Kaspersky and GData have criticized these tests, with the usual bad faith; but Dr Web of course has used the result for marketing purpose:
http://news.drweb.com/show/?i=678&c=5
Regarding Prevx and its aggressive marketing, it has already been discussed here; it's up to the "injured" editors to take a lawyer and to do the right things!
Rgds

 

Just common marketing ploys, regardless of whether it's true or not.

 

I do know that!I was trying to be slightly sarcastic about this ruling,every company(even Kaspersky)is guilty of taking bits and pieces from tests etc to try and give their product an edge over the competition,I have never once seen any advertisements for any AV product claiming if you install it it will cause all sorts of problems/slowdowns or that they are using their customer base for extended beta testing,have you?

 

No, thats the job of the EULA...

 

probably well hidden in that as weel,3/4 of the way through cos nobody reads more than the 1st half and then skips to the end!

 

I admit that I am not competent to discuss test technically so would appreciate your assessment of the various test organizations in regards to the top two or three.

Thanks,
Jerry