Comodo vs AVC

Exactly, that is why they have to remove the word Independent of their website, because they ARE NOT.

Not everybody is reading the fine print.

 

Independent: adjective 1. not influenced or controlled by others in matters of opinion, conduct, etc

They are not controlled by others opinion they are merely being paid to test.

 

Paid but not influenced
The are dependent of their payments

Wiki:
A dependant (British English) or dependent (American English) is a person who relies on another as a primary source of income.

 

Every testing organization is dependent on some sort of payment. Just because you are getting a payment does not mean it influences your ratings.

 

Yes you are right, but the major difference is were the money is coming from.
From readers of a magazine, a consumer organisation, a university etc.
Or the AntiVirus vendors .
And as Melih of Comodo has made clear , there is influence of the vendors
And mail transfer on which kind of test will be best etc.
If an vendor thinks a certain test will not be good for them, they will not join, that is Major influence isn't it?

For me this thread ends here ..

 

You seem to forget that this test was commissioned by Comodo.
Do you know of any independent testing organization or magazine that does commissioned tests for free?
And why in the world would a university subsidize commissioned testing?
*
Let them subsidize new testing methodology from which the general public can profit but throwing money at one particular AV vendor? That's not what a uni is for.

 

Magazines make money from advertising, I doubt a university would pay for antivirus testing. Besides do all these other places have the tech and skill to properly test AV products?

We can't go by one side of the story, Comodo is the first to claim that AV-C is being influenced by cash. If other vendors feel the same way why are they all paying to get in and not saying anything.

 

Please, take a look at -http://www.av-comparatives.org/images/stories/test/ondret/avc_od_aug2011.pdf

Look at the comments section, and tell me if, based on your description, it's an independent test.

I'm just trying to figure out what the word independent truly means in this testing organization, that's all. According to the definition you provided, they shouldn't have been influenced by those AV vendors. Correct?

 

Even with making those changes it does not seem to have anything to do with charging. They are not giving anyone an advantage; each vendor could make a request to config their product a certain way.

 

That's exactly what I think as well

 

I just read the title of this thread, had a look at Melih's post and am typing this post now. I haven't read through 4 pages of this thread (I will now), but I am going to put on record my initial impressions:

Totally unprofessional act by Melih. I have seen Comodo sling so much mud around that I really doubt they are a serious company. Comodo has a history of slinging mud around (Read about the McAleaveys and BOClean, the issues with MRG, now this) and disguising it under the pretext of "being honest with our users".

So what's with paid tests? Well, do you think people don't need to make a living? Tests out of the ordinary that are used to see how my product is doing for my purposes only - I don't see why I shouldn't pay for this, because I realize all too well that testing is a difficult, time consuming job and there are bills to be paid here.

Let me put something in perspective. A hard worker works hard regardless of how the world treats him. If I keep complaining about how my life is so difficult, I cannot progress as a person. This is true for any task. If Comodo doesn't like something, stop dealing with them and don't talk about it.

Do you see Dr.Web diss about AV-C (for example)? I'm sure they've had their complaints, like a few others. The way Comodo acts is similar to an angry teenager who is blaming everybody around him. This is not mature and not professional.

I'm not commenting as of right now on the acts of AV-C, whether they are in the right or wrong, but I see a significant problem with how Comodo handles these things. That in itself lowers my trust in this company.

 

I don't see a problem with testing products with settings requested by vendors as long as that's made clear in the results. The comments section states that they prefer to test products with "default" settings because it's likely many people will use the products with those settings. It also says they recommend that vendors make "High" detection settings the default. I didn't see anything there that would make the results unreliable or unfair. In fact it seems very necessary to address the issue of settings since the effectiveness of a product can vary a great deal depending on how it's configured.

 

There are no backhanded deals going on in AV-C that are not already mentioned in the terms and conditions on their website. Careful analysis of AV-C's results since 2004 will show that chances of bias are very less since a lot of products have changed positions repeatedly over these seven years.

Let's put it in perspective: If I take bribe from a different one of my 7 contacts every year to call them as great, the only "benefit" I will get is the one of burning bridges with all of them since they have all been competing for the same thing.

Also, AV-comparatives has never explicitly made a recommendation for any product, the methodology has always been clearly defined, and every released paper declares that the test results are not the sole indicator of viability of a product. How many "paid" and "biased" tests explicitly say these things?

http://en.wikipedia.org/wiki/Independent_test_organization

It is clearly implied that a testing organization can be paid and independent at the same time. This does not hamper the independence of AV-comparatives' testing process in any way. There may be a few niggles in the methodology that some people don't agree with, but these things are to be discussed at forums like AMTSO and not to be used as a means of slinging mud.

It is painful to see that hard working people get mud thrown at them because some corporate hobo (sorry for the expression) thinks the other way.

And regarding the 'independent' thing: What we are seeing here is a difference of perspective. You are 'technically' independent if your working is unaffected by the actions of others. You are 'financially' independent if you don't rely on others' money to fund your operation.

What Melih did here is classic: mixing up industrial independence with commercial/financial independence. One of the many ways of twisting words to suit your purposes, because the way he puts it, every single non-profit organization is influenced because it is reliant on external money!

 

Its been known for years that avs pay for their testing.Do you think that independent AV testers should do it for free?.Av companies also know that the prestige they get having good results published in these tests might mean more publicity and revenue for them.Its hardly a revelation.

 

It's not cheap to test. You need equipment and employees and lots of time.

 

Paying testers is not the problem as long as all vendors pay the same fee structure and all get the same information, options for configuration and the facts are disclosed. Its a leveled playing field.

The problems occur when payment is different from one vendor to the rest. Like for example sponsored tests where the sponsoring vendor chooses or provides the methodology, testing scripts, defines cases which should be tested and even goes as far as providing the samples. If you want to complain about non-independent tests then go after the sponsored tests. Those sponsored tests are the easiest to manipulate so that they say exactly what you want them to say.

 

I for one find it kind of funny that Melih is making such a big deal about AV-C's terms. Comodo doesn't seem to mind paying for the Matousecs' tests, where Comodo always scores 1 or 2. I know it's a different type of test, and I don't know what Matousecs' terms of payment are, but wouldn't it be a similar sort of agreement? Please correct me if I'm mistaken in this.

 

What do you mean with charging? Paying more or something like that? If that's it, that's not what I meant.
I merely responded that according to what you have previously mentioned, Independent: adjective 1. not influenced or controlled by others in matters of opinion, conduct, etc, then this is not what happened; they were influenced in matters of opinion, correct? AV vendors said/asked they wanted to be tested in a certain way.

This is another definition for independent, which I believe reflects the nature of AVC:

not influenced by the thought or action of others: independent research. (it was taken from dictionary.com)

After all, this is a research; they're researching the most efficient antivirus/antimalware. Correct?

Maybe these testing organizations shouldn't call themselves independent, simply testing organizations, because independent implies without influence. And, I'd imagine influence of any kind.

 

Well, it seems that neither AV-C nor MRG nor Av Test.org is suitable for Comodo, it is only languy's tests that are suitable. So stop wasting your
time with this thread.

Regards

 

Gizmo criticizes the Matousec's tests. It's a technical reading, but seems fair (http://www.techsupportalert.com/content/matousec-personal-firewall-tests-analyzed.htm).
Others point to an interest conflict on Matousec's tests, reducing their independence (?) (http://smokeys.wordpress.com/2008/04/20/matousecs-firewall-challenge-wrinkle-conflict-of-interests/).

 

Good point.

Thanks.

 

Just a simple correction: It is not languy's test, but languy99's test. There is a difference. .


Thanks. .

 

You are right...
But I just wanted some healthy discussion...

 

Comodo agrees to pay $50,000 to AV-Comparatives.org for test auditing & validation
http://www.melih.com/2011/12/07/com...omparatives-org-for-test-auditing-validation/

I thought they didnt like AV-C

 

We are already in the process to get ISO 17025 accreditation http://www.iso.org/iso/catalogue_detail.htm?csnumber=39883

Strange that they first post it online and hours later they mail us to let us know.
Anyway, we told Comodo that they can donate the money to a charity organization after we have been accredited.

Meanwhile you can have a look at the tests of other ISO17025 certified labs like e.g. ICSA.