AV-Comparatives - Anonymous Test of four Chinese Vendors

Discussion in 'other anti-virus software' started by SweX, Oct 30, 2011.

Thread Status:
Not open for further replies.
  1. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    I find this quite hmm... I guess interesting that these vendors choose to be anonymous.

    Go here:http://www.av-comparatives.org/ And then browse to the Singel Product Reviews page.
    And in the PDF one can see Missed samples, but also how many FP's each vendor got.

    Vendor 1 did OK, but Vendor 4 did not so good.
     
    Last edited: Oct 31, 2011
  2. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    Wow anonymous?
    Hahaha i guess they are testing their engines :D
     
  3. Rilla927

    Rilla927 Registered Member

    Joined:
    May 12, 2005
    Posts:
    1,710
    I would not touch any of them.
     
  4. sm1

    sm1 Registered Member

    Joined:
    Jan 1, 2011
    Posts:
    520
    The least percentage of missed samples is 16.9% So the best detection rate is 83.1% which is not that great.
     
  5. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    3,764
    Location:
    Outer space
    :thumb:
     
  6. ziaul

    ziaul Registered Member

    Joined:
    Aug 14, 2007
    Posts:
    239
    +1

    Ziaul
     
  7. Montmorency

    Montmorency Registered Member

    Joined:
    Oct 9, 2011
    Posts:
    181
    + 100
     
  8. cruelsister

    cruelsister Registered Member

    Joined:
    Nov 6, 2007
    Posts:
    976
    Location:
    Paris
    Let's see if I can guess one:

    Chinese AV + Not in August Comparative = Kingsoft?
     
  9. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    I guess that Rising must be one of them also. And maybe Jiangmin as well :doubt:
     
  10. tuatara

    tuatara Registered Member

    Joined:
    Apr 7, 2004
    Posts:
    772
    Unbelievable !

    AV-Comparatives has lost it completely.
    AV's today, are using cloud as an important part of their solution,
    which is needed because of the amount of malware out there.
    AV-C decided not to include this in his tests, and using malware several months old, that doesn't give you a good indication on how a AV in "real life" performs!

    That is bad, but now it even get worse.
    So 4 AV's are tested, but he doesn't want to say which ones.
    That is good for the general public ?!?, they really need that info.
    It is like: i have tested 4 European cars, but i don't tell you which ones,
    and these are the results o_O


    - The fact that antivirus brands need to pay him, for these tests isn't good to start with. It gives all kinds of complications like:
    - i don't want to be tested the way suggested, so i don't pay, that way i get no bad press
    - i will pay, but i like to choose with which settings, but you are allowed to report that
    - How good is a product that is never tested , let's speculate :( etc. etc.

    And now, i like to test 4 Chinese brands, but they will not pay .... hmmm,
    And if they don't pay , ... i don't mention their names, so they don't get free press..!!

    Terrible !

    It is time Anti Virus companies decide HOW tests should be done.
    And when tests can get their AMTSO (www.amtso.org) certificate.

    If not, tests will be getting worse and worse, although more unclear and unusable results as these can not be expected,

    :thumbd:
     
  11. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    - it was tested with cloud. (inform yourself before making statements)
    - it was commissioned by chinese companies (not AV company); reasons why no names are included can be read in the report (read before making statements)
     
  12. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,648
    Location:
    Hawaii
    Poster's mouse is moving vigorously, but CPU is not yet active, I think. :cautious:
     
  13. Dermot7

    Dermot7 Registered Member

    Joined:
    Dec 20, 2009
    Posts:
    3,196
    Location:
    Surrey, England.
    You're priceless, bellgamin...if you don't mind me saying so :D...Love to read your posts :thumb:
     
  14. SweX

    SweX Registered Member

    Joined:
    Apr 21, 2007
    Posts:
    6,429
    Ehm...It sounds like you didn't read the PDF at all :ouch:
     
  15. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    Now that i read why was the reason they were anonymous i totally understand!! :thumb:
    To be honest i never even read what was the reason behind it but after having some spare time i decided to do so
     
  16. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    Of course, but if we take this fact, than the test published has not really many informations for public.
     
  17. Noob

    Noob Registered Member

    Joined:
    Nov 6, 2009
    Posts:
    6,468
    And to be honest that is true also, oh well i think these tests were more oriented for the people that financed it (Chinese IT Company) i guess they just released it as a teaser for us :D
     
  18. tuatara

    tuatara Registered Member

    Joined:
    Apr 7, 2004
    Posts:
    772
    IBK wrote

    1 - i referred to previous tests here, see our previous discussion on this :)
    2 - exactly, the one who pays , or not, can decide if the test results are published.

    But let's be honest,
    What is the use of publishing test results without mentioning WHAT you have tested?
    It will only lead to speculation and a to wrong conclusions.

    See the previous posts in this thread..

    And by the way, in your document, i read "as an independent non-profit organization"
    how independent is this organization, if it lives of the payments of vendors that
    pay to have their software tested ? Or test what the client want to have tested as in this case?

    And what about the malware tested, again most of the malware found
    can be considered as old.

    Another thing is why are the number of False Positives always in absolute numbers,
    but the missed samples in percentage?

    In your Graphs 150 False Positives seems the same as 32000 missed samples (16 % of 200000) ;)

    btw i don't mind if you say i didn't read your document :)

    :thumbd:
     
    Last edited: Nov 4, 2011
  19. Baserk

    Baserk Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    1,317
    Location:
    AmstelodamUM
    Actually reading up on AV-C, would learn you that "AV-Comparatives is an Austrian Non-Profit-Organization" which closely cooperates with the University of Innsbruck.
    Reading up a bit further on other tests (or a simple Google search) f.i. shows; "The whole product dynamic test is a collaboration between AV-Comparatives and the University of Innsbruck, with financial backing from the Austrian government."
    AV-C doesn't just rely only on money from participating vendors, so it doesn't live of the payments of vendors.
    Again, if you would have read a bit about their tests, you would learn that the sample set for FP's is a different one than the malware sample set.
    The FP graph clearly shows a maximum of 400, doesn't it. Of course, you're free to interpret 400 as 200000. :)
    Why not inform yourself, it can prevent unnecessary speculation and wrong conclusions. And semi-troll like posts. :thumbd:
     
  20. tuatara

    tuatara Registered Member

    Joined:
    Apr 7, 2004
    Posts:
    772
    Yes, i have read that too,
    of course it is easy to say that i draw the wrong conclusions
    based on nothing.
    But as a matter of fact i have a business relationship with
    several antivirus brands.
    And that is were i get additional info.

    But AV-C can make this very clear by answering a simple
    question:

    Do antivirus brands have to pay or not to have their
    products tested, or is the university the only one funding this?

    Guys, as you can see on the account creation date i
    have been around here for quite a while.
    And i know what AV's have to pay.

    Ok, you might call this trolling, but i think it is time
    that the general public hears the other side of this.

    So, please respect someone who has another opinion,
    you might disagree, but i am well informed on this.
    - please calculate the absolute number of misses and compare those with the absolute number of FP and see for yourself.

    With this statement i will end my part if this thread .
     
  21. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    Why you do not read the PDF on the website (from 200:cool: where is says that there is a fee for provided services and licensing of logo usage?
    If you know so much, why you do not also say that our fees are for the provided services the cheapest ones?
    Are you aware that everyone, also all testing organizations (also the ones which do not do it for profits), have expenses that need to be covered in order to deliver services?
    Can you state with which vendor you have a relation?

    btw, there is only one type of test a AV-C which is done without cloud, and that is done by design/scope. All other tests are done with cloud. Also the statement about several months old malware is wrong. I think you are misinformed or do not want to inform yourself by reading known information.
     
  22. PJC

    PJC Very Frequent Poster

    Joined:
    Feb 17, 2010
    Posts:
    2,959
    Location:
    Internet
    Decide HOW and Pay FOR or Pay... ACCORDINGLY?
    The issue is to have 'Fully-Independent from the AV Vendors'AV-Testing Organizations...
    Do the AV-Testing sites (Dennis Technology Lab, ICSA Labs, and PC Security Labs)
    which are AMTSO members ,
    have such an AMTSO AV-Testingcertificate? Just asking/wondering...o_O :doubt:
     
  23. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,819
    Location:
    Innsbruck (Austria)
    such a certificate does not exist. please read the FAQ.
    P.S. VirusBulletin and CheckVir are also current members.

    [sorry for the off-topic]
     
  24. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,648
    Location:
    Hawaii
    More than "semi" I think.
    ~~~~~~~~~~~~~~~~~~~~~~~

    Test organizations need an income stream. Some of us feel objectivity is questionable when the income stream is obtained from the companies who own the programs being tested. But -- is there a VIABLE alternative source of income for test organizations? I for one cannot think of one (the stumbling point is that word, "viable").
     
  25. PJC

    PJC Very Frequent Poster

    Joined:
    Feb 17, 2010
    Posts:
    2,959
    Location:
    Internet
    I know Andreas, and my Question was deliberate...;)
    I am aware of VB, too.
    I mentioned the specific sites on purpose...;)
    Just check past Threads with their Tests...:blink:
     
Loading...
Thread Status:
Not open for further replies.