Investigation in Progress by AV Comparatives

Discussion in 'other anti-virus software' started by Charyb, Apr 26, 2015.

Thread Status:
Not open for further replies.
  1. siketa

    siketa Registered Member

    Joined:
    Oct 25, 2012
    Posts:
    2,718
    Location:
    Gaia
    So, these are two products that participate in all three tests.
    We can already eliminate some vendors from offenders' list.
     
  2. Bob D

    Bob D Registered Member

    Joined:
    Apr 18, 2005
    Posts:
    1,223
    Location:
    Mass., USA
    This all makes me question the methodology of testing; using “submitted” AVs vs. download / purchase of commercially available products.
    i.e: Consumer Reports when testing cars (or anything else) does not accept vehicles from manufacturers, rather, they go out to a dealer as a consumer and purchase them (to eliminate the likelihood that they may end up testing a “doctored” version).
     
  3. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,879
    Location:
    Innsbruck (Austria)
    "submitted" does not mean "send"; it only mean they tell e.g. which version to use (e.g. free, AV, IS, TS, etc.) the products are downloaded from official websites like you guys do.
     
  4. aztony

    aztony Registered Member

    Joined:
    Sep 9, 2012
    Posts:
    720
    Location:
    The Valley Arizona
    Contrast that with a scenario where the vendor/developer is paying the tester to test their product(s).
     
    Last edited: Apr 28, 2015
  5. FreddyFreeloader

    FreddyFreeloader Registered Member

    Joined:
    Jul 23, 2013
    Posts:
    527
    Location:
    Tejas
  6. vojta

    vojta Registered Member

    Joined:
    Feb 26, 2010
    Posts:
    830
    I posted this in 2013:

    And now my favourite reply from one of the many right-click-scan-tests lovers here at Wilders back then:

     
  7. FreddyFreeloader

    FreddyFreeloader Registered Member

    Joined:
    Jul 23, 2013
    Posts:
    527
    Location:
    Tejas
    If you guys downloaded from an official site, how was it possible you got the rigged version? If it was on their website and available for public download - where's the problem? Was this a beta version? Not following how YOU got this version at all.
     
  8. Infected

    Infected Registered Member

    Joined:
    Feb 9, 2015
    Posts:
    1,084
    That's very interesting. Never thought of that.
     
  9. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    AV-C do say in their blog that
    so is it not possible that for a short time the 'rigged' version was publicly available to everyone, including the testers?
     
  10. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,638
    Location:
    Slovenia, EU
    If I remember correctly it was only limited time download, removed after testing was done. But I might be wrong...

    EDIT: TonyW beat me to it... :)
     
  11. FreddyFreeloader

    FreddyFreeloader Registered Member

    Joined:
    Jul 23, 2013
    Posts:
    527
    Location:
    Tejas
    Then AV-C's testing protocol is faulty. What if all AV vendors started doing that? If they did AV-C wouldn't know whether to ~ Snipped as per TOS ~ or go blind. How many vendors actually have done this in the past?
     
  12. aztony

    aztony Registered Member

    Joined:
    Sep 9, 2012
    Posts:
    720
    Location:
    The Valley Arizona
    That's the million dollar question, FF.
     
  13. vojta

    vojta Registered Member

    Joined:
    Feb 26, 2010
    Posts:
    830
    More or less everyone. If not this, something similar.
     
  14. metmichallica

    metmichallica Registered Member

    Joined:
    Mar 16, 2005
    Posts:
    183
    I am lost so you are downloading programs like we do? "Oh yeah av-comparatives downloaded our program. We can take the rigged program down." That seems a little hard to swallow.
     
  15. JRViejo

    JRViejo Super Moderator

    Joined:
    Jul 9, 2008
    Posts:
    77,095
    Location:
    U.S.A.
    Removed Off Topic Posts. Instead of Speculating, Let's Wait Until There's an Official Answer. Thank you!
     
  16. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    8,230
    Location:
    The land of no identity :D
    Let's say I am an AV vendor.

    <x> month: FDT testing starts. Time to reduce the sensitivity of my heuristics and bump up the signatures, add more samples quickly.
    <y> month: Real-world testing starts. Time to "reconfigure" my cloud for a few days....

    <rest of the time>: holidays for us folks, back to usual :D

    Does this happen? Maybe yes, maybe no.... but this way, I do not need to release or submit and "special" versions of the software :p
     
  17. wtsinnc

    wtsinnc Registered Member

    Joined:
    Oct 3, 2008
    Posts:
    943
    If I am the CEO of one of the vendors and I know my company did not cheat, I want AV Comparatives to identify as soon as possible the guilty parties- if any.
    I do not want my company name associated with flawed testing methodology or with cheating.
    Certainly, I want the identities of those companies who cheated revealed so to exonerate my brand.
    As of now, all participants are under scrutiny- deserved or not.
    This indecision on the part of AV Comparatives is probably costing the honest vendors considerable revenue, and they have done nothing wrong.
     
  18. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    5,746
    Location:
    DC Metro Area
    At this stage of the game, unless AV-C is unsure of it's initial analysis, I can think of no justification whatsoever for AV-C not making public the name of the offending party. It does involve a breach of contract, but it is unimaginable that AV-C would agree to a non-disclosure agreement as part of any settlement with the offending company. If AV-C did that it might as well turn off the lights.

    Unless there is a lot more to the story, AV-C appears to be handling this situation irresponsibly and unprofessionally. While it has no legal duty to disclose the cheater, it has some moral obligation to it's followers to do so as soon as possible.
     
    Last edited: Apr 28, 2015
  19. Eggnog

    Eggnog Registered Member

    Joined:
    Nov 17, 2012
    Posts:
    129
    Location:
    United States
    How so? It's an investigation. You don't start dropping names right off the bat. You wait until you have the information and release the results at that time. Just because we don't know who is being investigated doesn't mean it's being handled poorly.
     
  20. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    5,746
    Location:
    DC Metro Area
    "AV-C has uncovered an infringement of the testing agreement by one of the vendors participating in its tests. It has been found that a product submitted for testing by the vendor had been specifically engineered for the major testing labs, including AV-C; public availability of this version was limited...."

    "http://weblog.av-comparatives.org/?p=501

    Fiction:

    Statement by Cambtell Soup. "We are recalling all of our latest shipments of tomato soup. It has been contaminated. We are continuing to investigate this occurrence."

    You don't sit on information and let peeps unknowingly buy from or use a product from a dishonest/unethical vendor if releasing the information you are sitting on can prevent it.

    Frankly, I am stunned that any vendor would be dumb enough to do this.
     
    Last edited: Apr 28, 2015
  21. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    In hindsight it might've have been better for AV-C to wait until their investigation was completed before making any announcement. Once the analysis and communications were finalised, a press release could be published along the lines of:

    Following recent tests conducted in [specify when], we discovered vendor XYZ did...
    We investigated and collaborated with [name of other testing groups] to verify facts...
    The results of this investigation show...
    We, the collective testing groups, have agreed to take the following action(s)...


    No doubt they will do something like this, but by publishing the initial blog post it has produced speculation upon speculation here and elsewhere. I'm sure it's something they could've done without, but the fire has already been stoked.
     
    Last edited: Apr 28, 2015
  22. coolcfan

    coolcfan Registered Member

    Joined:
    Nov 1, 2008
    Posts:
    130
    Maybe somethins like this:

    I know XXX test starts this month so I put a version with special optimizations on my website and update the common download link and wait for the lab guys to download. :p
     
  23. Krusty

    Krusty Registered Member

    Joined:
    Feb 3, 2012
    Posts:
    9,548
    Location:
    Among the gum trees
    That's what I would of thought was a better way to go. :rolleyes:
     
  24. hawki

    hawki Registered Member

    Joined:
    Dec 17, 2008
    Posts:
    5,746
    Location:
    DC Metro Area
    That would be wrong. The fact is that they have clearly stated they have a identified a cheater. In the meantime, peeps are left to guess who it is. Peeps will be unknowingly buying or using a product made available by an dishonest/unethical/cheating company. (I am not assuming that the product available to the public is bad, though it is likely to be not as good as its test results indicate.) You can't let the cat half out of the bag. Meanwhile, honest vendors suffer as peeps put off buying decisions knowing that there is a rat in the market. Further collaborative investigation can continue after the cheater who has already been identified is made known.

    The simple fact is that AV-C is being irresponsible by not disclosing the name of the cheating company. There is no way around that fact. There is only a partial argument for not releasing the name of the cheating vendor. To determine how AV-C 's procedures allowed this to occur and whether it was negligent and bares some responsibility for the failure, what procedures it can put into place to assure it does not happen again, and to re-examine all it's prior tests to see if this cheating also occurred in those tests.

    Naming the vendor will not preclude a deeper investigation. What? It's unfair to the already known cheater to release it's name before AV-C completes an investigation to see if the second suspected cheating vendor actually cheated?? It's wrong to make known the name of the cheater until all testing labs discover how and every way and in which tests the cheating was done?

    AV-C is unwittingly collaborating with the cheater by possibly harming current buyers in the marketplace. It is an accessory after the fact. If the police have proof that someone committed rape but also suspect another of doing the same, should they should wait until a full cooperative investigation with the FBI is completed in determining the full details of what the known rapist did to his victim and how many other rapes he has committed previously and completing their investigation of the second possible rapist before letting known to the public the name of the person they know has committed rapeo_O?? This cheater is currently raping the AV marketplace and harming other vendors and possibly harming those who buy or use the product.

    AV-C has made a huge error in judgement . It has however possibly accomplished three things by its half-baked announcement. It has told it's followers to ignore the results of all of its completed tests (it does not say which test(s) the cheating occurred in) and that there is at least one and possibly two dishonest/unethical/cheating vendors loose in the wild, and that AV-C needs to review its procedures to assure that this does not occur again. AV-C is to be given credit for making itself totally irrelevant until the full investigation is completed and all the final discoveries are made public.
     
    Last edited: Apr 29, 2015
  25. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    at least this has got folk talking about the testing used and whether these tests in the end mean anything,if tests are done to a known timetable it is relatively easy for the vendors to manipulate them,just look historically at the VB tests,eset didn't miss any examples thrown at it for years but in the real world it always offered less protection than kaspersky which at that time missed examples fairly regularly in the tests
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.