some big names fail the latest VB certification

Discussion in 'other security issues & news' started by bigc73542, Oct 2, 2008.

Thread Status:
Not open for further replies.
  1. bigc73542

    bigc73542 Retired Moderator

    Joined:
    Sep 21, 2003
    Posts:
    23,873
    Location:
    SW. Oklahoma
  2. emperordarius

    emperordarius Registered Member

    Joined:
    Apr 27, 2008
    Posts:
    1,218
    Location:
    Who cares
    Failed for 1 false positive! That's terrible! Let's all and change our avs!:eek: :D
     
  3. minacross

    minacross Registered Member

    Joined:
    May 12, 2002
    Posts:
    657
    Passed:
    Agnitum,AhnLab,Alwil,AVG,Eset,Fortinet,FRISK,Kingsoft,McAfee,Microsoft Forefront,Norman,Rising,Sophos,Symantec,AEC (Trustport) and VirusBuster.

    Failed:
    ArcaBit,Avira,CA eTrust,F-Secure,Kaspersky,MicroWorld,Quick Heal and Redstone.
     
  4. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    4,222
    To fail with 93 misses (Arcabit ArcaVir) is one thing, to fail with 1 false positive (all of the others except for CA eTrust 1 miss, Redstone 1 miss + 1 false positive) makes the whole test a bit of a joke the way they word their results.
     
    Last edited: Oct 4, 2008
  5. Arup

    Arup Guest

    VB looses its credibility for good.
     
  6. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    Why? False positives can be a major problem.
    It's only one test. Nothing more, nothing less.
    The report allows companies to fix their false positives.
    This is a good thing.
     
    Last edited: Oct 4, 2008
  7. saberfox

    saberfox Former Poster

    Joined:
    Jul 23, 2008
    Posts:
    84
    Nobody is saying that FP tests should be abolished. Osaban's point is that to declare a product as having "failed" the test just simply because there was a false positive is ridiculous, even more so when we all know the products that "passed" the test certainly do not have 0 FPs in real life.

    Yes, and the value of tests is based on whether they can provide an accurate and/or meaningful gauge of the tested products' abilities. VB100% cannot.
     
  8. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    Originally Posted by saberfox:
    This criteria has been in place for many years, unlike other tests that select arbitrary cut offs for ratings.

    Originally Posted by saberfox:
    The same could be argued for any test. That's why many AVs have removed themselves from testing.
     
  9. GES/POR

    GES/POR Registered Member

    Joined:
    Nov 26, 2006
    Posts:
    1,490
    Location:
    Armacham
    Many? No only those that were in the bottom of detection and only in those test they failed, not the ones where they shine.
     
  10. saberfox

    saberfox Former Poster

    Joined:
    Jul 23, 2008
    Posts:
    84
    Yes, it's been for many years: since the late 80s, if I remember correctly. Given how quickly the malware landscape is changing, you're doing nothing but kidding yourself if you think a metric from 20 years ago is still useful today.

    This thread is about VB100%, not other tests. If you want to bash other tests, feel free to create your own thread.
     
  11. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,736
    Location:
    New York City
    Originally Posted by saberfox:
    .
    Speak to the folks at VB100 to change their criteria.
     
  12. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,648
    Location:
    Hawaii
    "Many," you say?

    Hmmm... 10 would qualify as "many". Please name at least 10 AVs that have removed themselves from testing. :cautious:
     
  13. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,187
    Location:
    Sydney, Australia
    The VB100 test is what it is: Pass or Fail the test as set.
    Not getting all the mals is bad, very bad. The FPs are bad too.
    The issue, obviously, is the statistical problem of the tiny sample.
    (although I dont know the sample size of the 'clean' sample)
    Ahh, we have all come to live with and love metrics...

    Whatever the results, the VB100 organisation has support from the vendors, even Trend who have withdrawn from the test attend their conferences.

    And as we know a group of tech ubergeeks can really live the high life
    http://www.sophos.com/blogs/gc/g/2008/10/03/vb2008-lipstick-pigs-anti-virus-and-pony-tails/
    :D

    PS Anyone want to submit a collective adjective for that group :D
     
    Last edited: Oct 5, 2008
  14. AaLF

    AaLF Registered Member

    Joined:
    Feb 20, 2005
    Posts:
    986
    Location:
    Sydney
    I think Shaun Nicols catchy headline is the problem here. "Failed" is a bit strong.

    But journos have gotta eat too.
     
  15. Osaban

    Osaban Registered Member

    Joined:
    Apr 11, 2005
    Posts:
    4,222
    Agreed this is only one test, nothing more, nothing less, and VB100 is a very respectful organization. Still the perception that one has about the performance of every AV is flawed: In the August AV Comparatives test as an example, Avira's detection was the best with 99,2% (failed VB100) compared to McAfee's 84,4%(passed VB100).

    Avira which is the best AV in detection, at the moment, wasn't even mentioned as a significant company in the article 'Security giants fail Virus Bulletin test' thanking VB100 "antiquated" services.
     
  16. Longboard

    Longboard Registered Member

    Joined:
    Oct 2, 2004
    Posts:
    3,187
    Location:
    Sydney, Australia
    :cautious: Yes.
    A problem re credibility isn't it.
     
  17. doktornotor

    doktornotor Registered Member

    Joined:
    Jul 19, 2008
    Posts:
    2,047
    Eh, these tests are apparently a piece of junk not reflecting real world at all ... :thumbd:
     
Loading...
Thread Status:
Not open for further replies.