DrWeb and NOD 2.50.2 heuristics!

Discussion in 'other anti-virus software' started by Firefighter, Apr 20, 2005.

Thread Status:
Not open for further replies.
  1. SDS909

    SDS909 Registered Member

    Joined:
    Apr 8, 2005
    Posts:
    333
    That was the point I was trying to make in my one sentence that most people failed to grasp what I was trying to indicate. In otherwords, this test is worthless. Heuristics are designed for specific field of threats.

    Throwing a random testbed of 3600 samples (including constructors and jokes) of all types at a couple AVs, then posting the results is doing nothing but a disservice to those that might believe(incorrectly) the test.
     
    Last edited by a moderator: Apr 21, 2005
  2. Happy Bytes

    Happy Bytes Guest

    HURRA! Somebody who understands my postings! :cool:
     
  3. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    Doesn't show anything? Well i greatly disagree with that. Good heuristics will certanly pick larger range of malware than weaker ones. That shows how flexible heuristics are. If the malware is just a new modification of older one,there is still a good chance of being picked by heuristics instead missed by signatures (even though,sample was already added using signatures).
    Also these "NEW" threats will someday become "OLD" ones and they will be still picked up by AV right? So it's the same for all malware that is not new anymore (samples in FF testbed).
     
  4. Happy Bytes

    Happy Bytes Guest

    And just one comment more:

    It has NOTHING to do with that i do not apriciate Firefighters efford!

    But to be very honest - i could spend a lot of time and efford in learning japanese language - I DO NOT BECOME WITH THIS AUTOMATICALLY A JAPANESE LANGUAGE TEACHER! I have to follow there some rules and have to take advices from people which living there!
    Even the schoolkids would laugh if they hear me! :D

    That said: Efford is one thing - missleading results is the other one...

    A lot of people reading such posts - most of them do not know how a AV works - they just believe in such test results! Bummer!

    Is this test bad for NOD32?
    No!
    NOD32 scored EXCELLENT in this "test"

    And i'm still complaining !
    As a ESET Mod, and therefore the devil in person, i complain that DrWeb could score much more better if the Testbed would not be flawed! NOD32 probably too, but this results are just USELESS!

    8^) HB.
     
  5. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Every test has some message. When I tested BitDefender less than a month ago against my 3624 infected samples, it scored 3114 (85.9 %). Today BitDefefender scores 3390 (92.5 %) against my 3665 infected samples.

    The message? Be carefull about BitDefender. It's collecting hundreds of crap defs! :D

    Best regards,
    Firefighter!
     
  6. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    If you do not want to send samples to AV companies, you can send to me a CRC list of the samples in order that I can tell you which files are known garbage that should not be used for tests. Its not much, but it is the only way to help you without that you have to send out your samples. Please take that chance to help you, otherwise it could look wrongly like you do not want to improve your test-set on purpose.

    @~~~: ~~~~~~
     
    Last edited by a moderator: Apr 21, 2005
  7. Happy Bytes

    Happy Bytes Guest

    You have always to adjust heuristics - according to the current malware situation! That said it is NOT guranted that a heuristics will keep all old detections of malware types which do not even exists anymore!
    Mainly for scan speed / performance reasons RejZoR....

    Next thing is how many polyengine construction kit's have been sighted in the wild? They are not even dangerous! Only the result what comes out might be! And this result is a virus and therefore DETECTED! Maybe even via heuristics! But detecting such constructors and joke viruses via heuristic is just NONSENSE. And firefighter has a lot of such construction kits included!

    Or Batchviruses :rolleyes: Do you get regulary batchviruses in your inbox? I mean not the executables which are just using a .BAT extension - i mean real batch files....

    I give you now the fact in big red letters - and this will be my final statement to this test:


    • The Form Virus in the 90's took more than 3 years to become 'in the wild' (Bootsektor Virus)
    • The Melissa Virus 1998 3 days
    • The Loveletter 1999 4 hours
    • The SQLSlammer 2003 20 minutes
    • and finally we are already in 5 min infected with Mytob, Bagle and the like in the year 2004/2005 !!!

    Do you know HOW MANY $$$$$$$$$$$$$$$$$$ a good scoring heuristic TRIMMED TO EXCACLTY SUCH THINGS saves from a customer - let's speak about a company where a lot of people working and most of them just clicking on every attachment if was send by someone whom they might know?! (worms spoofing email addy's...)

    You do not have here this time anymore - WE ARE NOT LIVING IN THE 90's - WELCOME TO THE REAL WORLD 2005 !
     
  8. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    Sorry,but i know which year it is. And guess what,i checked only entries for Trojans,Backdoors,Trj-Downloaders,virii and other Win32 malware. And yes,i can see the point of FFs test. If you don't,fine. Everyone has it's own opionion about things. And if Constructors are not bad,why are they then detected by AH ? Makes you wonder doesn't it? Well, not me...
     
  9. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    OFF-TOPIC:
    Should anyone in future again claim that the retrospective tests of av-comparatives.org are not showing a real world test scenario (even if AV companies and agrees on the used method)... well you know what I mean... FF makes a test by deleting sigs and he scans on not new samples for which the AVs already have signatures and tries to get heuristic results. Common... today is the 20th April, not the 1st ;-)
     
  10. RejZoR

    RejZoR Lurker

    Joined:
    May 31, 2004
    Posts:
    6,426
    Well give us some better idea,other than improvisation? There is not many AVs that offer signature disabling... basically i know only NOD32 taht can do such thing...
     
  11. IBK

    IBK AV Expert

    Joined:
    Dec 22, 2003
    Posts:
    1,886
    Location:
    Innsbruck (Austria)
    Well, the best idea is IMO probably the way I choosed and use at av-comparatives. I understand that that means lot of work, but good tests always need much work and time and can not be done in some hours.
    I mean, who does in real world disable signatures?
     
  12. LowWaterMark

    LowWaterMark Administrator

    Joined:
    Aug 10, 2002
    Posts:
    18,278
    Location:
    New England
    This is back, but the more dramatic exchanges were trimmed. See comment in first post about how this was done. This may stay locked for good, though maybe it'll open later. I don't know yet.

    I thought it was better to cut deep (hostile, offensive and personally directed comments) versus deleting the entire thread. Other than not being as exciting to read as it was before, it still has all the technical commentary, and some of it is pretty good regarding AV testing in general.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.