Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statement

Discussion in 'NOD32 version 1 Forum' started by anton, May 9, 2003.

Thread Status:
Not open for further replies.
  1. anton

    anton Eset Management

    Joined:
    Oct 25, 2002
    Posts:
    210
    Dear Clients,
    Dear Participants,


    Eset has received many thousands of inquiries and calls from concerned users across the globe since the results of a recent antivirus test performed by GEGA IT-Solutions GbR were published in PC World (Italy) and PC Welt and PC Welt–Special (Germany) magazines. These results have been widely publicized and discussed on the Internet.

    Eset’s reaction comes delayed due to the time required to complete our research into all the related issues involved in the tests and their interpretation. Our lab technicians have performed exhaustive tests on the viruses reported by GEGA IT as “missed” by NOD32, and not one test confirmed the negative results on which the PC Welt and PC World articles were based. The NOD32 version released immediately before the test, the actual version tested by GEGA IT, and the version released immediately after the test, produced detection rates an order of magnitude higher than the detection rate on the allegedly “missed” .BAT viruses reported in the magazine articles. These findings have been presented to all the parties involved, and Eset fully expects complete retractions of the flawed test results and reviews.

    NOD32 continues to provide as close to 100% reliable protection as possible, and is the only system in the world which has not missed a single In the Wild virus in the past five years of testing by the world’s #1 antivirus product evaluator, Virus Bulletin. No other antivirus product in the world, past or present, comes close to this record.

    (A current “hot topic” in security forums is the number of “unpacking engines” found in various antivirus programs. “More is better” according to the amateur “virus experts” who persistently try to trash NOD32’s detection rating - but most current ItW viruses are “packed” with one of the many available runtime compressors, and NOD32 consistently detects 100% of these viruses in independent professional tests.)

    =====

    The following comments do not represent a complete and thorough analysis of antivirus product testing - they are simply a brief outline of the facts, and an attempt to stress the key aspects.

    Two very important elements are essential to ensure the proper testing of any antivirus program . . . . .

    The first requirement is a verified set of virus-infected files which are representative of current real world infiltrations – those viruses referred to in the antivirus industry as “In the Wild” (“ItW”) viruses. Non-ItW viruses (lab samples and viruses which pose little or no real world threat) are called “Zoo” viruses, and many testers include a selection of these in their tests – but even Zoo viruses must be verified if they are to be used in a test.

    The second requirement is bulletproof testing methodology which guarantees consistent procedures and produces reliable (and replicable) results.

    THE SET OF TEST VIRUSES:
    Selection of the viruses used in the test set is crucial. There is a wide variation in the degree of clear and present danger posed by different viruses and different types of virus. Zoo viruses are way down near the bottom of the list, and there is NO place in antivirus product testing for crippled/broken/inactive/corrupted/simulated or otherwise non-viral files – files referred to as “crud” in the antivirus industry. Detection of “crud” might be good for boosting “detected virus” numbers in advertising, but it has ZERO importance in the real world.

    Antivirus experts treat any test or review which places high importance on the detection of Zoo viruses and/or simulated viruses with derision – detection of In the Wild viruses is infinitely more important - but the average computer user is not an antivirus expert, and the skewed results of tests which include thousands of Zoo viruses and “crud” are highly misleading when presented to general public as relevant decision-making criteria.

    AntiVirus Product Development (AVPD) consortium members, under the auspices of International Computer Security Association (ICSA) Labs, have agreed that certain types of older viruses will be phased out of professional test sets, as these no longer represent a real world threat.

    The most important feature of a modern antivirus program is the consistent accurate detection of ItW viruses. A quick look at MessageLabs’ detection statistics from January 2003 through March 2003 reveals that, of the almost 3 million viruses intercepted during that period, ALL were In the Wild viruses. NO Zoo viruses were detected.

    Educated end users should demand the best available protection against current and future real world threats from their antivirus vendor – not detection of thousands of obscure Zoo viruses and “crud” files which will never see the light of day.

    THE TEST PROCEDURE MUST BE BULLETPROOF:
    To avoid errors, a test center should allow antivirus vendors to verify the test results. This requires disclosure of the list of real, undetected viruses – and both the actual virus test set and the tested product version should be archived for future reference.

    Test centers should be available to answer vendor’s questions and provide evidence of 100% compliance with proper testing methodology. Based on a clear description of the test methodology used, an expert in the field should be able to duplicate a particular test with 100% accuracy and, if the test was valid, produce identical results.

    Simple oversights or procedural errors in testing (like rendering viruses inert by changing their file extensions to “non executable”, thereby converting them to “crud”, as happened with the recent GEGA IT test set) will always produce false “misses” and inaccurate results.

    By design, NOD32 ignores “crud” files because they are NOT live viruses – but in the case of the GEGA IT test, if the extensions are changed back to “executable” then NOD32 detects them as viruses.


    INTERPRETATION OF THE TESTS:
    Not all tested criteria are equally important and relevant. Statistical significance of the tested parameters should be presented to avoid misleading interpretation of the results and misrepresentation of the product’s value. (DOS viruses, for example - regarded today as trivial by antivirus experts - are regarded as “just as bad as the latest Win32 worms” by the less-informed end user - yet detection is largely unnecessary, since such viruses are usually found “live” only in a lab.)

    A participant in this forum recently said that NOD32 is successful only in “easy Virus Bulletin tests”. Perhaps he would like to tell us why, if the VB100% test is so “easy”, so many of the “big boys” have failed to pass it so many times?

    The fact is, winning the VB100% award is far from easy. Virus Bulletin charges no fee to participate in these tests - and unlike testing organizations which charge a fee and give antivirus vendors a chance to rectify their mistakes, Virus Bulletin gives each tested product one chance and one chance only.

    The list of In the Wild viruses is published monthly and is always publicly available shortly before each VB100% test is performed. This allows Virus Bulletin to check the flexibility of a particular vendor to fine-tune his product to detect the latest and the most dangerous infiltrations written only ‘hours’ earlier.

    Virus Bulletin’s “pass” criteria are very simple and straightforward - “Detect 100% of the current ItW virus test set in both on-demand and on-access tests without producing a false alarm and you’ll be awarded VB100% certification for that test.” Everyone plays by the same rules, and the tests are equally fair to all participants.

    =====

    Computer magazines, professional test centers, and antivirus vendors each share a portion of the collective responsibility to end users. Vendors should deliver an efficient antivirus system. Test centers should perform rigorous and verifiable tests on validated viruses. When offering advice and making recommendations, computer magazines should provide information which takes into account the significance of real world virus threats and each product’s consistent track record in protecting computers against such threats. Doing all this properly and ethically is the only way to provide the best service and advice to consumers.

    The real value of an antivirus product lies in its ability to consistently and accurately detect real world virus infiltrations and, ideally, to detect the newest viruses by means of heuristics – and Eset’s primary focus is to provide our clients with a state-of-the-art antivirus system capable of heuristically detecting present and future threats in real time with the least consumption of system resources.

    The soon-to-be-released NOD32 Version 2.0 features an exclusive advanced heuristics engine which breaks new ground in virus detection technology. Anyone who would like to test drive this heuristics engine can do so via IMON, or by command line execution of the on-demand scanner using the switch /AH (Advanced Heuristics). Beta 5 is available for download from http://www.nod32.com/download/download.htm

    Anton Zajac


    CEO
    Eset Software
     
  2. mrtwolman

    mrtwolman Eset Staff Account

    Joined:
    Dec 5, 2002
    Posts:
    613
    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    Nice reading. Glad to see ESET did nice work in cheking the facts.
     
  3. krazykidjoe

    krazykidjoe Guest

    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    Anton, thank you for the explanation. I really appreciate it.
    I am a NOD32 user/customer and these test results goo or bad tend to sway or change the perception of a user. Thanks again..Joe
     
  4. rodzilla

    rodzilla Registered Member

    Joined:
    Jun 15, 2002
    Posts:
    653
    Location:
    australia
    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    In case you missed it in the other thread .........

    > This thread was dying out, then rodzilla came a blasting away, at me,

    Ask yourself why I'm "blasting" you.

    Plenty of other people have made adverse comments about NOD32 here in Wilders and elsewhere when they've had problems with the program's performance or detection. I didn't "blast" them ... but they didn't profess to be "Security Experts", they didn't have a blind unreasoning hatred of NOD32, they had (on the surface, and in their own minds) valid reasons to make those comments, and they didn't come across like VXers trying to impress the AV world.

    No doubt there are hundreds, maybe thousands, of viruses which have never seen the light of day ... viruses on VX websites with meaningless "disclaimers" like "These viruses are for educational purposes only. The author takes no responsibility for what you do with them." ... which is basically an invitation for wankers and wannabes to download them and turn them loose.

    By your own admission, you've downloaded some of them ... and by your own admission you think those viruses pose a real and present danger to the public. One could be forgiven for wondering why, if you really are the "Security Expert" you hold yourself out to be, you lack the social responsibility to forward those undetectable viruses to the various antivirus vendors.

    You continually trash NOD32 with your stupid irrational comments and your stupid rigged "tests" on never-seen-In-the-Wild viruses ... viruses which you deliberately seek out and/or modify to make sure NOD32 won't detect them.

    You don't look for discussion and you don't attempt to impart useful information ... you simply troll for arguments with your childish codswallop and meaningless "tests" ... which, btw, it seems most regular Wilders members treat with the contempt and derision they deserve. (I have yet to see anyone agree with your drivel.)

    Since my return a few days ago several Wilders members have asked me to ban you from the Eset forums ... not because of your persistent NOD32 bashing but because they regard you as a deliberately disruptive clown with nothing valid to contribute to the forums. I agree with them ... and I'm very close to doing just that.

    You're skating on very thin ice, wannabe.

    Consider this your one and only formal warning.
     
  5. JacK

    JacK Registered Member

    Joined:
    Jun 20, 2002
    Posts:
    737
    Location:
    Belgium -Li?ge
    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    Hello Rod and Co,

    Some of the virus mentionned by Vampirefo (which are not in ITW list) are in NOD32 DB for quite a while.

    If those are not detected, only 2 possibilities : they are corupted and thus ineffective and they have not to be detected or they were purposely unpacked/packed with another packer in order to circumvent detection : I could do it to cheat any AV/AT product with any virus/worm/trojan

    Those so-called tests have no value at all AFM : it's just kindergarten playing :)

    I don't say you product is perfect (no product is) but I use it for quite a while and it does what it claims fast and with effectiveness.

    Rgds,
     
  6. JacK

    JacK Registered Member

    Joined:
    Jun 20, 2002
    Posts:
    737
    Location:
    Belgium -Li?ge
    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    Hello,

    Those versions came form VX collections and are not in NOD32 DB and there is no reason for them to be in :8 : they are zoo virus and are no thread.
    As I told before, I might do it with any virus/worm/trojan or all the virus/worm/trojan and spread them by myself undetected by any AV/AT product.
    That's REALLY easy and that's why I said kindergarten/scriptkiddies playing.

    That's the truth :)

    Rgds,

    JacK


    Hello,
     
  7. JacK

    JacK Registered Member

    Joined:
    Jun 20, 2002
    Posts:
    737
    Location:
    Belgium -Li?ge
    Re:Does NOD32 Personnel Agree With (Test by GEGA IT-Solutions)?? - Eset statemen

    Possible as it may also possible that this variant is not in NOD32 DB : as you know, all AV developpers don't always give the same .* for variants :
    for instance KAV may say virusname.x and another developper virusname.y or alias.z
    So Alcaul.N in NOD32 DB could be named Alcaul.* by others.

    Rgds,
     
Thread Status:
Not open for further replies.