Heuristics versus Signatures

Discussion in 'NOD32 version 2 Forum' started by Bunkhouse Buck, May 31, 2007.

Thread Status:
Not open for further replies.
  1. Bunkhouse Buck

    Bunkhouse Buck Registered Member

    Joined:
    May 29, 2007
    Posts:
    1,056
    Location:
    Las Vegas
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    It never ceases to amaze me that hundreds of people in this forum do not get that heuristics is far more important than outright detection of known malware. If your system is clean, there is no malware on it by definition. What keeps your machine safe? The highest probability (by far) is that NOD32's heuristics will protect you. Do not be fooled by promotion of other engines-ESET's is far superior.
     
  2. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    You haven't really done a good job of explaining why that is so. Assume product X scores 90% on proactive detection, but only catches, say, 90% of all malware. Contrast that with product Y with 30% proactive detection but 98% detection rate on all malware. Which is better? Is product X really "far superior"?

    Heuristics are only a means to an end. You can have as powerful heuristics as you want, but if the end the price you pay is a slip in overall protection, then it's really all for naught.
     
  3. joel406

    joel406 Registered Member

    Joined:
    Aug 21, 2006
    Posts:
    43
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    NOD32's engine works with it's threat sense monitor. To kill both known and as yet unknow threats. You will find that no other AV on the market has a better protection probability then NOD32.

    And if you reflect back on previous comparisons NOD32 has been consistent with it's ability to protect any system. Over the last 6 comparatives NOD32 ranked a adv+ and reciving only 1 adv rating. Compare that to any other AV that they rate, even kasp(cough..cough).

    Congrats ESET I am proud to not only be a user as well as a retailer.
     
  4. Blackspear

    Blackspear Global Moderator

    Joined:
    Dec 2, 2002
    Posts:
    15,115
    Location:
    Gold Coast, Queensland, Australia
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Complete opposite actually, and this has been discussed before:

    https://www.wilderssecurity.com/showthread.php?p=761671

    Blackspear
     
  5. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Actually, not quite.

    Heuristics only comprises of one of the ways a scanner can detect malware. In the case of zero-day outbreaks; yes, this is where heuristics obviously shines. My point was that: if a scanner has a higher chance (yes, it's still a chance, and it's not guaranteed) of catching a mass zero-day outbreak (of which there are less and less of nowadays) BUT suffers in its overall detection, it still leaves you vulnerable to a greater subset of malware than another scanner with weaker proactive detection but catches a greater percentage of malware.

    Unless your computing environment is such that zero-day outbreaks are the ONLY method in which you can possibly get infected... you need to look at the big picture. :D
     
  6. Blackspear

    Blackspear Global Moderator

    Joined:
    Dec 2, 2002
    Posts:
    15,115
    Location:
    Gold Coast, Queensland, Australia
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    The "Big Picture" is you can never write enough signatures fast enough as you are always behind the 8 Ball when malware writers are continually changing the goal posts. Heuristics are the future, signatures are the past, this is why every software manufacturer is trying to develop better heuristic engines.

    We have taken this off topic enough. Start a new thread and continue any discussion about heuristics that you want.

    Blackspear.
     
  7. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    For outright "overall" protection you need an AV with best "overall" detection,to have a better chance(and its only a chance not a certainty!)of catching new/unknown malware you need best heuristics,its a pity they are not combined in any one AV:-we need Nod to improve overall detection or someone like Kav to improve their heuristics,if either company did that then we would have protection worth shouting about!
     
  8. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Again, you're only focusing on theoretical new, unknown malware while ignoring the substantially larger subset of other malware out there, and harping on the one area where heuristic shines while choosing to not look at overall protection. I don't call that looking at the big picture. Do you? ;)

    I thought you'd might say that. :D
     
  9. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Even the best heuristic based av(nod) still relies on sigs,just a pity they are a bit slow releasing them at times
    If you go to your GP with an ailment would you prefer him to prescribe a drug that "may or may not cure that ailment"(heuristics!)or would you prefer to be precribed a drug that was known to cure your ailment(sigs!)
     
  10. mrtwolman

    mrtwolman Eset Staff Account

    Joined:
    Dec 5, 2002
    Posts:
    613
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    IMHO your example is not the best one when we are speaking of new malware threats. The point is: your second doc can prescribe only medication to the ilness he/she already knows while the first one can prescribe the medication which works (let say in 30 - 60 per cent of cases) even for the health condition he/she never met.
     
  11. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    You sound confident enough in heuristics to dump sigs altogether:-even the best heuristic engine doesn't offer anywhere near good enough protection by itself,if any AV was tested and gave same protection results as Nods heuristics it would be considered "hopeless":-no matter what "Nod fans" would like us to believe sig based AV's(Nod included)are going to be around for a while yet for the simple fact that on their own heuristics do not give anywhere near good enough protection:-its a fact,not marketing hype!anyone thats feels differently just install Nod,configure it to how you want it to work and then just leave it:-don't update it and report back has to how it protects you in a hostile environment(thats if your PC is still able to acces the net lol)
     
  12. Joliet Jake

    Joliet Jake Registered Member

    Joined:
    Mar 1, 2005
    Posts:
    911
    Location:
    Scotland
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    You're not seeing the bigger picture. No signature for a brand new threat gives you zero protection whereas heuristics offers a decent chance of protection.

    A breakdown I'd be interested in seeing is what nasties each AV misses. Are they the real bad ones or less destructive.
     
  13. Blackspear

    Blackspear Global Moderator

    Joined:
    Dec 2, 2002
    Posts:
    15,115
    Location:
    Gold Coast, Queensland, Australia
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    What issue do you have with continuing a topic in its own thread :rolleyes:

    Blackspear.
     
  14. Blackspear

    Blackspear Global Moderator

    Joined:
    Dec 2, 2002
    Posts:
    15,115
    Location:
    Gold Coast, Queensland, Australia
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Would you prefer 10,000 needles or a single broad spectrum shot, and for that dose to be continually adjusted to catch new outbreaks :blink: ;) :D

    If you have 1000 malware writers today, 2000 writers tomorrow, 3000 the day after... would you hire greater and greater amounts of staff to combat it by writing signatures and always trying to play catchup, or would you think outside the box and try to develop a catchall system to prevent the escalating increase.

    Heuristics, the way of the future.

    Cheers :D
     
  15. Detox

    Detox Retired Moderator

    Joined:
    Feb 9, 2002
    Posts:
    8,507
    Location:
    Texas, USA
    Another product comparison (ABC vrs XYZ) post removed. Let's keep the product comparisons out of the NOD32 forum.
     
  16. Marcos

    Marcos Eset Staff Account

    Joined:
    Nov 22, 2002
    Posts:
    14,374
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    That's exactly what is happening. Malware writers can create a robot that will produce new variants every second. Adding signature for each of the variant would be beyond human's capabilities, needless to say this approach would dramatically increase the signature database as well as memory consumption.
     
  17. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Well, I'd say we've been getting a pretty good demonstration of how well this "catchall" system works in the latest AV-Test and AV-Comparative on-demand reviews.

    Again, you're focusing on the one narrow area where heuristics obviously outperforms traditional signatures, and ignoring the bigger picture. Compare NOD32 with other products out there which have weaker heuristics but offer better and faster signature updates, and I think it is clear that the recent antivirus comparatives will show you quite a few of such products that detect a greater amount of malware than NOD32 and hence offer better overall protection, which is what's really important in the end.

    You can have the world's best heuristics engine, the world's best unpacking engine, etc etc etc. The million dollar question is, however: how much malware does the program detect? That's what counts, not some theoretical fancy cutting-edge technology that in the end fails to detect as much malware as the competition.

    On the contrary. Newer technologies are proving more and more effective at combating unknown malware. It's simply heuristics that's most well-known and has most press coverage at the moment.
     
  18. Bunkhouse Buck

    Bunkhouse Buck Registered Member

    Joined:
    May 29, 2007
    Posts:
    1,056
    Location:
    Las Vegas
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    You did not read my post carefully. I stated that heuristics are more important if you have a clean system-that was my premise. If have a clean system, the efficacy of a high rate proactive detection engine (NOD32) is far superior to a high rate of signature based detection.
     
  19. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Re: NOD32 get's "Advanced+" in AV-Comparatives test May 2007

    Would you mind providing any explanation on why this might be true?
     
  20. steve1955

    steve1955 Registered Member

    Joined:
    Feb 7, 2004
    Posts:
    1,384
    Location:
    Sunny(in my dreams)Manchester,England
    thought this thread had been closed onceo_Oo_Oo_Oo_O??(by Detox!)
    Blackspear:-
    I like heuristics and agree they are the "future" but at the moment even the best ones are not good enough to offer enough protection on their own(and doesn't seem much improvement being made any time soon),that is why I feel sig based AV's will be around for a while yet!
    You prob Know which AVs I use:-I would love one with the heuristics of one of them(Nod)combined with the speed of sig update of the other(use you imagination which one that is!):-I cannot understand why one company can update things so quickly and others can't(or won't)
    When Eset started banging on about heuristics,a while back now,it was always in the back of my mind that they may start relying on them to protect users rather than updating virus bases as fast as they could,and I cant help feeling that at times this seems to have happened,this is one of my main concerns with heuristics,they can make AV companies a little complacent(and lazy??)

    PS cd was the thing of the future,problem is it doesn't sound as good as vinyl:-newer isn't always better!
     
  21. JAB

    JAB Registered Member

    Joined:
    Apr 17, 2007
    Posts:
    36
    I'm surprised everyone is still looking for a product with great heuristics, a superb signature database and rapid updates. That product clearly already exists. All you need to do is read the aforementioned AV-Comparatives reports to find it. The key is that every product has weaknesses, and it's apparent that there are other factors that some people feel dominate the "perfect" AV described above. False positives and ability to repair come to mind.

    Nonetheless, what is of concern with regard to NOD is that they don't provide both, when they clearly can. NOD's on-demand detection rate, as measured by AV-Comparatives, has clearly been falling. Of equal concern is when NOD adds updates. If you look at the chart on page 3 of the last AV-Comparatives on-demand comparative, AV-Comparatives shows if and when AV vendors added detection of the samples missed during the previous on-demand comparatives. Eset is unique among the top performing vendors in adding the majority of the samples they missed only 30-days before the next test, and that statement is true of the past four on-demand comparatives. To me, that looks like benchmark management rather than trying to provide timely detection.

    To claim that only new malware detection is important denies the fact that you can be infected with already known malware. To claim that only detection of known malware is important denies the fact that new malware is being created all the time. Obviously, both are important. Whether NOD is doing a good job on both depends on what you think of the AV-Comparatives test set and their benchmarks in general.

    Personally, I would very seriously consider deploying NOD in my enterprise, if it only supported exclusions for the on-demand scanner. Without that feature, running an on-demand scan is like playing Russian roulette with your domain controllers, Exchange servers, SQL servers, etc.

    /jab
     
  22. Blackspear

    Blackspear Global Moderator

    Joined:
    Dec 2, 2002
    Posts:
    15,115
    Location:
    Gold Coast, Queensland, Australia
    It was, and then reopened, we are not going to have a comparison thread running in the the NOD32 Support Forum, if it starts to head that way then this thread will be moved again further down into another forum so the discussion can continue.


    I'm not saying that either, however as Marcos pointed out with bot generation there needs to be another approach looked at, some form of generic detection/filter such as Heuristics. Now it may end up being called something else other than "Heuristics", however the concept will remain the same, and no, we are not there yet, not even close.

    Cheers :D
     
    Last edited: Jun 3, 2007
Thread Status:
Not open for further replies.