Secunia Test of Internet Security Suites

Discussion in 'other anti-virus software' started by Oldjim, Oct 13, 2008.

Thread Status:
Not open for further replies.
  1. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,160
    Location:
    UK / Pakistan
    What is in the lab today, might be in the wild tomorrow.

    Detection of one exploit might be equal to etection of hundreds and thousands of malicious codes.

    I will not discuss whether the exploits used by them were important or not. They started a new trend. May be some one can perform same tests in a more better way, using in the wild exploits etc.
     
  2. kwismer

    kwismer Registered Member

    Joined:
    Jan 4, 2008
    Posts:
    240
    and will also interfere with using the exploit for the legitimate purpose it was intended, thereby making it a false alarm...
     
  3. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    Aigle, I think you have missed the crux of the test.

    Secunia has NOT TESTED for vulnerability detection of systems. But has checked if AV suites can via ON-DEMAND SCAN detect EXPLOIT CODE for various vulnerabilities from within various sample files.

    Thats why the tests have been blasted. As stated by Alex Eckelberry in Sunbelt Blog :

     
  4. aigle

    aigle Registered Member

    Joined:
    Dec 14, 2005
    Posts:
    11,160
    Location:
    UK / Pakistan
    Ok, I am not an expert so it will be interesting to see what Norton people say about this in this case.

    Anyway I am not holding my beath on this test. I just thought detection of exploits is very important for AVs.
     
  5. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    IMHO, the answer lies in Patching and updating Behavioral/HIPS components to prevent exploits. Eeye excels in this.
    Signature based scanning for exploit is a very costly method of solving this problem.

    So the test is lopsided. If someone tests the whole package in totality for such exploit possibilities, then its worth taking notice.
     
  6. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    No, because I don't just rely on my AV to protect me from such exploits. I do, however, keep my system patched and updated with newer releases of the kind of products Secunia keeps track of.
     
  7. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    First: Detecting Exploits via Signature in on-demand scan is USELESS. And Secunia should know that! Of course you can add a exploit signature ( for example the shell code ) or even a "generic" signature with several known things such as a NOP 0x90 bed-in and out. However, that's not the point here. AV Vendors DO provide specific detections for this that actually WORK when the exploit is executed. With on-demand scan the exploit is NOT executed, it's just scanned like a normal worm/backdoor/trojan sample. For detecting exploits in a on-demand scan you would have to have some REAL virtualization system and not just a emulation in order to detect that. Only a very few AV vendors have that.

    That said: The idea to test that isn't bad. However, how they did that was extraordinary amateurish and unprofessional.
     
  8. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    Btw: Good morning Kurt :D
     
  9. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    I've to agree with Mike and Stefan.

    Just scanning a bunch of self-made exploit test files and claiming security software can't block or prevent exploits it's not really a big test.

    Detecting exploits is not trivial. As Stefan said before, just detecting the vulnerability can result in a number of false positives. How do you really know if a corrupted file was intentionally corrupted to be bad or, instead, it was corrupted because of any other reason?

    Then, if you want to do a better detection, you've to add a good shellcode generic detection. As Stefan said, shellcodes are often similar because virus writers are really lazy. Even doing so, sometimes this can result in a increase of system resource usage. You can't just scan every file looking for shellcode-like piece of codes, it's crazy. You could tune up the scan for specific files.

    But the best thing shouldn't be basing only on signatures to have an exploit detection. There are other ways to isolate exploits, even generically, when they get executed.

    Claiming 'poor exploit detection' and basing this sentence only on a signature scan it's not useful to anyone. Well, maybe only for someone.

    Anyway, the positive thing - as Mike said - is the concept on which the test has been based. Exploit detection testing isn't a bad idea at all.
     
  10. kwismer

    kwismer Registered Member

    Joined:
    Jan 4, 2008
    Posts:
    240
    except that it ignores the legitimate use of exploits in operational security...

    detecting exploits with actual malicious payloads is one thing, but cart blanche detection of exploits in general has obvious (to me) unintended consequences...
     
  11. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Having a product that tells that you are unpatched (even when it works correctly which Secunia does not in many cases), is not very useful if you dont apply the patches. Lets face it.. no one applies patches.
     
  12. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    I think you are missing the point. The reason you want to detect the exploit rather than the payload with is a generic signature is to ensure you provide protection that is proactive. You dont have to keep revving the signature just because the shell-code used in the exploit changed or the script used the GUID of the activeX instead of the name.

    Yes, I have played with a lot of ADODB.Stream exploit scripts and Kaspersky's protection is pretty easily bypassed. The bad guys know this (unfortunately).
     
  13. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Finally someone that understand the real impact on your security if your product cannot generically detect these exploits.. if you cannot detect the exploits, you have no shot at detecting the millions of payloads that these xploits can drop on your machine.
     
    Last edited: Oct 17, 2008
  14. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Actually you are only partly correct. Read the PDF carefully. THe web exploits were hosted on a webserver and then they browsed to them using a vulnerable browser. ON-DEMAN scans were not used for this part of the test.

    ON-DEMAND scans were used for the exploits that existed in files... that I agree was a poor idea. They should have double-clicked those files using a vulnerable versio of the app.
     
  15. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Once again.. another sore AV vendor that didn't read the PDF carefully. It clearly says under Test Methodology that the web exploits were hosted on a web server and they browsed to them using a vulnerable browser with vulnerable versions of 3rd party ActiveX's installed. The security product better detect and block the exploitation of the ActiveX before it exploits the browser. The did not do an ON-DEMAND scan when testing the web-exploits..

    Where they did screw up was use the on-demand scan when testing for exploits in vulnerable file formats like swf, pdf, xls etc. That was bad.

    Does sunbelt have generic detection for the RDS.Dataspace exploit ?
     
  16. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    I agree with regard to your web-script vulnerability test jib. The fact remains, most suites don't have a HTTP scanner. Which do are far to generic and don't delve into checking JS,VBS,SWF for exploit code.

    For web based exploits sandboxing or virtualization is the only answer. Since the browser is running with admin rights in most cases and any exploit defeating the browser becomes all powerful. Whereas the browser is usually in the whitelist for AV suites, so their HIPS tend to miss the same.

    Another very interesting observation is about Norman and their *sandbox* technology. If it was truly sandbox it would have protected against atleast some attacks. But it looks like Norman Sandox is just a over-rated-marketing-hyped heuristic scanner.
     
  17. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Read the PDF.. they did host the web exploits on a web server and browse to them. Its the file exploits that were statically scanned. You dont need shell code or NOP string detection to detect exploits. Check out the browser protection in NIS2009.

    As an example, I challenge you to create a working RDS.Dataspace exploit that that when hosted on a webserver will bypass NIS's detection. I have not been able to create one. Just because your product sucks at generically detecting web exploits, doesn't mean you should bash the tests.
     
  18. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Why ? Because they are self made ? Thats BS. If Secunia can self-make them, so can many other hackers in the world. And they are making them.

    Check out the Browser Protection in NIS2009. Browse to a webpage containing any variation of the RDS.Dataspace exploit (or any other exploit for that matter) using an unpatched XP SP2. I used RDS.Datapsce exploit since its the #1 exploit on the web. See that NIS will detect and block it. Try creating your own using <OBJECT> tags, "new" instantiation, GUIDs, encryption, VBScript whatever... NIS will detect all of them.

    Test and learn. If you can come up with a JScript/VBScript that can bypass NIS2009, PM it to me. I have tried all kinds of polymorphic JScript code. If NIS2009 has a signature for it, then no amount of obfuscation will bypass it.


    Then, if you want to do a better detection, you've to add a good shellcode generic detection. As Stefan said, shellcodes are often similar because virus writers are really lazy. Even doing so, sometimes this can result in a increase of system resource usage. You can't just scan every file looking for shellcode-like piece of codes, it's crazy. You could tune up the scan for specific files.

    But the best thing shouldn't be basing only on signatures to have an exploit detection. There are other ways to isolate exploits, even generically, when they get executed.

    Claiming 'poor exploit detection' and basing this sentence only on a signature scan it's not useful to anyone. Well, maybe only for someone.

    Anyway, the positive thing - as Mike said - is the concept on which the test has been based. Exploit detection testing isn't a bad idea at all.[/QUOTE]
     
  19. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    I do, don't you?
     
  20. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    If its not an automatic update, then no I don't. Just dont have the time. Besides there is no way to absolutely be certain that you have patches for very single application, and ActiveX object on your machine. You could use Secunia (surprise, surprise), but its FP-prone.

    So overall I would bet that even for most people on this forum, there would be at least one ActiveX or application that is still vulnerable on your machine, whether it be a Winzip ActiveX, or a RealPlayer ActiveX, or a WebThunder ActiveX (for our Chinese readers).

    Ofcourse all this applies only if there IS a patch available in the first place. And we all know how long that takes.
     
  21. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    True, but one can make sure they have the latest version of the application installed. It may not protect you from the newest vulnerabilities discovered since the last release of the program or module, but goes some way to protecting against previous loopholes until the next update or a patch is available.
     
  22. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Latest versions of how many applications. 20 ? 30 ? 50 ? Some of the activeX objects dont even have "installers" and certainly no installer updaters. I think its next to impossible.
     
  23. Zombini

    Zombini Registered Member

    Joined:
    Jul 11, 2006
    Posts:
    469
    Update on the Secunia Test

    Secunia has just updated the PDF to include the details of all the Web Exploits they used for tested. Earlier they had only included the file exploits. Now all the htmls have been included as well. VERY IMPORTANTLY, to reiterate, the Web Exploits were tested by first installing the vulnerable ActiveX object, and then browsing to a web page hosting the malicious html. Thats a perfectly legit way to test drive-by downloads and most products sucked.
     
  24. emperordarius

    emperordarius Registered Member

    Joined:
    Apr 27, 2008
    Posts:
    1,218
    Location:
    Who cares
  25. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,741
    Location:
    UK
    An interesting point here is:

    * All security suites were installed with default settings
    * All security suites were tested in the same way

    Wonder what the results would have been if they were tested at maximum settings.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.