Secunia Test of Internet Security Suites

[Oldjim]

Is this really a valid test http://secunia.com/gfx/Secunia_Exploit-vs-AV_test-Oct-2008.pdf

[rolarocka]

A bit more info:
http://secunia.com/blog/29/

[doktornotor]

{QUOTE-> Is this really a valid test http://secunia.com/gfx/Secunia_Exploit-vs-AV_test-Oct-2008.pdf <-QUOTE}

Antivirus solutions won't plug in the holes in your applications no matter how much some people would like to (and then we get into the area of those HTTP scanners and similar wannabe solutions). You'll do yourself a much better service if you install something like Secunia PSI and schedule it to run daily to get alerted about security hotfixes.

(The above completely disregards stuff like vulnerabilities in AV software itself, where the cure proves worse than a disease (http://secunia.com/advisories/23278/) and in fact those AVs help the attackers compromise affected systems.)

[Stefan Kurtzhals]

While Secunia has a point in claiming that AVs don't focus too much on detecting vulnerabilities. But the detection rate of the AV products would look ALOT better if Secunia had tested exploits that actually download/drop malware and execute it. I am pretty sure most of the HIPS/behaviour blockers are very well optimized for that. Who cares which vulnerability in Word, Excel or PowerPoint allows you to execute code and launch malware - if your HIPS does put Microsoft Office under special surveilance and will catch attempts to drop/launch malware from within the memory of Microsoft Office. That approach is independent of the exploit actually used.

In the end, Secunia wants to sell a product, the message of this test is accordingly.

Besides, just detecting the vulnerablity will somtimes give you false positives on randomly corrupted documents/files. I had quite a few of that with my Office vuln. detections. Try explaining a customer that his precious document is just corrupted and not an actual exploit, even though the corruption exactly triggers the vulnerability... :(

[Abeltje]

I totally agree with Stefan. They just create some artificial "malware" exploiting vulnerabilities .. but as far as I see this no real damage is done.

If it was real malware programs with excellent detection rates like Avira would probably have cought it.

Furthermore, programs with built in behavior blocker, like KIS, would probably have yelled that something very dangerous is going on / or directly blocked it.

But if there is no real damage .. why block it?

Hence, why this test?

[trjam]

well geez, no wonder their tests are crap when they leave out my sig.8)

[Zombini]

This test has two types of sub-tests, one where a folder was scanned (which is pretty useless), and a second where they browsed to a web-page that was infected (this is a very valid test).

As expected NIS2009 killed the competition. Notice how poor Kaspersky's detection of exploits is since they don't write generic sigs for the exploits for the most part. Rather to reduce FPs, some parts of the shell-code are included in the signature pattern.. very poor.

This test exposes KIS and all other vendors that dont write generic signatures to detect exploits.

NIS2009 detected 10 times more exploits than Kaspersky and others...K fan boys please note.

[TonyW]

{QUOTE->
As expected NIS2009 killed the competition. Notice how poor Kaspersky's detection of exploits is since they don't write generic sigs for the exploits for the most part. Rather to reduce FPs, some parts of the shell-code are included in the signature pattern.. very poor.

This test exposes KIS and all other vendors that dont write generic signatures to detect exploits. <-QUOTE}Though Norton IS apparently trounced the competition, they still failed on 236 exploits out of 300.

With regards to Kaspersky, it should be pointed out KIS 2009 uses the Secunia database when scanning for vulnerabilities.

[Stefan Kurtzhals]

My my, when I get real-world exploit samples from customers (some are targeted attacks), Symantec does not look so shiny (if you can call 20% shiny) anymore - and the rest doesn't fail so "badly" anymore aswell.

What's so bad about detecting shellcode? Of course, it can be easily replaced, but there is a pretty large amount of lazy malware writers that keep using the same shellcode. If you have a good generic shellcode detection, you could even catch a total new exploit if the malware author was too lazy to obscure the shellcode well enough.

[vijayind]

Why are they using Trend Micro IS 2008 , instead of the 2009 version ? TMIS 2009 was released 3 days after Norton 2009.


Anyways, I agree that scanning for vulnerability exploit code is not a solution. The answer is in ensuring that system is patched from vulnerability or the behavioral/HIPS component can thwart possible exploits.

[Longboard]

@Stefan K:
{QUOTE-> My my, when I get real-world exploit samples from customers (some are targeted attacks), Symantec does not look so shiny (if you can call 20% shiny) anymore - and the rest doesn't fail so "badly" anymore aswell. <-QUOTE}
What are you referencing there?

[Jin K]

{QUOTE->
This test exposes KIS and all other vendors that dont write generic signatures to detect exploits.

NIS2009 detected 10 times more exploits than Kaspersky and others...K fan boys please note. <-QUOTE}

im a fan of kaspersky and also for avira!!

anyway i will tell the truth kav generic detection is realy weak!! they dont write generic signatures not just for exploits but also for malware!!! thats my big disappointment on kav :thumbd:

even avast has kill it with its own generic detection!!!

[Zombini]

{QUOTE-> My my, when I get real-world exploit samples from customers (some are targeted attacks), Symantec does not look so shiny (if you can call 20% shiny) anymore - and the rest doesn't fail so "badly" anymore aswell. <-QUOTE}

What are you trying to say ? That Secunia rigged the test. If the rigged the test then it would only be in Kaspersky's favor because KIS2009 uses their technology.

{QUOTE->
What's so bad about detecting shellcode? Of course, it can be easily replaced, but there is a pretty large amount of lazy malware writers that keep using the same shellcode. If you have a good generic shellcode detection, you could even catch a total new exploit if the malware author was too lazy to obscure the shellcode well enough. <-QUOTE}

I'll tell you whats so bad.. You have to keep revving the signatures to keep up with all the variants of shell code out there generated by all the polymorphic shell code generators. Kaspersky labs has revved the Psyme signature 1220 tmes http://www.viruslist.com/en/find?search_mode=virus&words=psyme

[proactivelover]

why don't they test ESET SMART SECURITY

[ASpace]

{QUOTE-> why don't they test ESET SMART SECURITY <-QUOTE}

It is really obvious . Just have a look at the conclusion/final results.

{QUOTE-> In the end, Secunia wants to sell a product, the message of this test is accordingly. <-QUOTE}

[TonyW]

{QUOTE->
I'll tell you whats so bad.. You have to keep revving the signatures to keep up with all the variants of shell code out there generated by all the polymorphic shell code generators. Kaspersky labs has revved the Psyme signature 1220 tmes http://www.viruslist.com/en/find?search_mode=virus&words=psyme <-QUOTE}I wouldn't be surprised if some of those earlier Psyme detections have now found their way into generic signatures.

[Baz_kasp]

{QUOTE->

NIS2009 detected 10 times more exploits than Kaspersky and others...K fan boys please note. <-QUOTE}

Jeez and there was me thinking the "poke fun at people because your antivirus has a bigger peni* than their antivirus" was old.

Cant we just discuss the test without the pointless sideswipes all the time ;D

If your favourite product did well in a test be happy but that doesn't give a license to go around proclaiming everything else sucks (which is pretty silly really)


Now...back to your statement

{QUOTE-> This test exposes KIS and all other vendors that dont write generic signatures to detect exploits.
<-QUOTE}

So?

They tested the file scanner. Big deal. Does Kaspersky and the other products tested consist of only a file scanner or multiple layers of security?

It doesn't matter how a file is detected, as long as it stopped. Its the same thing as saying "Hey, that gun wasn't detected by the Xray machine but it was detected by the metal detector at the door"...that means our security is crap(?)

[TonyW]

Some of you may be interested to read Alex Eckleberry's blog on this test, which include comments from Andreas Marx:

http://sunbeltblog.blogspot.com/2008/10/another-useless-test-grabs-headlines.html

[maymoons]

{QUOTE->

Internet Security Suites fail to block exploits and do little to protect users against exploits, according to a recently released "test" [here] by Secunia, a Danish vulnerability notification firm. I quoted the word "test" as it's very common to see vulnerability companies use close-to-unethical tactics to oversell problems with the AV industry in order to promote their own services [another example here].

Now its Secunia's turn. In their "test" they assume that anti-virus products have poor performance in detecting vulnerability exploits because of their limited focus on traditional AV signatures. So along comes Secunia's Chief Technology Officer (CTO) Thomas Kristensen with the bright idea of testing 12 different Internet Security Suites from McAfee, Norton, Kaspersky, Panda and others against a testbed of exploit files. So far so good, it’s an interesting idea for comparing technologies and I believe it should be performed.

However when testing exploits one very important aspect is that these products don't just rely on traditional signature detection. Yet Secunia's "test methodology" only takes into consideration manually scanning 144 different inactive exploit files. This is very much like saying that you're going to test a car’s ABS breaks by throwing it down a 200 meter cliff. Absurd, sensationalist and misleading at best.

Just to clarify, if you only test 1 part of a product against exploits, which by the way is the part of the product which IS NOT designed to deal with exploits, and leave out of the test the part of the product that DOES deal with exploits and vulnerabilities, there's a very good chance the results will be misleading. Mr. Kristensen, as a Chief Technology Officer, should know this and should be very well aware of the consequences of a faulty methodology. So the question remains, why did he ignore it and just go for the yellow sensationalist approach?

But the absurd doesn't stop with Secunia's flawed testing methodology. Mr. Kristensen concludes that "… major security vendors do not focus on vulnerabilities. Instead, they have a much more traditional approach, which leaves their customers exposed to new malware exploiting vulnerabilities." Well duh, if you only test traditional signatures and neglect the other technologies included in the product which ARE designed to block exploits, what do you expect? Oh, wait, I just saw on their website that Secunia actually sells a vulnerability scanner! Hmmm, I wonder if that has something to do with the flawed conclusions of this test... Internet Security Suites do not rely on signature detection alone since many years ago. Panda's (and other) products integrate behavioral analysis, context-based heuristics, security policies, vulnerability detection, etc. However none of these technologies were tested by Secunia.

Let's just take 1 of the many protection technologies included in Panda Internet Security 2009 which DOES deal with prevention of vulnerability exploitation and see how it behaves against these exploits if tested correctly. I'm talking about Kernel Rules Engine, a security policy technology incorporated in 2004 to all Panda products which effectively prevents zero-day exploits of PDF, DOC, XLS, PPT and many other vulnerable applications. While Secunia's test grants Panda a lowly 1.59% detection rate of the important threats, if they would have tested correctly they would have found out that just with Kernel Rules Engine Panda's product is able to generically and proactively block 56% of the important threats. And this just with KRE technology. But Panda's products also include other technologies such as TruPrevent's Behavioral Analysis, URL Filters and the Vulnerability Detection module which would prevent other exploits if Secunia cared to run their tests with a minimum level of professionalism.
Note to Secunia:
The following exploits (at least), which in your study are marked as "not detected by Panda", are actually detected generically with the correct testing methodology. Hint: have you tried actually "running" the exploits?

SA14896 CVE-2005-0944 PoC.mdb
SA20748#1 CVE-2006-3086 PoC.xls
SA21061 CVE-2006-3655 POC1.ppt
SA21061 CVE-2006-3656 POC2.ppt
SA21061 CVE-2006-3660 POC3.ppt
SA22127#1 CVE-2006-4694 PoC.ppt
SA23540 CVE-2007-0015 PoC.qtl
SA23676#2 CVE-2007-0028 Exploit1.xls
SA23676#2 CVE-2007-0028 exploit2.xls
SA23676#2 CVE-2007-0028 PoC.xls
SA23676#3 CVE-2007-0029 PoC.xls
SA23676#4 CVE-2007-0030 PoC.xls
SA23676#5 CVE-2007-0031 PoC.xls
SA24152 CVE-2006-1311 PoC.rtf
SA24359#1 CVE-2007-0711 PoC.3gp
SA24359#3 CVE-2007-0713 PoC.mov
SA24359#4 CVE-2007-0714 PoC.mov
SA24359#8 CVE-2007-0718 PoC.qtif
SA24359#9 CVE-NOMATCH PoC.jp2
SA24659 CVE-2007-0038 GameOver.ani
SA24664 CVE-2007-1735 PoC.wpd
SA24725 CVE-2007-1867 GameOver.ani
SA24784 CVE-2007-1942 Exploit.bmp
SA24784 CVE-2007-1942 PoC.bmp
SA24884 CVE-2007-2062 GameOver.cue
SA24973 CVE-2007-2194 GameOver.xpm
SA25023 CVE-2007-2244 PoC.bmp
SA25034 CVE-2007-2366 GameOver.png
SA25044 CVE-2007-2365 GameOver.png
SA25052 CVE-2007-2363 GameOver.iff
SA25089 CVE-2007-2498 PoC.mp4
SA25150#1 CVE-2007-0215 PoC1.xls
SA25150#1 CVE-2007-0215 PoC2.xls
SA25150#3 CVE-2007-1214 PoC.xls
SA25178 CVE-2007-1747 PoC.xls
SA25278 CVE-2007-2809 GameOver.torrent
SA25426 CVE-2007-2966 PoC.lzh
SA25619#1 CVE-2007-0934 PoC.vsd
SA25619#2 CVE-2007-0936 GameOver.vsd
SA25619#2 CVE-2007-0936 PoC.vsd
SA25826 CVE-2007-3375 PoC.lzh
SA25952 CVE-2007-6007 PoC1.psp
SA25952 CVE-2007-6007 PoC2.psp
SA25952 CVE-2007-6007 PoC3.psp
SA25988 CVE-2007-1754 PoC.pub
SA25995#1 CVE-2007-1756 PoC.xls
SA25995#2 CVE-2007-3029 PoC1.xls
SA25995#2 CVE-2007-3029 PoC2.xls
SA25995#3 CVE-2007-3030 PoC.xlw
SA26034#4 CVE-2007-2394 PoC.mov
SA26145 CVE-2007-3890 PoC1.xlw
SA26145 CVE-2007-3890 PoC2.xlw
SA26433 CVE-2007-3037 PoC.wmz
SA26619 CVE-2007-4343 Exploit.pal
SA26619 CVE-2007-4343 GameOver.pal
SA27000 CVE-2007-5279 PoC.bh
SA27151 CVE-2007-3899 GameOver.doc
SA27151 CVE-2007-3899 PoC.doc
SA27270 CVE-2007-5709 GameOver.m3u
SA27304#1 CVE-2007-5909 GameOver1.rtf
SA27304#1 CVE-2007-5909 GameOver2.rtf
SA27304#1 CVE-2007-5909 PoC1.rtf
SA27304#2 CVE-2007-6008 PoC1.eml
SA27304#2 CVE-2007-6008 PoC2.eml
SA27361#4 CVE-2007-2263 PoC.swf
SA27849 CVE-2007-6593 GameOver1.123
SA27849 CVE-2007-6593 GameOver2.123
SA27849 CVE-2007-6593 GameOver3.123
SA28034 CVE-2007-0064 PoC1.asf
SA28034 CVE-2007-0064 PoC2.asf
SA28034 CVE-2007-0064 PoC3.asf
SA28034 CVE-2007-0064 PoC4.asf
SA28083#2 CVE-2007-0071 PoC.swf
SA28092#1 CVE-2007-4706 PoC.mov
SA28209#10 CVE-2007-5399 PoCbcc.eml
SA28209#10 CVE-2007-5399 _PoC_cc.eml
SA28209#10 CVE-2007-5399 PoC_date.eml
SA28209#10 CVE-2007-5399 PoC_from.eml
SA28209#10 CVE-2007-5399 PoC_imp.eml
SA28209#10 CVE-2007-5399 PoC_prio.eml
SA28209#10 CVE-2007-5399 PoC_to.eml
SA28209#10 CVE-2007-5399 PoC_xmsmail.eml
SA28209#11 CVE-2007-5399 PoC.eml
SA28209#12 CVE-2007-5399 PoC.eml
SA28209#13 CVE-2007-5399 PoC.eml
SA28326 CVE-2008-0064 GameOver1.hdr
SA28326 CVE-2008-0064 GameOver2.hdr
SA28506#1 CVE-2008-0081 Exploit.xls
SA28506#1 CVE-2008-0081 PoC.xls
SA28506#2 CVE-2008-0111 PoC1.xls
SA28506#2 CVE-2008-0111 PoC2.xls
SA28506#2 CVE-2008-0111 PoC3.xls
SA28506#4 CVE-2008-0114 PoC.xls
SA28506#7 CVE-2008-0117 Exploit.xls
SA28506#7 CVE-2008-0117 GameOver.xls
SA28506#7 CVE-2008-0117 PoC.xls
SA28563 CVE-2008-0392 Exploit_CommandName.dsr
SA28563 CVE-2008-0392 GameOver_CommandName.dsr
SA28765 CVE-2008-0619 PoC.m3u
SA28765 CVE-2008-0619 PoC.pls
SA28802#1 CVE-2007-5659 GameOver.pdf
SA28802#1 CVE-2007-5659 PoC.pdf
SA28904#2 CVE-2008-0105 PoC1.wps
SA28904#2 CVE-2008-0105 PoC2.wps
SA28904#3 CVE-2007-0108 GameOver.wps
SA29293#1 CVE-2008-1581 PoC.pct
SA29321#2a CVE-2008-0118 PoC.ppt
SA29321#2b CVE-2008-0118 GameOver.ppt
SA29321#2b CVE-2008-0118 PoC.ppt
SA29620 CVE-2008-0069 GameOver.sld
SA29650#5 CVE-2008-1017 crgn_PoC.mov
SA29704#1 CVE-2008-1083 PoC.emf
SA29704#2 CVE-2008-1087 PoC.emf
SA29838 CVE-2008-1765 Exploit.bmp
SA29838 CVE-2008-1765 GameOver.bmp
SA29934 CVE-2008-1942 PoC_ExtGState.pdf
SA29934 CVE-2008-1942 PoC_Height.pdf
SA29934 CVE-2008-1942 PoC_MediaBox.pdf
SA29934 CVE-2008-1942 PoC_Width.pdf
SA29941 CVE-2008-1104 Exploit.pdf
SA29941 CVE-2008-1104 PoC.pdf
SA29972 CVE-2008-2021 PoC.ZOO
SA30143#1 CVE-2008-1091 PoC.rtf
SA30953 CVE-2008-1435 PoC.search-ms
SA30975 CVE-2008-2244 PoC1.doc
SA30975 CVE-2008-2244 PoC2.doc
SA31336#2 CVE-2008-3018 PoC.pict
SA31336#4 CVE-2008-3020 PoC.bmp
SA31336#5 CVE-2008-3460 PoC1.wpg
SA31336#5 CVE-2008-3460 PoC2.wpg
SA31336#5 CVE-2008-3460 PoC3.wpg
SA31385 CVE-2008-2245 PoC.emf
SA31441 CVE-2008-4434 PoC.torrent
SA31454#X CVE-NOMATCH PoC.xls
SA31454#2 CVE-2008-3005 Exploit.xls
SA31454#2 CVE-2008-3005 PoC.xls
SA31675#3 CVE-2008-3013 PoC.gif
SA31675#4 CVE-2008-3014 PoC.wmf
SA31675#X CVE-NOMATCH PoC.emf
SA31675#X CVE-NOMATCH PoC.wmf
SA31675#5 CVE-2008-3015 PoC.ppt
SA31821#6 CVE-2008-3626 PoC1.mp4
SA31821#6 CVE-2008-3626 PoC2.mp4

<-QUOTE}

http://research.pandasecurity.com/archive/Exploits-vs-Antivirus-_2D00_-The-Last-Stand.aspx

[chrisretusn]

Test seem pointless to me. It simple says that Norton has a larger signature database.

I routinely download viruses (I check every malware link I get.) Downloading is not a hazard, more often than not the file downloaded is a variant and quite often the file it self is not detected as malware; however, executing that file triggers alarms, assuming I allow it to run in the first place. Just checking a file is great, but more important is stopping the virus from installing. I would suspect that many of those listed products would have fair much better if these malware files had been executed.

Edit: My Security Suite was not tested; which is fine with me.

[vijayind]

Secunia exploits security suites flaws :
http://news.cnet.com/8301-1009_3-10066975-83.html

{QUOTE-> The Secunia test departed from the traditional testing done by organizations such as AV-test.org and AV-comparatives.org, which use collections of malware to demonstrate the on-demand and heuristic capabilities of the security products. Secunia used exploits--not viruses and worms--to demonstrate the need for users to patch vulnerabilities as well as have a good firewall, antivirus, and other anti-malware protection. The company said exploits are what criminals are most likely to use these days, and faulted the tested security vendors who said their products could protect against any threat.

Secunia did single out one product, Kaspersky Internet Security, as providing a vulnerability scanner, yet Kaspersky also did poorly on the test.

But Alex Eckelberry of Sunbelt Software criticized Secunia's report as being a "useless test." And others, too, have criticized the metholodgy used.

<-QUOTE}

[aigle]

AV vendors are crying as most of them persorm poorly in the test.

IMO detection of explits is very important. So I will say it,s sgood test indeed. You will now see the vendors adding sgnatures for the exploits more n more.

[Medank]

{QUOTE-> AV vendors are crying as most of them persorm poorly in the test.

IMO detection of explits is very important. So I will say it,s sgood test indeed. You will now see the vendors adding sgnatures for the exploits more n more. <-QUOTE}

I totaly agree with you..

And i think most of the user's here says this av test sucks just because their AV did not score good maybe? it's a question?

[rolarocka]

At least this test made the vendors think about this exploits and maybe they will consider detection of them.

[kwismer]

{QUOTE-> AV vendors are crying as most of them persorm poorly in the test.

IMO detection of explits is very important. So I will say it,s sgood test indeed. You will now see the vendors adding sgnatures for the exploits more n more. <-QUOTE}

no, detection of malicious code is very important, not detection of lab-grown exploits... how are you supposed to use those exploits to test how vulnerable your applications are if your av is blocking access to them? the entire premise of the test is ridiculous on that basis alone...

the fact that they misused the security suites means their methodology was retarded, and the fact that secunia has a financial incentive to try and tarnish the reputation of av vendors just makes it all the more like some unethical marketing exercise...

[aigle]

{QUOTE-> no, detection of malicious code is very important, not detection of lab-grown exploits...
<-QUOTE} What is in the lab today, might be in the wild tomorrow.

Detection of one exploit might be equal to etection of hundreds and thousands of malicious codes.

I will not discuss whether the exploits used by them were important or not. They started a new trend. May be some one can perform same tests in a more better way, using in the wild exploits etc.

[kwismer]

{QUOTE-> What is in the lab today, might be in the wild tomorrow.

Detection of one exploit might be equal to etection of hundreds and thousands of malicious codes. <-QUOTE}

and will also interfere with using the exploit for the legitimate purpose it was intended, thereby making it a false alarm...

[vijayind]

{QUOTE-> AV vendors are crying as most of them persorm poorly in the test.

IMO detection of explits is very important. So I will say it,s sgood test indeed. You will now see the vendors adding sgnatures for the exploits more n more. <-QUOTE}
Aigle, I think you have missed the crux of the test.

Secunia has NOT TESTED for vulnerability detection of systems. But has checked if AV suites can via ON-DEMAND SCAN detect EXPLOIT CODE for various vulnerabilities from within various sample files.

Thats why the tests have been blasted. As stated by Alex Eckelberry in Sunbelt Blog :

{QUOTE-> In most cases, it is simply not practical to scan all data files for possible exploits, as it would slow-down the scan speed dramatically. Instead of this, most companies focuse on some widely used file-based exploits (like the ANI exploits) and some companies also remove the detection of such exploits after some time has passed by (as most users should have patched their systems in the meantime and in order to avoid more slow-downs).

There are a lot more practical solutions built-in to security suites, like the URL filter (which checks and blocks known URLs which are hosting malware or phishing websites) and the exploit filter in the browser (which would also block access to many "bad" websites). Some tools also have virtualization and buffer/stack/heap overflow protection mechanisms included, too.

Then we have the traditional "scanner" -- and even if some exploit code gets executed, a HIPS, IDS or personal firewall system might be able to block the attack. For example, some security suites are knowing that Word, Excel or WinAmp won't write EXE files to disk -- so potentially dropped malware cannot get executed and the system is left in a "good" state.

<-QUOTE}

[aigle]

Ok, I am not an expert so it will be interesting to see what Norton people say about this in this case.

Anyway I am not holding my beath on this test. I just thought detection of exploits is very important for AVs.

[vijayind]

IMHO, the answer lies in Patching and updating Behavioral/HIPS components to prevent exploits. Eeye excels in this.
Signature based scanning for exploit is a very costly method of solving this problem.

So the test is lopsided. If someone tests the whole package in totality for such exploit possibilities, then its worth taking notice.

[TonyW]

{QUOTE->
And i think most of the user's here says this av test sucks just because their AV did not score good maybe? it's a question? <-QUOTE}No, because I don't just rely on my AV to protect me from such exploits. I do, however, keep my system patched and updated with newer releases of the kind of products Secunia keeps track of.

[Inspector Clouseau]

First: Detecting Exploits via Signature in on-demand scan is USELESS. And Secunia should know that! Of course you can add a exploit signature ( for example the shell code ) or even a "generic" signature with several known things such as a NOP 0x90 bed-in and out. However, that's not the point here. AV Vendors DO provide specific detections for this that actually WORK when the exploit is executed. With on-demand scan the exploit is NOT executed, it's just scanned like a normal worm/backdoor/trojan sample. For detecting exploits in a on-demand scan you would have to have some REAL virtualization system and not just a emulation in order to detect that. Only a very few AV vendors have that.

That said: The idea to test that isn't bad. However, how they did that was extraordinary amateurish and unprofessional.

[Inspector Clouseau]

Btw: Good morning Kurt ;D

[EraserHW]

I've to agree with Mike and Stefan.

Just scanning a bunch of self-made exploit test files and claiming security software can't block or prevent exploits it's not really a big test.

Detecting exploits is not trivial. As Stefan said before, just detecting the vulnerability can result in a number of false positives. How do you really know if a corrupted file was intentionally corrupted to be bad or, instead, it was corrupted because of any other reason?

Then, if you want to do a better detection, you've to add a good shellcode generic detection. As Stefan said, shellcodes are often similar because virus writers are really lazy. Even doing so, sometimes this can result in a increase of system resource usage. You can't just scan every file looking for shellcode-like piece of codes, it's crazy. You could tune up the scan for specific files.

But the best thing shouldn't be basing only on signatures to have an exploit detection. There are other ways to isolate exploits, even generically, when they get executed.

Claiming 'poor exploit detection' and basing this sentence only on a signature scan it's not useful to anyone. Well, maybe only for someone.

Anyway, the positive thing - as Mike said - is the concept on which the test has been based. Exploit detection testing isn't a bad idea at all.

[kwismer]

{QUOTE-> Exploit detection testing isn't a bad idea at all. <-QUOTE}

except that it ignores the legitimate use of exploits in operational security...

detecting exploits with actual malicious payloads is one thing, but cart blanche detection of exploits in general has obvious (to me) unintended consequences...

[Zombini]

{QUOTE-> With regards to Kaspersky, it should be pointed out KIS 2009 uses the Secunia database when scanning for vulnerabilities. <-QUOTE}

Having a product that tells that you are unpatched (even when it works correctly which Secunia does not in many cases), is not very useful if you dont apply the patches. Lets face it.. no one applies patches.

[Zombini]

{QUOTE-> Jeez and there was me thinking the "poke fun at people because your antivirus has a bigger peni* than their antivirus" was old.

Cant we just discuss the test without the pointless sideswipes all the time ;D

If your favourite product did well in a test be happy but that doesn't give a license to go around proclaiming everything else sucks (which is pretty silly really)


Now...back to your statement



So?

They tested the file scanner. Big deal. Does Kaspersky and the other products tested consist of only a file scanner or multiple layers of security?

It doesn't matter how a file is detected, as long as it stopped. Its the same thing as saying "Hey, that gun wasn't detected by the Xray machine but it was detected by the metal detector at the door"...that means our security is crap(?) <-QUOTE}

I think you are missing the point. The reason you want to detect the exploit rather than the payload with is a generic signature is to ensure you provide protection that is proactive. You dont have to keep revving the signature just because the shell-code used in the exploit changed or the script used the GUID of the activeX instead of the name.

Yes, I have played with a lot of ADODB.Stream exploit scripts and Kaspersky's protection is pretty easily bypassed. The bad guys know this (unfortunately).

[Zombini]

{QUOTE-> What is in the lab today, might be in the wild tomorrow.

Detection of one exploit might be equal to etection of hundreds and thousands of malicious codes.

I will not discuss whether the exploits used by them were important or not. They started a new trend. May be some one can perform same tests in a more better way, using in the wild exploits etc. <-QUOTE}

Finally someone that understand the real impact on your security if your product cannot generically detect these exploits.. if you cannot detect the exploits, you have no shot at detecting the millions of payloads that these xploits can drop on your machine.

[Zombini]

{QUOTE-> Aigle, I think you have missed the crux of the test.

Secunia has NOT TESTED for vulnerability detection of systems. But has checked if AV suites can via ON-DEMAND SCAN detect EXPLOIT CODE for various vulnerabilities from within various sample files.

Thats why the tests have been blasted. As stated by Alex Eckelberry in Sunbelt Blog : <-QUOTE}

Actually you are only partly correct. Read the PDF carefully. THe web exploits were hosted on a webserver and then they browsed to them using a vulnerable browser. ON-DEMAN scans were not used for this part of the test.

ON-DEMAND scans were used for the exploits that existed in files... that I agree was a poor idea. They should have double-clicked those files using a vulnerable versio of the app.

[Zombini]

{QUOTE-> First: Detecting Exploits via Signature in on-demand scan is USELESS. And Secunia should know that! Of course you can add a exploit signature ( for example the shell code ) or even a "generic" signature with several known things such as a NOP 0x90 bed-in and out. However, that's not the point here. AV Vendors DO provide specific detections for this that actually WORK when the exploit is executed. With on-demand scan the exploit is NOT executed, it's just scanned like a normal worm/backdoor/trojan sample. For detecting exploits in a on-demand scan you would have to have some REAL virtualization system and not just a emulation in order to detect that. Only a very few AV vendors have that.

That said: The idea to test that isn't bad. However, how they did that was extraordinary amateurish and unprofessional. <-QUOTE}

Once again.. another sore AV vendor that didn't read the PDF carefully. It clearly says under Test Methodology that the web exploits were hosted on a web server and they browsed to them using a vulnerable browser with vulnerable versions of 3rd party ActiveX's installed. The security product better detect and block the exploitation of the ActiveX before it exploits the browser. The did not do an ON-DEMAND scan when testing the web-exploits..

Where they did screw up was use the on-demand scan when testing for exploits in vulnerable file formats like swf, pdf, xls etc. That was bad.

Does sunbelt have generic detection for the RDS.Dataspace exploit ?

[vijayind]

{QUOTE-> Actually you are only partly correct. Read the PDF carefully. THe web exploits were hosted on a webserver and then they browsed to them using a vulnerable browser. ON-DEMAN scans were not used for this part of the test.

ON-DEMAND scans were used for the exploits that existed in files... that I agree was a poor idea. They should have double-clicked those files using a vulnerable versio of the app. <-QUOTE}
I agree with regard to your web-script vulnerability test jib. The fact remains, most suites don't have a HTTP scanner. Which do are far to generic and don't delve into checking JS,VBS,SWF for exploit code.

For web based exploits sandboxing or virtualization is the only answer. Since the browser is running with admin rights in most cases and any exploit defeating the browser becomes all powerful. Whereas the browser is usually in the whitelist for AV suites, so their HIPS tend to miss the same.

Another very interesting observation is about Norman and their *sandbox* technology. If it was truly sandbox it would have protected against atleast some attacks. But it looks like Norman Sandox is just a over-rated-marketing-hyped heuristic scanner.

[Zombini]

{QUOTE-> First: Detecting Exploits via Signature in on-demand scan is USELESS. And Secunia should know that! Of course you can add a exploit signature ( for example the shell code ) or even a "generic" signature with several known things such as a NOP 0x90 bed-in and out. However, that's not the point here. AV Vendors DO provide specific detections for this that actually WORK when the exploit is executed. With on-demand scan the exploit is NOT executed, it's just scanned like a normal worm/backdoor/trojan sample. For detecting exploits in a on-demand scan you would have to have some REAL virtualization system and not just a emulation in order to detect that. Only a very few AV vendors have that.

That said: The idea to test that isn't bad. However, how they did that was extraordinary amateurish and unprofessional. <-QUOTE}

Read the PDF.. they did host the web exploits on a web server and browse to them. Its the file exploits that were statically scanned. You dont need shell code or NOP string detection to detect exploits. Check out the browser protection in NIS2009.

As an example, I challenge you to create a working RDS.Dataspace exploit that that when hosted on a webserver will bypass NIS's detection. I have not been able to create one. Just because your product sucks at generically detecting web exploits, doesn't mean you should bash the tests.

[Zombini]

{QUOTE-> I've to agree with Mike and Stefan.

Just scanning a bunch of self-made exploit test files and claiming security software can't block or prevent exploits it's not really a big test. <-QUOTE}

Why ? Because they are self made ? Thats BS. If Secunia can self-make them, so can many other hackers in the world. And they are making them.

{QUOTE->
Detecting exploits is not trivial. As Stefan said before, just detecting the vulnerability can result in a number of false positives. <-QUOTE}
Check out the Browser Protection in NIS2009. Browse to a webpage containing any variation of the RDS.Dataspace exploit (or any other exploit for that matter) using an unpatched XP SP2. I used RDS.Datapsce exploit since its the #1 exploit on the web. See that NIS will detect and block it. Try creating your own using <OBJECT> tags, "new" instantiation, GUIDs, encryption, VBScript whatever... NIS will detect all of them.

Test and learn. If you can come up with a JScript/VBScript that can bypass NIS2009, PM it to me. I have tried all kinds of polymorphic JScript code. If NIS2009 has a signature for it, then no amount of obfuscation will bypass it.


Then, if you want to do a better detection, you've to add a good shellcode generic detection. As Stefan said, shellcodes are often similar because virus writers are really lazy. Even doing so, sometimes this can result in a increase of system resource usage. You can't just scan every file looking for shellcode-like piece of codes, it's crazy. You could tune up the scan for specific files.

But the best thing shouldn't be basing only on signatures to have an exploit detection. There are other ways to isolate exploits, even generically, when they get executed.

Claiming 'poor exploit detection' and basing this sentence only on a signature scan it's not useful to anyone. Well, maybe only for someone.

Anyway, the positive thing - as Mike said - is the concept on which the test has been based. Exploit detection testing isn't a bad idea at all. <-QUOTE}

[TonyW]

{QUOTE-> Lets face it.. no one applies patches. <-QUOTE}I do, don't you?

[Zombini]

{QUOTE-> I do, don't you? <-QUOTE}

If its not an automatic update, then no I don't. Just dont have the time. Besides there is no way to absolutely be certain that you have patches for very single application, and ActiveX object on your machine. You could use Secunia (surprise, surprise), but its FP-prone.

So overall I would bet that even for most people on this forum, there would be at least one ActiveX or application that is still vulnerable on your machine, whether it be a Winzip ActiveX, or a RealPlayer ActiveX, or a WebThunder ActiveX (for our Chinese readers).

Ofcourse all this applies only if there IS a patch available in the first place. And we all know how long that takes.

[TonyW]

{QUOTE-> Besides there is no way to absolutely be certain that you have patches for very single application, and ActiveX object on your machine. <-QUOTE}True, but one can make sure they have the latest version of the application installed. It may not protect you from the newest vulnerabilities discovered since the last release of the program or module, but goes some way to protecting against previous loopholes until the next update or a patch is available.

[Zombini]

{QUOTE-> True, but one can make sure they have the latest version of the application installed. It may not protect you from the newest vulnerabilities discovered since the last release of the program or module, but goes some way to protecting against previous loopholes until the next update or a patch is available. <-QUOTE}

Latest versions of how many applications. 20 ? 30 ? 50 ? Some of the activeX objects dont even have "installers" and certainly no installer updaters. I think its next to impossible.

[Zombini]

Update on the Secunia Test

Secunia has just updated the PDF to include the details of all the Web Exploits they used for tested. Earlier they had only included the file exploits. Now all the htmls have been included as well. VERY IMPORTANTLY, to reiterate, the Web Exploits were tested by first installing the vulnerable ActiveX object, and then browsing to a web page hosting the malicious html. Thats a perfectly legit way to test drive-by downloads and most products sucked.

[emperordarius]

New info

http://secunia.com/blog/30/

[TonyW]

An interesting point here is:

* All security suites were installed with default settings
* All security suites were tested in the same way

Wonder what the results would have been if they were tested at maximum settings.

[cruelsister]

This is a follow-up on the Secunia tests by Aleks at Kaspersky:

October 17, 2008 | 12:57 GMT

By now most people have seen the Secunia test results and all the ensuing discussions. Frankly, I was a bit surprised by the vehemently negative reaction from a number of AV vendors.

And it doesn't seem to be about the 20% difference between the 'winner' and the rest. Criticism has focused on the testing methodology, which many people thought was dubious. Some of the suggestions were useful - mostly those from Andreas Marx, the well-known AV solutions tester from Germany. The general tone, though, seems to be that many AV vendors thought their results would have been a lot better if the test methodology had been different. And maybe they're right.

But I think people are too focused on looking for mistakes in the tests and/or attempting to explain their poor PoC detection rates. Sure, criticizing Secunia's testing methods is justified, but only if we're discussing testing methodology, and nothing else.

As I see it, Secunia wasn't trying to highlight the weaknesses of AV solutions - I think they were trying to make a different point...

At Kaspersky, we've taken a decision not to detect PoC vulnerabilities - it's far more sensible to focus on protecting users from the real threats and exploits that are being used by malware authors in the real world. That's what our antivirus databases are for. The point isn't so much that detecting PoCs is a pretty difficult task (although the test results clearly show that even Microsoft and Symantec, with all of their resources, didn't fare all that well) but that detecting PoC s is a dead end, and doesn't address the fundamental problem.

So what is the problem?

An abundance of vulnerable applications. And the solution for this problem doesn't lie in detecting 65% or even 99% of PoCs. Nor does it lie in good or bad AV testing methodology. The only real solution is proper patch management. In the context of the post test discussion, I get the feeling that a lot of people are conveniently forgetting or ignoring Secunia's "What to do" list:

Users and businesses need to take the threat seriously and realise that firewalls and traditional security software, such as that included in Internet Security Suites, isn't sufficient to protect PCs and corporate networks.
Because the security industry can never offer a protection that matches that of a properly patched program, consumers and businesses have to put more effort into patching their programs. If your programs are vulnerable and unpatched, then you're left quite exposed to new attacks.
What makes patching even more attractive is the fact that it is free-of-charge. It only costs the amount of time invested in downloading and installing the patch/update. With tools such as the free Secunia Personal Software Inspector (PSI) and the similar functionality offered by Kaspersky Internet Security 2009 it is very easy to identify the programs that needs patching.

Fortunately, the AV industry is taking steps to tackle the patching issue. Our product, Kaspersky Internet Security 2009 is so far the first and only product to contain a vulnerability scanner. It identifies applications that have unpatched vulnerabilities - a log gives details of the vulnerability, including a name, threat level and what needs to be done to install the necessary patches.

This is just a first step towards a fully-functional system for managing risk on personal computers, and we're going to continue active work in this area.
We need to treat the disease, not the symptoms. In this case, the disease is all the vulnerable applications which pose a potential risk that is exacerbated by users' lack of knowledge. And this is not something the AV can, or should, tackle alone - it's a matter of security in general.

Moreover, no AV vendor, no matter how well they do on such tests, has the right to say 'Great, we protect you against all exploits, so you needn't patch'. No company would dare say this, and everyone agrees patching is necessary. This fact alone leaves those who are hotly discussing Secunia's test results and methodology without a leg to stand on.

We're happy about the increased awareness of vulnerabilities and the responsibilities of AV vendors that we're seeing. The AV industry can't begin solving the problem of patching soon enough for me. We need both new technologies and user education - we need to talk about patch management until home users understand that it plays just as big a role in security as AV software does.

Patch management begins with the head, and not with the software.

http://www.viruslist.com/en/weblog

[vijayind]

I find it surprising that they missed Eeye Blink (http://www.eeye.com/html/products/blink/features/index.html). Which advertises to be a suite which provides protection against vulnerabilities.

{QUOTE-> Blink delivers integrated multi-layered endpoint protection in a single, lightweight client that replaces multiple security agents, protecting against known exploits, zero day attacks, and all other attack vectors.

FeaturesIntegrated Client Security Agent
Integrated antivirus, antispyware, antiphishing, plus firewall technologies.
Integrated Endpoint Protection
Host vulnerability assessment, intrusion prevention, plus intrusion detection.

Automatic System Protection
Complete system, registry, buffer overflow, plus rootkit protection.
Resource Efficient Security Client
Extremely lightweight, the Blink endpoint protection agent runs at around 66MB.

Prevent Panic Patching
Protects without the need for patches or signatures.
Centralized Policy Enforcement
Single, integrated policy for all security components.

Removable Storage Policy
Prevent data leakage through storage device control (such as USB and FireWire).
Network-Aware Policies
Dynamically set policies and configuration settings based on physical network location

<-QUOTE}

[kwismer]

{QUOTE-> New info

http://secunia.com/blog/30/ <-QUOTE}

looks like he still doesn't get that if you're testing suites you should test all parts of those suites.... otherwise you aren't really measuring the protective capability of the entire suite...

[EraserHW]

{QUOTE-> Why ? Because they are self made ? Thats BS. If Secunia can self-make them, so can many other hackers in the world. And they are making them.


Check out the Browser Protection in NIS2009. Browse to a webpage containing any variation of the RDS.Dataspace exploit (or any other exploit for that matter) using an unpatched XP SP2. I used RDS.Datapsce exploit since its the #1 exploit on the web. See that NIS will detect and block it. Try creating your own using <OBJECT> tags, "new" instantiation, GUIDs, encryption, VBScript whatever... NIS will detect all of them. <-QUOTE}

First at all, thank you for that kind word (BS). I can see you've misunderstood what I've written, and that's because most likely I haven't written it rightly (english is not my native language).

I'll try to explain better my thoughts.

First: This test that Secunia has done is quite interesting, at least in my humble opinion. The ability for a security suite to prevent exploit attacks is quickly becoming a priority. I personally think an exploit is always intended to be "malicious", both if it's only a PoC or it has a malicious payload. An exploit is, by definition, a piece of software, a chunk of data, or sequence of commands that take advantage of a bug, glitch or vulnerability in order to cause unintended or unanticipated behavior to occur on computer software, hardware, or something electronic. Unintended or unanticipated, that would mean something unwanted and not explicitly asked by the user. In other words, something that should be detected.

Now, the problem arises on how to detect exploit attacks. As I've already told, that's not trivial detecting exploits basing the detection only on a plain signature. I've no doubt Symantec can detect RDS.Dataspace or every other HTML/JS/VBS exploit (that wouldn't be totally true, if you agree with Secunia test).

Though when you come to other kind of exploits, then other problems appear. First, as Stefan said in one of his posts, the problem of FPs. Just an example: corrupted files that "look like" but, instead, they are only corrupted files. You've to explain then to your users all these FPs, and sometimes you've to apologize for them.

Often is choosed to add signatures only for those exploits that are really in the wild and used to spread malicious softwares, leaving all the rest to other proactive technologies. Here you can read what Panda Software (and Mike on an earlier post) was saying.

When an exploit is executed, then you could detect it in a easier and more generic way than adding specific signatures for every new exploit discovered. Term "executed" doesn't necessarily mean that it has already run its malicious payload (if there's one) and it has infected the machine.

That's what Pedro Bustamante was saying on the Panda Software blog, that their software use generic heuristic rules to proactively prevent and block exploits...but the exploit has to be run! If it isn't, then it's only a piece of code that can be detected only using plain signatures (With all their pro and cons).

Moreover, when you test security suites, you've to test the whole suite. If you test only the antivirus engine, than it's not a test of the suite. Security suites are designed to give the user more layers of security, to try to prevent every kind of attack. What does it mean? For instance that an exploit that could have bypassed the antivirus engine is then blocked by (just an example) an HIPS module. Or, for instance, the malicious payload is then heuristically blocked. And so on. That's how security suites are designed, to give more layers of security to the user.

In the end, what I want to say with this post is that the test done by Secunia is in my opinion a good idea but it has been run in the wrong manner. You can say: "They have run html based exploits". Yes, true. That's what they should do for every kind of exploit, to give a full overview of security suite's abilities.

I hope I've made my thoughts expressed in my previous post a bit more clear.

Best regards

[Macstorm]

{QUOTE-> This is a follow-up on the Secunia tests by Aleks at Kaspersky:
http://www.viruslist.com/en/weblog <-QUOTE}
It's clearer than water.
Thanks for posting the article :thumb:

[Zombini]

{QUOTE->
In the end, what I want to say with this post is that the test done by Secunia is in my opinion a good idea but it has been run in the wrong manner. You can say: "They have run html based exploits". Yes, true. That's what they should do for every kind of exploit, to give a full overview of security suite's abilities.

<-QUOTE}

I agree. Half the test is wrong, but the other half where they tested web based exploits is correct. So even if Panda could detect exploits when they are run, then they should have been able to detect the web-based exploits because they were in fact run. But they didn't, so I think Panda is trying to blow smoke up everybody's a*s.

There were 157 web-based exploits that were all "run". Panda detected ZERO!! Kaspersky detected ZERO!

[Zombini]

{QUOTE-> It's clearer than water.
Thanks for posting the article :thumb: <-QUOTE}

Yep. Thats the Green K's PR machine in over-drive. Its one think building a product that tells you that your patches are missing (heck.. Microsoft even does that for their products). Its another thing to find all those patches from dozens of different websites and automatically run them (which Kaspersky doesn't do).

And.. its yet another for the customers to even know that they need to run the vulnerability scanner. Overall IMO Kaspersky's approach is a PR gimmick.

Symantec has the best vulnerability database in the world with SecurityFocus. They could easily build such a vulnerability tool. Why do you think they aren't building one ? Because it DOESN't WORK!

[Macstorm]

Oh btw.....as for the thread title..

Secunia Test: gently brought to you by S_ _ _ _ _ _ c . ::)

[Zombini]

{QUOTE-> Oh btw.....as for the thread title..

Secunia Test: gently brought to you by S_ _ _ _ _ _ c . ::) <-QUOTE}

Actually its a cleverly orchestrated marketing stunt by Secunia and K_ _ _ _ _ _ _ y. Just look at K's press release after the "test". Sure does help "K" prove their point that there is a HUGE need for vulnerability assessment (which does not work in practice btw), that COINCIDENTALLY, Secunia happens to be selling.

How convenient::)::)::)::)::)::)::)::)

[cruelsister]

I really think that a majority of the posts on this topic are a trifle harsh. Secunia's point for a while has been that computer users are ignoring a facet of security that is very important and very easily fixed, and that is keeping up to date versions of code that malware writers love to hack into (ie Quicktime, Flash, etc).

I've been using Secunia PSI for quite a while (free, by the way) and it has always alerted me to updates that were critical and would only be found by reading security news.

PSI is a very valuable tool for any out there who don't know what versions of Flash, etc they are currently using and who aren't sure what the most updated version is. To bash a free tool that will provide this information seems to me to be counterproductive especially on a Security forum.

[kwismer]

{QUOTE-> I personally think an exploit is always intended to be "malicious", both if it's only a PoC or it has a malicious payload. An exploit is, by definition, a piece of software, a chunk of data, or sequence of commands that take advantage of a bug, glitch or vulnerability in order to cause unintended or unanticipated behavior to occur on computer software, hardware, or something electronic. Unintended or unanticipated, that would mean something unwanted and not explicitly asked by the user. In other words, something that should be detected. <-QUOTE}

it's a shame this ignores how a great many people in security in enterprises work... they use exploits in order to test the vulnerable surface area of their systems so as to better prioritize patches (because they often have way more patches than they can easily deal with) and to make sure patches actually work... that won't be possible if the anti-malware industry starts adding detection for benign exploits...

ignoring these customers will hurt the industry and ultimately hamper security...

[huangker]

{QUOTE-> I really think that a majority of the posts on this topic are a trifle harsh. Secunia's point for a while has been that computer users are ignoring a facet of security that is very important and very easily fixed, and that is keeping up to date versions of code that malware writers love to hack into (ie Quicktime, Flash, etc).

I've been using Secunia PSI for quite a while (free, by the way) and it has always alerted me to updates that were critical and would only be found by reading security news.

PSI is a very valuable tool for any out there who don't know what versions of Flash, etc they are currently using and who aren't sure what the most updated version is. To bash a free tool that will provide this information seems to me to be counterproductive especially on a Security forum. <-QUOTE}
Well said. Patch management should be an integral part of maintaining security on your system. That is the point of secuina's test. Why are we bagging out certain AVs?

[TonyW]

{QUOTE->
I've been using Secunia PSI for quite a while (free, by the way) and it has always alerted me to updates that were critical and would only be found by reading security news. <-QUOTE}I ran the PSI tool for a test drive, and it seems I'm "patched". The KL vulnerability scanner doesn't alert me to missing patches either. I suppose this is because I've kept my programs current with the latest releases. That's not to say a vulnerability may appear between now and the next release, but I can't do anything about that till the patch/software update is available.

By keeping on top of things, you're narrowing the chances of getting hit, and I think this is the whole point with regards to patch management.

It's worth noting that by the time vulnerabilities are disclosed by the likes of Secunia, patches or updated versions of the software are usually available.

[cruelsister]

PSI only alerts you if the patches exist. The whole point of it is to alert you to update, not to expose vulnerabilities.

It's useful, not theoretical.

[doktornotor]

{QUOTE-> PSI only alerts you if the patches exist. <-QUOTE}

Not exactly; it alerts you of end-of-life and unpatched vulnerable stuff as well as long as it's in their vulnerabilities DB. And yeah, it's useful of course, don't really get all the fuss here...

[Macstorm]

{QUOTE-> Actually its a cleverly orchestrated marketing stunt by Secunia and K_ _ _ _ _ _ _ y. Just look at K's press release after the "test". <-QUOTE}
Secunia obviously knows how to choose their technology partners :thumb:

Go Kaspersky go! 8)

[Zombini]

{QUOTE-> Secunia obviously knows how to choose their technology partners :thumb:

Go Kaspersky go! 8) <-QUOTE}

Yep; they sure did pick the "winning" horse ::)

[TechOutsider]

{QUOTE->
NIS2009 detected 10 times more exploits than Kaspersky and others...K fan boys please note. <-QUOTE}

Well kudos to Symantec. NIS09 obviously has definitions for exploits; Symantec has a whole library of "bloodhound.exploit.XX" definitions, and I actually stumbled in one of those detections recently.