Enterprise Endpoint Testing Results (Jan 2011): Real World Malware Blocking

Commissioned by Trend Micro and performed by AV Test GmbH:

http://us.trendmicro.com/imperia/md...erprise_endpoint_comparative_report_final.pdf

 

Having a hard time finding those results serious.

 

Thanks, I needed a laugh.

 

AV-Test.org Efficacy Benchmark: Real World Testing Report (Commissioned by Symantec Corporation, date of the report: January 27th, 2011, last update: February 10th, 2011).

AV-Test.org Efficacy Benchmark: Remediation Testing Report (Commissioned by Symantec Corporation, date of the report: January 27th, 2011, last update: February 10th, 2011).

 

My AV-Test results trumps your AV-Test result . . . . .

Sigh.

So why did 2 reports from the same tester (AV-Test) have 2 very different results? It is all in how you configure the test. This report was carefully configured to test Trend's strength - URL filtering and not to test their weakness - malware detection.

The Trend report is all about malicious URLs - no malware was delivered by email, USB, network share, CD/DVD . . . .

Trend's first line of defence, and their strength is URL reputation. They block known malicious URLs. Will they do as well with malware delivered by email or a drive-by download from a legitmate site that was hacked? This report doesn't say. The Trend product only had to detect 6 malware samples - because they had most of AV-Test's set of malicious URLs in their database. URL filtering is great, I'm all in favor of it. But it is not sufficient. Again, what if the malware came by network share, USB, email or from a legitimate site? What if the malware sample came from a site that Trend didn't have on their block list?

The bottom line, this wasn't a test of malware detection, it was a test of known malicious URL detection. Was his sample set of URLs sufficient? Statically significant? Where did he get it from? The report doesn't address those questions.

Oddly, in a test of enterprise products, AV-Test chose to test Symantec's older, SEP SBE (Small Business Edition) instead of testing Symantec's newly announced (though not yet shipping) SEP 12.1 for enterprises. He certianly knew of 12.1 - since he was testing it for us. But Andreas Marx never asked to use it in Trend's report.

His other reports (done for my employer, Symantec) can be found here:
http://www.symantec.com/content/en/...rces/b-real_world_testing_report_OR.en-us.pdf

 

they need their own base to sell you know.

 

200 URLs represents less than 2% of my current live watch list and this does not include any 'stepping stone' URLs.

I post this for a little perspective on what 200 may or may not represent.