Tests other than Matousec?

Are any anti-malware or HIPS softwares evaluated anywhere else? It seems that Matousec's testing methodology is severely limited, and the effectiveness of many of these programs in the real-world is suspect.

How can so many people justify spending money on, for instance, an anti-keylogging software with unproven effectiveness? This would be like hiring a security guard based on their own personal statement alone.

It doesn't make sense. If no one is properly testing these softwares, then why doesn't an expert from this forum take up the task? Or is it possible that this market is too small, scam-ridden, and unprofessional to be worth benchmarking in the first place?

 

I would suggest doing your own tests. Then you'll know for sure which software works best for you.

 

I didn't claim to be an expert capable of doing an evaluation like that properly. I don't think 'personal experiences' are worth much in terms of informing users of the value of a program, either (e.g., feedback like "it stopped a virus one time" is near worthless).

Are there publicly accessible databases of malware samples?

 

Do you really think this can be done on a lazy afternoon?

Gerard

 

Yes there are, but you have to find them yourself, it will not be posted here.

Gerard

 

Actually Matousec's tests are quite extensive, & (IMO) ARE meaningful when applied primarily to HIPS programs (or to programs with a HIPS component).

My dislike of Matousec's tests has to do mainly with the following:

1- He applies HIPS tests to several apps which do NOT have HIPS.

2- His business practices apparently manifest an implied pressure for security apps to sign-up/pay-up.

 

You would think the financial imperative is lessened by the affiliate sales comissions he generates from click through sales of the top performers.

Always been weary of test results that have a buy it now button alongside them.Too many untrustworthy rogue testers/webpages use that model.

Just my 2cents but a sole up front fee or service charge is better

 

I have never found the "do it yourself" advice helpful. Few are able to do much testing due to lack of expertise, and equipment. I also agree that the results would only be useful for that equipment, for those samples, and on that day.

I doubt that those who give such advice are capable of doing it well, and if so then let us know what testing you have done, the methodology, and the results.

Regards,
Jerry

 

Many other do the same like MRG.

 

You have evidence for that? From what I've seen MRG are more than happy to go legal if you slander them. So you really ought to have evidence for that or retract it.

 

If I understood ok bellgamin is just saying that if you want collaboration or any detailed information about the test you need to pay.

 

Bellgamin is saying that about Matousec. You're saying that about MRG. Like I say, you better have evidence or retract it.

 

As far as I know MRG does not work for free, and don't see nothing illegal on this, or something bad.

 

Hi guest,

Let me clarify how MRG Effitas operates.

We provide a range of services, including efficacy assessments, support services etc. for security vendors, financial institutions etc. Nearly all this work is private and therefore never seen by the general public. Clearly, for these services, clients have to pay.

For the kinds of tests you see us do – the flash tests, PUA reports, online banking reports etc. vendors do not necessarily need to be a client (some are – but we don’t disclose names) to be included.

If a vendor wants to use any of our reports for marketing purposes they have to license them from us. Existing clients generally have the right to use these reports as part of their contract with us.

We will supply missed samples to any vendor who contacts us – indeed, we have, on occasion, supplied many millions of samples to vendors without charge.

We sometimes cut or waive fees for smaller developers. A disproportionate amount of innovation is coming from small vendors so it’s important to help them if you can.

Regards,
Sveta

 

And Ok, I already understood your business
What I wanted to say that there is nothing bad about tester requesting money for their work and help to fix problems. And as far as I know matousec do the same, if you request details you have to pay, if you want to be tested again you pay, but all the leaks tools used are available to download them for free. You can be also tested for free if you wait to the next big round. The bad side, he test app's without full HIPS.

If Scoobs72 see a legal problem on this is his problem.

 

Gizmo criticizes the Matousec's tests. It's a technical reading, but seems fair (http://www.techsupportalert.com/content/matousec-personal-firewall-tests-analyzed.htm).
Others point to an interest conflict on Matousec's tests, reducing their independence (?) (http://smokeys.wordpress.com/2008/04/20/matousecs-firewall-challenge-wrinkle-conflict-of-interests/).

 

One more problem issue with Matousec tests- they do not include sandbox security solution into the test, it's only for traditional, popup-based solutions. The more popups the better, if you are disagree, there is no place for your there.

 

Less is more