MRG Flash Tests 2011

I get what you mean. In MRG's tests Vipre is blocking every o-day so far. But in this test it was knocked because it stuggled to do exactly that. http://www.pcworld.com/article/217609/gfi_vipre_antivirus_2011.html

 

Hi Ianb,

I’m afraid you are looking at the methodology used for last years tests. We have published the methodology for the current tests on our website here http://malwareresearchgroup.com/malware-tests/flash-test-results/ and also in our forums.
You will see, the methodology clearly states that a live URL is used. Given this fact, IP blocking can be assessed.

Regards,
Sveta

 

We couldn’t comment on another organisations test results, only our own, to confirm VIPRE has detected every sample so far by signature (mostly generic).

Perhaps any queries would receive more insightful response or feedback if directed to the vendor.

Regards,
Sveta

 

Thanks for that Sveta.

Don't get me wrong, I think your testing is important but people shouldn't see it as the holy grail (or any other test for that matter).

I know there is some Comodo|MGR history but can I ask why don't you include Comodo in your testing ?

 

Hi Ianb,

MRG focuses on designing tests which model how security applications are used in the real world – this is why all our tests as a minimum use an infection vector, are dynamic and where viable, use live URLs. We run a range of tests for our clients as no single test / methodology will yield results to represent the real world panorama.

The efficacy assessments we provide for vendors often comprise of thousands of individual tests, using malware of various types and ages along with custom designed simulators and POC tools. These flash tests are not designed to be or presented as being a rigorous assessment of any of the products efficacy, but instead are intended to be viewed as a light overview of how a range of applications perform against a single early life threat each day. If nothing else, we hope the tests serve as stimulus for a recognition of the impact of malware which is < 24 hours old has on the total threat landscape.

Regards,
Sveta

 

I wasn't expecting you to comment on PC mags test. Rather my comments were in response to atomomega.

 

I hope that you will not understand me wrong but MRG flash testing looks like a typical one-man testing that many Wilders members performed every day. It is very easy to find the (relatively widespread) malware that Vipre can not detect. No antivirus, which relies solely on signature-based detection (even with an exceptional heuristic), can detect 100% of 0-day malware!? That's why Sunbelt has developed ClearCloud and Vipre has proactive protection in the form of blocking unknown programs. They are professionals who know best that signature-based detection simply are not enough.

If we are well informed, you are using Vipre with default settings (without blocking of unknown programs) and without ClearCloud, of course. Therefore, I am confused because the results of your tests but maybe it is merely a coincidence because you use really a very small number of samples?

 

I didnt relaize there was one standard site for others to learn from.

Personally I like MRG and these tests. You can make whatever you want to from them but to me, it makes for interesting reading. I dont see any harm in that. Thanks Sveta.

 

No harm I echo your sentiments entirely.

 

I guess they also focus on widespread malware not only 0-day.
Malware that will more likely infect the most users and are kinda new.

 

Indeed, in contrary to most useless tests which use so-called zero day malware which will never ever target a consumers pc.

I don't see the harm of these tests, I like them, and think they are an good addition to all the other tests performed. If someone doesn't like these tests, why go through the effort to read and reply on here?

 

If they would used malware that is widespread, then at least some of the top-notch signature-based antimalware would have to achieve much better results. Widespread/well-known malware is normally not a problem for Avira, NOD32, MSE or Kaspersky. It seems that the MRG "24h later" testing points in that direction.

Nosirrah already has provided some valuable explanations regarding the malware that MRG using:

post # 150

post # 162

I have respect for the MRG flash testing. Explanations provided by the Sveta concerning the purpose of the test are fully acceptable to me. I try to point out certain circumstances which are not clear enough for me and that may be important for the validity of the test and interpretation of test results.

 

To clarify...I was referring to the widespread lame malware that is simple to define. Anyone that knows anything about security knows that bankers, zbot, spyeyes and TDSS are indeed widespread.

 

So, they are using bankers, zbot etc. that are widespread but in the same time:

I belive you'll agree with me that widespread malware is easy to find on the web?

I think that information regarding dispersion of the malware they using for testing is of significant importance. If they use widespread malware, Vipre cannot be that better than others signature-based antivirus, particularly those top-notch such as Avira, NOD32 or Kaspersky. If they use malware that are not widespread and which due to is not easy be found, test results have only limited importance.

 

Due to Vipre's steady results, I'd like to know what settings exactly are being used.

As you wrote;

Does anyone know the exact Vipre's config, used in the tests?
I haven't come across this info yet.

The focus on <24 hour/0-day malware in these particular tests would indicate that it can't be that widespread.
Then of course we would also have to know the definition of 'widespread' as being used in this thread.

 

This is actually quite far from true. These have rapidly changing IPs and domains and often use all kinds of tricks to avoid direct downloading the simplest being required cookies and referrals. You might be able to play catchup on the fallout with ease but by then the dropper is long gone.

As far as Viper goes, I have a feeling that I know why they are doing as well as they are but as I do not work for them, it is only speculation. it is likely related to putting more of a 0day effort in than most though.

 

There is a problem here where widespread is being used to cover two things that are not always equal. Widespread can refer to the availability of samples online or it can refer to number of people affected by certain families of malware. Sometimes these are 1:1 in nature but for the very nasty stuff all too often capturing the fallout wont allow prevention of new variants at all. In these cases the impact and availability of fallout samples may be quite high but that does nothing to stop the spread of new variants.

 

I asked the same question and got the answer:

"All security applications are installed with default settings and the most recent build and signatures used in each test."

http://malwareresearchgroup.com/malware-tests/flash-test-results/

So, Vipre was tested without proactive protection (blocking of unknown programs) and, for example, MBAM was tested with IP Blocking.

 

My annoyance may be useful, eventually.

This is exactly what I'm most interested in. If Vipre relied solely on the signatures and heuristic how it managed to achieve 100% success in detection of sophisticated 0-day malware. New kind of heuristic or something else?

Nossirah would you be so kind? What's wrong with speculation?

 

ya, thats my point too, 0day malware cannot be detected so quickly and accurately all the time using signatures

moreover i see in Vipre , i see in default settings , HIPS is disabled yet its catching everything in MRG tests ??

btw , i am using VIPRE and really liking it

 

Hi Nevis...
Are you saying Vipre has HIPS? I am not seeing that. Did I misunderstand you? It sure looks like a good software.

 

yes , it does according to setting but disabled in default settings :

 

They have a pretty good technology called MX-V which is the base of the heuristics engine. This is from Vipre's webpage, look for the MX-V Virtualization paragraph almost at the end of the page.

The HIPS module in Vipre is not a HIPS 'per se'. I mean, it's not a classical HIPS, so it really doesn't have a real impact on Vipre's detection capabilities.

 

ok, thx for clarification

 

If I interpret nosirrah's posts about VIPRE right, VIPRE is doing that good on zero-day malware because they are actively searching on the web and sites as Malware domain list to add the malware signatures for the latest variants. (Hence why ClearCloud DNS is such an attractive addition to one's security setup.)

It would be very difficult to conclude that certain qualities of a vendor are because a fancy technology with a fancy name (such as MX-V or Genscen or whatever), written down in a comparison table. All vendors have such technologies although they might market it differently or have a different approach on how decisive such an technology should be in their product (for example in Norton the combination of SONAR and Norton Insight blocks huge amounts of malware, but there is only one qualification for their heuristics: they suck. That doesn't make Norton bad though, vendors might have a different approach.)