Prevx scored no.1 in test

well, maybe its just me, but i install my security software as my FIRST installation after a fresh OS installation, so id pretty much 100% say, my systems before i install my security are clean. (so, not impossible)

im sure, there are many others who install their security as their first installation after a fresh OS.

if DW says it needs to be installed on a clean system, why test it differently?

also, your stating that DW is easily bypassed but wont state how, what is the developers thoughts on this?

 

Sveta MRG

can u test appguard 1.4.7? it provide a protection even if u got infected by monitoring critical system area ..

http://www.blueridgenetworks.com/products/appguard.php

 

If you are asking me did someone commission us to conduct this test, the answer is absolutely no. This is an official test and as such, nobody outside MRG knew about it.

You asked me how we are funded. We conduct private testing, analysis and research for numerous vendors. Private tests are not made public and their use is for analysing and improving products only.

If anyone wants to use our tests in any way, they must first purchase a license.

We reserve the right to maintain client confidentiality and therefore will not disclose client names are without their express permission, nor will we disclose fees.

 

Installing security applications first is not a common practice by many people.

You also have to take into consideration that most people stick to their systems for a long time and install applications on it, so in 99% of the cases you can't guarantee that the system is clean.

 

well, M.R.G seems to test Prevx's detection, and it always comes off as one of the worst detectors out-there.

I can run as many as possible, ranging from new Zero-day samples, to ones a few weeks, months, it doesn't matter to Prevx, hardly anything gets through Prevx, and this isnt even running highest settings. (High/Med/Med)

according to HMP with all its engines, this was my remaining infections:



Im sure you guys are just right clicking a folder of believed-to-be-infected files and cleaning the machine, seeing which files are left.



* & just because everyone else does, i ran MalwareBytes just to be sure.




So, i decide to find that one missing file (copy>paste to desktop) , and perform a simple right-click scan. (this is after a full advanced system scan - which completed and told me clean/protected)

and now i get this:



I am not sure why Prevx acted like this on that last file, but are we now seeing Prevx perform 100% in one of my own little tests, never seen that before on my machine.


*************

Ok, We are not seeing Prevx get a 100% on my machine, here is the results of that 'delayed detection' on that one remaining file.

Its a screen that ive personally never seen before, and its nice to see. (strange i feel like that really, but i think its good that if the software fails on one file, that this process is not left un-attended )

of course, this has happened during my some-what frequent tests of my software, and this was the only file that Prevx has left through, so VERY VERY impressed.

 

the puzzle is not quite over,

although Prevx is saying it has failed in its removal of this one file, MalwareBytes seems to think differently.



however, I think MBAM is incorrect in its assumption, Prevx is correct in its failure, both Prevx and HMP still detect them, gonna try HMP's removal now.

To all whom who may be interested, Hitman Pro could not remove it either, both prevx and HMP still detect it.

weird thing is, not only did prevx not immediatly detect it, the prevx engine in HMP is still not detecting it.

just thought id share that.

 

This is the only thing in your post that's actually of some use. You tested DefenseWall by running the executables as untrusted?
That would be completely different from what you and others implied in previous posts.

 

Thank you, Sveta, for the clarification. In the future, I recommend that you include this information as a footnote in your published “official tests.”

 

I'd suspect that if you run another scan, it will likely clean it up. Detection for your sample was added purely automatically after it was seen to bypass Prevx initially on your PC - that screen prevents cleanup from running around and instead tells the user that they will likely need assistance. However, in your case, it looks like detection was added after one scan which triggered the cleanup message erroneously: if we had run a normal cleanup after, it would have likely cleaned it up without a problem.

However, it's definitely worth bringing that screen to public view more so than it has been It is a relatively rare screen to see, but it is how we ensure that our customers receive the best experience. If we see that a sample was not successfully cleaned on the first round, we will immediately send them to that page and tell them to contact our support directly - even if we would clean it on the next scan, it is generally better to have us intervene in case there is an improvement that we can make

 

Sveta has been responding to most of these posts, but I'd like to echo this one in particular. This is precisely the case we see with our users - while it is definitely an ideal situation if you can get a user to install their security (whether it is DefenseWall, Prevx, or any other security product) immediately after installing their OS and then keep it up-to-date, this is an extremely rare case, albeit a common one amongst forums like Wilders.

Users "in the wild" have any number of bizarre pieces of malware already on their PC - while it is still useful to install a security product on top of that, the test which MRG has performed is looking to see the effectiveness of a security product with pre-existing infections, the most likely case for an infection that would steal banking details.

Even if an infection is not pre-existing, in the case of sandboxing applications, the sandbox would have to cover every application all the time and additionally cover every possible point in kernel mode... which is not possible. So, as long as you have an application which can access kernel mode or have one which enters the system through a non-sandboxed program, you will be susceptible to these types of threats. That is where SafeOnline and other products step in - if a threat already exists on the PC before these products are installed, these products try and circumvent the threat while banking online.

Granted, this is not perfect and indeed there is no silver bullet in any of these solutions - SafeOnline is built on top of Prevx 3.0 and many of the other products require the user to use an up-to-date AV solution.

Although it may seem that I am biased because of how SafeOnline scored, I stand behind the methodology that MRG have used as it is valid for testing threats in these conditions.

 

This is precisely the reason that the MRG test is flawed. SafeOnline has been developed in order to fill a gap left by other security products (including Prevx itself). It's not valid to compare SafeOnline against products that have been designed for an entirely different purpose and which have different operating parameters.

Before SafeOnline was developed, had Prevx anti-malware been included in this test, it might have fared very badly. The methodology used for the test correctly ignored the capability of the products to detect, prevent, and clean malware; as the purpose of the test was to test the ability of the products to protect the browser on an infected system. In this scenario, Prevx would likely have been the first to point out that the methodology was flawed and that Prevx anti-malware must be tested as a whole, including its ability to detect, prevent, and clean; that nothing is perfect and a layered defense is always best, etc, etc.

You just can't take programs using different approaches such as policy restriction, virtualisation, whitelisting, blacklisting, heuristics, HIPS, etc and start comparing them willy nilly with scant regard for what each program does, how it does it, and the range of scenarios that determine and limit its proper use.

EDIT: Minor clarification added.

 

The issues of 'installing the software on a clean system' aside, all of the software tested has specific anti-logger capabilities. The approaches may be different (e.g. HIPS, policy restriction etc) but they all have specific functionality that can be classed as anti-logging. So yes, you can compare the anti-logging capabilities of such programs.

 

I take your point but the test was described as an "Online Banking Browser Security Test", not just a generic anti-logging test. Even Prevx anti-malware (minus SafeOnline) on its own has some anti-logging capability.

Of the programs tested, only Prevx SafeOnline and Trusteer Rapport have been specifically designed for the express purpose of browser security on an infected system. Both programs put a wrapper around the browser, operating as a kind of reverse sandbox. IMHO this puts these two programs in a different class to the other programs tested, also witnessed by the fact that both are aiming at gaining acceptance by the banks. In this respect they are direct competitors in a fairly new genre in which there are as yet few players. It didn't surprise me that SafeOnline did so well in this test, but it did surprise me that Rapport did so badly, given that the test methodology was tailor-made for these two programs to shine. For anybody considering deploying online banking security software, it's useful to know how these two programs stack up against each other.

I'm not looking to start a fight, just expressing the view that the test wasn't a level playing field; and I'll say it again - kudos to Prevx for achieving such a good test result, and kudos to MRG for carrying out the test.

 

It works, just plain works, and that is what some cant stand. To bad, Prevx did it and it is a fact. Perosnally, I am really a follower in the fact Prevx with SafOnline is all you need.

 

Originally Posted by Sveta MRG

He's totally correct, so conducting a test in this manner is more than just appropriate, it is a Real World test. Of course no one can possibly predict exactly what nasties ALL those people out there have already been infected with, but even so it's a very worthwhile exercise, and i wish there where more like it.

 

I agree with the way the test was done also. If the test was about installing on a clean system and then running the malware and using the anti-malware apps as intended, Defensewall and others would have done a lot better, but as the test was about a real world scenario the tests do portray exactly what would be the real world outcome.

 

For the non paranoid relatively safe surfer Prevx would indeed suffice.

 

I think it is crucial to ignore the detection aspects of these products in this test, especially when testing leaktests. It is very easy to detect a single sample, or even a class of samples by behavior, but what happens when the malware authors test their creations against your product until it doesn't detect it any more? It would likely take less than an hour to make a known piece of malware completely undetectable from every AV-style product, so I think taking the detection of files out of the picture is the only valid way to test this type of protection.

 

I don't have a clue
but I like PrevX

 

That's the spirit Konata!!!!

 

Sorry Sveta, Either they were already on the system or Defensewall was bypassed, not both, which is it?


Trust is a big issue with anything security related, be it a security program or security software testing.
Personally I do not trust MRG.

For those who missed this informative thread..............
https://www.wilderssecurity.com/showthread.php?t=251113

 

That's not what Sveta is saying. He is saying that independently of these tests MRG have found a bypass to DW. Hence the argument that DW would have prevented the system being infected in the first place doesn't hold water either.

 

DW was bypassed & then the tests started with samples on the system?

 

Scoobs72, thanks for clearing that up for me. (if it is indeed true)

Yet I still cannot trust MRG.

 

Hope he contacts Ilya with details on the bypass.