Prevx scored no.1 in test

maybe they run DW test while "trusted" the files?
they didn't say.
and also , why they didn't test sandboxie? it also can score 100% (set only browser.exe to run )

so lame test

 

Hi Demoneye.

The reality is that Sandboxie fails all components of this test other than SSL logging.

I’m pushed for time right now, but, I think it could be useful for Joe to post a short article here just outlining the fundamentals on infection vectors and their implications in the real world, as its clear many people are missing the basics here. Without understanding these, I feel people will never truly appreciate why Prevx have implemented their SafeOnline technology in the way that they did and they may falsely believe other applications are protecting them effectively.

Best regards,
Sveta

 

1. P2P clients are covered by DefenseWall.
2. What are the "numerous other vectors" you mention here?

 

Hi, just a simple question. Can you tell us which verctor/ vectors you used to infect the system while using DefenceWall?
I am too curious about this.

 

Hi Ilya,

Please read the methodology detailed in the test. All applications are run with default settings.

Does Drfensewall run ALL available email, p2p, irc, usenet, chat, clients and all browsers etc as untrusted by default? If it does not, then you can’t guarantee it will prevent malware entering the system.

You will see from the methodology that the inactive samples were already on the system as it is a fact that none of the applications tested (including DW) could, by default, prevent these getting there in the real world, therefore, given this fact, the purpose of the test was to see which applications could prevent data being captured from within IE.

Regards,
Sveta

 

The methodology has a bit of flaw. From where these samples came on ur test PC after all? I suppose they were not born from within the test PC!

 

DefenseWall must be installed on a clean system.
All Internet facing apps must be run as untrusted.
It might help in your testing if you understood how a particular security software works.
May I suggest reading the help file on any app before testing.

 

No, the methodology is ok. The way I read it is that the malware has entered the system by whatever method (including and possibly most likely by user error). This test is to test whether the software prevents subsequent theft of data, despite the malware infection.
You could argue that DW should not be included in this test on the basis that it is designed to protect threatgates, not to prevent data theft, and I think there is some validity to that argument. Nonetheless, a key point for me is that DW will not stop, for example, a clipboard logger - it will just alert to the event after it is happened.

Overall, I think this test validates the concept that Prevx SOL will protect your critical data regardless of how the malware enters the system - which is important for novice computer users that may be regularly infecting their system through their own action or inaction.

 

Yes, exactly

We can always argue that this or that program should or should NOT participate is particular test, but overall they should all be able to deliver the performance in terms of protection.

If you pick samples that will suite particular application, you will get excellent results which will mean nothing in real world

Regards,
Sveta

 

Yes, that is correct. DefenseWall do runs all the browsers, e-mail, P2P, IM, IRC and multimedia clients as untrusted by default.

This case why did you include DefenseWall into the test as, according the protection scheme, it must be installed on clean system only? DefenseWall just out of the test methodology and, thus, people are mislead with its results.
Just exclude it from the test and that's it.

 

True. First page of the DW help file:
"Ensure that your system is 'clean' before installing DefenseWall.
Any pre-existing malware or infected files on your system prior to installing any security software can still actively cause data loss and damage."

DW should probably have been excluded from this test.

 

There's no doubt about that,in the case of DW the test was fundamentally flawed.

 

When they are testing a real world scenario, we can,t just suppose that malware entered by watever method, they must tell us the vector/ vectors of these malware samples in the test scenario.

I asked a very simple question and the tester never replied to it.

 

I don't mean to be rude, i really don't, but do you realize how ridiculous this sounds?
You admit you don't know the application, you put the samples on the machine before installing DW and you're still typing away replies?

How strange this methodology. If only i could get through the whole bizarre 'Cure experience'. I could use a proxy of sorts, is that how people read these tests?

 

Clearly flawed test when DefenseWall is included. Giving it a zero, ridiculous.

 

I have no experience of DefenseWall at all.
So please humor me.
The DefenseWall advocates/fans etc seem to be saying that if DW is installed on a 'clean' PC it cannot be bypassed/defeated.
Wow!

 

DefenseWall is no silver bullet solution. However, if used and configured correctly it can provide extremely strong protection.

 

Thats what I thought, so if malware can get past DW then there is no point in saying 'no-fair' to a test that makes that assumption.

 

As stated, this test does not at all test DefenseWall's capabilities nor is it a reflection of DefenseWall. Therefore coming to the conclusion that DefenseWall has failed is false.

 

Not entirely true.

No single piece of software is designed to handle every situation, which is why a layered defence, consisting of multiple approaches, is usually recommended.

Used correctly, software that primarily works by policy restriction (e.g DW) will create a very strong defensive wall against malware infection, but it is important to install it on a clean system as the ability of this type of software to protect an already infected machine will be limited. That's not its intended purpose and Ilya is right to object to his program being included in this type of test.

When doing tests, it's important to do like-for-like comparisons. The whole point of this test was to compare software whose stated aim is to protect the browser against an already infected machine - a kind of reverse sandbox.

As far as I'm aware, only two of the products tested have browser protection as their direct goal, and those two are: Trusteer Rapport and Prevx SafeOnline. The other programs tested only do this indirectly as a by-product of their main function. IMHO, this test should have restricted itself to the two main contenders in this class of software where a direct comparision is valid.

What I found useful about this test is that it validates my decision to go with Prevx SafeOnline, rather than Trusteer Rapport which is freely available via my bank. Kudos to MRG for putting in the effort to test this relatively new breed of software, even though the test methodology was somewhat flawed.

EDIT: Minor clarification.

 

yep, thats pretty much what DW does, puts a wall up to everything, while keeping your system behind that.

DW if used correctly, is as close to 100% detection than any other security software on the market (and yes, i no longer use it)

to me, DW was a little bit of a pain in the butt, manual allowing of software to run as trusted so they would install properly without giving me problems, i know it doesnt sound much, but it did become a bit of a pain to me.

however, I do still really like the software.

 

That's true for almost all the HIPS products, you need to install them on a clean PC, they're primarily for prevention not detection.

MRG didn't even bother to read the help file.

 

I TOTALLY agree with Ilya Rabinovich , u should take DW out of this test or test it while it is working (eg browser is under DW protection).
u can also include acronis true image and tell after system got infected u can always restore image and gain 100% protection , or even eaz fix/ rollback RX .
this test is good for some security software (mostly hips/ behavior blockers ones) for sure not to such as DW (or SB) like software.

 

OK, lets all take a deep breath now

This is the last we shall say on this particular matter, else, this could go on indefinitely.
Let us state some facts:

1) It is a fact that it is impossible to guarantee that any system is 100% clean.
2) It is a fact that it is possible to bypass DefenseWalls default protection and download and run malware on the system using standard Windows functionality.

Given these facts, it is also a fact that:

1) Even if DefenseWall had been installed on the “clean” OS to start with, the samples / code / malware used could still have been installed on the system, bypassing Defensewall.

2) When installing DefenseWall on any system, you can never be sure there is not malware on there already, which, with DefenseWalls default settings, will run as trusted.

Given these facts, it is also a fact that:

1) Having the samples installed on the OS to start with / not describing or including an infection vector has made no difference to the validity of the test since they could be placed on the system and run as trusted anyway.

DefenseWall is described by the vendor as:

“the simplest and easiest way to protect yourself from malicious software (spyware, botnets, adware, keyloggers, rootkits, etc.) and identification theft, that can not be stopped by your anti-virus and anti-spyware programs, when you surf the Internet!”

Given the product is described and marketed in this way, it is entirely proper that it should have been included in the test.
As a matter of interest, even if the tests are run as “untrusted”, DefenseWall is still unable to prevent all data being captured.

In conclusion and to repeat:

1) The protection provided by DefenseWall is easily and completely bypassed using standard Windows functionality.

2) Even if we purposefully run the tests as untrusted, Defensewall fails to prevent all data capture and so fails the overall test.
I hope we can now draw a line under this and move on to more productive discussions.

PS. Please do not ask how we bypassed DefenseWall as we will not disclose this publically. We have described the method to the developer and hope he is able to cover this in future releases.

Regards,
Sveta

 

Sveta, simply for the sake of transparency, can you please clarify if any of the vendors examined in the Online Banking Browser Security Test (March, 2010) has or had a financial relationship of any kind with the Malware Research Group?

Personally, I do not believe that the existence of a financial relationship between a vendor and a testing organization necessarily implies that the results are illegitimate. Yet, it is always wise, in my opinion, to disclose any such potential conflicts of interest.

Thank you.

P.S.: I hope you do not interpret my question as “confrontational” or “offensive” -- neither is intended.