SafeOnline vs Leaktests

Hello all,
I've built a list of leaktests which we are aware of for testing SafeOnline. If anyone is aware of any other leaktest, please let me know and I'll add it to the list, analyzing it and adding protection if necessary. The list is fairly comprehensive to cover a number of techniques, but in no way does it show every technique protected by SafeOnline - each of these keyloggers uses a slightly different approach but let me know if you'd like any validation on specific threats.

We've manually tested each of these leaktests on the full range of operating systems but if anyone wants us to replicate the test results, please let me know and I'll make a video demonstrating it.

To better explain the scope of testing SafeOnline, please take the following into account:

- SafeOnline will provide full protection on 32bit operating systems. On 64bit operating systems, SafeOnline relies on the layered protection of Prevx 3.0 for blocking known screen grabbers and clipboard stealers. This is because of fundamental architectural differences in 64bit operating systems preventing modifications to the "shadow service descriptor table" because of PatchGuard. We are planning a workaround for this, but it will still have some reliance on the antimalware components.

- SafeOnline provides protection over credentials and browser instances. Once Prevx sees you are visiting a secured website (i.e. https://www.paypal.com) it will load protection onto that instance. Keystrokes typed into the browser will be protected, but not necessarily keystrokes entered into another program (like Notepad).

- Clipboard protection will prevent any untrusted program from seeing clipboard data. By default, every program is untrusted and Prevx has to manually add a program to be trusted.

- Using build 3.0.5.85 or earlier, a "Compatibility Mode" exists which causes Prevx to not load all of the screen grabber protection. This mode is triggered when configuring protection down from Maximum to High. If you've ever done this within your Prevx installation, it will not be reset by changing configuration back up to High. This is to allow us to persist the configuration past a reboot, but we've since reassessed this approach and will have it changed in the next build. However, currently you will need to uninstall/reboot/reinstall if you have changed configuration options like this.

- SafeOnline is partially incompatible with Zemana AntiLogger and both products cannot load screen protection at the same time.

- SafeOnline will only protect against credential-relevant threats, intentionally not bothering with webcam loggers or sound loggers.

We strongly recommend using real infections to test the protection of Prevx, but we do think that leaktests are a valuable way to test the protection. SafeOnline's protection is incremental to the rest of the protection provided by Prevx 3.0 and while it provides a significant layer on top of what Prevx 3.0 provides, it isn't a silver bullet. However, we have yet to find a leaktest or threat which bypasses SafeOnline when everything is fully configured on a compatible operating system. In the event that something would eventually get past SafeOnline, Prevx will be immediately aware of the threat and will block it using the Prevx 3.0 antimalware functionality.

Our current (partial) list of leaktests is:

• PASSED - Firewall Leaktester - AKLT - GetKeyState
• PASSED - Firewall Leaktester - AKLT - GetAsyncKeyState
• PASSED - Firewall Leaktester - AKLT - GetKeyboardState
• PASSED - Firewall Leaktester - AKLT - DirectX
• PASSED - Firewall Leaktester - AKLT - LowLevel Hook
• PASSED - Firewall Leaktester - AKLT - JournalRecord Hook
• PASSED - Firewall Leaktester - AKLT - GetRawInputData
• PASSED - Firewall Leaktester - AKLT - Screenshot 1
• PASSED - Firewall Leaktester - AKLT - Screenshot 2
• PASSED - Zemana - ScreenLogger Simulation Test
• PASSED - Zemana - ClipBoardLogger Simulation Test
• PASSED - Zemana - Keylogger Simulation Test
• PASSED - Zemana - SSL Logger Simulation Test
• OUT OF SCOPE - Zemana - WebcamLogger Simulation Test
• PASSED - Alpin Software - Through the Eyes of a Keylogger - Key logging
• PASSED - Alpin Software - Through the Eyes of a Keylogger - Screen Logging
• PASSED - Alpin Software - Through the Eyes of a Keylogger - Clipboard Logging
• PASSED - SpyShelter - AntiTest Keylogging
• PASSED - SpyShelter - AntiTest Screenshot
• PASSED - SpyShelter - AntiTest Clipboard monitoring
• OUT OF SCOPE - SpyShelter - AntiTest Webcam Capture
• OUT OF SCOPE - SpyShelter - AntiTest System protection
• OUT OF SCOPE - SpyShelter - AntiTest Sound record
• PASSED - Chpie - Rootkit.com - Global Specific I/O Address Space Trap Keylogger
• PASSED - Greg Hoglund - IDT-based Basic Keyboard Sniffer
• PASSED - Atif Aziz - IECache Viewer
• PASSED - NirSoft - IE Cookie Viewer
• PASSED - NirSoft - IE Password Viewer
• PASSED - NirSoft - Protected Storage PassView
• PASSED - NirSoft - PasswordFox
• PASSED - Pro Data Doctor - Password Recovery
• PASSED - Amecisco, Inc. - Invisible Keylogger Stealth 2.1
• PASSED - Security Xploded - RemoteDll
• PASSED - Keylack Software - Asterisk Password Viewer
• PASSED - Komodia - SSL sniffer
• PASSED - ram verma - LoginMgr (BHO)
• PASSED - YourBankHere Demo - PatchDemo Browser Hooking
• PASSED - YourBankHere Demo - OverlayWindow
• PASSED - Snadboy - Snadboy's Revelation v2 - Password Viewer
• PASSED - System Safety Ltd. - Keylogger 1 (GetKeyState)
• PASSED - System Safety Ltd. - Keylogger 2 (GetAsyncKeyState)
• PASSED - System Safety Ltd. - Keylogger 3 (Low Level Keyboard Hook)
• PASSED - System Safety Ltd. - Keylogger 4 (Journal Record Hook)
• PASSED - DiamondCS - KeyHook
• PASSED - Unknown - Keyboard Listener

Let me know if you find anything else we should test!

 

How about trying Prevx against corporate/commercial keyloggers? They are usually the best at staying undetected and a lot of products choose not to detect them, because most of the time their use is legal.

 

Even if a keylogger is a commercial keylogger, we will block it unless SafeOnline has been explicitly turned off by the organization. I've intentionally left them off of this list as I've focused primarily on leaktest-style keyloggers, but I can definitely compile a list of popular commercial keyloggers and directly test them (or feel free to send in suggestions if you'd like specific ones tested). Every build of Prevx which we send out goes through about a dozen of the major commercial keyloggers (including private builds) across all platforms before being released publicly.

 

Thank you.

I'm pleased to say that, following PrevxHelp's advice about uninstalling and re-installing PrevX 3 to get out of compatibility mode, I now have SafeOnline successfully blocking Zemana's keyboard logging, clipboard grabbing, and screen grabbing test tools.

• keyboard
• clipboard
• screen

I must have enabled into compatibility mode at some point during my testing.

I did lose all the confidential data from my configered sites, but I can enter this again no problem.

I will look at some of the other leak tests when I get the chance.

(Now all we need is the paste protection for confidential data, which is "coming soon", I believe. )

 

Nice list: will keep an eye out for any others.
Might I suggest this be kept as a "sticky" for reference.
The list can be added to by PrevX Help in the original post as/if needed.

Bring back the on site anti-exe

 

I nearly forgot.

Several years ago I wrote a small VB Application to take screen shots of windows (this was before I discovered SnagIt etc.). It wasn't a very sophisticated program; it just used BitBlt to copy pixels from area of the screen occupied by a Window. ( http://msdn.microsoft.com/en-us/library/aa930997.aspx )

Anyway, I dug my program out and ran it and SafeOnline stopped it working.

 

how will PrevX antimalware functionality do that? How will it detect the threat?

 

Prevx 3.0 leverages the added behaviors from SafeOnline to detect malware sooner. SafeOnline is able to see a very granular picture of what a program is trying to do to other programs on the system and data on the system (even more so than the behavior monitoring within Prevx 3.0 alone).

This, in turn, helps feed detections and dramatically improve the proactive aspects of Prevx

 

We've started a new initiative within Prevx to try and formalize the testing of products like SafeOnline - I've started a thread - https://www.wilderssecurity.com/showthread.php?p=1636942 which links to our blog post.

Let me know your thoughts!

 

Good to see vendors actively involved in all manner of testing


Here's a list for you, some of them might not be exactly what you're looking for, but i posted them just in case

Host-based Intrusion Prevention Software (HIPS) Leaktests

keylogger.exe Simple keylogger leaktest. 4 methods to spy keystrokes

www.syssafety.com appears to have gone ? but i have all the files/info etc saved.


Spycar

Spycar is a suite of tools designed to mimic spyware-like behavior, but in a benign form. Intelguardians created Spycar so anyone could test the behavior-based defenses of an anti-spyware tool.

http://www.spycar.org/Welcome to Spycar.html


Scoundrel Simulator

I simulate what a virus, trojan, or other malicious program can do to your computer.

http://www.geeksuperhero.com/scoundrelsim.shtml


gswdemo.vbs

The demo is a simulation of intrusion attacks, virus and mal-ware activity, including:

Information Disclosure attacks, copying confidential files
Infecting executables
Deleting documents
Code injection
Sending control keystrokes to windows (shatter attacks)
Process termination through implicit context of WMI service
Installing a backdoor attacks

http://gentlesecurity.com/files/gswdemo.vbs


Test Your Vital Security Policy

The tools below let you test your Vital Security policy. Each of the tests below focuses on a specific security domain in the product’s security policy.

Anti Virus

Java Script / VB Script

Denial of Service (DoS)

Remote Code Execution (RCE)

Phishing

Code Obfuscation of Malicious Script

Java Applet

ActiveX Control

URL Filtering Security Engine Testing

http://www.finjan.com/Content.aspx?id=577


Free Security Test

Even with good anti-virus, how vulnerable is your PC to Internet attacks?

In the following demo, we will simulate what will happen when you receive a malicious file. It could come in through any number of ways: browsing, as an email attachment, from a USB storage device, just to name a few.

We will attempt to prove that none of your security system's defense layers will identify or alert you to our intrusion attempt.

Note: This is only a demo and no actual damage will be caused to your PC.

TrojDemo.exe and TrojDemo2.exe

http://www.trustware.com/Free-Security-Test/


DFK Threat Simulator

DFK Threat Simulator v1 and DFK Threat Simulator v2

Note this is only for highly experienced users. Don’t play with this thing unless you really know what you are doing.

Although the security community has relied on the "Eicar Antivirus Test File" for years, the complex advances in malware requires a more modern and thorough threat simulation. To this end the "DFK Threat Simulator" was created. Bundling a declawed collection of dropper, rootkit, virus, trojan, spyware, keylogger, leaktest, and alternate data stream technology, the DFK Threat Simulator is a serious representation of the modern dangers facing computer users today.

http://sunbeltblog.blogspot.com/2005/10/new-threat-simulator.html

www.morgud.com appears to have gone ? but i have all the files/info etc saved.


The Zapass Trojan Test

Zapass consists of a Control Interface (zapass.exe) and an Implant dll (zapass.dll), that should be extracted in the same folder. Apart from these two files, absolutely nothing will be written to your system. At any time, you can completely remove Zapass from your system just by deleting these two files.

http://www.whirlywiryweb.com/articles/zapass.zip


I also have multiple collections of various other Leaktests etc that i've aquired over the years. If you or anybody else are interested in them, just ask and i'll upload them offsite for you.

 

Very interesting, thank you.

I can reach morgud.com, but I had to add its Ip address to MalwareBytes' AntiMalware Ip protection whitelist.
Moreover I cannot find the mentioned file, perhaps that site has been defaced, since it looks like a fake one.

Please, see image attached

 

@leofelix

Thanks, yes i can reach the same one you did, but it's not the original site. Seems like Morgud has let it go and now it's just parked waiting for someone to buy it. Shame because he wrote some very articles etc. I couldn't even see any of his old pages in Google's cache ?

If anybody knows what's happened to him please let us know.

 

I disabled SafeOnline; too much problems.

Will that decrease my HOST protection? (Not the webcomponent stuff!)

Btw: Here it does not look like good old PrevX detection rates. As i saw the results i was disappointed.
In my feeling the development of SafeOnline takes too much ressources. In my humble opinion PrevX should concentrate on the really important things: it's detection rate.

 

No it won't - but could you let me know what problems you're seeing with SafeOnline so that we can try nad correct them?

The only link I found in there was to a VT result which says we missed the detection on a file... however, we do detect it - VT uses a significantly different engine from the Prevx 3.0 product so the results found on VT don't really reflect the performance of Prevx. The development of SafeOnline takes place in a completely different part of Prevx than the research department so don't worry as there is no loss of productivity in our antimalware departments If you or any users have found samples we don't detect, please send them to report@prevxresearch.com.

 

How about Matousec.com's tests?

http://www.matousec.com/downloads/
http://www.matousec.com/projects/proactive-security-challenge/faq.php

 

Nice to hear that Prevx also protects against them, I don't have any specific suggestions(just like to be protected against commercial keyloggers too) but testing them would be nice

 

The last big problem with SafeOnline was that i was not able to print Webpages even if i turned SafeOnline completely down. I had to reboot to fix the printing problem, that is absolutely anoying!

And i can hardly see an advantage for me using SafeOnline.

My girlfriend is using SafeOnline but had some trouble too. She wanted to print an ebay ordering affirmation site (https). That does not work. So she had to turn SafeOnline down or define a rule for that specific site. After that it was necessary to close the Browser. Zap, that site was deleted and she could not get back to it.
So all in all SafeOnline is not very popular here..

I think it will be a never ending story to fix all the old and all the new bugs.

PS: Please excuse my terrible english; last night was a bit heavy... ^^

 

Hi Habackuck, thx for your posting!

Indeed. And totally unacceptable! - At least to me and everyone I know (using Prevx).

Exactly. Especially if you have to turn the thing off all the time because of this and that problem and what not! And don't forget ... this whole SafeOnline software was meant for the part of malware that otherwise is not detected (so far) by Prevx to close a small gap. Question: All that 'pain' for such a (hopefully!) very small gain? Really?! - Well, I don't think so.

Thanks for this wonderful (better: horrifying!) example of what SafeOnline can do to you! - Yes, also to you dear fanboy reading this!

Btw: Such things I meant when I many weeks ago doubted that this software is ready to go 'final' or released to the masses soon! - Just imagine the average-Joe user who doesn't probably even see the connection between such problems like this and Prevx and what a nightmare that must be for him if nothing works as it should)!

100% ACK. I am sorry, but it's true and lying about that would help no one!

But maybe it's just a german thing as million other users are so very happy with SafeOnline and having no issues (detected!) ... or PrevxHelp wants us make to believe!*kidding*

This is what I thought from the very beginning where nearly every program needed a special fix ... sure they fixed many things so far, yeah great and I mean it, but now it's many months later and still there are so many programs existing (and updated all the time!) and countless possibilities for (always new) incompatibilities! So I am very skeptical too, but who knows ... maybe they can do that? If they didn't give up already?

But how they could have the brilliant (WTF?!) idea to implement a mysterious 'compatibility' (= f*cked up feature workaround) mode silently without any documentation, which is (or at least was) completely non-transparent to the user (who wants to trust a slider bar if it says maximum of course, don't you think?!) is beyond me! - At least my slider bar works now - also very mysteriously - only since a few builds and weeks even if PrevxHelp told me nothing was changed regarding this though I could of course clearly see the difference on as much as 4 different systems - regarding screenshot protection and black screens.

However ... if especially this very serious printing problem isn't fixed (so that one can have at the *same* time maximum protection for https) this whole SafeOnline thing is anyway just a very bad practical joke device to me, sorry!

And GREAT!!! Now I have to inform my relatives to shut down SafeOnline if they don't want to be bitten some day by that nasty printing issue in an important situation like the girlfriend of habakuck had to go through! (Thx for the warning!)

And to get finally to the topic: All those leaktests you want to talk about here are a total waste of time in my opinion if such ordinary things like printing websites aren't solved way *before* that. If I want to cripple my system and make it unusable I just delete my system folder or whatever, don't need Prevx and total 'SafeOnline paranoia maximum annoyance' for that, i am really sorry!

 

Simple: Dont like it, Dont use it.

 

Indeed Clearly the printing issues are limited in scope... we aren't in the business of breaking people's printers but the solution is to change configuration from Maximum to High if there is an issue. This only disables screen grabber protection so it isn't that major of a problem in our opinion.

If the user would see a printing problem, it would happen immediately - we've made significant progress in fixing the issues in the recent builds, but if anyone is still experiencing them, please let me know and I'll see what we can do to schedule a remote session to debug through the problem.

 

yeah, i think he just used that persons post just to continue is rant from before, people take things all too serious, all that text just for another moan. *lol*

i enjoy the simple things, like being able to get back on my PC as ive finished work for the weekend.

 

Here we are Joe; that did not work at all on this maschine!

I had to completely shut down SafeOnline and reboot to fix that problem!

Even if Safeonline was configured to "only detection" the problem persists.

I will never try out SafeOnline again. I took several chances but SafeOnline wasted a lot of time here so for now i wont use it again.

 

I am using version 3.0.5.91.

I have been trying to get Zemana AntiLogger to "play nice" with SafeOnline. I have had some success with AntiLogger on another computer that does not have PrevX 3.0 installed.

However, I can't get the two programs to work together. I have turned Zemana AntiLogger screen protection off but left SafeOnline on Maximum. I also set SafeOnline to High. Whatever I do, these programs don't seem to provide independent layers against key logging (using the AKLT tester). Sometimes, when both protections are on, key logging is possible, sometimes its possible with either layer on.

Am I expecting too much to get these to behave together? If so I will just delete AntiLogger (I didn't pay for it, loaded it from a free promo link here at Wilders).

Edit. Don't worry too much about this. There are more serious issues -- see post #25.

 

Yes! Why would you possibly want to run both together? They provide the same scope of protection. Even if you do manage to appear to get them both working, they may conflict and neither offers protection when you most need it. It's like running two anti-virus - don't do it.

 

You're right of course scoobs72. I just tried it because AntiKeylogger is supposed to have some limited compatibility with SafeOnline.

I have since removed AntiKeylogger and have been re-testing SafeOnline on its own against the AKLT leak test. I previously thought this was working, however, I now find my initial tests were not thorough enough. Here are the results against the seven key logger techniques which AKLT offers. Using v 3.0.5.91.

(NOTE I restored my disc image to a date before I started playing with AntiLogger, so I know there are no remnants for these tests).

All tests were run on an HTTPS site with Configuration at Maximum.

• GetKeyState. Most characters blocked but number keys 0-9 are not blocked.
  
• GetAsyncKeyState. Most characters blocked but number keys 0-9 are not blocked.
  
• GetKeyboardState. All characters blocked.
  
• DirectX. Not blocked. (lower case are converted to upper case, numbers are scrambled)
  
• LowLevel Hook. Not blocked. (lower case converted to upper case, numbers not blocked)
• JournalRecord Hook. Unable to test (blocked by UAC)
• GetRawInputData. All characters Blocked.

I could swear that it passed the DirectX test a few days ago. But now it doesn't. I tried doing an uninstall, then re-install of PrevX but I get the same results. I have also gone back to version 3.0.5.85 and it gives the same results as well.

Something's wrong.