AVLab - "Test of software for online banking protection"

I am a fan of the creativity AV Labs uses in their tests. However, this time they "crossed the line."

AV Labs is not an AMTSO member: https://www.amtso.org/members/ . Therefore they are not bound by the AV lab testing standards established and jointly agreed to by all AMTSO members.

One is advised to stick with an AMTSO member AV lab when determining the effectiveness of their security solution against banking malware; and all malware for that matter. One such lab is MRG which has performed quarterly banking malware testing for some time: https://www.mrg-effitas.com/wp-content/uploads/2019/01/MRG-Effitas-2018Q4-Online-Banking_level1.pdf. In this report, refer to the section titled "Samples used in the In-the-Wild real financial malware test" as the proper way to conduct a banking malware test.

 

Well now I personally think AMTSO is a scam. This is like having a committee made up of banks executives dictate how banks will be regulated. That was tried before and they ended up creating the biggest mortgage fraud scandal ever. Or a committee of insurance companies who set the standards for how insurance companies set rates. Or letting the pharmaceutical company oversee their own testing. Or a consortium of car makers who dictate how cars will be tested for safety features.

Nope. Not for me. I think any organization who wants to evaluate anti-malware solutions, whether it be AV-Labs or NSS-Labs or whoever should be allowed to test these programs any way they want, AS LONG AS they test every product the same way, and testing is not designed to favor one program over another.

 

I figured you would "chime in" on that posting. And I am not going to debate you on the issue since I know where that will lead. So as the saying goes, "Each to their own opinion."

 

Anyway all testing labs are AVs marketing proxies, their methodologies are somehow oriented and favors AVs, at best they are just estimations and extrapolations, can be close to reality, but will never be same.

 

VoodooShield finally has a current review! The results are good, just don't take them as any more than an approximation.

This is what they, the chemical industry and Big Oil do in the U.S. "Police" themselves! Self-regulation is the order of the day.

 

You can contact with me at kontakt(at)avlab.pl to get some PoC.

No. It was not necessary for SpyShelter. Tested on default settings. Read a green row.

It was a editor mistake.

All malicious files was delivered to PCs via HTTP protocol by Chrome browser. Malware was written in Python scripts and compiled to EXE file. It was a naturally situation when users download trojans via browser.

For something completely different - in the antivirus test, the antivirus protection is tested. The antivirus. Not user reaction to the pop-up. Some of the comments are wired and accuse that were not real malware used. What does mean a real malware? In-the-wild banking Trojans are not suitable for use in the test because you cannot control C&C server. You do not have access to the malware code. You can not modify anything. You must write your own malware to test the antivirus reaction, for example, data theft. That is the best and easiest method. Moreover it is not easy to capture working properly in-the-wild trojan. The trojans usually does not work. And if it downloads payload, it may not communicate with C&C. Is this a real situation? Maybe MRG Effitas has access to the RTTL malware database as an AMTSO member. We do not have.

Regarding AMTSO. We are not a member, because we have a few reasons. For example (as of today), we do not want to pay for membership a few thousand$ per year and have their a badge on our website. We are a very small company. It is too expensive. We only carry out 2-3 max tests a year and there is no financial reason for us. We specialized in AV testing but it is not our primary fileld of earning. However we use the methods described at amtso webpage. Mostly. Please note that not each producer belongs to AMTSO. For different reasons.

 

As far as I am concerned, this states why malware needs to be tested as true as possible to actual delivery method. As I stated previously in this thread, most major AV products are designed to prevent malware from being downloaded in the first place.

The problem with post-execution malware detection is simply the malware might have modified the system prior to it being detected. A backdoor is a classic example. It can reside in a dormant state for days, months, and in a few recorded instances, years.

In others words, you took existing malware and completely modified is actually delivery method. Yes, they are a few malware that have or could use PyBuilder, PyInstaller, etc. to do so. But they are still a rare occurrence and fall into the targeted advanced persistent category: https://unit42.paloaltonetworks.com...omware-coinmining-worm-targets-linux-windows/ .

 

It does not matter for what AVs are designed. Have been programmed for secure the system. Remind how the ransomware paralyzed half of Europe in 2017 through the SMB vul. The SMB protocol is not popular at all, and yet it was used to attack.

Wrong. It was our scripts. Once again, what does it matter to shows the weak points of AVs? If you would like to hack your neighbor, what will you do? Will you use malware captured in-the-wild? No, because you do not have control the code, so the malware will not work the way you want to. If you want to check anti-keylogging, you cannot use in-the-wild malware, because you can not check if the malware has sent correctly stolen data to hacker's server. You have no control over the entire phase of the attack.

 

Another point about this test that I am surprised no one has commented on.

Since the test malware was "packaged" Python executables, Win 10 native SmartScreen should have blocked/warned on the execution of them since it is assumed they all would be flagged as "unknown." This implies:

1. Win 10 native SmartScreen was disabled or,
2. None of the executables had the "mark-of-the-web" associated with them.

As far as no. 2, this because is the executables weren't actually downloaded from the Internet but rather from a server within the test network.

Also if for some reason SmartScreen didn't detect the Python executables, WD's "block-at-first-sight" and subsequent cloud scanning is also dependent upon "mark-of-the-web" tagging.

In any case, this disqualifies any of the Windows Defender test results since Win 10 native SmartScreen is an integral part of its security protection. Additionally since native SmartScreen is enabled by default in Win 10, it should have been so for all AV product tests.

 

OK, so you downloaded EXE files via the browser and then executed them manually? To clarify, I'm not criticizing the way that it was tested, but I'm just trying to understand how the testing was done. But from what I understand, you simulated real banking trojans? Nothing wrong with that, because I forgot that you can't modify real life malware to perform certain things.

But can you explain how SS reacted to all tests? Did it present you alerts, and did it also auto-block certain tests? For example, how did it stop the RAM Scraping Attack?

I'm not following. Bad guys will simply try to get malware running on the system. When people are being tricked, certain anti-malware measures won't help. I sometimes run into sites that are not blocked by web-filters, and they try to make you download fake video players or fake AV's. Let's say that I'm a noob, I will probably ignore SmartScreen and AV's often won't spot this kind of fresh malware, at least not via signature. So then, behavior blocking is your last line of defense, and that's exactly what is being tested in this particular test.

 

You said,

I am saying we cannot assume we know how the bad guys will react either. If they were that predictable, it would be much easier to block their next move.

I don't see this being a problem with most noobs - at least not the adult ones. Even regular news shows regularly warn to not be "click happy". It is not uncommon for me to get calls from newbie clients saying, "I got this warning but was afraid to click anything so I'm calling you." The problem is more with kids who, of course, think they are invincible and nothing can happen to them, or sadly, the elderly (those older than me and I'm 67) who still think most people are honest and friendly. And sadly still, those are the ones I wish would call me, but often don't because they don't want to bother me.

 

If one wants to test Python packaged malware, at least use the "the real stuff":

https://www.fortinet.com/blog/threa...s-nsa-exploit-to-propagate-monero--xmr--.html
https://securityaffairs.co/wordpress/76146/malware/pylocky-ransomware-machine-learning.html

 

You didn't read the report for WD. SmartScreen was enabled.

 

To get "some semblance of reality" to this test procedure, do this.

Since it can be assumed your previous Python .exe's are now being flagged on VirusTotal, create a new test Python .exe encapsulating a malware sample not previously used. Also create a zipped version of the .exe. Upload to a file sharing service both the .exe and the zipped folder. Post the link to where the files can be downloaded from.

I will then proceed to download the file/folder and determine how native SmartScreen on Win 10 1809 performs. Since I am also using IE11 w/SmartScreen enabled, I strongly suspect I will get an immediate unknown file warning on the attempted download.

 

I already explained that to me personally it doesn't matter if SmartScreen was enabled or not. You have to look at what the goal of this test was. If SmartScreen blocks 100% of all malware, then how are you going to test AV's? And in real life, people may choose to ignore SmartScreen.

 

That is certainly true. However the fault here is not the security mechanism but the individual.

Likewise using SpyShelter for an example, you're going to get plenty of alerts from it. If you click on "Allow" each time an alert presents, you are likewise going to get infected. So using your stated logic, I guess we have to blame SpyShelter for that.

 

I see that the description of the Panda Dome Advanced results has been corrected and now correlates correctly with its performance.

 

That's why I asked how SS blocked these attacks. AFAIK, it's the only tool that will show alerts and let the user make the decision. Other tools, will block this stuff automatically, so not a completely fair test. Same goes for SmartScreen, it will give an option to ignore the warning. In my opinion, these tests should be about automatic blocking of malware.

 

"Now you're talking" my friend.

 

Since it appears this "Python malware testing business" is getting out of control, the proper way to use Python for testing malware is to develop the malware using Python scripts. Then bundle those plus the Python engine components into an .exe. In this regard, I am posting links to an excellent three part primer series on how to do so.

First, a brief summary of the articles. Part 1 shows some code for a couple basic keyloggers. Part 2 continues with more advanced keyloggers; a few I wasn't aware of. Finally, Part 3 gets into the real "nitty gritty" stuff such as cookie and credential stealing.

A "must read" is the first section of the Part 1 article. The basic keylogger code shown was bundled into a Python .exe and submitted to VT. No one detected it as shown via the VT scan link posted in the article. Out of curiosity and noting that scan is close to two months old, I initiated a rescan. Guess what? Only three vendors presently detect it; Kaspersky, Tencent, an ZoneAlarm. The conclusion drawn from this is if your relying on your security solution to protect you against keyloggers and its banking protection is not scrambling keystrokes which is the only way to protect against all keyloggers, you have a problem. Additionally, security solutions employing key scrambling would have falsely failed these lab/individually tester created simulated tests since the keylogger .exe would not have been detected. Finally, the keylogger code shown in these articles would be a great way to evaluate SpyShelter detection capability since after all, it is at its core an anti-keylogger.

https://0x00sec.org/t/malware-writing-python-malware-part-1/11700
https://0x00sec.org/t/malware-writi...gging-with-ctypes-and-setwindowshookexa/11858
https://0x00sec.org/t/malware-writing-python-malware-part-3-stealing-credentials-and-cookies/12099

 

It will block the keylogger described in part 2, but SS doesn't block credential/password stealers. I have requested such a feature, but developers don't seem to think it's important, or perhaps it would require to much time to develop it.

 

Test it using a Python .exe with its engine and script bundled via PyInstaller.

 

Since this test is about creating stealthy malware, let's do it right. Let's create some malware that will bypass not only AV's but all anti-execs.

Python is not the only thing that can be encapsulated into a .exe, so can the command processor. Before I get into this, note that this is a remote PowerShell execution attack. To perform one, two elements are necessary on the targeted device; a reverse shell and a PowerShell script. You will have to read the "Part 1" of this article series on how to set up your attack server and build the encrypted, packed, obfuscated, etc. PowerShell script. This technique will make the PowerShell script download totally invisible on the targeted device. And again folks, PowerShell is running remotely and any mitigation you have to prevent its execution locally is not applicable.

Additionally, the downloaded .txt file executable is not detectable with the "show hidden files" option enabled. In fact, the only to detect the .txt file is manually and is given at the end of the below linked "Part 2" article.

Now for the details:

https://null-byte.wonderhowto.com/h...le-payload-part-2-concealing-payload-0185060/

-EDIT- Warning: For those wishing to experiment, the author warns in a separate web site posting dedicated to BAT2EXE, that it should be considered "dangerous" if downloaded from the creators website. It should always be run in a secure sandbox or VM prior to use; or recommended, always. Additionally, many AV's will flag the download regardless of malicious status. The article author has a "sanitized" and older version of it on his web site that can be downloaded.

-EDIT- It gets better. The developer of BAT2EXE also has a VBS2EXE ver. that will encapsulate wscript plus the .vbs into a .exe. His web site will also do so avoiding an install allowing for a direct download of the encapsulated .exe. Additionally, online you bundle a .js script instead. I would avoid anything online from the web site. Additionally, you can't Trojanize the payload since you can't edit the scripts.

 

Thanks for this, it's very insightful

On a side note, after reading through the thread one thing I wanted to comment on was in regards to smartscreen. I thought it was possible to not allow bypassing smartscreen by changing the setting to block from warn. I think warn is the default setting, but I have mine set to block and I know there wasn't an option to allow the app through without changing the setting of smartscreen itself. I also am running a SUA, so I dont know if using block and SUA together make it so that you cannot bypass the prompt?

 

As a general rule, auto block is always more secure than anything thing requiring prompt decision making. The user can make a wrong decision or the prompt alert itself could be hijacked somewhere in transit to the desktop. As far as Win 10 1809 goes, the default setting for Microsoft Defender SmartScreen for non-Store apps is block. For Store apps, I believe it is prompt.

The problem with both SmartScreen and Windows Defender is they are dependent upon the "Mark-of-the-Web." Here's an article on that: https://textslashplain.com/2016/04/04/downloads-and-the-mark-of-the-web/ . As far as issues with it, refer to the "What Could Go Wrong?" section. Also, some issues noted have been rectified. For example, archive extracts now will contain the the "Mark-of-the-Web" if the archive was downloaded.