AVLab - "Test of software for online banking protection"

Discussion in 'other anti-malware software' started by ichito, Mar 19, 2019.

  1. ichito

    ichito Registered Member

    Joined:
    Jan 14, 2011
    Posts:
    1,851
    Location:
    Poland - Cracow
    https://avlab.pl/test-software-online-banking-protection
    Full report
    https://avlab.pl/PDF_avlab/AVLab-Test-of-software-for-online-banking-protection.pdf
     
    Last edited by a moderator: Mar 19, 2019
  2. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    Interesting, but it doesn't explain how it was tested? Was this tested with real life malware or simulator? And I noticed that SpyShelter was the only who passed all tests. But here is the problem, SpyShelter doesn't auto-block most of the stuff that was tested. So I would love to know which alerts were presented. And all of the other solutions do try to auto-block these attacks, with behavior blocker or safe browser.
     
  3. Azure Phoenix

    Azure Phoenix Registered Member

    Joined:
    Nov 22, 2014
    Posts:
    999
    You didn't notice Norton?
     
  4. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    Kaspersky and possibly Sophos Intercept(rebagged HMP-A + Sophos IS), can't remember also received perfect scores. -EDIT- Also Comodo and ZoneAlarm passed all the tests w/"banking mode" settings.

    Appears to me this test combines elements of MRG banking test plus things SpyShelter's own test tool detects such as clipboard scrapping, etc.. So I would except it to do very well on the test.
     
    Last edited: Mar 20, 2019
  5. bjm_

    bjm_ Registered Member

    Joined:
    May 22, 2009
    Posts:
    3,313
    Location:
    .
    IDK
     
  6. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    In other words, they "cranked up" SpyShelter's HIPS to full alert mode. Which brings up the applicability of actual use. Anyone who has used SpyShelter in this mode will in short order reconfigure it to something one can live with on a daily basis.
     
    Last edited: Mar 20, 2019
  7. Bill K

    Bill K Registered Member

    Joined:
    Sep 19, 2018
    Posts:
    45
    Location:
    Naperville IL
    I have to question the validity of the reported results when the description of the Panda Dome Advanced section refers to Kaspersky and appears to be a copy of their results. Maybe an editing error but if they can't get that right why should any credibility be given to the rest of the report? :doubt:
     
  8. Azure Phoenix

    Azure Phoenix Registered Member

    Joined:
    Nov 22, 2014
    Posts:
    999
    Spyshelter got perfect score both in default and modified settings.
     
  9. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    The last time I tested SpyShelter Anti-Keylogger at default settings, there really was no reason to configure further. The alerts were for my purposes unbearable.

    -EDIT- The also brings up the "usability" factor in this test. It appears that wasn't addressed in this test in regards to "hardened" settings. The average PC user is not qualified to be answering a barrage of alerts from his security product.
     
    Last edited: Mar 21, 2019
  10. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    The one test result I am having an issue with is Norton's. It had a perfect score in both test modes. I really can't believe it has improved that much.

    Also note that in many of the products, the firewall was set to Interactive mode.
     
  11. guest

    guest Guest

    Standalone HIPS are not made Average Joe, it is niche product for security geeks.
    All "usable" HIPS are coupled with cloud lookup, whitelisting, and whatever they can to reduce prompts which obviously reduce its full power.

    I use Spyshelter in "Ask user mode" so I answer dozen of prompts when running softs for the first time or when updating them.
    For some LOLbins, I set SpS to never remember my choice.

    HIPS are made for full control, by not doing so, you defeat its purpose.
     
  12. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    My bad, I guess I had a bad day yesterday, they also explained how it was tested LOL. So looks like Norton, Panda, ZoneAlarm, SpyShelter, Norton, Kaspersky, and Sophos Intercept X all passed when banking protection was enabled. And it was tested with a simulator, I'd rather see them tested against real banking trojans.

    The thing is, all other tools try to block these attack techniques automatically. I'm guessing that SS can only block test 3, 5, 7 and 8 automatically with the keystroke encryption and anti-SSL sniffing feature. For all others it will present alerts. I'm surprised it could block all DLL injection methods. Would like to see more proof for that, I wonder which alerts were presented.
     
  13. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    Based on this:
    the only simulator tool used was the Bettercap tool mentioned for MITM test.

    His reference to use Python scripts leads me to believe this is how he deployed the malware used in the test. In other words, it was real malware samples although not deployed as they were originally in the wild. This could possibly adversely affect the Next Gen products whose ML algorithms are dependent on that behavior for a probability exceeding threshold positive value. Although, it appears no Next Gens were included in this test.
     
  14. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    I'm sorry, but I can't visualize it. What would make sense to me is that they simply run a simulator or real banking trojan on a system and see how tools react. Of course, AV's may be able to detect it as malicious, but I believe the goal was to test the pro-active protection from all of these tools.

    So in this scenario, malware is already running in memory and trying to perform all of those tests. But now that I think of it, how was Win Defender not able to block it? Probably because the attack was simulated in a different way, would like to have some more info about this.
     
  15. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    The use of Python scripts in malware testing is nothing "new or suspicious." The PC Security Channel: https://www.thepcsecuritychannel.com/ , has used then for some time in its security product testing. I also suspect they are used by the major AV labs. Simply put, they are a way of automating the test procedure. However, they are not appropriate for all malware testing such as ransomware.
     
  16. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    I never said they were, but I don't understand exactly how they are being used. So in order to simulate such an attack, do you need to click on some script file? Like I said, to me it would make more sense to simulate a real life attack.

    So let's say you're browsing, and you get to see on some site that your PC is infected, so you will now download a fake AV. You will ignore SmartScreen and UAC, because you really think you're infected. This fake AV is zero day, so your AV won't detect it. So now the question, which tool can block this fake AV from stealing money and data? Some will use the safe browser, and others will use behavior blocking.
     
  17. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    Here's how the PC Security Channel explains its Python script usage:
    Does this in any way mirror how the malware payload was actually delivered in the real world? Of course it doesn't. Note that in the realtime tests by AV-C and AV-Test, they actually download the malware from known malicious URLs which is a more realistic test. However, it excludes other malware delivery methods such as via an e-mail client and the like.

    Bottom line - automated testing bypasses AV products front-end protections such as web filtering which employ among other things, IP blacklisting and the like. In other words, tests like this only evaluate malware protection after the malware is resident on the disk. The whole emphasis of modern AV protection is to prevent the download from occurring in the first place.
     
    Last edited: Mar 23, 2019
  18. imdb

    imdb Registered Member

    Joined:
    Nov 2, 2011
    Posts:
    1,558
    so true. :thumb:
    and the same goes for sw like sap and vs, whereas ag and nvt erp provide rock solid protection for the geeks who know what they're doing.
     
  19. Bill_Bright

    Bill_Bright Registered Member

    Joined:
    Jun 29, 2007
    Posts:
    3,532
    Location:
    Nebraska, USA
    Another simulated test that is supposed to prove something that really proves nothing. :(

    Odd that McAfee didn't participate. Makes you wonder why? Don't see Malwarebytes, Zemana, Hitman Pro and a few others either.

    Never heard of mks_vir even though it goes back 32 years!
     
  20. Surt

    Surt Registered Member

    Joined:
    Jan 23, 2019
    Posts:
    77
    Location:
    USA
    Spy Shelter... not bad.

    Anyone think the results would be the same for Premium given its Internet Security feature versus Firewall with its additional two-way rules/trusts Firewall Protection and anti-executable Application Execution Control?

    For recall:
    https://www.spyshelter.com/internet-security/ (module details)
    https://www.spyshelter.com/download-spyshelter/ (comparison chart)

    My only hands-on was a ways back with the now defunct Free version.
     
  21. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    Well, if you're not able to download malware because of web filtering, then how the heck will you be able to test behavior blocking capabilities? That's why you should simply assume that people are somehow being tricked into running malware. So you should run malware and see how AV's react. If they can't stop it via signature/heuristics, they should be able to block it via behavior blocking or safe banking browser.
     
  22. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    7,721
    Location:
    U.S.A.
    Ah ….., "the chicken or the egg" scenario. But most appropriately, the applicability of the truism "an once of prevention, is worth a pound of cure."

    Simply put, one doesn't have to worry about process execution risks if there is no process to execute in the first place . The purpose of AV Labs is not to exclusively evaluate post execution blocking capability, but all malware blocking capability.
     
  23. Bill_Bright

    Bill_Bright Registered Member

    Joined:
    Jun 29, 2007
    Posts:
    3,532
    Location:
    Nebraska, USA
    I think it is more to make money and to provide advertising fodder for those products that score well. :(

    To be sure, I am not a fan of big pharma but how do we know if a particular drug is good enough? The best and most conclusive way is to conduct "real-world" studies with 10s of 1000s (or more) patients out in the real world in real-world scenarios, monitoring them over extended periods of time to see how well the drug does.

    Because such a study adapted to computers is impossible (not to mention prohibitively expensive for such test laboratories), these simulated tests in laboratory environments are just that, simulated and not in the real world. It is not the same. These labs remind me of those who try to guess what should go into annual flu shots. Did you know the flu shot is considered a good match if ~50% effective! :rolleyes:

    Of course they are guessing for the next flu season so the analogy is hardly perfect. But the antimalware testing labs also throw in 1000s and 1000s of different malware at these products even though 1000s have not been seen in the wild for many years, or they only affect XP or other long superseded and obsolete operating systems. Then they rate the program on how well they do against that malware too. Is that realistic or truly indicative of actual threat seen today in the real world? No.

    I am not blaming the labs, they don't have the resources to monitor, in real time, on a massive global scale, exactly what malicious code is being a threat today. That would take a company that has a massive global reach, the deep pockets to monitor and continually analyze the current real-world threat, and the initiative to rid the world of malware to do that - a company like Microsoft, the only company in that list of antimalware makers who would love to see malware go away - and is actually doing that sort of research as we speak!

    Note all the other companies need malware to thrive for them to stay in business. If malware goes away, so do those other companies. This is exactly why Microsoft does NOT code their program to score well on simulated tests. They don't need the advertising fodder to generate revenue, or to entice consumers to choose their product over the competition.

    I am not saying Windows Defender is the greatest solution out there. It clearly is not. But is it good enough? Of course it is! At least with Windows 10. If not, there would be millions and millions of infected W10 systems out there, with more and more getting infected every day. And that is not happening. Yet many sure want us to believe it is. Makes you wonder, why?

    I mean look at where the link in this thread's opening post takes you. What is the one and only product displayed in that introduction? But NOOOOO! That is not a biased report. :rolleyes: Yeah right.
     
  24. Rasheed187

    Rasheed187 Registered Member

    Joined:
    Jul 10, 2004
    Posts:
    12,126
    Location:
    The Netherlands
    Exactly, but in a test you can never completely simulate real life. Because you can never know how people will react. In the scenario that I described, people will simply ignore warnings from SmartScreen, Safe Browsing and perhaps UAC. You can always be tricked into running malware, think of tech support scams. And don't forget about the so called "supply chain" attacks on CCleaner and now apparently also on ASUS Live Update.

    https://www.zdnet.com/article/suppl...s-through-hijacked-asus-live-update-software/
     
  25. Bill_Bright

    Bill_Bright Registered Member

    Joined:
    Jun 29, 2007
    Posts:
    3,532
    Location:
    Nebraska, USA
    Those people include the bad guys.
     
Loading...
Similar Threads
  1. Peter2150
    Replies:
    41
    Views:
    3,691
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.