MRG Effitas Online Banking Browser Security Assessment Project Q3 2013 – Q1 2014

Discussion in 'other anti-malware software' started by malexous, Jun 18, 2014.

Thread Status:
Not open for further replies.
  1. malexous

    malexous Registered Member

    Joined:
    Jun 18, 2010
    Posts:
    830
    Location:
    Ireland
  2. ky331

    ky331 Registered Member

    Joined:
    Jun 25, 2008
    Posts:
    158
    Dismal :-(
     
  3. Fabian Wosar

    Fabian Wosar Developer

    Joined:
    Aug 26, 2010
    Posts:
    838
    Location:
    Germany
    I stopped reading after this:
    So the PDF can be summed up to: "If you are infected, you are screwed." I guess most people wouldn't have to read a 12 page PDF to know that.
     
  4. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    Wow, so many failures. This shows how far ahead the malware criminals really are in front security vendors.

    I agree with Fabian though.

    Also:

    So they actually went way outside a real world scenario in order to install the browser extension. Who, besides a developer, would even do this?
     
  5. Fabian Wosar

    Fabian Wosar Developer

    Joined:
    Aug 26, 2010
    Posts:
    838
    Location:
    Germany
    I wouldn't say that to be honest. For example: A lot of security vendors do watch for hidden browser extension installation and block it. I know we do for example. Of course, if you assume a pre-infected system, it's not much of a protection test. We have entered the domain of damage control at that point.
     
  6. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    This is a test in which a highly unlikely scenario would take place. I think we have to see this test as it is. The anti-malware applications are installed post-infection and the test shows what security applications prevent malware from stealing. So you're right about damage control. This test should probably be seen as how good damage control each security application does with each infection. Not many did a very good job with the damage control... but then again, most applications would probably prevent infection from happening in the first place.
     
  7. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    6,555
    Location:
    New York City
    The report raises concerns for Webroot where identity protection is a key component of their defense strategy.
     
  8. shadek

    shadek Registered Member

    Joined:
    Feb 26, 2008
    Posts:
    2,538
    Location:
    Sweden
    Indeed.
     
  9. trott3r

    trott3r Registered Member

    Joined:
    Jan 21, 2010
    Posts:
    1,283
    Location:
    UK
    Quarri armoured web browser?
    Has anyone used this and can recommend it over bitdefender safe pay or is it as sluggish?
     
  10. FOXP2

    FOXP2 Guest

    Who? Some one who finds the developer mode to be exploitable?
     
  11. emmjay

    emmjay Registered Member

    Joined:
    Jan 26, 2010
    Posts:
    1,540
    Location:
    Triassic
    I would like to know this too. Anyoneo_O?
     
  12. SLE

    SLE Registered Member

    Joined:
    Jun 30, 2011
    Posts:
    361
    Fullack.
    Funny thing is that they introduce their report with the mention of relevance for real world testing.

    And what to they do?
    - simulators. How can those in-house tools claim to be real world?
    - install on clean system before security software
    ...
    I recommend everybody to read this report carefully and think about it. For every test case they describe there are enough arguments that it has absolutely nothing to do with real world. Thats why: no relevance, no need for security products to detect anything of all that stuff
     
  13. BoerenkoolMetWorst

    BoerenkoolMetWorst Registered Member

    Joined:
    Dec 22, 2009
    Posts:
    4,868
    Location:
    Outer space
    A pity avast wasn't tested with it's Safezone, still interesting results.

    A lot of security software has some special banking mode/browser these days, which should protect you, in case you're infected.
     
  14. ArchiveX

    ArchiveX Registered Member

    Joined:
    Apr 7, 2014
    Posts:
    1,501
    Location:
    .
    Wontok SafeCentral? :eek:
     
  15. FleischmannTV

    FleischmannTV Registered Member

    Joined:
    Apr 7, 2013
    Posts:
    1,094
    Location:
    Germany
    They seem to have adjusted their testing methods in order to make the newly introduced participants shine out. I wonder what Miladinov's liege Kaspersky thinks of this insolence.
     
  16. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,899
    Location:
    localhost
    I guess they starting to be out of ideas/exploits to make their testing attractive to the readers (showing big losers / big winners).
    Therefore they moved to another scenario (pre-infected system) to keep their breaking news style...

    They also lost Trusteer that was always giving good results... probably IBM is not anymore interested in their tests ;)
     
  17. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,592
    Location:
    U.S.A.
    Don't really understand the assumptions behind this kind of testing. If they are that the majority of PC uses are ignorant about the security status of their PC and the majority still are, I guess this is a valid test. And as shown, a stand alone "armored" browser is the only thing that will protect you if your infected. On the other hand, there are "issues" with these stand alone browsers. I recently tested Quari's product and it is not compatible with EMET 4 and above as verified by Quari after I informed them of the issue. Also, Trusteer is not compatible with EMET.

    There have been lengthy discussions about Bitdefender's Safe Pay on Wilder's with posts stating that the stand-alone browser Safe Pay uses is based on an older version of Chrome that has numerous recorded exploits.

    My contention has been for years that no financial transaction is or every will be safe using the Internet. What is required is a separate private network dedicated to financial activity.
     
  18. Sveta MRG

    Sveta MRG Registered Member

    Joined:
    Aug 16, 2009
    Posts:
    209
    Hi Guys,


    First of all we would like to thank you for the comments regarding our tests. Without negative feedback we won’t know if we are at a good direction or not. We are also happy to have this discussion on the deep technical level.


    Allow us to react on every comments made on the forum:


    Comment 1. (Fabian)

    "The simulators are installed onto clean systems without protection, thus simulating a pre-infected state.

    So the PDF can be summed up to: "If you are infected, you are screwed." I guess most people wouldn't have to read a 12 page PDF to know that."


    Our reply:

    Although technical people might be aware of this fact, but average users don't know this - as this is clearly not communicated to the users. We believe both Emsisoft and MRG have the same goal - which is less malware infection in the world, but this could not be achieved when average users have a false sense of security.


    Please be aware that our simulators in this test don't use any rootkit/hiding technology at all, and they run in the user space.

    For example, taking the Firefox extension as a good example, I don't see why any of the AVs who failed the test would block the installation of this extension.

    -----------------------------------------------------------------------

    Comment 2. (shadek)

    "Because of the way Chrome allows the installation of browser extensions outside of the official extension store, the installation of the browser extension needs “developer mode” to be enabled in the browser. The Chrome browser warns the user about developer mode, every time the browser is started. This means that in a real-world scenario, it is less likely that users will install a malicious Chrome browser extension, than is the case for other browsers."


    Our reply:

    Here are two articles to point out why Chrome extensions are still a threat:

    hxxp://ddanchev.blogspot.hu/2014/01/dissecting-ongoing-febiposcarfekab.html

    hxxp://www.pcworld.com/article/2089580/spammers-buy-chrome-extensions-and-turn-them-into-adware.html


    On the other hand we saw malwares which can click on GUI automatically (either AutoIt based, or self-developed), so it is not that hard to write a malware which will install itself into Chrome by clicking on the GUI.

    -----------------------------------------------------------------------

    Comment 3. (SLE)

    " Funny thing is that they introduce their report with the mention of relevance for real world testing.


    And what to they do?

    - simulators. How can those in-house tools claim to be real world?

    - install on clean system before security software ...

    I recommend everybody to read this report carefully and think about it. For every test case they describe there are enough arguments that it has absolutely nothing to do with real world. That's why: no relevance, no need for security products to detect anything of all that stuff"


    Our reply:

    We understand that a lot of AV companies disagree with the use of simulators, and we also know that testing with simulators is far from perfect. On the other hand, testing with known malware is a bit of outdated (and won't show realistic scenario as well), as of nowadays most of the malware is 0day malware. And our simulators are just another 0day malware. Please beware that most of our simulators have been created by reverse engineering known malware, and we implemented the same attack (e.g. how it injects itself into the browser). Thus, it acts as a new, unknown malware. But it has the very same functionality as a malware, and works like a malware.


    Comment 4. (fax)

    “They also lost Trusteer that was always giving good results... probably IBM is not anymore interested in their tests ;)


    Our reply:

    We have not lost Trusteer, they remain a core client we have a very close relationship with them and continue to conduct work with them to this day. Also specifically addressing your comments about IBM, we would like to point out that we are in fact an IBM MSP and we are working very closely with them and in fact we are employing their cloud computing technology in our new tests. If you look in our blog, you will see a video IBM made about us and our next generation of tests.

    -----------------------------------------------------------------------


    General comment:

    The focus of the whole test is about "Safe browsers". Safe browsers are marketed in a way that no matter if you use AV or not, and no matter if your system is infected or not, use our "Safe browser" during online banking/shopping, and you will be safe. If you look at our test with this in your mind, if a customer sees a traditional AV/Internet security suite, which fails at some of our tests, the customer can see if he needs additional protection or not. If the customer checks the solutions which have some kind of "Safe browser", the customer can decide whether it is worth to use it, or is it just another snake oil.


    We do believe that some Safe browsers are worth using in a defense-in-depth approach as a last line of defense. These safe browsers don't protect against malware X, which if modified in a bit, won't be detected anymore, but they do prevent suspicious activities, like code injected into a browser. If you look at the numbers, there is an infinite number malware (hypothetically) which can inject itself into the browser, but there are a finite (and very small number) of ways, how the malware can inject itself into the browser. Creating a new variant of the malware costs nothing for malware developers, but finding new ways to inject into the browser, that takes a lot of resources. And this is what our (MRG and AV vendors) main goal is, to increase the effort of malware developers, so in the future it won't be worth to develop malware.


    MRG Effitas has always tried to bring the testing closer to the real world, and there are many scenarios out there. When you read about APT’s for example, they were discovered only when the systems were already compromised and all we saw later was just damage control. We have to face the reality; the bad guys are winning at the moment because we the testers are attacked if we push things too far, but when a big attack occurs (that could have been avoided if such attacks were simulated in the labs and vendors were alerted about this in time…) then vendors are attacked for not protecting their users.

    At the moment we have the following situation:

    Bad guys -> Vendors -> Testing Labs -> Users

    What if this changed in the future and Testing Labs were given more autonomy….

    Bad guys -> Testing Labs -> Vendors -> Users

    This could be viewed as just another quality control process that ensures greater level of security for the end user.

    We have quite a few tests that will be published soon; you will see that we cover the test from lots of different angles, and those tests will map the real life threats better than this test. But we would like to highlight that if an AV/Internet Security Suite/Safe browser failed at one of our simulators, there is a very strong chance that those will fail in real life as well.

    There is no perfect test! We do our best to evolve our tests every day, so customers can make an informed decision. This is reflected by our use of the IBM cloud computing technology described in the video on our blog. This technology is being used in our current Banking tests, the first of which will be published at the end of this month.


    Regards,
    Sveta
     
  19. emmjay

    emmjay Registered Member

    Joined:
    Jan 26, 2010
    Posts:
    1,540
    Location:
    Triassic
    Sveta, Thank you for sharing your testing results and participating here. It is refreshing to see someone who can deal with negative responses. The strength of the community is in its diverse opinions.
     
  20. This "test" only applies to damage control after shoot in the foot user error's. There is no software remedy against user stupidity, except when you prevent them making mistakes, e.g.through rights restriction.

    Untitled.png
     
  21. FreddyFreeloader

    FreddyFreeloader Registered Member

    Joined:
    Jul 23, 2013
    Posts:
    527
    Location:
    Tejas
    Can you please test HitmanPro.Alert?
     
  22. Frank the Perv

    Frank the Perv Banned

    Joined:
    Dec 16, 2005
    Posts:
    882
    Location:
    Virginia, USA
    Well.... I think that was a pretty good and fair response by Sveta MRG.

    Yes, there will always be issues with testing methodologies.

    But the tests with their inherent limitations still have meaning... it is another data point to evaluate a product.

    It is popular sport here to bash the tests... and we all stand-by for the bashers to do better testing.


    That is all.

    10-4, Over and Out.
     
  23. fax

    fax Registered Member

    Joined:
    May 30, 2005
    Posts:
    3,899
    Location:
    localhost
    Interesting.... but even more weird as you always included them in banking tests. A conflict of interest and preferred not to publish the bad results? I guess we will never know as you are bound to confidentiality.
     
  24. phalanaxus

    phalanaxus Registered Member

    Joined:
    Jan 19, 2011
    Posts:
    509
    Hi Sveta,

    MRG Answer 1: Although technical people might be aware of this fact, but average users don't know this ....
    Comment 1: Believe me most of the average users never heard of MRG or any similar testing organization, and unless this changes drastically the target audience of these tests are "technical people".

    MRG Answer 2:On the other hand we saw malwares which can click on GUI automatically (either AutoIt based, or self-developed), so it is not that hard to write a malware which will install itself into Chrome by clicking on the GUI.
    Comment 2: Most of those random packers used to compile will be blocked in default settings, especially popular ones like AutoIt. And other points made before still stand.

    MRG Answer 3:
    Comment 3: I will pass this one as I see some value to simulators.

    MRG Answer 4:
    Comment 4:I don't know anything about this so no comment.
     
  25. Frank the Perv

    Frank the Perv Banned

    Joined:
    Dec 16, 2005
    Posts:
    882
    Location:
    Virginia, USA
    I know nothing about phalanaxus knowing nothing about that.

    So I have no comment on that.

    That is all.
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.