How Well Does Your AV Solution Detect Archived Malware?

Discussion in 'other anti-virus software' started by itman, May 25, 2018.

  1. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    Whereas the AMTSO Desktop Test site performs tests against 6 common archive formats, I found a web site that performs 18 different tests using the EICAR test malware as the payload. It includes formats such as .cab files, etc..:

    https://www.fortinet.com/offers/test-your-system-malware-detection-capabilities.html

    Eset scored 17/18 only failing the password protected archive test. It actually did detect it as suspicious and submitted it to the cloud for further analysis. Fortinet rated Eset as excellent in archive protection.

    Curious to see how Windows Defender performs.
     
  2. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    I tested Kaspersky and got mixed results. On AMTSO site all archived samples are blocked, on Fortinet site only plaintext sample was blocked. So it seems that it doesn't scan archived files in transit.
    If I enable scanning of archives in File Anti-virus component, all files except password protected archive, were detected by this component.
     
  3. hamlet

    hamlet Registered Member

    Joined:
    May 10, 2005
    Posts:
    229
    I guess my question about this test is "can malware embedded in an archived file infect or otherwise harm a computer while it is still archived?" If the answer is no, then I am not sure that I see the point of the test. Wouldn't it be a better test to unzip the archived files and see if the malware is detected at that point? I am not asking to be argumentative. I really don't know.
     
  4. xxJackxx

    xxJackxx Registered Member

    Joined:
    Oct 23, 2008
    Posts:
    8,616
    Location:
    USA
    Mostly my thoughts as well. My response to the question "How Well Does Your AV Solution Detect Archived Malware?" was going to be "I don't care". It's like being afraid the chicken nuggets in your freezer are going to peck at your toes.
     
  5. Buddel

    Buddel Registered Member

    Joined:
    Apr 28, 2015
    Posts:
    1,919
    ... and my thoughts as well. I really don't care. As hamlet said, detecting malware the moment it's unzipped is what really matters.
     
  6. roger_m

    roger_m Registered Member

    Joined:
    Jan 25, 2009
    Posts:
    8,627
    No it can't. It will only infect the computer, if you manually open an infected file. Even if you extract the archive, nothing will happen, unless you open one of the infected files, you extracted.
     
  7. marcuskng

    marcuskng AV Expert

    Joined:
    Feb 19, 2010
    Posts:
    74
    Apart from the previous criticism of this test, there seems to be something technically wrong with it. Sometimes it turns red even though the files were properly blocked.
     
  8. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    This article will explain most of your questions:

    BEST PRACTICES FOR DETECTING THREATS IN COMPRESSED FILES
    http://www.infosecurityeurope.com/__novadocuments/86437?v=635670694317400000
     
    Last edited: May 26, 2018
  9. hamlet

    hamlet Registered Member

    Joined:
    May 10, 2005
    Posts:
    229
    Interesting, thanks. For the record, I ran the test against three different programs. Windows Defender and Emsisoft AntiMalware detected none of the items and F-Secure Antivirus detected all of them except for the password protected item.
     
  10. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Well, yes and no.
    The test site are meant to test NGFWs and UTMs on the edge of your network.
    It's not meant to test endpoint protection on your local pc, and therefore the results listed on screen if you do the automated test anyway on a pc will not be reliable.

    On a pc, you just do the test manually instead.

    Instead of pressing the dark grey button labeled "Run all tests", you press the red button below that are labeled "Expand all".

    The 18 test cases are now expanded and you can see that in each test case there are a download button next to the infected file belonging to that specific test.

    All 18 test cases are generated with a fresh screenshot included in each test archive the moment you load the page.
    So no cloud can know the fingerprint of any of the test archives before you start.

    Now click the 18 download buttons next to the 18 infected test files and see the reaction from your local endpoint protection.

    Running the test on a clean, fresh Windows 10 1803 with Windows Defender, I see 17 of the 18 test cases blocked before ever touching the disc.
    The password protected test file are allowed to be saved of course, and the moment I press unpack (the password are "infected" ), then Windows Defender detects, blocks and remove this test case also.

    So a perfect 18/18 protection score with Windows Defender. :thumb:

    People running other AVs can just follow procedure mentioned above and test their setup. :)
     
  11. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    In regards to the Fortinet test is this:
    http://metal.fortiguard.com/about/

    I would assume if this testing required that each test be run individually for AV client installations, Fortinet would have stated same. Therefore, this issue should be clarified with Fortinet itself.

    -EDIT- I came across a comment in this forum indicating that the tests are javascript based: http://www.dslreports.com/forum/r31176317-Fortinet-test-your-metal . If your browser was FireFox, you had the noscript option enabled, and your AV scored 0/18, this is possibly the reason.
     
    Last edited: May 28, 2018
  12. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    As far as verification that the test malware archive files "never touched your disk," you should observe in your AV security solution quarantine file that what was blocked is prefixed by http:// as shown in the below Eset quarantine file screen shot:

    Eset_Metal_Detection.png
     
  13. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Ladies and gentlemen, there's no magic involved here.

    All the test script does is get test case 1, wait for error in browser, draw pretty graphics.
    Get test case 2, wait for error in browser, draw pretty graphics.
    And so forth. .. .. .. .. ..

    The test script has zero clue as to what any endpoint protection on your pc did or didn't do.

    If your endpoint protection blocks the test cases, but does not raise an error in browser - then the test script does not have a clue what happened.

    And as soon as the script gets confused for the first time, then it stops right there.

    There are zero difference between having the script get a test case or having the user press download button to get a test case.

    The only difference is that the user has to count successful blocks themselves (or look in their AV logs afterwards), instead of watching pretty graphics on screen.

    Anyone wanting to do this test on local endpoint protection, should just do it manually. And get reliable results. :thumb:
     
  14. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Form testing with Kaspersky I came to conclusion that script probably checks if file was successfully written to disk. If it was page shows you AV failed, if not it claims it blocked it. If AV doesn't check archive files in transit (on network level) it would be shown as if it failed.
     
  15. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Not at all.
    Test script looks for a network error. That's all.
    It has zero clue as to what happened on your pc or what your AV did or didn't do.
     
  16. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    Yes, that what I meant. If file (for whatever reason) could not be saved on disk (transfer completed), they assume that AV blocked it.
     
  17. Krusty

    Krusty Registered Member

    Joined:
    Feb 3, 2012
    Posts:
    10,193
    Location:
    Among the gum trees
    Firstly, the tests would not run without me allowing scripts in NoScript.

    After I'd done that though Norton scored 1/18.
     
  18. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Test script has no clue if anything was written or if intercepted and blocked before that.
    Test script only looks for network error.

    Which is understandable, since test is a sales promotion for Fortigate.

    That's also why I posted, that if people want to see how their endpoint protection handles these archived threats, then they should push the 18 download buttons.
    Either they see a lot of auto-blocks or else they have 18 downloads waiting for them in their download folder afterwards. :)
     
  19. Minimalist

    Minimalist Registered Member

    Joined:
    Jan 6, 2014
    Posts:
    14,881
    Location:
    Slovenia, EU
    OK, I understand what you mean. I don't know about network error which script is looking for and what it means (I'm not a programmer).
    I came to my conclusions only by testing it with Kaspersky.
     
  20. Trooper

    Trooper Registered Member

    Joined:
    Jan 26, 2005
    Posts:
    5,501
    I'm half asleep and don't know why, but the tests will not run. I keep seeing at the top of the page and the test stops.

    "Are you sure your FireWall is correctly configured?"
     
  21. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    As I posted in reply #11, the tests are JavaScript based. If you browser is blocking its execution, the tests won't run.
     
  22. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    Hi Trooper,
    Just ignore the auto-test and do it manually as mentioned here :)
     
  23. itman

    itman Registered Member

    Joined:
    Jun 22, 2010
    Posts:
    8,591
    Location:
    U.S.A.
    Pertaining to this Fortinet test, it is a test designed for corp. network firewall appliances. These devices sit on the edge of the corp. network and are designed to block Internet traffic prior to entering the corp. network. As such, they employ web filters to do so. These filters not only perform archive scanning for which this test is designed for but also other network traffic filtering activities such as scanning of HTTPS and client e-mail traffic to name a few such activities

    Consumer AV solutions such as those employed by Eset, Kaspersky, etc. are essential the same internally as their endpoint solutions. As such, they also employ web filtering. They do so by intercepting Internet traffic at the network stack level either through use of a network adapter mini-port filter driver or by using the Windows Filtering platform which interfaces with the network stack. In essence, these AV solutions offer retail users many of same network protections dedicated corp. appliances provide.
     
    Last edited: May 29, 2018
  24. Trooper

    Trooper Registered Member

    Joined:
    Jan 26, 2005
    Posts:
    5,501
    Will do. Thanks!
     
  25. Martin_C

    Martin_C Registered Member

    Joined:
    Dec 4, 2014
    Posts:
    525
    You are welcome, Trooper.
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.