ESS 5 fails !

Discussion in 'ESET NOD32 Antivirus/Smart Security Beta' started by dorgane, Jul 10, 2011.

Thread Status:
Not open for further replies.
  1. dorgane

    dorgane Registered Member

    Joined:
    Oct 17, 2007
    Posts:
    362
    hi,

    video :

    part 1 (just software) : -http://www.youtube.com/watch?v=7XvJKu07ZxI
    p2 : -http://www.youtube.com/watch?v=GDF_l6sq6VA
    p3 : -http://www.youtube.com/watch?v=-Kikg0KPqh4
    p4 : -http://www.youtube.com/watch?v=WLfK2lZDsKA
    p5 : -http://www.youtube.com/watch?v=LCDPQDB2eLA
    p6 end : -http://www.youtube.com/watch?v=ICwN5gqhGnU

    i am not author of this videos.

    but after see :

    -> not detect 5xx of 6000 virus ( good but not perfect) in archive
    -> cloud is very poor, file is unknow but not action of the cloud ( file is running )
    -> HIPS lot lot lot notification and don't stop


    idea :

    cloud : green = allow ; yellow = ask (run, hips, firewall) ; red/unknow = block
    in the cloud : can be terminate process and connection
    in the cloud : auto-upload and notification unknow file send

    ---

    hips : better stop notification
    hips : block startup modifications in automatic


    end ideas for the moment !


    thank you for reply, ideas, comment...
    sorry for my bad english :/

    Arnaud
     
  2. Marcos

    Marcos Eset Staff Account

    Joined:
    Nov 22, 2002
    Posts:
    14,374
    1, no security solution detects and protects against 100% of malware
    2, I saw corrupted Autoit application and some dubious Chinese soft in the video. How can one then be sure all those samples are valid for a test?
    3, was the reasonably huge collection of samples saved to the disk with real-time protection enabled or was it at least scanned by the on-demand scanner prior to executing them?
    4, I went through the videos quickly but didn't notice them testing an archive with 6000 samples. If it was there, couldn't it be that they used the very famous archive with 5917 samples from vx sites containing a lot of prehistoric DOS COM files, corrupted files and benign files? In that case, ESET detects everything that should be detected and the rest is just junk that is not subject to detection.
     
  3. dorgane

    dorgane Registered Member

    Joined:
    Oct 17, 2007
    Posts:
    362
    hi

    thank you for reply

    but i don't talk about just detection...Marcos, i talk about improvement and bugs !

    why cloud is not a cloud ? lol
    i have always read on the web the cloud is real time analysis with server and communauty (okay i do esay explain my english is bad)
    but in eset
    , the cloud is more outdated !

    when I see Norton ( Sonar), Kapersky ( KSN i believe), and eset ...i laught
    in "my cloud" :

    prevx.png

    sorry but i don't want bad with eset, i like lot software but when i see prevx in cloud in orange with 3 month...the cloud is not live analysis...

    The real time cloud is very good for stop, how you say old virus, 0 day attack...if it is real time...sorry but eset cloud is nothing,If I run SysInspector it's the same as watching the cloud.


    And i don't talk about HIPS allow unknow file to modify startup key :gack:

    edit : don't be angry Marcos, i just want eset improve eset5 ;)
     
  4. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    im the video author..

    1- its true..
    2- yes all files is valid for the test and he was tried by hash (6950 samples)
    3- Methodology: The folder of malware are present at the installation of the product, and i make a on-demand scanner on c:
    4- Its not a malwares of vx site, this malwares was collecting on different url's and back-up for testing product and tried by hash. the malwares have different categories: virus, rogues, trojans, javascript etc etc...its not a "prehistoric files" ^^.
    For example spysheriff contains the trojan Pskill and is was not detected by eset...eset don't detect everything same other product to..

    Sorry for my bad english.

    The vm was so infected, its not possible to make a sysinspector rapport, he bug at 64%... A alureon rootkit in a false xvid set-up infected the vm etc etc...

    The very good evolution since the 4.2 version is a better detection of malware in memory, and better Rogues Blocking
     
  5. Marcos

    Marcos Eset Staff Account

    Joined:
    Nov 22, 2002
    Posts:
    14,374
    I noticed a sort of adware / riskware in the test and also that running an Autoit file resulted in an error message (ie. the script must have been corrupted).
    In order for any test to be taken seriously, authors should adhere to the principles set by AMTSO. One of the principles is that tested samples must be provided to the vendors for verification of the quality and objectivity of tests.
     
  6. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    Hello Marcos, thanks for reply,

    But i want testing the av product in "customers profile".

    And my malwares folder, contains set of malware meet by the user daily.

    Im not a "professionnal", its true, but i think eset don't make products only for professionnal.

    If 30 malwares/500 is corrupted, 470 was not detected...the number is not imortant, its the danger family the important...

    I remain at your disposal for further inquiry
     
  7. Marcos

    Marcos Eset Staff Account

    Joined:
    Nov 22, 2002
    Posts:
    14,374
    Given that these "tests" are not professional as you admitted, they should not be taken seriously as they are performed on a testbed consisting of corrupted samples and samples of dubious quality. The samples haven't been verified by the vendor which is required step before prestigious testers publish their test results If you wish, you can supply us with the missed samples for verification so that we can tell how many of them were actually supposed to be detected.
     
  8. toxinon12345

    toxinon12345 Registered Member

    Joined:
    Sep 8, 2010
    Posts:
    1,200
    Location:
    Managua, Nicaragua
    Are you testing the beta or release candidate?
     
  9. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60

    hello, I have testing the two versions... The Beta and the Release candidate.

    The release candidate is more effective...
     
  10. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    Most customers who use your products are not as professional.

    The few samples which you think is a problem, can not justify such a gap detection.

    Most files are verifiable on VT.

    More like old programs are still not detected (spywarethis etc etc...) the collect date of this sample is 2010 (old programs)

    if I take as an example antivir, it leaves 284 samples and Eset 550, been thought that this difference justifies the corrupted files

    collection of these files and the fruit of my labor and time consuming, I do not provide a lab

    Please see this link, the protection test,Eset was under industry average.. Particulary blocking malware on post execution.
    http://www.av-test.org/reports/2011q1/avtest_report_eset_110941.pdf
    Its more professionnalo_O
     
    Last edited: Jul 11, 2011
  11. toxinon12345

    toxinon12345 Registered Member

    Joined:
    Sep 8, 2010
    Posts:
    1,200
    Location:
    Managua, Nicaragua
    Is easier for you to provide such samples to ESET

    the fact another scanners detected the samples is insufficient, that only could show the reputation of a given file, in such cases can be False positives
    testing is not an easy task, and could create confusion between customers, if the samples have poor quality

    edit: prevalent samples are more important than zoo samples collected in VX sites or posted in sites as MDL
     
  12. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    Is not easier:
    1- i have a poor internet connection.
    2- its a 2Gbytes malware folder ^^ Too long in upload.
    3- I am not employed, im a customer, its not my job.

    False positive is impossible in my malware folder...I spent many hours to verify that...

    My samples are représentaifs that can meet the users on the Web.

    If you want i make another video-test with samples of threatcenter:
    http://threatcenter.crdf.fr/

    I already try the result is the same...and you will see that the problem does not come from of the quality of my malware folder.
     
    Last edited: Jul 11, 2011
  13. toxinon12345

    toxinon12345 Registered Member

    Joined:
    Sep 8, 2010
    Posts:
    1,200
    Location:
    Managua, Nicaragua
    Have you mentioned you tried with file hash?
     
  14. dmaasland

    dmaasland Registered Member

    Joined:
    Nov 10, 2010
    Posts:
    468
    The fact remains that you're just one guy, doing a test you designed, with malware you say is genuine. That makes it unreliable. Like Marcos said, if you're really serious about AV testing, take a look at AMTSO.
     
  15. patch

    patch Registered Member

    Joined:
    May 14, 2007
    Posts:
    178
    Maybe you could burn a DVD and post it.
    I am keen to see ESET improve their detection when ever possible.
     
  16. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    Its true im one guy, and one guy bypassed Eset with malware representative of threats present on the Web..

    I don't need AMTSO...I need a browser and Threatcenter samples...as any users..

    I do not see the interest of protecting against threats that the user has little chance to meet and pass the most common.

    I think AVtest.org are not "one guy".

    After everyone's opinion, my goal is to share with other users, not polemics
     
  17. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina

    The fact remains, he also could be exactly right with his findings. We sit here and ridicule because he isnt a professionnal tester, but when the professional testers post their results, we claim they are idiots. :gack:

    In a situation like this there isnt a way to prove it for all of us, so in the end it really means nothing. Over time, if he is correct it will get validity from other testing sites, with Esets ability or lack of, to detect malware.
     
  18. dmaasland

    dmaasland Registered Member

    Joined:
    Nov 10, 2010
    Posts:
    468
    I'm not saying he's wrong, I'm saying that there is no way to validate the results.
     
  19. Temp Member

    Temp Member Registered Member

    Joined:
    Mar 28, 2009
    Posts:
    263
    Location:
    Glasgow

    But in 2 threads now you jump on someone for speaking out against ESET!

    Everyone who pays for ESET is entitled to say their piece on it be it good or bad and ESS v5 has a lot of work to be as good as it can get.

    In both threads you did not even present a good argument as the peeps were both in the right!

    I personally think it is good someone who knows a bit more than the average Joe does some testing as it is the closest we can get to real user usage!
     
  20. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    Arguing is pointless. This road never ends. Like I said he may be right, but I did some testing the other night and the RC was outstanding. Blocking all and stopping the one Panda let through that tried to wipe my hard drive.

    So it just really doesnt matter, to each his own
     
  21. toxinon12345

    toxinon12345 Registered Member

    Joined:
    Sep 8, 2010
    Posts:
    1,200
    Location:
    Managua, Nicaragua
    i wonder who is that user wanting protection and they infect the system intentionally with samples posted in sites that 1% of users will visit

    and not taking into account the samples are not randomly selected and some samples are corrupted



    that does not make sense,
     
  22. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina

    Actually it does make sense and proves that this thread is pointless.
     
  23. Marcos

    Marcos Eset Staff Account

    Joined:
    Nov 22, 2002
    Posts:
    14,374
    As I always say - judge security products according to your personal experience. What's important is how well they protect you and not how many "samples" they detect in various serious or amateurish tests. If you get infected frequently you won't like the product no matter how high it ranks in tests.
     
  24. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60

    exceptional ... it's a bit much either?
     
  25. vigen

    vigen Registered Member

    Joined:
    Mar 28, 2011
    Posts:
    60
    Of course but the problem is that some of these have malware was collected on sites so your argument of scarcity does not ..... but if I take the case of site keygenguru now correctly detected by Eset, there 'a few were not there. And even if it is clear that the behavior of the user is most important, we know that unfortunately this type of site is very popular, so now what you think you are free men.
     
Thread Status:
Not open for further replies.