ESS 5 fails !

hi,

video :

part 1 (just software) : -http://www.youtube.com/watch?v=7XvJKu07ZxI
p2 : -http://www.youtube.com/watch?v=GDF_l6sq6VA
p3 : -http://www.youtube.com/watch?v=-Kikg0KPqh4
p4 : -http://www.youtube.com/watch?v=WLfK2lZDsKA
p5 : -http://www.youtube.com/watch?v=LCDPQDB2eLA
p6 end : -http://www.youtube.com/watch?v=ICwN5gqhGnU

i am not author of this videos.

but after see :

-> not detect 5xx of 6000 virus ( good but not perfect) in archive
-> cloud is very poor, file is unknow but not action of the cloud ( file is running )
-> HIPS lot lot lot notification and don't stop


idea :

cloud : green = allow ; yellow = ask (run, hips, firewall) ; red/unknow = block
in the cloud : can be terminate process and connection
in the cloud : auto-upload and notification unknow file send

---

hips : better stop notification
hips : block startup modifications in automatic


end ideas for the moment !


thank you for reply, ideas, comment...
sorry for my bad english :/

Arnaud

 

1, no security solution detects and protects against 100% of malware
2, I saw corrupted Autoit application and some dubious Chinese soft in the video. How can one then be sure all those samples are valid for a test?
3, was the reasonably huge collection of samples saved to the disk with real-time protection enabled or was it at least scanned by the on-demand scanner prior to executing them?
4, I went through the videos quickly but didn't notice them testing an archive with 6000 samples. If it was there, couldn't it be that they used the very famous archive with 5917 samples from vx sites containing a lot of prehistoric DOS COM files, corrupted files and benign files? In that case, ESET detects everything that should be detected and the rest is just junk that is not subject to detection.

 

hi

thank you for reply

but i don't talk about just detection...Marcos, i talk about improvement and bugs !

why cloud is not a cloud ? lol
i have always read on the web the cloud is real time analysis with server and communauty (okay i do esay explain my english is bad)
but in eset
, the cloud is more outdated !

when I see Norton ( Sonar), Kapersky ( KSN i believe), and eset ...i laught
in "my cloud" :



sorry but i don't want bad with eset, i like lot software but when i see prevx in cloud in orange with 3 month...the cloud is not live analysis...

The real time cloud is very good for stop, how you say old virus, 0 day attack...if it is real time...sorry but eset cloud is nothing,If I run SysInspector it's the same as watching the cloud.


And i don't talk about HIPS allow unknow file to modify startup key

edit : don't be angry Marcos, i just want eset improve eset5

 

im the video author..

1- its true..
2- yes all files is valid for the test and he was tried by hash (6950 samples)
3- Methodology: The folder of malware are present at the installation of the product, and i make a on-demand scanner on c:
4- Its not a malwares of vx site, this malwares was collecting on different url's and back-up for testing product and tried by hash. the malwares have different categories: virus, rogues, trojans, javascript etc etc...its not a "prehistoric files" ^^.
For example spysheriff contains the trojan Pskill and is was not detected by eset...eset don't detect everything same other product to..

Sorry for my bad english.

The vm was so infected, its not possible to make a sysinspector rapport, he bug at 64%... A alureon rootkit in a false xvid set-up infected the vm etc etc...

The very good evolution since the 4.2 version is a better detection of malware in memory, and better Rogues Blocking

 

I noticed a sort of adware / riskware in the test and also that running an Autoit file resulted in an error message (ie. the script must have been corrupted).
In order for any test to be taken seriously, authors should adhere to the principles set by AMTSO. One of the principles is that tested samples must be provided to the vendors for verification of the quality and objectivity of tests.

 

Hello Marcos, thanks for reply,

But i want testing the av product in "customers profile".

And my malwares folder, contains set of malware meet by the user daily.

Im not a "professionnal", its true, but i think eset don't make products only for professionnal.

If 30 malwares/500 is corrupted, 470 was not detected...the number is not imortant, its the danger family the important...

I remain at your disposal for further inquiry

 

Given that these "tests" are not professional as you admitted, they should not be taken seriously as they are performed on a testbed consisting of corrupted samples and samples of dubious quality. The samples haven't been verified by the vendor which is required step before prestigious testers publish their test results If you wish, you can supply us with the missed samples for verification so that we can tell how many of them were actually supposed to be detected.

 

Are you testing the beta or release candidate?

 

hello, I have testing the two versions... The Beta and the Release candidate.

The release candidate is more effective...

 

Most customers who use your products are not as professional.

The few samples which you think is a problem, can not justify such a gap detection.

Most files are verifiable on VT.

More like old programs are still not detected (spywarethis etc etc...) the collect date of this sample is 2010 (old programs)

if I take as an example antivir, it leaves 284 samples and Eset 550, been thought that this difference justifies the corrupted files

collection of these files and the fruit of my labor and time consuming, I do not provide a lab

Please see this link, the protection test,Eset was under industry average.. Particulary blocking malware on post execution.
http://www.av-test.org/reports/2011q1/avtest_report_eset_110941.pdf
Its more professionnal

 

Is easier for you to provide such samples to ESET

the fact another scanners detected the samples is insufficient, that only could show the reputation of a given file, in such cases can be False positives
testing is not an easy task, and could create confusion between customers, if the samples have poor quality

edit: prevalent samples are more important than zoo samples collected in VX sites or posted in sites as MDL

 

Is not easier:
1- i have a poor internet connection.
2- its a 2Gbytes malware folder ^^ Too long in upload.
3- I am not employed, im a customer, its not my job.

False positive is impossible in my malware folder...I spent many hours to verify that...

My samples are représentaifs that can meet the users on the Web.

If you want i make another video-test with samples of threatcenter:
http://threatcenter.crdf.fr/

I already try the result is the same...and you will see that the problem does not come from of the quality of my malware folder.

 

Have you mentioned you tried with file hash?

 

The fact remains that you're just one guy, doing a test you designed, with malware you say is genuine. That makes it unreliable. Like Marcos said, if you're really serious about AV testing, take a look at AMTSO.

 

Maybe you could burn a DVD and post it.
I am keen to see ESET improve their detection when ever possible.

 

Its true im one guy, and one guy bypassed Eset with malware representative of threats present on the Web..

I don't need AMTSO...I need a browser and Threatcenter samples...as any users..

I do not see the interest of protecting against threats that the user has little chance to meet and pass the most common.

I think AVtest.org are not "one guy".

After everyone's opinion, my goal is to share with other users, not polemics

 

The fact remains, he also could be exactly right with his findings. We sit here and ridicule because he isnt a professionnal tester, but when the professional testers post their results, we claim they are idiots.

In a situation like this there isnt a way to prove it for all of us, so in the end it really means nothing. Over time, if he is correct it will get validity from other testing sites, with Esets ability or lack of, to detect malware.

 

I'm not saying he's wrong, I'm saying that there is no way to validate the results.

 

But in 2 threads now you jump on someone for speaking out against ESET!

Everyone who pays for ESET is entitled to say their piece on it be it good or bad and ESS v5 has a lot of work to be as good as it can get.

In both threads you did not even present a good argument as the peeps were both in the right!

I personally think it is good someone who knows a bit more than the average Joe does some testing as it is the closest we can get to real user usage!

 

Arguing is pointless. This road never ends. Like I said he may be right, but I did some testing the other night and the RC was outstanding. Blocking all and stopping the one Panda let through that tried to wipe my hard drive.

So it just really doesnt matter, to each his own

 

i wonder who is that user wanting protection and they infect the system intentionally with samples posted in sites that 1% of users will visit

and not taking into account the samples are not randomly selected and some samples are corrupted



that does not make sense,

 

Actually it does make sense and proves that this thread is pointless.

 

As I always say - judge security products according to your personal experience. What's important is how well they protect you and not how many "samples" they detect in various serious or amateurish tests. If you get infected frequently you won't like the product no matter how high it ranks in tests.

 

exceptional ... it's a bit much either?

 

Of course but the problem is that some of these have malware was collected on sites so your argument of scarcity does not ..... but if I take the case of site keygenguru now correctly detected by Eset, there 'a few were not there. And even if it is clear that the behavior of the user is most important, we know that unfortunately this type of site is very popular, so now what you think you are free men.