New test where Prevx detection is low :(

Is this test good enough to tell something on performance?

http://mtc.sri.com/live_data/av_rankings/

Prevx doing badly

Does heuristics explain this, ie. in real life most samples would have been stopped?

 

The site uses VirusTotal.com, hence its just a on-demand scan without any HIPS/Behavioral blocking. Plus VirusTotal.com scanners are not updated as often as (say) an avg. connected PC.

Hence such results for new malware infection, should be taken with a pinch of salt.

 

I posted the same words some time ago

First: VirusTotal's implementation of Prevx is just an on-demand scanner that doesn't use any advanced heuristic and behavioral analysis components - it just check if a specified file is determined as malware or not. Who has used Prevx before, clearly know how limiting is a test like this. Prevx is not born to be tested like all standard antivirus softwares.

Second: Lately there has been a large proliferation of tests based on VirusTotal. I won't comment on those automatic tests but instead I would focus on a question: where is written how tested samples are really choosed? How samples are really tested if they are really malware and if they are really dangerous and not in any way corrupted files?

On that webpage there's only written "on the limited corpus of malware binaries captures by our honeynet" . It's not really the best way, for a lot of reasons.

Just an example: some antiviruses detect corrupted files as corrupted, some other detect corrupted files as malware (presence of the signature) even if they are clearly corrupted and they can't be executed at all, some other don't detect corrupted files.

So do antiviruses that don't detect corrupted files score worse than other antiviruses? Why?

Then I read on the main page: "The results do not take into consideration the false positive rate of a given tool, and thus a tool that declares everything to be infected would appear to have the highest true positive percentage rate".

Well, I won't either comment on this

 

I know, I know. This comment was for this page:

http://winnow.oitc.com/AntiVirusPerformance.html

What somewhat worries me with this new test is that for all samples PX missed the detection rate of other AV:s has been usually quite high:

http://cgi.mtc.sri.com/popups/av_rankings/10-12-2008/Prevx1_Missed_MD5s.html

So most samples are probably legitimate?

Yes, but is the heuristics really this efficient: 70%-80% of malware detected through heuristics?

 

There are a number of options to explain this. First is that I won't go (really no time ) looking for how many samples are really working. Most of those samples catched by honeypots are often corrupted, even if everything looks like ok. Antiviruses still detect the signature, but the file is harmless.

Second thing is that Prevx still (*still*) doesn't handle detection for infected files (Virut, Parite etc..etc..) and I've already found them in the list of missed malwares.

Third: every sample we don't detect and that is uploaded to VirusTotal is usually analyzed and added within few hours in the database. Indeed, if you check missed files by Prevx on VirusTotal, they have been added. I've already checked some of them, and they have been already added (no time to check whole list).

Most of time yes We could not have a specific signature for the last IRCBot because it has been repacked, crypted, rebased ..blablabla.. but at the end the behavior is the same. On VirusTotal we wouldn't detect it, but indeed we detect it.

Cheers,

Marco

 

Last question:

Recently my setup has been F-secure (firewall only)+Prevx+Defencewall. How much does the lack of "detection for infected files" decrease the level of security? Should I switch FS antivirus on?

Thanks for answers!

 

File infectors are no where near as popular as they used to be, however, they seem to make up a startlingly high percentage of submissions to VirusTotal. Frankly, when hit with a file infector, there is not a whole lot you can do - generally, the best thing to do would just be to restore completely from a backup as no AV can correctly clean all file infectors.

A file infector's best chance of spreading is to piggyback on an already popular infection (as many do), and in this case, we would block it completely. The other method of spreading is to infect a user's system from an executable from another user's system. Generally, safe computing practices say that you should never accept a random executable from someone else and in the case of file infectors, this dramatically reduces your risk.

For 99% of users, our products can be used as the only security solution. However, we don't claim to detect everything (hopefully no one else does because if they do, they're lying ). The big benefits of Prevx's technology are protection against targeted threats (which virtually all antivirus products don't even consider), protection against 0day/emerging threats via behavioral analysis (which VirusTotal and similar engines can't use), protection against complex and mutating malware, and removal of complex malware which existing vendors tend to ignore.

Sure, we might not find some complex variant of Win32.Polipos for instance, but that infector is seen by faaaarrr less than 0.005% of any antivirus vendor's user base and in the time that it took for some vendor's researchers to add protection for that one virus, our automated rules added protection for tens of thousands of pieces of malware

 

I have found Prevx has a very good rate of detection in my own use, which includes surfing to the dark side looking for stuff. Anything I have come across that it doesn't detect I upload using the research tracker and it isn't usually too long getting added. I usually run in Pro mode though so if it dosen't detect it and I know full and well it is malware I can stop it in its tracks manually and send it in to get added to the database.

 

Thanks for bringing attention (cleaning limitations), to this very important aspect of file infecting viruses.

Only once with the Parite virus was NOD32 able to recover a reasonable number of infected files when i was researching it awhile back, but even then it wasn't 100% effective and no one should expect any AV to have renewal powers when it comes to those bugs.

The only sure recovery is a saved "clean" image or in my case a ISR recovery from FD-ISR's archives which i was lucky enough to had presence of mind to store in case of malicious leakage i call it.

 

Definitely true - a system image is definitely the best way to go for restoring from a file infector. And, to make matters worse, many file infectors completely corrupt executables which make them entirely impossible to restore.

Some vendors are using technology which makes copies of executables as the system runs to facilitate true restoring on demand and this is a very good solution... HOWEVER... if the user is already infected before the AV installs, they are out of luck.

Hate to say it, but if you get hit with a file infector, the safest thing to do is to reimage. Sure, its possible to replace every exe in an alternate boot environment, but for the short amount of time it takes to reimage (assuming you have backups ) that would just be far easier. Non-file infectors, however, are a different story and in most cases, it would be an unnecessary use of force to reimage entirely when the malware is generally relatively self contained and removable by specialized tools/products.