Prevx detection rate (again)

Discussion in 'other anti-malware software' started by ako, Feb 1, 2008.

Thread Status:
Not open for further replies.
  1. ako

    ako Registered Member

    Joined:
    Nov 16, 2006
    Posts:
    627
    Prevx scanner misses suprisingly many new _EXECUTABLE_ malware samples, see http://malware-research.co.uk/ .

    See also http://virusinfo.info/index.php?page=testseng
    Prevx detecs directly there only a few new samples. A lot more is seen by its heuristics, but experience tells that Prevx heuristics at Virustotal gives too many false positives to be useful/reliable.

    How much would the likelihood for detection be improved on the average, if malware would be executed?
     
  2. Hermescomputers

    Hermescomputers Registered Member

    Joined:
    Jan 9, 2006
    Posts:
    1,069
    Location:
    Toronto, Ontario, Canada, eh?

    I had several "Obvious" new virus samples... everyone of them known by at least 4 AV's on Virustotal.com all but one where ignored during a full HD scan. The one that got picked up, was only detected the next day...

    I guess the strength of Prevx isn't in the detection realm yet but in the prevention of executables running wild. It does block those from being fully executed. A good reason perhaps to keep an AV on hand just to compensate for this deficiency.
     
  3. Perman

    Perman Registered Member

    Joined:
    Nov 23, 2005
    Posts:
    2,160
    Hi,

    I was a Prevx aficionado until very recently. I have begun to feel uncomfortable with these facts:

    (1) it has not rolled out ANY new build, let alone any new version, for some long time, folks at Prevx probably know this much better than I do.

    (2) it can not give any affirmative disposition, if their mighty databases do NOT have that info on file. And its behaviour mechanism is a so-so me-too chapter. THEREFORE,

    I would not surprise to learn that Prevx's rating in overall is deteriorating. I do hope folks at prevx can wake up to improve it, and please do not shift your focus--you guys have spent too too much time wasting in corporate sector; those fat bodies will not let their network secrets exposed to your mighty community bank concepts. They may have a bright brain as smart as you guys are.

    Take care.
     
  4. ako

    ako Registered Member

    Joined:
    Nov 16, 2006
    Posts:
    627
    At least at http://winnow.oitc.com/AntiVirusPerformance.html Prevx has not improved at all during last year :(

    Below a comment on this by Tom Shaw from OITC:

    ( copied from http://www.castlecops.com/t203755-Prevx_performance.html )

    The reason we are posting to this thread is due to an email sent to us asking to comment on this thread. It is important to correct the record so we will copy most of our response to the original email.

    I hope this clarifies our response. Weel thought out and crafted questions will be responded to; flames not.

    ---------

    Dear x,

    Thank you for contacting us.

    First, it should be noted that no one from Prevx has ever contacted us to discuss our methodology including Prevx Host.

    Now on to Prevx Host's comments,

    "You should bear in mind that these stats include non-executable malware; like scripts, rougue html, macros, boot sector threats etc. These are out of scope for Prevx 2.0 - which is focused on executable file-based malware only."

    This is a really bizarre comment as

    1) as stated above no one from Prevx has ever contacted us to ask us how we obtain (and thus the mix) of what comes into our system and is tested. Our methodology is documented at http://winnow.oitc.com/avreadme.html. Note that we don't care how the malware entered onto our honeypots. The vector is not important to us. Once each malware file enters into the honeypot (or in your system) it is an "executable file-based threat" to use the terminology of Prevx Host

    2) many file-based malware are scripts (PHP, PERL, VBL and even SH) so we do not understand Prevx Host's comment that these are non executable maleware especially since there are huge numbers of bots that are PERL or PHP based.

    Lets just keep to the facts. Our system does not check performance against "yesterdays database" whether it is 100,000 entries or 100 million entries nor does it check how good a AV system is in detecting and removing malware.

    What we do analyze is the real-world performance of AV systems on "new" malware (eg near 0-Day outbreaks) and this is displayed on the running graph which has been calculating and accumulating running statistics for over a year. We also provide combinatorial statistics for near 0-Day outbreak detections.

    We also maintain a host of other and complementary statistics for near 0-Day outbreak detection which are proprietary.

    Given that we are maintaining a running, real world, detection statistics, what is interesting is Prevx's lack of detection rate change over the last year after initial stabilization. This documents definitively that Prevx's detection performance has not improved over the last year for the detection parameters we are interested in.

    What does concern us is Prevx Host and other Prevx defenders making up data used to defend their indefensible positions. Here Prevx Host made up what our system processes and then disparaged our statistics based upon his made up information. In other instances for which we have been privately asked about, such as this offline question of yours, Prevx individuals have made up other negative remarks to defend their system.

    What is sad is that Prevx Host and others "attack the messenger" rather than look at what Prevx might be lacking and attempt to correct and improve the Prevx system's performance.

    We do not popularize the data we publish on the web. We just publish our data as a public service and make it available to the community as data with no fan fare. How you interpret our data is your own issue but, please, interpret it based upon exactly how it is derived and not based upon rumors and innuendos of how it is derived.

    If I was looking for an AV system, I surely would not select one from a group that could not characterize malware properly ("these stats include non-executable malware; like scripts") and lowered themselves to make up data to disparage alternative views and who never ever contacted us to discuss and understand what our data really meant.

    I hope this answers your question.
     
  5. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    I don't think Prevx is good enough yet to be a stand alone but combined with a good antivirus it does provide that little extra bit of protection that just may come in handy.
     
  6. Hermescomputers

    Hermescomputers Registered Member

    Joined:
    Jan 9, 2006
    Posts:
    1,069
    Location:
    Toronto, Ontario, Canada, eh?
    Prevx while not being the sharpest knife cutting the virus bug... It is still a product that combines some of the best concepts. I just wish the product was not so reliant on "Multiple negative report" concept before it lists something...

    I think somewhere in my wish list there is something about better W/B list cooperation among all products to speed up hostile inclusions... Regardless of brand/products or types...
     
  7. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    That would be near impossible to pull off when you consider all the software that is out there. Prevx does have a good concept in doing this with the community based approach but I think it would be impossibe to ever build a database that would provide everyone with a 100% solution. I still consider Prevx a work in progress but they are making strides in the right direction.
     
  8. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
  9. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    Thanks Eraser....very good read. For english not being your mother language you did a good job answering the question and adding to what Ghiser attempting to explain. For anyone who hasn't checked out that thread, Ghiser 1 made a fairly good response to Tom Shaw's post and Eraser filled in the rest of the blanks, I won't copy and paste the responses so to be fair you should check out the other side of the story.

    I think the big problem is that most people can't get their heads past traditional AV thinking. Prevx is a whole new beast and like all other security tools, is not perfect but it is another good tool in the battle against malware.
     
  10. ako

    ako Registered Member

    Joined:
    Nov 16, 2006
    Posts:
    627
    Thank you for an interesting and useful reply.
     
  11. Hermescomputers

    Hermescomputers Registered Member

    Joined:
    Jan 9, 2006
    Posts:
    1,069
    Location:
    Toronto, Ontario, Canada, eh?
    If you where a hammer you'd just hit that nail right on the head! :thumb:
     
  12. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Part of the problem is also, I think, the lack of a trial version of Prevx for people to see for themselves if the tall claims are true. And in the meanwhile Prevx is merrily continuing to post that very misleading detection graph on their homepage.
     
  13. Hermescomputers

    Hermescomputers Registered Member

    Joined:
    Jan 9, 2006
    Posts:
    1,069
    Location:
    Toronto, Ontario, Canada, eh?
    Personally I think the PREVX CSI free scan is really good trialware... It would provide many with an overview of infections on their computers and as such they can assert its potency as a security tool... as for the graph... Someone else can answer to that as I lack the appropriate stats...

    I have personally used CSI on infected systems successfully. While it did not find all active infections it certainly allowed me to see there was a problem within less than 4 minutes... I find that remarkable.
     
  14. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Only because it only scans active processes, startup entries and system files, hence its quick scanning time. If you have a good eye you can do the same with SREng or GMER in one.
     
  15. Hermescomputers

    Hermescomputers Registered Member

    Joined:
    Jan 9, 2006
    Posts:
    1,069
    Location:
    Toronto, Ontario, Canada, eh?
    Yes, but those tools certainly are not for the "Average Joe" solcroft... You, I and most of the Geeks @ Wilders are more than comfy with those types of tools certainly not the little old lady whose pc was used by a hack to broadcast viagra adverts...
     
  16. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    My point was more that "4 minutes" is because it only scans a very small area of your computer.

    I'd have no problems believing that, DrWeb CureIt!, for instance, would be able to accomplish the same in a similar amount of time if you told it to scan only the areas that Prevx CSI does.
     
  17. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    When they first started Prevx was free until your first infection clean up and then you had 30 days to buy it before it disabled itself. Problem was that people would use it and and not really have to buy it until it caught something. Not a good way to generate revenue and with the way Prevx is set up with the Central Community Database controlling everything I would imagine it would be an expensive proposition to maintain when you take all the computers connected by the Prevx agent constantly talking back and forth with Home Base rather than a server just pushing out updates a few times a day. I see why they had to drop their original way of doing business. Although I would agree that they should have gone half way with it and offer a 14 or 30 day full trial and then it completely disables.
    I have only used the CSI scan a few times on my VM so I can't really comment on that. I opted for the full blown Prevx on my main computer and so far have been satisfied with it.
     
  18. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    I agree 100% with you Hermes and I think that Prevx is looking to create the "HIPS for the non techie" with their product.
     
  19. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    Although we do not scan every single file in the system with Prevx CSI, our fine-tuned scan task list allows us to narrow down our focus into the most important places. We check all active areas (registry included, it's implicit) along with most important inactive system areas (and this doesn't mean only Windows System directory, of course).

    The probability of finding executable malware within documents or in the configuration files for programs is actually very low (read carefully, it isn't impossible but if it would be an active threat and not only a copy of malicious code - as peer to peer worms do for example when copying itself into other directories - we would detect them anyway). We are working on adding a full system scan to a future version which would be invoked when suspicious files/data is found to give full view of the system, but, this is a very expensive module for us because of the thousands of probably clean files which need to be scanned.

    If you do have any suggestions on other areas which CSI appears to not scan, please let us know. We are willing to work with the community to provide the best, free scanner we can provide.

    Thanks for your thoughts.

    Best regards,

    Marco
     
  20. solcroft

    solcroft Registered Member

    Joined:
    Jun 1, 2006
    Posts:
    1,639
    Marco,

    Actually that wasn't meant as criticism of CSI in the first place; I was just responding to Hermes' "4 minutes" comment. You're indeed right that scanning only those select files and folders would give a very good indication of whether a machine is infected. Which is why tools like SREng and GMER don't give you a whole listing of files on the HD either, only active processes and information from select parts of the registry.
     
  21. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    Don't worry :) My reply wasn't intended as a criticism to you. I tried to better explain to people who haven't well understood how Prevx CSI work.

    Sorry, english isn't my mother language so sometimes I could appear more arrogant when usually I'm not or I could use wrong words :)

    All the best,

    Marco
     
  22. Threedog

    Threedog Registered Member

    Joined:
    Mar 20, 2005
    Posts:
    1,125
    Location:
    Nova Scotia, Canada
    So the file scan option on Prevx 2.0 basically does the same thing as CSI ? I've only used this option once when I first installed it and noticed it did the scan fairly fast. I never bothered doing a full blown file scan. I can see where the full file scan would be very costly for you guys given that every little file would have to be OK'd by the Community when there is really no need to do all the files.
     
  23. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    There are several options when you do a file scan on Prevx 2.0: Smart Scan, Full System Scan and Custom Scan.

    If you're referring to the first option, yes, more or less they are based on the same concept. Prevx CSI has anyway definitely improved it.
     
  24. Perman

    Perman Registered Member

    Joined:
    Nov 23, 2005
    Posts:
    2,160
    Hi,

    That was exactly why I hooked into it and subsequently recommended it to many of my acquaintances--all average joes/janes. But the "no news" situation from Prevx has worried me. Don't you worry a bit when your favorite app stalls and has not rolled out any NEW build/version for some time ?

    The malware-fighting front can not be so quite-everyone else are doing the same thing as Prevx does; updating database frequently. But many of them have updated their core component. Prevx's core section can not be so solid that it does not require any improvement.

    I think Prevx folks have shifted their focus from Prevx2 to something else, and would not acknowledge it. IMO.
     
  25. EraserHW

    EraserHW Malware Expert

    Joined:
    Oct 19, 2005
    Posts:
    588
    Location:
    Italy
    As I replied to another thread like this, if there aren't many updates actually to Prevx 2.0, this doesn't mean that we're not giving anymore importance to the software.

    I don't see any major antivirus company that release major improvements to their product as update. Usually they release signature updates or, sometimes, bug fixes and minor improvements to their core engine.

    Minor improvements? We're going to release an update to our drivers to fully support incoming Windows XP Service Pack 3 and Windows Vista Service Pack 1. This is just a "minor" improvement, together with other small bug fixes.

    We don't release signature updates, that's true and if you've understood how Prevx works you know why.

    But I'm sure you can understand we can't release a whole new version of Prevx in an update. And, moreover, the fact that we're more quiet lately - at least on the front of Prevx 2.0 - couldn't it be a hint that we're developing something new, some major improvement (redesign of the whole software, for example) that can't be released as simple updates?

    Besides, since the release of Prevx 2.0 we've released a number of bug fixes and minor improvements to our core engine. This mean that actually we've reached a good stability and we're working on something totally new.

    Has ever Symantec released Norton Antivirus 2008 in small updates during Norton 2007 period? Not as far as I know. Minor improvements and bug fixes can be released, but when you see everything is quiet, this could both mean we don't care you and we're hardly working on something new that will replace Prevx 2.0 with a new release.

    Two major releases usually take something like one year at least between them.

    Best regards,

    Marco
     
Loading...
Thread Status:
Not open for further replies.