I applaud Prevx’s openness to sharing information

Building upon the comments of Macstorm (see post #161 in this thread), I applaud Prevx’s openness to sharing information about “threats that your current security products missed” on the home page of their website. Nonetheless, there are several aspects of the reported statistics that raise questions.

• If you examine the list of threats that each security vendor “missed,” there appears to be significant overlap across vendors. It is possible that the same threat might be missed by Eset, by F-Secure, by Kaspersky, by Panda and by Symantec—but, it does not seem probable. The combined resources of all these major companies are substantial and their methods are independent. To believe that each and every one of these vendors is failing to detect the threats identified by Prevx is questionable.
  
• Prevx does not report the number of PCs scanned for which there are no identified threats by vendor, although it clearly must be in possession of that information. I encourage Prevx to post these data as well, in order to present a more complete analysis of the situation.
  
• If you accept that the list of “threats that your current security products missed” on the Prevx website is accurate, then it still does not logically follow that Prevx would have prevented the installation of this malware for the same users under the same circumstances. The distinction is one between detection versus prevention, and numbers reported on the Prevx website are only applicable to the former.
  
• The counts of the “threats that your current security products missed” on Prevx website appears to be a count of suspicious files found—not a count of the number of threats detected. Since a single threat will most likely involve multiple files, the numbers are “exaggerated.” I encourage Prevx to consider correcting the labeling of their chart from “threats that your current security products missed” to “suspicious files that your current security products missed.”
  
• The counts of the “threats that your current security products missed” on Prevx website may, to an unknown degree, be residual traces of malware that the security vendor has disabled but has not completely deleted. As the consequence, these counts reported by Prevx may, to some degree, represent inactive threats with no risk to the user.
  
• The counts of the “threats that your current security products missed” on Prevx website may, to an unknown degree, be false positives. It is typically the case that products with the highest detection rates (e.g., G DATA and AVIRA) also have the highest false positive rates (see Anti-Virus Comparative No. 21, February 2009). I encourage Prevx to participate in the Anti-Virus Comparative (and in other independent anti-virus ratings) so that users can better understand the strengths and weaknesses of its product versus the competition.

I welcome comments, clarifications and corrections from forum members as well as from Prevx on these points.

 

Re: New MBR rootkit goes undetected

Hello Pleonasm,
Thank you for the criticism - it is great to open discussion about topics like this in a professional manner. Here is my response to each point in order as they appear:

1) We gather our data from the Windows Security Center so if we detect a threat on a computer with an AV installed as the primary AV on the system in the security center, we report it as having been missed by that AV. I agree that it doesn't seem probable that a vast majority of the antivirus solutions on the market miss similar threats, but the fact is that it is true. Threat life cycles are much shorter today than they used to be. In the past, it was acceptable to release definition updates once per week as threats propagated slowly. Today, many companies release new definitions on an hourly basis - but this is still not fast enough. The problem is that users have to actually download the new definitions and then scan the system again - an unfeasible request. The difference with our approach is that the centralized signatures allow us to get realtime updates with absolutely no delay: the instant that a file is detected, every Prevx customer is protected by it. The model of downloading signature updates can be powerful against certain types of threats, but not against highly mutating threats that are unique to each PC. Because of this, the only way to protect against threats first seen in the previous few seconds is to check with an instantly updated database which applies realtime heuristics. Because of this, we have a unique view on infections which other companies don't have, allowing us to detect threats they miss on a conceptual and practical level.

2) About a year ago, we showed this data on our homepage as well but we took it down because it was an unfair comparison. Users looking for our products already suspect they are infected and are looking for a solution to their problem or a way to prevent it in the future so the percentage of users who come to us are higher than the global average of users who are infected. That being said, around 50% of users coming to us, regardless of their security product, are infected with some form of malware.

3) Based on our architecture, any threat which we detect we would also prevent at the time of detection. We of course don't block everything and in the off chance that we do miss a threat, we add protection very quickly. The tests which we show are valid because the infections already exist on the computers and had you encountered all of those infections for the first time on the day you bought Prevx, you would be protected against all of them.

4) The count is indeed the number of malicious files found and if we counted the number of threats, the number would actually be far higher. We count duplicates of a file in different locations as a single file and in our reporting we don't display any information about registry entries or other malicious components. The files may not each be a complete infection, but it is impossible to say that a group of files came from the same unique infection so we are forced to show the information on a per-file basis. Granted, that one file may not house the entire infection, but each component does perform at least some malicious action so we detect them independently. We have measures in place to prevent abuse as well, one example being that we limit the number of reports coming from a single computer/IP.

5) This is true to some extent, however, based on the way that we scan the system, we only focus on real threats - ones currently registered in the registry, existing in a system location, loaded in memory, being accessed by other programs, and anything which exhibits rootkit-like behavior or is hidden to the OS. Our scanning is able to get around the locks which AVs put on files, but from what I've seen, AVs tend to move files into quarantine rather than leave them in place. When a file remains in its original location, it has a high propensity for reinfection as it can easily be re-registered by another, undetected component or it can just continue to spread out from under the locking if the AV doesn't correctly disable the in-memory components of the file. We don't go scanning through quarantine folders or other repositories which the AVs may have on the local system, so I don't think that the numbers are skewed because of this.

6) We are susceptible to false positives just as any other vendor is, however, once we correct a false positive, the reports online are immediately updated to reflect the change. The false positive rates of AV companies as reported by tests are frequently misleading when applied to a test like ours. If we were to have a false positive rate of even 1/100th of 1%, we would have been out of business long ago because virtually every one of our millions of users would be coming in complaining that we cleaned some file which was legitimate. However, for the sake of the test, feel free to assume a FP rate of 1%, or even 5% (which would mean on this computer I'm writing to you on where 28,839 files were scanned, 1,441 files would have been detected when it has always come up clean ).

The problem with current antivirus testing organizations is that they assess products against old threats. If the test results which show an AV blocking 99% of threats were actually applicable to the threats seen on a day-to-day basis, we would not be seeing the dramatic, exponential increase in malware we see today. We detect literally thousands of new infections every day, and these are infections coming directly from our users, not from our malware research teams. The core difference between our metrics and the metrics from other tests is that ours is backed by real world data, seen by users today, not threats collected over the previous 10 years that no users have ever encountered.

We keep track of the count of Prevx users who have actually seen a specific threat. In some smaller AV tests, we have looked at the static malware samples and seen that a surprising majority of files had never actually been seen by one of our users. Sure, we aren't as large as Symantec or McAfee, but why should we focus on threats which simply... aren't threats?

Conventional AV testing currently focuses heavily on non-threats and until this changes, we don't agree with it and really can't be tested in this manner as it is so terribly inaccurate to actually assess the effectiveness of a security solution.

Sorry for the long reply here, but I think it is a very important topic to discuss and lay our cards out in the open. Our goal of showing the data on other AV vendors is not to put them up against each other, rather, its to try and teach the average, non-technical user that their security is not 100% - every product is fallible, our included. It is with a combination of security solutions that the margin of susceptibility decreases.

Please let me know your thoughts and comments. We're very open to any suggestions and want to make all of our information as clear to users as possible.

 

Re: New MBR rootkit goes undetected

Based on Prevx support in this forum...I would conclude that any statistics brought by Prevx are as close to the truth as one can get.

 

Re: New MBR rootkit goes undetected

I am not a PrevX user, but I really admire their competence and professionalism.

I have PrevX free on some systems and until now they have not found any threat that my other security suites have missed. Which IMO, is due to the fact that in my network we have simply restricted access to corridors to common sources of malware infection.

So unless we are on the hitlist of malware writers ( or disgruntled employees) the current traditional model is enough. I am guessing the same is true for most other places, else by now PrevX would have become a large giant which would start bundling toolbars and search engine integration without a care in the world.

 

Re: New MBR rootkit goes undetected

Haha Don't worry - we are growing fast but will never have the need to pack spyware/useless-ware into our software

 

Re: New MBR rootkit goes undetected

PrevxHelp, I sincerely appreciate your prompt and thoughtful reply to my comments. Some additional observations follow.

The implication is that other security vendors would have the same virus signatures as Prevx, although – in some cases – perhaps not as quickly. In other words, if I understand your comments correctly, the difference between Prevx and other security vendors is primarily one of responsiveness.

Nonetheless, there still appears to be something awry in the argument. Consider the case of PDFUPD.EXE, which is reported as one of the most common threats now detected by Prevx which is missed by AVG, Avira, Eset, F-Secure, Kaspersky, Microsoft, Symantec and ZoneLabs. According to the Prevx notes on this threat, it first appeared in May, 2008 and more recently in March and April, 2009. Are you saying that despite the existence of this threat for nearly one year, none of these major anti-virus vendors has been successful in protecting against it? The claim appears to be incredulous—but may, of course, be true.

This is only partially true, since it is not a complete description of how tests are performed. In the case of AV-Comparatives, for example, a retrospective test is employed to get “an idea of how much new malware a scanner … can detect … , before a signature is provided for the malware” (review the testing methodology).

The comment may be true of “some smaller AV tests,” but it is not reflective all AV tests. As AV-Comparatives notes in its methodology document, “…the era where ‘zoo’ samples were submitted by the malware authors only to labs, and which therefore existed only in labs and collections, ended years ago.”

Of this number of infections, how many are unique threats?

Assuming this to be true, it should be the case that a comparison of Prevx to other security vendors would show Prevx to be vastly superior. Yet, Prevx has not, to the best of my knowledge, been assessed by any independent organization, such as AV-Comparatives, ICSA Labs, NSS Labs or West Coast Labs. The unwillingness of Prevx to have its claims subjected to review, at face value, does not appear to reflect well on the assertions by the company.

Are you recommending that Prevx be used in conjunction with the products of other security companies as a supplement in order to heighten the certainty that malware is not present on a PC?

 

Re: New MBR rootkit goes undetected

The problems here are both responsiveness and economical viability of threat detection. If a threat only affects a portion of a userbase of an AV company or doesn't generate any prompts which cause users to become suspicious that they're infected, the AV company would never have any knowledge of it. A quick googling says that PDFUPD.EXE is a new PDF exploit: http://mnin.blogspot.com/2009/04/malware-forensics-how-ironic-can-it-get.html

My guess is that this threat first started last May but has now started gaining traction, possibly because it was first used as a targeted attack. AVs cannot handle targeted attacks because they need to focus on signatures that affect the largest number of users possible because of the sheer volume of threats which come out on a day-to-day basis.

Retrospective tests are more "in the right direction" with being able to test our technology, however, we can't "freeze" our definitions as the other vendors can because our database is highly dynamic so the tests which are done with 1-2 month old signature bases against today's threats simply cannot be done with our technology.

That's hard to say - take Vundo for example: are all Vundo infections counted as one threat, or is each of the few thousand variants counted separately? Trying to count infections or signatures or unique threats is probably not a logically viable thing to attempt just because of how dynamic everything is and how much crossover there is between threats. (i.e. is a sample of Conficker infected with Virut considered a new, unique threat or just Conficker or just Virut? )

Again, we aren't unwilling to have our claims tested - we just conceptually can't be subject to the types of tests that these organizations provide. We'll have to see what AMTSO ends up deciding on, but our solution is based in-the-cloud so trying to run it in a secluded environment offline or with old definitions just won't work, and that's how most of these tests work today. We also do require data to be sent up about samples for analysis and testing organizations have "disqualified" us in the past because of this, but there is simply no way around it because our analysis is centralized.

Correct - you CAN use Prevx by itself but it won't block everything. You can also use Symantec by itself but it won't block everything... we aren't making claims that we're better than everyone else, just that we block the threats that the others miss. Sure, they block things that we miss as well, which is why its recommended to use multiple layers of security. Many users feel adequately secured based on their browsing habits to just use Prevx 3.0 but if they want, they can easily put a free AV or any other security alongside it, without complaints from us or reduced system stability.

We've developed Prevx 3.0 to be a complete incremental security solution - rather than forcing it to work as the only security in your system as many AVs do, we allow you to install any other security products alongside it to achieve the best protection you can. We've explicitly engineered it for maximum cross compatibility and we test this thoroughly on every release.

Hope that helps! Let me know if you have any other thoughts.

 

Re: New MBR rootkit goes undetected

hey PrevxHelp,
any chance of a verison for usb sticks so i can clean machines without having to install anything on clients machines?
In bulti updator for a small fee would be ideal.

 

Re: New MBR rootkit goes undetected

PrevxHelp, this is an interesting discussion. To continue the conversation…

Please say more. It doesn’t seem reasonable that “freezing” the definitions in a database is an impossibility. Point-in-time snapshots of live databases are easily taken using tools such as ShadowProtect.

This comment implies that anti-virus vendors knowing allow their users to be at risk, because they issue signatures against some – but not all – of the threats that they detect on a daily basis. If true, then that is a condemnation of their practices, in my opinion. However, allow me to ask: is your comment a ‘theoretical’ understanding of how AVG, Avira, Eset, F-Secure, Kaspersky, Microsoft, Symantec and ZoneLabs operate – or, are you stating that they in fact knowingly and willingly allow users to be placed at risk by ignoring a subset of detected threats?

Prevx is not a member of the Anti-Malware Testing Standards Organization (ATMSO) – correct? In that case, how does the company intend to influence its testing methodology standards?

Please explain the rationale and technical limitations in more detail.

Even if it is true that there exists no organization on the planet which is able and willing to perform a test of the Prevx product, then why not hire a well-respected independent organization (e.g., maybe TÜV-IT, an independent German security auditor) to perform the testing of Prevx against other major anti-virus vendors using a methodology of which you approve?

I do not understand. If Prevx is claiming to block substantial numbers of threats that other security vendors miss, how is that not the same as saying that Prevx is “better than everyone else”?

What data can you share about this? Are the number of detection failures larger or smaller for Prevx as compared to other vendors?

Thank you.

 

Re: New MBR rootkit goes undetected

Well you may be underestimating the size and complexity of our databases I don't think ShadowProtect would work very well on many terabytes of data, across multiple datacenters worldwide, being fed live data from millions of agents simultaneously Our architecture cannot simply be "stopped" and it isn't exactly an economically wise decision to try and duplicate millions of dollars of servers just for a test

I'm not meaning that they are knowingly allowing their users to be at risk - frankly they probably don't know at all. The problem lies in the reporting: it is very easy for a threat to fly completely under the radar if it has only ever been seen by a single user, or even 1000 users if it is a targeted threat. Our analysis can detect a file after its been seen by only a single user, even if it is completely unknown to our database (granted, it may take a bit of extra time to gather the behaviors from the single system but in most cases we will end up detecting it). Our architecture allows us to look at threats seen by single users, and the volume of targeted threats is actually quite significant. An AV company can only work with what they have - which is user submissions and web crawling, otherwise, a majority of them do not have any centralized reporting (this is changing, but they have some work to do before the solutions are completely viable).

We are considering joining AMTSO but at this point it is hard to change the entire AV testing industry as we are a single company going up against 100+ others.

The core limitations start because of how Prevx products are structured - they require online scanning to check with the newest definitions and data needs to be sent up to our databases. This first strike was a deciding factor for a number of AV testing firms because they don't want any data about their samples to leave the testing environment.

The second limitation is one which many other vendors are starting to run into is that our protection engines can't just be tested by right clicking and scanning a file. That will find a file if we have it in the blacklist or in the first few layers of heuristic analysis but it really is no demonstration of the accuracy or power of our technology. The best way to test Prevx products is to install them on an infected computer or to use driveby infections/etc. to test in a real world situation.

We work to protect users - not to change our technology so that it will perform well in certain tests. This is also why we have a policy to not detect leaktests unless they're being used by malware - they have significantly different behavior from real threats and it would be no actual benefit to customers to add protection for them.

The best way to test us is to try and get infected by real world threats, and that's what testing companies should do - not try and horde samples together and throw them all en-mass, but to use today's threats, spreading by today's exploits/techniques, against up-to-date detection from AVs. If something gets past at that point, that would be the best way to assess the security level/strength. This is what we do every day internally - we always question our technology by trying to get infected with the latest and greatest infections and if we find any problems, we fix them immediately.

We aren't saying that we're better than everyone else - we're saying that we fill the gaps where they miss. It depends on the individual user's situation and detection is just one of the many aspects to consider when purchasing an AV.

I don't have the inverse data as no other vendor provides the realtime view like we do, but that would really be the most accurate assessment of a product's effectiveness, in my opinion.

 

Re: New MBR rootkit goes undetected

If I understand it correctly... TF is very different from Prevx. Instead of being a mix of AV and pure proactive, databased, heuristical protection, what it does is don't give a crap about how old or new something is, but basically check "what's this thing actually doing?", even before looking in its black- and whitelists - TF that's. I think that makes it less likely to produce FPs since it'll first check: is something doing dangerous stuff? Then, if it does, see if it's a legitimate program doing so, so that it should let it be, or if it's actually something known malicious so it can take care of it directly and give the user the information about it.

Personally (yeah, right now I'm a fan-boy ), I think that also makes it likely to be more effective. If it's something that's unknown and does dangerous stuff - fine, I'll tell the user what's and then he or she can decide.

 

Re: New MBR rootkit goes undetected

PrevxHelp, a few more observations...

Is Prevx uploading information from a user’s PC to a centralized database? If so, what protections exist to ensure that the user’s privacy is maintained? (I noticed that Prevx does not have a link to a privacy policy on the home page of its website.)

What is preventing Prevx from performing precisely that procedure? Why not deploy a set of identically configured PCs, each running the same script to simulate usage, and evaluate which anti-virus products are detecting/preventing threats most effectively?

* * * * * * * * * * * * * * *​

In summary, it may be the case that Prevx is as good as – perhaps better than – some of the anti-virus competition. However, in the absence of any independent proof of that conjecture, it remains simply a ‘personal perspective’ of Prevx rather than an empirical technological assessment. I strongly encourage Prevx to address this significant shortcoming, so that the user community can evaluate evidence about the strengths and weaknesses of Prevx as compared to the competition.

 

Re: New MBR rootkit goes undetected

The privacy policy for the product is in the EULA pre-installation, which describes the privacy policies for the data. No personal data is collected - only data related to executable programs - and that data is encrypted at all levels of transmission and storage.

We do this frequently in-house, but we couldn't just release these results by ourselves, and paying an independent organization review your software always throws up some concerns for the legitimacy of the review. However, the real problem is trying to run a statistically significant branch of tests across a wide range of vendors in a real world environment. Its very possible to do it, but it is an expensive test to run, although I expect tests like this to be performed by the major testing firms soon at which point I suspect we will be more than welcoming to join in the testing.

The best way to really see how effective Prevx is would be to use it. Independent tests cover ranges of infections which would never be encountered by most users (geographically-specific infections or non-spreading infections). The simple fact is that we do provide an incremental layer of protection over whatever security you have, as shown by the graphs on our homepage.

Even if you don't pay to use our full version, you can use it indefinitely for free. If something does happen to slip past your existing security and we find it, you'll immediately see the benefit and reason for using us.

The overhead of Prevx 3.0 is very light, especially when combined with another AV, and in a world where no one company can find everything, it is worth putting one's best foot forward to try and get the best security possible by combining complementary technology.

 

Re: New MBR rootkit goes undetected

PrevxHelp, thank you for continuing the conversation.

I do not understand why….

This simple fact, however, is not the issue. The question isn’t whether Prevx provides “an incremental layer of protection over whatever security you have,” since using any other anti-virus product to supplement a primary defense would accomplish the same objective. Instead, the question is whether Prevx is worse, the same, or better than the competition – whether Prevx is used either as a primary or secondary anti-virus layer.

Unfortunately, based upon this conversation, it appears that there exists no independent and empirical evidence whatsoever that Prevx is the same or better than the competition. Please do correct me, if I am mistaken.

P.S.: I am sincerely impressed with the thoughtfulness and timeliness of your responses.

 

Re: New MBR rootkit goes undetected

With all due respect I see a fatal flaw there already... since when has the Windows Security Center been a reliable indicator of the presence/status (or not) of a security application?

As you yourself have noted on many occasions...e.g. here and here to quote you "The Security/Action Center in Windows is a highly unreliable feature".

So when you check the WSC status, you may "see" an antivirus that was previously removed, and that is no longer resident on the computer as being currently installed. This is not an indication that it has "failed" to detect anything.

Then we come onto the question regarding how do you know that the user has not chosen to ignore that file, or has changed the settings of their program/has not bothered to update which means it is running with out of date signatures (meaning there is a PBKAC and the program/vendor itself is not at fault)...we could also take spread/geolocation of a threat into account meaning it may be a localised epidemic in timbuktoo but a user from mainland Europe may never come into contact with this threat...there are just too many different factors associated to make any of those statistics remotely reliable or have any other purpose than FUD, in my opinion.

So, showing the number of computers that were "infected" is an unfair comparison (by your own admission) whilst absolutely crapping on your competitors is fair?
I see a large contradiction there, and wonder why many other vendors refrain from such dubious marketing on their homepages.
Any one of those vendors that you mention on your homepage could produce a similar chart stating "Hey, look what Prevx misses and what WE detect"....but none of them (as far as I am aware) do so because it is a blatant misrepresentation of information and a completely unreliable method for gauging how effective a product is at protecting a user- since for every piece of malware that is caught by vendor X, there is large number of pieces that are not caught by vendor X but may be caught by vendor Y and vice versa. Why, for example, does this forum have a no VirusTotal linking policy- I would assume it is for exactly the reason I have mentioned above- doing such X vs Y comparisions is amost always completely useless.

It is all well and good for you and other prevx representatives to post replies/explinations here on wilders and watch posters get caught up in the general good spirits regarding your service/responsiveness but until you (prevx) decides to raise the bar and cease publishing such equivocal messages on your website and cease using some slightly unsavoury marketing practises I will not be convinced.

You may have a brilliant product, staff, and operate a legitimate business- this itself brings with it a certain sense of responsibility that you must uphold and set an example for other firms to follow, while at the same time it does not give you free reign to employ borderline deceptive marketing. I am not admitting other vendors do not employ such tactics at times too- you are all commerical organisations which are out to make a profit at the end of the day- but the way your organisation chooses to present themselves in this area leaves a number of questions in my mind.

Please do not mistake my observations and comments as an unjustified, outright bitter attack on prevx, I am simply stating what I see to be the shortcomings in your model.

 

Re: New MBR rootkit goes undetected

hi, me too i'm not convinced with prevx edge but when i install PE and test it in real world (see below link)....PrevxEdge (together with Sandboxie) did the job done. my KIS is silent.

https://www.wilderssecurity.com/showthread.php?p=1438975#post1438975

 

Re: New MBR rootkit goes undetected

Hi fce,

That is one of the points I was touching on- I never said Prevx wasn't good at what it does, and in some areas the whole concept of Prevx can mean response to 0 day threats and detections where the resident AV is not able to detect the "infection". I was commenting more on the aspects of "hey look what we can detect that the others don't know about"...it doesn't make sense because the same thing could be done with other samples that prevx doesnt detect but antivirus X does.

 

Boy. All this makes things appear almost complicated if not redundant in some respects for me. LoL

My motto is if it works then it's worth it, if it misses something or somethings down the line, report it and expect reasonable explaination which in this day and age seems the real problem is theres a growing populus of students who are picking up fast on ways to circumvent windows security vendors protections and thus add a feather to their cap and pick up some popularity or added credits. It would be hoped these type projects was done to correct code mistakes because we're all human and the smallest distractions can divert attention away at a time that created a missed programming key somewhere.

Lengthy and detailed as all this discussion is, it's still noteworthy in many respects and hopefully helps clear the air on some grievances as well as expectations, but i still say, if it works for you, keeps the PC shielded and at the very least alerts at-once to something/anything possibly time-wasting or destructive to you, be grateful and pitch in when you can to help it become an even better program for the future of all.

EASTER

 

@Pleonasm

Like Easter, if the security supplied by the application suits user or network needs and usage, then it is a successful tool - for that environment. If not, then one needs to move to something else. No "single" product is likely to satisfy "all" potential user/network environments.

Dicsussions about efficacy being determined by testing agencies is a slippery slope indeed. Numerous, and IMHO - unresolvable, issues arise regarding the metrics to be applied to the testing and the relative merit of catching various malware specimens across a various time spectra. The measuring criteria themselves are as dynamic as the threat criteria and as dynamic as the evolution of the anti-malware technology.

For the average user, the measure that matters, amost likely the only measure that matters, will always be - did I get infected or not...or conversely, was I protected or not. It may be that the answer to that question, for the average user environment (other environments will vary), would best be arrived at by real-world benchmarking a nominally configured system...say Windows XP SP3 fully updated + Windows Firewall + whatever "single" security application one wishes to test...and placed directly on the web (not behind any router - i.e. hardware firewall). Then let the user go forth into the wild...so to speak.

Whatever security app one is testing, it will be breached...by something at some point. The measure of one app versus the wild and of one app versus other apps can be made from the frequency and quantity of system breaches and from the severity of various types of breaches. Even utilizing this type of testing philosophy still leaves one with issues of how to evaluate one app versus another due to what "priorities" one may place on various types of threat severities versus/coupled with frequency of breaches...a truly maddening situation if one is attempting to compare the efficacy of various security apps. And...addtionally maddening if one wishes to parse efficacy against differing user/network environments.

Discussion of testing, performance, comparable efficacy, etc. is a very slippery slope and is frought with a broad scope of variables and criteria - none of which are likely to have universal acceptance or agreeability. All the information one gathers from any source(s) matters not if "your" user/network environment does not match the "testing" environments from which the information was assembled.

The bottom line must be established by seeing if a given application - knowing that none stop all threats - does in fact offer "adequate" protection for "your" use...obviously a determination required to made by "you"...

galileo

 

Other anti-virus vendors may refrain from such tactics in part because “attacking the competition” typically isn’t a successful long-term marketing strategy for building a brand. While PrevxHelp has acknowledged that the statistics on the company’s website about “threats that your current security products missed” do not support the case for the superiority of Prevx, it certainly creates that impression for the casual observer. In that respect, the display of the information may indeed be a “misrepresentation.”

And, in my mind as well. The circumlocution of Prevx creates the subjective impression that it has something to hide. At this time, I prefer to give Prevx the “benefit of the doubt,” however, in the hope that the company will take active steps to more thoughtfully compare itself to the competition.

I do agree.

I would not argue that empirical testing results by an independent organization should be the only factor considered when choosing one anti-virus product over another. Nonetheless, more information is better than less information, and thus such comparatives are beneficial, in my opinion.

Ditto. Prevx may indeed be a superior product, but without the active participation by the company in empirical and independent comparisons, none of us will ever know.

PrevxHelp, I hope that you will return to this thread and either confirm or correct this statement.

 

Pleonasm,
I understand your initial point and in theory I agree. However knowing some things about competition between companies ( not in the security sector ) I can tell you that Prevx's marketing strategy seems really innocent...I don't want to say the word naive ( no offense Prevx ). I have seen worse...worse...worse...worse.

My opinion is that they are sincere. In any case, every company that considers that the marketing strategy of Prevx damages its interests and image, should ask from a court to examine the case. Otherwise they should shut up or reply using the same means and methods. Why do you think that marketing should be ethical and right? I have never seen a marketing campaign that is 100% ethical, right and respects 100% the competitors.

So all these discussions that I have read here ( interesting reading though when you want to learn more about Prevx and security in general ) for me have no meaning at all. And this because you have missed the marketing side of the story.

Personally, I would expect some more strict rules about the privacy, not only from Prevx...I'm talking about all the security products. Some kind of certificate. All these "new" products with the in-the-cloud technology should be more trasparent. Not to me...but to the authorities that protect the privacy of the citizens. Actually I support the idea that everyone who touches my files should have the means...anytime...to provide me a list of what he examined...uploaded...etc. Even if you scan 40 millions files I want...at least the authorities to be able to press the "magic" button and see a complete log of the scan and communication (network traffic) between the local client and the server.

 

Hi Folks,

Based on these discussions I installed PrevX to go with my Avira and Online Armor. However I still do not have the whole gestalt of what it does differently since the scan showed nothing (which is good in a sense). I will have to study the looonnng thread and the website, time permitting. Based on this thread I can give my view so far.

1) PrevX looks like a fine company

2) PrevX has made some advertising and product representations that muddy the waters and they should take to heart the prodding here to improve that element. Most of us have some forebearance on hyperbole in advertising, yet since integrity is the heart of the security industry, the more accurate and helpful the representations, the better.

3) Their model of full continual, ongoing scan but not cleaning (which one here very unfortunately attacked as rogue based on some external similarities that overlook fundamental differences) is quite understandable in the context of their program. The model should be under continual review, yet all models have pluses and minuses between free and commercial, as PrevX painstakingly pointed out. Which explanation was fair and quite helpful. PrevX has been very open about the fact that you can "discover" something with PrevX and then bounce the discovery off with them or Wilders or others and use any tool you want and that they will work quickly to correct false positives. And if you buy on a false positive, I accept their representation that they will refund on request. (I think correcting false positives is easier for a small company to do than the anti-virus bureaucracies, based on a recent Emsi experience). No complaints at all from me on this point. I even got a laugh about the fella complaining that you might rush to buy the product because your credit card information was being stolen and you did not have time to wait patiently for forum responses. Overall, I don't think I would mind the small purchase of the good product that made the malware discovery in that instance !

4) Similarly I accept their point about the laboratory testing facilities. That their model is different and especially that the limitations of the labs that prevent uphill information goes against their model and they are encouraging and awaiting a more sensible test.

At any rate, if they would improve #2, and the user is taking a "layered" approach to defense, the issues about % detection become much less germane that reports of what works in the 'wild', as pointed out by many posters. What works today for the malware that just came forth. At the very least, at this time I am willing to give PrevX the benefit of the doubt on the testing issue.

Thus, please reconsider the hyperbole claims pointed out on the thread (the ones where the competition could say the exact same about you, but don't bother because you are small or that is not their style) try to describe your niche better, and expand on the very fine Wilder's and security community goodwill that come forth from your product and the participation in the forums.

Shalom,
Steven Avery

 

NoIos, I too do not doubt the sincerity of Prevx. Clearly, this company is one of the “good guys” seeking to protect the interests of PC users.

NoIos, from a marketing perspective, the core issue isn’t about respecting the competition—rather, it is about respecting your company’s own customers (and prospects). All serious marketers today know the importance of trust in growing customer lifetime value. If a company makes statements that are potentially unclear, misleading and/or deceptive, then customer trust is eroded—thereby destroying customer equity. For a company operating in the realm of security, the criticality of trust is further heightened.

NoIos, I am unfamiliar with the details of the Prevx privacy model myself, but I do share your concern. Personally, I would never install or use any application that automatically uploads files from my PC, without my having given explicit prior permission on a file-by-file basis.

Steven, very well stated.

I fear that unsophisticated PC users may easily misinterpret the statistics about “threats that your current security products missed” on the Prevx website. It is worthwhile to note that the company itself claimed that these statistics represented “a real world measure of comparative antivirus protection” when they were first introduced on June 28, 2008 (see here). In my opinion, the “marketing” of these statistics in this way is subjectively unfortunate and distasteful, and objectively questionable.

Personally, I would feel much more comfortable if Prevx expanded the explanation that appears when you click “Explain this chart” on the website to repeat what PrevxHelp said earlier: “We aren't saying that we're better than everyone else” and “{competitive products} block things that we miss as well,” since Prevx wants “to make all of our information as clear to users as possible.” I hope that Prevx will seriously consider at least this suggestion, which I am providing in the spirit of constructive criticism.

Steven, every problem has a solution, and I believe – if Prevx were strongly motivated to do so – it could overcome any such obstacles within a testing methodology. The scientist within me simply cannot accept the idea that the comparative quality of the Prevx product is unknowable. Indeed, even PrevxHelp stated that the “best way to test Prevx products is to install them on an infected computer or to use drive-by infections/etc. to test in a real world situation” – a procedure which has no insurmountable obstacles for implementation.

* * * * * * * * * * * * * * * * * * * *​

PrevxHelp, for the “threats that your current security products missed” reported on your website, what percent were classified by Prevx as low, medium and high risk?

* * * * * * * * * * * * * * * * * * * *​

I sincerely wish Prevx the best of luck in developing and promoting its products. As more companies compete in the anti-malware market, users are certain to benefit.

 

Re: New MBR rootkit goes undetected

Hello all,
Sorry for the delay - I've been on a plane and preparing for our presentations at RSA 2009 (come visit us at stand 2732!)

If we were to release the results of our internal tests, they would be criticized as biased. We are working on finding a testing organization which can perform the levels of tests required, but testing live infections is time consuming and expensive.

Because Prevx operates conceptually very differently from other AVs, we're able to find an entire class of malware which others simply cannot find based on their definition models. We detect a high volume of "normal" threats as well, but those aren't difficult to find The real challenges for every AV vendor right now are: server-side polymorphic malware, rootkits, targeted threats, drive-by infections, and high-volume malware releases, all of which we handle extremely well.

Until AV testing organizations can provide a reliable way to test vendors against these forms of threats, it will be difficult to see who is better than who. Currently I'm unaware of any organization which has correctly assessed the abilities of security products against these forms of threats - and these threats are the ones affecting a majority of users today.

 

Re: New MBR rootkit goes undetected

Sure, that is possible, but the issues don't happen all THAT frequently and they tend to only happen on computers which have uninstalled/reinstalled multiple times. WSC is technologically relatively unreliable, but a majority of enterprises/large companies use it to see the status of their user's PCs.

Ignoring a file from one AV which is detected by another vendor (us) would really tend to mean that it is malicious and that would indeed be a PEBKAC situation. However, a majority of users are no where near technical enough to configure an override policy in their antivirus program, and the chances of this happening on a file which is indeed malicious and detected by us has to be extremely low.

The concept of geolocation/spread applies to other antivirus tests as well - antivirus testing organizations use threats from all over the world, most of which a user is never going to see just because they have no reason to visit a website which would spread the threat. I think this is an issue only as much as it is an issue in other tests. For instance, Asian malware is covered extremely well by Asian antivirus programs, but they tend to not detect as much non-Asian malware (from what we've seen). Does this make them any less effective? Honestly, I don't think so - if you are an Asian user, I think a strong Asian antivirus program would be a great addition to your arsenal, just as Russian users benefit by using Kaspersky's localized research, etc. Our view of signatures is global and we have users from basically every country across the world who all act as analysis points, reporting data centrally which we analyze. Geography does play a role in our heuristics, as well as the popularity of a program and the age of a program within a geographic region.

No, showing the percentage of computers is unfair because most of our users think they may be infected, which is why they're coming to us. The data we're reporting is raw data - the number of infections seen across the community of users.

It is true that every product finds things that other products don't, but THAT is the point we're trying to make with our graphs. The average, every day user does NOT understand that concept, which is why we're showing the statistics on what gets past AVs every day. Users need to understand that regardless of what they're using, it isn't 100%. We aren't using the charts to say that X av is better than Y av, rather, we're saying that both X and Y av miss things - the things they miss may be completely different, or the same, but that doesn't matter - we're focusing on the fact that they miss samples and we provide the data to back that up.