Dr Web and AV Comparatives

Well, I don't know what exactly was stated. Perhaps I have misunderstood. Perhaps Bryndis was just trying to be polite. In any case, I don't make policy at FRISK - I just work there. (Had I been the one responsible for such a policy, I would have withdrawn our product from your tests even when they were free. It is my firm belief that participating in such low quality tests undermines our credibility. But, hey, that's just me. People who know me personally know that I tend to have a very short fuse and very little tolerance for incompetence and stupidity. Perhaps I'm also a bit biased, given that I was the one who personally had to deal with at least part of the crap you were sending to us as "undetected malware".

I think I already addressed this issue. Yes, a small fee for a testing company with no other revenue is not unreasonable. Your fee is neither small nor reasonable, though. Hell, you ask for a single (incompetent, crappy) test about as much as I make in a year! And you don't have even a minuscule fraction of the knowledge, expertise and aptitude of what takes to be a good AV tester, let alone an AV researcher.

I have never heard of any AV testing entity charging this ridiculous amount of money. I think even the ICSA charge less - and they are well-known in the industry for essentially selling incompetent overpriced "certifications".

But, hey, to each their opinion. I've stated mine. You're welcome to yours. I wish you luck in your endeavor and I am happy than the company I work for is no longer part of it.

Regards,
Vesselin

 

well, ok, your opinion. things changed in past years.
i heard that other testers ask for by far much more than we do (at least this is what peoples told me, if its true or not, i do not know).

 

Somehow I missed this when I was replying. As anybody who can read and has access to Virus Bulletin knows, their tests are by far not based exclusively on the WildList. Apparently, you fail on at least one of these two conditions.

Of course. Nobody is perfect. However, I've had to deal with the "quality" of your tests (or, more exactly, the lack thereof) for several years already and I am convinced now that you don't just "make a few excusable mistakes". Over all these years, their quality hasn't improved an iota. The only thing that has changed is the number of samples used.

Now, I've had the unfortune to study Marxism at school, and there there is the principle that "increased quantities lead to a change in quality" or something like that (not sure how exactly it translates in English). In reality, however, a pile of crap is still crap - even if it is a very big pile.

{snip - Blue}By now I am convinced that you'll never be able to reverse-engineer a piece of malware and understand how it works (or why it doesn't) and that you'll never grasp the fundamental principles on which the various anti-virus products are based.

All you can do is gather a huge pile of files from various dubious sources, run a bunch of scanners on them, process the results in tables (which might even be wrong - I can't tell, because I don't have access to the raw data), and call it a "test".{Snip - Blue}

{Snip - Blue}

Oh, well. Have a nice day.

Regards,
Vesselin

 

Umm, would this be the same as my Big Mac theory.

So in the end, the tests have always been crap, Eset really does rank about in the middle as most other places show,CSJ was right over the last 2 years questioning the validity of them, and life goes on.

 

To all,

Let us keep the discussion centered on topics, not people.

Blue

 

LOL don't bring me into it jeff,

Keeping a low key on here lately, I shall leave the disagreements to everyone else.

I don't really question whether the tests are valid' or not, I just like to make up my own mind about things, rather than being told.

 

Actually, real life is a lot more nuanced than that.

There are very clear tradeoffs that one makes to achieve a large and contemporaneous testbed. The key question is whether those tradeoffs improve or detract from the overall insight provided by these test results.

Blue

 

Well actually you do. Its called price discrimination. McDonalds near my university (University of New South Wales) offers reduced prices for students.

Cinemas offer discounts for students and pensioners. Holiday makers pay less than business flyers. The list goes on.

What about the figures/technical details behind these claims?

bontchev, can you provide some figures to support your claims? the missed samples sent to you, how many functional? how many weren't?

IBK, can you provide some information of your virus collection methodology? Also how do you analyse to ensure that they are all functional and relevant?

 

I second this.

Bontchev's claims don't sound very consistent at all when you consider the kind of scores that other products consistently achieve, which indicate that they must put a lot of effort indeed into detecting "crap". I have no problems believing that Avira, for instance, would be the king at doing this, but not so for Kaspersky, and DEFINITELY not so for NOD32 and Symantec.

So I guess the question is... does Mr Bontchev have the statistics to back up his personal insults?

 

well personally, I dont think Eset is crap. To the contrary. I also think that Bontchev is smarter then to make such claims without knowing his stuff. So who knows, or as one person said, who cares. The bigger question is why did it take this long for some to speak up if this is what they think. That is what bothers me. Good or bad.

 

The disagreement between AV vendors and testers certainly surprises me. Having listened to both ends, I'm almost sure that we will never have objetive/peer-reviewed/statistically meaningful/accurate data to compare AV performance.
Sorry for the folks who depend on their AV to be protected. Better start learning and applying safe computing principles in-depth.

 

Along with firewalls, anti spam, Hips, all of which dont provide 100 percent protection. And some virtualization products. AVs are not the bad guy. The frigging user is. I am so sick of people saying that all the pain in the world is the fault of their AV.

 

Never said they were - at least not in my previous post. I think you need to read it again carefully to see what I said.

Coincidentally, smart people - especially those of his station - also do not make such claims in public without being able to back them up. Mr Bontchev claimed earlier that he has little patience with stupidity. The question now is whether he's one of the smart people.

 

Yep, suddenly IBK's testbed is full of garbage files. Until recently, AV-C tests were considered to be reliable by the industry, with garbage files kept to a minimum.
IMO, this disagreement will hurt both AV testers and AV vendors. This is not good.

 

I agree on both points.

 

Right with firewalls. Not so with HIPS and other non-signature apps. With non-signature apps. you know what they can and can't do (if you do a bit of research and asking) so you're somewhat aware of the possible failures. Also, the practice of safe computing will keep you well protected (safe surfing, e-mail security, patch policy, LUA+SRP, common sense, etc) and with fall-back solutions (imaging, back up, etc)
AVs need testing, you need to know if the signatures they're pumping are keeping up the pace with malware writers, you need to know if AVs can clean a missed infection when the signatures are updated, etc.

 

after having some issues with my wifes computer trust me, people create more problems that av's can cure. I'll also tell you I have come to the conclusion that no test (no disrespect intended) & I mean none can tell you how well a product will work for an individual. That is one reason why I disagree with the current Wilders policy of not comparing AV's. People are the best testers we have as they have hands on testing & IMHO that's what I need to hear. From people who run product X & had product Y & here's why it's better. To not allow that discuss to me take away the ability to intelligently discuss products from a hands on experience. I'd rather hear that than read test results...

 

the only issue with that Larry, and I include myself in this, is that most will tell you theirs is the best because it is their trademark, their hero, their freebie. What they wont tell you is why, like what it caught that another didnt. To be perfectly most, most dont even know. So yes, if people posted based on solid usage instead of fanboy material, it would prove worthy. But that isnt the case and again, myself is included.

 

Correct, but you're missing one point. I can tell you how well product XXX runs on my machine, take screenshots of it, play with the options, etc but I'm unable to tell you if it will protect you. In the end, almost all AVs run well in the majority of configurations, but not all AVs protects well the non-security savvy user and there's no fall-back.

 

cool post larry.

But expect LWM to squash it

But you already know this,right?

Its amazing how people can defend thier own decisions, even though its not needed, and I myself am at fault for this too. LOL

 

well there was a fall back, but they quit making it.

 

The most conflicting information we give here is when we have threads like this and then turn around have threads like this. We are really good at sending conflicting signals sometimes. The only thing we all agree on is there isnt a perfect solution. But there are a lot of damn good ones involving different products and vendors. To a new person coming here I can only hope they walk away with knowing that more stuff works then doesnt work, because otherwise I would not blame them for turning the power off to their PC and getting up and walking away.

 

your right but we're her to discuss topics on AV's & yes we all have our fav's but I can pull myself from the fav to have an intelligent conversation about others I tried. A newbie shoud be able to come here give us his machine specs, needs & as intelligent folks throw out ideas to help him/her choose. The person should be able to decide on the product but we have experience handling programs setups & malware. Others should benefit from that experience. I still think real world running of a product tells 10 times more than product x finished 1st in testing...& FWIW NO product can guarantee you'll have a clean machine forever. If the person is a sloppy surfer they are going to get hit...

 

Let it go Luke, let it go.

 

Here is some advice for those posting in this thread....

• Stay on the nominal topical thread, which has moved a tad recently
• Stay off speculating on future moderating actions - if this continues it will result in immediate closure of this thread. The continuing lack of any clue on this point really continues to astound me.
• Stay off personally directed comments. Comment on technical/logisticals points to your hearts content as long as it's topically appropriate
• Be civil and contribute with some thoughtful and reasoned comment.

Regards,

Blue