New Report, New Website and Greetings from PCSL

PCSL's test is desinged to simulate the whole infection flow in one single test.
Static scan: one a threat can be recognized by the product, then qua or delete it.
Dynamic test: a missed sample is executed, and blocked by behaviral engine or stopped by av product, then qua or clean it.
Static False positive test: to test if there is static fp.
Dynamic false positive test: To avoid the agrresive behavioral block technology.

so you can find that we have restored the real use of normal client.

For getting used to products. We have to use professional tool to guard the AV products while doing dynamic test while to get used to clinet products is a must. Consumers can know litter about that but we have to know a lot. So comes the question If you think we know little like the normal consumers, will youn trust my result>?

 

I stronly recommand you read the english result pdf file list above(in floor 5).
I suppose that you take PCSL's total detection rate to other test's static detection rate.

Total detection rate combined static detection rate and dynamic block rate(e.g. obtained by Panda Truprevent).

And I mentioned above, PCSL has own monitor system to catch freshest samples but it does not mean vendors can not do such things.

I can also give you several links that will produce thousands of malware producted by robot, same variety different MD5, mostly, they will produce millions of malware per year. But, please notice that those samples will seldom infect computer users and if I put these samples into sample set, can you believe that the test refelct the real status? I think I have explained a lot above and if you have time, please read those first and maybe you will get a better understanding .

Thanks and will follow your further questions.

 

I think if vendor's are given the opportunity to provide 200 of their own samples, found on actual user systems, they would also try and supply samples that are common to them and uncommon to other companies.

For example, twister might supply samples affecting asian users, gamers etc, some samples that say dr web might not detect, and vice versa. Dr Web would supply common malware along with some obscure samples picked up on russian systems.

Although the test results show many of the AVs today are effective, I personally think that's the case. Too often you see tests where one product is labelled as poor because it doesn't detect samples which can't be found through a common search engine request, but only if you dig 50 pages deep in google's list. (I think my mum and dad's system will be safe for now).

Receiving samples from the vendors show which samples are in circulation and are affecting users.

Off-topic, with services like virustotal thriving, where users from all around the world are uploading samples every minute, my guess is vendors would have to be receiving samples through services such as VT. Maybe not? Well I would assume the vendors would contribute to the service (in some way) in-return to receive samples they missed. End result creates a more even playing field and a better end-product for users.

Back to Jeffrey's report, if you do compare it to other reporting (AV-C), most products do perform well and say the odd one (Kngsoft) is lagging behind.

 

Well said, I'm very happy with these results. The fact that so many AV's are scoring 90%+ shows that it's still worth having. Which defeats the "AV's are becoming useless" theories that some people propose.

 

Good job Jeffrey, keep up the good work

 

Unfortunately, the site that hosts the pdf is in chinese and can't find where to click to download it.

Yes, i am trying to find an explanation as to why all contenders are so closely clustered together , while in other tests there is more difference.

I know, you were actually the first AV lab to do dynamic test.

Please don't misunderstand me. I don't say that your methodology is any worse or better than other tests. I think it is close to impossible to make a "real life" test that represents with faithful statistics real life.

I think your test is as useful as anyother and it focuses mainly on samples that are exactly, more prevalent. Other tests include probably very and ultra rare samples.

And probably these uncommon are those that make the difference... Of course uncommon between so many contenders is something very difficult. It can miss 2,5,6 vendors, but there are much more in the test. And if you do submit those that are in users' systems, chances are they are more common than not. Many users may have uploaded them to VT themselves in the meantime.

I think this test covers more the area of "malware you 're more likely to encounter", than others. It's not something inherently bad. It is just a different sample with different characteristics.

 

Thank you for your reply Fuzzfas, if there is any misunderstanding from me , I am so sorry

For close detection rate of some vendors, I have to say that is why the main players are all developing behavioral detection techonology.

I have updated the draft picture of english report in floor 1st

 

Pcslinfo, from your perspective, how are the tests performed by PC Security Labs more informative or more comprehensive than those conducted by other organizations (e.g., AV-Comparatives or AV-Test)? What incremental insights are to be gleaned from your additional efforts?

Thank you.

 

You can call me Jeffrey hehe.
I have never said a word that my methodology is better or worse than other test labs', what I have said is to show what my test can show to the users.

To debase others to improve one's position is not a good way and polite that a mature and confident test lab should take. And of course, I will never do that. That is like, some one like Madonna, some one like ladygaga, some one like both, some one like neither. We do not need to change others' view, the only way is to do youself and do your best. That's my position.

 

The report suggests that VIPRE isn't doing well. Quite possibly the reason why they have not been tested by av-comparatives.

Two notes:
This is a Chinese organization. Where does the money come from, the Chinese government, local, central ?

The WHOIS data of www.pcsecuritylabs.net are anonymous.
I never like that, especially when it is about security.

 

VIPRE is improving and the score is higher for second time of test than his first time to take part in my test. For test issues between VIPRE and AVC, you can ask either of them.

Puzzled that you are curious about my operation of PCSL, seems that if I say I lack sponsorship, you can provide the whole operation fee for me

It is my human right to protect my personal information and also I do not think it is safe to provide personal information to everyone else.
One question, do you want to provide your credit card, your id number, your salary, where do you work and who do you serve to everyone? If you think that is nothing, then I will be glad to show the WHOIS information public.
One more word, my personal information is shared by the AV industry and most of the guys from AV vendors here know my personal information, and I think this is enough.

 

But it would help to instil confidence in those reading reports from "PC Security Labs" that this organisation has an actual registered office and contact number, as opposed to private WHOIS data.

 

Sponsorship shows who has a hand in a honeypot so to say. Like Norton sponsored A test, People had a harder time accepting that tests results because it was funded by Norton. It is a valid question if you ask me.

I believe AV Corp requires a entrance fee for the testing. So Pretty much any AV there has to pay for it hence making it less one sided.

 

I know that and I understand what fly and you mean.
I have said that I have shared my personal information(telephone, mobile, fax, office address) within the AV industry. If I do not have office, how do they deliver the product package to my office? If PCSL is visional, do you think they will cite my test result and I will have to honor to be a member of AMTSO?

I quite understand your query and what I want to say is that just take it easy

 

So you think PCSL is charging vendors to take part in the public test?

 

No. Someone has to be tho or otherwise you would not be in business still. Whether it be from a Sponsor like Norton, Eset, Avast whoever or from somewhere else. It shows if someone has a stake in the test or not.

 

So that is why I have said in my report, PCSL Total Protection Test is free not only the test, the report, but also the award logo, as long as he can reach the requirement of getting an award. PCSL will not charge any fee for vendors to take part in the public test, and that is why within one year's operation, most of the players are willing to take part in my test whether their result is already public or still in internal.

 

You still seem to be bouncing around the question. Who is FUNDING the tests. Vendor, Gov, Non Prof Organization or your old sick Grandma with to much money. Normally from testing agency this is pretty up front. It's starting to make me wonder with the dodging.

 

Can I also ask you a question.
Have you asked the other test lab how they operate their labs? The red line I have answered above.

And also what you will get if I give you the answer.

Also provide you a chinese saying: self-reliance and hard work.
FBI coming, hehe

 

Actually Yes. Other AV testers have been asked the same thing and have been very open with the community. Even Norton sponsored tests that are posted here, they are VERY open that they are the ones that sponsored the test. This cloak and dagger from a "Independent" Testing organization such as your self seems odd and really out of place. All it seems to point to to me is You don't want anyone to really know who your ether Affiliated with or to know that someone does secretly have a hand in the honeypot.

 

To clarify, it was not my intention to ask if your methodology is “better or worse” than that of another testing organization -- only to ask how it may be different. Every methodological approach has its advantages/disadvantages, since every test is an abstraction of reality by virtue of being a controlled experiment.

So, in your opinion, are there unique facets to the testing methodology used by PC Security Labs? Stated differently, has PC Security Labs advanced the discipline of anti-malware testing -- and, if so, how? (I'm just curious.)

 

If I told my test is using my own money, how much proportion do you think it is ture? If I told you I get research funds from university, how much proportion do you think it is true?

And what I get in your words is that sponsorship from AV vendors is a must to hold a public comparative test. And also please give me a method to balance if I get a sponsorship for public comparative test that I can also be independent. I know that VB and other test labs do not charge any fee for test itself, so do you think they are not independent or odd and really out of place?

I think you can ask PWC to issue an audit report and I am also willing to see the result both of mine and others'.

 

I have said a lot in my above words and I think you are also familar with the other test labs, please compare by yourself and there will be no comparation words from me.

I can also give you a link from panda blog,if you have time, you can take a look.
http://research.pandasecurity.com/panda-participates-in-new-av-comparative/

 

What caused the poor Online Armor result at the dynamic test?
No prompts or no quarantine/clean? Or wording...

Cheers

 

If your funds come from your own pocket and a university research program, then on my book, you 're the most independent tester i 've heard of.