New test from PC Security Labs

The Total Protection Testing includes Signature/Cloud/Dynamic testing.
Dynamic is based on behavior block technology.

http://www.pcsecuritylabs.net/news.php?readmore=12

 

Jiangmin Antivirus 2009 (Jiangmin) ?!?!?!?!?



ANYONE have any idea ?

 

Jiangmin is a famous security vendor in the mainland of china, here is the english homepage for him and you can check for more details:
http://global.jiangmin.com/

Regards

 

sorry, but that looks like a very poor test.

 

Can you tell me for what aspect you think is very poor?
And have you ever read the methodology and our procedure of sorting samples?
If not, please just look at the manual and I will be here discuss the details with you
Here is our methodology
http://www.pcsecuritylabs.net/document/PC Security Labs Manual.pdf

And for every products' detailed information, please check our reports(we have single product report along with the summary report and all the vendors have officially agreed to take part in our testing program):
http://www.pcsecuritylabs.net/document/PCSL-Total-Protection-Testing-Report(2008NO.11).zip

 

@ pcslinfo
I didn't meant no disregard at all.

I was waiting @ other more experience users with AV at this forum to post for that AV particulary.

I did visit the site before you posted the link. Thanks anyway.

As for the testing i can say that are too little vendors included to make any judgement.
The results are little bit too fishy. Obviously other aren't posting here because are afraid not to become this threat like SSU .

Seeing that Kaspersky, Panda and Jiangmin have 99+ % is questionable by itself.


As i know
a-squared Anti-Malware 4.0 (Emsisoft) uses IKARUS engine,
Kingsoft uses Kaspersky engine.... to be continued

You are getting my point ?!?!?!

Like test 8 products, but actually all are using same engine.( two products= 1 engine).

BTW what engine(or signatures) uses Jiangmin?

Jiangmin (Man , thay have to change the name if they want to become more widely known and used. At least to change the name for international usage.
If i suppose they are going to that market, and seeing that they have agreed on some comparision with other(more famous) AV-s).



They will have to consider also to give some version as free (against paid one) to attract more customers.

Just my 0.50 cents.

Yes GDATA, Trust port and even others are using other vendors engine, but they are multi-engine AV-s. And the test @ others testing site
includes all AV-s or most commonly used. That means 10-15 at least.

 

Maybe Kingsoft uses Kaspersky engine, even F-Secure uses Kaspersky technology, but not the exact same engine of the Kaspersky own products.

Regards.

 

This is possibly the greatest test in AV history.

 

1000 clean files is your FP Set? You kidding me.
And your methodology is also wrong. Very wrong. I'm not going to explain to everyone who wants to be an antivirus tester the whole stuff from scratch. But renaming all PE32 files to .EXE is definitive wrong since they are DLL's. You have to check the Library Flag in the NTHeaders. (pointerYourNtHdr->FileHeader.Characteristics) And if you don't understand what i'm writing here please don't bother to continue AV Testing.

 

Most definitely AV Learning.

 

Kingsoft has his own engine.

 

Yes, there are a little bit few AV vendors who take part in our testing platform, but... as you can see in our manual, I have explained clearly that why there are not so many AV vendors currently. We invite the AV vendors to join our testing. That means if they don't officially agree us to test their products and allow us to publish the testing result, we will not test them along with the other AV vendors who join our testing.
To add more AV vendors into the platform is not difficult in the technical aspect, can you understand what I mean? I am continuely inviting new AV vendors and you will sooner find more athletes




Dynamic testing means that we will run the samples static testing(on-demand testing) missing to do the real infection to the computer to test the defense ability of each AV products.
Kaspersky, Panda and Jiangmin all have behavior-based defense moudle, so it is not strange that they get a high mark in our total protection testing.
Probably you are referring to their on-demand scanning ability, please just make notice to the blue ones in this pic





As I have explained above, we do not only test their scanning engine but also their behavior-based moudle, the OEMer and the original AV engine may definately get different performance in our testing.
And also the different engine version and signature database(maybe) may also lead to a differerce.
For these 8 AV vendors.
Trend Micro (own engine)
Kaspersky(own engine)
Kingsoft(own engine--not kasperksy OEM)
IKARUD(own engine)
a2(IKARUS engine+own engine)
Panda (own engine)
Jiangmin(own engine)
Filseclab(own engine)
So please don't jump to the conclusion, friend



Thank you very much for your consideration

 

Hi Mike,
1000 clean files seams not a huge amount, but every month we will use 1000 fresh clean files. And the samples ever used in the testing will be uploaded to our server and AV vendors can download to add to their White list or to fix the false alarm. I will not use the samples(including malicious ones and clean ones) once had been used for our testing again for a second time.
And I haven't find some detailed number about single clean files ever used in a single test, so what's your suggestion? 10000? 100000?

For the PE files issue, please go through my methodology again and obviously you have misunderstand what my procedure is.
The samples I collected may not always remain what they original are(some with no extension). Yes Maybe there exsit dlls and even txts. For our testing, cause we have to run the samples and test the behavior-based moudle, so the samples finally added into our Malware-List should be functional exes.

First, we will decompress the samples and rename them all with exe extension.

Second, we will run all the samples in the VMware and watch their behavior by hips and net sniffer.
Dlls with .exe extension and txts with .exe extension are surely can't be executed and cause a real infection. So these samples will surely not be included into our final package which will be used to our final testing. For some samples who will detect the vmware and hide their real behavior, we will surely not add them into the final package. The only standard for the sample to be added into the final package is prevalent functional malicious file.
For checking the Library Flag in the NTHeaders, it is definately easy to select the exes from dlls, sys, txts,etc. But, as some bad coding, there exsits the corrupted ones and someone can't execute a functional infection. And finally, I have to run it again and watch the real infection status. So we just combine these two steps into one step, that is select the functional sample with .exe extension.

For PE format , we have also taken some methods to analysis
Here is an example,


BTW, I have given the account to Eric Howes to download 200809 Malware-List, which is the list package we use in this month's testing. You can check the samples and verify.

Thank you very much for advice and I will be here discuss the any issues about testing with you.

Any constructive suggestion is welcome

Jeffrey

 

Bahhh

That website is as dead done as a doornail. Didn't last very long.

 

Not at all dead on my computer & IMO. Twister looks good, as does A2.

 

@pcslinfo

Please provide me with solid arguments to pay attention to your tests and to spread the word.

TIA,

Smokey

 

Hi Smokey,
1) We have the ability to prove that the samples we use to test(monthly Malware-List package) are prevalent.
At the last step of samples sorting, we will scan the samples with local multi-command line scanner(with 21scanners)http://www.pcsecuritylabs.net/news.php?readmore=8
And then we will check the detection name in the AV vendors library such as www.viruslist.com and find the probable time AV vendors add signature to detect them and how prevalent they are. That means we will prevent adding old samples, zoo samples into malware-list package. It will cost a lot of time, but as for a testing, the quality of the testing material is very important. So I think it is worth doing that procedure.


2) We have the ability to judge whether a sample is malicious & functional. We do not reply on the detection result of AV vendors to judge whether it is a malware or a clean file. We use HIPS to detect the behavior and also to check the functionality of a single sample. We do not use web-based sandbox analyzer either and the only way to judge is run the sample and watch the behavior. Every month I will run tens of thousands of samples


3) We provide the samples to AV vendor to add to their database after we finish the testing.


4) We are always glad to any constructive suggestions and we are also discussing the methodology with experts from AV vendors, individual researchers and also IT pros in forums such as WSF. We will improve our methodology with the development of the Anti malware technology

Really thank you for your consideration and have a nice day
Regards
Jeffrey from PCSL

 

Hi Jeffrey,

First, txs for the kind wishes

I will make well matured considerations.

Regards,

Smokey

 

Thank you very much and I will also improve myself
Yours Sincerely
Jeffrey