New test from PC Security Labs

Discussion in 'other anti-virus software' started by yanzilme, Nov 10, 2008.

Thread Status:
Not open for further replies.
  1. yanzilme

    yanzilme Registered Member

    Joined:
    Mar 28, 2007
    Posts:
    13
    Last edited: Nov 10, 2008
  2. renegade08

    renegade08 Registered Member

    Joined:
    Aug 26, 2008
    Posts:
    431
    Jiangmin Antivirus 2009 (Jiangmin) ?!?!?!?!?



    ANYONE have any idea ?
     
  3. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Jiangmin is a famous security vendor in the mainland of china, here is the english homepage for him and you can check for more details:
    http://global.jiangmin.com/

    Regards:)
     
  4. xan K

    xan K Registered Member

    Joined:
    Sep 15, 2008
    Posts:
    152
    Location:
    Dominican Republic
    sorry, but that looks like a very poor test. :(
     
  5. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Can you tell me for what aspect you think is very poor?
    And have you ever read the methodology and our procedure of sorting samples?
    If not, please just look at the manual and I will be here discuss the details with you:)
    Here is our methodology
    http://www.pcsecuritylabs.net/document/PC Security Labs Manual.pdf

    And for every products' detailed information, please check our reports(we have single product report along with the summary report and all the vendors have officially agreed to take part in our testing program):
    http://www.pcsecuritylabs.net/document/PCSL-Total-Protection-Testing-Report(2008NO.11).zip
     
    Last edited: Nov 10, 2008
  6. renegade08

    renegade08 Registered Member

    Joined:
    Aug 26, 2008
    Posts:
    431
    @ pcslinfo
    I didn't meant no disregard at all.

    I was waiting @ other more experience users with AV at this forum to post for that AV particulary.

    I did visit the site before you posted the link. Thanks anyway.

    As for the testing i can say that are too little vendors included to make any judgement.
    The results are little bit too fishy. Obviously other aren't posting here because are afraid not to become this threat like SSU .

    Seeing that Kaspersky, Panda and Jiangmin have 99+ % is questionable by itself.


    As i know
    a-squared Anti-Malware 4.0 (Emsisoft) uses IKARUS engine,
    Kingsoft uses Kaspersky engine.... to be continued

    You are getting my point ?!?!?!o_O

    Like test 8 products, but actually all are using same engine.( two products= 1 engine).

    BTW what engine(or signatures) uses Jiangmin?

    Jiangmin (Man :ouch: , thay have to change the name if they want to become more widely known and used. At least to change the name for international usage.
    If i suppose they are going to that market, and seeing that they have agreed on some comparision with other(more famous) AV-s).



    They will have to consider also to give some version as free (against paid one) to attract more customers.

    Just my 0.50 cents.

    Yes GDATA, Trust port and even others are using other vendors engine, but they are multi-engine AV-s. And the test @ others testing site
    includes all AV-s or most commonly used. That means 10-15 at least.
     
  7. harlan4096

    harlan4096 Registered Member

    Joined:
    May 6, 2008
    Posts:
    113
    Location:
    Almería (Spain)
    Maybe Kingsoft uses Kaspersky engine, even F-Secure uses Kaspersky technology, but not the exact same engine of the Kaspersky own products.

    Regards.
     
  8. Frank the Perv

    Frank the Perv Banned

    Joined:
    Dec 16, 2005
    Posts:
    882
    Location:
    Virginia, USA
    This is possibly the greatest test in AV history.
     
  9. Inspector Clouseau

    Inspector Clouseau AV Expert

    Joined:
    Apr 2, 2006
    Posts:
    1,329
    Location:
    Maidenhead, UK
    1000 clean files is your FP Set? You kidding me.
    And your methodology is also wrong. Very wrong. I'm not going to explain to everyone who wants to be an antivirus tester the whole stuff from scratch. But renaming all PE32 files to .EXE is definitive wrong since they are DLL's. You have to check the Library Flag in the NTHeaders. (pointerYourNtHdr->FileHeader.Characteristics) And if you don't understand what i'm writing here please don't bother to continue AV Testing.
     
  10. Miyagi

    Miyagi Registered Member

    Joined:
    Mar 12, 2005
    Posts:
    420
    Location:
    Honolulu, Hawaii
    Most definitely AV Learning. :D
     
  11. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Kingsoft has his own engine.
     
  12. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157




    Yes, there are a little bit few AV vendors who take part in our testing platform, but... as you can see in our manual, I have explained clearly that why there are not so many AV vendors currently. We invite the AV vendors to join our testing. That means if they don't officially agree us to test their products and allow us to publish the testing result, we will not test them along with the other AV vendors who join our testing.
    To add more AV vendors into the platform is not difficult in the technical aspect, can you understand what I mean?:) I am continuely inviting new AV vendors and you will sooner find more athletes;)




    Dynamic testing means that we will run the samples static testing(on-demand testing) missing to do the real infection to the computer to test the defense ability of each AV products.
    Kaspersky, Panda and Jiangmin all have behavior-based defense moudle, so it is not strange that they get a high mark in our total protection testing.
    Probably you are referring to their on-demand scanning ability, please just make notice to the blue ones in this pic
    200811-1.JPG




    As I have explained above, we do not only test their scanning engine but also their behavior-based moudle, the OEMer and the original AV engine may definately get different performance in our testing.
    And also the different engine version and signature database(maybe) may also lead to a differerce.
    For these 8 AV vendors.
    Trend Micro (own engine)
    Kaspersky(own engine)
    Kingsoft(own engine--not kasperksy OEM)
    IKARUD(own engine)
    a2(IKARUS engine+own engine)
    Panda (own engine)
    Jiangmin(own engine)
    Filseclab(own engine)
    So please don't jump to the conclusion, friend;)



    Thank you very much for your consideration:)
     
    Last edited: Nov 11, 2008
  13. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Hi Mike,
    1000 clean files seams not a huge amount, but every month we will use 1000 fresh clean files. And the samples ever used in the testing will be uploaded to our server and AV vendors can download to add to their White list or to fix the false alarm. I will not use the samples(including malicious ones and clean ones) once had been used for our testing again for a second time.
    And I haven't find some detailed number about single clean files ever used in a single test, so what's your suggestion? 10000? 100000?:)

    For the PE files issue, please go through my methodology again and obviously you have misunderstand what my procedure is.
    The samples I collected may not always remain what they original are(some with no extension). Yes Maybe there exsit dlls and even txts. For our testing, cause we have to run the samples and test the behavior-based moudle, so the samples finally added into our Malware-List should be functional exes.

    First, we will decompress the samples and rename them all with exe extension.

    Second, we will run all the samples in the VMware and watch their behavior by hips and net sniffer.
    Dlls with .exe extension and txts with .exe extension are surely can't be executed and cause a real infection. So these samples will surely not be included into our final package which will be used to our final testing. For some samples who will detect the vmware and hide their real behavior, we will surely not add them into the final package. The only standard for the sample to be added into the final package is prevalent functional malicious file.
    For checking the Library Flag in the NTHeaders, it is definately easy to select the exes from dlls, sys, txts,etc. But, as some bad coding, there exsits the corrupted ones and someone can't execute a functional infection. And finally, I have to run it again and watch the real infection status. So we just combine these two steps into one step, that is select the functional sample with .exe extension.

    For PE format , we have also taken some methods to analysis
    Here is an example,
    sample.jpg

    BTW, I have given the account to Eric Howes to download 200809 Malware-List, which is the list package we use in this month's testing. You can check the samples and verify.

    Thank you very much for advice and I will be here discuss the any issues about testing with you.

    Any constructive suggestion is welcome:)

    Jeffrey
     
    Last edited: Nov 11, 2008
  14. EASTER

    EASTER Registered Member

    Joined:
    Jul 28, 2007
    Posts:
    5,632
    Location:
    U.S.A. (South)
    Bahhh

    That website is as dead done as a doornail. Didn't last very long.
     
  15. bellgamin

    bellgamin Very Frequent Poster

    Joined:
    Aug 1, 2002
    Posts:
    5,648
    Location:
    Hawaii
    Not at all dead on my computer & IMO. Twister looks good, as does A2.
     
  16. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,513
    Location:
    Annie's Pub
    @pcslinfo

    Please provide me with solid arguments to pay attention to your tests and to spread the word.

    TIA,

    Smokey
     
  17. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Hi Smokey,
    1) We have the ability to prove that the samples we use to test(monthly Malware-List package) are prevalent.
    At the last step of samples sorting, we will scan the samples with local multi-command line scanner(with 21scanners)http://www.pcsecuritylabs.net/news.php?readmore=8
    And then we will check the detection name in the AV vendors library such as www.viruslist.com and find the probable time AV vendors add signature to detect them and how prevalent they are. That means we will prevent adding old samples, zoo samples into malware-list package. It will cost a lot of time, but as for a testing, the quality of the testing material is very important. So I think it is worth doing that procedure.


    2) We have the ability to judge whether a sample is malicious & functional. We do not reply on the detection result of AV vendors to judge whether it is a malware or a clean file. We use HIPS to detect the behavior and also to check the functionality of a single sample. We do not use web-based sandbox analyzer either and the only way to judge is run the sample and watch the behavior. Every month I will run tens of thousands of sampleso_O


    3) We provide the samples to AV vendor to add to their database after we finish the testing.


    4) We are always glad to any constructive suggestions and we are also discussing the methodology with experts from AV vendors, individual researchers and also IT pros in forums such as WSF. We will improve our methodology with the development of the Anti malware technology:)

    Really thank you for your consideration and have a nice day:)
    Regards
    Jeffrey from PCSL
     
  18. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,513
    Location:
    Annie's Pub
    Hi Jeffrey,
    First, txs for the kind wishes:)

    I will make well matured considerations.

    Regards,

    Smokey
     
  19. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Thank you very much and I will also improve myself:)
    Yours Sincerely
    Jeffrey
     
Loading...
Thread Status:
Not open for further replies.