PCSL Total Protection Testing (2009 May's report)

Discussion in 'other anti-virus software' started by pcslinfo, Jun 8, 2009.

Thread Status:
Not open for further replies.
  1. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
  2. Thankful

    Thankful Savings Monitor

    Joined:
    Feb 28, 2005
    Posts:
    3,744
    Location:
    New York City
    Creating a readable report would be a nice start. I can't view the results in either Firefox or I.E.
     
    Last edited: Jun 8, 2009
  3. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    Thks for the results, Jeff.

    A suggestion: Since 12 products are almost neck-n-neck for the 5 star rating. Could you add details of what malware was missed by which product. So that one can evaluate better the difference between the products.
     
  4. subset

    subset Registered Member

    Joined:
    Nov 17, 2007
    Posts:
    825
    Location:
    Austria
    There should be an explanation on the website what exactly is...
    - Detection Rate in Static Testing
    - Total Detection Rate
    - Final Score
    - Award

    But not... Marking system=(A+B)/C*100-lg(D+1) :argh:

    Cheers
     
  5. progress

    progress Guest

    A2 and the false positives - a neverending story :rolleyes:
     
  6. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    You can check that in detection rate in the static testing

    Hmmm, I will add manual in the package next time:)


    Thank you for your suggestions. Good Luck:D
     
  7. raven211

    raven211 Registered Member

    Joined:
    May 4, 2005
    Posts:
    2,567
    Hey, Panda did fair very well here (well? freakin' first :D). Would you say that it's in the top by now and that their proactive features are really improving? P-CAV will probably be quite dangerous once Beta 2 is released. :D AV-Comparatives testing of Panda (ProtectStar reports) have been different when it comes to Panda... :)
     
  8. MaB69

    MaB69 Registered Member

    Joined:
    Dec 9, 2005
    Posts:
    540
    Location:
    Paris
    Hi,

    It seems that i am the only one unable to load PCSL Website, using OpenDNS as DNS provider

    Sans titre.jpg

    I have no idea what's wrong :doubt:

    Will try later but thanks for your work

    Regards,

    MaB
     
  9. 1000db

    1000db Registered Member

    Joined:
    Jan 9, 2009
    Posts:
    718
    Location:
    Missouri
    From those results it seems to illustrate difference in detection methods. For example, Avira and A2 seem to focus mainly on fingerprints or definitions while others like Bitdefender and Twister make heavy use of dynamic (non-signature detection as I understand it) detection. BTW what's up with Twister only having 1 FP!
     
  10. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    LOL! Lately there is a reduction on Twister's FP. I myself have seen none for some months. They must be working on that. I 've also seen a reduction on the on-demand scores. My guess is, they have neglected signature adding, in order to put more manpower to the new version. As a result, the FDDS (the behavious blocker), has increased its share of detection (in the above test, the FDDS accounts for roughly 25+% of Twister's total detection!).
     
  11. Sputnik

    Sputnik Registered Member

    Joined:
    Feb 24, 2005
    Posts:
    1,198
    Location:
    Москва
    Thanks Jeffrey! :thumb: Will check out the report asap.
     
  12. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Sorry to hear that, DNS problem of your ISPo_O?

    Waiting for your suggestions:)
     
  13. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    Kaspersky was tested in automatic mode, isn't it?
     
  14. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    it is:)
     
  15. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    Hi Jeff,
    That's just a number.
    If you could display the exact malware missed. Then one can evaluate how good the protection of AV is. For ex: If ACME Anti Virus misses say conflicker.c sample then even though it has a 95% detection rate, its probably not all that good.
     
  16. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    in proactive mode you will have complete power of KIS HIPS and detection/protection will be ~100% :eek:, you should consider running it in proactive mode for ("missed") "on execution" test next time... ;)
     
  17. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    Actually proactive mode is for advanced users, auto mode is for most of the users. Normanl users actually can not just whether it is malicious or not, anyway, through the test, KIS's auto mode is good and strong enough:)
     
  18. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    You mean the exact missed samples' detailed information? But if one missed a lot, how can it be displayedo_O :rolleyes:
     
  19. Einsturzende

    Einsturzende Registered Member

    Joined:
    Apr 14, 2008
    Posts:
    390
    Location:
    neubauten
    fair enough, but should be noted there is "reserve power" in KIS and protection can be further enhanced by adjusting KIS settings :)

    EDIT:
    BTW. this test is example in which way professional tests should go, if it is possible add for next testing small amount of "malware from the source", complete vector of infection to see particular antimalware solution in real environment, e.g. from malicious url to malware execution and disinfection...
    many thanks for testing
     
    Last edited: Jun 8, 2009
  20. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    That's why they offer proactive mode for the advanced users like you:)
     
  21. vijayind

    vijayind Registered Member

    Joined:
    Aug 9, 2008
    Posts:
    1,413
    Well, I thought it wouldn't be too hard for you. Since you seem to number your malware files with unique names. And if the AV is set to scan and delete/quarantine you will only be left with undetected samples in your malware folder.

    So just see the files left and list out corresponding names. I think you could even right a script for it ( using grep ).

    At this point, I am just making assumptions. So if your load is high, if possible list the missed malware you think were important. So that a reader can differentiate between the performance of all AVs getting 95+%.
     
  22. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    well actually F-Secure should bump up a slot with 2010 and the Bitdefender engine instead of Kas 6. Along with their own engines to compliment it, looks like good times ahead.
     
  23. Pleonasm

    Pleonasm Registered Member

    Joined:
    Apr 9, 2007
    Posts:
    1,201
    Jeffrey, an idea to consider: It would be interesting to extend the analysis to pairs of anti-malware products, under the assumption that increasingly multiple security products are utilized for protection (e.g., a primary real-time anti-virus coupled with a secondary product for on-demand, manual scanning). Such an analysis would answer the question, “Which two products should I use in combination to maximize protection (and to minimize false positives)?” Obviously, not all products “play well” in combination with one another, so that practicality might limited the number the pairs examined.

    Question: What is the process by which vendors are selected for inclusion in your analyses?
     
  24. kdcdq

    kdcdq Registered Member

    Joined:
    Apr 19, 2002
    Posts:
    657
    Location:
    Southwestern Massachusetts
    Jeff, thanks for your work and I enjoyed reading your test results.

    I do have a question, though:

    Is there a reason that you did not test AVG? It would have been nice to see how AVG compared to the other AVs tested, me-thinks.

    Thanks in advance.
     
  25. Ford Prefect

    Ford Prefect Registered Member

    Joined:
    Oct 31, 2008
    Posts:
    103
    Location:
    Germany, Ruhrpott
    Hi Jeff,
    thanks for your report.
    A list of important false positive detected files would be interesting, too.
    Regards,
    Ford
     
Loading...
Thread Status:
Not open for further replies.