PCSL Testing Report 2009 NO.1

Discussion in 'other anti-virus software' started by Lawliet, Jan 15, 2009.

Thread Status:
Not open for further replies.
  1. Lawliet

    Lawliet Registered Member

    Joined:
    May 19, 2008
    Posts:
    15
    PCSL 2009 Total Protection Testing Report NO.1 is release now
    Please to our official website
    http://www.pcsecuritylabs.net/news.php?readmore=20


    And Panda Blog
    http://research.pandasecurity.com/archive/Panda-participates-in-new-AV-comparative.aspx
     
  2. firzen771

    firzen771 Registered Member

    Joined:
    Oct 29, 2007
    Posts:
    4,815
    Location:
    Canada
    Yet again, good job to the people at Avira, twister did really well as well, that was nice to see
     
  3. Coolio10

    Coolio10 Registered Member

    Joined:
    Sep 1, 2006
    Posts:
    1,124
    Wow, good job to filseclab heuristics. No fps aswell!
     
  4. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Way to go Twister, little fella! :argh:

    It does have heuristics in the on demand scan, but most is due to the FDD (behaviour blocker). Even on default settings, it's quite aggressive. It's not a surprise that it works well, it is known to score always well in these tests and to complement the signatures in a good percentage.

    The absense of false positives is pure luck. Believe me, i know! :D Just a few days ago i submitted once more the binary of Abiword that as in every new release, is detected as Trojan Zhelatin. (It's now fixed...again). So... it has no false positives... Hmm... Probably no other legitimate application was installed on the test PC. So all it could find was the real malware! :D

    In the next version there will be improvement in both engine and FDD.
     
  5. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Avira is getting boring... It spoils all the mystery being always first! :D Well, Emsisoft got the same percentage...

    Both are taking the crown off KAV's hands in most tests out there.

    On another note, what has happened to TrendMicro?! It used to be very famous. Up to a few years ago, in most motherboards they were giving PC-Cillin on a CD.
     
  6. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    Way too many 99%'s and the sample size is too small for me to take this seriously.
     
  7. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Just between me and you, i don't trust any AV test, but they 're good for product promotion. ;) So, yay Twister! :D

    There is one half-truth Panda is saying here:

    * Freshness of malware samples. Only the newest samples from the previous month are tested, not year old samples.

    "Fresh" = previous month. If they were more fresh, most wouldn't arrive at 80% most probably. 90+s are good for the vendor. It promotes a safety feeling for the customer. Year old samples are good in that too, because most likely everyone detects them, so they all start on a good basis, which is good for business. The "very fresh" are the real problem that make the difference...
     
  8. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,633
    Location:
    UK
    I'm more surprised at Panda's spelling of "kewl" in their blog.
     
  9. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    We use default setting as Filseclab suggested
    Default setting will lead to some reduction of detection rate and less false positive.:D


    Our testing is to test the comprehensive competence of a single antivirus testing. As you see, we run all the samples each scanner missed in the static testing to simulate the real infection may occur during the normal use of PC users. And we sort the sample by prevalent level and every time we use fresh samples to hold this testing. And we judge the sample malicious or not by behavior analysis to ensure the functionality of each sample:)


    Our testing is focusing on the comprehensive competence of a single antivirus testing. The very fresh samples you mentioned are used to hold the response time testing:D
     
  10. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Still, it is a miracle. Probably you didn't have a good number of legitimate applications around. :D Don't get me wrong. I am fan of Twister and i love it. But as false positives are concerned, even with default settings, well...


    I am no big fan of AV tests, but it's fine by me and i hope you will will create a reputation. The detection rates in absolute numbers, depend much on what samples you use (age is one important factor). So, the more AV tests out there, the more reliable the results can be, for the relative detection rate of AVs. Meaning i do believe Avira is 1st, because in most AV tests it is 1st. Now, the 99% is another story, depends much on sample age, malware geographic origin etc.

    So, keep up the good job! I particularly like the fact that you also do dynamic testing and pubblish in detail that result too. This way i could evaluate how good FDD is.
     
  11. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    I will try my best to perfect our testing(better methodolody, larger clean file database, more representative malware samples) and thank you for your consideration and advice:)
    Have a nice day and also Good Luck:D
     
  12. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    I am sure you will. I also enjoyed your malware links in your forum. I read you closed access to the pubblic to the malware database, i hope you will keep posting the web based malware in the forum. It's good for enthusiasts to be able to test their defenses once in a while.

    Also thanks for including Twister. You 're the only one that did so. I am fond of this underdog and most probably, being unknown and with small userbase/lifetime license, they can't afford paying the $$$ that well known magazines or tests require.
     
  13. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    As there are too many things while the new year comeo_O I will redesign the website in the Chinese Spring Festival and probably restart the web malware link at that time:)
     
  14. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    Ah, new design too! Best wishes! And, yes, please do restart the web malware link then! It's very helpful! :thumb:
     
  15. GES/POR

    GES/POR Registered Member

    Joined:
    Nov 26, 2006
    Posts:
    1,490
    Location:
    Armacham
    Can you include f-prot n prevx edge in your tests?
     
  16. Saraceno

    Saraceno Registered Member

    Joined:
    Mar 24, 2008
    Posts:
    2,404
    Nice forum too. Links to allow the testing of 'shifty stuff'! :ninja: :D

    Your testing is great as well, launching the file the same as a user would experience in the real environment. Much better than a simple right-click on a specified folder and then scan.
     
  17. pcslinfo

    pcslinfo Registered Member

    Joined:
    Jul 18, 2008
    Posts:
    157
    We invite the AV vendors to join our testing program, and after the AV vendor offically announces to participate, we then add them into the testing list. When there is new AV vendor who accept the invitation, I will post it in my homepage and let everyone knows:)
    Thank you for your consideration and have a nice day
    Regards
    Jeffrey


    I will try my best;)
    Thank you and enjoy yourself:)
     
  18. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,633
    Location:
    UK
    I don't think that's a good idea for the same reason forums like this one don't allow links to malware.
     
  19. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    This forum doesn't also allow AV A vs AV B, but it allows firewall A vs firewall B, antimalware A vs antimalware B, any other software A vs any other software B, hardware A vs hardware B etc.

    People with bad intentions know exactly where to find malware (they have dedicated sites and fora), they don't wait for PCSL to put 10 infected sites to find malware. On the other hand, PCSL is a test lab, not a forum (the forum is merely an extra) and having a few links with infected sites can be an easy way for users that don't want to go to malware sites to find some malware. You click on your own risk. It's not that they try to shove you malware down your throat.

    The other week i wanted to find malware for my own testing. I had to go to the "dark" side, in sites that i didn't know if they had some exploit for the unaware visitor. It's much safer to know that i can find a few samples from a "secure" site than wondering from one malware site to another.

    If "Pcslinfo" decides to eliminate those links too, i only hope that it can make them accessible to whoever wants to via password. In this case click-happy users that don't know what they are doing will have less to complain.
     
    Last edited: Jan 16, 2009
  20. TonyW

    TonyW Registered Member

    Joined:
    Oct 12, 2005
    Posts:
    2,633
    Location:
    UK
    If they're going to allow links to malware, that might be a better approach rather than have them openly accessible. Yes, I know sites and forums exist where such things are, but they're not freely advertised here mainly for safety reasons.
     
  21. Fuzzfas

    Fuzzfas Registered Member

    Joined:
    Jun 24, 2007
    Posts:
    2,753
    They did have a malware pool but they password protected it and made accessible for testers. Now the only "freely" accessible "malware source" are a few links in the forum, that point to infected sites. But they are quite old now, most AVs will detect them.

    I know that here such links are forbidden and it's ok. This is a forum and they have their policy. PCSL is a more particular site though, i think that a few links to infected site can be allowed to exist for those who want to test their defenses. On their own risk (put a nice warning sign).

    Or password protect that too, but not limit it to AV researchers. I prefer going to PCSL to find some malware than to malware fora or sites.
     
  22. Saraceno

    Saraceno Registered Member

    Joined:
    Mar 24, 2008
    Posts:
    2,404
    But at the same time, listing malware links shows you which sites are continually pumping out software loaded with 'problem files'. For example, lot of links to the brothersoft site. What I learned? I wouldn't probably download software from that site.
     
Loading...
Thread Status:
Not open for further replies.