c't magazine 1/2005 AV test

Discussion in 'other anti-virus software' started by halcyon, Dec 28, 2004.

Thread Status:
Not open for further replies.
  1. halcyon

    halcyon Registered Member

    Joined:
    May 14, 2003
    Posts:
    373
    The respected German computer magazine c't just published a test roundup of 16 AV scanners in their 1/2005 issue. The test were done in co-operation with AV-Test.de/University of Magdeburg.

    The article is way too big for me to go into details here. Nor does my sorely lacking German skills help the issue :)

    I'll just summarise that they did test ITW (on-demand and on-access), Backdoors, Trojans, heuristics with old signature base and modified ITW viruses.

    For me, the surprisingly good contenders were:

    - Bidefender Free 7.2: very good in ITW viri and backdoors/trojans (as good as Bitdefender 8 for-pay)
    - AVG Free edition 7 (as above, but much worse with trojans)

    Both had quite many false positives (8 and 11 respectively), compared to others.

    Of course, KAV, F-Secure, McAfee and AVK (G Data) were "up there" as well.

    NOD-32 didn't quite measure up as well in the backdoors/trojans test. It even lost to Avast, but with better heuristics.

    I think I'll consider Bitdefender Free now as my backup scanner of choice, if I can configure it to be on-demand only with light demands.


    regards,
    halcyon

    NB! The above is based on my limited understanding after having read the article (basically with a dictionary in hand). If you know the article and speak fluent German, please chime in.
     
  2. RejZoR

    RejZoR Registered Member

    Joined:
    May 31, 2004
    Posts:
    6,426
    What was this test,a paper magazine article or article on web? If it's on the web you should paste the link...
     
  3. bigbuck

    bigbuck Registered Member

    Joined:
    Jul 7, 2004
    Posts:
    4,877
    Location:
    Qld, Aus
    RejZoR, I think it's a German site..... http://www.heise.de/ct/
     
  4. nadirah

    nadirah Registered Member

    Joined:
    Oct 14, 2003
    Posts:
    3,647
    I don't understand German language.
     
  5. dan_maran

    dan_maran Registered Member

    Joined:
    Aug 30, 2004
    Posts:
    1,053
    Location:
    Stamford, CT
    I cannot find a link to the article from the website so I am assuming it is only in the print edition. It is on the PAge posted above about a third of the way down.

    Beware crappy translator,
    "Attack goal PC
    Viruses and worms become always more refined and mix themselves
    increasingly with Spyware, what the Antiviren-manufacturers places before always new challenges. Next to one viruses scanners correctly configured help preventive precaution to prevent the worst.

    • - The state of the things in the viruses warehouse
      - 16 Viruses scanners in the comparison
      - Precaution for the PC
    "
     
  6. tazdevl

    tazdevl Registered Member

    Joined:
    May 17, 2004
    Posts:
    837
    Location:
    AZ, USA
    Use google language tools... does a better job translating.

    Linky
     
  7. halcyon

    halcyon Registered Member

    Joined:
    May 14, 2003
    Posts:
    373
    Yes, only on print.

    Very interesting test.

    I recommend you pick up a copy of this magazine, if you are interested and can find this magazine.

    This is just FYI.

    I don't have neither the time nor the inclination to start translating and copying the multi-page article here :)
     
  8. nameless

    nameless Registered Member

    Joined:
    Feb 23, 2003
    Posts:
    1,184
    What's this--the billionth source to issue such a statement? Yet still the NOD32 fans dispute or dismiss it?
    -
     
  9. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,513
    Location:
    Annie's Pub
    Without a link to the full article, it's impossible to read it and make any conclusions, and for sure not any judgemet at all.

    Maybe to you everything is clear, but not to me.


    http://ladyinblack.com/animals/Smokey.gif
     
  10. halcyon

    halcyon Registered Member

    Joined:
    May 14, 2003
    Posts:
    373
    I feel for you, but that's what I can do.

    At least we know about the test now and you know some of the results.

    Without me posting, you wouldn't even know that :)

    I'm sure one of our German members will chime in sooner or later and comment more on the review.
     
  11. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,513
    Location:
    Annie's Pub
    That would be nice!;)


    Ciao,

    [GLOW="gray"] Smokey[/GLOW]
     
  12. halcyon

    halcyon Registered Member

    Joined:
    May 14, 2003
    Posts:
    373
    Quick partial summary of the free ones + KAV & NOD32.


    Code:
    Software                ITW OD/OA   Backd./Troj.    Heur3/Heur6/Modif.
    ----------------------------------------------------------------------
    Avast Home 4.1.418      100%/98,6%  84,6%/83,3%     8,4%/5,5%/54,8%
    
    AntiVir PE 6.28         100%/100%   66,2%/56,2%     7,5%/5,7%/32,30%
    
    AVG Free 7.0.289        100%/100%   91,4%/31,9%     6,9%/3,8%/32,3%
    
    BitDefender Free 7.2    100%/n/a    99,7%/99,1%     23,5%/14,4%/100%
    
    F-Prot DOS 3.15b        100%/n/a    91,1%/87,6%     9,8%/6,4%/80,6%
    
    Kaspersky Personal Pro  100%/100%   99,7%/98,7%     19,7%/13,4%/48,4%
    
    NOD32 2.0 (1.906)       100%/100%   82,4%/70,2%     45,5%/33,1%/96,8%
    
    LEGEND:
    ITW OD = On-demand scan In-the-wild viri (recognition)
    ITW OA = On-access scan In-the-wild viri (recognition)
    Backd. = Backdoors (recognition)
    Troj. = Trojans (recognition)
    Heur3 = heuristic with 3 month old signatures (recognition)
    Heur5 = heuristic with 6 month old signatures (recognition)
    Modif. = Virus modifications (recognition)


    The above is just a small sampling from the test. There are many more test
    (speed, packers, false-positives, etc.). Also there were many other software
    tested: Avast Pro, AntiVir Pro, AVK, Bitdef Pro, F-Secure, McAfee, Norton,
    PC-cillin and VirBot.

    Please don't ask me to type more. It's copyrighted by c't and I think the
    above should be within quotation limits and enable initial discussion on the
    test itself.
     
  13. RejZoR

    RejZoR Registered Member

    Joined:
    May 31, 2004
    Posts:
    6,426
    Hm,very interesting score of avast!, especially for backdoors and trojans.
    NOD32 had the highest heuristics score as expected.
    But as i can see,very old versions were used for these tests. Thats very odd.
    avast! 4.5 is out for quiet some time now and it's much better than 4.1.
    NOD32 also had many improvements lately (trojans)
     
    Last edited: Jan 1, 2005
  14. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    A bit of curious. How many BackDoor, Trojan, TrojanDownloader, TrojanDropper, TrojanSpy, Worm, Win32 viruses vere tested?

    Best regards,
    Firefighter!
     
  15. Ianb

    Ianb Registered Member

    Joined:
    Nov 26, 2004
    Posts:
    232
    Location:
    UK
    Interesting indeed. I can't see how Avast (much as I like it) did so well on Backdoor\Trojans, it never does well on these type of files at Jotti.

    Also, Avast Home only has heuristics for email (as I remember) :doubt: Did they test the heuristics by email o_O
     
  16. Acadia

    Acadia Registered Member

    Joined:
    Sep 8, 2002
    Posts:
    4,048
    Location:
    SouthCentral PA
    Please, don't laugh, how did Norton do? :oops:

    Acadia
     
  17. Smokey

    Smokey Registered Member

    Joined:
    Apr 1, 2002
    Posts:
    1,513
    Location:
    Annie's Pub
    O.k., I don' laugh, I cry:'( !

    As I said before, without producing here the full test it's impossible to get a correct view and judgement about the test and it's results.

    So there is still hope for you and Norton!;)


    Ciao,

    [GLOW="gray"] Smokey[/GLOW]
     
  18. _anvil

    _anvil Registered Member

    Joined:
    Jun 18, 2003
    Posts:
    56
    @Firefighter
    They did some usual ITW testing (mostly worms and some virii, as you know) and a backdoor/trojan test. In the backdoor/trojan testset, there were about 60,000 samples. I don't know, if there is still much "crap" in Marx' backdoor/trojan testset (clients, server editors, ...) :rolleyes:

    I also have question to you: is your own testset "clean", i.e. free of harmless clients, editors etc. (which are usually included in a trojan package)? I ask, because I read elsewhere, that TH did very poor in your test, which could theoretically be due to the fact, that TH generally doesn't detect those things (unlike most other AV/ATs).
     
  19. uni-hamburg

    uni-hamburg Guest

    Just another payed for/add test - as there are meany popping up. Hire a tester/testing company - define your goal - get the results wanted - publish. This has been not that uncommon, especially in regard to Mr. Andreas Marx, who actually cashed in for the job as it seems. No bad intend: we all have to earn a living.

    My advice: better ignore sponsored tests - and this one surely fits in. Better stay tuned on independend test an testers, such as Andreas Clementi running the trustworthy non-commercial 'av-compatitives' site. Commercial strings attached = providing the results payed for. It's as simple as that.

    Ignore - for the benefit of all.
     
  20. Edwin024

    Edwin024 Registered Member

    Joined:
    Nov 14, 2004
    Posts:
    1,000
    I totally agree with you, mate!
     
  21. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    I'm not trying to hijack this thread but when I scanned BitDefender 7.2 Free against my 3518 infected archived samples, BDF 7.2 scored far away that 99+ % against all trojan like nasties. Propably I was able to pick very odd backdoor and trojan samples or how many there really were samples like that in this test?

    PS. I hate 100 % detecting rates, they are always skewed, those samples have to be straight from some av-vendor's database, my first rule of testing.
     

    Attached Files:

    Last edited: Jan 1, 2005
  22. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    I'm 100 % sure that my trojan like samples aren't 100 % clean as you wrote, but if a Kaspersky engined av scored over 99 % and some other about 90 % and Ewido 92.1 %, that's enough for me.

    When TrojanHunter scored only 175 detectings from my 3518 infected archived samples and the McAfee & TH combo scored only 6 more than the McAfee only, I think that it isn't my fault. Those samples that I have during my testing were inside WinRAR 3.40 archives packed by "zip" format. Maybe that's why TH detected so poor.

    In my mind TH is VERY good AntiTrojan, but if I launch each 3518 file one after an other, I'm sure that TH scores much better. Because of this, you understand, why I just can't do that.

    Best regards,
    Firefighter!
     
  23. RejZoR

    RejZoR Registered Member

    Joined:
    May 31, 2004
    Posts:
    6,426
    Yes,avast! uses heuristics only in Internet Mail provider (for POP3).
    Thats why i was even more amazed since avast! picked them only by generic signatures (mainly Win32:Trojan-gen). Not bad for AV without true heuristics.
    But you can see how NOD32 performs with probably the strongest heuristics along with Panda's TruPrevent and Norman's Sandbox.
    Score is skyrocket higher...
     
  24. Ianb

    Ianb Registered Member

    Joined:
    Nov 26, 2004
    Posts:
    232
    Location:
    UK
    I agree, that is impressive.

    Any chance you can test Avast Home (with and without Ewido) on your samples Firefighter ? Also latest KAV Home version, it would be interesting to see how the newer engine compares to the old escan one. Pleeeeeeeasee ;)
     
  25. Firefighter

    Firefighter Registered Member

    Joined:
    Oct 28, 2002
    Posts:
    1,670
    Location:
    Finland
    Apparently you have not seen these, posts 1. 17. ad 19.

    https://www.wilderssecurity.com/showthread.php?t=58597&page=1

    PS. Today Ewido scores 2452, so I believe that Avast & Ewido combo is able to beat McAfee VSE 8.0i. That newest Kaspersky Home I just don't want to test because eScan Free 4.7.6 uses those super secure definitions of KAV (_x in the end of URL:s). Before there were no differencies to mention, only samples larger than 1 Megs were not detected by eScan and I have only two of them.

    Best regards,
    Firefighter!
     
    Last edited: Jan 1, 2005
Loading...
Thread Status:
Not open for further replies.