New AV-Test.org malware testing (Avira finished 1st, CA eTrust finished last)

Discussion in 'other anti-virus software' started by InfinityAz, May 23, 2007.

Thread Status:
Not open for further replies.
  1. InfinityAz

    InfinityAz Registered Member

    Joined:
    Jul 23, 2005
    Posts:
    828
    Location:
    Arizona
    I just saw this on AppScout, unfortunately they don't give a link to the actual results (so if anyone knows please post). Here's the link to the text (with the actual text following it):

    http://www.appscout.com/2007/05/antivirus_shootout_in_magdebur.php#more

    AV-Test.org, based in Magdeburg, Germany, has just released results of an exhaustive malware-detection test. They threw over 600,000 malware samples at thirty-odd antivirus programs and measured how many each product detected. That's just detection - there was no effort in this test to measure whether the products could clean up the malware they found. The only real surprise to me is how well the best products did; quite a few reached 98% or even 99% detection.

    The samples shook out into four distinct categories - Trojans, worms, backdoors, and bots (zombies). Only Windows-based threats were used, and only current threats (meaning they're no older than 12 months). Overall, the products were most successful at detecting worms and bots. The range of ability in detecting Trojans was quite a bit wider.

    Limiting the list to products that we've reviewed in PC Magazine, the top scorer was Avira's Antivir, with 99% detection overall. F-Secure, Symantec, and Kaspersky all came close with 98% detection. Avast!, AVG, and BitDefender weren't far behind at 96%. From that point, though, scores start to trail off. Panda got 92%, Trend Micro got 91%, NOD32 88% and McAfee 87%. Microsoft's OneCare didn't come in dead last, as it did in an earlier test, but it only detected 81% of the threats.

    Grisoft's Ewido Anti-spyware didn't do so well, detecting only 75% - but then, it's strictly an anti-spyware product, not an antivirus. As noted, AVG (also from Grisoft) scored 96%. VirusBuster, sold both as a standalone product and as the licensed antivirus in Agnitum's new security suite, only detected 73%. And bringing up the rear, Computer Associates eTrust-VET antivirus detected just 62%.

    That's quite a range! And it would surely be a still greater spread if AV-Test had gone on to analyze how well the products removed what they detected. How well did your antivirus do?

    Posted by: Neil Rubenking (PC Mag)

    Results:
    Avira's Antivir - 99%
    F-Secure, Symantec, Kaspersky - 98%
    Avast!, AVG, BitDefender - 96%
    Panda - 92%
    Trend Micro - 91%
    NOD32 - 88%
    McAfee - 87%
    Microsoft OneCare - 81%
    Ewido Anti-spyware - 75%
    VirusBuster - 73%
    Computer Associates eTrust-VET - 62%.
     
    Last edited: May 23, 2007
  2. NAMOR

    NAMOR Registered Member

    Joined:
    May 19, 2004
    Posts:
    1,526
    Location:
    Arkham Asylum
    Wow Trend made a huge jump in detection when comparing this test to the PC world one.
     
  3. NAMOR

    NAMOR Registered Member

    Joined:
    May 19, 2004
    Posts:
    1,526
    Location:
    Arkham Asylum
    I found this link on PCMAG.COM. I'm surprised at the detection of Rising.

    http://www.pcmag.com/article2/0,1895,2135053,00.asp

    http://img.photobucket.com/albums/v219/NAMOR/PCMAG.png
     
  4. AshG

    AshG Registered Member

    Joined:
    May 7, 2005
    Posts:
    206
    Location:
    East TN
    This gives me some real food to chew on. I know there's no single uber-test that will give a definite answer as to which program is the best, and even the top tier testing sites have different rankings. The one thing I'm sure of is that NOD32 seems to be slowly sliding down the rankings. It's still at the top, but it's consistently at the bottom of the top lately and that concerns me.

    My Eset license runs out in July, so I have a month and a half to make the hard choice - Kaspersky 7.0 or Eset v3. I've happily sent Eset my money ever since I stumbled onto v1 but it just hasn't been the same since the good Inspector moved on. Gah, decisions....
     
  5. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    Now THIS is interesting. Lots of great twists to the tale this time :D

    1) AVG - 96%

    I do not have any idea why AV-test uses AVG Pro instead of Anti-Malware in all its tests (at least this has been the case in all its past tests), but this score for AVG Pro means that the competitors had better start looking out. Even AVG Free will score the same in this particular test due to absence of adware/spyware samples. :D

    2) Rising - 96.02%

    Again AV-test is in stark contrast to what we saw on AV-comparatives about Rising, even if we discount the virus and otherOS malware as well as script malware categories from the AV-comparatives test. This AV is definitely something worth tracking. :D

    But from a honest perspective, Rising has been working hard to add samples in the past few months. So, this detection rate is a good surprise to see. :)

    3) BitDefender - 95.68%

    BitDefender scored slightly less than I expected, but its still OK :p

    4) Trend Micro - 90.97%

    This AV differs from the PC World test results because the PC World test results had additional categories, mainly file viruses, polymorphic viruses, script malware etc. So speaking purely from a perspective of Trojans/Worms/Bots/BackDoors, it seems Trend is not so bad.

    5) Dr.Web - 85.84%

    :(:(:(:(:(:(:(:(:(

    6) F-Prot - 85.27%

    I don't know, this result seems very strange to me, did Marx test version 3.x again? I expected a bit better than 85% from F-Prot. :doubt:

    7) NOD32 - 88.32%

    This score is again somewhat inconsistent with AV-comparatives even if we discount the scores NOD32 achieved in the Viruses/Script malware/OtherOS malware categories :doubt:

    Overall, it is VERY NICE to see regular tests coming out from AV-test. Gives regular eye openers. I hope AV-test continues to publish tests frequently throughout the year. :)
     
  6. Jarmo P

    Jarmo P Registered Member

    Joined:
    Aug 27, 2005
    Posts:
    1,188
    Avira's AntiVir results are for the Premium edition I think, but they are consistent with most of the recent tests I have seen.
    Have not seen any tests lately for AntiVir Classic though it might detect also some malware by heuristics that are not classified as viruses&trojans.

    Same time I again got yesterday the large 1MB Avira update that did not require a reboot. And together with Sandboxie and Comodo running, Comodo started after the update telling me about invisible applications starting firefox instead Start.exe from SB, so Comodo was somehow in a mess, but reboot solved the problem.

    NOD32 is definately on the downhill ride.
     
  7. NAMOR

    NAMOR Registered Member

    Joined:
    May 19, 2004
    Posts:
    1,526
    Location:
    Arkham Asylum
    Maybe some can answer a simple question for me. How is it that some AV's tested on AV-Test.org and Av-comparatives have very similar detection % and others are completely off? If it's because they use different test beds, wouldn't we see more of a discrepancy between all of the AV in detection % instead of the 3 or so AV that have drastic differences.. You have the high-detection AV's that are comparable on both sites, then you have some mid-detection AV that are comparable, and even some lower-detection ones that are also comparable. But, then you have a few that are just all over the place in-terms of detection %'s.
     
  8. Durad

    Durad Registered Member

    Joined:
    Aug 13, 2005
    Posts:
    591
    Location:
    Canada
    For me not trustworthy test.

    We calculated once that this collector is "collecting" several malwares every 10 minutes 24/7, 365. No sleeping, eating, working, relaxing etc... :)

    He is using similar way to test detection rate as Virus.gr used. The diference is that Virus.gr is on some kind of black list while this one is not, guess why :)

    For me this looks like a mini war between antivirus vendors and their secret weapons :D
     
  9. FRug

    FRug Registered Member

    Joined:
    Feb 7, 2006
    Posts:
    309
    Durad: av-test is THE most reputable test centre out there, they employ 15 people not doing this as hobbyists but as their main job for years (av-test exists since '91, as a company since '96).
    Their test centre is running more than a hundred PCs and they have more than 60Terabyte of test data (malware/clean).

    Comparing their infrastructure and professionalism (i.E. they check whether all files in the testset are actually still executable and not dead samples) to some greek VX kiddie with an attitude problem is ridiculous.
     
  10. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    Can you explain what you mean by "attitude problem"? o_O
     
  11. The Hammer

    The Hammer Registered Member

    Joined:
    May 12, 2005
    Posts:
    5,619
    Location:
    Toronto Canada
    LOL. I don't think FRug had you in mind. NObody thinks of you as a geek VX kiddie.
     
  12. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    firecat dont forget, they are still testing 4.33

    so to come within a few percent of nod32 and mcafee and beat f-prot, its not too bad :)

    yayyy for panda

    i too wish they would test removal and all the other little details.
     
  13. EliteKiller

    EliteKiller Registered Member

    Joined:
    Jan 18, 2007
    Posts:
    1,138
    Location:
    TX
    Overall I am very impressed with AVG's performance. Why would they test Ewido when it is not an AV, and why don't they refer to it as AVGAS? McAfee has better detection than the good Dr. :p
     
  14. Firecat

    Firecat Registered Member

    Joined:
    Jan 2, 2005
    Posts:
    7,927
    Location:
    The land of no identity :D
    No, I was meaning to ask what he meant by saying VirusP had an "attitude problem". :)
     
  15. trjam

    trjam Registered Member

    Joined:
    Aug 18, 2006
    Posts:
    9,057
    Location:
    North Carolina
    Wow, Avast and Trend Micro are both better then Nod. I just dont see how they are going to turn things around. Not good at all.
    Kudos to Avira, even with some current issues to work through.
     
  16. FRug

    FRug Registered Member

    Joined:
    Feb 7, 2006
    Posts:
    309
    1) he's a VXer freely exchanging stuff with anyone who can provide him with more samples, that alone is unprofessional and as such an "attitude" problem when it comes to computer security (at least on my list)
    2) he doesn't listen to criticizm or tries to improve his testing methods. To me that also qualifies as "attitude problem"
     
  17. Blackcat

    Blackcat Registered Member

    Joined:
    Nov 22, 2002
    Posts:
    4,010
    Location:
    Christchurch, UK
    The big difference between this result and Command AV, suggests that the new version was tested.
     
  18. BlueZannetti

    BlueZannetti Administrator

    Joined:
    Oct 19, 2003
    Posts:
    6,590
    To me, there's a bit of a disconnect on the testbed numbers.

    607,000 samples, all within the past 12 months and supposedly fully functional. That means an average number (based on a 5 day workweek, no vacation days) of 607,000/260 ~ 2335 samples/day validated as functional. That's a major effort, possible, but quite large.

    The other thing that is as one might hope and expect, on average the results largely agree with AV-comparatives. There a percent or three here and there above and below, but on average most results are in-line. However, both NOD32 and F-Prot are well outside the concordance achieved with all other products examined in both tests. The disagreement of both are a little more than two standard deviations away from the mean disagreement - which would suggest that a little test tweaking might be in order - one side side or the other. The primary difference, I believe, is the relative newness (past 12 months only) of the testbed for AV-Test.org

    Blue
     
  19. MalwareDie

    MalwareDie Registered Member

    Joined:
    Dec 8, 2006
    Posts:
    500
    I am quite suspicious of the 900 000 samples one month ago and now the 600 000 samples. I do not think all the samples were new.
     
  20. Diver

    Diver Registered Member

    Joined:
    Feb 6, 2005
    Posts:
    1,444
    Location:
    Deep Underwater
    Has anyone seen a response from Eset (or one of their boosters) explaining the rather poor showing of Nod32?

    On the other hand, AVG (without anti malware I assume) had an unexpectedly good showing, compared to other tests.
     
  21. apm

    apm Registered Member

    Joined:
    Mar 15, 2006
    Posts:
    162
    Avast! 584,574 96.32% is way above Nod32 536,043 88.32% & Dr Web 520,959 85.84% this time, impresive:eek:
     
  22. Legendkiller

    Legendkiller Registered Member

    Joined:
    Jun 29, 2006
    Posts:
    1,052
    Wow,norton detected more samples then even kaspersky...:eek: :eek:

    It seems norton has sustained its good performance since the test done last year....where it faired well.;) ;)

    surprisingly nod and mcafee didn't rather well...:doubt: :doubt: ,i hope things will improve with the new versions...:rolleyes: :rolleyes:
     
  23. Legendkiller

    Legendkiller Registered Member

    Joined:
    Jun 29, 2006
    Posts:
    1,052
    and wait a minute....:ninja: :ninja:

    avast and avg have done better trend,nod,mcafee,panda........i mean...:cool: :cool:
     
  24. budfox

    budfox Registered Member

    Joined:
    Apr 5, 2005
    Posts:
    103
    Its good to see Fortinet finish high. I hate software Av's...they always seem to F'up XP kernals.


    I hardware firewall/AV/IPS has zero impact on system stability/performance. Also all computers on the network are all protected.

    If you like software av's I tried out AVK 2007 and really liked it. Make sure to turn off registry protection. It also has a really cool feature to disable realtime for x minutes (5mins to 8 hours i think). If you using a CPU intensive program, but you dont want to forget turning the reatime back on, this is a perfect thing for a AV program to have.
     
  25. C.S.J

    C.S.J Massive Poster

    Joined:
    Oct 16, 2006
    Posts:
    5,029
    Location:
    this forum is biased!
    norton is still a massive player in the av market, and with improved 2007 version, they must be laughing, as they are back on top.

    and yes avg as always, provides quality... and yes i still use the suite on the PC.
    (kaspersky licence doesnt get used, and my trend expired)
     
Loading...
Thread Status:
Not open for further replies.