Simplistic 7 Step Choice Algorithm to Select Anti-Virus Products

March 9, 2007
Simplistic 7 Step Choice Algorithm to Select Anti-Virus Products
Introductory Comments
After permission was obtained on March 1, 2007 Footnote 1 and some advice from a moderator I decided to contribute this paper to try to reduce the negative effects of brand loyalty and emotional issues when our favourite AV doesn’t do as well as before as new results are published over time. I felt that human factors have tended to cause unnecessary concern, fear, defensiveness and product switching.

The choice algorithm uses the November 2006 proactive and the August 2006 independent AV Comparative test results to select top 5 Anti-Virus products as candidates for your PC. When reading this work please note the footnotes where provided.

Note as well that I have deliberately excluded the latest AV comparatives and will shortly redo the first 3 steps for you using the latest findings. For the record, I use BitDefender which ranked last in this analysis, but as it is still on the list of 5 I will continue with it.

Choice Algorithm
(1) The performance on “new” parasites is used as the first selection filter to find 5 products (plus ties). Only these products are considered further. Of course this assumes that heuristic ability should be given 1st priority.
(2) Next, the performance in on-demand tests to further filter these 5 (plus ties). This assumes that on-demand ability ranks second in priority.
(3) The scores from step 1 and 2 are added. Sort these 5 by total score (plus ties). Remove the bottom 2 products.
(4) If your existing product is included anywhere on this list relax and enjoy life and don’t bother with the remaining 4 steps. Those steps are suggested only if you don’t have any AV product or if your product has developed bad performance habits on your PC
(5) Determine required O/S, CPU, RAM and HDD, record scan speed and trial each products footprint on your PC to break ties (if any) and to uncover any unresolved compatibility issues with your system BEFORE investing $!
(6) During the trial periods, test vendor’s support group(s) with similar questions and their track response time and the quality of solutions offered
(7) Include visits to public forums to determine un-edited vendor reputation and any unknown product benefits and issues. If ties remain, use price as a final selection variable of course free wins over paid.

Completing step 1, after scanning the AV latest new malware results, the top 5 (using November 2006 proactive results) are
listed, ties included are:

1. Avira PE Edition 7............53%
1. Nod 32 AV 2.5................53%
2. Trust Port AV WS 2.0......46% Footnote 2
3. AntiVirusKit 2006............43% Footnote 2
4. BitDefender Pro 9.5.........42%
4. VAB 32 workstation 3.1...42%
5. Dr Web 4.3...................34%



Completing step 2, after scanning the AV latest results, the top 5 (using August 2006 on demand) are listed, ties from step 1
are still included:

Avira PE Edition 7.....................99.69%
Nod 32 AV 2.5.........................99.07%
Trust Port AV WS 2.0...............99.06%
AntiVirusKit 2006......................99.79%
BitDefender Pro 9.5..................96.53%
VAB 32 workstation 3.1.............82.91%
Dr Web 4.3.............................92.25%

Completing step 3, adding scores and sort from steps 1 and 2 results ties included are:

Avira PE Edition 7.....................99.69+53=152.69
Nod 32 AV 2.5.........................99.07+53=152.07
Trust Port AV WS 2.0...............99.06+46=145.06
AntiVirusKit 2006.....................99.79+43=142.79
BitDefender Pro 9.5..................96.53+42=138.53
Dr Web 4.3.............................92.25+34=126.25
VAB 32 workstation 3.1.............82.91+42=124.91

Dropping to bottom 2 the top 5 are:

Avira PE Edition 7...............152.69
Nod 32 AV 2.5...................152.07
Trust Port AV WS 2.0.........145.06
AntiVirusKit 2006...............142.79
BitDefender Pro 9.5............138.53



Complete step 4





Complete steps 5, 6 and 7



These results in the 2 tables are used to break ties or eliminate products that fail to meet your expectations for quality.

Footnote 1
Hi,there are some sites (reviews) which already do similar things: they use av-comparatives data and then summarize somehow the results to get a winner. Personally I prefer to deliver only the raw data and to let choose users their av's only after they tried them out by themself. I have nothing in contrary if data is used for such a method like you propose, but i do not want to do it by myself. atm best regards, andreas

Footnote 2
AntiVirusKit=KAV + BitDefender
TrustPort = Norman + BitDefender

 

I suggest some weighting criteria when step 1 and 2 are added. Adding different percentages doesn't compute in my head
Just constructive criticism. I'm curious now as to how AV-Comparatives does this. Checking it now.

 

erm..........

to this post

 

It seems that they just add the results without weighting (subjective by nature of course).
Still, add the results, and then make percentage, or weight the results by number of samples (category/total).
Or divide the percentages by two, but then you are giving both the same importance (you're weighting), no matter the number of samples in each category. About the same thing you got, but at least you don't get 152.69, which begs the question: 152.69 what? potatoes?

I know maybe i'm going too far for this, but i can't reason with it either
Not as is

It is a good initiative

 

Hi Someone:

Suspect you are working too hard on this simple method.

The 1st filter just uses AV's public results to find 5 candidates. The next filter retains those heuristic scores and combines them with the on demand scores. The assumption is that the better a product is ON both results in detecting parasites the higher it would rank. Not potatoes of course but parasites/malware whatever word you like. Another way you could do it is use the raw numbers of parasites AV found rather than %, that wasn't what I did but it could be done. Why not do that and report back?

I say simplistic since it is, others can create a better method, I will be an early adopter of that when it surfaces.

The point is to use a neutral method to rank and reduce the negative human factor effect.

Glad you like the initiative, others will not like it as is their privilege!

Cheers

 

Yes, it's perfectly ok Escalader. Simplicity has good qualities.
I'm not inclined to do a similar test, but maybe sometime, who knows?

I was just replying with some thoughts, my reaction to it. It doesn't invalidate it, far from it. You explained well.

 

I wonder what it would be like if you could download a spreadsheet or little applet that shows something like the entire av-comp results but with no names of the products on the columns. Next to each item or test, there could be little pulldowns or check boxes where you select the positive attribute on each of them. Then once you have gone through the whole sheet, it then displays the name of the top product for you, based on all of your selections. Of course at that point it could also show all the other product scores you didn't select through the process. If someone could create something like this then it would certainly eliminate and human bias towards or against a particular product. Kind of like a blindfolded taste test and such.

Anyhow just a thought...

 

Funny way of getting to your ultimate security solution.
Conclusion speaks for itself I think. BitDefender is doing some great things I believe.

It's a young en rather new company which services great security tools for their users. Next to theier software they are really investing in their service as you can see at their live assistance 24/7 and their great support forum 24/7


The One

 

Hi Atomic_Ed: And a good thought it is!

av agreed to my using of the "raw" results in the way I did (see footnote 1). Not my method or conclusions those are mine for good or ill. I don't know if they would agree to downloading their results into such a "blind" spread sheet or not. But I like the idea of masking them off and revealing them only at the end.

 

Thanks. It's okay with me as well. I'm unconcerned if members "invalidate" it. Flaws can be used to improve methods.

One thing I'll get too this year is some basic statistical work to see if the av on demand % differences are really statistically different when they are 99% or so, but that will be much later. If some one else does that first that's even better.

If they are statistically the same on the top 5 then I don't need on demand as a tie breaker in the choice algorithm and would need a different measure to break ties in heuristics.

Just a thought at the moment.

 

Hi One:

Well yes but please note that BD came last in the list of 5, it happens I have BD 10 at the moment BUT if when my subscription is to be renewed and it has fallen off my list I will switch to number 1. I was disappointed it was last but this 7 step choice tool does that.

That's the whole reason, to remove/reduce bias and rapid switching so BD on this current list is holding on by a thread.

 

AV-Comparatives makes at the end of the year a similar evaluation (see Summary Report of 2006) and came to the conclusion that in 2006 the AV which scored best was NOD32.

 

Thanks, it very well may come out #1 in this 7 step process as well.

Looking at the NOD32 and Avira scores, they are basically the same so there is no argument from me re NOD32 being #1 and Avira being #1.

Avira PE Edition 7.....................99.69+53=152.69
Nod 32 AV 2.5.........................99.07+53=152.07

The post went through the first 3 steps, the remaining 4 steps are to be done via testing the products on the individual users PC's.

Ties are broken by comparing the technical and vendor factors then the individual would make their own choice.

The goal was not to come up with #1 per se, but to introduce simple steps for all the reasons stated earlier. My advice is anyone with a product in the set of 5 would stand pat and not rush out to change,

 

Actually,

Avira PE Edition 7.....................99.69 + 53.49 = 153.18
Nod 32 AV 2.5.........................99.07 + 53.09 = 152.16

Best regards,
Firefighter!

Btw, and

Kaspersky................................99.45 + 99 = 198.45 (with PDM)

 

That is all very interesting. However, the most important things are the GUI and any bugs that really affect you. They are all fine AVs. The difficult part is finding one where you like the GUI and as you use it some nasty bug doesn't crop up (like for me now with Avira and the constant access every 60 seconds by Avira to my floppy drive after I burn a CD using Nero). I also liked KAV's GUI (not as much though as avira's and BD's) but then it went and messed with chkdsk after several months of use. NOD32 has a lousy GUI (IMO) so I don't consider it anymore mostly for that reason as I had considerable irritation and confusion when I used it for two years because of the non-intuitive GUI. BitDefender cannot be used with Spybot and Spyware Blaster so I can't use it. Since they are all good AVs, as is the new Norton 360 (but it has no right click scan ability), your tests are not very relevant because you end up using whatever AV you find that actually works properly on your computer and has a GUI that you like. I really liked the new F-Prot until I found that it crashed every time I tried to scan a folder I have of 300+ viruses and F-Prot support could not reproduce the problem. I think a person is very lucky these days if they can find an AV that works right on their computer with their applications and where they actually like the GUI.

 

Mele20:

Actually I use BitDefender and Spyware Blaster without difficultly. So maybe you have clashing shields at work with SpyBot.

I read your post and agree that you need an AV that works on your PC.
That is what the remaining 4 steps are for to trial the top products on your machine and then take the first one that runs properly. GUI yes, but it is a cosmetic feature IMHO to help us drive the AV car we pick.

Just curious what AV and ASW do you actually use now for active protection?

 

I was using BD Pro 9 and then was auto updated to 9.5 which had the new anti-spyware scanner and I immediately got a BSOD on boot caused by the BD driver. Up until that forced upgrade to 9.5, I had no problems with BD and Spyware Blaster and Spybot (no TeaTimer running) working together. When I sought help for the BSOD, BD eventually asked for a Belarc Advisor to be run and the results sent to them. They saw I had Spyware Blaster and Spybot and told me that I had to uninstall them as they were not compatible with BD 9.5. I uninstalled BD instead and I really liked BD before the problems. Support never said anything about did I have those two antispyware programs or that I should uninstall them until they requested the Advisor report and latched on those two as the cause of the BSOD on boot. It was stupid because the BSOD is in their knowledge base as a known problem with earlier versions of BD and when I got the BSOD with 9.5, I think they just didn't want to believe it was their driver causing the problem because they thought they had fixed that much earlier so they decided it was the two antispyware programs causing the problem. Javacool was really puzzled by what they said and Spybot forum just shrugged their shoulders and said that BD had been hostile to Spybot for a long while. BD support refused any further support until I removed the "offending" programs so I got disgusted and removed BD instead.

As for what I am using now, I found that after using Nero if I reboot that stops Avira from accessing the floppy drive every 60 seconds. I don't like to have reboot ...it is a hassle especially since I use VMWare and have to shut down virtual machines before the main one and I have Ping Plotter Pro running to several targets all the time so I have save the data and shut it down...but since I am not currently using Nero frequently....I can live with this work around. I hope Avira will take this seriously and fix it within the next few months. I was going to purchase Avira but I won't do that unless they fix this. I really like Avira so if I have no further problems with it and the floppy drive ...just the need to reboot after using Nero...I can live with that considering I'm using the free version of avira. But my floppy drive eject button is frozen so when I get another drive and actually use it...avira may have more problems that rebooting won't fix...but I hope not.

 

Hi Escalader

I know it became 5 on this list but 5 in the list of all providers it isn't that bad.
Also you are talking about 9.5 in your list. The upgrade to 10 was a really great one so it would have done a bit better.
Your list is a list of several great anti virusses so 5 isn't that bad.

I'm very curious what NOD32 will be doing the next couple of months. I think their competitors are going fast in improval and I believe that NOD32 is standing still right now.

Maybe they are working hard on v3 ( I hope)


The One

 

Hi One:

Yes, I also am on BD 10 and working fine. It is beginning to look like a suite (are they inevitable?).

BD 10 has:

1. AV and ASW scanning functions
2. Privacy Protection (ZA Pro has this but I haven't compared yet)
3. Registry Control ( like SpySweeper)
4. Dialer
5. Cookie control
6. Script
7. Email IN/OUT scanning

Before month ends, I will publish my updated Simplistic 7 Step Choice Algorithm to Select Anti-Virus Products using the latest AV comparatives.

Some changes I'm considering based on member feedback are:

(1) using numbers of parasites detected rather than %. This may avoid the rounding issues of 2 decimal places.

(2) include a wider list of products at step1 filter, thinking of including any AV comparative proactive product that ever reached Advanced + in the last 3 proactive reports. If say BD fell to advanced from advanced + in one report that would be insufficient to exclude it.

 

I think it is rather strange that nobody mentions the fact that

TrustPort = Norman + BitDefender => was like that in the past.

Now Trustport Workstation = Norman + AVG + Ewido (spyware) + BitDefender.

YES Trustport has 4 engines now

So why buy BitDefender? I would prefer TrustPort WITH BitDefender.
The latest AV-Comparatives (Comparatives Feb 2007)
shows that TrustPort has better test results then:
Mcafee, NOD32, Symantec Norton, Avast and AVG.

I must say that after i switched to TrustPort i was very impressed,
with the things it found that Kaspersky left behind on my machine.
SO i am a licensed user now.
It also has a very good Firewall AntiSpam and other goodies.

And the performance impact on my system is not so bad.
Is there anyone else here that has tested Trustport already ?

 

It does look good. How much does it cost for one computer.

 

I am highly impressed with TrustPort, but couldn't find any link to purchase the product. If you can show me where I can buy it, maybe

 

TrustPort would be awfully slow at scanning though.

 

I think trustport is not sold to individuals. It has rather a sophos type selling method, only through distributer. I may be wrong .

 

http://www.aec.cz/index.php?id=198,0,0,1,0,0
You have to register and download a trial. It´s shareware I think.