The effect of security software on the peformance of low-end PCs

Hello,

I have written an article that examines the methodology of the Design of Experiments through a subject dear to us all - security software.

The article examines the impact of three factors - anti-virus, firewall, and RAM on the boot up time and scan time (by Spybot), at two settings, the "light" and the "heavy" one, using the statistical approach of the Design of Experiments.

This could be an interesting read for the engineers among us - but everyone is welcome.

The article is a PDF (see link on the page below), 18 pages long - mostly graphs!

http://www.dedoimedo.com/computers/doe.html

Comments and suggestions are welcome.

Cheers,
Mrk

 

Too shocking ...?
Boring?
Mrk

 

I'm trying to work at the moment - to make money - so will read more fully later.
I assume that in your tests you didn't have any "programs" installed ? The boot up times could be much greater with a full working computer ?

I stopped using AV real time on my old machines a long time ago because they just slowed things down too much - and in my case never caught anything anyway ( because there was nothing to catch)

Not to shocking nor too boring - looks like a great piece of work

 

Hello,
As I explained in the article, the only real-time software except the tested was ProcessExplorer. Other than that, the installed programs included only CCleaner, RegSupreme, ATF-Cleaner, and Spybot.
Cheers,
Mrk

 

Just read it. Nice reading, however would be more interesting to run the same tests with some more programs installed and running...
Would also be interesting to see how strong impact other suites such as Symantec's and ZoneLabs' would have on system performances. LOL
AVG has often been found to be one of the lightest antivirus out there and as far as I know Sygate is pretty light and even Comodo, while not being particularly low on CPU or MEM, doesn't slow down the system in a really noticeable way - at least on newer machines. McAfee isn't that great about impact on performances, but isn't the worst either.
Considering how many users are stuck with Norton still nowadays, chances are that real situations are likely to be even worse than what is shown in these tests...

 

I will have a look later, any real-world senarios tested ?

 

Hello,
Definitely. Real software, real world.
Mrk

 

Get alot more software and include it. Such as all these

http://www.av-comparatives.org/

Gdata Avk uses 2 engines at once. Its runs slow needs very high spec to run.

Job well done.

 

The results do underscore that detection is only one of many performance dimensions that need to be assessed in selecting products or solving the problem by other means.

An unfortunate reality is that combinations of multiple light options do not necessarily yield a light combination. Interaction terms can be important in this type of situation.

Blue

 

Hello,
I agree. I even stated in the article that there are many other possible factors, like cpu speed, hard disk speed etc. And definitely the interaction. But the goal was to demonstrate the methodology first and foremost. I leave the real testing to professionals...
Cheers,
Mrk

 

Did I miss baseline figures for machine booting without any software installed ?

 

Hello,
Nope. No baseline. That's not the goal of this experiment. The goal was - given a number of factors, which might interact, how to find, through an optimal number of runs, the maximum influence (through variances) of the given combination, something that sounds quite trivial, but for which companies pay $200/hour for a consultant to setup in their production floors.
Mrk

 

You are only measuring relative influence. Does not give any indication of significance, which is important, especially if you are calculating average values and considering error.

For example of your 25% increase in boot time, 88 v 110 (rounded).

It will make a lot of a difference if the baseline boot time is 87s vs 7s for example.
Taking 87s baseline means that the product with 110s boot time actually has 23 times (2300%) the performance hit of the other product, but taking a 7s baseline means only a 27% performance hit, there is a lot of difference 2300% v 27% increase in boot time.

 

Hello,

Looking at the problem like that, you can then treat the baseline as another level, call it the lightest, where the actual light level would be considered a centerpoint.

The DoE at hand was a 2-level full factorial. With more levels involved, it gets more complicated. I could have done a 2-level with centerpoints, I admit.

Mrk

 

Good article. I occasionally read medical research articles, and the stats are in line with those. Good read, thought provoking. I consider this another rung in my computer education.
Allen