VMWare, Virtual PC, Parallels, VirtualBox Comparison

[charincol]

I have done some basic comparing and benchmarking of all 4 products. The versions used for this comparison are:
VMWare Workstation 5.5.3
Microsoft Virtual PC 2007
Parallels Workstation 2.2.2112
VirtualBox 1.3.6

My PC specs are:
AMD Athlon XP 2900 2 GHz
1 GHz DDR SRAM in Dual Channel mode
128 MB GeForce 5200 AGP Video
80 GB IBM/Hitachi Sata Drive (Fastest Drive - OS and Apps reside on it)
160 GB WD Sata Drive (Virtual Images reside on it)
40 GB Seagate EIDE
40 GB WD EIDE

My OS is Win XP Pro that has a majority of the pork stripped out of it with nLite. It is completely functional for everything I need/want to use it for. All startup apps/programs and services not needed to run the VM's were turned off. (Wasn't worried about my temporory lack of OS security, I'm behind a FreeBSD firewall box.) Task manager shows 13 processes running using around 79 MB after boot. (Normal boot with all things enabled is around 143 MB.)

My virtual machine OS is also Win XP Pro. But it's an even more stripped version that's still very functional for most tasks. It installed all 4 times and detected all virtual hardware without a hiccup. It takes up about 450 MB on the virtual disk without the pagefile. Task manager on each VM showed 12 processes using around 44 MB on boot. I installed each program's Virtual Tools that help integrate the mouse, keyboard, and display nicely into the desktop. I then installed PerfectDisk and Crap Cleaner to optimize the 1.5 GB virtual disk in each VM. The virtual drive files are on the same 20 GB partition and were completely defragmented before starting. There is 128 MB RAM "installed" in each VM. I was able to use bridged networking on all except VirtualBox.

I used Fresh Diagnose to benchmark the "important" hardware components of each VM. I was not able to test networking on Parallels or VirtualBox. The VM in Parallels was able to see my LAN shares just fine, Fresh Diagnose wouldn't. VirtualBox was the only one I couldn't get bridged networking to work on. Every time I tried to create a virtual NIC that would allow bridged networking, VirtualBox would lock up. The interface would lock up sometimes when doing a custom resize of the virtual desktop. Also, the desktop size sometimes reverted to 640x480 when rebooting the VM. VirtualBox works fine with NAT networking but that doesn't let the OS see LAN shares. I'm sure that these bugs will dissappear as VirtualBox matures.

I ran each benchmark on each VM 5 times and then averaged the results. Higher numbers are better on every test. The display benchmark does not indicate anything other than a comparison score. The display scores are based on 7 different types on image speed tests that I combined to get the comparison. They accurately reflect relative display performance and nothing more.

All VM's were configured with default hardware settings
All VM's have a virtual HDD of 1.5 GB and 128 MB RAM and pagefile of 200 MB
All RAM usage was done with Windows Task Manager
Highest score is in Blue, Lowest is in Red

VMWare___is in Column 1
Virtual PC__is in Column 2
Parallels___is in Column 3
VirtualBox__is in Column 4


____________________________________1__________2__________3__________4

RESOURCE SPECS
Program Installer Size (MB)_____________92_________30_________19_________13

Program Installed Size (MB)____________105_________35_________24_________24

Program RAM usage (MB)_______________17_________23__________8_________26

w/ VM Running (MB)___________________61________175_________53________231

XP VM Boot Time (Seconds)____________25_________26_________25_________18

XP VM RAM usage (MB)________________44_________44_________44_________44

w/ Tools Installed_____________________48_________49_________47_________47

VM Tools RAM usage (MB)_______________4__________5__________3________1.7

VM Tools # of proccesses_______________2__________3__________1__________1

CPU BENCHMARKS
CPU Dhrystone (MBIPS)______________5812_______5553_______5603_______5682

CPU Whetstone (MWIPS)_____________4224_______3839_______3910_______4023

CPU Speed (MHz)___________________2002_______1911_______2010_______2001

CPU Multimedia Extensions Score______2191_______1317_______1421_______2249

MEMORY BENCHMARKS
Integer Assignment Score___________17781______16587______16199______16671

Real Assignment Score______________18467______16649______16335______16719

Integer Split Score_________________20686______19867______19439______19885

Real Split Score____________________24221______20752______22714______23440

DISPLAY BENCHMARK
Default Video RAM (MB)________________16__________8_________16__________8

(Score was computed from 7________137007_____121175_____142031______68250
categories of how fast the
virtual adapter can draw
different shapes per second)

HARD DISK BENCHMARKS
Write (MB/s)________________________20.52______19.10______18.02______18.78

Read (MB/s)________________________22.79______21.89______15.88______24.43

NETWORK BENCHMARKS
Write (MB/s)_________________________9.80_______9.96___Not tested__Not tested

Read (MB/s)________________________27.52______25.26___Not tested__Not tested

Just some obvious results:

VMWare has best all-around performance (no surprise).
Virtual PC has worst CPU performance.
VirtualBox has solid all-around VM performance, but had the most bugs on my system. (There are lots of people with no problems using WinXP as a guest on Windows or Linux.)
Parallels has worst RAM performance.

Here are some other things to help you compare them.

Virtual PC and Parallels video adapter RAM can't be increased. Virtual PC video RAM can be decreased, however. VMWare and VirtualBox support up to 128 MB video RAM. VMWare has beta support for Direct3D. Virtual PC emulates an S3 Trio which DOS-based games like a lot (more of them recognize it over the other virtual adapters.) I increased VirtualBox video RAM to 16 MB and it didn't do anything to increase video performance.

VMWare and Virtual PC allow you to create a base VM and then create multiple differing snapshots from it. This allows you to have mutiple VM's at a fraction of the hard drive space. Both also have the ability to start the VM in a "mode" that allows all changes to be discarded when the VM is powered off (great for malware or untrusted software testing.) VirtualBox also has similar features but it requires in-depth study to figure them out.

Parallels and Virtual PC have excellent guest OS support for OS/2 Warp (which isn't much to brag about these days, but it still has a following.)

All of the services from any VM program installed into the host Windows OS can be changed from Auto to Manual startup and the programs will run fine. I had no issues from turning them off.

Virtual PC and VirtualBox both completely unload from host RAM when they are closed. Both are also free.

VMWare has the best networking and host-to-guest USB and printer support.


I was able to run all 4 virtual machines and open www.google.com (http://www.google.com) in IE6 on all of them at the same time. I just had to disable floppy and CD drive access to boot them all. There was a noticeable lag in OS functions but it was not significant.

VirtualBox 1.3.8 was released 2 days ago. I haven't tried it yet.


Here's some things about another very good Virtual Machine Product called Qemu (which VirtualBox is based on.)

It is free and open-source like VirtualBox.
It will allow you to run an OS (even Windows) from a USB drive.
It has an Undo Disk capability.
I have managed to use it to install Windows XP Pro x64 as a guest on my Windows XP Pro x32 host (but it runs slow, takes like 5 minutes to boot.)
However, it is not for the newbie or casual VM user. I did not test it here because it does not have any VM Tools.

[lucas1985]

I agree with your findings. In my case, VirtualBox seems to perform better with faster hardware (dual-core) than VMware. But VMware has the best USB/printer/network support as you said.

[Peter2150]

Interesting test. Another interesting comparison might be how well the handle the roll back out of bad stuff.

For example, I have taken a snapshot in a vmware workstation machine, and then trashed the disk drive. Rolled back to a saved snapshot and everything including disk was just fine. Can't do that even with the host.

I've notice performance comparible to the host, but I gave my VM machine a full 1gig of ram to work with, as well as a 20g drive. VM even supports dual processors.

There is one other subjective thing I've discovered, and admittedly it probably won't matter to most. If you are into serious testing, or trying to solve a software problem, if you report to developers you are testing on a VM machine, they won't even question it if they know you are using Vmware machines. Don't know about the others. Reason for this as I've discovered is most of them use the VMware workstation vm machines themselves.

Pete

[namdog]

I like virtualbox, because it support remote access.

My wife has a powerful PC.
Now I can run virtualbox on her PC (without interface) while she is working. Then I use the birtualbox from my computer.

[aigle]

Hi charincol,
Nice testing.

[wilbertnl]

Thank you very much, Charincol, for all your efforts in this time consuming test and reporting the results so clear to us.
If you don't want to spend $190 for VMWare workstation, then it's a big deal to choose from the [free] alternatives.
I have tried QEMU lately, XP guest in XP host, and the guest installation took hours, it seemed to stall frequently... But I'm impressed with the small VM software that runs it all.
It's good to know that VirtualBox is progressing, it has a lot potential.

I value differencing disks the most. It enables me to create different/temporary setups, based on the same OEM
installation.

[Meriadoc]

Yes, thankyou Charincol, a nice study there. I'm using VMWare Workstation, VirtualBox and occasionally Parallels on a Mac and its nice to see a comparison - with your system:thumb: I was interested in how the VirtualPC compared as I've used it the least, again thanks for taking the time.

[charincol]

You're welcome. I mostly use Virtual PC for malware or suspicious file testing because of it's quick and easy undo disk settings access. The other VM softwares have a little more complicated process. But for running a VM for a while, VMWare is my choice.

[aigle]

-{ Quote: "Yes, thankyou Charincol, a nice study there. I'm using VMWare Workstation, VirtualBox and occasionally Parallels on a Mac and its nice to see a comparison - with your system:thumb: I was interested in how the VirtualPC compared as I've used it the least, again thanks for taking the time." }-
So what is ur own experience on Mac?

[Meriadoc]

Mac?
At work we've a mac with Parellels for running Windows and nix like OSes. Much easier than Apple boot camp (http://en.wikipedia.org/wiki/Boot_Camp) where you have to shut down, reboot, plus you have to organize the hard drive to begin with. - although I'm looking to change as I took over the mac for the Fusion beta which although comparable has a few more features. I've also used CrossOverMac (based on WINE) which makes the installing of Windows software really easy, COM will just recognise it and do the rest.