Interesting image creation times with top products

[Masejoer]

Over the weekend I tested Macrium Reflect free, Terabyte Image for Windows, and Drive Snapshot to find what works well now days for live image creation. My drives are as follows:

Intel X25-M 160GB SSD
C: 50GB used

Intel X25-M 160GB SSD
D: 100GB used

WD Caviar Black 2TB HDD
E: 400GB, 300GB used
F: 1400GB, 1200GB used

I backed the system up to a custom built NAS that pushes 100+MB/sec but uses standard windows file sharing so take the performance limitation of that into mind. The test NAS drive was a WD Green 2TB drive.

---------

On Friday I ran Macrium and found that it completed the backup using standard settings as a scheduled task in 5 hours. The backup file(s) ended at 1.03TB.

On Friday night I ran Drive Snapshot through the UI using standard settings and it took 8 hours to complete. The backup file(s) ended at 1.03TB.

On Saturday I ran Image for Windows through the UI using standard settings EXCEPT I enabled VSS mode as PHYLock couldn't obtain a lock and it took 7.5 hours to complete. The backup file(s) ended at 1.03TB.

On Saturday night I ran Macrium again to verify that I didn't get the numbers wrong earlier, through the UI using standard settings and it took 4.5 hours to complete. The backup file(s) ended at 1.03TB.

---------

There are no scheduled tasks on this machine, and it was only used occasionally for web browsing or cleaning up files on my server, logged in through remote desktop. I have not tested restoration performance yet as it doesn't mean as much to me in a home environment, but I will test smaller images later out of curiosity.

Does anyone have any ideas as to why Macrium is twice as fast as the other two products? I liked the interface and ease of setup using Image for Windows, but the backup performance is pathetic compared to Macrium. Macrium is a bit bloated in comparison. I'm on the fence about whether I like Snapshot as much as the other two products, each having their own merit.

[Brian K]

Masejoer,

A few months ago TeraByte Unlimited added several different compression levels to their products. There are now 10 levels. People wrote to the forum describing their findings. I found Nil Compression to be the fastest (although I wouldn't use this level in practice) but most of the other respondents didn't agree. The maximum compression level produced images only marginally smaller than other levels but the time taken to create the image was ridiculous. Nobody liked this level. Most, but not everyone, preferred Enhanced Speed - A as it was the best compromise between image size and imaging time.

Even a 1% difference in compression can make a big difference in imaging time but hardware made a difference too. Some computers performed better at a certain compression level and other computers performed differently. I gave up trying to understand it. I just use Enhanced Speed - A as it's the best for my computer. The images are 8% percent larger than Standard compression but the creation time is 40% of the Standard compression time.

[Masejoer]

Originally Posted by Brian K

Masejoer,

A few months ago TeraByte Unlimited added several different compression levels to their products. There are now 10 levels. People wrote to the forum describing their findings. I found Nil Compression to be the fastest (although I wouldn't use this level in practice) but most of the other respondents didn't agree. The maximum compression level produced images only marginally smaller than other levels but the time taken to create the image was ridiculous. Nobody liked this level. Most, but not everyone, preferred Enhanced Speed - A as it was the best compromise between image size and imaging time.

Even a 1% difference in compression can make a big difference in imaging time but hardware made a difference too. Some computers performed better at a certain compression level and other computers performed differently. I gave up trying to understand it. I just use Enhanced Speed - A as it's the best for my computer. The images are 8% percent larger than Standard compression but the creation time is 40% of the Standard compression time.

Brian,

I will try this setting this evening and see how it goes. I don't think the CPU was as heavily loaded in IFW as Macrium, but I will verify. The system has an Ivy Bridge quad core at 4.5Ghz and 16GB of RAM, not that it should change much in comparison between applications.

[Isso]

Some of the imaging products don't have transfer speed optimization, i.e. they are working in a consequent steps like: "read-compress-write". This is a very simple and reliable solution, but slow.
More advanced products use multi-stream (multi-thread) approach - where reading, compressing and writing is done simultaneously. This is more complex, but is much faster.
Another source of delays is the compression method. Some imaging products use quite slow compression algorithms. Here the CPU speed plays significant role.

Looking at your numbers - despite Macrium being much faster that the competition, still it's overall speed looks to be around 60 MB/sec, which is significantly lower than the 100 MB/sec NAS speed. So either even Macrium isn't fully optimized, or the real NAS speed is less than 100 MB/sec

[Brian K]

Originally Posted by Masejoer

I don't think the CPU was as heavily loaded in IFW as Macrium,

Masejoer,

I'll be interested in your findings. I don't notice any reduction in computer performance when IFW is running but I've seen a few reports of high CPU usage during image creation and the computer being "unresponsive". My images are created on a schedule and I just keep using the computer normally while IFW is running.

[Brian K]

Isso,

I made these figures some time ago on a 6 year old computer (two internal HDs)....

No compression = 94 MB/sec
Std compression = 21 MB/sec
Enhanced Speed - A = 59 MB/sec

[Masejoer]

Originally Posted by Brian K

Masejoer,

I'll be interested in your findings. I don't notice any reduction in computer performance when IFW is running but I've seen a few reports of high CPU usage during image creation and the computer being "unresponsive". My images are created on a schedule and I just keep using the computer normally while IFW is running.

Will do. Tonight I won't have much control over long backup durations as I spent the weekend cleaning up 15 years of "backed up" data , which previously I just stored in multiple places as data cleanup was a daunting task. Anyway, I officially started, and used treesize personal to find duplicate files. Everything I could has been moved to my server, and follow its backup schedules. Now it's just huge directory tree cleanup to do on photos, 11mbit digitized VHS, and 22mbit AVHCD.

I'm down to maybe 300GB of data to backup from the workstation so I will need to run a quicker backup in all three applications. I'm hoping I can get one of the others down to Macrium speed so I can use them. I love the speed improvement Macrium gives.

[Isso]

Brian,

"No compression" speed of 94MB/sec is close to the HDD limit, so I guess reading and writing in IFW is very well optimized.

But the speed with compression is lower than I would have expected, perhaps limited by old CPU. What was the CPU model? And do you by any chance remember if all its cores were loaded?

[Brian K]

Isso,

It is a Pentium D (I think the first of the dual core processors).

I found a figure for Maximum compression...

3 MB/sec

No wonder it's slow.

[Isso]

Brian,

Oh yes, for such a slow processor those are pretty good results

[TheRollbackFrog]

Originally Posted by Isso

Brian,

"No compression" speed of 94MB/sec is close to the HDD limit, so I guess reading and writing in IFW is very well optimized.

But the speed with compression is lower than I would have expected, perhaps limited by old CPU. What was the CPU model? And do you by any chance remember if all its cores were loaded?

Isso, when I use IFW on either my 2.8ghz Dual core or my 2.6ghz Quad, I find it is not optimized for either multi-thread or multi-core... most of its big work is done in only 1-core. I'm sure that affects its general thruput. I think you'll find Macrium somewhat optimized for multi threads/cores.

[Brian K]

I saw these figures done with IFW on a modern computer....

Backup using validate byte-for-byte.

Type: backup time / validate byte for byte time / size

None: 26m/25m/196GiB
Standard: 34m/16m/114GiB
Enh A: 53m/16m/106GiB
Enh D: 34m/25m/108GiB
Enh Spd A: 16m/16m/119GiB

In MiB/sec these work out to be...
129
57
34
54
126

[Isso]

TheRollbackFrog,

I agree when a program doesn't load more than one core it looks like it's not multi-core optimized. That looks strange, because IFW is a quite sophisticated software.

Also the test results posted by Brian for "Enh Spd A" indicate 125 MB/sec speed, which is the same as "None", i.e. it's the disk transfer maximum speed and the compression doesn't slow down the process.
So either the CPU is so fast that can handle it with a single core or maybe Brian is using newer IFW version that has multicore support (?). Brian, could you tell the CPU load and how many cores it has?

[Brian K]

Isso,

It's not my computer. TeraByte Support provided those figures about 3 months ago for ver 2.72.

[Masejoer]

Originally Posted by Brian K

Masejoer,

I'll be interested in your findings. I don't notice any reduction in computer performance when IFW is running but I've seen a few reports of high CPU usage during image creation and the computer being "unresponsive". My images are created on a schedule and I just keep using the computer normally while IFW is running.

My computer now has 350GB of used space after cleanup over the weekend.

Intel SSD C:
52GB used of 149GB

Intel SSD D:
87GB used of 149GB

WD platter F:
183GB used of 1.33TB

WD platter E:
28GB used of 500GB

------------

The times are below in hours:minutes to completion:

1:26 Macrium Reflect
Standard settings
279GB image size
20-30% CPU usage during imaging

2:15 IFW
Standard settings with VSS
277GB image size
10-20% CPU usage during imaging

3:11 Drive Snapshot
Standard settings
291GB image size
15-25% CPU usage during imaging

2:00 IFW
Enhanced Speed-A with VSS
280GB
I am checking this one remotely so no CPU % numbers

2:00 IFW
Enhanced Speed-B with VSS
279GB
I am checking this one remotely so no CPU % numbers

------------

The numbers are all over the place. I'm sure I can speed up the programs by launching one instance of imaging for each drive to run in parallel, but I'm still interested in these numbers being all over the board. The Macrium to Snapshort ratio is about the same as before when I had more data, but IFW is a bit quicker.

[Isso]

Thanks Masejoer, interesting numbers indeed. Macrium looks good. IFW apparently is not fully multicore optimized.
I'm wondering how Acronis and Shadowprotect compare to these - I tested them before and found quite fast.

[Masejoer]

Originally Posted by Isso

Thanks Masejoer, interesting numbers indeed. Macrium looks good. IFW apparently is not fully multicore optimized.
I'm wondering how Acronis and Shadowprotect compare to these - I tested them before and found quite fast.

I think that in general, I like IFW/IFD due to the ease of use on a PE USB drive, general speed (and easy parallelization), size, and cost. The base Windows program is around 10MB (plus dependencies) which is much better than Macrium Reflect, but it doesn't compare in size to the 260KB Drive Snapshort. What is interesting is that it is cheaper to get MANY more licenses and purchase up to the next price break. 500 licenses cost more than 1,000

11 licenses cost the same as 15
19 licenses cost the same as 25
36 licenses cost the same as 50
77 licenses cost the same as 100
472 licenses cost the same as 1,000

[Moosehead77]

Who cares. The point is whether or not in the moment you need any of the programs the most, they restore your system and you dont lose crucial information or have to waste time reloading all the programs you may have had on the PC at the moment of the crash.

[Masejoer]

Originally Posted by Moosehead77

Who cares. The point is whether or not in the moment you need any of the programs the most, they restore your system and you dont lose crucial information or have to waste time reloading all the programs you may have had on the PC at the moment of the crash.

Sure, but if I'm running this on my home server which also contains a lot of content changes throughout the week, I need speed. Doing my server will probably take 16 hours if I have 4TB of data. I don't trust having just optical burns/tape and offline backups of much of these files as it's my whole life, so much of it stays stored on my server and NAS also. High def video has certainly started taking its toll on my disk usage. I will figure out a backup method for these files that I like, when I come across it - for now, I'll stick with whole disk imaging as file backups have caused me grief in the past. Even the directory structure of data when using Adobe Premiere needs to be restored exact. For home use, restoration time doesn't matter so much. I will be testing this restoration time for production usage also though, since I think I would also like to use the IFW/IFD/IFL products there.

Pretty well-rounded application Terabyte has made.

[Brian K]

Originally Posted by Masejoer

2:00 IFW
Enhanced Speed-A with VSS

Masejoer,

Out of interest, how long did you wait for a PHYLock lock? I usually get a lock in 5 seconds but some of the regulars in the TeraByte forum have reported occasional waits of up to 10 minutes. The explanation given is something is "writing to the disk".

I've never used VSS so I don't know what it does to the imaging time.

If this is of benefit you can initiate a restore of a remote computer from your computer. You don't have to visit the remote computer. The backup image can be stored on your computer or remotely.

[Masejoer]

Originally Posted by Brian K

Masejoer,

Out of interest, how long did you wait for a PHYLock lock? I usually get a lock in 5 seconds but some of the regulars in the TeraByte forum have reported occasional waits of up to 10 minutes. The explanation given is something is "writing to the disk".

I've never used VSS so I don't know what it does to the imaging time.

If this is of benefit you can initiate a restore of a remote computer from your computer. You don't have to visit the remote computer. The backup image can be stored on your computer or remotely.

I have had PHYLock work once within seconds. Since then, it times out after the default 5 minute wait, so I just use VSS instead.

The computer is remote as I only setup remote access into one machine at home for security reasons through PPTP, and from that I do not provide myself access to other systems. This system also provides a few other internet services. Why have everything accessible from the internet when it isn't needed? pfSense is a great router distro and lets me lock down everything, and cheap (free) so the value is good.