Backup of data: Imaging vs replication programs

[Thorz]

After reading about the subject on other threads I think this topic deserves its own one, and I have decided to post it following Peter2150 suggestion.

Imaging programs like ShadowProtect or Acronis True Image are used to backup both system partitions and data partitions containing documents, pics, videos, etc.
Imaging stores the contents on proprietary container formats.

Replication / Sync programs are used to backup data, they are not viable for backing up system partitions.
Replicating uses just copy as a method for storing.

The objective of this thread is to discuss the pros and cons of both backup methods when used on data partitions.

The following is some info compiled by me about both methods:

IMAGING
PROS:

Compression: Possibility to compress the images for saving storage space.
Incremental: Can be used to quick imaging the changes.
Possibility of recovery of previous versions of files when using incremental
When using automatic schedules gives protection against virus or user error as replication will just automatically copy the infection or error (deleted folders/files) to the backup disk.
Easy storage as the backup contains fewer files to worry about.
Easy scheduling and automation as you are managing all your backup jobs from the same program you use for backing up your system partition.

CONS:

As all the data is stored inside a container, corruption of a part of the image file could render the rest of your backup useless.
For access to your files you have to mount or extract the image file meaning more time to access your data in case of a disaster.
You have to install or have access to the application in any other way (cd media, usb, etc) for having access to your files.
They are more expensive than good replicators that can be found for free (i.e. Karen's Replicator).


SYNC/REPLICATORS
PROS:

You will always have the files on your restore partition/disk ready for use, without needing to restore or mount anything.
Corruption is a minor issue for that it will not affect the hole backup but just some files if it happens.
The good replicators are cheaper than the good imagers, or free
They are as fast than image programs (see bellow Karen's Replicator vs ShadowProtect)

CONS:

If using scheduling/automation they can replicate a virus or user error (for example file/folder deletion) and render your backup useless.
You don't have access to previous version of files on your backup as with image incrementals.
You will have a lot of files to care about as your backup is an exact copy of your originals.


Here are some tests posted kindly by ErikAlbert in another thread comparing ShadowProtect with Karen's Replicator:

{QUOTE-> I checked the required space on my external harddisk :
1. Karen Replicator volume = 27.8gb (= volume of E:\Backup Data)
2. ShadowProtect's volume = 25.5gb (= volume of .spf-file)
So KR needs only 2.3gb more than SP on my external harddisk for the same backup, using standard compression on SP.
(...)
I also checked the pure scan-time of KR, in case there is nothing to backup.
1. First I ran KR one time to do the normal backup.
2. Right after that I ran KR again and it took only 7 seconds to check if there was something to backup or not.
That is the absolute minimum run-time of KR : seven seconds...
(...)
I formatted my second harddisk, which contains my Data Partition [D:] in order to test a full restore with KR.
KR restored my data partition in 17 minuts (28gb)

I also had an image-file to test the restore with the Recovery CD.
SP restored my data partition in 27 minuts (28gb)
<-QUOTE}
There are several other users on this board that have decided to use replicators over imagers for backing up their data. Most of them use imaging for backing up their system partitions.

What do you think about this topic?
Do you use your imaging program to back up your data files too or do you entrust this task to other methods, and why?

I personally will buy ShadowProtect 3.0 when it comes out this week for backing up my system partition, but have not decided yet if I will use it too for making incremental backups of my 400GB data partition or if I will use a program like Karen's Replicator for this task.

[Acadia]

{QUOTE-> What do you think about this topic?
<-QUOTE}
:thumb:

Acadia

[Peter2150]

I do both. All the data I really care about is part of my C partition anyway. So when I image it is part of the image.

Secondly my two machines are setup identically, so I use a sync program to sync to an external drive, and from that drive to the 2nd machine.

So the external drive and 2nd machine become backup's to my data.

The stuff I have like photo's and downloaded programs I just keep on external and 2nd internal drives, and just duplicate when I feel like. Non critical.

Pete

[ErikAlbert]

I also compared SP with KR, doing a full backup of my Data Partition.

SP = 21 minuts = 31.6 GB (normal compression)
KR = 22 minuts = 33.7 GB (no compression)

So 1 minut difference isn't much.
Of course a daily backup isn't a full backup.
During a daily backup KR will only backup files that have been changed or added or deleted.

[Thorz]

{QUOTE-> Of course a daily backup isn't a full backup.
During a daily backup KR will only backup files that have been changed or added or deleted. <-QUOTE}
As well as SP when doing an incremental.

Thanks for all this info ErikAlbert

[lucas1985]

{QUOTE->
1- Imaging stores the contents on proprietary container formats.

2- Replicating uses just copy as a method for storing.

3- You don't have access to previous version of files on your backup as with image incrementals. <-QUOTE}
Some minor corrections :)
1- DriveImage XML (http://www.runtime.org/dixml.htm) stores the image file in a XML-based format
{QUOTE->
Images are stored in XML files, allowing you to process them with 3rd party tools. Never again be stuck with a useless backup!
<-QUOTE}
2- Some file backup programs have the ability to backup to a single file/container, usually a ZIP archive.
3- Some file backup programs have the ability to store more than one version of a given file.

[ErikAlbert]

{QUOTE-> As well as SP when doing an incremental.

Thanks for all this info ErikAlbert <-QUOTE}
I can't use incremental backups, because my data partition is updated via two work snapshots.
Firstly, you can not install SP in two snapshots = piracy.
Secondly, incremental backups with two different SP's would create a total mess.

[lodore]

the point is most people who use shadow protect desktop dont use any snapshot programs.
if you do a incremental backup every example 30mins you wouldnt need the snapshot program.
its just a shame you need to boot from the winpe cd to do a restore.
since most backup program can reboot and do the restore of an OS drive in a prebooting environment.
lodore

[Peter2150]

{QUOTE-> the point is most people who use shadow protect desktop dont use any snapshot programs.
if you do a incremental backup every example 30mins you wouldnt need the snapshot program.
its just a shame you need to boot from the winpe cd to do a restore.
since most backup program can reboot and do the restore of an OS drive in a prebooting environment.
lodore <-QUOTE}

Not necessarily. During the busiest period of the business use on one of my desktops, I don't really have the time to update snapshots. The automatic incrementals of SP can be taken as often as every 15 minutes, and it is effortless and not even noticed. Then if the system hangs, and won't reboot(has happened) you can restore to a close previous point. Quite valuable if it happens.

Pete

[lodore]

Hi Peter,
do you backup with SP to a seprate partition or a a seprate drive?
im gonna problaly backup to a second internal drive with my new pc at least daily.
lodore

[Peter2150]

{QUOTE-> Hi Peter,
do you backup with SP to a seprate partition or a a seprate drive?
im gonna problaly backup to a second internal drive with my new pc at least daily.
lodore <-QUOTE}

Everything to a separate drive. All my drives only have one partition.

Pete

[Alphalutra1]

Whenever I backup my systems, I like to take a dump (http://netbsd.gw.com/cgi-bin/man-cgi?dump+8+NetBSD-current) :P This has been one of the most reliable ways of backing up for me, it is incremental, and the files inside the snapshots can be accessed and restore (http://netbsd.gw.com/cgi-bin/man-cgi?restore+8+NetBSD-current)(d) selectively as well. The nice things about these tools is that they are available for most filesystems, including ext2,ext3 (those are the main filesystems used in linux), xfs, ffs, ufs, and ufs2. See this (http://www.freebsd.org/doc/en_US.ISO8859-1/books/handbook/backup-basics.html) part of the FreeBSD handbook to see why it is simply the best tool period to use on *nix.

There is a dump and restore utility from ntfs/fat (http://sourceforge.net/projects/nfgdump/), but I haven't tried it yet and development seems to have stopped, but it looks pretty mature and stable from what I have read in the user manual.

But for backing up windows in a snapshot, I have always resorted to ntfsclone (see my post here (http://www.wilderssecurity.com/showpost.php?p=938752&postcount=2)). Depending on what option you select you can mount it and read the files from it.

Another tool I use is rdiff-backup (http://rdiff-backup.nongnu.org/) which is a replication program. It is awesome for backing up my usb drive so I always have an exact copy at home. Another great feature is that it keeps increments so if I change a file and have already backed it up, it stores an old copy automatically which is great. Another cool thing is it also works on windows ;D

Cheers,

Alphalutra1

[grnxnm]

{QUOTE-> I also compared SP with KR, doing a full backup of my Data Partition.

SP = 21 minuts = 31.6 GB (normal compression)
KR = 22 minuts = 33.7 GB (no compression)

So 1 minut difference isn't much.
Of course a daily backup isn't a full backup.
During a daily backup KR will only backup files that have been changed or added or deleted. <-QUOTE}

Interesting. You say that with KR the resulting files (with no compression) are 33.7GB. The fact that SP is only able to compress these down to 31.6GB is a strong indication that the files on your data partition are ALREADY highly compressed. In such a case you may achieve higher performance if you do not compress the data. Then again, compression is a funny thing because while you think that the added CPU load may slown down the overall process, it's most often the case that compression speeds up the overall imaging process as it results in less writes to the image file, and as the disk is (by far) the slowest device, it's generally very beneficial to use compression vs no compression. However, in this case, where the source data is nearly all compressed to begin with, you may realize increased performance with ShadowProtect if you image without compression.

[ErikAlbert]

{QUOTE-> Interesting. You say that with KR the resulting files (with no compression) are 33.7GB. The fact that SP is only able to compress these down to 31.6GB is a strong indication that the files on your data partition are ALREADY highly compressed. In such a case you may achieve higher performance if you do not compress the data. Then again, compression is a funny thing because while you think that the added CPU load may slown down the overall process, it's most often the case that compression speeds up the overall imaging process as it results in less writes to the image file, and as the disk is (by far) the slowest device, it's generally very beneficial to use compression vs no compression. However, in this case, where the source data is nearly all compressed to begin with, you may realize increased performance with ShadowProtect if you image without compression. <-QUOTE}
I'm not an expert, but I do alot of video-editing lately and the 33gb is mainly caused by video-files, which I collect on my data partition, until I have enough to burn them on DVD.
Most are documentaries of 47 minuts, not movies and I created AVI-files of the recorded videos, so that the volume of the AVI-file is MUCH smaller than the original video. Normally I can burn only 2 video-files on one DVD, now I can burn 11 video-files (AVI) on one DVD. So I think that this is the reason.

P.S.:the test in this case was not because of the small volume difference, but because of the small time difference between SP and KR. I was quite surprised by KR. After all SP is the fastest image backup software at this moment.

[Huupi]

Beware of the dangers if you get your sync app to set up the way it makes realtime continuous scans for differences .In this way your backup is always equal to the sourse,if you mess something on data you work on and like to replace it with an earlier backup copy,then you'r out of luck !!!

[ErikAlbert]

{QUOTE->
CONS:

If using scheduling/automation they can replicate a virus or user error (for example file/folder deletion) and render your backup useless.
You don't have access to previous version of files on your backup as with image incrementals.
You will have a lot of files to care about as your backup is an exact copy of your originals.

<-QUOTE}
I don't understand about viruses as a CON.
If my data partition is infected and I do an image backup with SP, also the viruses are included in the backup. If I restore that image, my data partition will be infected again. The same will happen with KR. Am I missing something ?

[Huupi]

The titel of this thread is utterly useless and disturbs everyone.Its like comparing a dog with a horse.Better compare KR with their own kind such as Syncback,AJCBackup,Debian and the like !!

[grnxnm]

{QUOTE-> I'm not an expert, but I do alot of video-editing lately and the 33gb is mainly caused by video-files, which I collect on my data partition, until I have enough to burn them on DVD.
Most are documentaries of 47 minuts, not movies and I created AVI-files of the recorded videos, so that the volume of the AVI-file is MUCH smaller than the original video. Normally I can burn only 2 video-files on one DVD, now I can burn 11 video-files (AVI) on one DVD. So I think that this is the reason.

P.S.:the test in this case was not because of the small volume difference, but because of the small time difference between SP and KR. I was quite surprised by KR. After all SP is the fastest image backup software at this moment. <-QUOTE}

What kind of performance do you get with ShadowProtect backing up that data volume (the one full of .AVI files) if you turn compression off?

[Thorz]

{QUOTE-> Firstly, you can not install SP in two snapshots = piracy. <-QUOTE}

?? Have you got this info directly from StorageCraft? I am well aware of your problem activating SP on 2 different FD-ISR snapshots, but I cannot see why doing it would be piracy. It is the same machine after all!

{QUOTE-> Beware of the dangers if you get your sync app to set up the way it makes realtime continuous scans for differences .In this way your backup is always equal to the sourse,if you mess something on data you work on and like to replace it with an earlier backup copy,then you'r out of luck !!! <-QUOTE}

You're right. This is the same situation that I was mentioning in the FP about user error being replicated, but I was using an automated schedule scenario and not real time, like when you create an unattended scheduled task to replicate your data partition every night at 2:00. If your data has been infected or messed up and you don't find this on time before the replication runs, the scheduled task ends up transferring all this mess to your backup.

{QUOTE-> I don't understand about viruses as a CON.
If my data partition is infected and I do an image backup with SP, also the viruses are included in the backup. If I restore that image, my data partition will be infected again. The same will happen with KR. Am I missing something ? <-QUOTE}
Not if you use incrementals with SP. You can restore your last incremental before the virii / user error has stricken and your data will be rescued.

{QUOTE-> The titel of this thread is utterly useless and disturbs everyone.Its like comparing a dog with a horse.Better compare KR with their own kind such as Syncback,AJCBackup,Debian and the like <-QUOTE}

This thread is not for comparing replicators, it is for comparing the 2 different backup methods (imaging vs replicating/synching) when the source are DATA partitions. I am sorry that you see this as useless, and I cannot see how it can "disturb" anyone.
Both backup methods are applicable to the subject and each one has its pros and cons. No one is better than the other one, it all depends on what do you need and what do you like to achieve.

I am trying to gather the most info as possible before deciding on which method I will use with my data disk. It is important to note that I am planning to run the backup always using a daily automated scheduled task, not manually, and if I decide for imaging, I will use incrementals.

[Acadia]

{QUOTE-> The titel of this thread is utterly useless and disturbs everyone. <-QUOTE}
Well, maybe for those of you who are smarter or more pc knowledgeable than I, this is true. However Wilders fortunately also caters to us dummies of the world, we need this kind of information the most, and I, for one, have found this thread most informative ... keep it coming, I'm soaking it up like a dry sponge.

Acadia

[Huupi]

{QUOTE-> ?? Have you got this info directly from StorageCraft? I am well aware of your problem activating SP on 2 different FD-ISR snapshots, but I cannot see why doing it would be piracy. It is the same machine after all!



You're right. This is the same situation that I was mentioning in the FP about user error being replicated, but I was using an automated schedule scenario and not real time, like when you create an unattended scheduled task to replicate your data partition every night at 2:00. If your data has been infected or messed up and you don't find this on time before the replication runs, the scheduled task ends up transferring all this mess to your backup.


Not if you use incrementals with SP. You can restore your last incremental before the virii / user error has stricken and your data will be rescued.



This thread is not for comparing replicators, it is for comparing the 2 different backup methods (imaging vs replicating/synching) when the source are DATA partitions. I am sorry that you see this as useless, and I cannot see how it can "disturb" anyone.
Both backup methods are applicable to the subject and each one has its pros and cons. No one is better than the other one, it all depends on what do you need and what do you like to achieve.

I am trying to gather the most info as possible before deciding on which method I will use with my data disk. It is important to note that I am planning to run the backup always using a daily automated scheduled task, not manually, and if I decide for imaging, I will use incrementals. <-QUOTE}

Sorry for disturbing some, and i for one also see some benefit with compare these app. The consencus is that they compare in speed,so ou can use either or both for same job.But bottom line is that i am more curious how they compare and can hold their own with their own kind,this tests will extracts their real potentials more than just a speed comparinas.

[ErikAlbert]

{QUOTE-> ?? Have you got this info directly from StorageCraft? I am well aware of your problem activating SP on 2 different FD-ISR snapshots, but I cannot see why doing it would be piracy. It is the same machine after all!
<-QUOTE}
Well I explained it all in detail in another thread. Nobody seems to understand the problem.
It's indeed the same machine, but not according StorageCraft.
SC does NOT verify your machine, based on HARDWARE COMPONENTS, that is exactly the problem.
SC creates its OWN Machine ID, not based on hardware components, when you install ShadowProtect and SC stores that Machine ID in their database and not only that, SC counts your activations.
The next time when you install SP in another snapshot for example, then SP creates another Machine ID, which won't match with the previous Machine ID of course and that is considered WRONGLY as piracy.
Then you have to send an email, explain your case and beg for a new re-activation.
Even when you re-install SP with the same Machine ID and you do this too many times, the counter will stop you and again a begging email.

I won't install ShadowProtect under Windows anymore, because there is no guarantee it will work and I'm not going to beg for a re-activation each time, that's not my style and it hurts my feelings to do this and I'm very sure that other users will find this very annoying, embarrassing and even consider this as an insult. After all they paid $70 for it and it doesn't work.

In the future, I will use the Recovery CD only to backup/restore my system partition and that's it.
Karen's Replicator will backup/restore my data partition and is now my most important backup software. :)

[Peter2150]

It's not that what Erik is doing is piracy it's the activation is there because of piracy, to answer the question asked.


But enough of this. Back on topic

Pete

[CatFan432]

{QUOTE->

SYNC/REPLICATORS

CONS:
You don't have access to previous version of files on your backup as with image incrementals.
<-QUOTE}

I used KR, and loved the program. Then the need arose for me to keep an accurate history of ongoing changes to some documents. I switched to ViceVersa Pro, (http://www.tgrmn.com/) which allows for archiving of files. Screenshot is of archive option screen.

Regards, CatFan

[ErikAlbert]

{QUOTE->
Not if you use incrementals with SP. You can restore your last incremental before the virii / user error has stricken and your data will be rescued.
<-QUOTE}
How do you know you have a virus ? These viruses don't shout "Hey, I'm here in this incremental backup."
Maybe your incomplete AV/AS/AT/AK/AR-scanners didn't find it ?

[Peter2150]

The advantage of incrementals over something like say KR or AJC sync is once you discover a problem you can work your way back and find the problem. Time consuming yes. But works.

I just went thru that. Had a strange problem with FDISR. Had no idea. Logs told me when problem started, and restoring an image from before that confirmed it was working back then. Did help me track down the problem, which was a software conflict(I think).

Once you sync thats it.

Pete

[Huupi]

{QUOTE-> I used KR, and loved the program. Then the need arose for me to keep an accurate history of ongoing changes to some documents. I switched to ViceVersa Pro, (http://www.tgrmn.com/) which allows for archiving of files. Screenshot is of archive option screen.

Regards, CatFan <-QUOTE}

Thats interesting,thats missing in KR, in this way you have some kinda of incrementals like you have in SP,ATI and Paragon.I will give it a whirl,sounds very interesting !I understand that there is the option to copy open files(you'r are working on),thats also not possible in KR !

[Thorz]

{QUOTE-> "Hey, I'm here in this incremental backup."
<-QUOTE}

:)

Virus was just an example. I am confident in my case that NOD32 can spot a virus if it strikes. I think that one of the great bennefits of incrementals is the way you can keep older versions of files and protect yourself from accidental deletion of corruption of files when you don't spot the problem at once. In the case of replicators/synchronizers, if you spot the problem before the replication schedule runs, or if you only use manual replication, this will not be a problem.

{QUOTE-> I switched to ViceVersa Pro, (http://www.tgrmn.com/) which allows for archiving of files. Screenshot is of archive option screen. <-QUOTE}

Very interesting. I agree with Huupi, this makes replication even more appealing as it adds that extra layer that keeping of old versions of files represents.
Thanks!

I am wondering something, and this one is for the SP / Acronis or other imaging users:

Let's say that you have a big file, f. e. a 13GB AVI file that you are editing with a NLE like Premiere or Pinnacle Studio. You image your drive today. You make some small changes to the file tomorrow and image again using incremental imaging. Does the imaging program add the hole 13GB file again to the image, or just the few sectors of hard disk that changed?

[Peter2150]

{QUOTE-> Thats interesting,thats missing in KR, in this way you have some kinda of incrementals like you have in SP,ATI and Paragon.I will give it a whirl,sounds very interesting !I understand that there is the option to copy open files(you'r are working on),thats also not possible in KR ! <-QUOTE}

AJC Archive Backup is perfect for that as it automatically saves certain specified files anytime you do a save or exit the program. You can then retrieve prior versions. Probably has save me more than anything.

[Thorz]

{QUOTE-> AJC Archive Backup is perfect for that as it automatically saves certain specified files anytime you do a save or exit the program. You can then retrieve prior versions. Probably has save me more than anything. <-QUOTE}

Can it archive files just replicating (copying) them or does it use its own container?

[Peter2150]

{QUOTE-> Can it archive files just replicating (copying) them or does it use its own container? <-QUOTE}

It uses it's own container so to speak. You can access back versions, by just going into windows explorer and right clicking on the file. Then you select by data and time, and either view the file, extract to a new place or e xtract and over write the existing file.

Nothing like it when you are working on a big doc and save it only to realize you accidently deleted key pages.

[Huupi]

{QUOTE-> It uses it's own container so to speak. You can access back versions, by just going into windows explorer and right clicking on the file. Then you select by data and time, and either view the file, extract to a new place or e xtract and over write the existing file.

Nothing like it when you are working on a big doc and save it only to realize you accidently deleted key pages. <-QUOTE}

Peter, Copies it only the changes to an point in time container,[real incrementals,with date and time]in sequence up to the current time or is every backup file the original with the changes,if so then you can run easily out of space [pictures],no problems with documents though,they are very small.

[Huupi]

In KR the incrementals are not saved in "buddy" containers,every older file is overwritten by the changes,so in KR there no real incrementals to use,this is a limitation compared with AJC Backup and Vice Versa and maybe more out there.Though KR is a juwel on its own,else doing the things what you expect of a top application,but i should be cautious .

[Peter2150]

{QUOTE-> Peter, Copies it only the changes to an point in time container,[real incrementals,with date and time]in sequence up to the current time or is every backup file the original with the changes,if so then you can run easily out of space [pictures],no problems with documents though,they are very small. <-QUOTE}

You probably would never use photo's with AJC's Active Backup. It doesn't save incrmentals, it saves the whole file. But space isn't an issue as you can specify which directory, which file, and how long to keep it. So I only save office data files, normally 7 days, but two critical files I kept 60 days, and yes it did get larger, but it was only a spreadsheet.

Point is you can control what it's doing.