Backup solutions for Linux

[tlu]

There have been several threads here regarding Linux backups.

I agree with Mrk's recent remark that it's probably best to use Clonezilla to create an occasional image of your root partition and to regularly use another backup solution for your precious data (/home and possibly other directories) so it's easy to restore specific files/directories if needed. So far I've been using rsync (there is also a GUI called grsync) which works reliably but I'm nevertheless interested in other solutions.

I've found the following alternatives:
sbackup (http://sourceforge.net/projects/sbackup/)
NSsbackup (https://edge.launchpad.net/nssbackup) which is the successor of sbackup. For Ubuntu it's available from a ppa (https://launchpad.net/%7Enssbackup-team/+archive/ppa).
Unison (http://www.cis.upenn.edu/%7Ebcpierce/unison/) which uses the rsync protocol.
Back in Time (http://backintime.le-web.org/) which is similar to Apple's Time Machine.
dd (Disk Dump). Its syntax is not easy but there is a GUI called AIR (http://air-imager.sourceforge.net/) (which is not in the Ubuntu repos but can be easily installed - the packages perl-tk and shareutils are needed)
TimeVault (http://www.dedoimedo.com/computers/timevault.html)
Pybackpack (http://andrewprice.me.uk/projects/pybackpack/) which is a GUI for rdiff-backup (http://rdiff-backup.nongnu.org/).
rsnapshot (http://www.rsnapshot.org/) which uses rsync and hard links "to keep multiple, full backups instantly available". Here's (http://www.rsnapshot.org/howto/1.2/rsnapshot-HOWTO.en.html) a how-to.
Mondo & Mindi (http://www.dedoimedo.com/computers/mondo.html)
HomeUserBackup (https://wiki.ubuntu.com/HomeUserBackup)
Keep (http://jr.falleri.free.fr/keep/wiki/Home)At first glance, rsnapshot looks best for me once properly setup. But i haven't tried the other alternatives yet. So I'm interested in your opinions about the (dis)advantages of these solutions.

[Mrkvonic]

I use grsync and tutorial coming up today, found it fastest and most reliable. Simulation mode and compression in transition are another great trait of rsync. Works fast, no stalls or hangs.

Unison is also good, but slow to index. Plus, it did not much like my NTFS-formatted external disk. Works well in most cases, though.

sbackup is not always reliable; trying to fire up the GUI twice can get the app to hang, plus the deamon is finicky sometimes. But works ok, except that it did not much like my NTFS-formatted external disk, like Unison.

Mondo, quite useful, especially the bootable iso with your data, but it's slow and more difficult to use by average users.

TimeVault, works fine as well, great for revision control, but takes a lot of space and runs all the time. Excellent for fire-and-forget users, who like to have their data backed up constantly and in many copies.

All in all, grsync gets my vote.

Regards,
Mrk

[tlu]

Mrk, thanks for sharing your view! Nice coincidence that a grsync tutorial of yours is about to be come out ;)

-{ Quote: "I use grsync and tutorial coming up today, found it fastest and most reliable. Simulation mode and compression in transition are another great trait of rsync. Works fast, no stalls or hangs." }-
Agreed. It's a nice GUI if you want to start the backup manually but not needed if you start rsync via a cron-job.

-{ Quote: " sbackup is not always reliable; trying to fire up the GUI twice can get the app to hang, plus the deamon is finicky sometimes. But works ok, except that it did not much like my NTFS-formatted external disk, like Unison." }-

Another disadvantage (possibly also for some of the other alternatives) is that the backup is saved in a single .tgz file which can easily become several gigabytes large so restoring specific files can become rather time-consuming.

-{ Quote: " All in all, grsync gets my vote." }-
Yes, but I tend more and more to an automated solution. Such a solution is probably best based on rsync, I'm not yet sure if I should add rsnapshot.

[Nick Rhodes]

I have had a play with TimeVault and Back In Time, there is also flyback.
I think all use rsync under the hood.

They are suited to user data files and do frequent backups, though I would not call them backups, more revisions as no point backing up to the same drive/machine as where the data is held.
I can't remember which one, but had great options for compounding old backups together so that you had 24 hourly backups, 7 weeks, 12 monthly etc.

When I rebuild my server, I will then rsync the backup-set across the network from my laptops to the server as a geniune backup.

[Pedro]

luckyBackup offers rsync backup also, it has alot of options, including the creation of cron jobs.
http://luckybackup.sourceforge.net
http://www.kde-apps.org/content/show.php/luckyBackup?content=94391
It's also important to note differences between the different backends, rsync, rdiff-backup etc. The ideal program for me does not exist :P http://www.wilderssecurity.com/showpost.php?p=1496430&postcount=4

[Pedro]

-{ Quote: "
I can't remember which one, but had great options for compounding old backups together so that you had 24 hourly backups, 7 weeks, 12 monthly etc.
" }-
rsnapshot?

[tlu]

-{ Quote: "luckyBackup offers rsync backup also, it has alot of options, including the creation of cron jobs.
http://luckybackup.sourceforge.net
http://www.kde-apps.org/content/show.php/luckyBackup?content=94391
It's also important to note differences between the different backends, rsync, rdiff-backup etc. The ideal program for me does not exist :P http://www.wilderssecurity.com/showpost.php?p=1496430&postcount=4" }-

Very interesting links, Pedro. Thanks a lot!

[tlu]

-{ Quote: "rsnapshot?" }-
I found a script on the German ubuntuusers Wiki that should perform something similar for rsync. You'd have to create the appropriate cron-job.

-{ Quote: "#!/bin/bash
# Simple backup with rsync

# SOURCES and TARGET must end with slash

SOURCES="/root/ /etc/ /home/ /boot/"
TARGET="/media/backup/"
LOGFILE="/root/backup.log"
EXPIREDAYS=0
RSYNC="--delete"

#SSHUSER="user"
#SSHHOST="hostname"
#SSHPORT=22

### do not edit ###

`/bin/date > $LOGFILE`

if [ -e $TARGET ]; then
LASTBACKUP=`/bin/ls -d $TARGET[[:digit:]]* 2>> $LOGFILE | /usr/bin/sort -r | /usr/bin/head -1 `
fi

TODAY=$(/bin/date +%y%m%d)

if [ $EXPIREDAYS -gt 0 ]; then
EXPIRED=`/usr/bin/find $TARGET[[:digit:]]* -maxdepth 0 -ctime +$EXPIREDAYS 2>> $LOGFILE`
for EX in `/bin/echo $EXPIRED`
do
/bin/echo "rm -rf $EX " >> $LOGFILE
`/bin/rm -rf $EX`
done
fi

for SOURCE in `/bin/echo $SOURCES`
do
if [ "$LASTBACKUP" ]; then
INC="--link-dest=$LASTBACKUP$SOURCE"
fi
if [ "$SSHUSER" ] && [ "$SSHHOST" ] && [ "$SSHPORT" ]; then
SSH="ssh -p $SSHPORT -l $SSHUSER";
SOURCEDIR="$SSHHOST:$SOURCE";
else
SOURCEDIR=$SOURCE;
fi
`/bin/mkdir -p $TARGET$TODAY$SOURCE` >> $LOGFILE
echo "/usr/bin/rsync -e \"$SSH\" -av $SOURCEDIR $RSYNC $INC $TARGET$TODAY$SOURCE " >> $LOGFILE 2>> $LOGFILE;
/usr/bin/rsync -e "$SSH" -av $SOURCEDIR $RSYNC $INC $TARGET$TODAY$SOURCE >> $LOGFILE 2>> $LOGFILE;
done" }-

[iceni60]

i've had this ridiff-backup link for ages with a small bash script to run it with. i didn't realise anyone had mentioned it till now -
http://www.liquidweather.net/howto/index.php?id=56

[pcalvert]

According to this article, rsync is not 100% reliable as a backup solution:
Backing up Linux and other Unix(-like) systems (http://www.halfgaar.net/backing-up-unix)

Phil

[Nick Rhodes]

rsync is not a backup tool, is it is file copying tool, just like tar is an archive tool and rdiff is a diff tool.
IMHO all require extra glue (UI, script etc) to be a complete backup sollution.

[Pedro]

Interesting post i found on Planet KDE.
-{ Quote: "TimeVault is alive! Ok, so maybe it was never really dead. But I’ve found some time to work it and after reworking most of the internals I’ve finally gotten it to back up again.

For those who don’t know, TimeVault was originally a gnome backup system that I used way back when I was on Ubuntu. I liked the concept but disliked the implementation and so sometime after moving to KDE I started working on my own TimeVault. The goal is to have a user friendly, dead simple backup system that runs in the background, knows about (and exclusively uses) external media and isn’t afraid to integrate into KDE frameworks wherever it can." }-
http://blog.chatonka.com/2009/12/timevault-progress-update/

[wat0114]

I used ShadowProtect to backup the root partition this afternoon and successfully restored it just a few minutes ago, after unsuccessfully trying the x64 version of Mint ( did not format the /Home partition).

[Nick Rhodes]

I've been using Back in Time for almost a month and found it to be good and reliable.
Its captured 5GB of data and I've tried rolling back various documents and large files such as movies old and new and worked a treat.
There is a noticable slowdown when it diffs large files (backups are stored on same HDD at the moment), but its no worse than any other backup tool.
I plan on getting an external drive store my backups in the Xms sales.

[wat0114]

Just to erase any doubt about the effectiveness of ShadowProtect Recovery disk as a Linux image/restore solution, I used it to backup both the "/' and "/Home" partitions individually, re-botted off the Linux Mint disk and used Gparted to formated both these partitions (left the swap partition alone) to ext4, re-booted off the SP Recovery disk and used it to restore the images to their respective partitions. All worked fabulously :)

The other solutions, although free, appear to be rather technically challenging for the average user.

[wat0114]

Okay, i have to score one for Clonezilla :thumb: Just used ver 1.2.3-20 tonight to clone the root & home parts, then the ultimate test: format root. Clonezilla restored it quickly and perfectly and I must say had no problems following the directions in cloning/restoring - far more intuitive than I thought it would have been. Very nice image/restore solution :)

[Ocky]

-{ Quote: "Okay, i have to score one for Clonezilla :thumb: Just used ver 1.2.3-20 tonight to clone the root & home parts, then the ultimate test: format root. Clonezilla restored it quickly and perfectly and I must say had no problems following the directions in cloning/restoring - far more intuitive than I thought it would have been. Very nice image/restore solution :)" }-
Hi wat0114,

You are progressing at a mind boggling pace,:) so I would like to ask you which version of Clonezilla you downloaded - was it the Ubuntu alternative or the stable ?
As far as I know Clonezilla does not support Grub2 yet, can you or anybody else confirm that this is indeed the case ?
BTW. Linux seems pretty stable Ubuntu and CentOS - never ever had to image - only once due to new HDD when I used Remastersys to get back my LTS Hardy.
Regards

[wat0114]

-{ Quote: "Hi wat0114,

You are progressing at a mind boggling pace,:) " }-

LOL, yeah, I better slow down ;D

-{ Quote: "so I would like to ask you which version of Clonezilla you downloaded - was it the Ubuntu alternative or the stable ?" }-

Yikes!!, not even sure, hard to remember, but I think off the page here:

http://www.clonezilla.org/download/sourceforge/all/iso-zip-files.php

-{ Quote: "As far as I know Clonezilla does not support Grub2 yet, can you or anybody else confirm that this is indeed the case ? " }-

I'm cloning/restoring Mint Helena w/grub 2 and no problems, other than earlier whenever I re-sized or moved the Linux root partition, the grub would not restore properly, but for some mysterious reason that problem seemed to resolve itself as I've had a couple successful restores in a row. So I'm not even sure if it truly supports grub2. I have grub installed on the root partition.


-{ Quote: "BTW. Linux seems pretty stable Ubuntu and CentOS - never ever had to image - only once due to new HDD when I used Remastersys to get back my LTS Hardy.
Regards" }-

You're right, Mint is excellent, and I've got Ubuntu installed in VirtualBox. It's just that I want to be proficient in using imaging/restore software/disks and then I'm always doing different things with my setup. I've moved and resized my partitions several times trying to achieve what suits me. Maybe I've got it figured out now.
I get easily bored keeping my machine static for too long :) I don't do anything too important on it production-wise and all data is backed up nicely so I tend to use my machine as a learning tool. Linux is an O/S I've been intrigued with for a few years now so I'm getting rather involved in it of late. In spite of some frustrations I rather like it and just feel better knowing something else other than Windows. It's nice to have alternatives.

[Ocky]

-{ Quote: "
Yikes!!, not even sure, hard to remember, but I think off the page here:

http://www.clonezilla.org/download/s...-zip-files.php]" }-

Thanks wat0114,

Yeah, it's the alternative version of course. Was just confused for a while. I am sure Grub2 is supported as there is a karmic .iso. Seems easy to use so will give it a go.

Try Remastersys, it's even easier to use, especially in backup mode. http://www.geekconnection.org/remastersys/remastersystool.html

I am completely Windows free since April. Just received an email from Avira reminding me to renew my AV licence .. nice being able to save on that and other software licence renewals.

Have fun ! :argh:

[wat0114]

Thanks for the link, Ocky! Not sure I'll ever be Windows free, since I've used it for so long and it offers benefits Linux can't, but that works both ways too. Linux provides benefits Windows can't, so I'll likely always have a use for both O/S'.

[Trespasser]

I've been using CloneZilla 1.2.2-31 stable for quite a while and it does support Grub 2.

CloneZilla is like my all-time favorite back up solution. It's absolutely the best. Works with Windows (XP thru Windows 7), and Linux (depending upon the filing system used).

As a side note, I need to start a thread about AptOnCD for Ubuntu users. It's a great application. I have every update, plus every application I've installed (Wine, build-essential, cairo-dock, cabextract, unrar, VLC, etc.), and its dependencies, safely tucked away in an ISO image that I've saved to pen drive. Great to have should you need to do a fresh install for yourself or someone else. I generally rebuild the ISO image once a week to keep it up to date.

Later....

[Ocky]

-{ Quote: "
As a side note, I need to start a thread about AptOnCD for Ubuntu users. It's a great application. I have every update, plus every application I've installed (Wine, build-essential, cairo-dock, cabextract, unrar, VLC, etc.), and its dependencies, safely tucked away in an ISO image that I've saved to pen drive. Great to have should you need to do a fresh install for yourself or someone else. I generally rebuild the ISO image once a week to keep it up to date.

Later...." }-
That will be nice. wat0114 will be pleased to see that it comes with Linux Mint. May I ask you to kindly clarify/confirm some things now i.e. before starting a new thread ?

For instance if you had made the AptOnCD DVD for your distro and then want to do a fresh install of the same distro for someone (to save download time and MB's) ...
With a fresh install the AptOnCD Restore window (where you can select packages to restore) will probably not be shown because the AptOnCD program is not installed.
So, am I correct in saying that when inserting the AptOnCD disc, you will be prompted to add the DVD/CD disc as a repository (Synaptic will open once this is done), and you can then install the required packages directly from DVD/CD - in this case nothing will be copied to /var/cache/apt/archives.
Now if one wants the installed packages to also be copied to /var/cache/apt/archives, is it correct to say that one first needs to install the actual AptOnCD program on the fresh distro installation, then start AptOnCD >Restore >Load DVD/CD.
This will bring up the packages window from where one can select which packages to install AND they will be copied to /var/cache/apt/archives.
I am not 100% sure on this so please correct whatever is not true.

[Ocky]

-{ Quote: "
Try Remastersys, it's even easier to use, especially in backup mode. http://www.geekconnection.org/remastersys/remastersystool.html
" }-

If you are using Ubuntu 9.10 (Karmic) you will see that the install icon is missing when you boot into live cd. Typing ubiquity in the terminal doesn't help either.
This is because in Karmic the ubiquity-frontend-gtk package is not installed by default.
once installed all is well - an Install RELEASE menu entry will be placed in the System>Administration menu.

214864

[Trespasser]

-{ Quote: "That will be nice. wat0114 will be pleased to see that it comes with Linux Mint. May I ask you to kindly clarify/confirm some things now i.e. before starting a new thread ?

For instance if you had made the AptOnCD DVD for your distro and then want to do a fresh install of the same distro for someone (to save download time and MB's) ...
With a fresh install the AptOnCD Restore window (where you can select packages to restore) will probably not be shown because the AptOnCD program is not installed.
So, am I correct in saying that when inserting the AptOnCD disc, you will be prompted to add the DVD/CD disc as a repository (Synaptic will open once this is done), and you can then install the required packages directly from DVD/CD - in this case nothing will be copied to /var/cache/apt/archives.
Now if one wants the installed packages to also be copied to /var/cache/apt/archives, is it correct to say that one first needs to install the actual AptOnCD program on the fresh distro installation, then start AptOnCD >Restore >Load DVD/CD.
This will bring up the packages window from where one can select which packages to install AND they will be copied to /var/cache/apt/archives.
I am not 100% sure on this so please correct whatever is not true." }-

That's pretty well it, Ocky. You've obviously used it before.

.....Well....so much for starting a thread about AptOnCD. Ocky's description pretty much sums up it's functionality (in a lot less words than I would have required I'm sure). One tidbit though...use a CDRW disc where possible. I have AptOnCD discs labeled A-L sitting in a corner of my computer desk gathering dust.

[Ocky]

-{ Quote: "That's pretty well it, Ocky. You've obviously used it before.
" }-
Hi Trespasser,

No, only tested then abandoned - don't know why. Will try again now that you have given me the green light. Must insist on your tutorial, with screenshots, the whole caboodle plse.
;) ;D

[wat0114]

That AptOnCD seems to be rather slick, something I'll take a look at. As for Clonezilla, I'm now on beta 1.2.3.27 and no issues at all, as with the previous beta.

[RonCam]

-{ Quote: "... no issues at all ... [with Clonezilla]" }-Hi wat0114,
As an experienced Clonezilla user, I need your input: have you got any idea why after restoring both the SSD and partition (I have only one on my system) I should be getting just a GRUB prompt (grub >) and failure to boot any farther?

I started out restoring the full disk, then dropped back to the partition, hoping for a difference, but there was none.

I am a Clonezilla newbie, just using it for a few days by now.

System Info:
Hardware, ASUS L7014G
OS: Leeenux Ubuntu v2
Clonezilla: most recent stable version

[wat0114]

I'm not sure why that's happening to you, and I've never used a SSD before, but I don't know why that might cause problems. Where is the directory located that you're mounting to restore/backup the images from?

BTW, Mrkvonic provides an excellent tutorial on Clonezilla here (http://www.dedoimedo.com/computers/free_imaging_software.html) just in case you want to verify you're going through the steps properly.

[RonCam]

To answer your question, where is the directory:
It's in an external FAT32 USB drive and I'm letting Clonezilla go to its root directory.

I'll check the tutorial, but wanted to get back to give a reply, as your fast response is much appreciated. I've been banging my head against the wall for a couple of days on this one, since everyone says Clonezilla works so well,for them!

New information:
A Google search on
"clonezilla" "failed to install grub"
is getting lots of hits, but so far, the only suggestion is to reinstall grub from the command line. Nothing on why it occurs or how to prevent it.

Details on Clonezilla's error message:
file /tmp/hd-img.2#aJj5/boot/grub/stage1 not read correctly.
also:
Failed to install grub !!
Yes, in red and with the exclamation points.

I'm really a linux newbie so details are appreciated. ;D

[wat0114]

How do you have your drive partitioned? You mention only one, so is it not similar to the way I have mine in three partitions (within an extended partition) as root (/) swap & /home?. I have grub 2 installed in the root partition. When I use Clonezilla to backup/restore, I backup/restore the root and /home partitions separately. I've never had an issue with grub not restoring.

[RonCam]

-{ Quote: "How do you have your drive partitioned?" }-Sorry, I wasn't counting Swap. It's Root and Swap.

This is the default installation from the Ubuntu variation I mentioned, but I think it's the way Ubuntu installs normally?

I have heard a Home partition is a great idea, but I want to make sure I can image and restore, before I begin customizing. So in effect, I was imaging an OS that had been installed minutes earlier, with nothing done to it.

Just tried to restore again, with the BIOS set for Install (as opposed to finish) but got exactly the same error.

[wat0114]

I don't understand what's going wrong with the backup/restore process, then?? You must be using Clonezilla v 1.2.3.27? Canyou boot off the live cd and use Gparted to scan the drive? Does it show the Root partition with the "Boot" flag?

[RonCam]

-{ Quote: "Does it show the Root partition with the "Boot" flag?" }-You raised a very good point, as I just found that through extensive 'Googling' someone who had this problem said the same (http://bbs.archlinux.org/viewtopic.php?pid=665802#p665802), and that it solved the problem.

I suppose once it's set, it should stay set? This is pretty new to me. :P

I take a look at this ... learning as I go! Probably have to continue in the morning, as it's getting late here!

[RonCam]

Yes, it's set as boot.
And there are three partitions, the Ext4 (/dev/sda1), an extended and swap partitions.

I see that grub can be in two places: the 'normal' place (would that be outside of the partition??) or in a Superblock inside the partition. Could grub be somewhere, on my system, where Clonezilla doesn't expect it? If that make any sense (not really, to me) then does this point to anything in your experience?

[wat0114]

The only other idea I have is that maybe with your USB stick inserted, Clonezilla is seeing it as, maybe, hd0 and your root part as hd1, whereas your grub menu is assigning hd0 to the root, as it should, so somehow, perhaps, Clonezilla is messing up the grub before the imaging process by assigning the wrong hdx (where x is 0 or 1)? You can check your grub entries by opening menu.lst which might be under: /boot/grubmenu.lst. You can see in my ss where it's pointing at (hd0,4) which just means harddrive 0, partition 4. Just grasping at straws because I'm otherwise out of ideas. There are many others in this forum who have far more expertise in this area, so hopefully someone will post some ideas.

[RonCam]

Thanks much for your efforts -- I will try this and if problems persist, go to another imaging solution compatible with Ext4.

I note Clonezilla is aware of this problem, but doesn't know what's causing it, either. The only work-around on their web site is a (very complicated) method of recovering the system by reinstalling GRUB.

[wat0114]

Hi Ron,

there is something wrong it seems with the Clonezilla disk cloning function. It restored the Windows part just fine, failed on the Linux parts, successful on my Backups part, failed on my Storage part. The "Restore Parts" function has been quite reliable for me, except last week when it failed to clone to an equal size part on my other h/drive. I'd say Clonezilla is kind of buggy :( In fairness, though, I'm using the latest test version (1.2.3-27).

[RonCam]

-{ Quote: "I'd say Clonezilla is kind of buggy :( In fairness, though, I'm using the latest test version (1.2.3-27)." }-Clonezilla has lots of happy users, but I agree with your comment. When the disk image-restore damaged grub, I also tried the partition image-restore function, but it did the same: boot failed, again, at the grub prompt. As I said, time to move on ...

So, I downloaded the .iso for something called SystemRescueCD, a 'live' disk that permits running fsarchiver (Ext4 compatible imager), from its own GNU/Linux Command Line. I reinstalled the OS, imaged, then restored my L701 SSD with fsarchiver, powered it up and booted -- grub worked fine, and then Leeenux started perfectly, and continues to do so, more than 24 hours later.

Thanks for confirming what I was suspecting ... it's nice to know I'm not the only one! ;)

[edit: the beta version of SystemRescueCD is recommended, for the latest fsarchiver. Also, the name is fsarchiver -- I think fsarchive may be something else. ]

[tlu]

-{ Quote: "
there is something wrong it seems with the Clonezilla disk cloning function. ...
I'd say Clonezilla is kind of buggy :( I" }-
I'm not sure if Clonezilla is buggy, but you can certainly run into problems when cloning to different hardware. That's what I was faced with just 2 days ago.

I cloned my system from an old IDE disk to a new SATA disk (and enabled AHCI in the BIOS in order to benefit from NCQ (http://en.wikipedia.org/wiki/Native_Command_Queuing)). I booted from the new disk, the Grub menu appeared - but then I got error messages like "device /dev/disk/by-id/ata-Maxtor_6Y120P0_Y34449ME-part6 not found". This was a reference to my old harddisk.

Therefore I booted again from the old disk and mounted the new one. To fix the problems I had to do 2 things:
I went to /dev/disk/by-id in order to find the disk-ID for the new harddisk. Then I simply replaced the ID of the old disk with the ID of the new disk in the corresponding lines in /etc/fstab and /boot/grub/menu.lst on the new harddisk. Example: In fstab
/dev/disk/by-id/ata-Maxtor_6Y120P0_Y34449ME-part6
became
/dev/disk/by-id/ata-SAMSUNG_HD252KJ_S0NJJDWQ108187-part6
Then I checked which drivers for the disk controller were used by executing
hwinfo --storage-ctrl
I got several info blocks. One of them looked like this:
28: PCI 1f.2: 0106 SATA controller (AHCI 1.0)
[Created at pci.318]
UDI: /org/freedesktop/Hal/devices/pci_8086_2923
Unique ID: w7Y8.9YfA3CDazC2
SysFS ID: /devices/pci0000:00/0000:00:1f.2
SysFS BusID: 0000:00:1f.2
Hardware Class: storage
Model: "Intel 82801IB (ICH9) 4 port SATA AHCI Controller"
Vendor: pci 0x8086 "Intel Corporation"
Device: pci 0x2923 "82801IB (ICH9) 4 port SATA AHCI Controller"
SubVendor: pci 0x1458 "Giga-byte Technology"
SubDevice: pci 0xb005
Revision: 0x02
Driver: "ahci"
Driver Modules: "ahci"
I/O Ports: 0xe600-0xe607 (rw)
I/O Ports: 0xe700-0xe703 (rw)
I/O Ports: 0xe800-0xe807 (rw)
I/O Ports: 0xe900-0xe903 (rw)
I/O Ports: 0xea00-0xea1f (rw)
Memory Range: 0xfa006000-0xfa0067ff (rw,non-prefetchable)
IRQ: 28 (58575 events)
Module Alias: "pci:v00008086d00002923sv00001458sd0000B005bc01sc06i01"
Config Status: cfg=new, avail=yes, need=no, active=unknown

Then I looked into /etc/sysconfig/kernel. Therein I found the following line:
INITRD_MODULES="thermal pata_jmicron ata_generic ide_pci_generic processor fan"
Ah - ahci was missing! So I added that to the above list of modules.Now I booted from the new disk and all was well :)

Thus, if you know what to do you can get around this trouble although it's not trivial at all for a normal user. The problem is that you don't find any hints on http://clonezilla.org/ (at least I didn't find any) although you should run into these problems whenever you clone your system to a different drive (regardless if your distro uses disk-IDs or UUIDs - in the latter case you should execute blkid to find them out).

I'm not aware of an easier method. Can SystemRescueCD fix such problems?

[Mrkvonic]

In general, when cloning an image (whether CloneZilla or just tar/cpio) to other machines (hardware), rebuilding initrd is always a good idea. And using by-name notation, as then there are no confusions. If devices contain partitions that are formatted, then using the same inode size.
Mrk

[wat0114]

-{ Quote: " To fix the problems I had to do 2 things:..." }-

Yikes, that looks like a lot, although I could probably handle it, I suppose. Maybe it is a disk id problem but I don't know why it should be in my case using the same machine hardware and physically identical drives. This has never been an issue for me using Acronis or ShadowProtect. However, both h/drives are exactly the make/model, and I'm cloning within the same machine, so there are no hardware changes between clones.

-{ Quote: "In general, when cloning an image (whether CloneZilla or just tar/cpio) to other machines (hardware), " }-

So that's just it, as I mentioned above, it's all within the same machine/hardware, and even the drives are both WD Caviar Black, 500 GB drives. Unless, as what happened to tlu, the drives are different id's and Clonezilla doesn't like that?? IMO, Clozezilla should have been able to handle the cloning process. Funny that it handled the Windows 7 system partition flawlessly and one of the ntfs storage partitions, but not the Linux parts or the other ntfs storage partition :-\

[tlu]

-{ Quote: "In general, when cloning an image (whether CloneZilla or just tar/cpio) to other machines (hardware), rebuilding initrd is always a good idea. " }-
Yes - but is that really easier than the method I described above? I mean if you apply mkinitrd you have to chroot (http://www.dedoimedo.com/computers/grub-2.html#mozTocId842078) the system (which is a problem for most users), and if you don't know which options (http://linux.die.net/man/8/mkinitrd) to use for mkinitrd, you have to make the changes in fstab, menu.lst and /etc/sysconfig/kernel anyway.

I may be wrong but I think for the average user the steps described in my posting above might be more familiar than experimenting with chroot and mkinitrd.

[tlu]

-{ Quote: "Yikes, that looks like a lot, although I could probably handle it, I suppose. Maybe it is a disk id problem but I don't know why it should be in my case using the same machine hardware and physically identical drives. This has never been an issue for me using Acronis or ShadowProtect. " }-

For cloning Windows or for cloning Linux?

-{ Quote: " However, both h/drives are exactly the make/model, and I'm cloning within the same machine, so there are no hardware changes between clones." }-

Yes, but I think the IDs should be different anyway. You can check it by attaching both disks, mounting the second one and going to /dev/disk/by-id and /dev/disk/by-uuid.

[wat0114]

-{ Quote: "For cloning Windows or for cloning Linux?" }-

Cloning Windows. So does the disk id play a role in Linux cloning?



-{ Quote: "Yes, but I think the IDs should be different anyway. You can check it by attaching both disks, mounting the second one and going to /dev/disk/by-id and /dev/disk/by-uuid." }-

You are right, I remember seeing different id's assigned to them when using the Clonezilla disk.

[tlu]

-{ Quote: "Cloning Windows. So does the disk id play a role in Linux cloning?" }-

Yes, since the entries in fstab and menu.lst are no longer correct.

-{ Quote: " You are right, I remember seeing different id's assigned to them when using the Clonezilla disk." }-

Thanks for confirming this. Everything else wouldn't have made sense ;D

[wat0114]

-{ Quote: "Yes, since the entries in fstab and menu.lst are no longer correct.



Thanks for confirming this. Everything else wouldn't have made sense ;D" }-

Thanks for your help, tlu! It's too bad disk id's make all the difference using Clonezilla with Linux, so I will try what you did and see if that works.

[tlu]

-{ Quote: "Thanks for your help, tlu! " }- You're welcome. :)

-{ Quote: "It's too bad disk id's make all the difference using Clonezilla with Linux, so I will try what you did and see if that works." }-
Not only Clonezilla but any other alternative has the same problem. I am not aware of a backup/cloning program that automatically adjusts disk-IDs (and driver modules) to new hardware. Perhaps somebody else ...?

[Trespasser]

RonCam,
I've had that problem as well not too long ago. What distro are you using? What version of CloneZilla are you using? If Ubuntu, what version? The problem I had related to the version of CloneZilla I cloned with and was using. Grub was the culprit. I switched to a different version of Clonezilla and was able to restore with no problem. Your image is probably fine.

I've been using CloneZilla a long, long time and I've never had a problem that wasn't of my own doing. I hope CloneZilla soon includes btrfs as an option. I remember they were quick to adopted ext4.

[wat0114]

-{ Quote: " Grub was the culprit. I switched to a different version of Clonezilla and was able to restore with no problem. Your image is probably fine.
" }-

I'd say you're right about grub 2 as the culprit. There really is nothing wrong with the image. I used the grub 2 repair procedure as explained here (http://tolearnfree.blogspot.com/2009/12/how-to-fix-grub2-on-ubuntu-910.html) and the problem is solved. Also, I had to first use the second option in clonezilla to use disk-disk, then choose clone local partition to local partition to restore the root part to my target drive (this avoids the Clonezilla error: "failed to restore partition...maybe image is corrupt" message), after which I just got the "GRUB" prompt as RonCam got, then after I ran the Grub 2 repair procedure, all is good again :)

This has ended up being a nice learning experience, a good thread, even if frustrating at times. The important thing is I've learned something new Linux-related. Thank you to all who've responded and offered suggestions :) RonCam, good to see you battled through some adversity and you arrived at a resolution :thumb:

[Mrkvonic]

-{ Quote: "Yes - but is that really easier than the method I described above? I mean if you apply mkinitrd you have to chroot (http://www.dedoimedo.com/computers/grub-2.html#mozTocId842078) the system (which is a problem for most users), and if you don't know which options (http://linux.die.net/man/8/mkinitrd) to use for mkinitrd, you have to make the changes in fstab, menu.lst and /etc/sysconfig/kernel anyway.

I may be wrong but I think for the average user the steps described in my posting above might be more familiar than experimenting with chroot and mkinitrd." }-

You're right, too much for an average user.
I'll think of something that might make (easy) sense for cross-platform cloning.
Mrk

[tlu]

-{ Quote: " I used the grub 2 repair procedure as explained here (http://tolearnfree.blogspot.com/2009/12/how-to-fix-grub2-on-ubuntu-910.html) and the problem is solved. " }-

Yes, that's basically what Mrk described here (http://www.dedoimedo.com/computers/grub-2.html#mozTocId842078). It's the chroot alternative to my approach already discussed with Mrk. I don't know if it's really easier. And it's certainly not enough if new driver modules are required.

[wat0114]

-{ Quote: "Yes, that's basically what Mrk described here (http://www.dedoimedo.com/computers/grub-2.html#mozTocId842078). It's the chroot alternative to my approach already discussed with Mrk. I don't know if it's really easier. And it's certainly not enough if new driver modules are required." }-

Okay, I have an older super grub 2 disk but it doesn't have an option that I can see to restore the grub (**EDIT** oh wait, I just saw it is necessary to run a terminal command after booting to live environment). I'll give the new one a try. Thanks! Still, I'm not cloning to new hardware, only to another drive to be used in the same pc as the source disk, so I'm not convinced the Clonezilla clone procedure works properly with grub 2. There should be no need to change id's or run the procedure I ran. At least there are workarounds, though :)

**EDIT**

Finally, just to add that I've figured something else out which applies to my situation where I want grub 2 installed to the linux root part (sda6 my case) and use EasyBCD in Win 7 to control where I boot from, I had to change entries sda to sda6 in these steps:

-{ Quote: "run the command grub-install again to install grub to your /dev/sda

sudo grub-install /dev/sda
sudo grub-install --recheck /dev/sda" }-

otherwise grub 2 woild install to the mbr. Now I feel I understand all this perfectly - at last! :)

[RonCam]

-{ Quote: "RonCam,
I've had that problem as well not too long ago. What distro are you using? What version of CloneZilla are you using? If Ubuntu, what version? The problem I had related to the version of CloneZilla I cloned with and was using. Grub was the culprit. I switched to a different version of Clonezilla and was able to restore with no problem. Your image is probably fine." }-

Sorry, wasn't watching the tread for a while.:-[ But ... it's possible that none of the other variables mentioned above are the real cause/cure for the problem.

According to Clonezilla's author, who was kind enough to reply (https://sourceforge.net/projects/clonezilla/forums/forum/663168/topic/3569602) to my question, the answer is that the -g parameter in Advanced Mode (http://clonezilla.org/clonezilla-live/doc/02_Restore_disk_image/advanced/09-advanced-param.php) should be UN-checked prior to the restoration, if you [I]don't want grub overwritten.

If this is the reason, you had very good fortune to finally recover the back-up, just by trying so many different combinations. You may want to look at the -g parameter in the "different version of Clonezilla" that finally worked, and see what it's status is. Wonder if the default for this parameter could be set differently, on different versions of Clonezilla??

[apathy]

I've used Back in Time/Image for Linux for years. I can't wait for the new stable for Back in Time and maybe Tbiview/IFL will start working again as well.

[tlu]

-{ Quote: "I found a script on the German ubuntuusers Wiki that should perform something similar for rsync. You'd have to create the appropriate cron-job." }-
Below is an updated and improved version (here for backing up /home) for a Debian-based system like Ubuntu:

#!/bin/bash
# Simple backup with rsync

# SOURCES and TARGET must end with slash

SOURCES="/home/"
TARGET="/media/backup/"

# mountpoint must end WITHOUT slash
#MOUNTPOINT="/media/server"

LOGFILE="~backup.log"
EXPIREDAYS=0
RSYNC="--delete"

PACKAGES="/tmp/packages.list"
#MAILREC="root@localhost"

#SSHUSER=""
#SSHHOST=""
#SSHPORT=22

### do not edit ###

/bin/date > $LOGFILE

MOUNTED=$(/bin/mount | /bin/fgrep "$MOUNTPOINT");
if [ -z "$MOUNTPOINT" ] || [ -n "$MOUNTED" ]; then

if [ -e $TARGET ]; then
LASTBACKUP=$(/bin/ls -d $TARGET[[:digit:]]* 2>> $LOGFILE | /usr/bin/sort -r | /usr/bin/head -1)
fi

TODAY=$(/bin/date +%y%m%d)

if [ $EXPIREDAYS -gt 0 ]; then
EXPIRED=$(/usr/bin/find $TARGET[[:digit:]]* -maxdepth 0 -ctime +$EXPIREDAYS 2>> $LOGFILE)
for EX in $(/bin/echo $EXPIRED)
do
/bin/echo "rm -rf $EX " >> $LOGFILE
/bin/rm -rf $EX
done
fi

for SOURCE in $(/bin/echo $SOURCES)
do
if [ "$LASTBACKUP" ]; then
INC="--link-dest=$LASTBACKUP$SOURCE"
fi

if [ "$SSHUSER" ] && [ "$SSHHOST" ] && [ "$SSHPORT" ]; then
SSH="ssh -p $SSHPORT -l $SSHUSER";
SOURCEDIR="$SSHHOST:$SOURCE";
else
SOURCEDIR=$SOURCE;
fi
/bin/mkdir -p $TARGET$TODAY$SOURCE >> $LOGFILE 2>> $LOGFILE;
echo "/usr/bin/rsync -e \"$SSH\" -av $SOURCEDIR $RSYNC $INC $TARGET$TODAY$SOURCE " >> $LOGFILE 2>> $LOGFILE;
/usr/bin/rsync -e "$SSH" -av $SOURCEDIR $RSYNC $INC $TARGET$TODAY$SOURCE >> $LOGFILE 2>> $LOGFILE;
done

if [ -n "$PACKAGES" ]; then
echo "dpkg --get-selections | awk '!/deinstall|purge|hold/ {print $1}' > $PACKAGES " >> $LOGFILE 2>> $LOGFILE
dpkg --get-selections | awk '!/deinstall|purge|hold/ {print $1}' > $PACKAGES 2>> $LOGFILE
echo -e "/bin/mv $PACKAGES $TARGET$TODAY \n" >> $LOGFILE 2>> $LOGFILE
/bin/mv $PACKAGES $TARGET$TODAY 2>> $LOGFILE
fi

/bin/date >> $LOGFILE
/bin/cp $LOGFILE $TARGET$TODAY/backup-$(/bin/date +%T).log 2>> $LOGFILE

else
/bin/echo "$MOUNTPOINT not mounted" >> $LOGFILE
fi

if [ -n "$MAILREC" ];then
/bin/echo -e 'Backup is finished!\n'$(/bin/date) | /usr/bin/mutt -s "Backup" -a $LOGFILE -- $MAILREC
/bin/rm $LOGFILE
fi
It also creates a packages.list file with all your installed packages. In order to restore them you just would execute:

sudo dpkg --set-selections < packages.list
sudo apt-get -u dselect-upgrade

or alternatively by:

sudo apt-get install $(<packages.list)

Please note that the script above makes incremental backups by creating a new subfolder with the backup date under the target folder and uses hardlinks giving the impression that all backup files are contained in every daily subfolder. This saves a lot of space on the target drive, yet it is possible to copy any daily backup to another medium.

[tlu]

-{ Quote: "You're right, too much for an average user.
I'll think of something that might make (easy) sense for cross-platform cloning.
Mrk" }-

Mrk, I've just remembered this topic, and I would like to ask you if a solution has come into your view?

[tlu]

Since Clonezilla was mentioned in this thread, I'd like to add the well-known Terabyte for Linux (http://www.terabyteunlimited.com/image-for-linux.htm) as an alternative. I had bought the program some years ago but hadn't used it for quite a while as ext4 had nor been officially supported. Now that it does (plus ReiserFS) I've tried the new version 2.58, and it looks really good.

It offers some advantages over Clonezilla:
Terabyte has the option to validate your images (even byte-for-byte, if required). That's something I've always missed in Clonezilla.
It offers differential backups.
It offers to save the image as a "Single File Set" or, alternatively, as a "Multiple File Set" which is comprised of one image for every individual partition. This makes it easier to restore single partitions rather than the whole drive. (Clonezilla offers the choice to save the image as one file or as, by default, 2 GB files if I remember correctly.)
It contains the tbiview program that lets you see the files/folders in an image.
It offers several restore options (like geometry settings) that go beyond the ones offered by Clonezilla. They might be useful in specific cases.I can't say anything about the reliability yet, but the Terabyte products have gained great renown in the past.The price of 29.95 $ is okay for what it offers IMHO.

[wat0114]

Actually I've had repeat disappointments with Clonezilla; Try restoring backed up partitions to targets that are re-sized even slightly larger than the originals they were created form, and you get a "unable to restore, image may be corrupt..." or to that effect, when in fact there's nothing wrong with the images. I understand target parts can't be smaller than the original source, but no way in the world should it be impossible to restore to larger targets, but Clonezilla fails miserably here >:(

[tlu]

-{ Quote: " I understand target parts can't be smaller than the original source, but no way in the world should it be impossible to restore to larger targets, but Clonezilla fails miserably here >:(" }-

Hm, I'm not quite sure but I think that I successfully restored partitions to larger targets in the past. You have to resize them with Parted Magic after that, of course.

Have you tried a newer version of Clonezilla? Perhaps this was a bug in an older one ...

[wat0114]

-{ Quote: "Hm, I'm not quite sure but I think that I successfully restored partitions to larger targets in the past. You have to resize them with Parted Magic after that, of course.

Have you tried a newer version of Clonezilla? Perhaps this was a bug in an older one ..." }-

I've always used GParted to resize the parts and several recent versions, including latest 1.2.5-35, of Clonezilla have failed to clone to resized-to-larger-than-original parts.

[wearetheborg]

-{ Quote: "That will be nice. wat0114 will be pleased to see that it comes with Linux Mint. May I ask you to kindly clarify/confirm some things now i.e. before starting a new thread ?

For instance if you had made the AptOnCD DVD for your distro and then want to do a fresh install of the same distro for someone (to save download time and MB's) ...
With a fresh install the AptOnCD Restore window (where you can select packages to restore) will probably not be shown because the AptOnCD program is not installed.
So, am I correct in saying that when inserting the AptOnCD disc, you will be prompted to add the DVD/CD disc as a repository (Synaptic will open once this is done), and you can then install the required packages directly from DVD/CD - in this case nothing will be copied to /var/cache/apt/archives.
Now if one wants the installed packages to also be copied to /var/cache/apt/archives, is it correct to say that one first needs to install the actual AptOnCD program on the fresh distro installation, then start AptOnCD >Restore >Load DVD/CD.
This will bring up the packages window from where one can select which packages to install AND they will be copied to /var/cache/apt/archives.
I am not 100% sure on this so please correct whatever is not true." }-

I tried aptoncd. Is the apt cache cleared at periodic intervals? I had installed Debian a year back, and I have downloaded and installed many many deb packages. But I only see a few recent ones in AptonCD.

[wearetheborg]

Pity that clonezilla is buggy.

So is the current leade fsarchiver?

I have a dual boot system (XP+Linux) and I want to make a backup of the two systems. Will fsarchiver also backup grub? Or does it need to be manually installed and configured?

The scenario I am considering is that I have to change hard disk, with a different sized one, with different partitions; and I want to clone XP&Linux back to the new hard disk.

[wat0114]

-{ Quote: "Pity that clonezilla is buggy.

" }-

It seems to be near-flawless as long as one leaves the source partitions that the images were created from alone, to make for dependable restoring of images back to them.

[wearetheborg]

-{ Quote: "It seems to be near-flawless as long as one leaves the source partitions that the images were created from alone, to make for dependable restoring of images back to them." }-

Unfortunately my main use is to make backups in case of HDD crash/change; so source partitions would be different....

Does clonezilla work with dual boot systems? Can it restore XP?

What about tools like DriveXML or Macrium Reflect or Paragon? Will they work on dual boot systems?