Backup & Recovery 10 Free Edition Warning: This is long because I tried lots of different methods to troubleshoot this problem. Paragon Backup & Recover 10 Free Edition Windows XP Pro SP-3 Did a full backup which completed with no errors. The program option to "Control integrity" was enabled which means checksums of the backed up files get saved into the archive. After the full backup was done, I ran an archive integrity check. It failed with: Operation failed Invalid archive file Error source: Hard Disk Manager Error code: 0x10025 No other information was provided in the popup error dialog. So what is the "Hard Disk Manager"? Is that some component of B&R10Free? Just where am I supposed to lookup the error codes (since they're used instead of providing plain language descriptions)? Note that before I performed this full backup that I already checked my hardware. I had ran memtest86 for 6 passes with no errors, Windows Memory Diagnostics for 5 passes with no errors, and Western Digital's Data LifeGuard Diagnostics found no problems with its quick and full check of the hard disk purposed only for backup storage. So it's not the hardware at fault. I rebooted into Safe Mode but the integrity check still failed. I did another full backup while in Safe Mode and then did an integrity check on that one. It failed, too, this time noting: Integrity error has occurred Archive integrity error between sectors: xxxxxx and xxxxxx! So is this an error reading sectors off the disk where the file was copied? Or is this a mismatch between the newly generated hash value for a file within the archive and the checksum that was stored in the backup (i.e., are these sectors from the image in the backup file)? From what I’ve read, causes for the integrity/verification/validation/or-some-other-name test to fail are: Bad memory: corrupted content in memory could result in invalid data used in the compare. Thorough memory testing has ruled this out. Bad hard disk: sectors/bytes won’t read reliably or at all. Thorough disk testing via “chkdksk /r” and using WD diagnostics rules this out. Also, other file operations on this same hard disk (the one used for the backup store) have not resulted in corrupted or damaged files. The checksum on the file(s) saved in the archive file generated during the backup process were invalid. No way for users to test this cause. If the checksums stored in the archive are wrong, users have no way to fix this. There is no utility provided that will generate a checksum of the file/sectors stored in the archive file that operates separately of B&R10Free to see what it generates and then extract the checksum that got stored in the archive for when the file got backed up and put into the archive file along with generating a checksum on the original (and unchanged) file that got backed up. With these 3 checksum (on original file, on archived file, checksum stored in archive), the user could see if it was the stored checksum in the archive that was wrong or if the checksum for the original and archived files were really different. It is unclear how the integrity check works. It may generate a new checksum from the archived file/sectors and compare it against the checksum value created during the backup that got stored in the archive. This checks only the archive contents are valid but works only if the stored checksum is the actual checksum of the content that got stored in the archive. It also presumes that the EXACT algorithms used to compile the checksum during the backup is the same used when performing an integrity check. This scheme would be faster because only one checksum need be generated: the one for the archived content. It may generate a new checksum against the archived file/sectors against a new checksum generated on the original file. The problem here is the file may have changed since the backup. Even if the file didn’t change, the slack space in the cluster isn’t part of the file but its content could change on the hard disk versus what was in the slack space when the sectors got backed up. This would be a slower integrity check because 2 checksums would have to get generated: one on the archived file and another on the original file. The new checksum generated from the files/sectors stored in the archive file don’t match the stored checksums in the same archive file. Could be the method used to generate the checksum during an integrity check results in a different value than how the checksum was computed during the backup. Could be the file was stored incorrectly into the archive file. That is, the file on the disk is okay but its copy in the archive is different (i.e., corrupted). Could be the archive file got corrupted sometime later; however, I’m doing the integrity check immediately after a backup completes. Could be anti-virus software is interfering with generating correct checksums; however, disabling the AV program or a reboot into safe mode where the AV isn’t active indicates this is not the source of the problem. Another cause could be the compression. If the compression doesn’t permit an accurate calculation for a newly generated checksum on the contents of the archive then they won’t match against the checksums stored in the archive. As noted in #3, the user has no means of separately generating checksums on the original and archived files/sectors and to extract the stored checksum to see which ones match and which don’t. Although B&R10Free is supposed to revert to using its own snapshotting driver if VSS fails, there is no indication given in the progress window during the backup to indicate that the backup program has discontinued using VSS (Microsoft’s Volume Shadow [Copying] Service) and switched to Paragon’s driver. VSS has been shown to have problems. Perhaps VSS has improved in later NT-based versions of Windows (Vista and 7) but I’m still back on Windows XP. VSS won’t work with FAT16/32 and requires NTFS (on, I believe, at least one partition). On all 3 hard disks, there is just 1 partition that spans the entire hard disk and NTFS is used in each. To test if VSS is the culprit, I configured B&R10Free to use “Paragon Hot Processing” driver instead of VSS. When I started the backup wizard, I checked the option to view that job’s options and “Paragon Hot Processing” was selected. After the full backup completed, a following integrity check still failed. I noticed that I was selecting the entire archive to include in the integrity check. So instead of picking the archive (which was a disk backup), I just picked the volume (C that spans that entire hard disk. Nope, the integrity check failed, too. As yet another test, I did backups with no compression. If the compression scheme is unreliable or otherwises causes problems in generating new checksums against the archived files/sectors then there will be a mismatch against the checksum generated at backup time that was stored in the archive. The disk properties showed 28GB was used and the prior full backups were creating 16GB “.000” files (I disabled image splitting since the backup location is a hard disk using NTFS). So an uncompressed backup would take longer but finish in maybe under an hour (the compressed full backups were taking 28 minutes). Without compression, the full backup now only took 18 minutes. Shorter backup time but a larger archive file (but only 20GB versus the 28GB consumed in the volume). The integrity check failed. I choose not to split up the archive into numerous files because it shouldn’t be required. A separate hard disk was purposed solely to store backups. It had only 1 partition which was NTFS formatted. However, if B&R10Free has problems reading huge files then it might be generating invalid checksums on its archived content. With the image split up into 4GB chunks, a following integrity check still failed. Another test was to not select the entire partition to backup but just the volume. That is, instead of selecting the disk (with its 1 partition, 1st track, and MBR), I just selected the volume (C to backup. The integrity check failed. Despite the integrity check failing and claiming the archive is corrupted, I tried pulling all the files out of the archive and saved them to an alternate location. I right-clicked on the archive and mounted it as drive N: and ran “xcopy n: c:\tempx /e /h” (since C: had enough free space to hold all the files). No errors reported by the xcopy operation. The xcopy command failed but not for what appears to be a “cannot read sector” or checksum error. The error after copying over 10,000 files was: …(other files)… Nocuments and Settings\lee_hodsdon\My Documents\My Videos\Desktop.ini Access denied Unable to create directory - C:\tempx\%userprofile%\My Documents\My Virtual Machines 10837 File(s) copied I don’t quite know if the error was bitching about the desktop.ini file or “unable to create directory”. This folder is not really a folder. It is a junction point. That is, it is a link that points somewhere else. VirtualPC 2007 wants to dump its virtual machine definitions under My Documents and I don’t want them there. Even if while defining a VM it is told its store folder is elsewhere, VPC2007 still creates this subfolder under My Documents. To alleviate it into thinking this documents folder already exists but actually have the VMs stored elsewhere, I created a junction point. VPC2007 sees the “folder” but which is really located elsewhere. This junction point on the C: drive under My Documents points to a folder on my D: drive. %userprofile%\My Documents\Virtual Machines (this is on drive C is a junction point to: D:\VirtualMachines\VirtualPC I can see why xcopy failed because it cannot create junction points so it cannot create the “folder”. I have to wonder if B&R10Free (and B&R2010FreeAdv that I tried earlier and had the integrity errors) don’t know how to properly handle junction points. For one, they should NOT follow junction points as this can lead to redundant data in the archive. As another example, I have a junction point under My Documents which points to C:\References. If the backup program follows the junction point then it would backup the files under there and later backup again the files it found under C:\References. While the expectation is that a restore could recreate junction points because they are inside of NTFS that doesn’t mean the backup program actually does handle junction points. I know that Acronis True Image does NOT follow junction points (but it does create them successfully on a restore). So I know why the xcopy aborted when it hit a junction point (but it did manage to copy the 3GB of files before it hit this junction point. I then ran xcopy on the rest of the 14GB of files in N:, the mounted archive which copied okay. So other than the junction point problem (something xcopy doesn’t support), all the files from the archive copied out okay. Maybe Paragon’s offerings don’t handle junction points, especially if they span across volumes. I have no spare drive or another same-hardware host in which to test restoring the entire archive (which is a disk image with 1st track, MBR, and volume). So I can’t do a disk or partition restore, just load the archive as a drive and copy from there to check if there are any errors accessing and copying the files out of it. As a last resort, I rebooted using the recovery CD. Perhaps the snapshot “technology” (Microsoft’s VSS or Paragon’s Hotshot) really aren’t stable. After all, they are trying to backup an active partition versus making sure it is completely quiescent during the backup. After I booted using the recovery CD, and after a length wait to get past a progress bar and then a blue screen after selecting Normal (i.e., the blue screen has no progress info), I tried to check integrity of the full backup. Got some error about invalid operation code. So I created another full backup using this boot-time version of the program. After it completed, I tried an integrity check but still got the invalid operation code error. Well, that was a wonderfully worthless tool. Figuring the backup created at boot (outside of Windows) would be of a quiescent partition where no changes were occurring and no snapshotting technology need be used (VSS or Hotshot), I rebooted into Windows and ran an integrity check on that backup. It failed, too. I’m not getting any errors during the backups. It is only afterward when I later run an integrity check that the product claims the archive is corrupted. Something else that I noticed is that the hard disk is properly identified when I selected it for backup. After the backup completes, its archive is listed as “Basic Hard Disk 1 (WDC WD1600JS-00NCB1 SCSI Disk Device )”. It’s a SATA drive and those are seen as SCSI devices. Yet when I go to run an integrity check, and when selecting the archive to test, it says “Basic Hard Disk 1 (Unknown Model)”. So the disk info is present in the archive but the integrity check suddenly can’t figure out what it was from the archive. I’m beginning to strongly suspect that either the integrity check doesn’t compute checksums using the exact same mechanism as used when doing the backups and storing the file/sectors inside the archive or the checksums computed at backup time and stored in the archive are wrong. This isn’t new. I forget which backup product it was many years ago but I happened upon one that was storing incorrect checksums in the archive which meant the verification or integrity check would always fail. It didn’t store the wrong checksum all the time. There were specific criteria found under which the wrong checksum got computed and then stored in the archive. The integrity check appears worthless because it cannot be relied upon to correctly validate an archive. It looks like I can extract all the files out of the archive. I can’t use the product’s own integrity check to verify the archives are usable. Well, I certainly don’t want to find out months later or even the next day that an archive is unusable because the integrity check is unreliable.