Matt Spathes: Is this misleading or not?

First, thanks for providing support on this forum and eliminating some of the speculation that occurs when users-only provide answers.

Below is a section from the B&R 12 Home manual but it exists in other versions and products as well.

This section contains a set of options that will be taken into account during backup/restore operations:
 Control archive integrity. Mark the checkbox to guarantee that all backup images created with the program are
100 percent flawless. If you decided not to control the archive integrity, the backup operation would take about
3-5% less time. ​

I would like to know if the statement concerning guaranteeing the images created are flawless is really true or is it misleading.

My understanding is: it allows checksums or other controls to be inserted into the image when it is created. If the integrity of the image is not checked by Verifying after creation then it does nothing for guaranteeing the image is flawless. If this is correct then the statement is seriously misleading because doing a restore or verifying before using an image to do a restore is not the time you want to find out it has an error.

If the image does need verifying as I suspect, why isn't it available as a "verify after creating" command that can be set via a checkbox in the image creation process?

Thanks for your help.

 

Hopefully, this will answer your question, but to explain it a bit better, that "archive integrity check" simply validates the image at the time of creation of the image, which performs a MD-5 integrity check, which validates the image in the process, while not altering the actual backup in any way. Other outside factors can still cause an image to be corrupted, i.e. pulling a drive without "safely removing", powering down a drive (hard shut down), interruptions in power without having a surge protector present are just some of the things I have seen cause corruption to an archive.

The actual "verify after creating" command that can be set via checkbox is a request that has been put into development and will eventually work its way into a future product, but otherwise you can always script that verifying process in, after the fact or run the archive integrity check which can found within the "advanced or full scale launcher" under the wizards tab under "backup utilities", from there simply choose check archive integrity.

Hopefully, that helps answer your question.

 

Thanks for the answer.

I don't think I'm totally clear on exactly what is happening but I get the impession that this does not confirm that the archive is accurately readable from the disk at the time of creation, such as in the case of the archive being written over an unknown bad cluster or a flakey disk cable. The "3-5%" increase in time mentioned is too small since implementing the "Verify" command takes about 80% of the image creation time.

I've had both the bad cluster and flakey disk cable problems with a competing product but they were picked up with the actual verify command.

I hope the Control Archive Integrity does give me confidence the archive is indeed properly readable from disk but right now I'm not sure.

 

As I've mentioned before I've had a HD with no file, etc. problems and archives that passed integrity check fail during the restore process. This is after the "same" archive successfully restored on a earlier date.

 

I had that with Acronis and the problem very definitely was that the disk holding the archive developed some bad sectors. It had verified right after creation but when I wanted to use it, it was bad. A couple of others which also had been verified were bad so I had to go back 3, IIRC, before I got one that hadn't been affected.

This says never to have only 1 backup, keep as many as you can and even go as far as having more than one device to store backups. Keeping all archives on 1 device is a single-point of failure problem.

 

Ultimately, everything has a point of failure it is why they recommend creating backups to multiple devices, HDD, or NAS devices. It is why redundancy is so important. I myself have backups saved to 5 different devices, 2 usb 2 ext hdd, 1 esata, 1 NAS, and 1 USB 3, now not all of them are up to date but at least I would have something if something went wrong.

When it comes down to it like I said before corruption can occur in other ways, it's why I create so my backups, but again that's just me personally.

 

I fogot to mention also that at the same time of the failure I also had a Windows 7 backup created at the same time on another external HD which failed also.

Paragon HDM11S stopped stating that it had a problem with some files with an option to look at the files but that window was blank. It also wanted to reboot but this was after some time into the restore but I believe I tried that but of course it didn't reboot. Tried a second time and it stopped at the same place with the same options. IIRC, I tried two more archives before one finally worked.

 

This topic is interesting. One thing that's still not clear to me is if it's possible that the MD-5 integrity check says that the image is ok while there are errors in the hard drive.

That is, if the image has been stored in flawed part of the hard disk, will the integrity check fail and thus alert the user about it?

 

That is indeed the queston and personally I don't think so. AFAIK, when PCs write files they do not pick up problems with a bad sector - how many problems have you encountered when writing a file assuming the disk is basically operational? None, in my case, they all are brought to light when the file is read which is the time the data is assembled and a CRC check is done by the disk hardware. In order to keep the speed high, computer hardware does not do a read-after-write check of the data which essentially doubles the time of a write.

The amount of time increase of 3-5% in image creation when the "Control Archive Integrity" is used is too small to do a check from the disk and I think it is creating the checksums placed in the image so it can be verified later and while this may do some degree of checking when the image is being assembled I don't consider it in the same category of a Verify which does read the disk, puts the data into RAM and re-computes the checksums from the data and compares them to the ones stored when the image was created. This is actually a pretty good test of the disk subsystem and RAM of the computer in addition to verifying the integrity of the image.

 

What´s the difference between verifying the image after its creation when the option "Control Archive Integrity" was used, and when it wasn´t?

Is the verification faster, or more accurate, when "Control Achive Integrity" was used? Or this option doesn´t alter the verification process at all?

 

Well, I did a test with B&R 11 Home and it just made it more confusing. Running 32-bit W7 nothing else running but idle network stuff and email program polling every 10 minutes.

I have a small partition with about 13.7GB used so I used it to make some images since the time would be fairly short. One image was made with the Check Archive Integrity box (via advanced options in the wizard) unticked that is no archive check. The image was then put through the Image Verify wizard.

Exited Paragon and restarted it in case there was any caching.

Made image again but this time I ticked it in the Advanced options in the wizard. Then the image was verified like the other image.

Create Images:
Wall-clock time to create image, Check Archive Integrity unticked: 3min59s
Wall-clock time to create image, Check Archive Integrity checked: 3min53s
Should have been other way around but I saw the same thing several months ago. Anyway the times are very close to identical!

Verify Images:
Verify image, Check Archive Integrity unticked:
Wizard reported: " The selected archive does not contain any data to control integrity as the required option has been switched off when creating it.
To Check Archive Structure instead please press Next"

This made sense since the box was unchecked. I pressed next and it took about 2min 48s to do the Archive Structure check. It said the archive was OK but what did it really check? Given the time is comparable to the Check Archive Integrity ticked case, I would guess it made sure it could read the archive off disk without a disk controller CRC error.

Verify Image, Check Archive Integrity ticked:
No message like above. Time to verfiy: 2min 30s by wall clock. So again is faster when you would expect the opposite.

Wondered if there was data included in the Check Archive Integrity ticked version that would help in reconstructing a damaged archive. One could infer this from the "archive does not contain any data to control integrity". Checked the file sizes of the archive files.
Archive Integrity Unticked: 10,407,192,044 bytes
Archive Integrity Ticked: 10,407,191,855 bytes
So there are 189 fewer bytes in the file that you might think should be larger. This certainly eliminates the idea of any error-correcting data being written and in fact rules out the idea that checksums are being included.

The small 172 KB .PFM file is 20 bytes larger in the Archive Integrity ticked case.

So I still have the question. What does ticking the box really do for you and how does it affect the stand-alone Verify function when it is not ticked? Hopefully this will answer the question why does everything look either backwards or as if it doesn't matter!

 

I did a test using HDM 12 from a WinPE USB key (with USB 3.0 drivers).

Although I usually create images with the option "Control Archive Integrity" enabled, this time I disabled it.

The description of this option in this program is: "Choose this option to allow writing of specific data that will later be used during restore to check the archive integrity. It can slow the backup operation".

According to this, the data added by "Control Archive Integrity" is only used during restore.

The image of the Windows 7 x64 installation (no data) was created in an external USB 3.0 disk, connected to an USB 3.0 port. The size was 13.88 GB, the time 4:58 min (47.7 MB/s average).

The time used to create the image was about 17% less than the time I´ve got recently with the option enabled, using the same disks and connections.

The image was verified in 3:00 min. No warning or special message appeared.

 

Your result of it being faster matches what someone else reported some time ago and has not been my experience. It also conforms with what is supposed to happen according to Paragon. Perhaps B&R11 has a bug.

In addition, my filesize with the supposed extra info included is not larger to incorporate it and the difference is only 189 bytes different.

As you point out it is to be used during the restore which means it does nothing for you to detect that the archive can be correctly read off the storage media so it seems you still need to do a verify to have higher confidence. The statement you referenced says that it is used to check archive integrity during a restore - not correct any data problem with the archive such as an error-correcting mechanism would provide.

My issue with checking during the restore is that it is not the time you want to find out by integrity checking the archive is flawed. You want to know when you create the archive so you can make another one if necessary.