ShadowProtect Incremental Backup Problem!

Discussion in 'backup, imaging & disk mgmt' started by Raza0007, Dec 3, 2009.

Thread Status:
Not open for further replies.
  1. Raza0007

    Raza0007 Registered Member

    Joined:
    Mar 30, 2009
    Posts:
    1,425
    Location:
    USA
    I have been trying ShadowProtect, especially the incremental backups and I have encountered a problem. I am unable to explain it myself and need some expert advice.

    I will explain with graphics as it will be easier.

    I setup a monthly schedule with full backups on every 29th and then just 1 incremental every day at 10:00 pm. Since the date had already clicked over and it was 30th Nov, so I took a full backup manually, and then let the scheduler take the incrementals every day at 10:00 pm.

    -----

    Full backup: See image below
    The full backup took 17 minutes, and generated an image sized 12.3 GB. Ignore the first full backup, it was just a test and I deleted the resultant image afterward

    00-FullBackup.jpg


    ---

    1st Incremental: See image below
    The first incremental ran at 10:00 pm on 30th, took 47 seconds and generated an image sized 119 MB.

    01-Ist incremental.jpg


    -----

    2nd Incremental:
    Ran on 1st Dec at 10:00 pm, took 3 minutes 15 seconds and generated an image size of 224 MB. Detail log was similar to the 1st incremental.


    -----

    3rd Incremental: See image below
    I took the third incremental manually at 11:26 pm same night. It took about 30 seconds to run and generated a file size of 20 MB. Notice the details say it was a manual incremental

    03-3rd incremental.jpg


    ----

    4th incremental: See image below
    The problem occurred on this fourth incremental run, which ran on the 2nd Dec at 10:00 pm, took 52 minutes!! and generated an image size of 295 MB. What went wrong? I mean 52 minutes! My full backup took only 17 minutes. Notice there is now a change in the details section of the log. 4th line says "VDIFF tracking is off" then "VDIFF tracking turned on". Also missing are the generation count and %of updated sectors.

    04-4th incremental.jpg

    ------


    5th Incremental:See image below
    Ran on 3rd Dec at 10:00 pm, took 31 minutes and generated an image size of 170 MB. Detail log again had "VDIFF tracking is off" "VDIFF tracking turned on"

    05-5th incremental.jpg

    -----

    What did I do wrong?

    Was I not supposed to take the manual incremental? Did the manual incremental break the scheduled incremental process?

    Since my first full backup was taken manually, so I do not think taking a manual incremental in between should have caused any problems. I would like to take manual incremental in between the scheduled ones from time to time.

    Why did my 4th and 5th incrementals took 52 minutes and 31 minutes respectively to run?

    Any opinions on my problem? Any solutions?

    Thanks.
     
  2. Peter2150

    Peter2150 Global Moderator

    Joined:
    Sep 20, 2003
    Posts:
    17,047
    i can't say before as I've never run manual incrementals. I run the continous incrementals thru out the day, and I don't mess with them. Sometimes a reboot or something major on the system will cause a real slow next incremental.

    If I want to do a manual image, I kill the incremental job, do what I want, and then restart the incremental job as a new one. Don't know for sure if it's necessary, but I do it that way.

    You might post this on the storagecraft forum and see what they say.

    Pete
     
  3. Raza0007

    Raza0007 Registered Member

    Joined:
    Mar 30, 2009
    Posts:
    1,425
    Location:
    USA
    Thanks, I will look into it further. I think the problem occurred when I took a manual incremental. The first three log details say "image will be created by VDIFF" whereas the next two problematic job logs state "incremental image will be created from existing full by using DiffGen".

    So, most likely, the 4th and 5th were actually incremental from the first full, in other words they were differentials.

    But the question why did it happen this way. I will start fresh and see if taking a manual incremental again ruins the scheduled incrmentals.
     
  4. 67mustang

    67mustang Registered Member

    Joined:
    Sep 2, 2008
    Posts:
    1
    ShadowProtect uses two methods for creating incremental backups. The first is the fastest method, in which sectors are tracked as they changed. This type of incremental occurs very quickly and uses very few system resources.

    The second type of incremental is created using 'diffgen'. This type of incremental is triggered when the sector tracking process has been interrupted (usually by a 'dirty' shutdown / hard reboot). When this process has been interrupted, ShadowProtect compares all of the sectors on the hard drive with the sectors contained in the last backup. The result is a small incremental file, but the process takes longer and is more resource intensive.

    Your scenario doesn't have anything to do with whether or not the backup was started manually or automatically. It has to do with the sector tracking process being interrupted.

    Hope this helps!
     
  5. Raza0007

    Raza0007 Registered Member

    Joined:
    Mar 30, 2009
    Posts:
    1,425
    Location:
    USA
    Yes, I did have a dirty shutdown two days ago, with a blue screen and all. So, SP will switch to this only when the dirty shutdown happens while it is running or will it do that when ever there is a dirty shutdown?
     
  6. LenC

    LenC Registered Member

    Joined:
    Jul 25, 2006
    Posts:
    846
    Location:
    CT, USA
    Raza0007 -

    I have a schedule for a weekly full backup and then daily incrementals. I had some occasional inexplicable results such as yours - I came to the conclusion they were caused by an option that causes a missed scheduled backup to run automatically. I'm pretty sure this was my problem, because I have had no problems since I unchecked that option.

    I think this scenario is what caused my prior problems - let's say the computer is turned off at a scheduled backup time and the backup is missed. I boot up the computer at a later time and am not aware the missed backup is now triggered and running. I turn off the computer before the backup is completed - causing the dirty shutdown as 67Mustang mentioned. I believe that scenario created problems for me with regard to subsequent scheduled backups.
     
  7. Raza0007

    Raza0007 Registered Member

    Joined:
    Mar 30, 2009
    Posts:
    1,425
    Location:
    USA
    I had that option unchecked already as I did not want the missed schedules to run as soon as I log back in. Most likely my problem was caused by a sudden system wide shutdown. A blue screen appeared and I was told there was a possible hardware/driver conflict and the pc rebooted. Now, I do not recall if SP was running at that time. That is why I wanted to know will it happen even if SP is not running in the background?

    I have seen this blue screen thrice last month and it is usually a signal that I need to reformat and start from fresh. I will probably do that soon.
     
  8. bgoodman4

    bgoodman4 Registered Member

    Joined:
    Jan 13, 2009
    Posts:
    3,131
    I remember seeing this option but cannot find it now. I am not sure if I have it selected or not but I suspect it is. Could someone tell me where to find the check box please?
     
  9. DLE

    DLE Registered Member

    Joined:
    Oct 6, 2008
    Posts:
    37
    bgoodman4,

    Edit Backup
    Continue Until You See Advance Options
    Select Advance Options
    Select Image Tab
    Select / Deselect Auto execution of unexecuted task
     
  10. bgoodman4

    bgoodman4 Registered Member

    Joined:
    Jan 13, 2009
    Posts:
    3,131
    Thank you.
     
  11. Raza0007

    Raza0007 Registered Member

    Joined:
    Mar 30, 2009
    Posts:
    1,425
    Location:
    USA
    Okay, I did a little more experimental research on my problem. It is indeed related to a dirty shutdown as pointed out by 67mustang. It is not related to taking a manual increment. You can take as many manual increments as you like, it will not effect the scheduled ones. Following might also be informative to other ShadowProtect users.

    If you have setup an incremental option in SP, then you need to be aware that if there is a dirty shutdown then SP will lose its sector tracking ability. The immediate next incremental (whether scheduled or manual) will not be quick and fast. In fact, it will be even slower than the full backup on that particular partition. Other incrementals that follow this one will be normal quick and fast. I have checked this by inflicting a manual dirty shutdown on my PC.

    If the log detail says "incremental is created using VDIFF" then the incremental will be created using the sector tracking ability of SP and will be quick and fast.

    If the log details says "incremental is created using DiffGen" then the incremental will be slow and will take longer than a full backup to run on that particular partition.

    It is related to the SP's "Enable Self-healing Incremental Recovery" option. I will quote the extract from their help file which seems to be self explanatory.


    My problem detailed in the first post seemed to be solved.

    Thank you for participating.
     
  12. LenC

    LenC Registered Member

    Joined:
    Jul 25, 2006
    Posts:
    846
    Location:
    CT, USA
    Raza0007-

    Very helpful information - thank you!
     
  13. bgoodman4

    bgoodman4 Registered Member

    Joined:
    Jan 13, 2009
    Posts:
    3,131
    Thank you from me as well. Good to know before there is an incident rather than after.
     
Loading...
Thread Status:
Not open for further replies.