Need the right Tool(s) to fix Hard Drive Nightmare

Discussion in 'all things UNIX' started by KookyMan, Aug 20, 2009.

Thread Status:
Not open for further replies.
  1. KookyMan

    KookyMan Registered Member

    Joined:
    Feb 2, 2008
    Posts:
    367
    Location:
    Michigan, USA
    Hey All... Having a problem, need some guidance to the 'right' tools to try and resolve an issue with minimal work, and minimal RTFMing.

    So far, when I've asked places, pretty much I get told RTFM. Or Google, then RTFM. I'm not opposed to doing my homework, but I'm at my wits end of people refusing to actually give me useful help "Fix it" don't count.

    Heres what I did, what happened, and where I stand now and what I am wanting to do (Suggest if there's a better way to do it.)

    I built a new rig, two 300GB Drives, two 1.5TB Drives.

    Linux is installed on one 300GB, then created two 1.5TB Partitions on the 1.5TB Drives using LVM. They share the same Volume Group (MainStorage), but I made them so each volume(Primary, Backup) is exclusive to each drive. (No split volumes.) At first I started copying files across my network via a a share. That ran for over a day and still only had moved half of the data. I stopped it, relocated my external to the Linux rig, and began a 'cp -aruv *' from the external to Backup. I did a little re-arranging of directories, then rsync'ed from Backup to Primary. I did some more significant directory resorting (massive changes to directory structure from external). Then rsync'ed from Primary to Backup with the new structure. I've set up Samba to share Primary over my network and started updating various files during 'live' use (Such as logs, saved downloads, etc).

    Last night I went to use an archive that I've had for over a year, and was having errors. After 45 minutes of searching, I discovered that the archive is corrupt. Both the Primary and Backup copies are corrupt. Luckilly, I have not deleted the source material (the external drive) yet, so I still have a working copy, which leads me to my question/current dilemma.

    The way I figure it, I need to do the following things:
    1. Determine what files don't match the external.
    2. Delete those files, provided they arn't newer. (Ie: updated logs)
    3. (Maybe) Push the smaller Updated files (logs) to the External?
    4. Recopy the files from the external back to storage
    5. Re sync the Primary Backup with the 'good' versions.

    Adding to the complexity is I now have 1.1-1.2TB of data on the Storage drives, but only 1.0TB of capacity on the external. I've moved quite a bit from a *Possibly* Failing Laptop drive. (Didn't know of this when I had started).

    Ok to summarize:
    I have:
    3 Drives.
    1.5TB, named Primary, that has 980GB of Data on it, partially (total amount unknown) corrupted, It has files on it that don't exist elsewhere (only copies of said files).
    1.5TB, named Backup, that has 950GB of Data on it, partially (total amount unknown) corrupted, and should be 95% identical to Primary, since it was an sync of Primary, but is now about 3-5 days out of sync..
    1.0TB, named FreeAgent(Seagate FreeAgent Drive), that has 920GB of data on it, Not Corrupt (To my knowledge), with a radically different directory structure. Outdated by 1-2 weeks at most.

    I want:
    To resync the files I have originals of on the FreeAgent to Primary, keep the newer or unique files on Primary with a minimum of data transfer (as transferring 1TB takes awhile even via Firewire).
    Determine the number of files that wound up different between the two drives. I realize that newer/updated/unique files will appear as 'different' but I can look and see based on path if its a updated/unique or if its corrupted. A number of the uniques are torrents, to which I still have the original torrent file, and as such I can use my torrent application to do consistency checks to check for/correct corruption.

    Any/All help is greatly appreciated.

    ---

    Update: Ran Rsync while I was away today with -nacvy against a directory with 346 files it (162GB). Of those, ~60-65 files are corrupted. If you don't have ideas as to what may have caused it, ideas on cause?
     
    Last edited: Aug 20, 2009
  2. KookyMan

    KookyMan Registered Member

    Joined:
    Feb 2, 2008
    Posts:
    367
    Location:
    Michigan, USA
    Finally found out the cause.

    It was apparently a bad BIOS / Bad BIOS Setting. I have an ASUS M4A78T-E board (Two of them actually) with BIOS 1001, and tweaked some settings (No overclocking, just other settings). Caused all my problems. I flashed to the current 2001 I think, and left the settings pretty much default and all my problems have gone away. Not sure if it was a bug in the BIOS or a bad setting. I might tinker later and see if I can reproduce it with the new BIOS, but just FYI for anyone else who might run into it.

    (I have a similar setup running Vista Business 64, and haven't experianced any issues, so I don't know if its a Linux issue or what.)
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.