TrueCrypt container - determine files affected by bad sectors

Discussion in 'encryption problems' started by lechuza, Jul 8, 2014.

Thread Status:
Not open for further replies.
  1. lechuza

    lechuza Registered Member

    Joined:
    Jul 8, 2014
    Posts:
    4
    Greetings,

    Could someone tell me how to determine which files are affected by bad sectors in a simple NTFS TrueCrypt (TC) container? I know which sectors are bad should that help.

    Here are the details:

    I created a raw, sector-by-sector image of a failing drive. There are dozens of unreadable 512-byte sectors. I have a list of the bad sectors' LBAs. I have determined that all the bad sectors are in a large, simple NTFS TC container. All of the bad sectors are zeroed in the image.

    I can open and mount the TC container, browse the filesystem, and examine files. How can I determine all files in the TC container affected by the zeroed sectors? I'm guessing that the knowledge of which sectors are zeroed is useful, but I could use some guidance in how to apply that or any other techniques that anyone would like to offer.

    I'm working on a Linux system but thought about booting an MS Windows system and running chkdsk on the mounted container's NTFS. However I think that would only check the filesystem's integrity and not identify any damaged files. Is that correct?

    I have a brute-force idea that I'd prefer not to use: mount the TC container from the failing drive and copy each of the thousands of files to a good filesystem, noting which files generate a read error. I'm hoping for a more intelligent approach, in particular, one that won't do further damage to the failing drive.

    Thanks.
     
  2. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    991
    Location:
    Hawaii
    Wow, sounds like fun! I can think of ways to do this using WinHex, but it's going to be fairly tedious, as each "bad" sector will have to be navigated to manually in order to determine which file it might contain a portion of. (WinHex can list the file's pathname whenever you place your cursor within a sector that contains data). How many "previously-bad, now zeroed" blocks do you need to examine? You might want to focus on finding an automated solution, but if you wish I can post a manual method.

    But before I do, please answer one question: Is the TrueCrypt volume partition-hosted (i.e. a fully encrypted partition), or file-hosted (i.e. a file container)?
     
  3. lechuza

    lechuza Registered Member

    Joined:
    Jul 8, 2014
    Posts:
    4
    Thanks for your reply and offer. It's a file container with a single (no hidden) NTFS. There are 80 zeroed sectors within a fairly close range (< 256 KB).

    I thought I'd write a script to use "hdparm --read-sector" on each bad sector of the failing drive as a last try to recover more data. From the man page, "This can be used to definitively check whether a given sector is bad (media error) or not (doing so through the usual mechanisms can sometimes give false positives)."

    Do you have an automated solution in mind? I haven't used WinHex but I'm willing to dig into it. However it appears to only be available for MS Windows. Are you aware of an equivalent program for Linux? If not, I can find a Windows system to use it. Do you know if the evaluation version of WinHex provide the features required by your manual method?
     
  4. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    991
    Location:
    Hawaii
    Short version:
    WinHex (as well as other hex editors) allows you to view the contents of any sector. If the sector happens to contain a portion of a file then under most conditions that file's pathname will be displayed in the sidebar.

    The only tricky part will be converting your list of physical sector numbers into the relative (or logical) sector numbers that will be in effect when you mount your TrueCrypt volume and explore the mounted volume with WinHex.

    Basically, you first need to locate your unmounted TrueCrypt container file on the disk and note down its starting (physical) sector. (And let's hope that the container file is fully contiguous and is not fragmented in any way, otherwise the situation becomes a bit more complicated).

    Additionally, you need to include an adjustment factor for the two 64KB headers that reside at the beginning of the volume. Assuming 512-byte sectors, this will account for an additional 256 sectors.

    So add 256 to whatever physical sector represents the starting point of your TrueCrypt file to determine the overall adjustment that you will need to apply to each entry in your bad sector list.

    Merely subtract that amount from each sector number on your bad sector list to determine the relative sector number that you will be using within your TrueCrypt volume. (In WinHex you merely open the mounted TrueCrypt volume according on its logical drive letter and then select "Navigation: Go to Sector")

    This will be a lengthy task. To help organize things, I would probably set up a spreadsheet as follows:
    Column A = list of physical sectors containing bad blocks
    Column B = formula to convert the disk's physical sectors into the TC volume's relative sectors
    Column C = pathname of the file (if any) that has some of its contents stored in that sector
    Column D = status of the file, after attempting to view/recover the file's contents
    Column E = notes

    Anyway, that's the general idea. I left out a few details and I may not have explained it all perfectly, but hopefully you get the picture.

    There might be ways to partially automate the process, but researching the various tools would take more time than I can spare. And I'm sorry but no, I don't personally know of any Linux equivalents for WinHex, but there is probably something. I think WinHex will run under WINE, though. And I'm pretty sure the evaluation copy will do all that I described.
     
    Last edited: Jul 9, 2014
  5. lechuza

    lechuza Registered Member

    Joined:
    Jul 8, 2014
    Posts:
    4
    Thanks for your guide.

    So far I've run "hdparm --read-sector" as mentioned in my original post and did some other things to be satisfied that I can't recover any of the 80 bad sectors. It does look like using WinHex will be a lengthy task. So since the failing drive is of no further use, I'm currently doing the brute-force copy I had mentioned. It's taking a long time but it's automated. It has already identified one bad file. Give its size relative to the size of the region with bad sectors, I suspect it is the only file that's affected.

    The copy's result will probably tell me everything I need to know, but you've got me intrigued about using WinHex. Once I'm done with the drive in the Linux box, I'd like to try your idea in Windows.

    I don't know much about NTFS so I'm learning how to locate the container file on the drive and if it's fragmented. I used ntfsinfo & ntfscluster but don't see anything obvious about sectors. I do see 17 $DATA attribute records from MFT records. Comparing that to the output using other files, I've concluded that the container file has 17 fragments. Does that sound right?

    I read about nfi.exe (File Sector Information Utility) from Microsoft. Maybe that will help when I move to Windows. Do you have a better idea?
     
  6. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    991
    Location:
    Hawaii
    Have you tried ddrescue or similar? I think that would be a good way to get "every last drop" of available data from the disk.
    Boy, I hope not. Ideally that file should be contiguous. But it's been too long and (happily) I've forgotten many details of the inner workings of the NTFS file system. Don't make me go back there!
    No, sorry.
     
  7. lechuza

    lechuza Registered Member

    Joined:
    Jul 8, 2014
    Posts:
    4
    Yes, first I used dd manually in two stages (a simplify version of ddrescue's algorithm) and then I used ddrescue including a retry run. I did a hash of the two images and they're identical. And I did the hdparm low-level reads of the bad sectors as mentioned. I'm convinced those 80 sectors are not going to be recovered. And the tar with a diff confirmed only one file was corrupted which I can recover. So from a practical perspective, my job is done. I thank you again for your interest in helping along with your useful suggestions.

    Given the size of the container file, it wouldn't surprise me that it's fragmented. As an exercise I may create a new NTFS, copy it there and reexamine it with the tools I've mentioned. I may even experiment with WinHex if I determine the container file is contiguous. I'll update this thread if I do pursue the academic line in case it's of interest to you or anyone else.
     
  8. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    991
    Location:
    Hawaii
    You're right about the possibility of fragmentation. Large file-hosted containers that are stored within NTFS file systems are often fragmented because their very size forces them to span the MFT and related structures. I prefer to create smaller containers in order to avoid that issue, and of course I check each one to ensure that it is contiguous. A little fragmentation doesn't affect performance or reliability very much, but contiguous encrypted files are much easier to recover, should that ever become necessary. It can be hellish trying to manually reassemble a fragmented TrueCrypt container, as there is absolutely no margin for error. The volume will not fully decrypt unless every bit of data is located in exactly the right place.

    It sounds like you've had a decent outcome. Thanks for the update. I've enjoyed reading your well-written posts.
     
Loading...
Thread Status:
Not open for further replies.