Experts wanted: Truecrypt container recovery.

My situation started off like others I have read about. Upon restarting my Windows 7 box I realize Chkdsk is running its services on my mirrored array that houses my Truecrypt container file. It had already run for some time so I just let it finish.

After rebooting I noticed my TC container file was 0KB in size. I was unable to mount it using Truecrypt.

After reading many threads detailing recovery methods I found a hex editor, learned how to use it and was able to recover the entire 1.81TB file. I had several backups of the container on other, external hard drives, of which the newest was about 45 days old. I'm very confident that I have recovered the entire file as I am able to compare it with the recently backed up file in the hex editor, both looking at the hex values at the beginning of each file as well as the total file size.

I am able to mount the recovered container file in Truecrypt and it accepts the password for the outer volume as well as the password for the hidden volume where all of my data resides. I do not ever use or mount the outer volume.

However, when I try to access the either the mounted outer or hidden volume within Windows I receive the following message:

V:\ is not accessible. The volume does not contain a recognized file system. Please make sure that all required file system drivers are loaded and that the volume is not corrupted.

I've restored the volume header from both the embedded backup as well as from a backup of the header that I manually created through Truecrypt. This made no difference in accessing the mounted volume.

I've tried accessing the mounted volume on two different computers running Truecrypt as was suggested by one thread contributor, but without success.

I've ran Testdisk and attempted a boot sector rebuild after selecting the drive in the list, selecting None as the partition table type and subsequently selecting NTFS as the partition type but that ultimately failed with the following messages:

Boot sector
Status: Bad

Backup boot sector
Status: Bad

Sectors are not identical.

A valid NTFS Boot sector must be present in order to access any data; even if the partition is not bootable.

I am at a loss as to where to go from here. I'm sure someone out there is able to point me in the right direction. I respectfully request that that someone offer their thoughts.

-pb

 

OK, but give me some time to get back to you. I've been fairly busy helping others today and I need to get some of my own stuff done for a change.

 

Think carefully before you answer the following questions, as the sequence of events matters, and the correct answers will help us to understand what may have gone wrong during your recovery of the container file:

1) After recovering the file, did both the volume header and the hidden volume header work right off the bat when you tried them? That is, was your password accepted without any error messages? (This is irregardless of any problems that you might have had accessing either of your volumes' contents. The contents of the volume are completely irrelevant as far as the action of the headers is concerned.)

*If so, this would show that your recovered file contains the true starting point of your lost file, so that part is ok.

2) If your passwords were not accepted, I'm assuming you needed to restore the headers from a backup copy before you could get them to work. Is that how it went?

*If so, this would indicate that either you did NOT recover the correct starting point of the file, or one or both headers were somehow damaged during the accident.

3) If you restored your headers from a backup, did you try restoring them from the embedded backup headers first (that is, before you used your external, file-based backup headers), and did the embedded backup headers work? (Please think before you answer). Or were you blocked by the "incorrect password or not a TC volume" message, which forced you to use your external backup headers?

*If the embedded backup headers worked right off the bat then this would indicate that you correctly recovered the file's true endpoint. If they did not, then you missed the endpoint. Two common reasons: Either you selected the wrong amount of data, or the file was fragmented (interrupted by foreign data), and thus the actual endpoint of the file was not where you expected it to be.

There are also a few other possibilities. Your answers will tell me a lot about which portions of your volume sustained the damage and what you can do about it.

4) Did you do this comparison right away, or was it after you restored the headers? Restoring the headers changes the header data at both ends of the volume. The backup header is not merely copied onto the volume. Rather, TrueCrypt creates a new, unique header and uses it to replace portions of the old ones at both ends of the volume.

5) Did you also try to compare the files' endpoints?

Now, some info for you:
1) The TC header has nothing to do with the volume's contents or the health of its internal file system. All the header does is require you to input the correct password (and keyfiles, if used) before TrueCrypt will process the file and on-the-fly encryption/decryption can occur. The volume's contents are completely irrelevant. You could completely wipe out the contents of the volume, or fill it with unformatted garbage, and TC would still mount the volume. You could reformat the volume and fill it with data, and TC would not be able to tell the difference (and it would continue to work).

2) When I want to know if a header is "working", I want to know two things: a) Is the password being accepted, and b) is the volume decrypting. (An incorrectly placed header, or the wrong header entirely, can still pass the first requirement, but not the second). If "a" and "b" both pass, I don't ask "How does the file system look?" because the header has nothing to do with that.

3) A 1.81TB file is a damn big file. Depending on the size of the partition that it was stored in, plus other factors (such as the location of any other data that was already present in the partition when you created the file), it might not have been fully contiguous. If the partition was just a little larger than the file then you can be almost certain that the file was fragmented because it was forced to span the MFT and other file system structures.

And recovering a large, fragmented container file is not that easy to do. You can't just locate one of the endpoints and then "copy the whole thing" based upon its known size, because the recovered file will contain unwanted gaps wherever it was fragmented. These gaps will show up as intrusions when you mount the volume, and they are likely to break the file system. (If they occur at the end of the volume then it may not be so bad, but if they occur too soon then you can end up losing access to almost all of the volume.)

Anyway, if you're good with that hex editor then maybe I can give you some tips on how to troubleshoot your volumes, once we figure out what went wrong. Here's a starting tip: Mount the outer volume, then use your hex editor to examine the contents of the mounted volume. Look for any sign of decrypted (non-random) data. You should see it right away. Do the same thing using the hidden volume.

It would also be helpful if you described the procedure that you followed to recover the file. That's it for now! Dantz out.

 

dantz,

Thanks for the detailed reply.

Before answering your questions let me provide you information about the recovery process as you requested at the end of your post.

After booting back into Windows after Chkdsk had finished and after realizing that I was unable to mount the now 0KB container file I made a concious decision to preserve the current state of the data on that drive by vowing never to alter any of the data while it resides on the drive nor write to the drive. Instead I used a second drive to save chunks of data to using the hex editor (HexEdit 4.0 via hexedit.com).

To begin, I mounted the drive containing the container file in HexEdit and started to search for the beginning values of a known good copy of the TC container file using HexEdit's search function. This stalled or appeared to stall; presumably due to the large drive capacity it had to search through. If I'd click on the HexEdit window it would hang and stop responding. So, I abandoned that process.

I decided to manually look for the start of a large chunk of random data following a bunch of zeroes as suggested in various threads. After some time I found the start of a large chunk of random data and assumed that was the start of my file. So I simply calculated the file size from that point to an end point that would match the size of a known good copy of the container file so I could save that data out to the spare drive. After specifying the end point I received a message in HexEdit that mentioned something to the effect that the location I specified for the end point did not exist and it offered to take me to the end of the drive, so I accepted. The saving process took about a week. Needless to say, the size of my TC container file has been a detriment to this whole process.

After the saving process finished I realized that the HexEdit message I received was important as the resulting saved file was smaller than the known good TC container file I had. Time wasted.

I then examined the end of the known good TC container file and noticed that it was comprised of a large chunk of contiguous zeroes. So I decided I would simply append the file that I created with the correct amount of data from the end of the known good TC container file. I calculate how many bytes I needed and counted backward from the end of the good file and appended that data to the saved file. Now the file I had was the correct size. After attempting to mount it in TC I got a message telling me that the password was incorrect or that it was not a TC file. So, I knew something was wrong.

After letting the issue rest for a few days I decided that I'd use the search function again to look for a string of values that matched the begining values of the known good TC container file. But, instead of searching from the beginning of the drive I started the search from the point at which I found that first chunk of random data. After a while it found the beginning of another chunk of random data that followed a chunk of contiguous zeroes and that also matched the values from the known good file. That was promising.

So, I began the process of saving a large file to my spare drive again. And again I received the HexEdit message telling me that the end point I specified did not exist so I chose to include all data to the end of the drive. After the file was saved out to the spare drive I again opened the known good container file and calculated backward from the end of the file the amount of data that I needed to append to the saved file to make it the correct size. Again, this appeared to be just a bunch of zeroes that I was appending.

After that process was finished I mounted the hidden volume in TC and it accepted my password. I then tried the outer volume and it again accepted the password.

Now, let's answer your questions.

1) Yes, as mentioned above the passwords were accepted once I specified the correct starting point of the file, saved the file to the spare drive and appended the appropriate amount of data to correct the file size. As a result of my ignorance, coupled with not being able to access the data after mounting the hidden volume where my data resides, I elected to restore the embedded backup header. Still no access to the data. I then restored a backup of the header I had stored elsewhere. Same thing, no data access accompanied by the message:

V:\ is not accessible. The volume does not contain a recognized file system. Please make sure that all required file system drivers are loaded and that the volume is not corrupted.

I learned from reading your post that a working header means nothing in the way of being able to access the data once the volume is mounted.

2) N/A

3) N/A in the respect that ultimately the backup headers did not have any effect on TC's ability to mount the volumes.

4) The comparison was conducted prior to the restoration of the backup headers as well as afterward. In both

cases the beginning and end of the files matched. Mind you I only visually compared less than 100 bytes of data from the beginning of the headers.

5) Yes, as mentioned above, the ends of the files are zeroes.

Responses to your second section of numbered items:

1) That is good info to know, thank you.

2) a) Yes b) how do I determine whether the volume is decrypting if I can't open the volume in a hex editor?

3) It's big, I know. I created the container file on a blank, newly formatted NTFS 2TB hard drive with one partition of the largest capacity allowed. I agree that the file may be fragmented, especially when considering that I could not proceed far enough ahead on the drive from the starting point when saving the file out to the spare drive.

I'm not terribly good with a hex editor beyond what I've already accomplished but I'm certainly a capable learner.

I am unable to examine either the mounted outer or hidden volume as they're not recognized as having valid file systems.

-pb

 

I'll add some in-depth comments later on when I have more time, but I will say a couple of things now:

Your recovery process was less than optimal, but it sounds as though you were at least able to recover the first large block of the file. From your description, the various blocks (fragments) are probably not arranged in order on the drive, so it will take considerable trickery, plus a big reassembly job, to get it right.

You could have saved a considerable amount of time at the start by testing the location on the disk where you thought the headers might reside, before doing the big copy/save operation. I've outlined the procedure in quite a few posts on these forums.

From your description of events, it seems unlikely to me that you would have been able to restore the embedded backup headers. These are located near the end of the file, and TC looks backwards a specified distance (128KB back from the file's ending offset) to find them. In your case, the only way that this would be possible would be if you had only appended a small amount of filler data onto the end of the file. If you truly did restore the embedded backup headers then you were somehow able to find the true endpoint of the file, which will be very helpful when it comes time to reassemble the various blocks.

I'm surprised that the ends of your (unmounted) good container files seem to contain large blocks of zeros. That should not be! Container files should look like random data from beginning to end. I suggest you test the embedded backup headers on your backup copies of the file to see if they are functional, but DON'T try to restore them, as this will alter the file. Rather, just try to use them to mount the volume and see if your password is accepted. ("Mount Options: Use backup header embedded in volume if available").

It was not really necessary to append filler data onto the end of the file to make it the correct size.

You should definitely be able to use a hex editor to inspect the contents of the mounted volume, whether or not it contains a valid, functional file system. If your current hex editor can't do it, try WinHex. The evaluation copy will be sufficient for that purpose. "Tools: Open Disk", then select the correct Logical Volume (select the drive letter that you assigned when you used TrueCrypt to mount the volume.) You will have to click through a couple of "invalid this or that" error messages, but it will work.

 

dantz,

Thanks for the reminder regarding testing for the header location. I recall reading about that but neglected to remember it in my haste.

I can confirm that I was still able to mount the file after both restoring a backup header from an external source as well as the embedded backup header. In fact, just now, I mounted the recovered container file using the embedded backup header.

I, too, was surprised to find that the end of a working copy of the container file was comprised of zeroes. I tried to mount a known 'good' copy of the container file using the embedded backup header and it did not accept my password. What does this indicate?

I opened the mounted volume in WinHex and it appears that TC is not decrypting the data.

-pb

 

It should work without you needing to restore the header. And whether or not it can mount isn't what's important. You need to have the right header in the right location, otherwise it will not be able to decrypt your data.

You need to find the original header wherever it is on the disk, and start your copying at that point. If a backup header is merely tacked onto some random chunk of data then of course it won't be able to decrypt it. The header has to be in exactly the right location in relationship to the specific data that is has to decrypt. The easiest way to accomplish that is to find the header on the disk (by testing for it), and then copy the header and all data that follows it, up to the size of the volume or fragment thereof.

FYI, you can restore a TC header onto absolutely anything. Any file, any partition, any disk. (Don't do this!) If you do then not only will you destroy the object that you restore it to, but TrueCrypt will actually allow you to assign a drive letter and mount the object. However, nothing will be decrypted. The contents of the mounted volume will consist of random data from start to finish.

 

TrueCrypt looks backwards a specific distance from the end of the file to find the embedded backup header. So either the length of the file is incorrect, or the embedded backup header has been damaged (or is not present.)

What are you finding when you use WinHex to look at the mounted volume? Any strings of zeros? Any words? This would indicate that the volume is decrypting (as it should, if you merely copied the header and the data in one piece from the source disk.)

Since you have a backup copy of the file, you should be able to copy the first 10 bytes or so and paste them into the WinHex search feature, then search for those bytes on the disk that contains your lost file in order to find the beginning of the file. (Assuming that your backup copy is a true backup of the file, as opposed to a separately-created container, that is.)

 

dantz,

I presume then that the embedded backup header is damaged as I've compared the size of the recovered file to a working copy of the container file and they're the same. So, it appears that the embedded backup headers in both the recovered file as well as the working version are damaged.

I don't know if this is a result of a previous Chkdsk run in the past whose occurrence I was unaware of or something else.

In WinHex I am finding random data when viewing the mounted volume. No sign of zeroes or readable data. What does that mean?

The working backup of the container file is actually a mirrored copy of the now-inaccessible file so the header is the same.

Here's what I've done since I last wrote.

1) Opened the drive that contains the fragmented container file in WinHex
2) Scrolled down a quarter of the way or so and placed the cursor in the middle of some random data
3) Searched upward for 00 00 00 00 00
4) After several hours it located the first instance of the search string which ended up being adjacent and prior to the beginning of the presumed header
5) I saved a 20KB block of data starting at the beginning of the header
6) I was able to mount that file using the password for the outer volume but not the password for the hidden volume
7) I saved a 65KB block of data starting at the beginning of the header
8 )I was able to mount that file using the password for the outer volume but not the password for the hidden volume
9) I saved a 128KB block of data starting at the beginning of the header
10) I was able to mount that file using the password for the outer volume but not the password for the hidden volume

As I understand in completing step 9 I should have saved the entire header as a file. If that's the case why does TrueCrypt not accept the hidden volume password when trying to mount the file, while at the same time I am able to mount the recovered file and a working copy of the container file using the hidden volume password? I'm also unable to mount a 128KB file containing the header data from a working copy of the container file using the hidden volume password but it mounts using the outer volume password.

How is Hawaii today? It's 1˚F here in MN.

-pb

 

128KB is just below the minimum size for testing a hidden header. You need to save a slightly larger test file. Try making it at least 16 bytes larger. However, I suggest making the test file about 2MB in size, as this will allow you to check the volume's contents to ensure that they are decrypting.

It's a pretty fine day as long as you don't mind being pounded by 50' surf. It's frickin' huge! I just went for a swim this morning, but on the opposite side of the island where it's still manageable.

(Edit): Under normal circumstances you shouldn't even have to worry about whether or not your test volume will decrypt. As long as you block-copy and recover the header and the data that follows directly after it in a single operation, it will decrypt. The only time users get into trouble with non-decrypting headers is when they try to restore the headers themselves and they restore them to the wrong locations.

If the data in a mounted volume doesn't decrypt, it generally means one of two things:
1) The correct header was restored to the wrong location. It has to be restored to exactly the right spot in relation to its data, or it won't decrypt.
2) An incorrect header (that doesn't even go with that data) was restored to any location.

 

More likely the embedded backup headers are merely out of place. You probably haven't found the true endpoint of your lost file, and thus TrueCrypt has no way of finding the embedded backup header. TrueCrypt doesn't actually search for the header, it merely counts backwards 131071 bytes from the end of the file and expects to find it there, where it's supposed to be. If you arbitrarily add or subtract bytes from the end of a container file then this is the sort of thing that will happen.

And as I mentioned earlier, your lost file was most likely fragmented (because it is so large that it most likely spanned the MFT, etc.), so you can't just use a hex editor to save a big block of data (of the expected length) from your hard disk and expect it to be correct from start to finish. The situation is much more complicated than that. You need to save each fragment separately, leaving out the parts that don't belong there, and then reassemble the fragments into a single file. It's tricky.

 

dantz,

I could definitely go for a nice helping of slamming waves. We're dipping down to -43˚F with the wind factored in tonight.

I'd like to extend my gratitude for the continued assistance with this; thank you.

I saved out a 2MB block of data and was able to mount the file using the hidden volume password and received a message that stated "Q:\ is not accessible. Reached the end of the file". I then opened the volume in WinHex (Tools, Open Disk) and received the same message along with subsequent messages in the WinHex Messages window stating "Cannot read from Sector" repeatedly appended with different sector numbers.

The hex values in each sector are all 55 4E 52 45 41 44 41 42 4C 45 53 45 43 54 4F 52 which translates to UNREADABLESECTOR.

In one of my first posts I mentioned that during my first go at recovering the file I designated the starting point as the first byte of a large block of random data that followed a bunch of zeroes and later realized that this was not the proper location for the starting point of the file [header]. What if I saved a block of data from the location at which we know the header is now all the way to the end of the random data block and then appended that file with the first block of random data that I discovered, assuming that was the only other chunk of random data on the disk and also assuming that in total their size equaled that of the working copy of the container file? Then, try to mount the file using the embedded backup header? Or, am I going about this incorrectly? The notion just popped into my head so I figured I'd throw it out there.

[edit]
Or, should I find the end of that first chunk of random data and count back 2MB and try to mount it?

[edit2]
Although, if my working copy of the file is in fact that, a copy, and its embedded backup header is missing, judging by the zeroes at the end of the file and TC's inability to mount the file using the embedded backup header, I may be looking for a backup header that does not exist.

-pb

 

You can ignore the various error messages that WinHex is displaying, as they are normal and expected for this situation. WinHex is basically telling you that it was expecting the file to be much larger than it it, and thus there are a lot of things missing (like a working filesystem and a whole lot of data). This is happening because TrueCrypt takes its file size information from the header, which doesn't change, and it mistakenly reports that the mounted volume is the full size of your lost volume, wheras we know that the file is only 2MB in size. We've essentially tricked both TrueCrypt and WinHex, but it's worth it because now we have identified the lost file's starting point and have confirmed that the data is decrypting, and that's really all that we wanted at this point.

You've almost got it right, but since you've located the intact volume header in the correct location, you don't need to use the embedded backup header to mount the volume. Just use either the regular volume header or the hidden volume header for everything. At this point the only thing that the embedded backup header will be good for (if you can find the original copy that's currently embedded inside the file, that is) is to help us determine the volume's true endpoint. It would be great if you could do that, because then the correct assembly of the fragments would be much easier.

Keep in mind that the matchup of the fragments has to be exactly right, otherwise the data from that point forward won't decrypt. It may take numerous tries to get it right. If, based on the overall size of the file, some data seems to be missing then we can insert filler if necessary, but every byte of data that you wish to recover MUST be located at the correct distance from the header, otherwise it won't decrypt.

No, not 2MB. If you selected the correct endpoint then 128KB would work, since the embedded backup header is normally located 131,072 bytes back from the end of the file.

I'd assume that the embedded backup header is lost somewhere within the file. It will only "work" if it is located at the correct distance from the end of the file. Perhaps you could ignore the zeros and count backwards the specified distance from the end of the random data, and find it there. You would need to test it, of course. But that can come later.

If I were you I would start by recovering the entire first fragment, clear to the end of the first big block of random block of data, and then mounting it and seeing if any of your data is present. If you're lucky then you might be able to copy off and recover some data.

Then copy the second block and attach it to the end of the first block (and hopefully the transition points at the end of the first block and the beginning of the second block are crystal-clear, otherwise there will be trouble.)

To save time and effort, you could probably do a small-scale test of the concept first. Merely copy a small amount of data from the beginning of the second block and paste it onto the end of the first block, then examine the mounted volume in WinHex to see if the portion that you pasted on contains any decrypted data.

I'd have to do some testing to figure out what the optimal amount of data to paste would be. I'm guessing that 1 or 2 MB would be enough. (I don't have time to run the tests right now.)

 

dantz,

Thanks for the notes.

Unfortunately I've already done this as outlined at the beginning of the thread. The only difference is that I appended a bunch of zeroes to the end of the file. I've since mounted the file, and smaller versions of it, all of which include the working header, and see only random data when viewing the volume in WinHex.

Does this mean that the header is adjacent to the wrong chunk of random data? From what I understand I should be seeing decrypted data in WinHex at this point since we've located the working header.

-pb

 

Open the 2MB file again, using the outer password, then inspect the mounted volume with WinHex and look for non-random data. I realize that you are much more interested in the hidden volume, but the hidden volume resides within the outer volume, so we need to get the outer volume working first. Please focus on the outer volume. If we can recover the entire outer volume then the hidden volume will work. If we can't then it won't.

At this point you won't be able to see any decrypted data if you mount the test volume using the hidden header because the test file is way too small to include any of the hidden data.

 

dantz,

I only have one file stored in the outer volume which I placed there immediately following the original creation of the volume. It is a text file called Padlock Combo.txt. The file contains a numerical sequence in the format ##-#-##.

I searched for the file name, part of the file name and for the numerical contents of the file using WinHex after I mounted the test file. It did not find anything.

When I mount the outer volume using a working copy of the container file TC reports that the volume is of Normal type (non-hidden) and that it is 199GB, which is correct. When I mount the test file using the outer volume password TC reports that the volume is of Normal type as well but that it is 1.81TB, the size of the hidden volume, which is incorrect. Not sure if this tidbit helps.

-pb

 

You don't quite understand what I'm asking you to do. You don't need to find any actual file data. The mounted test volume probably doesn't contain any of your actual data, as it's too small. It should, however, contain some traces that there is a working file system, and that's all we are hoping to see at this point. So don't look for filenames or file contents, just look for non-random data. To demonstrate that a volume is decrypting, all you need to see is something that has a pattern or is recognizable in some way. It will usually be visible immediately, but if necessary you can scroll down a little bit.

To be more specific: When you mount the 2MB test volume using the outer volume password, and then open the mounted volume in WinHex, do you see any zeros in a row? Like this: "00 00 00 00 00". Or any words in the text column such as "NTFS" or "FAT" or "partition" or "the" or any other words that you recognize? That's all we want to see at this point.

And try it again after mounting the volume with the hidden password, since you have mentioned that the volume sizes seem to be coming out wrong.

PS: According to your steps 1 through 10, several posts back, your test file begins with the outer volume's header, which is correct. Do you still have some of the smaller test files? Mount them and check the volume size (by clicking on Properties in the TrueCrypt main screen) to see if their size is correct. The listed size should be very close to the size of your lost file.

 

dantz,

I scrolled through the contents of the file in WinHex after mounting it using the outer volume password. I could not see any trace of non-random data.

WinHex is unable to find any data in the volume when mounting the file with the hidden volume password. (UNREADABLESECTOR)

None of the smaller (20K, 65K, 128K) files report the correct volume size in TC when mounted using the outer volume password.

-pb

 

(I have some time now. Sorry about the delay.)

If you copied your test data off the hard drive, saved it as a file and were able to mount it using the outer volume password, then you have very likely found the beginning of the lost file, and it should both mount and display some non-random filesystem data.

However, since the mounted test volume is not displaying any non-random data, and is also not listing the correct volume size, it seems that you have not located and copied the correct data from the hard drive. Perhaps you copied the remnant of an earlier TrueCrypt container that used the same password, or something like that?

Since you have a backup copy of the lost container file, this should actually be quite easy. Just use the WinHex "Search" feature to search for the first 8 bytes (or whatever) of the known good header. If you can't find it, try searching for some other portion of the file that would not have changed, as follows: If your backup file is a direct copy of the lost file then the first 128KB and last 128KB of the file should be identical, even if the contents of the outer or hidden volumes have changed. This provides you with quite a few search options.

Oh, and while you're at it, mount the outer volume of your backup copy, click on Volume Properties (in the TC interface) and note the size of the volume. Is it correct? Is it the same as the test files?

I'd really like to understand what's going on here, as you have reported several unusual discrepancies and things are not working the way they ought to.

 

dantz,

No worried, thanks for reporting back.

I don't recall every creating a another container prior to the creation of the lost file so that doesn't seem plausible.

I'm not following what you want me to do here. Are you saying that the first 128KB of the working file should match the last 128KB of the working file? If so, it doesn't. Remember that a substantial portion of the end of the working file is a block of contiguous zeroes. Even if I back up to the last portion of random data it does not match the first 128KB of the file.

What I did do was saved out a 10MB block starting with the beginning of the header in the working file and mounted it in TC using the outer password and then opened the disk in WinHex and was able to view some non-random data, as expected.

Does this tell us that the header in the recovered test files is somewhat corrupted? Able to be mounted but not decrypt?

I should clarify one of my earlier statements. I initially mentioned that when mounting the test files using the outer volume password it was reporting an incorrect file size. It wasn't. The mounted volume reports the same file size whether you've mounted the file using the outer or hidden password. This is by design I presume. This realization came from mounting a working copy of the file using both passwords and comparing the volume sizes. They're equal. The confusion on my part came from mounting a smaller (199GB) file which contains another hidden and outer volume which resides inside of the larger (1.81TB) hidden volume. I am trying to recover data that reside both in the larger hidden volume as well as within the smaller hidden volume. I was simply reporting to you the volume size after mounting the inner TC file rather than the main file. My bad.

In summary, the volume sizes are reporting correctly when mounting a working copy of the file and smaller recovered test files from the botched drive.

[EDIT]
I compared the contents of the 128KB test file to the contents of a 128KB file saved from the working copy of the file and they're not the same, at least according to Microsoft Word. I highlighted and copied the blocks of data in a hex editor and pasted the contents in separate Word documents, then used the Compare function which will show you the differences between the documents. There were many differences.

-pb

 

No, I'm saying that you claim to have a direct backup copy of the lost file, and thus the four 64KB headers on the backup copy (there are two at each end) should be identical to the four 64KB headers on your lost file. You should be able to compare them to one another, copy small sections of them and use them in search strings, etc. in order to find both ends of the lost file. Apparently you've already found the starting point, but the problem is, it doesn't seem to be working as expected.

A header is "all or nothing". The first 512 bytes are the crucial portion of the header, and the rest of it does nothing at all. If the first 512 bytes of the header was even slightly corrupted, with just a single incorrect byte, then the volume would not accept your password, and that would be that.

Your results tells us that there's something wrong between the header and the data that follows it. Either the header isn't the correct one for that data, or it is in the wrong location relative to the data, or something of the sort. Here are the main possibilities:

1) Wrong header: The header was accidentally or deliberately replaced with an incorrect header (which some users actually do on purpose in order to further obscure their data)

2) The data that follows the headers has been altered or moved while the volume was unmounted, or the header itself was moved. If the location is off by even a single byte then nothing will decrypt.

3) The data was wiped and replaced with random data while the volume was mounted

4) The data was overwritten while the volume was unmounted.

I would focus on the first 512 bytes of each volume. Are these bytes identical between both your backup copy and your recovered file, and the several test files? They should be, unless that is you have altered one of the headers, perhaps by changing the password or by restoring a header.

 

1) I did not purposely, nor do I recall accidentally replace the once working header with data from an incorrect header.

2) This is likely since chkdsk.exe had been run on the drive

3) Unlikely

4) Unlikely

The first 512 bytes of data are identical when comparing the header on the drive containing the lost file to the header of the working copy of the file. Beyond that is where the differences begin.

As an aside, what is the recommended method for encrypting data on a non-system drive using TrueCrypt? File container? Encrypted partition? Encrypted drive? Top considerations are static vs dynamic storage allocation and ease of recovery in the event something like this happens again. I realize more frequent backups would contribute to making the second consideration less of a determining factor when choosing the encryption scheme.

Thank you,

-pb

 

dantz,

I'm still here if you've got any additional input. Hope you're doing well.

Thanks,

-pb

 

Your situation is not behaving as expected. Things have gotten screwed up somehow (beyond the normal level of screwups, I mean). Even your backup copy is messed up in a very strange way. I can only assume that more has been going on than what has been described thus far. But I'll give it further thought, maybe there's something we can do. I'm very busy right now, though. Maybe by tomorrow.