Encrypted (TrueCrypt) Raid 1 External Recovery Help

I am at a loss. I have a fully encrypted 2TB external HD that I can no longer access. I know the password is correct. I know the drive must be sort of working. The WesternDigital Drive Manager says the drive is healthy. I have a feeling that because my tech level is low when it comes to this drive science stuff, I have made the problem worse.

The Problem:

When I went to mount the drive via TrueCrypt, the encrypted device (931 GB) looked different from the other selectable deceives i.e. my internal HD's. The internals have the following below the device name;

Harddisk 0: (Device Name)
\Device\Harddisk0\Partition2

This second line info is missing from the raid1 external encrypted drive. Well at least I think it was anyway. So I then proceeded as normal. Entered the correct password. Went to open the drive as usual however this time I received a message that said the drive is unrecognizable and must be formatted. So of course I did not format because I do not want to loose my data.

How I Fudged It Up:

I scratched my head for a bit and then decided to restore headers, because it sounded like a good idea at the time(s). Reading these forums, I now am under the impression that was a big mistake. I may have done this multiple times as well.

Please Help:

I do have hope however that the data is still obtainable because of the mirroring. I need to understand more before I shell out cash for WinHex. Unfortunately I think WinHex will be needed to identify the problem. Are there any free tools out there that can help me diagnose the issue and verify that data still exists on the drive before I invest in data recovery software?

 

I don't know much about this stuff except what I've learned from my thread (https://www.wilderssecurity.com/showthread.php?t=329860)

But I think restoring the header was a mistake. It apparently screwed up my Truecrypt partition, and then I had to resort to a backup that turned out to be useless. Thanks, DriveimageXML... And you would think TC would be robust enough that attempting to restore the header wouldn't cause all data to be lost.

First I think you should back up everything. WinHex can create an image file and so can Easeus, but Easeus is free. I'm sure there are plenty of other programs. What I wish I had done was backup with two different apps, to increase the odds of success.


The free version of WinHex lets you view the contents of the drive but not export any files bigger than 200KB.

BTW I don't see how mirroring would help... if you messed up the backup headers, then the change would affect both drives.

 

It sounds like you used to have a encrypted partition, and now it's gone. The partition table may have been overwritten by a software error, or possibly the beginning of your external drive has experienced physical damage. This sort of thing happens more often than you might think. The encrypted data that used to be on that partition is probably still there, but now it's in free space. If that's truly the situation then the safest solution will most likely involve imaging the drive, restoring the image to another drive and then attempting to recreate the partition table to its original configuration. However, before you try any of that we need to understand more about what happened and what you have done so far. You say it's a Raid-1 setup? If so then please describe where the mirrored data is located. Is Raid-1 built into the external drive, perhaps as a dual-drive set?

Contrary to your current beliefs, restoring the TC header almost never causes problems. However, if you use an external header then it's possible to restore the wrong header, and under certain circumstances it's also possible to restore the correct header to the wrong location. How, exactly, did you restore the header? It's important to explain this fully. Did you use an external (file-based) header or the embedded backup header? Is the embedded backup header still available, or has it stopped working? Did you try using it to mount the volume? (This happens automatically after 3 "incorrect password" prompts.)

When this sort of thing happens, the first step should be to attempt to mount the volume using the embedded backup header ("Mount Options: Use backup header embedded in volume"), not to restore the volume header. It's perfectly fine to mount the volume this way. If you try this and you are successful in mounting the volume then the first order of business is to immediately copy all of your data to a safer location. You can try to restore the volume header later if desired.

You can use the demo version of WinHex to examine the situation. You can also use the freeware hex editor HxD if desired. First answer the above questions, then I'll see if I can help.

If you get to the point where you are able to mount your volume then you can try the demo version of GetDataBack to see if it can find your data. There are also a few freeware data recovery programs that might help.

 

I will do my best to retrace the steps on the path of my self destruction.

I went to mount the device as normal and I'm pretty sure I noticed that the partition info line was missing at that time. I then became alarmed when I could not open the drive after entering my correct password after at least a couple of attempts (I'm sorry I'm not sure if I got it the 2nd, 3rd or 4th time). Unfortunately I panicked and attempted to restore and mount multiple ways and multiple times. Also, unfortunately, I do not recall the order. The worst part is after I recklessly tried to restore the header I remembered that I may have a thumb drive that I backed the header up to. I'm still not sure that I did do this because when I (finally) located the thumb drive and tried to mount with backup header from external device, TrueCrypt said the size did not match. The file on the thumb drive I believe is called BKcert. Not sure if I named the file myself or not I'm sorry. I am very sorry I am being so vaugue. I know I did try to mount using the embedded header and it didn't work. I'm not sure how to locate the embedded header. I'm afraid I tried every available option to mount and restore that the software allowed me before hitting theses forums. I should have reversed that order.

The 2TB drive itself contains 2 seperate physical drives inside the external casing. The Raid 1 or mirroring was configured by the Western Digital utility that was included. It was a real easy process, just a couple clicks. This is same utility that currently tells me the drive is "healthy".

I apologize for grammar and spelling as I am writing from a mobile device.

 

It sounds as though you were never actually able to restore the header, either from the embedded backup header or from an external file, is that right? Did the "Incorrect password or not a TC volume" error message pop up each time you tried? That's ok, it's the outcome I would expect based on what you described earlier.

Here's how it normally works: When you try to restore the header from the embedded backup, TC goes to the location where it expects to find the backup header and then tests the first 512 bytes of that location against the password that you provide. If the intact header isn't where it's supposed to be, or if you supply the wrong password, then you see that error message. I suspect that TC can't find your header because the partition definition has been lost. For partition encryption, TC normally looks backwards a specific distance from the end of the partition to find the embedded backup header. However, you're apparently selecting the entire disk, so TC is looking backwards from the end of the disk and is not going to the right place.

If you are able to separate your RAID array and disconnect one of the drives then you can use that as your backup copy. If you'd rather not try to do that then you need to decide if your data is worth making a complete backup copy of the drive before we try to fix anything. However, we can still do some safe (read-only) procedures right now using WinHex. The evaluation copy of WinHex (http://www.winhex.com/) will get you started. Based on how the recovery effort goes you might need to purchase a license, but first let's see how it goes.

Does this sound reasonable so far? If I have misunderstood your situation then please let me know.

The first step is to look at your drive using WinHex to see if we can find out where the lost partition used to be. If possible we should find both ends, and at that point we can try to find the TC headers. If we find them then we can create a new partition to match the original settings, and it should work. Alternatively, we can use WinHex to copy the entire contents of the lost partition to another drive. Either method can be successful, if we get that far. Here are a few steps to get you started:

1) Open WinHex

2) Options; Edit Mode; ensure that Read-Only mode is selected; click OK (or click Cancel if you didn't have to change anything, which is normally the case).

2) Tools; Open Disk; select your external drive under Physical Media, click on Ok.

3) WinHex displays the contents of your disk. Each row of data is displayed in both hex (left-hand column) and text (right-hand column). The Offset column tells you where you are on the disk, and you can move around by using the scroll bar on the right, plus some other methods.

If there are any partitions or files present they will be listed near the top of the screen, just above the data. If any partitions are listed then you can still try to follow the remaining steps just to see what happens, but they won't give the desired results.

The Offset column can display in either Decimal or Hexadecimal notation. Each time you click it toggles from one to the other. Set it to display decimal notation for now. Scroll down if you're not sure which mode it's in, as it will be easier to tell lower down. The display mode is also listed in the information panel.

4) Position; Go to Offset; New Position = 32256 Bytes (decimal); relative to Beginning; click OK.

5) Offset 32256 is the most common location for the beginning of the first partition. What do you see here? Does the area above your cursor contain a lot of zeros (00 00 00 00 etc.) in the Hex column? Is any of the data below your cursor readable (in the Text column)?

6) Press Ctrl+End to go to the end of the disk. Is it all zeros? Scroll up a little ways (use the scrollbar) until you get to some data. Click once to place your cursor in the data.

7) Search; Find Hex Values; 0000000000; Search: Down; click OK.

8.) You are hopefully at the transition point between a giant block of random data, and a lot of zeros. This might mark the end of your lost partition. However, if there are any other partitions present on the drive then this probably isn't the spot we're after.

That's plenty for now. Try some of this stuff out and let me know what you find.

 

I'm not sure if I know how to tell if there are any partitions or not. It all looks like gibberish to me. There are some numbers and letters in the second two columns. I believe you said the far right being the text column. This column is unreadable as far as plain English text goes. It is just has numbers. symbols and letters. This continues for about 32 rows down then there are zeros and nothing in the text column.

All zeros (00 00 00 00 etc.) on the entire view-able screen. I scrolled up and down with and saw only zeros in the middle column. Nothing in the text column either.

As soon as I hit Ctrl+End the middle column and text column have numbers letters and symbols (symbols only being seen in the text column as before). I scrolled up and this continues as far as I could tell. I could not locate at which offset the text column started to be blank again (scrolling up). I also could not find where it started to be blank (scrolling down) after the first 32 rows. Will I need to find this point?

 

We're kind of looking for that now. I guess the partition didn't start at 32256. Let's try another potential location:

(In WinHex, with physical drive selected):
Position; Go to Offset; New Position = 1048576 Bytes (decimal); relative to Beginning; click OK.

See if the area above your row is mostly zeros, and the area at and below your row is filled with random-looking (unrecognizable, patternless) data.

Did you partition the drive yourself, or did it come like that? What OS are you running?

Another trick: Place your cursor in the middle of what appears to be a big block of random data, then search UP for the pattern 00 00 00 00 00. Modify Step 7 (in my previous post) to do that. (Search for 0000000000 hex, up from current position)

You typically shouldn't expect to recognize much of what you see in the right hand (text) column, but words or letter patterns will sometimes be present, especially near the beginning of drives or partitions. The exception is encrypted partitions, which always look totally random from start to finish.

If this doesn't seem to get us anywhere then we'll try a different approach, possibly involving TestDisk. We also might have to check with the drive manufacturer to see how the drive's partitions were set up. Meanwhile you're learning how to use WinHex, which will probably come in handy when we find what we're looking for.

 

Wow you nailed it! Right at offset 1048576 is where the middle and text column start to contain letters and numbers. Yes this is true below, but above there are zeros.

Yes, I partitioned the drive myself if you are referring to the RAID. I just used the Western Digital Dive Manager utility. It's almost too easy to switch from RAID 1 to 0 and vice versa. I could just click there right now and select RAID 0 and boom...this problem, along with any surviving data (if any) are gone. I setup up the TrueCrypt part of it per a youtube video. I don't think I got into setting up separate partitions within the drive. Although that might be a better idea for the maybe the future?

The drive is a the 2TB version of this. A Western Digital My Studio II.

I am using Windows 7 Professional. If I had Win7 Ultimate I probably would have used bit-locker instead.

 

Ok, that's probably the starting point of the lost partition. Let's find out. The first 512 bytes of an encrypted partition normally contains the most important portion of the TC volume header. We need to copy that, plus some 'padding' data, to a file and then test it to see if it can be mounted by TC. Here are the steps:

1) Open WinHex, select the correct physical drive

2) Click once in the Offsets column to switch to Decimal mode (if it's not already in that mode)

3) Edit; Define Block; Beginning = 1048576 [beginning of block]; Ending = 1248576; OK

(we have selected the potential header plus a little under 200KB of adjacent data in order to make the resulting file exceed the minimum size so that TC will be able to mount it)

4) Edit; Copy Block; Into New File; (choose a folder on another drive, assign a filename such as "1048576 test.tc"); Save

5) Notice that the newly created file appeared in a new tab or window in WinHex. Close that tab or window and then close WinHex.

6) Open TrueCrypt; click on a free drive letter; Select File; specify the file "1048576 test.tc" (or whatever you called it); Mount; enter the password for the lost partition; click OK.

If your TC header is still intact then your password will be accepted. If not then you will see the "incorrect password or not a TC volume" error message.

7) If you are able to mount this tiny sample of your lost volume, hooray! Click on Volume Properties and write down the exact Size of the volume in bytes.

8. At this point you can dismount the volume and post your partial success story so we can go on to the next step. Don't bother trying to view the volume's contents using Windows Explorer or WinHex, though, as there will hardly be any legible data. This is just a test to see whether or not you have found the intact header and the partition's starting point.

 

Okay, so sorry it took so long and thank you for responding.

I believe this was succesfull. I followed the steps and was able to mount the file/block with my TrueCrypt password, thank you. Just so you know, I did attempt to open it and I received an error message that states, "The disk structure is corrupted and unreadable". I'm sure this is expected.

Anyways, after clicking volume properties in TrueCrypt the volume size reads, "1000195186688 (bytes)".

I hope this is all good news.

 

Sounds good. That's great news. Now we have to decide how to safely fix the problem. There are two main approaches:
1) attempt to recreate the original partition table using something like Microsoft DiskPart (included in Windows), or

2) use WinHex to block-select what we believe to be the entire huge (~1TB) partition and save it onto another drive as a file, which will then be mountable by TrueCrypt. This is basically a giant-sized version of what you already did when you created the test file, but this time it will include the entire partition instead of just a snip from the front end.

The DiskPart approach will be quicker, easier and cheaper (you won't have to buy a WinHex license), but it is also considerably riskier. You really ought to make a backup before doing this, in case things go wrong. I'm not sure if your RAID controller and/or software creates a special circumstance, and I'm also not certain what sort of partition you created using Western Digital Drive Manager, as I'm unfamiliar with that tool. We can probably do this, but I have to warn you that I'm more of a TrueCrypt expert than a partitioning expert, and these sound like non-standard conditions.

Can you remove one of the drives and connect it directly to your system? Either in a drive caddy, or installed internally? Then your other drive can be the backup. I would have said just pull one drive out of your RAID array and run on the other one, but I'm fairly sure your RAID enclosure would complain and would wait for you to insert a fresh drive so it could rebuild the mirror.

Alternate plan: make an image of the entire disk before we proceed any further.

I know most people would just bull ahead and attempt to recreate the partition definition, and it would probably work, but I tend to be more cautious, especially with other people's data.

 

Well, unfortunately, it may come down to available and/or obtainable resources. For option 2 I will need to also come up with another TB of free space for the new partition. Is that correct?

I should be able to install one of the drives internally. From what I can tell from the documentation, the two drives inside are just normal 3.5 drives. There is a section on self-service a.k.a. how to remove a failed drive.

You are correct. It also says in this section, not to use it while your rebuilding it.

Dumb question; when you image a drive, does it just mean copy the drive? Therefore I would also need another TB of space for the image/copy? I guess my question is, would the image be the same size as the partition that is imaged?

 

Yes, the image file would be the same size as the drive, especially in this situation since the drive's contents are fully encrypted and thus are almost entirely incompressible. Plus, you would need to do a sector-by-sector raw image (or clone, your choice) of the entire drive to ensure that no data is skipped, even if it's currently in free space.

It sounds like the simplest approach would be to pull one of the two drives and connect it directly to your system. If you don't have an external caddy and if you'd rather not install the drive internally, you can actually run a bare drive outside the case temporarily. I probably shouldn't recommend this, but I do it myself, so what the heck. Just remove the side of the computer case, plug a spare SATA cable into the motherboard and the drive, drag out an unused power connector and plug it in, set the drive on a small stack of books if necessary to get it up to the right height, then boot the system and run the drive it that way. Seems OK for small jobs.

The DiskPart procedure will take just a few minutes, and if it works then you can reinstall the drive into the RAID array and rebuild the mirror (and make sure you do this in the correct direction!)

Incidentally, make sure you save the small test file that you created earlier, as it contains a backup copy of your encryption header. Might need that later on if we screw this up!

 

Okay, so I think I'll try option 1 this first time. If that doesn't work I'll prepare for option 2 utilizing the backup or mirrored drive at a later time.

I opened disk part and it appears to be a DOS prompt. Is that right?

 

Sorry, I have been too busy to respond. Do I understand that you want to go ahead and try the diskpart procedure without first removing one of your mirrored drives or otherwise making a backup? Sounds unnecessarily risky to me.

Yes, diskpart is run at a dos prompt. You could also probably do this using Windows 7 disk manager by creating a new partition of the appropriate size, as long as you are careful not to format or assign a file system to the partition.

 

No, I am going to remove a drive from the external encasing and install it in my system. If the diskpart procedure fails, I will purchase a WinHex license (as soon as I can) and utilize the second drive. If it comes to that point, I will theoretically have a 1 TB drive to use to the image the remaining drive.

 

If your lost partition originally filled the entire drive (which is what it sounds like, from what I can tell) then you can just use Windows 7's disk manager to recreate the default (maximally-sized) partition. Make sure you don't assign a file system or format it.

Alternatively, follow the diskpart instructions from this thread:
https://www.wilderssecurity.com/showthread.php?t=327959 (below):

Be aware that even if this works, the endpoint of the partition may be incorrect, which among other things means that your embedded backup headers will no longer function.

Once you have access to your data, I suggest backing it up to another drive right away. Also, back up your TC volume header. If the partition's endpoint is incorrect (as seen by the fact that you can't mount the volume by using the embedded backup header) then it would be best to wipe the drive, create a fresh partition, encrypt it and then copy your data back in.

Incidentally, I have to ask what benefit you hope to gain by using mirrored drives. Are you under the impression that this constitutes a reliable backup system? It doesn't, as you have just seen. The primary purpose of drive mirroring is to minimize downtime or data loss in the event of a single drive failure.

 

Yes, I thought utilizing a mirrored configuration would nearly eliminate my chances of data loss. What configuration would you recommend?

I will need to come up with some storage so I can back up the data, when (if) I can get to it. Will keep you posted, thank you!

 

Alright, here's where I'm at. I ordered some SATA cables to install the drive in my machine. I ordered from Amazon and did not realize they shipped from China. I just got them this week. This is partly why it took so long for me to get back. Anyways, so I plugged in the SATA and power cables and when I try to boot my machine, I get stuck after the BIOS screen. It says that I need to select a bootable device...then select any key. I restarted, and hit F8 to select boot device. Selected the drive with my OS and still no full boot into the OS. When I unplug the SATA cable, I can boot into the OS just fine. Any ideas?

 

Not sure what's going on. Does your bios recognize the drive? If not, maybe the jumpers are set incorrectly.

 

I thought jumpers were not necessary for SATA drives. Is that true? I can't remember. I'm trying to find some of those little plastic pieces. I might need to pick some up.

 

Well, f-word. Finally got the drive recognized by the BIOS. Everything was working great in DiskPart. Created the partition. The partition name now showed up in TrueCrypt. I got very excited. However, when I try to mount, TC says, "Incorrect password or not a TrueCrypt volume". I never changed my password. I'm not sure how this is possible? I shouldn't need to place the drive back in the external casing, right?

 

Hmmm, that's weird. Especially since in Post #10 you stated that you were able to get the header working (in a test file) and it was apparently located correctly on the disk. Did you use the exact steps I posted to recreate the partition? Did anything seem to go amiss during the process? You might want to re-try the procedure in Post #9 and see if it still works for you (but use a different file name for the test file so you won't overwrite your original test file).

You could also use WinHex to look at the partition manually to see if its first sector (the header) is still intact and if the partition actually begins at 1048576 (decimal). But even if the header is damaged, no worries, you have a copy. The original file that you created from the Post #9 procedure contains a working copy of the header, so if necessary you can back up that header and then restore it to the new partition. I think that'll work, but don't do it yet, as we haven't yet confirmed that the partition is located correctly.

But first, try using the embedded backup header to mount the volume. There's a slim chance that this might work. (Mount Options, Use Backup Header embedded in volume etc.)

 

Everything seemed to go smoothly when recreating the partition. I believe I performed the steps accurately. I re-performed the steps in post #9 and was not able to mount the file. I then made a copy of the original test file we did in #9 and successfully mounted it in TrueCrypt.

In WinHex, I tried to Go To offset 1048576 (bytes) but received a message that it did not exist on the volume.

I find the password issue intriguing. I hope I did not mess something up.

 

You would go to that offset on the hard drive, not the test file. Make sure you are in Hexadecimal Decimal mode first. You should see what you saw last time, which was an obvious transition point between a lot of zeros and the beginning of a very large block of random data.

The fact that you are no longer able to create a mountable test file using the same procedure as before indicates that either you are doing something differently, or the disk got altered at that location. The DiskPart commands that I listed would not normally write to that portion of the disk. Hmmm, I'll have to think about this one.