Formatted external with truecrypt volume

Discussion in 'encryption problems' started by donouann, Aug 12, 2013.

Thread Status:
Not open for further replies.
  1. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hi dantz. Yes the container is almost the size of the external. I named it JPU1TB and put all my files in them. I have 2 other externals and I also did the same to provide protection for my files.
     
  2. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Here are screenshots of my 2 externals where I created TC containers in them. The Elements in drive H is 1.5TB and I created a 1TB container and it can be seen as "My Volume". the other Elements in drive J is the one i am having a problem with now. before the format, i could see the container "JPU1TB", but now, it just indicates This folder is empty.
     

    Attached Files:

  3. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    Depending on how much time and effort you want to spend, it might be possible to recover some or all of your lost TrueCrypt file, but it's not going to be easy. For various reasons, lost TC files can be notoriously difficult to recover. Here are some key points that you need to understand before you begin.

    The file itself consists of random-appearing (encrypted) data from beginning to end. This can help in identifying the starting and ending points of the file, especially if obvious transitions between random and non-random data can be identified on the disk (for example, "00 00 00 00 00 E1 DF AD 76 CB", but on a much larger scale). These so-called transition points aren't always present, depending on what sort of data happens to be adjacent to the lost file's endpoints. (This paragraph assumes you are examining the disk using a hex editor such as WinHex, of course.)

    Finding the exact starting offset of the lost file is crucial for two reasons: 1) The first 512 bytes of the TrueCrypt header, which we desperately need to locate, are located at the very beginning of the file, at the file's starting offset. Of course, the header looks just like random data and it blends in perfectly with the rest of the file. 2) The file will not decrypt unless the header is placed at the very beginning of the file. If you're off by even a single byte, it won't decrypt. So if we try to use the embedded backup header instead, or an externally saved backup header if you had one, we would have to insert it into exactly the right place or it won't work.

    We know approximately where to start looking. The file's starting point will be in the general vicinity of the partition's starting offset, and since the file almost filled the partition, its endpoint should be in the general vicinity of the partition's ending offset. The endpoint might be easier to find, because there won't usually be any confusing data next to it, but it will also be less useful because even if we can recover the embedded backup header (which is located a specific distance backwards from the endpoint) we can't use it to decrypt anything unless we can also find the file's exact starting offset. Sometimes this requires a "trial and error" approach until the correct location is found.

    Your file, by virtue of it's size, is almost certainly fragmented, as it must span the partition's MFT and other file system structures. Thus, manual recovery of the entire file will be very difficult. The first large chunk might be recoverable without too much suffering, but trying to find the remaining pieces and fit them all together such that there are no missing or extra bytes could prove to be hellishly difficult. Any missing bytes or extra bytes will break the "chain" of decryption, resulting in the failure of the remainder of the file to decrypt.

    I've done several TC file recoveries, and some of them have been quite difficult, but I'm not an actual data-recovery expert. I have some specific knowledge and I've pieced together an approach that sometimes works, that's all. The methodology is crude and painstaking. Using WinHex, I visually examine the disk in an attempt to locate the file's starting offset and/or ending offset, then I test the likely prospects (using TrueCrypt) to see if I've located an actual volume header (either the main header, or if I must, the embedded backup header). Sometimes I have to sample dozens of times, and it doesn't always work. There are also programmatic ways of performing the above whereby the program "walks the disk" while testing for the presence of the headers, but I haven't resorted to that yet.

    Finally, I use WinHex to reassemble the found pieces onto another large disk, saving them as a single file which can then be mounted by TrueCrypt. Sometimes it works. Sometimes I can only recover the first fragment, in which case the incomplete volume has a broken file system, in which case I use "data-carving" data-recovery tools to pull out whatever data I can from the mounted partial volume. The success of these tools depends on the types of data files that need to be recovered, their fragmentation level, and whether or not they are supported by the tool. And actually, WinHex can also do this type of data recovery.

    There are people out there who know much more than I do about this sort of thing, if you wish to contact them. There are also probably better ways to do this than what I have come up with. The most fruitful approach would be to find somebody with real technical knowledge, hopefully a file-recovery expert, and pair them with me so I can provide them with the specific techniques that they will need for TrueCrypt files. (Some of these techniques and explanations can be seen in my other posts on this forum). It will probably cost you.

    As for doing it all on the forum, I have limited time and can give you limited instruction, but we can try to get started and see how it goes. Hopefully you will be able to recover something. However, if you want to bring in an expert you will probably have better luck, as I don't have detailed knowledge of Windows 7 and the NTFS file system and I will probably miss some important things.
     
    Last edited: Aug 13, 2013
  4. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    Hopefully I didn't just scare you off. If you want I can walk you through a relatively easy WinHex procedure that might, if you're lucky, be able to locate your TC volume header and get you on your way to recovering at least some of your data. There can be no guarantees, of course. Sometimes it's easy, and sometimes it isn't.
     
  5. Amanda

    Amanda Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    2,115
    Location:
    Brasil
    May I suggest TeamViwer? It's would be a lot faster than typing :p
     
  6. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    It might come to that, but I prefer to stay clear of other people's encrypted data, as you never know what might actually be in there. This is also why I don't accept hard drives in the mail, etc.

    Besides, isn't it more fun if everybody gets to watch?
     
  7. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hello. thank you for your response. yes I am willing to go through the Winhex process. You can walk me through the steps. How should I do it and what are the steps? Here's the latest screenshot taken by Winhex..
     
  8. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Latest Winhex screenshot of the drive...:)
     

    Attached Files:

  9. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    I'll write you up a procedure, but I won't be able to finish it and post it until tomorrow, as I am out of free time at the moment.
     
  10. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Ok I'll wait for it Dantz. Actually, though I am not well-versed technically about all these recovery procedures, i'm quite excited in doing this and learning a lot from somebody who knows so much more about the procedure. i am just eager to find the solution step by step. And i thank you lot for painstakingly taking the time to assist me...:D
     
  11. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    OK, try this:
    1. Open WinHex
    2. Tools: Open Disk
    3. From the list of Logical drive letters, select the partition that contains your lost file.
    4. Click OK

    The following is a very brief description of what you're seeing and how to move around in it:

    The "directory browser" at the top of the screen lists the general contents of the partition (such as files and folders). In your case, you're most likely looking at the normal NTFS file system structures such as you would see in almost any freshly formatted Windows 7 partition. You can jump to any of those locations by clicking on them (although we don't actually need to, so don't.)

    The hex and text display (underneath the directory browser) shows the specific location and the exact data content of the area that you are currently looking at. Note that each line of hex is followed by its equivalent line of text. These are basically two different ways of showing the same data.

    The Offset column shows you your location within the partition. The numbers are displayed in hexadecimal format, which you're probably not used to working with (yet).

    Some quick navigation:
    Click your mouse somewhere inside the hex display. Then press Ctrl+Home to move to the very beginning of the partition. Notice that at the bottom of the screen, your Offset is shown to be "0". Try pressing Ctrl+End to move to the very end. Notice the offset has changed. Try dragging the scroll bar up to return back up to the beginning (or nearly so). Press Ctrl+Home to get the rest of the way up. You should be at Offset 0 again.

    About the data:
    Look at the hex data. Each pair of numbers/letters represents a single byte. At this location you should be seeing some groupings of zeros (00 00 00 00 00) mixed in with the rest of the data.

    For our purposes, this implies that you are seeing normal (non-encrypted) data, also known as plaintext. Repetitive "00" bytes are fairly common in an unencrypted formatted partition, even if it does not contain any user data. Truly random data (such as you might find within an encrypted TrueCrypt volume) will almost never contain that many identical bytes in a row.

    You can also press PgDn/PgUp to move one screenful at a time. From Offset 0, press PgDn once while looking in the Text column. Do you see any recognizable text inside the column, such as the embedded error message "A disk read error occurred", the word "NTFS", etc.? This also shows that you are looking at plaintext rather than random data. Any time you see recognizable words or patterns, in either the hex or the text column, you are most likely looking at plaintext.

    I mention this because you'll need to learn how to spot the difference between plaintext and encrypted data so you can find the beginning of your lost file.

    OK, it's time to start looking for the lost file. The file's contents basically consist of a huge block of random data that almost fills the partition, interrupted by some file system structures (the MFT, primarily).

    My so-called "plan" is to position your cursor somewhere inside the random data of your lost file, and then let WinHex search backwards (up) for the hex string "00 00 00 00 00", which we are hoping will be located right before, or at least fairly close to, the point where your lost file begins. Maybe this will work, and maybe it won't:

    5. Grab the WinHex scroll bar and drag it down about 5% to 10% of the distance to the bottom. Don't go too far or the backwards search will take too long. Look at the displayed data and make sure you can't recognize and words or patterns in either the hex or the text. If you do see words or patterns or zeros then we're not inside the lost file, in which case you can try dragging the scroll bar a little lower and trying again. Once you seem to be in random data, go to the next step.

    6. Search: Find Hex Values

    7. Type "0000000000" (ten zeros, without the quotes) into the search box

    8. In the "Search" box, change the direction of the search to "Up".

    9. Click OK and let it run.

    I hope that this will place your cursor in the vicinity of the beginning of your lost file, in which case we can test it to see if the header is intact. Something that looks kind of like this would be an ideal outcome:

    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
    9F 4D BD FE F4 42 1B CF 96 80 74 E6 3B F5 1A ED
    D5 52 01 BE FF 86 76 A3 71 F9 1B 7B EA 15 AB 36
    etc.

    It's not always that obvious, but one can hope. Post your results and I will post additional instructions.

    (Note: The above instructions are based on an older version of WinHex. Hopefully all of the commands are basically the same, but there could be some differences. I'll be updating my version soon and will cross-check it then, but in the meantime just let me know if there are any problems.)
     
  12. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hi dantz. Wow this looks interesting. i'm excited to do this. Will thoroughly study your instructions and follow it step by step. Fingser-crossed....:cool:
     
  13. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hi dantz, i am doing it now. Just to make a few clarifications, I noticed the groupings of zeroes and patterns only come at the very start of the hex display. Should I stop directly when I can no longer notice any pattern? Where should i stop? : 5-10% from the top of the scroll bar or from the bottom?
     
  14. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hello dantz. i just finished with the Winhex search and yhis is what I got. Looksa similar to the sample you posted. i hope this is a good result..:thumb:
     

    Attached Files:

  15. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    From the top, but it's just a rough guess so it might need to be repeated elsewhere.

    Well, your results look encouraging! But don't get your hopes up yet, as we need to test the data starting at CC840000 for the presence of the TrueCrypt header before we can get too excited. To do this we will create a test file based on the data at this location, and then attempt to mount it using TrueCrypt.

    If the intact header is where we hope it is then the password will be accepted and the test file will mount. (No data will be recovered yet, this is just a test to see whether or not we've found the header.)

    In WinHex, with the partition selected, do the following:
    "Edit, Define Block"

    Set up the Define Block dialog box as follows:
    Beginning: CC840000 [Beginning of block]
    End: CC8701FF [End of block]

    If you entered the numbers correctly (I'd suggest using copy & paste) the selected block should encompass just under 200KB of data. (I'd prefer to save a larger block, but I don't know whether or not you are using the evaluation copy of WinHex, which is limited to creating files up to 200KB. If you have a licensed copy then we can set a better file size, one that is more likely to work properly with TC. Let me know, but in the meantime carry on with the instructions to see if it will work.)

    Now save the block as a file:

    "Edit, Copy Block, Into new file"
    Make sure you save the file to a location on a different partition in order to avoid overwriting any of your lost container file. Give the test file a meaningful name such as "CC840000 Test File.tc"

    Close WinHex, open TrueCrypt.
    Select a free drive letter
    Click on "Select file"
    Navigate to your test file, select it, then click on "Open"
    Click on "Mount", provide the password to your lost file, and click OK.

    If your password was accepted then we're in great shape! Don't bother going to Explorer and trying to browse the contents of the mounted volume, because the test file is too small to contain any of your data. All we wanted to do was to see if your password worked, which indicates that we've found the all-important header and the beginning of the lost file.

    If you succeeded in mounting the test file then click on Volume Properties in the TC interface and write down the Size of the file in bytes. Then dismount the volume and close TC. And don't delete the test file, as it can now be used as a header backup.

    I hope it works! Let me know the outcome, then we will move on to the next phase.
     
  16. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    OMG!!!!! IT WORKED dantz!!!! Here is the screenshot of the testfile mounted on tc and the password accepted. Also is the screenshot of the volume properties..OMG!!!! I can't breathe......o_O
     
  17. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Here is the screenshot. Much to my excitement, the first image exceeded 300kb and could not be uploaded..apologies...OMG!!!!...:D
     

    Attached Files:

  18. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    Wow, I can't believe it worked on the first attempt! You've been quite lucky so far. It's usually not this easy.

    The next step will be to try to recover a large chunk of your volume. We should be able to recover the first large portion (up to the point where it straddles the MFT) without too much trouble. The procedure is similar to creating the test file (we just take a bigger chunk) but we have to use a different method because the file is so large that it will overwhelm the "copy block into new file" command. We'll have to use WinHex's imaging features instead.

    However, I don't know whether the file system of the partially-recovered file will still function, as there will no doubt be some important pieces missing. We can try this first, and if it doesn't work well enough to get you back most of your data then we'll have to try to piece together the fragments of the lost file, and this will be quite a bit harder than what we've done so far.

    So, the next step will require a large, empty, formatted partition to store the recovered data in. Maybe if I were more skilled we could try to recover your file "in-place", but I don't know how to do that, so I generally just transfer it all onto another disk. It's safer, anyway.

    Are you using a licensed copy of WinHex? Because if you're using the evaluation copy, I don't think it will be able to do this. Also, how much data was stored in the Truecrypt volume, in proportion to the size of the file and partition?
     
  19. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Yes i am using a licensed copy of Winhex dantz...:)
     
  20. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    O dantz, i will prepare another external to store the recovered volume..If i am not mistaken, I have stored around 600Gb of file in the container I created..Yes i am using a licensed copy of Winhex dantz...:)
     
  21. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Hello dantz. I already have the spare external. We are ready to go. What is the next step?...geeeezz,, im ex;) cited much...
     
  22. dantz

    dantz Registered Member

    Joined:
    Jan 19, 2007
    Posts:
    1,034
    Location:
    Hawaii
    This one's going to be tricky because the file is both huge and fragmented. We may need to try it a few different ways to see which method gets you the most data. I'll test some recovery scenarios & get back to you soon.
     
  23. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Ok dantz, Will wait....:)
     
  24. wearetheborg

    wearetheborg Registered Member

    Joined:
    Nov 14, 2009
    Posts:
    667
    This thread is very interesting, and its a pleasure to see a Jedi (dantz) work magic.
     
  25. donouann

    donouann Registered Member

    Joined:
    Aug 8, 2013
    Posts:
    42
    Location:
    Philippines
    Yes indeed wearetheborg. dantz is really great in what he does. i'm patiently waiting while he tests the recovery scenarios so i can get my files back...;)
     
Thread Status:
Not open for further replies.
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.