TrueCrypt: file, partition, drive

Hi there,

I have a 500Gb external USB HD which I want to encrypt with TrueCrypt to work with a Win7 computer(s). I only need a single encrypted volume on it. My options are (I believe):

1) format a single NTFS partition on it, create a large TrueCrypt file-container that nearly fills it, and format it internally as NTFS or FAT32.

2) format a single NTFS partition (of FAT32?) on it, then encrypt the partition wit TrueCrypt.

3) encrypt the the entire drive with TrueCrypt

What are the technical differences between the three?

I almost understand option 1) and its consequences (slightly? slower performance, no deniability, no automated mounting on plugging the device, but less potential for things to go wrong and easier to recover if they do). But what is the difference between 2) and 3), and why one would be better than the other?

I was able to find some advise on the forum, but mostly based on personal opinions - nothing really explaining the differences clearly.

One of the reasons I'm asking (and trying to simplify it) is that previously I had a USB HD partitioned into small unencrypted FAT32 partition and larger NTFS partition, encrypted with TrueCrypt and formated as NTFS internally). Things went wrong at least 3 times over 2 years using it as a backup (corrupted FAT in either one or the other partitions or corrupted TC header). I was able to recover most of the data, but I couldn't figure out why it was happening. I'd like to simplify it, hoping that it won't happen again - but in a process I started getting interested with the device vs. partition difference as well.

Cheers!

 

I found this thread which seems to correspond with my problems of loosing partititions on USB drives whose one partition was encrypted with TrueCrypt. I don't use BootIt, but the description seems to be similar to the problems I had - the lost partitions probably happened wh en the drive was connected and the computer either crashed or was rebooted.

I don't really know how to diagnose the problem but perhaps there are people that do or recall some other threads that describe similar problems so that I can educate myself (I'm searching).

Would that support the idea of having a file-based container as being the least dangerous option, as far as data loss is concerned? Are there some ways of preventing windows with messing with the FAT if a drive/partition is encrypted? Which one makes more sense: drive or partition?

Cheers

 

When you attempt to mount a volume TrueCrypt does not actually search for its volume header, it merely goes to the expected location, reads the first 512 bytes and attempts to run the data through its validation routine. If the header is not 100% correct and/or the wrong password is supplied then the validation fails and TrueCrypt displays the "Incorrect password or not a TrueCrypt volume" message.

For partition encryption, TrueCrypt expects to find the first 512-bytes of the volume header in the first sector of the partition. If, however, the user alters or overwrites their partition table, whether by accident or on purpose, then the partition's starting offset is changed (although the data often stays put) and as a result TrueCrypt cannot find its header. Many users end up breaking their encrypted partitions in this manner. The simplest solution is to re-create the original partition structure exactly as it was, but most users don't store backup copies of their partition tables, so it's not always that easy.

The same methodology applies to the embedded backup header. For partition encryption, TrueCrypt goes to the ending offset of the partition and looks backwards a specified distance in order to "find" the embedded backup header. Of course, this attempt will fail if the partition's ending offset is changed, as sometimes happens.

Thus, partition encryption relies on the partition table and the partition boundaries remaining intact and unchanged. For whatever reasons, many users can't seem to stop themselves from screwing these things up, resulting in the loss of vast amounts of encrypted data. Partition encryption is especially dangerous for that reason.

"Entire device" encryption is much simpler. The volume header is located at the very beginning of the physical drive, and that's where TrueCrypt looks for it. This is a physical location that can't be altered or screwed up by the user, so TrueCrypt will always look in the correct location. The embedded backup header is similarly located, as it is located a specified distance back from the actual end of the physical drive. Thus, the headers can't get "lost". They can be overwritten, and this sometimes happens, but as long as you keep header backups you can just restore them to the drive and regain access to the volume.

The biggest problem with entire-device encryption is that Windows will not recognize the device as being a valid partitioned, formatted drive, and thus it will periodically offer to "fix" it by initializing the drive, writing a fresh MBR & partition table, etc., and any of these actions will overwrite the volume header. You have to say 'no' everytime you are asked, and you dare not leave the device plugged in during a Windows reinstallation or upgrade or Windows will perform the "fix" without even prompting you. This happens to a lot of users. Luckily the embedded backup header is not usually damaged, so a full recovery is usually attainable. (I say 'usually' because sometimes the overwritten area extends into the data, especially if the user allows Windows to format the drive). I consider entire-device encryption to be the safest, as long as the user keeps a backup copy of the headers.

Container files are comparatively safe, as Windows tends to protect all files that are stored within a valid file system regardless of their contents, but there are other issues that create risk. For example, users sometimes delete these files by mistake. Larger files are often fragmented, especially if they fill almost an entire NTFS partition, as this forces them to span the MFT and other structures. It can be quite difficult to find all of the pieces and put them back together again. Data recovery programs can't identify TrueCrypt volumes because they have no common signatures. They need to rely on file pointers in MFT or FAT, and based on the nature of the accident, the records might not be there. The fallback option, hand-assembly using a hex editor, is a bear, as TrueCrypt-encrypted data is quite delicate and must be assembled perfectly or it won't decrypt.

Bottom line: In my opinion, entire-device encryption is the safest as long as you are careful with the disk and you keep backup headers. Partition encryption is the riskiest. Of course, whichever method you choose, backing up all important data is the best approach.

 

You kick butt, dantz Thanks for the lesson.

PD

 

A little bit more about partition encryption:
If you are quite sure that you're not at risk of accidentally altering or overwriting your partition table then you can use encrypted partitions with reasonable safety. However, there's still the problem of Windows continually offering to format the partition, and you might mistakenly hit "OK" one of those times.

If, for example, your partition type is set to 07 (NTFS), which is common, then Windows expects the partition to be formatted as NTFS, but when it looks at the partition contents it can't find a matching NTFS boot sector (because the partition is fully encrypted and consists of purely random data from beginning to end). "Oh, it's broken! Let me fix that!" says Windows, "I can write you a fresh NTFS bootsector and format the partition so it will start working again!" You can either say 'No' every time it comes up, or you can change the partition type from 07 (NTFS) to 83 (Linux) so Windows will stop asking. The TrueCrypt volume will work either way. (However, before switching the partition type make sure you back up your data, just in case you screw up and alter the partition size and/or location by mistake.

@PD: Thanks!

 

Thank you v. much, Dantz! I couldn't agree with PD more!

Given your explanation, backing up disk structure info, in addition to TC headers, is the key to recovery when things go wrong. But what do I back up and how?

For example, is it possible to backup and restore what's needed (MBR? Partition Tables?) using TestDisk? If so, would you know of a 'recipe' of sorts that one could use to decide what needs to be backed up in case of 1) encrypted drive 2) a single partition on encrypted drive 3) multiple partitions on encrypted drive. And what do I need to try to restore (and in what order) when things go wrong?

Would things need to be done differently for NTFS and FAT32 partitions?

Also, even if TC partition or container can be accessed, it contains 'internal' Partition Tables, I understand. Would it make sense to backup these as well? I presume they may not be as vulnerable as 'external' partitions, but I'm not sure. But where to store the backup of the internal tables - inside some other encrypted container?

I thought that perhaps it was just me having a problem, but as I was searching for answers, it seems that there is a lot of people having similar questions or problems. While I'm sure that there is no universal recipe, I think that some sort of sensible and practical guideline on what and how to backup drive partitioning info and then how to diagnose and try to fix the problem when TC encrypted partitions/drives/containers go wrong would be immensely appreciated by the community.

And the last problem, which although may deserve a separate post, may be related enough to partition losing that I ask it here: how about defragmenting from the performance and data safety perspective, not the security or deniability which is covered in TC manual. In particular, for file-hosted containers on a NTFS drive, is defragmenting even needed?, is it risky? does it have any tangible benefits? Then, if it is needed, what do I defragment: the NTFS drive with unmounted TC container or the files inside the mounted TC container or both (and why)?

This thread summarizes the question well.

Cheers!

 

@dantz

Really good and simple explanations you gave there. I've learned a lot. It'll help me in the future.

 

One additional question regarding device encryption: does the fixed location of the TC header and it backup at the physical beginning and the end of the drive, if the entire disk is TC-encrypted, hold true in a wear-leveling devices such as USB flash sticks?

Cheers!

 

Good question, but I don't know how to answer it. I haven't studied wear-leveling mechanisms that much.

 

There are a number of utility programs that can back up the MBR and nearby sectors. I'm pretty sure TestDisk can do it using the "Backup" and "Load Backup" options, but I haven't explored those areas of the program. I personally use WinHex to back up the so-called "Start Sectors" of the physical disk.

I'm sorry mcjtom, but your post contains so many questions that it would take me hours to reply to them all properly, and I don't have that much free time right now. However, keep in mind that disks themselves can fail, whether the data on them is encrypted or not, so the best approach is always to keep multiple backups of important data.

 

Many thanks, Dantz! My apologies for excessive brainstorming.

Would you be able to perhaps just comment on defragmenting an encrypted drive:

1) Is it effective at all?
2) Defragment inner/encrypted file system in the mounted container?
3) Defragment the partition in which the unmounted TC file-container is kept?

This question has been asked here and here previously.

Cheers!

 

Dantz....I tried to send you a private message but I was unable to.

Can you contact me? I have an opportunity for you

Jeff