Default Truecrypt encrypted system unbootable - salvage the data possible? [[SOLVED]]

Hello everyone!
This is my first post to the forum, unfortunately it's a call for help... I've seen some TC discussions here and thought people might have some advice.

I'm dual-booting my laptop (on a single hard drive): Windows Vista (TC full system encryption, don't remember exact version but it was the latest about 6 months ago) and Ubuntu 10.04 (fully encrypted with LUKS, on which I type right now). Using GRUB on MBR, its menu allows loading one of the two - I copied the TC bootloader to an additional partition (unencrypted, /boot) where GRUB files lie, and I chainload to it.

Yesterday, after deleting and moving around some partitions (not the encrypted ones!) and testing Windows 7 installation, some system process decided to "do me a favor" and to "fix" my encrypted Vista partition without asking... Which apparently wrote over the beginning of the partition, erasing the TC header data and practically making it a simple NTFS partition (I was horrified to find out that I can simply mount it from Linux as if it wasn't encrypted! In fact there are files there that take up a few good MB. Using a hex editor I can see that it's no longer truecrypt's "random garbage" it once was). Obviously, this meant the TC bootloader is useless as it keeps saying "No bootable system found" or something similar. Trying to mount this from Linux as a TC drive yields a similar result - "Wrong password or not a TrueCrypt volume" (the password is correct). What all this means is that without the headers all the data is lost. My entire university studying material is there, lectures, notes and all the rest (shame on me for not making proper backups)!

Using the original Rescue Disk I restored TC's bootloader to MBR (worked in the past when GRUB was screwed) - doesn't work. used it to restore the standard volume header, it says the restoration is complete, yet still doesn't work. Looking at the partition with the same editor I can in fact see that the headers weren't written over at all! It didn't restore anything.

I can deal with having the entire system unbootable - if only I could fix the headers thing I might be able to mount it through linux and salvage the data! However the rescue disk doesn't work. Does anyone know what else can be done, or why the rescue disk doesn't do what should (maybe it's looking for a non-ntfs partition)? If I knew where in the disk the data is kept I might copy it over to the partition.

I'd welcome any advice of any kind as this is unknown territory for me... I'd hate to lose all this data. Thanks in advance!

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

I hate bumping threads, but I'm pretty much stuck and have no new ideas. If no one can think of a reason why the rescue disk doesn't work or how to restore the headers, I'll have no choice but to format the drive.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

I'm not an expert on TC system encryption, but I do have some ideas.

What's the partition layout of the Vista system? Did you encrypt your data partitions separately, by any chance? If so, their headers can't be restored using the rescue disk, but they should contain embedded backup headers that might still be good.

I assume you've restored both the key data and the bootloader and you're trying to get "mount without preboot authentication" to work? The rescue disk restores the the key data (the header of the encrypted system) to the last sector of Track 0, not to any encrypted partition. Typically the encrypted system partition immediately follows the key data. Where is your Vista partition located, and did you encrypt it in that location or move it there afterwards? I ask this because I believe that when you restore the bootloader you also restore the original partition table.

Possibly the multiboot config is getting in the way. If you really need to recover your actual Vista system partition and not just a separate data partition then it might be possible to simplify things by pasting together a solution on an external drive. Copy Track 0 (with the restored key data) to an external drive, set up a single partition on the drive, copy the encrypted system partition into it, then try to mount it without preboot authentication. Or something like that. I'm just throwing out some ideas here, as I've never tried this myself, but it might be worth a shot. If it was just a single partition it would probably work, but if you encrypted an entire Vista system containing multiple partitions then it gets trickier. A few tests would show how all this works, but I don't have the time right now.

You would format the drive and throw away all that important data? I would buy another drive and set that one aside, or at least make an image to an external drive first.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Thanks, dantz! I wasn't aware of the backup at the end of the volume, thought I can only restore from the rescue disk / external backup. However, I thought it just restores it to the beginning of the encrypted partition / volume, where it's read from before the system can be decrypted - did I get it wrong?

Yes, I'm trying to unencrypt and mount it in any way possible, even if the system itself will be left unbootable. I finally managed to clear some space and dumped the partition to a single file on the linux partition.
However, I couldn't restore the partition's headers from its local backup through truecrypt on linux, after choosing to do so for that device and entering the password I still get "Incorrect password or not a truecrypt volume"; this is strange since the backup copy should contain the same original password, so I can only assume it has something to do with the FS type (I see it as NTFS now - it starts with non-random data for about 150MB - don't know if that changes anything...).

Recreating the original structure on a different drive would be tricky, I don't really remember the exact sizes of the partitions, if that's important. Originaly:
0 - MBR
1- Vista (Full-system encryption, just a single partition with OS and user files)
2 - Vista's recovery partition (unencrypted)
-- both came with the machine
3 - /boot (unencrypted, held GRUB files etc.)
4 - Linux (logical volume, encrypted with LUKS)

moved the recovery to the end, created a new partition and installed Windows 7 there, then all of this happened.

Now that I have a backup of the partition I'll feel better about messing with it. Unfortunately I forgot to get up early yesterday and sign up for this semester's classes, as all my old timetables and personal notes were still encrypted. Nothing I can't fix, though.

Thanks again, I'll update if I have new info!

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Sorry if I misled you. The embedded backup headers are present only on non-system volumes that were encrypted independently from the operating system. I only mentioned this as a possibility in case you had encrypted your data partition separately, but now I see that you encrypted your Vista system in a one-shot deal and that your operating system and your data are stored within the same system-encrypted partition.

When you restored the bootloader, did you also restore the key data? You should have at least gotten to the point where you could boot to the TC bootloader and your password was accepted, even if the partially-trashed OS was unable to load past that point. And what happens when you try to boot directly from the rescue disc? This bypasses everything but the "key data" at the end of Track 0.

Also, wasn't your GRUB bootloader overwritten when you restored the TC bootloader? If so, then how are you booting into Linux?

The TC bootloader and the key data (aka the encryption header) reside in Track 0 of the drive. The bootloader takes up much of the first 62 sectors and the 512-byte key data occupies the final sector. The rescue disk is capable of restoring these items only to their original locations. Incidentally, it also contains a copy of your original Windows bootloader, which includes your original partition table if you should need it.

I'm not sure if the "mount without preboot authentication" feature will still work if you move the encrypted system partition away from its standard location (and I'm not too clear about your current layout). This feature is probably the only way you will be able to access whatever data remains in your trashed system partition. And as I mentioned earlier, the header of a system-encrypted volume is not attached to the partition.

You probably ought to set the whole thing up on a separate drive and simplify it down to its basic components rather than trying to juggle all of the pieces into place within your multiboot system. Can't you get your hands on a spare drive somewhere?

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Never mind my previous post. I just did some quick tests using a TC rescue disk, a BartPE and a spare hard drive (which I temporarily set as the boot drive). First I used the rescue disk to restore the key data to the end of Track 0, then I booted from the BartPE, ran TC from the hard disk, selected the first partition of the boot drive and mounted it using "mount without preboot authentication". Piece of cake. If your encrypted Vista installation is in the first partition then I would expect this approach to work for you.

edit: Pretty sure the Vista partition needs to be marked as "active". Of course, if the encrypted partition was partially overwritten then once you mount it you will have to deal with its broken file system. You may need to run data-recovery tools on it.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Sorry for not replying, but I didn't have any progress yet. I've tried different things and none of them worked - mainly, I'm having a hard time finding a spare drive that's big enough to occupy the partition; I managed to find an old one but it had so much read/write errors is corrupted the FS and made it impossible to work with. I might get an external drive from a friend today, then we'll see.

To your question - I returned the bootloader to MBR but since the /boot partition is right after it and is marked as bootable, I can simply hit Esc. and reach GRUB, from which I can boot linux.

I really appreciate the fact that you tried the solution yourself! This reassures me a lot. my first priority now is to get a second drive and copy the partition over, then restore the bootloader and key datawith the rescue disk. I've never used BartPE before but I'll give it a shot - mounting that partition should probably still work from a local, installed Windows system, right? (as in, not from a bartPE disk but from a system on a second drive.)

If everything goes well and I can mount it, even if the FS is completely overwritten I can still use several tools to restore deleted files, which was my intention to begin with (the OS can be fixed later and linux is a good alternative for now). However, I'm concerned by the fact that the partition is now a proper NTFS, mountable and containing written files! (it's been "fixed" and the first 150 MB or so were written over.) How would TC recognize it properly? I encrypted a small partition and copied some 150MB over to the beginning of the broken one, thought maybe now I could fix it and mount it using the backup header - no go, I probably just misunderstood how the whole thing works.

I'll update as soon as I'm making progress. Thanks again!

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Yeah, you don't need to use a BartPE. If you can slave the drive to a Windows-based system (with TC installed) then that would be just as good if not better. Do whichever is more convenient for you. Slaving it to a Windows system will provide more flexibility, especially since you'll probably need to install and run data-recovery software. You can even reinstall Windows and use your own system after you get the partition safely copied to an external drive (if you fully trust the cloning and/or copying process, that is.)

The thing about a BartPE is, you have to either build or borrow one, because you can't just download it (from normal channels, that is). Plus, you need either Windows XP or Windows Server 2003 to build it. Other alternatives include getting your hands on a WinPE disk or basically anything that will let you boot into a Windows environment so you can run TC from there.

I don't think you can use Linux for this, since TC doesn't support system encryption for Linux and thus I would not expect the Linux version to provide the "mount without preboot authentication" option either. However, I'll check into that just to make sure.

PS: If you could get yyzyyz interested in your thread then he could probably provide you with some valuable advice. I assume you've seen this thread? https://www.wilderssecurity.com/showthread.php?t=274342

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Failing to get a large enough HD, I did what I wished to avoid and removed the laptop's drive, and simply connected it to my home box (didn't have the machine with me up until a few days ago). I used the rescue disk to restore both bootloader and header data to the drive, booted into my old WinXP and attempted to mount the partition without pre-boot authentication... and still, "Incorrect password or not a TrueCrypt volume"!!! This is incredibly frustrating. I KNOW the password is correct because the rescue disk accepts it in order to do both restorations, and the funny thing is that when I try to boot from the corrupted partition and enter this password I actually get a Windows error message (but again, the partition is by now a NTFS one and the error doesn't come from the encrypted system part). This is supposed to work, if the password is right and the proper information is in track 0, what's in the way? I can only assume this has something to do with the FS being recognized as NTFS, but I can't figure this out. Am I missing something?

By the way, I can't boot XP on my laptop, Compaq Presario CQ61 has hardware for which no drivers were released for XP, part of an attempt by Microsoft to force new buyers into using Vista...

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

I played around a bit with different partition locations and I noticed that on my computer, the encrypted system partition has to be located in the very first partition, otherwise I get the same error message as you. I'm not sure what criteria TrueCrypt is using here (whether it's looking for the "first partition" or the "active partition" or something else), but apparently your system partition isn't located where or how TC it expects it to be.

I don't think your current problem has anything to do with the contents of the partition. Until your password is accepted TrueCrypt isn't able to decrypt the partition in the slightest, so it has no idea what's in there. The partition could be completely blank for all TC knows, and it would still be able to "mount and decrypt". In fact, I've done some testing using blank partitions, and it works fine. Of course, there's no filesystem once you get it mounted, and nothing actually gets decrypted into a useful state (TrueCrypt "decrypts" the solid block of zeros and displays them as random data), but the whole time TrueCrypt behaves normally and thinks it's doing its job.

Where is your partition located? What precedes it?

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

My system IS the first partition, always has been. The structure is, currently -

==========
MBR - with GRUB, TC bootloader or anything else.
-----------
[Vista]
[GRUB partition]
[Ubuntu, logical volume]
[Windows 7 - reinstalled, now not working]
==========

Vista hasn't moved once since I got the laptop and encrypted it. It was followed by some sort of Vista Restoration partition (unencrypted), then the GRUB part, then Linux. Today I noticed a few unallocated MBs preceding Vista, couldn't remember if they were there to begin with, so I deleted the encrypted partition, merged both and re-wrote the partition back from the dump file. Didn't change a thing.

Disregard what I said about Windows error I got, it wasn't from the encrypted Vista. Nevertheless the rescue disk accepts the password required for recovery so that's got to be correct.

I see... If FS type changes nothing, or even the contents of the part., than it must either be the password, the headers not restored properly, or the original location of the partition? How can TC know if it's "Not a TrueCrypt volume"?
Eventually I'll have to reinstall the laptop But until then I'll keep trying to fix this.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

I kind of wish you hadn't done that. Changing the partition boundaries on an encrypted partition is not a good idea. I've done a lot of work with standard (non-system) TrueCrypt encrypted partitions and I've found that if you alter the beginning of a partition in relation to its data (e.g. enlarging or shrinking the partition) then the data will not decrypt to plaintext no matter what you do. It will still mount and appear to decrypt, but the decrypted contents will be complete gibberish. The only solution is to put the partition boundary back where it belongs. I haven't tested this scenario with system encryption, but it could be the same.

I suppose that if you restored the original bootloader from the rescue disk then the partition layout will be restored, as long as you didn't actually move any of the data when you merged the partitions. But you're playing a dangerous game. This sort of thing shouldn't be tried unless you have a reliable backup (such as a sector-by-sector image of the Vista partition) that you can fall back on. And I would personally be doing all of this type of work on a clone, not on the original drive. And the approach I would have taken would be to move the the entire partition so that it started at Sector 63, rather than merging it with the unallocated space that preceded it.

The "incorrect password or not a TrueCrypt volume" message is kind of misleading. The message normally means that TrueCrypt isn't able to match up the password with the header. Either the password is wrong, the header is damaged or the header isn't where it's supposed to be. For system encryption it apparently can also mean that the volume and the header aren't located properly in relation to each other, but I'm still not sure what criteria TrueCrypt is using to decide that. I'll have to check around and maybe do some more testing to see if I can figure it out.

Is that a typo? Did you mean without GRUB, TC bootloader etc.?

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

During installation on drives with unallocated space, Windows 7 tends to create a small (usually 100MB or 200MB) hidden partition at the front of the drive. Maybe this is what happened to your drive. It isn't supposed to do this to drives that are already partitioned, but maybe it chose to treat your fully encrypted (and thus completely unrecognizable) partition as though it were unallocated space. This might account for the extra MB that you found just before your Vista partition. If so, Windows probably just took the space out of your existing partition, so the fact that you merged it back into your partition might not have hurt anything. I'm just guessing, though. There are too many unknowns and the situation is getting kind of tricky, so I really think you ought to be doing your recovery attempts on a spare drive. I have some ideas on how to proceed using a spare drive, but I'll wait and see what your plans are.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Hi, dantz.

You are absolutely right... In retrospect that was a dumb thing to do, but I only realized it some time later - I was being reckless and impatient. I do have the image of the partition as it was in the beginning; by deleting the partition it became unallocated as well, then I created a new partition and wrote the image over it. But then again, I was running out of options and I wasn't sure (then and now) that this unallocated space was part of the disk partitioning to begin with.

If Windows 7 tried to create a hidden partition on that part of the drive than restoring the original partition to the beginning of the drive would have solved this; it didn't, so I assume that wasn't the case.

Even after connecting the drive to an external windows system and restoring both bootloader and header, I couldn't mount the partition using "without pre-boot authentication".

I can safely say that the password is right and that the header is fine (the one thing I do have is a backup of the original header). The fact of the matter is that if this results from a mismatch between where TC expects to find the encrypted partition and its actual location, than it's pretty much over since I don't have the exact disk layout at hand. Serves me right for not doing enough backups!

No, sorry - over the last year or so that part of the disk contained either GRUB, TC's bootloader or Vista's original bootloader. I guess I need to clarify that statement - right now it's the TC bootloader.

Something I found in the official documentation: http://www.truecrypt.org/docs/?s=encryption-scheme

If the beginning of the partition was written over, wouldn't this make it impossible to decrypt? Also, what would happen if I created a similar encrypted partition with the same encryption algorithm, and simply copy over this part onto my damaged partition - would this make it decryptable with the right password (in which case I can mount it and start restoring files)?

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

No, that quote refers to the header itself, not the encrypted system partition. The encrypted header contains the string "TRUE" and this is confirmed as a part of the header decryption process. The password is used to decrypt the header (which contains the encrypted master key) and the master key is then used to decrypt the volume.

Sorry, that wouldn't work. The master key would be completely different.

I'm still thinking about your problem and I hope to come up with other ideas, but I need to learn a bit more about some of the finer points of system encryption.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Where does Partition 1 begin on your disk? Is it at decimal offset 32256 (XP style), or decimal offset 1048576 (Vista/Windows 7 style)? A clean install typically uses the Vista alignments, wheras an upgrade over XP (or to an XP-partitioned drive) uses the XP alignments.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Well, it couldn't possibly be an XP upgrade since XP cannot support my hardware (as far as I know) - part of Microsoft's plan to force consumers to use Vista. It came pre-installed.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

I've been doing a bit of testing to see if I can write you a procedure that will allow you to access your encrypted system partition via the "mount without preboot auth" feature. My current theory is that your modified (multiboot) MBR is redirecting the boot to a non-standard location rather than handing the boot directly over to the first bootable partition, and this is preventing TC from finding the components that it expects find in the standard locations. I've played around with this quite a bit and have tried to mount every partition on my various internal and external drives (except my OS partition), and it appears that TC will only mount the partition that the MBR would normally transfer the boot sequence to. This even holds true on external non-booting drives. The partition can even be empty. For volume mounting purposes TC doesn't seem to care about the contents of the partition, it just wants everything to line up where it's supposed to be.

One more question: When you first used TC to encrypt your Vista system, was your system already set up to multiboot between Vista and Ubuntu? I ask this because I want to know the status of the MBR that TC copied and saved to the TC rescue disk right before you let it encrypt your system. That is, if you used the TC rescue disk to "restore the original Windows bootloader" would you be restoring a conventional default-style Vista MBR or a modified/customized multi-boot MBR?

I also want to let you know that I'll be leaving tomorrow for about 4 days and will be unable to post until I return. Maybe I'll be able to post the procedure before I leave, but if not then I should be back on Monday or Tuesday.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

If what you're saying is correct this actually make sense, considering what Windows 7 might have done.
Originally it was just Vista and its restoration partition. If I remember correctly, the general steps I took were:
1. resized the Vista partition to make room for linux.
2. Installed linux (Fedora) on the newly-created partition.
3. As part of the installation also installed GRUB on MBR, which allowed dual-booting between the two.
4. Returned to Vista - Full-system encryption (at which point the rescue disk was created, and the new bootloader erased GRUB on MBR).
5. through some live-CD created a backup of TrueCrypt's bootloader from the MBR into GRUB's partition, re-installed GRUB and chainloaded to that backup file.

Don't know if Windows 7's installation process could have altered the boundaries of this partition (without actually starting a full install), but if it did and the bootloader works as you describe then it would certainly explain why it can't find the partition it's looking for. In that case, if I find where the partition originally started (when TC was installed) and move it there, and then restore the TC header and bootloader, it should be possible to mount. (?)

I'm completely unfamiliar with how the bootloader actually works - I was sure it's just a simple program that looks for a bootable, encrypted partition right at the beginning of the drive, and attempts to decrypt it.

Unfortunately I can't find a spare drive (large enough to occupy this partition, anyway), and already asked everyone I know, so I can't test things without using the laptop; otherwise I would have simply emulated this scenario on VMware or Virtualbox... Once the holiday ends and stores will open again (in a couple of days) I'll just go and buy a new one, then I'll be free to experiment. Thanks again, dantz, I really appreciate your efforts! I'll update if I get anything to work while you're gone.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Here's what I would try next. I'm currently too busy to post a complete procedure, so I'll sketch it out instead:

1) make sure that your encrypted system partition is the first partition on the drive and that it begins in the standard location (e.g. decimal offset 1,048,576 for a fresh Vista installation, or whatever is appropriate for your system, 2) download and install a good MBR backup program such as MBRFix, MBRWizard, etc., 3) back up the MBR of the slaved drive, 4) run FixMBR from the Vista disk to write a fresh MBR to the slaved drive (and use the correct parameters such as "FixMBR \device\harddisk2" so you don't overwrite your boot drive). Or if you prefer, use TestDisk for this instead. TestDisk allows you to select the desired hard drive from a list so you can clearly see which drive you're writing to. Some of the MBR backup programs also have this capability, although I haven't tried them personally, 5) try to mount the system volume using the "mount without preboot auth" feature.

If the password is not accepted then it might be worth looking at the drive with a hex editor to confirm that the 512-byte key data (the system volume's header) is still present in Sector 62, as expected. If it's there and everything looks normal but the password still fails then restore the previous MBR from your backup to get back to "normal" and rethink the situation. No harm done (hopefully).

If the password is accepted but the data does not appear to decrypt (as confirmed by examining the mounted volume with a hex editor) then consider whether or not the original starting point of the system partition has been moved in relation to the rest of its data, for example by inserting or cutting sectors from the left side of the partition such that the overall size of the partition has been either enlarged or reduced. I believe this will result in a failure to decrypt after the password has been accepted, in which case the only solution will be to restore the partition to its correct size by adjusting the left side.

I'm not guaranteeing anything, but that's the testing approach I would use if I couldn't get my hands on a big enough external drive to safely assemble the entire scenario away from your original drive. I haven't tested the above steps under your specific conditions and I might be a bit off, so you might have to adapt things a bit. Good luck!

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

Sorry for not updating, I've been extremely busy and this took longer than I planned.
I eventually got a 1TB drive, aligned it properly and was able to make an image of my laptop's drive. Then i went on in an attempt to mimic the original drive and configuration.
- On VirtualBox, created a drive similar in size to mine and installed Vista as first partition, and a fake restoration partition right after it.
- Installed Fedora (original installed linux dist.) as LVM and encrypted it, included the boot partition and installed GRUB.
- Encrypted Vista with TrueCrypt and its bootloader.
I took a snapshot at every step. I then started to physically copy the broken encrypted partition from the laptop to the Vista partition on the virtual drive (this would preserve the virtual partition's boundaries and location but run it over with data), which took a while... Unfortunately I had to quit it, pack my computer and move to my new apartment, so it was only half done - but from what I understand so far, this wouldn't change much as the problem is with the partition's location.
- Booted from another virtual system (a Vista drive I already installed in the past) with the mentioned virtual drive attached, installed TrueCrypt there, attempted to mount without pre-boot authentication - It works! I haven't had a chance yet to try some undelete utility on it to make sure it's accessible (the files would be partially destroyed anyway, as I only copied half the data), but if this works than I have two choices:
1. Write down the exact disk layout (at which block the partition starts) and change the original accordingly, then mount in a similar way and salvage what's possible.
2. It might actually be easier to simply copy over the whole broken partition image to the virtual partition and extract it from there - since I know it works - to a shared folder on my machine.
Just wanted to update the thread, I'll try the next steps tomorrow if I have some spare time.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

That is AMAZING! If it actually works, you really ought to write this up somewhere.

 

Re: Truecrypt encrypted system unbootable - salvage the data possible?

This is weird... I rebooted the virtual machine (into working Vista), tried mounting again but was told the "device was in use". Somehow Vista mounted it as NTFS, and when I explored the drive I saw it contained several system directories - much like what I saw when this all began. Attached the drive to another guest OS I had (XP), same thing (can't mount, device busy, NTFS).

I decided to finish what I started: reverted to latest snapshot, dd'd the entire broken partition over night to the virtual one. dd reported that it only copied 33 GB Out of a 78 GB partition (no idea why!). Nevertheless, I took a snapshot, rebooted, tried to mount without preboot authentication - success again. I ran a full scan with Undelete (worked for me in the past), and it found almost nothing, no file structure, barely a few generic files that I couldn't recognize. Ran TestDisk and tried to undelete, reported back that BS is "bad". At this point I checked the mounted drive with a hex editor to make sure - it's random garbage (a good sign - much better than NTFS).

I don't know what to relate these problems to. Is Vista "fixing" the encrypted partitions, or installing files there? Is this a complication of working on a virtual drive? If the decrypted partition was corrupted would an undelete utility find anything?

If everything is alright I should have been able to salvage something. I can at least be certain now that the bootloader works, the password is correct and that the partition in this layout begins as expected. This doesn't comfort me, though... If I dump the disk layout and try it on the original drive I'll also ruin Linux. I need to start thinking about reinstalling this laptop, need a working Windows environment for my studies and Ubuntu is no substitute, and It's been over a month.

I'll keep tinkering with it, but I'll have less and less spare time. I'm looking into converting a physical partition to a virtual one to avoid the dd issue. I think I made some progress there but I doubt this could go any further. As always, if anyone wants to conduct an experiment I'm willing to be the Guinea pig.

 

I have some good news - with the help I got here I finally managed to retrieve most of the lost files from the corrupted partition. Although the thread has been dead for almost a year, I feel it's important to report how this was solved, so that people with a similar issue who find the old tread will have the complete picture.

The issue in summary (already covered above): First partition on the drive was Windows Vista, that had complete system encryption with TrueCrypt. A Windows 7 installation (that was meant to be installed at the end of the drive) recognized the encrypted partition as unallocated space, and so a 100-200MB hidden partition was created there as part of its installation process. This wrote over the first 50MB or so of the encrypted partition, making it unbootable (even undecypherable - couldn't mount externally with TC (TrueCrypt) even after restoring the key data and bootloader using the rescue CD). There was a new NTFS partition where the encrypted Vista stood before, that could be mounted and accessed...
Solution, in essence:
1) Backup the partition as-is.
2) find out what were the exact beginning/end of the partition before this mess took place ( = where TC expects to find it).
3) recreate such layout on an external (or virtual) drive.
4) copy the data over to the new partition created.
5) restore (using the rescue CD) the key data.
6) mount the broken partition on a working Windows environment with TC, "without pre-boot authentication".
7) And finally, run some undelete utility on the newly created drive (the now-decrypted partition).
* This is all to avoid shifting the partition locally on the disk (I learn my lessons the hard way...), if necessary I suppose this could be simplified to a one operation on the same drive but it's unrecommended.

Below is the solution in more detail, if ever needed. I hope no one finds him/herself in my shoes, but in case they do I hope this part helps. Sorry for the huge wall of text below, tried to make it as short and clear as possible. Feel free to skip it.
--------------------------------------------
1. After finding out what happened, booted to the working Linux partition I already had on the drive, and used "dd" to dump a raw image of the corrupted partition to a single file on an external drive ("dd if=/dev/sda1 of=/mnt/external/sda1_raw_image_dump").

2. TrueCrypt was looking for an encrypted partition starting at a specific point on the drive, but failed to find it after Win7's installation; to find out where that was exactly (I couldn't remember) I used VirtualBox to recreate the drive as it first was when I encrypted Vista. Took note of exact drive layout ("fdisk -l /dev/sda" on linux but any other thorough HD utility works just as well).

3. Bought a large HD to hold all the files (VirtualBox and VMware systems, drives, image files etc.). On VMware created two separate guest systems: 1) Linux (Ubuntu) for handling the raw image with the virtual disks, and 2) Windows XP (for TC's mounting, as mounting without pre-boot authentication isn't available on the linux version, and also for the restoration utilities (see 7)). Both OSs were given small root drives but a lot of available RAM (disk I/O operations were heavy on resources). In addition, created a separate virtual disk, the same size as the one I was trying to restore, and attached it to both system.

4. Through the linux Virtual Machine, created a partition starting where the encrypted one was expected (fdisk), then copied the raw image file onto that partition (dd if=./sda1_raw_image_dump of=/dev/sda1). Note that this required, in my case, installing VMware Tools (to access the local filesystem (raw image) through shared folders, both in linux and Windows). Went to make a cup of coffee, this process takes a couple of hours. Now TC could find the partition where it should be, regardless of the state of its data.

5. Mounted the TrueCrypt Rescue CD on one of the VMs and booted it, used it to restore the bootloader and the key data for the virtual disk.

6. On the WinXP Virtual Machine, used TC to "mount without pre-boot authentication" the partition with the copied-over corrupted data (as read only for safety). A new drive letter appeared - at last I had limited access to the decrypted data.

7. Drive was still inaccessible as-is (corrupt FS), but a recovery utility could now be used on the (temporarily decrypted) data. I used R-Studio's "undelete" and Runtime Software's "GetDataBack for NTFS", both are pretty extensive despite being user-friendly and proved useful in the past. The FS was obviously broken (folder tree was partial at best) so I had to sift through hundreds of nameless folders; eventually I was lucky to find everything I needed intact and in one folder, recovered it all outside of the VM to a shared folder on the local linux host. Also found a short story I wrote that I thought I lost for good, which is nice (it still needs a lot of work, though). Note that scanning 100GB of virtual drive looking for file fragments takes a few hours, so coffee is your friend.
--------------------------------------------

Some notes:
- I know there are easier, simpler ways - this was a patchwork of very small, unsuccessful steps I took over the last year on what little free time I had (had several failed attempts). As I'm on a break from the university now I got the time needed to finish this. If you're in the same situation just do everything on a separate physical drive - a day's work or so and you're done.
- For me it was much easier emulating everything instead, allowed to continue working on the running host, as I couldn't afford hours of downtime on every failed attempt. Also, this enables taking snapshots of the disks and the OS at every step, in case you need to revert back to a prior phase.
- Switched from VirtualBox to VMware at some point - performance issues, among everything.
* If working on VMware, a possible way to convert a raw dd-like image directly to vmdk format (VMware's virtual disk) is using a QEMU command (a process simulator). On linux - "qemu-img convert dd_raw_image_file.dd -O vmdk new_VMware_disk.vmdk" . Note that this will convert an image of a single partition into a drive of its own (not needed in this scenario), however this could be useful if you imaged the entire drive and didn't want to bother running "dd" on a working virtual machine for several hours... The conversion only takes a few minutes.
- I understand there's a way of installing Win7 without creating the hidden partition, though I haven't checked it out yet. If you wish to prevent this situation from happening you might want to look into that (or avoid the issue altogether by installing on a separate drive).

- I'm not very knowledgeable on TrueCrypt, but I think the general approach is correct and that enough of the relevant info is in the thread above. if someone wishes to correct me / add to what was said earlier to make it clearer or more useful, please do so!

Thanks for having this forum open, I hope I can contribute myself in the future. And dantz, thank you for all of your help! I learned a lot from your input, and managed to piece together what went wrong and fix it. Thanks especially for taking the time to simulate this on a small scale to show that a solution is possible. I'll buy you a beer if I ever meet you in person.

 

Wow, that was quite a project! Thanks for sharing your solution. It should come in handy if a technically-proficient user is ever faced with a similar situation. I'm going to study this thread for awhile and maybe try a few things. Since this thread began I've developed a lot of new tricks involving WinHex and system encryption, and I think I see some significant shortcuts that could be used next time. Thanks!