How to verify secure deletion

[SOLVED] How to verify secure deletion

I'm evaluating a file-wiping utility. I thought it would be nice to verify that it is overwriting files as they claim it is... So I'm trying to figure out how to do this (within reason--I at least want to know if the correct number of passes are being done). Sysinternals Filemon doesn't seem helpful in this regard. Anyone have any ideas? I'd be very surprised...

Please, don't tell me to use Eraser.

 

Hi nameless

Simple... try to recover a file erased by your secure eraser utility.

Try this:

Recovery
http://jon.swelter.net/rest2514/

PCInspector File Recovery
http://www.pcinspector.de/file_recovery/fr/welcome.htm

etc.

If a recovery utility may recover your "erased file"
therefore... ;-)

 

Thanks, but simply trying to recover a wiped file doesn't prove much, though. A simple, single pass of zeroes will prevent all software-based recovery attempts from working. I was hoping to verify that when it says it's doing a 3- or 7-pass "DoD" type of overwriting, it actually was.

 

You are looking for forensic data recovery. Not cheap, and some are restricted to law enforcement only.

http://www.e-evidence.info/vendors.html

 

No; that would be testing how well the data recovery vendors could recover the data. It wouldn't say anything, directly, about how the data had been overwritten. For example, if they recovered a file that I had (supposedly or actually) overwritten with a 7-pass DoD method, would that mean the vendor was really, really good, or that the file hadn't really been overwritten? Or would it mean the file had only been overwritten one time, not 7 times? Or... The possibilities go on. What you're suggesting is akin to determining what type of gun a victim had been shot with, based on whether or not a doctor could save his life.

The advice I was looking for would be something like: "Create a large file, begin overwriting it, then suspend the process and examine the file. Resume the process and examine it again..." (Something like that.) Or, "Here is a utility that can be used to examine the actual I/O being written to disk..."

Thanks anyway...

 

You mentioned FileMon, but how about DiskMon? If you know the sector that the file is residing on, you could potentially shut down all running apps and use DiskMon to see how many times that sector was written to. I know there's some apps out there that will tell you what file is on what sector. I believe that O&O defrag was among them, at least older versions. Alternately just create the file while looking in DiskMon (with all other apps shut down to minimize other disk writes).

 

Yeah... Diskmon is a good thought. Already tried it though. It doesn't report much info.

I succeeded though... I had the right idea to begin with; I just needed to tweak my approach a bit. The short explanation is that I:

1. Created a large (512-MB) file using WinHex. The large file size was necessary so I'd have time to do what I needed to--a small file would be overwritten too quickly.

2. Opened the file from step 1 in WinHex, using WinHex's "Physical Media" method.

3. Started securely deleting the file using the utility being evaluated, using the 7-pass DoD algorithm.

4. After a few seconds, I suspended the process of the secure deletion utility. (At first, I used Sysinternals Process Explorer for this, but I quickly switched to using the "Command Line Process Viewer/Killer/Suspender"* at a command prompt, and I'm glad I did, because it turned out to be much easier and faster.)

5. With the process suspended, refreshed the file in WinHex, to see if the content had changed.

6. Resumed the process that had been suspended.

Then, I repeated steps 4-6 until the secure deletion was complete.

What I saw was that indeed, the file was being filled with specific content for each of 6 passes, and then was filled with what looked to be pseudorandom data on the 7th pass. I didn't verify that the 2nd, 4th, and 6th passes were complements of the 1st, 3rd, and 5th passes... It's too late and I'm not caffeinated enough.

--------

* Note: Some anti-virus software flags this excellent command line utility as "riskware" or whatever cute term they prefer to use.

 

By the way...
what's wrong with Eraser?

 

I hate version 5.8, it has annoying bugs such as duplicate context menu entries. Version 5.7 is better, but all versions of Eraser fail to overwrite certain files. I think this happens with very small files that fit entirely into the MFT, but I can't be bothered to investigate--it happens, I'm certain of it, and I'm just as certain that the current "maintainer" of Eraser won't do anything about it.

Edit: I initially forgot another issue I had, which I mention below... Eraser would sometimes (not often, but once is enough!) mysteriously decide to overwrite free space when I was simply trying to delete a single file.

 

Directory Snoop. Without a doubt. I've used it for years to look at the raw data on my drives.

"Directory Snoop is a cluster-level search tool that allows Windows users to snoop through their FAT and NTFS formatted disk drives to see what data may be hiding in the cracks. Use Directory Snoop to recover deleted files you thought you would never see again or permanently erase sensitive files so that no one will know they ever existed. Supported media include local hard drives, floppy disks, Zip disks, MO disks, and flashcard devices."
It allows you to look at a directory - at a glance - and at individual files. Looking in the "raw mode" can be quite enlightening.

http://www.briggsoft.com/dsnoop.htm

http://www.briggsoft.com/images/ds1.gif

 

I'm not sure why you recommend Directory Snoop, but as I mentioned above, I already use WinHex, which is capable enough.

 

nameless - Would not SDelete take care of that? From this page:
http://www.microsoft.com/technet/sysinternals/FileAndDisk/SDelete.mspx :

"On NTFS drives SDelete's job isn't necessarily through after it allocates and overwrites the two files. SDelete must also fill any existing free portions of the NTFS MFT (Master File Table) with files that fit within an MFT record. An MFT record is typically 1KB in size, and every file or directory on a disk requires at least one MFT record. Small files are stored entirely within their MFT record, while files that don't fit within a record are allocated clusters outside the MFT. All SDelete has to do to take care of the free MFT space is allocate the largest file it can - when the file occupies all the available space in an MFT Record NTFS will prevent the file from getting larger, since there are no free clusters left on the disk (they are being held by the two files SDelete previously allocated). SDelete then repeats the process. When SDelete can no longer even create a new file, it knows that all the previously free records in the MFT have been completely filled with securely overwritten files."

I run SDelete occasionally only because of the issue you mentioned in the quote above. I'm not even really sure that Eraser doesn't already do it, to wit (from Eraser's "Help" file):
"If you are running Windows NT or 2000 and the file system on the drive is NTFS, Eraser will next overwrite the free space on the Master File Table (MFT). The reason why this is done is that on NTFS file system, clusters are not necessarily allocated for files smaller than the size of a MFT record, but the file is stored completely in the MFT (the file is then said to be resident). If you have insecurely deleted such a small file, the free space on the MFT still may contain the file body and therefore, it must be erased as well."

I only run SDelete occasionally now because the latest version seems to take forever (like about six hours) - the previous version didn't take that long. (Which leads one to wonder whether or not the previous version was doing things correctly or conversely whether the new version is buggy). Pete

 

I'm not sure at all as to why Eraser 5.x used to skip certain files on me. All I know is that I'd seen it happen many a time.

I've been using SDelete 1.51 for awhile now. I find it extremely fast--especially when overwriting free space. It's so fast, I have a scheduled task overwrite the free space of my Windows partition every night; and sometimes, I do it for grins and giggles, since it takes only a few minutes. It's so fast, in fact, I have wondered if it's doing the job correctly.

I've also been using R-Wipe&Clean, which I have verified, though only on a file-based 7-pass overwrite.

 

nameless - I checked into the "long run-time" issue I was having with SDelete and found out that the script I use to run it was set for three passes (which I had known but forgotten).

Changed it to 1 pass, instead, which should be more than sufficient considering everything else I "clean" with here: https://www.wilderssecurity.com/showpost.php?p=843140&postcount=17

Ran a "Zap Free Space" on G drive (Total Size: 66.6GB - Free Space 59.6GB) and it took an hour.

If yours is supposedly running a lot quicker than that, yeah, I'd be concerned that it isn't working correctly. Of course, I'm sure the amount of RAM, HD speed, etc. could account for some of the difference - but not all if SDelete is running too quickly. Pete

 

Interesting thread guys
A seminar in itself

Nameless can you be more specific re which files Eraser has skipped?
I have used Eraser5.7 for some time with great success, a fantastic utility.
Agree totally about your comments on 5.8 a great shame; severley burnt some users who trusted the update.

I have been using the "verify" function in eraser out of interest since you first started this thread and have seen no problems yet. Seems to be doing as claimed.

Using Sdelete probably bypasses some of the fiddling. Another great tool.
Good that MS acknowledges the authors and history of sysint. tools.

Any clue as to whether Eraser will be Vista compatible.?

 

No, as I never looked into it, and there's no way I can remember offhand. By the time this problem started happening, Eraser had been abandoned by its original author, Sami Tolvanen. AFAIC, all meaningful development of Eraser died that day.

Sorry, no, I have no clue on that. I have next to zero interest in Vista.

 

It has been so long, I forgot about another problem I had quite a few times with Eraser 5.x... It would occasionally go nuts and, while overwriting some file, blow the file up to fill all available free space. This is yet another issue that I saw repeatedly--and did not attempt to report or diagnose. If it doesn't happen to you, great, but it most definitely did happen to me.

 

"but I can't be bothered to investigate--"

"never looked into it, and there's no way I can remember offhand."

"and did not attempt to report or diagnose."

You do realize how completely weak all that sounds, right?

I find it rather odd that you can't provide even a single, substantiated (by something other than your word) instance of an actual occurrence of any of the quoted statements (for whatever reason) - or even remember what the files were?

Eraser is the best donationware option for rock-solid data erasure when it's used correctly - period. And I say that after years of use and after having "donated" to support the program.

Twice.

Try checking out the numerous computer forensics sites - somewhere or other on all of them you'll find a statement to the effect (by people who know far better than you or I) that a single free-space wipe with Eraser on a disk where all the relevant information has been properly marked as free-space makes that data totally non-recoverable.

That's good enough for me. Pete

 

I'm not sure why you're going off on me, but anyway... I don't care if it sounds "weak". Your first mistake would be to think I mentioned any of this to persuade you or anyone else one way or the other on the issue. That isn't meant to be argumentative, but literal: I'm saying if Eraser works well for you, then great! Keep using it. I'm not out here to slam it or prevent anyone else from using it!

I only said what I did because someone asked me what I found "wrong" with Eraser. They asked, I answered. I didn't investigate or report (most of) the issues, and I have no obligation to. I also have no obligation to prove anything here. The issues I mentioned happened on my system, I have no reason to lie about them, and that's all there is to it. There is no reason to find it "odd" that I can't substantiate anything with evidence--it happened quite a long time ago, and I didn't take notes. The software behaved badly for me, repeatedly, and I couldn't be bothered to diagnose it, mainly because by that point, I had long deemed Eraser a neglected project.

I have a high opinion of Eraser's efficacy, always have, and never said otherwise.

Again, the problems I mentioned were very real, and no one is going to tell me otherwise. But since Eraser works well for you, cool!

 

Ladies and Gentlemen, back to the topic at hand and let's leave the personal swipes out of this.

Blackspear.

 

None of the quotes that follow, taken from the Eraser forum, were posted by me.

OK, here you go, other users reporting a failure to overwrite certain files:

http://bbs.heidi.ie/viewtopic.php?t=1990&sid=084ce0e0dbd42ddf86d26556e34faa10
http://www.cipherserver.com/phpbb2/viewtopic.php?t=1031&sid=bcd4d07343d279d9b48c0f4892fa4466
http://www.snugserver.com/phpbb2/vi... reached&sid=057d6c415eb3b8e8e414230988b79360

(P.S. When I experienced this problem, it was not related to NTFS compression.)

Regarding the issue where Eraser would go nuts and inflate the size of a file being overwritten...

http://bbs.heidi.ie/viewtopic.php?t=1521&sid=084ce0e0dbd42ddf86d26556e34faa10
http://www.snugserver.com/phpbb2/viewtopic.php?t=58

(P.S. When I experienced this problem, it had nothing to do with the "Enable Background (slow) Entropy Polling" option--I'd always kept that disabled, since it has long been a bug under Win2K/XP. Also, file fragmentation was not at issue, since I've always been a defragoholic.)

I could go on and on, because there are quite a few references to the same issues I described.

Experience tells me that it's best I stop watching this thread now, or I may find myself on "post watch" again. Catch you later.

 

Indeed--thank you!

 

Substantiation provided and accepted (gotta love that! ). My apologies if I was out-of-line. Pete

 

Maybe because it can verify if a wiping utility has done its job?
Isn't that what you asked about?
If you have it all figured out, why ask the question? By the way, which post "resolved" your question as I just read the whole thread again and you had a come back comment about every suggestion. I didn't see where anything helped you "resolve" your question as you claim in your edit of the subject title. Frankly, it appeared to me as if you asked your question and then went on to answer it yourself as if responding to an original post started by someone else. It was your thread. Some of us tried to offer suggestions and you seemed to already know the answer to your own question and went on to tell us why our suggestions were either wrong or irrelevant. Just a little truth here, not meaning to be argumentative.

 

Well, actually, you need to some software that monitors access to a certain cluster on disk, but I think that it doesn't worth doing so.

What are DoD (or NSA) standards - these standards specify the number of passes (7, 10, 14, 1024?), but if it really makes your files more secure? The answer is NO. With today’s hard disks it doesn't matter if you overwrite file one or twice or 10 times. The information will be overwritten once and it is quite enough.

Why publishers of file wipers like talking about number of passes? Actually, because some government agencies do so. These standards were invented for old hard disks (even types) many years ago and ... actually, they are making people see in the wrong direction...

What is the right direction to see? It's what files are actually still at your hard disk deleted and not overwritten any number of times!!

You can check with any file recovery utility, if the files like ~@sometext are recoverable? If you see some then you can recover them and have a data from them - what is it? These files are temporary files of MS Word and contains a copy of data from actual word file.

What can be done about this? Well, the best ideas, I think, is using background mode file shredder utility, so that it could capture all files that are deleted in your Windows system, but you, by programs, by system.

Another good idea is using wipe free space function, but in some cases it will not work correctly, as you will be able to do wipe free space only once a week or once a day, but secure files are changed more often.

Home this was useful. Feel free to ask me any questions about file shredders.