How to verify secure deletion

[nameless]

I'm evaluating a file-wiping utility. I thought it would be nice to verify that it is overwriting files as they claim it is... So I'm trying to figure out how to do this (within reason--I at least want to know if the correct number of passes are being done). Sysinternals Filemon doesn't seem helpful in this regard. Anyone have any ideas? I'd be very surprised...

Please, don't tell me to use Eraser.

[Climenole]

Hi nameless :)

Simple... try to recover a file erased by your secure eraser utility.

Try this:

Recovery
http://jon.swelter.net/rest2514/

PCInspector File Recovery
http://www.pcinspector.de/file_recovery/fr/welcome.htm

etc.

If a recovery utility may recover your "erased file"
therefore... ;-)

:)

[nameless]

Thanks, but simply trying to recover a wiped file doesn't prove much, though. A simple, single pass of zeroes will prevent all software-based recovery attempts from working. I was hoping to verify that when it says it's doing a 3- or 7-pass "DoD" type of overwriting, it actually was.

[Devinco]

You are looking for forensic data recovery. Not cheap, and some are restricted to law enforcement only.

http://www.e-evidence.info/vendors.html

[nameless]

No; that would be testing how well the data recovery vendors could recover the data. It wouldn't say anything, directly, about how the data had been overwritten. For example, if they recovered a file that I had (supposedly or actually) overwritten with a 7-pass DoD method, would that mean the vendor was really, really good, or that the file hadn't really been overwritten? Or would it mean the file had only been overwritten one time, not 7 times? Or... The possibilities go on. What you're suggesting is akin to determining what type of gun a victim had been shot with, based on whether or not a doctor could save his life.

The advice I was looking for would be something like: "Create a large file, begin overwriting it, then suspend the process and examine the file. Resume the process and examine it again..." (Something like that.) Or, "Here is a utility that can be used to examine the actual I/O being written to disk..."

Thanks anyway...

[Notok]

You mentioned FileMon, but how about DiskMon? If you know the sector that the file is residing on, you could potentially shut down all running apps and use DiskMon to see how many times that sector was written to. I know there's some apps out there that will tell you what file is on what sector. I believe that O&O defrag was among them, at least older versions. Alternately just create the file while looking in DiskMon (with all other apps shut down to minimize other disk writes).

[nameless]

Yeah... Diskmon is a good thought. Already tried it though. :) It doesn't report much info.

I succeeded though... I had the right idea to begin with; I just needed to tweak my approach a bit. The short explanation is that I:

1. Created a large (512-MB) file using WinHex (http://www.x-ways.net/). The large file size was necessary so I'd have time to do what I needed to--a small file would be overwritten too quickly.

2. Opened the file from step 1 in WinHex, using WinHex's "Physical Media" method.

3. Started securely deleting the file using the utility being evaluated, using the 7-pass DoD algorithm.

4. After a few seconds, I suspended the process of the secure deletion utility. (At first, I used Sysinternals Process Explorer (http://www.sysinternals.com/) for this, but I quickly switched to using the "Command Line Process Viewer/Killer/Suspender" (http://beyondlogic.org/solutions/processutil/processutil.htm)* at a command prompt, and I'm glad I did, because it turned out to be much easier and faster.)

5. With the process suspended, refreshed the file in WinHex, to see if the content had changed.

6. Resumed the process that had been suspended.

Then, I repeated steps 4-6 until the secure deletion was complete.

What I saw was that indeed, the file was being filled with specific content for each of 6 passes, and then was filled with what looked to be pseudorandom data on the 7th pass. I didn't verify that the 2nd, 4th, and 6th passes were complements of the 1st, 3rd, and 5th passes... It's too late and I'm not caffeinated enough.

--------

* Note: Some anti-virus software flags this excellent command line utility as "riskware" or whatever cute term they prefer to use.

[tigerbiter]

By the way...
what's wrong with Eraser?

[nameless]

{QUOTE-> By the way...
what's wrong with Eraser? <-QUOTE}
I hate version 5.8, it has annoying bugs such as duplicate context menu entries. Version 5.7 is better, but all versions of Eraser fail to overwrite certain files. I think this happens with very small files that fit entirely into the MFT, but I can't be bothered to investigate--it happens, I'm certain of it, and I'm just as certain that the current "maintainer" of Eraser won't do anything about it.

Edit: I initially forgot another issue I had, which I mention below... Eraser would sometimes (not often, but once is enough!) mysteriously decide to overwrite free space when I was simply trying to delete a single file.

[Genady Prishnikov]

Directory Snoop. Without a doubt. I've used it for years to look at the raw data on my drives.

"Directory Snoop is a cluster-level search tool that allows Windows users to snoop through their FAT and NTFS formatted disk drives to see what data may be hiding in the cracks. Use Directory Snoop to recover deleted files you thought you would never see again or permanently erase sensitive files so that no one will know they ever existed. Supported media include local hard drives, floppy disks, Zip disks, MO disks, and flashcard devices."
It allows you to look at a directory - at a glance - and at individual files. Looking in the "raw mode" can be quite enlightening.

http://www.briggsoft.com/dsnoop.htm

http://www.briggsoft.com/images/ds1.gif

[nameless]

I'm not sure why you recommend Directory Snoop, but as I mentioned above, I already use WinHex, which is capable enough.

[spy1]

{QUOTE-> Version 5.7 is better, but all versions of Eraser fail to overwrite certain files. I think this happens with very small files that fit entirely into the MFT, <-QUOTE}

nameless - Would not SDelete take care of that? From this page:
http://www.microsoft.com/technet/sysinternals/FileAndDisk/SDelete.mspx :

"On NTFS drives SDelete's job isn't necessarily through after it allocates and overwrites the two files. SDelete must also fill any existing free portions of the NTFS MFT (Master File Table) with files that fit within an MFT record. An MFT record is typically 1KB in size, and every file or directory on a disk requires at least one MFT record. Small files are stored entirely within their MFT record, while files that don't fit within a record are allocated clusters outside the MFT. All SDelete has to do to take care of the free MFT space is allocate the largest file it can - when the file occupies all the available space in an MFT Record NTFS will prevent the file from getting larger, since there are no free clusters left on the disk (they are being held by the two files SDelete previously allocated). SDelete then repeats the process. When SDelete can no longer even create a new file, it knows that all the previously free records in the MFT have been completely filled with securely overwritten files."

I run SDelete occasionally only because of the issue you mentioned in the quote above. I'm not even really sure that Eraser doesn't already do it, to wit (from Eraser's "Help" file):
"If you are running Windows NT or 2000 and the file system on the drive is NTFS, Eraser will next overwrite the free space on the Master File Table (MFT). The reason why this is done is that on NTFS file system, clusters are not necessarily allocated for files smaller than the size of a MFT record, but the file is stored completely in the MFT (the file is then said to be resident). If you have insecurely deleted such a small file, the free space on the MFT still may contain the file body and therefore, it must be erased as well."

I only run SDelete occasionally now because the latest version seems to take forever (like about six hours) - the previous version didn't take that long. (Which leads one to wonder whether or not the previous version was doing things correctly or conversely whether the new version is buggy). Pete

[nameless]

I'm not sure at all as to why Eraser 5.x used to skip certain files on me. All I know is that I'd seen it happen many a time.

I've been using SDelete 1.51 for awhile now. I find it extremely fast--especially when overwriting free space. It's so fast, I have a scheduled task overwrite the free space of my Windows partition every night; and sometimes, I do it for grins and giggles, since it takes only a few minutes. It's so fast, in fact, I have wondered if it's doing the job correctly.

I've also been using R-Wipe&Clean (http://www.r-wipe.com/), which I have verified, though only on a file-based 7-pass overwrite.

[spy1]

nameless - I checked into the "long run-time" issue I was having with SDelete and found out that the script I use to run it was set for three passes (which I had known but forgotten).

Changed it to 1 pass, instead, which should be more than sufficient considering everything else I "clean" with here: http://www.wilderssecurity.com/showpost.php?p=843140&postcount=17

Ran a "Zap Free Space" on G drive (Total Size: 66.6GB - Free Space 59.6GB) and it took an hour.

If yours is supposedly running a lot quicker than that, yeah, I'd be concerned that it isn't working correctly. Of course, I'm sure the amount of RAM, HD speed, etc. could account for some of the difference - but not all if SDelete is running too quickly. Pete

[Longboard]

Interesting thread guys
A seminar in itself :thumb:

Nameless can you be more specific re which files Eraser has skipped?
I have used Eraser5.7 for some time with great success, a fantastic utility.
Agree totally about your comments on 5.8 a great shame; severley burnt some users who trusted the update. >:(

I have been using the "verify" function in eraser out of interest since you first started this thread and have seen no problems yet. Seems to be doing as claimed.

Using Sdelete probably bypasses some of the fiddling. Another great tool.
Good that MS acknowledges the authors and history of sysint. tools.

Any clue as to whether Eraser will be Vista compatible.?

[nameless]

{QUOTE-> Nameless can you be more specific re which files Eraser has skipped?
I have used Eraser5.7 for some time with great success, a fantastic utility. <-QUOTE}
No, as I never looked into it, and there's no way I can remember offhand. By the time this problem started happening, Eraser had been abandoned by its original author, Sami Tolvanen. AFAIC, all meaningful development of Eraser died that day.

{QUOTE-> And the Any clue as to whether Eraser will be Vista compatible.? <-QUOTE}
Sorry, no, I have no clue on that. I have next to zero interest in Vista.

[nameless]

It has been so long, I forgot about another problem I had quite a few times with Eraser 5.x... It would occasionally go nuts and, while overwriting some file, blow the file up to fill all available free space. This is yet another issue that I saw repeatedly--and did not attempt to report or diagnose. If it doesn't happen to you, great, but it most definitely did happen to me.

[spy1]

{QUOTE-> Version 5.7 is better, but all versions of Eraser fail to overwrite certain files. I think this happens with very small files that fit entirely into the MFT, but I can't be bothered to investigate-- <-QUOTE}

"but I can't be bothered to investigate--"

{QUOTE-> I'm not sure at all as to why Eraser 5.x used to skip certain files on me. All I know is that I'd seen it happen many a time. <-QUOTE}

{QUOTE-> No, as I never looked into it, and there's no way I can remember offhand. <-QUOTE}

"never looked into it, and there's no way I can remember offhand."

{QUOTE-> It has been so long, I forgot about another problem I had quite a few times with Eraser 5.x... It would occasionally go nuts and, while overwriting some file, blow the file up to fill all available free space. This is yet another issue that I saw repeatedly--and did not attempt to report or diagnose. If it doesn't happen to you, great, but it most definitely did happen to me. <-QUOTE}

"and did not attempt to report or diagnose."

You do realize how completely weak all that sounds, right?

I find it rather odd that you can't provide even a single, substantiated (by something other than your word) instance of an actual occurrence of any of the quoted statements (for whatever reason) - or even remember what the files were?

Eraser is the best donationware option for rock-solid data erasure when it's used correctly - period. And I say that after years of use and after having "donated" to support the program.

Twice.

Try checking out the numerous computer forensics sites - somewhere or other on all of them you'll find a statement to the effect (by people who know far better than you or I) that a single free-space wipe with Eraser on a disk where all the relevant information has been properly marked as free-space makes that data totally non-recoverable.

That's good enough for me. Pete

[nameless]

{QUOTE-> "but I can't be bothered to investigate--"

"never looked into it, and there's no way I can remember offhand."

"and did not attempt to report or diagnose."

You do realize how completely weak all that sounds, right?

I find it rather odd that you can't provide even a single, substantiated (by something other than your word) instance of an actual occurrence of any of the quoted statements (for whatever reason) - or even remember what the files were?

Eraser is the best donationware option for rock-solid data erasure when it's used correctly - period. And I say that after years of use and after having "donated" to support the program.

Twice.

Try checking out the numerous computer forensics sites - somewhere or other on all of them you'll find a statement to the effect (by people who know far better than you or I) that a single free-space wipe with Eraser on a disk where all the relevant information has been properly marked as free-space makes that data totally non-recoverable.

That's good enough for me. Pete <-QUOTE}

I'm not sure why you're going off on me, but anyway... I don't care if it sounds "weak". Your first mistake would be to think I mentioned any of this to persuade you or anyone else one way or the other on the issue. That isn't meant to be argumentative, but literal: I'm saying if Eraser works well for you, then great! Keep using it. I'm not out here to slam it or prevent anyone else from using it!

I only said what I did because someone asked me what I found "wrong" with Eraser. They asked, I answered. I didn't investigate or report (most of) the issues, and I have no obligation to. I also have no obligation to prove anything here. The issues I mentioned happened on my system, I have no reason to lie about them, and that's all there is to it. There is no reason to find it "odd" that I can't substantiate anything with evidence--it happened quite a long time ago, and I didn't take notes. The software behaved badly for me, repeatedly, and I couldn't be bothered to diagnose it, mainly because by that point, I had long deemed Eraser a neglected project.

I have a high opinion of Eraser's efficacy, always have, and never said otherwise.

Again, the problems I mentioned were very real, and no one is going to tell me otherwise. But since Eraser works well for you, cool!

[Blackspear]

Ladies and Gentlemen, back to the topic at hand and let's leave the personal swipes out of this.

Blackspear.

[nameless]

None of the quotes that follow, taken from the Eraser forum, were posted by me.

OK, here you go, other users reporting a failure to overwrite certain files:

http://bbs.heidi.ie/viewtopic.php?t=1990&sid=084ce0e0dbd42ddf86d26556e34faa10
http://www.cipherserver.com/phpbb2/viewtopic.php?t=1031&sid=bcd4d07343d279d9b48c0f4892fa4466
http://www.snugserver.com/phpbb2/viewtopic.php?t=1518&highlight=end+file+reached&sid=057d6c415eb3b8e8e414230988b79360

{QUOTE-> I've found that when I use the shell extension in I.E. to erase a single small file, it won't erase. A single large file will erase (several mb), but a single file of only a few k doesn't erase. Multiple small files erased together work fine. ...This is within Windows Explorer -- I right-click on a file, select Erase, I get the Confirm Erasing box, select Yes, and the box disappears but the file is still visible in Explorer. <-QUOTE}

{QUOTE-> Using an older version (don't know if this was fixed in later versions) a single file often won't erase. If another file is copied to the same directory and then both selected, they will erase. <-QUOTE}

{QUOTE-> Yep, you can't erase small files at all...Eraser works on my work computer but doesn't work with my home computer. Yes I tried to delete 1 byte file with both computer. 5.7 works some times. <-QUOTE}

(P.S. When I experienced this problem, it was not related to NTFS compression.)

Regarding the issue where Eraser would go nuts and inflate the size of a file being overwritten...

http://bbs.heidi.ie/viewtopic.php?t=1521&sid=084ce0e0dbd42ddf86d26556e34faa10
http://www.snugserver.com/phpbb2/viewtopic.php?t=58

{QUOTE-> I've been using Eraser 5.7 since I first discovered it off a website somewhere...But every now and then I would run the program, and it would start to run wild. I'm trying to erase multiple files, then Eraser gets "stuck" erasing one of the files, so I try to stop it, and freezes up! And, at the same time, the file that Eraser was supposed to clean suddenly becomes huge!--sometimes huge enough to use up all the space on my hard drive! <-QUOTE}

{QUOTE-> I've had this same problem intermittently as well <-QUOTE}

{QUOTE-> Ive had the exact same problem since converting my partitions to NTFS a few weeks ago...the wiping progress meter appears to freeze and so I click cancel. Despite doing this, disk activity continues and the file grows until the partition is full before the eraser task closes. <-QUOTE}

{QUOTE-> 3 years later . . . I am using Eraser 5.7 and one of my PCs recently started to show this behavior too. I have never seen it before and I have been using Eraser for over 2 years. This particular machine is running Win XP SP2...I believe that I solved the issue. Just degragmented the hdd that was badly fragmented. <-QUOTE}

(P.S. When I experienced this problem, it had nothing to do with the "Enable Background (slow) Entropy Polling" option--I'd always kept that disabled, since it has long been a bug under Win2K/XP. Also, file fragmentation was not at issue, since I've always been a defragoholic.)

I could go on and on, because there are quite a few references to the same issues I described.

Experience tells me that it's best I stop watching this thread now, or I may find myself on "post watch" again. Catch you later.

[nameless]

{QUOTE-> Ladies and Gentlemen, back to the topic at hand and let's leave the personal swipes out of this. <-QUOTE}
Indeed--thank you!

[spy1]

Substantiation provided and accepted (gotta love that! ;D ). My apologies if I was out-of-line. Pete

[Genady Prishnikov]

{QUOTE-> I'm not sure why you recommend Directory Snoop, but as I mentioned above, I already use WinHex, which is capable enough. <-QUOTE}

Maybe because it can verify if a wiping utility has done its job?
Isn't that what you asked about?
If you have it all figured out, why ask the question? By the way, which post "resolved" your question as I just read the whole thread again and you had a come back comment about every suggestion. I didn't see where anything helped you "resolve" your question as you claim in your edit of the subject title. Frankly, it appeared to me as if you asked your question and then went on to answer it yourself as if responding to an original post started by someone else. It was your thread. Some of us tried to offer suggestions and you seemed to already know the answer to your own question and went on to tell us why our suggestions were either wrong or irrelevant. Just a little truth here, not meaning to be argumentative.

[Sam Miller]

Well, actually, you need to some software that monitors access to a certain cluster on disk, but I think that it doesn't worth doing so.

What are DoD (or NSA) standards - these standards specify the number of passes (7, 10, 14, 1024?), but if it really makes your files more secure? The answer is NO. With today’s hard disks it doesn't matter if you overwrite file one or twice or 10 times. The information will be overwritten once and it is quite enough.

Why publishers of file wipers like talking about number of passes? Actually, because some government agencies do so. These standards were invented for old hard disks (even types) many years ago and ... actually, they are making people see in the wrong direction...

What is the right direction to see? It's what files are actually still at your hard disk deleted and not overwritten any number of times!!

You can check with any file recovery utility, if the files like ~@sometext are recoverable? If you see some then you can recover them and have a data from them - what is it? These files are temporary files of MS Word and contains a copy of data from actual word file.

What can be done about this? Well, the best ideas, I think, is using background mode file shredder utility, so that it could capture all files that are deleted in your Windows system, but you, by programs, by system.

Another good idea is using wipe free space function, but in some cases it will not work correctly, as you will be able to do wipe free space only once a week or once a day, but secure files are changed more often.

Home this was useful. Feel free to ask me any questions about file shredders.

[n8chavez]

Have there been any updates to this? I ask because I am currently stuck in my decision making process between sdelete and eraser. I like eraser. But for me there is something about command line driven utilities that I like. I guess I like the amount of control the user is given, and the fact that I can add and customize context menue entries for that utility. That was something Eraser could not do. I am curious as to the effectiveness of both of these; whether sdelete is better at what is claims than eraser or visa versa.

[spy1]

http://www.microsoft.com/technet/sysinternals/Security/SDelete.mspx

"The reason that SDelete does not securely delete file names when cleaning disk free space is that deleting them would require direct manipulation of directory structures. Directory structures can have free space containing deleted file names, but the free directory space is not available for allocation to other files. Hence, SDelete has no way of allocating this free space so that it can securely overwrite it."

If you can live with the fact that file names won't be over-written by SDelete, then go for it.

If you can't live with that, then simply use both programs. Pete

[sussane]

Use LINUX to delete the files permanently in secure way , use dd command..

[n8chavez]

spy1,

The paragraph above the one you posted give somewhat conflicting information.
{QUOTE-> To overwrite file names of a file that you delete, SDelete renames the file 26 times, each time replacing each character of the file's name with a successive alphabetic character. For instance, the first rename of "foo.txt" would be to "AAA.AAA". <-QUOTE}

It would seem then that filenames are taken care of, but they are not "deleted" ujst overwritten; there would then be a bunch of files names AAA.AAA, or something to that effect. Am I correct here? Perhaps there is something I'm missing, but it seems to me that would be just as good as "deleting" the filenames because after using sdelete they would be effectively indecipherable.

[coolbluewater]

Back at the Farm, we use a torch

[ottocart]

Try a program like Hex Workshop from BreakPoint software (http://www.bpsoft.com/). You can view the actual disk with this program by going to My Computer and right-clicking on the drive you want to verify had the file(s) erased. Then search for some text that was unique to that file and Hex Workshop will search your entire disk for that text. If it was not found, then the files you wanted to delete are GONE.

[AJohn]

I personally respect Privacy Eraser Pro and E3 (Radsoft) the most. ottocart has a nice suggestion for verification.

[Franklin]

The next time I need to wipe a HD I am going try the below.
{QUOTE-> Something called Secure Erase, a set of commands embedded in most ATA drives built since 2001. If this is so wonderful, why haven’t you heard of it before? Because it’s been disabled by most motherboard BIOSes.

Secure Erase is a loaded gun aimed right at all your data. And Murphy’s Law is still in force. But hey, if you’re smart enough to read Storage Bits, you’re smart enough to not play with Secure Erase until you need to. <-QUOTE}
http://blogs.zdnet.com/storage/?p=129

[Longboard]

Isnt that the same as "Boot and Nuke" ?
http://dban.sourceforge.net/

[Franklin]

{QUOTE-> Isnt that the same as "Boot and Nuke" ?
http://dban.sourceforge.net/ <-QUOTE}
Don't really know but I will try the app from your link as the one in my post didn't seem to do anything.

[AJohn]

It's at http://cmrr.ucsd.edu/Hughes/SecureErase.html "Download Freeware Secure Erase Utility"