Scan for file corruption & errors + Disk wiping

1. 1. Hello. I would like to see if all data files in my computer are still healthy, i.e. they haven't been damaged or corrupt yet. Manually opening one by one is not an option. There are literally millions of files.

I knew the System File Checker but it is only for Windows system files. I want to check the health of all my data files. Disk checking, as far as I know, has nothing to do with file corruption. The drive can be healthy even if some files have been corrupt unrelated to drive failure reasons.

I tried to search on the net but couldn't find any.

Any method or workaround which could check for file corruption & errors, even if the check is not comprehensive or faultless, is much appreciated. Thank you.


2. I would like to ask whether there is any good program which could:
(A) *really* wipe specific files and folders irrecoverably
(B) *really* wipe the data of whole drive/disk irrecoverably
I hear that some programs claim it does but actually not (it's still there when you use the recovery program) so I want something which is really working, be it free or paid. Thank you.

 

File corruptions:

I assume you just want to know whether your files are corrupt or not? You need a Hash checksum generator and checker like "Hashcheck" http://code.kliu.org/hashcheck/

This utility will generate a hash for a set of files. You can keep that hash file safe somewhere and if you need to check whether or not your files are fine just run the hash check. Hashcheck will not correct the corruption for you but it will tell you if corruptions are there. It is also free. I have used it and can confirm it works great.

File wipe:

There are many good utilities out there, I have used Tuneup utilities, Ashampoo winoptimizer myself and both have DoD 5220.22-M (single pass overwrites file 7 times) and Peter Gutmann (single pass overwrites file 35+ times) methods for wiping files. Both methods wipe reliably.

Single pass DoD is fine if you want to protect your info from everyday run of the mill thieves etc. If you want to hide you information from CIA or FBI or IRS, use 1 or 2 passes of Peter Gutmann. (...for wife/girlfriend use 3 passes of Peter Gutmann or get a good lawyer )

Nowadays I use Bestcrypt's BCwipe with single pass DoD to wipe my sensitive information.

 

This method has some problems. Let's say you frequently edit your data files. You want to check if the files are still healthy when something bad occurs (eg crashes, power failure and so on).

The "hash" method works only if the data is rarely edited. Otherwise it's hard to keep an updated of hashes for all files all the time.

The several programs that you recommend looks great. I probably get one which is dedicated to data destruction. I don't want to install too many bloated programs.

PS: Agree on 3 passes for wife/girlfriend.

 

1. For single or multiple secure file deletion/shredding, I use Spybot Search & Destroy with a slightly higher than the default "chop" setting. I do not know how secure it is, but the files that I delete usually aren't that sensitive.

2. For whole disk wiping I have used Terabyte Unlimited's "CopyWipe". I have used only two of the wipe settings: 'Zero Write' and the 'Strong Random Pattern + Zero Write'. I have not used the stronger CopyWipe wipe options because it may take as much as several days to perform the wipe of a single hard drive.

http://www.terabyteunlimited.com/copywipe.php

There's also the "widely used" Darik's Boot and Nuke ("DBAN"):

http://www.dban.org/

 

Thank you.
But after reading a bit about data destruction, I start to suspect there is absolutely no way to securely delete specific files/folders. (that is the case even if we exclude cases like where FBI which has a lot of brainpower and money want to recover your data)
Full drive secure deletion is the only way to do.

Do you know what technique does it use to securely delete the data? The description on its website is too brief to be useful:
http://www.dban.org/about

 

For CopyWipe all I have are the explanations on Page 20 of the CopyWipe manual.

http://www.terabyteunlimited.com/downloads/copywipe.pdf

DBAN - I ran across this, but I do not know if this is up-to-date. I have almost no experience with DBAN.

http://www.security.ku.edu/download/DBAN_Tutorial.pdf

 

Yes, hash check method only works if the files are not edited, in fact it is designed to prevent edits/changes to your files.

If you just want to prevent data corruption due to power failure etc. good old chkdsk should be fine.

If you are concerned that a specific sector in your hard disk might fail, I recommend Spinrite. It reads and writes to all sectors of your hard drive, ensuring all sectors work fine. Spinrite does not destroy existing data.

 

You won't find any, except for some specific file types. It's not theoretically possible to do this in the general case though.

To check that the filesystem is intact, do chkdsk /f, as has already been mentioned.

I'd recommend to periodically make incremental backups of your data files. Then, if you discover that a file is corrupt, you can at least get an older version back, provided an older version exists. Another option is to use a program such as FileHamster to automatically make a copy of a data file any time it changes.

You mentioned that you did not want to use a hash checking program, but perhaps consider using a program such as NIS FileCheck that can hash check only files with extensions of your choosing. This allows you to hash check only those file types that you wouldn't expect to normally change.

 

Well then at least we can check some file types. So what are they?
Archives: RAR, ZIP, 7z
Office files: DOC, XLS, PPT, PDF
Images: JPG, GIF, PNG

I haven't noticed programs like FileHamster exists. Let me take a look at this.
About incremental backups I heard that incremental backup is not a good idea. It's because the 6th or other incremental backups afterward would become useless if 5th incremental backup becomes corrupt/damaged.

Notes: Does FileHamster keep backup of deleted files too, by any chance? You know users sometimes accidentally delete the files by mistake (people do have fat fingers). Data recovery program is fine but recovery-based method is not 100%. Backup-based method is much preferable.

Well I don't oppose using it but it is just impracticable in our cases. I'm glad to apply it into usage if the hash checking can somehow keep tracks of the hash history automatically (or have an easy way to achieve the same thing). The data files are meant to be "active". They keep changing. Running the hash program every time when we change a file is virtually impracticable. I think we would get lost and confused soon before it can proves its usefulness.

That's sound. I will take a look at this too later.

 

I am impressed by the new HDDScan version : it can even detect the T° of external USB drives, where little other utilities can !

 

I wonder how accurate it is. The problem is how it can manage to do that when the external USB drives have no thermal sensor.

 

I suppose the sensors are within the 3"1/2 HDD itself, the intelligence of HDD Scan is how to access them through an USB/IDE or SATA interface. With my USB external HDD by WD (a MyBook Essentials), there is a special utility provided by WesternDigital that can access SMART parameters through the USB, but it only works with external WD HDD. Now with HDD Scan I can also access them, and those of a Buffalo USB/Sata external drive.

 

Another one that I can vouch for as working well is NTI Shadow.

It automatically makes a backup copy to a location you select whener a file is changed. It also keeps a selectable number of revision levels (including "none" if that is what you want).

It can also be configured for file synching instead of just copying, so that two drives always have exactly the same files and revision levels.

Once set, you never even notice it working, until you decide you need to recover something...then, IT IS THERE !

You can set multiple backup job configurations, so that the same file is backied up on multiple drives, or backed up on one and "synched" with another....basically any backup scenario you can invision.

 

I haven't researched this, but of these particular file types, I know that pdfsam will highlight corrupt PDFs when you do a merge operation. You could do some web searches for 'integrity check' along with a given file extension from your list. A search for 'file repair' might also work.

This is true, but you can use a backup program that has a verify backup option. Also, switch to a new backup set every once in awhile, to further reduce that possibility.

I think it asks the user after deletion, but I don't recall. It's been awhile since I used FileHamster. Do a web search for 'continuous backup' to find other similar solutions.

 

If you're concerned about files that ought to not have been altered but were altered nonetheless, you could do the following: Use any program that backs up files in native format, and does not consider file contents as a criteria. You could use SyncToy for this, with the 'check file contents' option turned off. Immediately afterwards, use a program that synchronizes files in native format, gives you a preview option, and does consider file contents as a criteria. You can also use SyncToy for this purpose. Any files that the synchronization program would alter should be inspected for corruption. It should be noted that Microsoft Office files sometimes legitimately change without a change in filesize or last modification date.

 

DBAN has options for all the common deletion methods including DOD and Gutmann.

 

Thanks. I'm still looking for ways to check some common filetypes.

Some zipping programs have a test feature to check if the zipped file is corrupt. Well, if you still have to run one by one. Any chance you know to automate the process of archives checking?

Good tips. A balance between space and safety.
I hope the "verify backup" really works and can save my day. I can only check for sure at the time I restore the image. But I can't do it all the time.

 

If you search for file repair programs, you'll encounter programs such as EasyRecovery FileRepair. Programs such as this should work for you, but whether there are any free ones (if that's a criteria) I don't know.

Offhand I don't know.