Secure deletion: a single overwrite will do it

Discussion in 'other security issues & news' started by ronjor, Jan 17, 2009.

Thread Status:
Not open for further replies.
  1. ronjor
    Offline

    ronjor Global Moderator

    Heise
  2. Mrkvonic
    Offline

    Mrkvonic Linux Systems Expert

    Well, I've only ever used simple dd single pass whenever needed, never felt the paranoia urge for more. But some people need to feel that their data is really, really, really safely erased and this has nothing to do with technology ... :)
    Mrk
  3. Cerxes
    Offline

    Cerxes Registered Member

    I'm using DBAN DoD short method (3-pass), not for I consider it safer than the quick erase method (1-pass, zeroes), it's rather because of pure laziness from my part. You see, you first have to press 'M', and then you can change the method...

    /C.
  4. dw426
    Offline

    dw426 Registered Member

    I only have one issue with this Ronjor, and that is that (not going to get political here) this information is based on what civilian recovery methods can/can't do, and on known methods at that. The way things are "behind the scenes" can be a different story. I don't need to bring paranoia into this, but one must always remember that there is always someone working on "what can't be done" and on new technologies.
  5. AKAJohnDoe
    Offline

    AKAJohnDoe Registered Member

    While a simple deletion often only disconnects the dataset from the VTOC (in archaic terms) and therefore can result in recovery, an actual overwrite of the full dataset contents with other data patterns is highly effective. If the technology does exist to recover these shadow patterns in the magnetic microstructure, the only folks with access to it would not be interested in the mundane doings of folks such as you and I. :cool: Well at least I, cannot really say about you!
  6. noone_particular
    Offline

    noone_particular Registered Member

    Agreed. We don't know what the recovery abilities of non-civilian (NSA, military) options are. It's likely to be much more than they admit to. The only thing that's certain is that we don't know the whole story.

    Unless you're erasing large files or huge numbers of them, the only difference is that a multiple pass overwrite takes a bit longer. Most PCs can perform the overwrite while doing other normal tasks. There's no productivity loss unless you're one who has to sit there and watch the overwrite take place. A multi-pass overwrite definitely doesn't hurt anything.

    In todays political climate, surveillance, mis-information, and misrepresentation of abilities are the norms. If using a multi-pass overwrite results in no benefit or liability, why the need to keep telling users that one pass is enough, unless it does make a difference to the "non-civilian" interests. IMO, that emphasis on "once is enough" is sufficient reason to use more on anything I consider sensitive, especially since it doesn't cost anything to do so.
  7. Searching_ _ _
    Offline

    Searching_ _ _ Registered Member

    I have found that it is not just the erasing of data that is the issue but whether a particular program can write to all sectors including HPA and DCO.
    For example; In Linux, Shred will only wipe the user space with the command /dev/hda1, but will wipe the partition table also with the command /dev/hda.
    dd does not wipe the HPA.
    If a malware can write to an HPA or create its own HPA, then wiping with a program that doesn't support wiping the HPA, is useless.
    HPA = Hidden Partition Area
  8. chronomatic
    Offline

    chronomatic Registered Member

    Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    I've been looking through this forum and have seen a lot of posts about secure deletion of hard disk data. It seems many people are under the impression that one must overwrite the disk numerous times (many say 35 passes) in order to make data irretrievable. Yet no one ever provides a single shred of evidence to validate that claim. This "multiple pass" theory has grown into what amounts to an urban legend.

    Well, some of you may know this already, but a research paper released earlier this year finally puts this theory to rest. Three researchers carried out experiments on multiple modern hard drives of numerous makes, models and sizes. They even used an electron microscope to see if data could be recovered (many people love to claim that an electron microscope can be used by government agencies -- again they claim this with no examples of it having been done). What were their conclusions when it was all said and done? One pass with zeroes is enough to make data unrecoverable by even electron microscopes.

    A quote from the abstract of the paper:

    And Craig Wright, one of the authors of this paper, said this on a forum:

    So, can we please stop telling people to use the Gutmann wipe method? One wipe with zeroes is all that is needed. Period.

    P.S. I apologize if this paper has been discussed here already.
  9. snowdrift
    Offline

    snowdrift Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Hear, hear!
  10. Nebulus
    Offline

    Nebulus Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    We can telling people to use only one zero wipe, but I'm afraid only very few will listen :).
  11. noone_particular
    Offline

    noone_particular Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    There's been a few discussions of this topic here. See this thread.
  12. caspian
    Offline

    caspian Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Is it necessary to use zeros? Or can you do a single random pass?
  13. Nebulus
    Offline

    Nebulus Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    We should wait for the full paper to be published, but I suspect that it's the same. If you are worried that overwritting with zeros can be discovered as an erasing method, I'm afraid that overwritting with pseudorandom data can be easily discovered as well. Sure, the data won't be recovered, but the fact that it was erased is very hard to hide.
  14. snowdrift
    Offline

    snowdrift Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    I use pseudorandom passes. I think it all goes to intent... if you don't mind someone trying to retrieve your data and *knowing* you wiped, use all 0's or 1's. If you want to minimize the appearance of wiping, then use pseudorandom data.
  15. snowdrift
    Offline

    snowdrift Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Also, don't forget to wash your hands after wiping. <snickers>
  16. Fly
    Offline

    Fly Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    I'm just paranoid and I'll keep it that way. :D

    I was not able to read the book/article (for free).

    But did you happen to notice that Symantec was involved in the study (5) ?

    Nice that it stated 'Magnetic devices DO NOT have memory. They experience a hysteresis
    effect. This is NOT Time based and is NOT going to supply the physical
    effects associated with tool marks etc'

    You don't know what a 'hysteresis effect' is ? I didn't either.
    A translation: 'The magnetization of ferromagnetic substances due to a varying magnetic field lags behind the field. This effect is called hysteresis, and the term is used to describe any system in whose response depends not only on its current state, but also upon its past history. The loss of energy per magnetization cycle per volume is given by Steinmetz's equation. '

    I'm most definitely not a physics expert nor do I have a Master's degree in Computer Science. I won't just accept that 'story' as the (full) truth because it's 'scientific' and uses fancy words that I don't understand.

    Magnetic devices have no memory ? Then, by what miracle am I able to retrieve data from my harddrive ? :cool:

    And about the 'modern drive': what is a modern drive ? I can imagine it's much more dificult to retrieve wiped data from a 1000 GB drive then from a 20 GB drive. Especially older systems (and maybe laptops) tend to have smaller
    harddrives.

    I admit, 35 passes is a bit much. But if I want to securely delete data I'll use more than one pass.

    I'm not a 'believer', I suspect there are people/organizations on this planet that can recover a lot more than you think. Maybe not the average hacker or trained police officer, but I bet there are those who can retrieve more than people believe.

    Better safe than sorry.
  17. BlueZannetti
    Offline

    BlueZannetti Administrator

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Many of us have tried, and apparently failed. I've tried a number of times myself, with little effect it seems.

    There's always a lurking "but what about...." black-helicopter-based scenario that's plausible enough ("Hey - I saw exactly the same thing on TV..." - it doesn't matter that it was the SciFi channel...) to appear real, but seemingly devoid of anything that could be realistically termed a technical detail, and the seeds of doubt are planted. Better safe than sorry as noted above.

    Here's the deal - if you overwrite, it's gone. There are a number of technical reasons for this (encoding schemes used, less mechanical slop in current generation read/write positioning mechanisms, scalability issues with extracting meaningful data from modern high information density platters, etc.) but I'm not expert enough in some of them to do justice. In the areas that I am sufficiently expert (e.g. the force microscopy technologies purported to be of forensic use) - my reaction is to get real and come to grips with the scalability issues on modern drives using this type of solution.

    If you want to use 0's, go with it. Does random strike you fancy? Great! But you only need to overwrite once.

    What most people don't seem to appreciate is that residual file metadata, MFT records, information kept in application caches and residual temp files, and system page/hibernation files tend to maintain oodles of readily accessible data that the 35-pass-worrywarts gleefully ignore. It's actually pretty ironic when you think about it.

    Blue
  18. wtsinnc
    Offline

    wtsinnc Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Fly wrote;

    I agree with Fly to this extent;
    we don't and probably never will know what is possible in terms of recovery.
    Like fly, I strongly suspect that law enforcement and National Security agencies do have the means to recover substantial amounts of data from a wiped drive.

    I keep very little sensitive financial data per se, only links to accounts that are password protected as well as encrypted, and absolutely no illegal data on any hard drive that I own.
    -But-
    I will continue to wipe prior to formatting for a reinstall and almost certainly wipe the drive three times (at least) if that drive is to leave my possession.

    For me, it's not paranoia, just a privacy issue.
    Last edited by a moderator: Jun 7, 2009
  19. snowdrift
    Offline

    snowdrift Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Investigations at CMRR at UCSD have shown that a single pass secure erase at lower frequencies results in no remaining data signals and a second erase reduces this signal only slightly more. The resulting data signal to noise ratio (SNR) at the magnetic drive head is below that required to recover data using a disk drive channel . The only recorded signal left in these experiments is a small amount of highly distorted track edge recording which is extremely difficult to recover data from even if the disk is removed from the drive and tested on a spin-stand.

    Many commercial software packages are available using some variation of DoD 5220, some going to as many as 35 overwrite passes. Unfortunately the multiple overwrite approach is not very much more effective than a single overwrite since it does not do much to the remaining track edges where most of the very low level distorted remnant data remains after an overwrite and it takes a lot more time (even with 3 overwrites it can take more than a day to erase a large capacity hard disk drive).
  20. StevieO
    Offline

    StevieO Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Wiping with a few passes with whatever method won't do any harm, so why not, just in case ! The only thing i would say is, i used to daily run a Guttman 35 pass on all the available Temp/Cache/Recycle/IndexDat etc etc files/folders. This was up until a couple of years ago. I was replacing more HD's than seemed normal considering i didn't play games, or watch movies, or p2p download music etc on the pc. These were 5400 and 7200 RPM drives. I thought at least one of the reasons could be, all that overwriting in the same disk areas over and over again every day.

    If you really want to go even further, then wiping the cluster tips is also recommended, if your App supports this function. These areas are where " supposedly " data can possibly be still read. Whether it's some/most/all is open to debate.

    Have a look at CDS in my screeny at an excellent App i highly recommend.

    BlueZannetti makes a very good point about data residing in, residual file metadata, MFT records, information kept in application caches and residual temp files, and system page/hibernation file.

    If you have enough RAM then i suggest disabling the Swap/Page file, if you don't have one then there's nothing to be found ! I always have and had no issues on 98SE and XP. You can do also it on Vista too.

    Attached Files:

    • CDS.png
      CDS.png
      File size:
      29.9 KB
      Views:
      4,185
  21. Warlockz
    Offline

    Warlockz Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    I read a case on a forensic forum where a private investigator made an image of a hd he was investigating for a suspecting spouse, the private investigator found illegal material on the image, so he had to turn his machine over to the FBI when he contacted LEA, the FBI let him have his machine back but only after making him wipe the Image with a 7 wipe pass....kinda makes you wonder why the Feds use these kinds of standards if a single pass is as effective as everyone claims.....
  22. BlueZannetti
    Offline

    BlueZannetti Administrator

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Sometimes it is as simple as "because that is the formally articulated standard designed to cover all situations" which was initially developed and it hasn't changed as technology has changed with time.

    In the days of low information density drives, some of these "overkill" schemes made a certain amount of sense since devoting huge efforts to unravel potential state secrets may have been feasible endeavor - but let's keep a sense of what's being bandied about here - protecting state secrets that may have cost hundreds of billions to develop vs. a personal HDD. Spending a few hundred million over a year or two to extract information potentially worth billions makes economic sense, doing the same thing to figure out what's on my HDD doesn't..., ever - both in unit cost and time required.

    Blue
  23. noone_particular
    Offline

    noone_particular Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Other than Gutmann's old paper and a few apps that still offer a 35 pass option, I don't see anyone telling people to use 35 passes. To me, the opposite appears to be the case, regular quoting the same "expert" and the same study claiming that "once is enough." We live in an age of suspicion, surveillance, and data collection on everyone. Windows itself looks to be designed for the same purpose.
    Add index.dat files, alternate data streams, hidden folders, registry stored MRUs, etc to the items you listed. It's difficult to come to any other conclusion other than that Windows was designed to store records of most everything you do. If it really makes no difference whether you use one pass or 35 to overwrite it, this "you don't have to do that" emphasis is enough to make me think that it does make a difference to someone. Our present state of surveillance and the extent of data storage would have been called paranoid thinking, impractical, and impossibly expensive 10 years ago. I wouldn't bet against data recovery methods that are equally as advanced, especially when some "expert" makes such an effort to claim that you don't have to do that.
  24. Nebulus
    Offline

    Nebulus Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    In my opinion this sort of study can be helpful, because it might make people realize that they should focus on other things than overwritting the erased data 3,7, or 35 times. Instead, they should pay more attention to the things like index.dat files, alternate data streams, hidden folders, registry stored MRUs (as noone_particular also said), which can give a lot more info about you than a file that was already overwritten once.

    This is not necessary the case... An expert could publish a study for a wide variety of reasons, not only to help some government or conspiracy to get your data easier. For instance, he can publish that to gain more visibility in his comunity, for fame, or even to help people - everybody has his reasons, and they don't always involve some hidden purpose.
  25. chronomatic
    Offline

    chronomatic Registered Member

    Re: Can We Please Put the "Multiple Pass" Wiping Paranoia to Rest?

    Dr. Craig Wright, one of the authors of the above mentioned paper, has a blog post that goes into some detail here.

    A few salient points from the post:

Thread Status:
Not open for further replies.