virtual disk=>full FILE-based restore,problem

Not sure, but I think WinPE = Windows Pre(boot) Environment, a miniscule Windows kernel loaded using TFTP from a network server. The ExplorerPE must be a version of the standard Windows Explorer made for WinPE.

 

Okay, time for an update. In short: Full, file-based restore works for me!

Armed with my trust TI 6.0 Rescue CD, I made a full image of my little laptop to my desktop PC. The disk layout on the laptop is this:

1. C-partition, 10 GB, NTFS, Primary & Active.
   Windows 2000 Pro SP-3 + Programs.
2. D-partition, 18 GB, NTFS, Logical.
   Data + Optional programs.
3. E, CD/DVD reader.
4. F-partition, 10 GB, NTFS, Logical.
   Spare partition used to host a temporary install of Win 2000 Pro SP-3 + Acronis True Image 6.0.

I rebooted into C, and loaded the temporary Win2K install on F. From there, I removed non-essential folders and files from C: leaving only a handful of system files, essential for the boot process:

Code:

22-07-2002  14:05              150.528 arcldr.exe
22-07-2002  14:05              163.840 arcsetup.exe
13-05-2005  01:05                  409 boot.ini
08-05-2001  15:00              229.792 cmldr
24-05-2005  21:22          267.894.784 hiberfil.sys
25-09-2001  20:38                    0 IO.SYS
25-09-2001  20:38                    0 MSDOS.SYS
03-12-2003  15:51               34.724 NTDETECT.COM
03-12-2003  15:51              214.432 ntldr
24-05-2005  21:22          402.653.184 pagefile.sys
14-04-2005  21:47       <DIR>          RECYCLER
14-04-2005  09:22       <DIR>          System Volume Information

Rebooted into the temporary Win2K on F, and trimmed the C-partition into a true anorexic boot partition:

Code:

22-07-2002  14:05              150.528 arcldr.exe
22-07-2002  14:05              163.840 arcsetup.exe
13-05-2005  01:05                  409 boot.ini
03-12-2003  15:51               34.724 NTDETECT.COM
03-12-2003  15:51              214.432 ntldr
14-04-2005  21:47       <DIR>          RECYCLER
14-04-2005  09:22       <DIR>          System Volume Information

Defragmented the C-partition, rebooted into F, and started to re-establish the C-partition by dragging and dropping files from the mounted image. With a heavy inhale of air, I shut down and rebooted into C. And it worked! How boring!

So, I rinsed and repeated, this time also deleted everything but the RECYCLER and the "System Volume Information" folder before I copied the missing files back from the mounted image. No problem at all.

So I guess I will have to try with the trial of TI 8.0 tomorrow. Even as a trial it should allow me to mount the original TI 6.0 image.

 

FOUND THE PROBLEM -- or at least one of the problems.
Can't write long right now, need to get some sleep, but wanted
to post this to prevent other people from wasting their time.

Don't bother to try to reproduce this problem unless you already
have (or are willing to have) Google Desktop installed.
Then delete the "Program Files/Google" folder and then restore
it from a TI image. Then try to reboot and login on the
affected partition. (Even better: before the restore, try to
scribble on the sectors occupied by the Google folder's files --
i.e., attempt to ensure that the restore will reside on
different sectors.)

btw, wrt to the successful (i.e. unproblematic) tests by
beenthereb4 and MiniMax -- so much for the claims of Acronis'
"experts" who claimed that msft *Windows* has *files* which are
sector-dependent.

 

Well I tried to satisfy my own curiosity but couldn't even get out of the starting blocks!! I'm just not able to boot from my external USB HD after restoring or cloning my C: drive to it, even though the BIOS was set to disable the SATA controller and boot from the USB-HDD. I don't particularly want to partition either of my internal drives and start phaffing around with a Boot Manager. So, for me, I guess the jury is still out.

MiniMax

Your doing a great job trying to prove/disprove the unmoveable files theory. However, I notice a slightly different approach between what A-C did and what you are currently doing. Namely, A-C formatted the empty partition before copying the files across from the mounted image whilst you didn't. Formatting obviously destroys any previous file structure which may or may not have an effect on the results you're obtaining.

Any chance of you humouring this old man by repeating some of your efforts, but this time format the C: partition prior to copying the mounted files across?

Regards
Tom

 

I will see what I can do. Problem is, A-C have his OS on partition #4 (that is, not C, the boot partition), so he can safely format P#4 without destroying boot.ini, ntldr, ntdetect.com and friends. If I wipe C, I will need to use the Win2K Recovery Console to rebuild those files before I can boot my temporary Win2K install in order to Explore the image. I will give it a go later.... Or.... I could scratch my D-partition, install Win2K there, image it, reformat D, and copy it back by file-by-file. Hmm.. Sounds like it should work, and it would be almost identical to A-C's setup....

 

Hi there MiniMax,

Fully understood. Hence the reason I was attempting to boot from my cloned external HD (failure to be resolved, hopefully, another day).

Kind regards
Tom

 

Fast fingers at work here....

I booted from an Windows XP install CD, deleted my D-partition (full format, NTFS), and installed WinXP. Logged in and out a couple of times.

Rebooted into the temporary Win2K on F, imaged the fresh WinXP on D on to local disc, and checked & explored the image.

Re-formated D, explored the image, and simpy dragged'n'dropped everything in the image onto the re-formatted D partition. Got a warning about the "System Volume Information" folder already being on the D partition. Say "Yes to All" for overwriting existing files.

Rebooted into the restored D partition - no problems.

 

I have installed the suspected "Google Desktop Search" on D now. Will create new image, and test again.

 

Thanks MiniMax. As I see it, that seems to have blown my understanding about unmoveable files out of the water (throwback to DOS/Win 9X days?). I stand corrected.

Kind regards
Tom

 

Save your fingers & time.
It's definitely GDS, and now I know why & how --
and boy, am I pissed off.
Going to bed now, will provide all the bloody details later
today.

 

Too late

Looking forward to that. As I said, I installed GDS and let it finish its indexing. Imaged the partition, reformatted, and dragged'n'dropped the files back. Rebooted and everything was fine.

 

Find a comfortable chair and a beverage: this takes a while.

[collisba/Brian, this may particularly interest you and anyone
else who supports an "enterprise" (i.e. corporate I.T.)
environment.]

THERE'S ALSO USEFUL INFO HERE FOR ACRONIS
PERSONNEL, even though the problem is not TI's fault.

The full post-mortem...

As MiniMax and I discussed, I "partitioned the problem space"
(A.I. lingo) by
(a) doing a full TI restore,
(b) progressively deleting (on the full-restore partition) and
then selectively restoring (from the mounted image) various
files & folders -- first within %SYSTEMROOT%, then within
%PROGRAMFILES%, and
(c) re-booting to the victim partition after each change, until
the problem appeared.
As reported, I narrowed it down to the folder containing GDS
(Google Desktop Search).

I'd already done a file-name/date/size comparison of an entirely
s-r'd partition against the mounted TI image, and found no
differences. This surprised me: ever since seeing that the
problem occurred only if the files (being s-r'd) didn't already
exist on the target at the moment of s-r, I'd suspected that
"something" was being dropped in the process of s-r (i.e., that
the omission in copying was masked if the "something" already
existed on the target). I say "something" rather than "file(s)"
because what was missed might be meta-data, such as Tonio's
suggestion regarding security attributes.

Then I did a binary-compare on all the files in the s-r'd GDS
folder, against the mounted TI image, and found no differences.

At this point, I suspected that the difference might be found
by looking at the MFT entries for the GDS folder & files.
I thought that perhaps encountering the problem depended on
something related to the layout/ordering of the files on the
disk or within the MFT. I even briefly wondered if Google had
used coding practices as archaic and invidious as hard-coding
physical metadata (e.g., sector numbers) within their files
as part of the installation process.

I proceeded to cut & paste (into a text file) hex-dumps of the
MFT entries for the 30+ GDS files (and each file's list of
allocated sectors) from the s-r'd partition -- all in one go
on that partition first, simply because it was easier and less
error-prone than alternating between the two partitions.
[Interesting factoid which I don't recall ever having known:
files which are "sufficiently" small are stored entirely within
the MFT entry, and occupy no other sectors.]

About a third of the way through dumping the MFT entries,
an hypothesis occurred to me, but I decided to defer pursuing it
until I'd finished dumping this first partition.

Have you guessed yet, what it was that I saw in the MFT entries,
which sparked the new hypothesis?

I opened a DOS box and did "dir /n/on/p/q/w/x" on the GDS folder
on the s-r'd partition, then on the mounted image.
For various files with long file names, the 8-dot-3 names had
changed during the copy process.

So, the next order of business was to determine where those
short names were being used in a manner which could cause the
original problem. I scanned all the sectors in the mounted
image for occurrences of "GO****~*.", *=wild-card. (Most of the
GDS short names start with GOOGLE~, but a few have a more
randomized form.) I was pretty sure what I would find, but I
wanted to confirm that none of the hits occurred within, e.g.,
.ini files.

There were a lot of false hits on residual bytes within sectors
which were unallocated or partially filled, but the only
credible hit was in a sector which was allocated to one of the
registry hives. I ran regedt32 and loaded all of the hives from
the mounted image, then ran regedit to scan those hives.

Sure enough, during the process of installing GDS, at least one
of the GDS short names (a dll) becomes registered with Win,
presumably specifying it as a dll which is essential to running
explorer.exe, which in turn will fail if the dll is absent.
(Even more likely, the cause of the failure is that the wrong
GDS dll is being used, since there are plural GDS dlls with
system-assigned short names which can be bollixed by copying.)
Now, I might be incorrect about the *exact* mechanism by which
GDS is involved in rendering the system unusable, but it's
undeniable that it's a result of the short-name problem --
which was completely unnecessary and avoidable by Google (see
below).

As confirmation, I again copied the GDS files from the mounted
image to the s-r partition, but this time using xxcopy with the
/nx parameter which preserves system-assigned short names --
and the problem disappeared.

No, I haven't forgotten that TonioRoffo suggested "using XXcopy
/clone... instead of windows explorer copy".
But, by that point, even if xxcopy had made the problem
disappear, I would have had insufficient confidence in the
long-term integrity of the restored partition, since I'd have no
idea of why the problem occurred.

Tying up some loose ends...
================
<google screed>
Google's sins...

1. GDS installs some files in its folder which are named
aa ### WARNING - Do not
ab ### move or delete these
ac ### files - your system
ad ### may stop working

This is atrocious and irresponsible human-factors design.

-- a. For one thing, at what point does Google expect users to
notice this warning? How many users do you think, every time
they install any new software, immediately inspect all of the
file-names which were installed?

-- b. Furthermore, the warning is ambiguous: perhaps the
reader will at least know that it's risky to move the folder to
another location. The most which Google can safely assume of
the reader, is that the reader will know that it's risky to
delete the files: the reader is even encouraged to this milder
and narrower interpretation by the remainder of the warning,
which says...
af ### To uninstall use
ag ### Add-Remove programs
ah ### in the control panel
ai ### or run
ak ### GoogleDesktopSearchSetup.exe -uninstall

-- c. Having read the warning, how many people do you think
will interpret the warning to mean that it's risky even to
*restore* the files (depending on the tools which are used)?

-- d. Among the people who do happen to see the warning, how
many will remember it months later while performing system
maintenance -- or while in the stressful and distracted
circumstances of peforming an emergency recovery?

2. This risk created by GDS was totally unnecessary and simply
avoided: all they had to do was to distribute the risky files
with Google-assigned 8.3 names, such as "gdsapi2.dll".
Apparently, some misguided (or simply arrogant?) developer (or
designer or product manager or marketing person) thought that
it's more important to use cosmetically attractive names such as
"GoogleDesktopAPI2.dll". As someone who has previously been a
developer for software vendors, I can confirm that even
decisions at such a picayune detailed level might be made by a
committee, frequently on the basis of standards layed down in a
document which dictates corporate standards for "branding".

3. I doubt that Google could even plead that this issue is a
matter of common practice among developers. On the system where
I experienced the problem, the Program Files folder contains
well over 200 products and packages, from a wide spectrum of
sources ranging from sole-developer share-ware, through open-
source, all the way to the behemoth world-famous vendors.
I scanned the registry for other occurrences of system-assigned
8.3 names, and found only five -- and none of those involved
functions whose failure could make a system unusable.


This next item might be of particular interest to
collisba/Brian, and anyone else who supports an "enterprise"
(i.e. corporate I.T.) environment.
Also, watch for another post about Google in the enterprise,
to follow shortly after this one.

4. As discussed earlier in this thread, perhaps the worst kind of
software deficiency is the kind which you discover a long time
(or never) after it has silently affected you -- especially in
circumstances which are "mission-critical" or related to loss of
business, or to loss of productivity, or to human safety.

So ask yourself:
"Is this really the level of quality which I want to introduce
into the environment which I'm responsible for supporting?"
How many of your supported users (or the people who support
them) will lose time to problems like these, perhaps needlessly
performing complete rollbacks of their desktop systems?

THIS QUESTION IS ESPECIALLY SALIENT,
NOW THAT GOOGLE HAS RELEASED
AN "ENTERPRISE" VERSION OF GDS.

</google screed>
================

Other loose ends...

-- I'm unsure why using explorer-copy produced different
short names than the GDS install, but I suspect that it depends
on the order in which the files are processed -- which might
well be different between install vs. copy.

-- The problem didn't occur when copying directly over
still-existing files. I'm guessing that the explanation is
that the relevant MSFT code explicitly opts to preserve the
existing short names in the about-to-be-replaced targets.
Note that the problem (or lack thereof) didn't change when I
used Directory Opus (instead of the win shell) to drag&drop the
files.

Finally, to Acronis...

-- Let's kiss & make up. I was disinclined to attribute the
propblem to your product, until I started seeing vague
hand-waving explanations of Win files with physical-location
dependencies.

-- I actually experienced a different problem related to
short names changing after using xcopy, years ago, and had
completely forgotten about it.
This kind of thing is easy to overlook. It would be NICE --
as in, "good corporate citizenship" -- if *any* and *every*
place you mention selective-restore (including web pages,
manuals, help files, dialogs, menus, etc.) would provide a
warning and reminder of this issue.

 

<google screed, continued>
More on the subject of Google in the enterprise environment...

5. To add insult to injury, GDS isn't even particularly
functional. AND I'VE SEEN NOTHING PUBLISHED BY
GOOGLE WHICH INDICATES THAT THE FOLLOWING
DEFICIENCIES ARE REMEDIED IN THE "ENTERPRISE" VERSION.

--a. GDS doesn't index the total content of its documents, only
the first <nnn> number of bytes.

--b. Furthermore, GDS doesn't even handle "OR" arguments,
which I was just recently told by Google, after months of
seeing puzzling scanty search results which I had been
attributing to faulty remembrance. In other words, if you
want to search for stored desktop items, or visited web pages,
which are related to overheating of your system, you can't
search for "cool OR cools OR cooled OR cooler OR cooling
OR coolant OR heat OR heated"...etc., but must instead perform
numerous separate searches (and then, of course, wasting your
time during each separate search by having to wade through all
of the redundant hits which you already encountered during the
previous single-word searches).

And if you're unaware of this limitation, all you know is that
you're getting few results for the search (or none), not knowing
that the reason is that GDS is treating your request as though
you were searching for items which contain ALL of the search
arguments.

Note what Google says at http://desktop.google.com/about.html :

-- "... provides FULL text search ... creates an index of ALL
your searchable information ... you can search the FULL TEXT of
your email, files, viewed web pages ..." (see item 5a. above).

-- "... search your personal items AS EASILY AS you search the
Internet using Google ... just do AN ORDINARY GOOGLE WEB
SEARCH ..." (see item 5b. above).

This amounts to deceptive practices.

I DON'T FAULT GOOGLE FOR THESE GDS DEFICIENCIES:
development priorities are a business decision. But I do fault
Google for publicizing and distributing GDS with no mention of
these issues, so that users experience these limitations
silently and insidiously. Despite Google's vaunted motto of
"Don't be evil", their behavior increasingly mimics that of
other vendors who engage in vapor-ware practices (such as
premature releases or announcements) in order to gain mind-share
and market-share over their competitors.

There are much better alternatives to GDS, including some free
ones. Some even allow more sophisticated, more precise,
time-saving compound Boolean arguments which are unavailable
even in Google's "real" search engine, e.g.
((cool AND cpu) OR (heat AND disk)) AND NOT ((cool AND shades)
OR (cool AND sounds))
Try the desktop-search offerings from Copernic, Yahoo, MSFT/MSN,
X1, DtSearch, etc.

6. Unrelated to the GDS issues, this seems like a good time
to mention -- especially to potential customers of expensive
Google "enterprise" products -- that even the "real" Google
search engine produces anomalous results. Try the following
at google.com, and then ask yourself, "Do I want to use this
same proprietary technology to run my business?".

--a. at the top of the google.com search page, click
"Preferences" and set *both* of "Interface Language" and
"Search only for pages written in these language(s)" to English.

--b. search for: dontdisplaylastusername

--c. Go directly to the bottom of the last page of results, and
click "repeat the search with the omitted results included".

--d. Go directly to the last page of results, and note at the
top where Google tells you the exact number of results it found.
At the moment I'm writing this, that number is 743.

--e. In the search box, after 'dontdisplaylastusername',
*add* the following to the search argument:
reason|reasons|bug|bugs|product|products|alert|alerts|alerted|alerting
In other words, you're asking Google to *narrow* your previous
results, to include *only* results which *also* contain one or
more of the additional words.
Now click Search.

--f. Repeat steps "c." & "d.".

With the modified (i.e., additionally restricted) search,
you would properly expect to get a count of results which is
equal or *less* than the previous number. But, at the moment
I'm writing this, that number is 764.

In other words, one or both of the searches produced
inconsistent results. Either the first search dropped some
matches, or the second search produced some bogus matches.

Now, this behavior might be acceptable (although incorrect)
when you're engaging in personal web searches, given the
fluidity of the entire set of web pages across the whole world.
But, if Google's purchasable enterprise products are using the
same core search-engine -- not an unreasonable conjecture --
then it's disturbing to think that there might be businesses
which are using that identical technology for important
data-mining applications.

</google screed>

p.s. -- I reported this bug to Google months ago. At the
beginning of April, they sent me a note which only said, "We're
aware of and investigating the problem." Almost two months
after their acknowledgement, the bug remains.
I'd hate to be one of the businesses which are paying Google for
placement of ads based on search results.

 

I had tee Earl Grey - hot (with milk, so I am a little bit out of character).

Confirmed!

This is sooooooooo utterly broken:

Code:

 Directory of D:\Programmer\Google\Google Desktop Search

27-05-2005  09:26       <DIR>                          .
27-05-2005  09:26       <DIR>                          ..
27-05-2005  01:19                  398 AA###W~1        aa ### WARNING - Do not
27-05-2005  01:19                  398 AB###M~1        ab ### move or delete these
27-05-2005  01:19                  398 AC###F~1        ac ### files - your system
27-05-2005  01:19                  398 AD###M~1        ad ### may stop working
27-05-2005  01:19                  398 AE###~1         ae ###
27-05-2005  01:19                  398 AF###T~1        af ### To uninstall use
27-05-2005  01:19                  398 AG###A~1        ag ### Add-Remove programs
27-05-2005  01:19                  398 AH###I~1        ah ### in the control panel
27-05-2005  01:19                  398 AI###O~1        ai ### or run
27-05-2005  01:19                  398 AJ###~1         aj ###
27-05-2005  01:19                  398 AK###G~1.EXE    ak ### GoogleDesktopSearchSetup.exe -uninstall
27-05-2005  01:19                  398 AL###~1         al ###
27-05-2005  09:26       <DIR>                          docs
27-05-2005  01:19              118.784 GOOGLE~1.EXE    GoogleDesktop.exe
27-05-2005  01:19              103.936 GOOGLE~1.DLL    GoogleDesktopAPI2.dll
27-05-2005  01:19              129.536 GOOGLE~2.EXE    GoogleDesktopCrawl.exe
27-05-2005  01:19               87.552 GOOGLE~2.DLL    GoogleDesktopDeskbar2.dll
27-05-2005  01:19               66.048 GOOGLE~3.EXE    GoogleDesktopDisplay.exe
27-05-2005  01:19               90.380 GOOGLE~1.DAT    GoogleDesktopEncdet.dat
27-05-2005  01:19               72.704 GOOGLE~3.DLL    GoogleDesktopIE.dll
27-05-2005  01:19              380.928 GOOGLE~4.EXE    GoogleDesktopIndex.exe
27-05-2005  01:19              109.568 GOOGLE~4.DLL    GoogleDesktopMozilla.dll
27-05-2005  01:19               20.935 GOOGLE~1.JS     GoogleDesktopMozillaStub.js
27-05-2005  01:19                3.119 GOOGLE~1.XPT    GoogleDesktopMozillaStub.xpt
27-05-2005  01:19                8.704 GOE462~1.DLL    GoogleDesktopNetwork1.dll
27-05-2005  01:19               93.184 GOE862~1.DLL    GoogleDesktopNetwork2.dll
27-05-2005  01:19               57.856 GO1FF9~1.EXE    GoogleDesktopOE.exe
27-05-2005  01:19              111.104 GOEC82~1.DLL    GoogleDesktopOffice.dll
27-05-2005  01:19              183.296 GO8D0E~1.DLL    GoogleDesktopResources_en.dll
27-05-2005  01:19              741.992 GOEB8E~1.EXE    GoogleDesktopSearchSetup.exe
27-05-2005  01:19               36.352                 gzlib.dll
27-05-2005  01:19              272.384 PDFTOH~1.EXE    pdftohtml.exe
27-05-2005  09:26       <DIR>                          temp
27-05-2005  01:19                  894 UNINST~1.ICO    uninstall.ico

It is GoogleDesktopIE.dll a.k.a. GOOGLE~3.DLL, a Browser Helper Object (BHO - an extension to Windows/Internet Explorer). Appearently, it is responsible for indexing the IE cache/Temporary Internet Files.

When I renamed GoogleDesktopIE.dll to GoogleDesktopIE.dll.ORIG, GOOGLE~3.DLL changed named too (GOOGLE~3.ORI, and from then on the Windows desktop refused to load. Renaming it back to its original name allowed the desktop to start again.

 

can someone help me interpret this note I just received from
Acronis?

"Please note that with the help of Acronis True Image 8.0 you
can restore only non system files with exlpore image option. The
algorithm Acronis True Image 8.0 uses to copy files from an
image is the same to Windows copy algotithm.

Acronis True Image 8.0 works properly if you restore the whole
image without any changes after it. You cannot later restore
system files, only the data.

Thank you."

wtf?
"algorithm Acronis uses to copy files from an image"?
Does TI now supply its own facility for copying *from* a mounted
image?

and does he really mean that "You *cannot* restore system files"
from an image?
If so, I'll *never* upgrade from v7!

 

Nonono,

What they mean is, using file based restore, you can't restore to a *online* filesystem (like any backup) - Because some files are constantly in use (restore.exe, registry, ...)

If you do a file based restore from another installation, or to another partition, then activate the other partition, you're all good.

Sector based restore from a boot disk will of course always work!