Windows Server 2003 stops responding with VSS Enalbled

I have one machine with Windows Server 2003 Enterprise SP2 on which i have installed Acronis True Image Echo Server (build 8.353).

I have enabled all related Volume Shadow Copy services and they are set to automatically start.

Randomly, but very frequently, server stops responding. No input from mouse or keyboard. It simply freezes.

The most significant log that appears in event viewer everytime this happens is an ESENT Error - ID 2004 - "lsass -488- Shadow Copy 14 time-out (70000 ms)". After that event, server stops responding and i have to hard-reset it.

I have applied all patches available from Microsoft and, as you can see, installed latest versione available from Acronis, but error still persists.

Any suggestion? Please HELP!!!

Thank you

Rob

 

Hello Rvecchi,

Thank you for using Acronis True Image Echo Server for Windows

Do you receive BSOD when server freezes? If so please create a screen shot and send it to us. Also please attach crash dumps to your next reply. Please do the following in order to enable mini-dumps:

- Right click on "My Computer" and choose "Properties";
- Go to "Advanced" tab;
- Go to "Startup" and "Recover Settings";
- Choose "Small memory dump" in "Write debugging information" box;
- Close all the dialog windows by clicking OK buttons.

Now, please reproduce the system crash and send us the mini-dumps created. This will allow us to investigate the problem thoroughly.

I would also recommend you update SnapAPI drivers, use the following KB article

Best regards,
--
Dmitry Nikolaev

 

Hello,

thank you for your reply. No, the server has never stopped with a BSOD.

It simply stops responding. I cannot click on any icon or issue any command. I have to hard reset the server, this is the only way i can put it back on a production state.

No problem with the same product on other servers with Windows Server 2003 Standard SP2. This only occurs on two server with Windows Server 2003 R2 Enterprise Edition.

At the moment, i have disabled VSS because this is a productive server and i want to make sure my company has business continuity.

I will try to install updated SnapAPI drivers as you suggested.

I will let you know what is gonna happen after that installation.

Thanks again for your reply.

Best Regards

R. Vecchi

 

I followed all the steps indicated in http://kb.acronis.com/content/1514.
I also upgraded the SnapApi Drivers. During registry key/strings deletion i have noticed that only "snapapi380" key/string was present. No "snapapi" key or string. I don't know if this can be related to the problem i have.
Anyway, after having completed all the steps listed in the KB, i can't find the "Enable VSS" Option anymore in Database support. The only items i can see now are "Pre-post Commands" and "Use Multi-snapshot".

Is that correct?

Tkank you

R. Vecchi

 

Hello Rob,

Thank you for using Acronis Corporate Products

VSS may lock the partition after a backup by changing the partition's attribute to read-only.

You can use batch files to stop the database for a second before the snapshot is taken and then immediately start it, while Acronis True Image proceeds with creating the backup. Please see the article Using Batch Files in Acronis True Image for more information.

Also you can unlock the partition using diskpart:

* Click Start -> Run;
* Issue diskpart;
* In the command-line issue list disk;
* select disk [number of the disk that has the locked partition];
* list partition
* select partition [number of the locked partition]
* att vol clear readonly
* att vol clear nodefaultdriveletter
* att vol clear hidden
* att vol clear shadowcopy

Check the partition attributes:* detail volume

It should display the following attributes:

* Readonly: No
* Hidden: No

Thank you.

--
Oleg Lee

 

Hello Oleg,

thank you for your reply.

What happens is that backup does not occur. So there is no problem with partition locked. When i see in the event viewer the error i reported on my first message, the machine freezes. Nothing but hard reset can be done and backup image is not created.

I see this happening ONLY with VSS enabled, that's the reason why i suspect it's a problem that occurs between Acronis and Microsoft VSS. In fact, if i disable VSS Backup (as i am doing right now) everything goes fine.

So the problem isn't after the backup, but even before the backup itself starts.

Thank you again for your reply.

Rob

 

This is exactly what happens (copied from a post in Acronis Forum https://www.wilderssecurity.com/showthread.php?t=212801)

Acronis VSS no clean state of VSS Writers after backup!

You can test it yourself.

1. Restart you server
2. after restart use this command

vssadmin list writers
the result should be something like this

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001 Microsoft Corp.

Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {c27d5895-9cf7-454a-b64c-f62cd1cb38d2}
State: [1] Stable
Last error: No error

Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {c55c0529-abc1-484c-bf0a-334595474f00}
State: [1] Stable
Last error: No error

Writer name: 'MSDEWriter'
Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
Writer Instance Id: {df560ba0-6377-4b35-96f7-bb2f9004502e}
State: [1] Stable
Last error: No error

Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {77d4f4a9-1313-4bf0-88a1-c2d84627779f}
State: [1] Stable
Last error: No error

Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {da2df630-ac75-4573-b6a2-e5fddefdff8b}
State: [1] Stable
Last error: No error

Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {64564f4e-b902-406b-ad48-9c6bd09ff814}
State: [1] Stable
Last error: No error

Writer name: 'Event Log Writer'
Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
Writer Instance Id: {2e13e156-e801-42cd-b644-554bdfc699ec}
State: [1] Stable
Last error: No error

Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {f01a1584-681f-44cd-98d5-413b6cb0fc51}
State: [1] Stable
Last error: No error

also with this command vssadmin list shadows you should get this:

vssadmin 1.1 - Volume Shadow Copy Service administrative command-line tool
(C) Copyright 2001 Microsoft Corp.

No items found that satisfy the query.

Now when acronis Runs with VSS enabled you will see that the states of alot of the writers get a failed status like this:

Writer name: 'System Writer'
Writer Id: {e8132975-6f93-4464-a53e-1050253ae220}
Writer Instance Id: {c27d5895-9cf7-454a-b64c-f62cd1cb38d2}
State: [10] Failed
Last error: No error

Writer name: 'MSDEWriter'
Writer Id: {f8544ac1-0611-4fa5-b04b-f7ee00b03277}
Writer Instance Id: {a6e89fb4-38d1-4ab0-8ed5-046239c4922e}
State: [1] Stable
Last error: No error

Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {c55c0529-abc1-484c-bf0a-334595474f00}
State: [10] Failed
Last error: No error

Writer name: 'BITS Writer'
Writer Id: {4969d978-be47-48b0-b100-f328f07ac1e0}
Writer Instance Id: {53647954-e76f-4210-ac97-045fe1f46c69}
State: [1] Stable
Last error: No error

Writer name: 'COM+ REGDB Writer'
Writer Id: {542da469-d3e1-473c-9f4f-7847f01fc64f}
Writer Instance Id: {5eac209d-3e6a-4131-b416-3580e827d513}
State: [10] Failed
Last error: No error

Writer name: 'WMI Writer'
Writer Id: {a6ad56c2-b509-4e6c-bb19-49d8f43532f0}
Writer Instance Id: {64564f4e-b902-406b-ad48-9c6bd09ff814}
State: [10] Failed
Last error: No error

Writer name: 'IIS Metabase Writer'
Writer Id: {59b1f0cf-90ef-465f-9609-6ca8b2938366}
Writer Instance Id: {dc994f77-43a8-425c-a411-61a47ac2c494}
State: [10] Failed
Last error: No error

Writer name: 'Event Log Writer'
Writer Id: {eee8c692-67ed-4250-8d86-390603070d00}
Writer Instance Id: {b5821155-02f8-4de4-8e21-2d1bdba344c4}
State: [10] Failed
Last error: No error

Writer name: 'Registry Writer'
Writer Id: {afbab4a2-367d-4d15-a586-71dbb18f8485}
Writer Instance Id: {805734d3-a231-485e-9312-386d8ea628a4}
State: [10] Failed
Last error: No error

ACRONIS this must be fixed!
9 out of 10 times this will go partly good. But 1 out of 10 times the server will just hang at Locking PARTITION part!


The only differences are that this user were using Acronis True Image Server 9.1 while i am using Acronis True Image Echo Server build 8.353 and luckily this was happening to him 1 out of 10 times while i experience this 7 out of 10 times!!
Some Writer Instance ID may be different, but this is the point and these are the effects. System hangs at locking partition phase BEFORE BACKUP and ESENT event ID 2004 - Shadow Copy Timeout - is written in Event Viewer.
I cannot unlock partition as you sugested (i am experiencing the issue BEFORE the backup) simply because i cannot interact with machine. No mouse, no keyboard, non BSOD, no nothing! System becomes unrensponsive to keyboard and/or mouse inputs. The only thing i can do is a Hard Reset to give network users server services.
And until now, no suggested operation have solved the problem.

 

Hello Rob,

Thank you for your response.

There no a solution at this moment. Here is the description of the technology:

Acronis True Image initiates VSS snapshot. This snapshot is created by MS VSS service via Acronis VSS provider, which is in fact a 'fake' VSS provider. Acronis True Image must start VSS snapshot creation through Acronis VSS provider to capture the moment when all VSS writers freeze and databases are ready for backup. This is the only purpose of Acronis VSS provider, e.g. VSS snapshot created in this step is not used at all and is in fact a 'fake' one. It was necessary to create this 'fake' VSS provider because MS VSS service does not have API functions to freeze VSS writers directly.

Acronis True Image sends a request to MS VSS service to suspend VSS writers. Acronis True Image creates a simple snapshot (not a VSS snapshot). This snapshot is created when all writers freeze and databases are ready to be backed up.

Acronis program aborts VSS snapshot initiated by MS VSS, and when the server stops responding, it appears to be a reaction on the aborted snapshot.

By the way, we reproduced the same issue with NTBackup and aborted VSS snapshot.

Thank you.

--
Oleg Lee