Good day
We have multiple Hyper-V (Server 2012) servers being backed-up by Data Protection Manager (2012 SP1). On a weekly basis we find Linux virtual machines that enters a saving/saved state and not being able to resume from there.
I know VSS will save any non-vssaware guest os to take the snapshot, but 40% of the time it gets stuck in this state and the only resolution is to reboot the Hyper-V server, which isn't a solution really.
During the saved-state, I noticed the backup that's 'in progress' on DPM - isn't transferring any data. I'm also unable to cancel the backup. I then need to stop DPM Service.
Once the DPM service has been stopped, I can at least kill the vmms.exe process and get the virtual machine into a 'stop' state - but I still can't start it since it only goes up to 10% and stays there...
I cannot find any logs accept the daily Hyper-V logs, the same on DPM. I have managed to find only two entries this week during a saved-state issue:
Log Name: Application<u5:p></u5:p>
Source: VSS<u5:p></u5:p>
Date: 2013-11-19 12:06:21 AM<u5:p></u5:p>
Event ID: 8229<u5:p></u5:p>
Task Category: None<u5:p></u5:p>
Level: Warning<u5:p></u5:p>
Keywords: Classic<u5:p></u5:p>
User: N/A<u5:p></u5:p>
Computer:
HOSTNAME.DOMAIN<u5:p></u5:p>
Description:<u5:p></u5:p>
A VSS writer has rejected an event with error 0x80042319, A writer did not respond to a GatherWriterStatus call. The writer may either have terminated<u5:p></u5:p>
or it may be stuck. Check the System and Application event logs for more information.<u5:p></u5:p>
Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer. <u5:p></u5:p>
<u5:p> </u5:p>
Operation:<u5:p></u5:p>
PrepareForSnapshot Event<u5:p></u5:p>
<u5:p> </u5:p>
Context:<u5:p></u5:p>
Execution Context: Writer<u5:p></u5:p>
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}<u5:p></u5:p>
Writer Name: Microsoft Hyper-V VSS Writer<u5:p></u5:p>
Writer Instance ID: {179aff09-4a58-4f39-a0ed-bc1839716251}<u5:p></u5:p>
Command Line: C:\Windows\system32\vmms.exe<u5:p></u5:p>
Process ID: 1996<u5:p></u5:p>
Event Xml:<u5:p></u5:p>
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"><u5:p></u5:p>
<System><u5:p></u5:p>
<Provider Name="VSS" /><u5:p></u5:p>
<EventID Qualifiers="0">8229</EventID><u5:p></u5:p>
<Level>3</Level><u5:p></u5:p>
<Task>0</Task><u5:p></u5:p>
<Keywords>0x80000000000000</Keywords><u5:p></u5:p>
<TimeCreated SystemTime="2013-11-18T22:06:21.000000000Z" /><u5:p></u5:p>
<EventRecordID>24995</EventRecordID><u5:p></u5:p>
<Channel>Application</Channel><u5:p></u5:p>
<Computer>HOSTNAME.DOMAIN</Computer><u5:p></u5:p>
<Security /><u5:p></u5:p>
</System><u5:p></u5:p>
<EventData><u5:p></u5:p>
<Data>0x80042319, A writer did not respond to a GatherWriterStatus call. The writer may either have terminated<u5:p></u5:p>
or it may be stuck. Check the System and Application event logs for more information.<u5:p></u5:p>
</Data><u5:p></u5:p>
<Data><u5:p></u5:p>
<u5:p> </u5:p>
Operation:<u5:p></u5:p>
PrepareForSnapshot Event<u5:p></u5:p>
<u5:p> </u5:p>
Context:<u5:p></u5:p>
Execution Context: Writer<u5:p></u5:p>
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}<u5:p></u5:p>
Writer Name: Microsoft Hyper-V VSS Writer<u5:p></u5:p>
Writer Instance ID: {179aff09-4a58-4f39-a0ed-bc1839716251}<u5:p></u5:p>
Command Line: C:\Windows\system32\vmms.exe<u5:p></u5:p>
Process ID: 1996</Data><u5:p></u5:p>
<Binary>2D20436F64653A20575254575254494330303030353332372D2043616C6C3A20575254575254494330303030323932392D205049443A202030303030313939362D205449443A202030303031313432302D20434D443A2020433A5C57696E646F77735C73797374656D33325C766D6D732E657865202020202D20557365723A204E616D653A204E5420415554484F524954595C53595354454D2C205349443A532D312D352D313820</Binary><u5:p></u5:p>
</EventData><u5:p></u5:p>
</Event>
----------------------------------------------------------------
Second log
----------------------------------------------------------------
Log Name: Application<u5:p></u5:p>
Source: VSS<u5:p></u5:p>
Date: 2013-11-19 12:06:21 AM<u5:p></u5:p>
Event ID: 8229<u5:p></u5:p>
Task Category: None<u5:p></u5:p>
Level: Warning<u5:p></u5:p>
Keywords: Classic<u5:p></u5:p>
User: N/A<u5:p></u5:p>
Computer:
HOSTNAME.DOMAIN<u5:p></u5:p>
Description:<u5:p></u5:p>
A VSS writer has rejected an event with error 0x80042319, A writer did not respond to a GatherWriterStatus call. The writer may either have terminated<u5:p></u5:p>
or it may be stuck. Check the System and Application event logs for more information.<u5:p></u5:p>
. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer. <u5:p></u5:p>
<u5:p> </u5:p>
Operation:<u5:p></u5:p>
PrepareForSnapshot Event<u5:p></u5:p>
<u5:p> </u5:p>
Context:<u5:p></u5:p>
Execution Context: Writer<u5:p></u5:p>
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}<u5:p></u5:p>
Writer Name: Microsoft Hyper-V VSS Writer<u5:p></u5:p>
Writer Instance ID: {179aff09-4a58-4f39-a0ed-bc1839716251}<u5:p></u5:p>
Command Line: C:\Windows\system32\vmms.exe<u5:p></u5:p>
Process ID: 1996<u5:p></u5:p>
Event Xml:<u5:p></u5:p>
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event"><u5:p></u5:p>
<System><u5:p></u5:p>
<Provider Name="VSS" /><u5:p></u5:p>
<EventID Qualifiers="0">8229</EventID><u5:p></u5:p>
<Level>3</Level><u5:p></u5:p>
<Task>0</Task><u5:p></u5:p>
<Keywords>0x80000000000000</Keywords><u5:p></u5:p>
<TimeCreated SystemTime="2013-11-18T22:06:21.000000000Z" /><u5:p></u5:p>
<EventRecordID>24995</EventRecordID><u5:p></u5:p>
<Channel>Application</Channel><u5:p></u5:p>
<Computer>HOSTNAME.DOMAIN</Computer><u5:p></u5:p>
<Security /><u5:p></u5:p>
</System><u5:p></u5:p>
<EventData><u5:p></u5:p>
<Data>0x80042319, A writer did not respond to a GatherWriterStatus call. The writer may either have terminated<u5:p></u5:p>
or it may be stuck. Check the System and Application event logs for more information.<u5:p></u5:p>
</Data><u5:p></u5:p>
<Data><u5:p></u5:p>
<u5:p> </u5:p>
Operation:<u5:p></u5:p>
PrepareForSnapshot Event<u5:p></u5:p>
<u5:p> </u5:p>
Context:<u5:p></u5:p>
Execution Context: Writer<u5:p></u5:p>
Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}<u5:p></u5:p>
Writer Name: Microsoft Hyper-V VSS Writer<u5:p></u5:p>
Writer Instance ID: {179aff09-4a58-4f39-a0ed-bc1839716251}<u5:p></u5:p>
Command Line: C:\Windows\system32\vmms.exe<u5:p></u5:p>
Process ID: 1996</Data><u5:p></u5:p>
<Binary>2D20436F64653A20575254575254494330303030353332372D2043616C6C3A20575254575254494330303030323932392D205049443A202030303030313939362D205449443A202030303031313432302D20434D443A2020433A5C57696E646F77735C73797374656D33325C766D6D732E657865202020202D20557365723A204E616D653A204E5420415554484F524954595C53595354454D2C205349443A532D312D352D313820</Binary><u5:p></u5:p>
</EventData><u5:p></u5:p>
</Event><u5:p></u5:p>
<u5:p></u5:p>
I may be looking at the wrong application, but I think it's related VSS. Whether it's the requester,provider or writer - i'm not sure.
Any ideas would be appreciated.
Thank you.