Hyper-V 2016 VMs stuck 'Creating checkpoint 9%' while starting backups

March 28, 2017, 1:40 pm

≫ Next: VM Very Slow After Migrating from Hyper-v 2012 to Hyper-v 2019

≪ Previous: Cluster node - Event 252 - cluster service crashed

We have a two clustered W2016 Hyper-V hosts, every couple of days one of the hosts gets stuck when the backup kicks off. In Hyper-V manager the VMs all say 'Creating checkpoint 9%' It's always the same percentage 9%. You can't cancel the operation and the VMMS service refuses to stop, the only way to get out of the mess is to shutdown the VMs, and hard reset the effected node. The backup works for a few days then it all starts again.

The only events I can see on the effected node is:

Event ID: 19060 source: Hyper-V-VMMS

'VMName' failed to perform the 'Creating Checkpoint' operation. The virtual machine is currently performing the following operation: 'Creating Checkpoint'.

Can anybody help please? Cluster validation is clean. Hosts and guests are patched up.

↧

VM Very Slow After Migrating from Hyper-v 2012 to Hyper-v 2019

November 25, 2019, 5:03 pm

≫ Next: Multiple VlanIDs on one Networkcard with Powershell for Hyper-V VMs

≪ Previous: Hyper-V 2016 VMs stuck 'Creating checkpoint 9%' while starting backups

My hyper-v 2012 environment has been migrated to hyper-v 2019, all VM`s have been migrated and are running correctly, except for a Windows Server 2008 Stardard VM, which runs an application and a SQL server 2008. After some hours of the migrated machine, the application begins to be very legendary.
I migrated using Export / Import, after migrating the VM to Hyper-v 2019, applied all available updates.
VM Specifications:
OS: Windows Server 2008 Standard SP2
Memory: 16GB
vProcessors: 12
1 NIC

I already reinstalled integraion services, but nothing worked.

↧

Multiple VlanIDs on one Networkcard with Powershell for Hyper-V VMs

November 25, 2019, 11:51 pm

≫ Next: Hyper-V 2016 failed to perform the 'Cleaning up stale reference point(s)' operation.

≪ Previous: VM Very Slow After Migrating from Hyper-v 2012 to Hyper-v 2019

Hi Guys,

i need help with with an problem. We want to use multiple vlanids on one vm networkcard. Yes i know, that we can add more networkcards at the Hyper-V Manager, but we got to the limit of 8 virtuell networkcards and we can´t add more. So we want to define more vlanids to one networkcard, to avoid the limit of the networkcards.

We used the Powershell command

Get-VMNetworkAdapter -VMName Redmond | Set-VMNetworkAdapterVlan -VMName Redmond -Trunk -AllowedVlanIdList 1-100 -NativeVlanId 10

of course with different values.

It doesn´t work...

We see that the networkcard ist changed to the mode trunk and the right VlanList, but it seems that no connection is established. The hardwareswitch is configurated to use these vlanid´s.

Did we do something wrong? Is there a KB where it explaned, how to use this command right?

If we use the command set-vmnetworkadaptervlan, all networkcards with the same name will changed, even if we use an pipe before, to specify a specific networkcard.

oh-vm Network Adapter Trunk 899,180,925
oh-vm Network Adapter Trunk 899,180,925

I hope you can help us,

Thanks

Dennis

↧

Hyper-V 2016 failed to perform the 'Cleaning up stale reference point(s)' operation.

November 18, 2019, 12:13 am

≫ Next: Host Windows Server 2016 standard OEM, 2 VM's with 2016 standard OEM - cannot activate? 0xC004FD03

≪ Previous: Multiple VlanIDs on one Networkcard with Powershell for Hyper-V VMs

Hello dear Hyper-V elites. I'm in need of help. I've googled without success, I can't find anything matching this issue.

We have a customer with 2x Hyper-V 2016 machines. HOST-A and HOST-B. In this case the issue seems to originate from HOST-B. 5 of the servers that is currently running on HOST-B and is set to replicate to HOST-A is giving us headaches.

The 5 servers are all running Linux (Ubuntu), and we have no issues with the other VM's that are running Windows. What we receive are alerts with the following description:

EventID: 19060

'Linux-Server-1' failed to perform the 'Cleaning up stale reference point(s)' operation. The virtual machine is currently performing the following operation: 'Backing up'. (Virtual machine ID XXXXXXX-XXXXXXX-XXXXXXX-XXXXXX1)

Directly after the following event-log entries comes up;

EventID: 19070 and after that 19080

'Linux-Server-1' background disk merge has been started. (Virtual machine ID XXXXXXX-XXXXXXX-XXXXXXX-XXXXXX1)

'Linux-Server-1' background disk merge has been finished successfully. (Virtual machine ID XXXXXXX-XXXXXXX-XXXXXXX-XXXXXX1)

However we never get the code 29174 which we receive from all other "working" windows VM's.

Example below;

Replica statistics for 'Windows-server-1' (Virtual machine ID XXXXXXX-XXXXXXX-XXXXXXX-XXXXXX2)
StartTime: 132184512000256627
EndTime: 132184943997080231
ReplicationSuccessCount: 144
ReplicationSize (bytes): 1194310144
ReplicationLatency (seconds): 3
ReplicationMissCount: 0
ReplicationFailureCount: 0
NetworkFailureCount: 0
MaxReplicationLatency: 1
Application-consistentCheckpointErrorCount: 0
PendingReplicationSize (bytes): 0
MaxReplicationSize (bytes): 79151616

They all run on the same backup solution (solarwinds msp backup & replication) and at the same schedule. We receive no errors on the backup console.

All servers are also "hosted/located" on the same disk. Since I've seen other topics mentioning that it could be a disk issue, we think this is not the case.

Help me obi-wan kenobi, you're my only hope.

//Regards, Andreas

↧

Host Windows Server 2016 standard OEM, 2 VM's with 2016 standard OEM - cannot activate? 0xC004FD03

August 24, 2019, 5:08 am

≫ Next: Trying to use GTX Nvidia card on a Linux VM through Hyper-V

≪ Previous: Hyper-V 2016 failed to perform the 'Cleaning up stale reference point(s)' operation.

Can someone please explain the correct activation and rules for the VM's. With the standard edition you can not use AVMA, supposedly you use the same OEM key you activated on the host. When I try to use the key I got the 0xC004FD03 error. I used telephone activation for the 1st VM. When I activated the Host I answered 1 computer, apparently you answer 0 for the first activation, 1 for the second (VM1) and 2 for the third (VM3). When I use 2 I get the error. BTW I did try the AVMA keys first before I tried the OEM key. My OEM is useless and keeps telling me to use AVMA but I do not have the datacenter version of windows server 2016 to use as the virtulization server. It seems that this problem is very common and Microsoft should clarify how to fix it! I have seen some vage info that the OEM should supply keys or authorize guests but I have not been able to get this done.

Thanks in advance for any help.

↧

Trying to use GTX Nvidia card on a Linux VM through Hyper-V

November 26, 2019, 6:38 am

≫ Next: Server 2019 - Error 0xc0000225 When Adding vTPM

≪ Previous: Host Windows Server 2016 standard OEM, 2 VM's with 2016 standard OEM - cannot activate? 0xC004FD03

Trying to use GTX Nvidia card on a Linux VM through Hyper-V and am aware of the vGPU support for Pascal GPUs (consumer GPUs) is not there. There must be a way to do this like the option of using through (a Linux host machine). I would prefer to use Hyper-V instead of switching back to linux host. Thanks.

↧

Server 2019 - Error 0xc0000225 When Adding vTPM

November 26, 2019, 7:56 pm

≫ Next: Adding more than 2 serial ports

≪ Previous: Trying to use GTX Nvidia card on a Linux VM through Hyper-V

Hello,

I'm testing Server 2019 out as a migration path but I'm having an issue with Hyper-V on where it fails to start any VM that has a vTPM enabled. I tried to enable a vTPM on a test VM, and initially it failed with the error:

TimeCreated  : 11/26/2019 8:12:15 PM
ProviderName : Microsoft-Windows-Hyper-V-VMMS
Id           : 15500
Message      : 'vtpm_test' failed to start worker process: The object was not found. (0xC0000225). (Virtual machine ID CFBC4A73-...)

Since this was a test hypervisor, rather than troubleshoot the error I wiped it clean, and re-installed Server 2019 DataCenter from scratch, re-installed the Hyper-V role, and tried to provision another fresh VM with vTPM. However I received an error that the key protector couldn't be unwrapped:

TimeCreated  : 11/26/2019 8:45:01 PM
ProviderName : Microsoft-Windows-HostGuardianService-Client
Id           : 2014
Message      : The Host Guardian Service Client failed to unwrap a Key Protector on behalf of a calling process. This event will normally correspond to a failure to startup a shielded virtual machine. Consult the description for further details. This could be related to an attestation issue, a Key Protection Server issue, or a network connectivity issue:

               System.IO.FileNotFoundException: Could not find file 'C:\Windows\system32\config\VSMIDK'.
               File name: 'C:\Windows\system32\config\VSMIDK'
                  at Microsoft.Windows.KdsClient.HgsClient.UnwrapProtectionDescriptor(Byte[] ingressProtectionDescriptor, Byte[]& encryptedTransferKey, Byte[]& encryptedWrappingKey, Byte[]& encryptedKeys)
                  at Microsoft.Windows.KdsClient.Interop.ManagedEntry.UnwrapKeyProtector(IntPtr keyProtectorPointer, IntPtr unwrappedKpPointer, IntPtr errorContextPointer)

Doing some digging around, this file is supposed to be auto-created but it wasn't, so I just went ahead and copied it over from one of my Server 2016 installs. After that, I still receive the same error when trying to add a TPM to a VM and start it:

TimeCreated  : 11/26/2019 9:13:15 PM
ProviderName : Microsoft-Windows-Hyper-V-VMMS
Id           : 15500
Message      : 'vtpm_test' failed to start worker process: The object was not found. (0xC0000225). (Virtual machine ID ED380AFD-...)

It's worth mentioning that VMs start without a vTPM and I've also tried installing the HostGuardian feature to see if that helps. Also, I'm not experiencing this issue on a fresh Windows Server 2016 install nor on the established Windows Server 2016 boxes I have. Any thoughts as to what may be causing this issue?

↧

Adding more than 2 serial ports

November 26, 2019, 8:08 pm

≫ Next: Server 2012 R2 Hyper-V with many .avhdx files

≪ Previous: Server 2019 - Error 0xc0000225 When Adding vTPM

The hurdle I have run into is COM ports. From within a vm I need software to control external devices that rely on Serial communication to control hardware so that very specific and precise tasks take place. At the moment there are only 2 com ports showing in the virtual machine and I need 6. I have not been successful in finding a way to add more. Is this possible? The com ports will appear through usb ports. Barney

↧

Server 2012 R2 Hyper-V with many .avhdx files

November 27, 2019, 2:15 am

≫ Next: How run Stop-VM without confirmation and without -Force

≪ Previous: Adding more than 2 serial ports

Hi everyone,

I have 2012 R2 Hyper V server with 3 VMs running. Everything works great, but for one machine which is Endian Firewall in Virtual hard disks folder I have multiple .avhdx files. These .avhdx filese are created everyday in the time when Windows backup starts and are not deleted. In Hyper-V manager I don't have Checkpoints for that machine, and I'm slowly running out of space on this server. This is production server and I have 100 (for 100 days) .avhdx files for that machine. That particular machine was Shutdowned in meantime but obviously files were not merged/deleted.

How can I delete that avhdx files?

Thank you

greets
Dario

↧

How run Stop-VM without confirmation and without -Force

November 27, 2019, 4:36 am

≫ Next: Hyper-v unable to create checkpoint

≪ Previous: Server 2012 R2 Hyper-V with many .avhdx files

When I run

Stop-VM -Name "VMNAME" -confirm:$false

I receive prompt for confirmation:

    Confirm
    Hyper-V cannot shut down virtual machine VMNAME because the Shutdown integration service is 
    unavailable. To avoid potential data loss, you can pause or save the state of the virtual machine. 
    The
    other option is to turn off the virtual machine, but data loss might occur.
    [Y] Yes  [N] No  [S] Suspend  [?] Help (default is "Y"): N

But when I run the script (for example, cron), I do not want any interactive and pause on this team.
I would want that the command returned an error or does not return anything (and later deal with the result of the command), but does not stop the execution of waiting for a input in the console.
And I want an answer "No" to such a request (-Force I understand it gives the answer is always Yes).

↧

Hyper-v unable to create checkpoint

November 27, 2019, 6:44 am

≫ Next: Hyper-V. Cannot share the hard drives with the default (quick create) ubuntu image in enhanced mode.

≪ Previous: How run Stop-VM without confirmation and without -Force

Issue : Unable to create checkpoints on three vm's in Hyper-V manager and backups are failing with error: Failed to create VM recovery snapshot.

OS Name: Microsoft Windows Server 2019 Standard

OS Version: 10.0.17763 N/A Build 17763

Backup solution : Veeam Backup

Antivirus: Symantec

VSS related services are working fine

Tried creating checkpoint on one of the VM,however getting error "the operation cant be performed while the object is in use"

checked the event viewer and could see event ID's 3280,3350,18012,18014

checked for any shadow copies through the vssadmin list shadows command however could not see shadow copy configured

configured and enabled shadow copy on the vm

The backup jobs for all three vm's are getting stored on the D drive (Backup Storage) of backup server

Restarted the vm but even after restart ,unable to create checkpoint from hyper-v manager

Could you please check if this is a bug or any known issue and help me

swathi

↧

Hyper-V. Cannot share the hard drives with the default (quick create) ubuntu image in enhanced mode.

November 27, 2019, 9:19 am

≫ Next: VHD and VHDX won't Compact

≪ Previous: Hyper-v unable to create checkpoint

Hello,

I am struggling to share the disks between my windows 10 and Ubuntu 18.04.3 LTS VM.

I did everything like described in the official article (cannot share link but the title is "

Use local resources on Hyper-V virtual machine with VMConnect")

When I am running:

VMConnect.exe <ServerName> <VMName> /edit

I see that all the drives are checked in the VM remote desktop settings.

but ubuntu: mounted 'shared-drives' disk is completely empty!

Remote Desktop Services are running!

Could you please, point out what else can I check?

Thanks

↧

VHD and VHDX won't Compact

July 12, 2012, 11:29 pm

≫ Next: Event ID 12240 - Cannot open VM attachment - Veeam Backup

≪ Previous: Hyper-V. Cannot share the hard drives with the default (quick create) ubuntu image in enhanced mode.

I also have the same problem.

If I compact a VHD or VHDX the compact runs for one second and ends, and the VM's remain uncompacted.

I am using REFS format on the drive, and the VM's are Windows 7 x86, if that makes any difference.

I have disabled System Restore in VM's, as that was a known problem with 2008 R2.

Can this be reported as a bug in Server 2012 RC.

↧

Event ID 12240 - Cannot open VM attachment - Veeam Backup

November 27, 2019, 3:43 pm

≫ Next: PXE boot a vm 1gen with legacy network adaptor win1903 - not working?

≪ Previous: VHD and VHDX won't Compact

Hi all, I apologies for the lengthy post, and any irrelevant information. I'm trying to provide as much information as I can.

I have been asked to log this ticket by Veeam, as they determine it is a Microsoft issue.

We use Veeam Backup and Replication to backup all of our Hyper-V guests.

We also use VeeamONE to report on issues.

Our environment is:

- a standalone physical server running our Veeam software

- two clustered servers with Fibre Channel storage (Cluster Shared Volumes) hosting our VMs (approx. 20 guests VMs)

For the most part, this all runs fine. Occasionally, however, VeeamONE reports the following error:

Description
Fired by event: 12240 Microsoft-Windows-Hyper-V-VMMS
Event description: '': Attachment 'C:\ClusterStorage\ServerName\Virtual Hard Disks\diskname_E40FF910-B414-4DFA-BDC3-BE02ACDCF27A.avhdx' not found. Error: 'The system cannot find the file specified.' (0x80070002). (Virtual machine ID )
Initiated by: DOMAIN\BackupAdmin
Knowledge
Cannot open VM attachment

It happens to random servers, on random dates, so no pattern to the issue.

The AVHDX files that is "not found" is the snapshot file created by Veeam during the Backup process.

Veeam uses an API to take the snapshot, then another one to remove it when the backup is complete.

So, in effect, the error is correct, as the file no longer exists, but why is it being logged, if the API is removing the snapshot correctly, should Windows be able to detect this and not log the error?

My main concern is that this then logs an error with VeeamONE, and makes our reports and KPIs look bad, with all these false reports.

I know that the easy solution is to turn off that Alert, but I would like to catch legitimate issues, not the false positives.

If relevant:

- All servers fully patched and restarted

- Integration services are up to date on most servers (issue happens regardless of version)

Again, I apologies for lengthy post. In summary, my question is: Why does Windows/Hyper-V log that error for a temporary Snapshot disk? Is it a bug in the API call? Is Veeam initiating the API call incorrectly?

↧

PXE boot a vm 1gen with legacy network adaptor win1903 - not working?

November 27, 2019, 11:42 pm

≫ Next: Microsoft Hyper-V 2016 Server

≪ Previous: Event ID 12240 - Cannot open VM attachment - Veeam Backup

Running windows 10 1903 - with hyper-v installed

PXE boot a vm 1 gen with legacy network adaptor - for running windows 7

getting this image - but it boot fine under win10_1809

Anyone with a solution for this?

#LnQ¯\_(ツ)_/¯

↧

Microsoft Hyper-V 2016 Server

November 27, 2019, 11:57 pm

≫ Next: Slow network speeds on internal network

≪ Previous: PXE boot a vm 1gen with legacy network adaptor win1903 - not working?

I build a vm in windows Hyper-v server 2016,but it can't receive packet from host after reboot it .It can work well after i logout and reload.

↧

Slow network speeds on internal network

November 28, 2019, 9:29 am

≫ Next: Problem installing Microsoft .NET Framework 4.7.2 and 4.8 on Two Physical Windows Server 2012 R2 x64 hosts

≪ Previous: Microsoft Hyper-V 2016 Server

hi,

we have multiple Hyper-V servers in a cluster.

we have a virtual service that uses internal network switches within hyper v and we are seeing slow performance on this.

to test connectivity i just spun up a basic windows server and added it to the internal network.

when i copy a file using the internal network IP range i barely get about 80MBps per sec (less than 1Gbps).

we have physical networking of 10Gbps and the internal network is showing a status of 10Gbps so i dont understand how the internal network which is purely virtual gets such slow network speed.

hope someone can help me out here.

thanks

↧

Problem installing Microsoft .NET Framework 4.7.2 and 4.8 on Two Physical Windows Server 2012 R2 x64 hosts

November 28, 2019, 9:40 am

≫ Next: Hyper-V - hypervisor.cpuid.v0 or hidden state=on equivalent

≪ Previous: Slow network speeds on internal network

Hi all, I'm unable to install 4.72 and 4.8 .NET on the two 2012 R2 x64 Physical servers. Has anyone dealt with a similar issue before and has a solution? - EventData

errorCode 0x80070005

Thanks in advance!

- System

- Provider

[ Name] Microsoft-Windows-WindowsUpdateClient [ Guid] {945A8954-C147-4ACD-923F-40C45405A658}

EventID 20 Version 1 Level 2 Task 1 Opcode 13 Keywords 0x8000000000000028 - TimeCreated

[ SystemTime] 2019-11-28T17:07:54.247990500Z

EventRecordID 3422040 - Correlation

[ ActivityID] {DB6D95C8-A605-0004-12FC-6DDB05A6D501}

- Execution

[ ProcessID] 176 [ ThreadID] 13136

Channel System Computer XXXXXXXXXXXX - Security

[ UserID] S-1-5-18

- EventData

errorCode 0x80070005 updateTitle Microsoft .NET Framework 4.7.2 for Windows Server 2012 R2 for x64 (KB4054566) updateGuid {AFDA57EE-ACB9-43D3-94F8-4D6A2551C9C2} updateRevisionNumber 200 serviceGuid {3DA21691-E39D-4DA6-8A4B-B43877BCB1B7}

↧

Hyper-V - hypervisor.cpuid.v0 or hidden state=on equivalent

June 19, 2019, 7:33 am

≫ Next: Hyper-V 2016 VMs stuck 'Creating checkpoint 9%' while starting backups

≪ Previous: Problem installing Microsoft .NET Framework 4.7.2 and 4.8 on Two Physical Windows Server 2012 R2 x64 hosts

KVM (hidden state='on' ) and Vmware(hypervisor.cpuid.v0) both provide a way to hide to the vm, the fact they are virtualized.

I've done:

ExposeVirtualizationExtensions : True

But it doesn't seem to have the same effect. The goal is to pass a nvidia geforce 1080 to a linux vm for the purpose of tensorflow. I've successfully done the passthrough, and it shows in lspci, but nvidia-smi returns "Unable to determine the device handle for GPU". In a windows guest I believe this manifests it's self as error 43. KVM and Vmware handle this by simply hiding that the system is virtualized. This fixes the issue in both windows and ubuntu on either virtualization platform

How can I achieve this with Hyper-V and an Ubuntu guest?

↧