Hi,

We have a server running Windows Server 2019 Standard with Hyper-V. The machine contains 4 Intel X722 LAN ports (on a single card) in a LAN team, functioning as a Virtual Switch. Also the machine contains 2 Intel i350 LAN ports, forming a team for LAN access as well.

Multiple guest OSes are running on this HV host, all of them being Server 2019. All guests experience issues visiting secure websites, sometimes with downloading, have a lot of certificate related issues, ... Some web sites can't be visited at all.

When visiting these from the HV host, everything is fine although it's the same OS.

Earlier we had issues reaching all servers (host and guests) using Teamviewer, but this was solved after applying the fix as mentioned in https://forums.lenovo.com/t5/ThinkSystem/SR630-amp-Intel-X722-issue-with-Windows-Server-2019/m-p/4442619#M867

At this point we have very good performance on our guests but suffer from issues with about anything that smells like using certificates, https traffic, ... For example the Belgian website waterinfo.be is not reacheable from within a guest, but is visited fine from the host.

Time and date settings are OK, all in sync with the domain controller (one of the guests), apps run fine, documents open without issues, SQL Server running nicely and fast ... 

We're at the end of our inspiration and hope to get some new here :-)

Thanks

K.

Hi all,

Brand new Fujitsu RX2540M4 server with 2 Xeon Silver CPUs, 128GB DDR4 mem and 6 x 480GB SSD in RAID-6. Server 2019 Standard ROK installed, Hyper-V role enabled. There are 2 VM's running on it, both Server 2019 STD, in production since about 2 weeks. SRV1-DC is domain controller with 8GB RAM assigned, SRV2-ADMIN is domain member and hosts a SQL SRV DB and acts as a terminal server for 6 thin client users (Office 365 ProPlus apps and ERP system users), with 96GB assigned to it. NIC team 1 consists of 2 Gbit adapters, used by the Hyper-V host, NIC team 2 consists of 4 Gbit adapters and is used as a virtual switch for the VM's.

Today, SRV2-ADMIN suddenly rebooted twice, the following errors can be found in the Hyper-V events on the HV host:

'SRV2-ADMIN' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x3B, ErrorCode1: 0xC0000005, ErrorCode2: 0xFFFF864C3F0BBAA9, ErrorCode3: 0xFFFF8C0BDAE62250, ErrorCode4: 0x0.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID B6FE9B8F-9129-4B6A-81AB-79BCE490AD6C)

Error code 0x3B assuming a memory access violation

'SRV2-ADMIN' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x50, ErrorCode1: 0xFFFFFC048E600000, ErrorCode2: 0x2, ErrorCode3: 0xFFFFF8005A252377, ErrorCode4: 0x2.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID B6FE9B8F-9129-4B6A-81AB-79BCE490AD6C)

Error code 0x50 PAGE_FAULT_IN_NONPAGED_AREA

The errors make me suspect the RAM in this server, but this is a brand new machine which has 4 sticks of 32GB ECC memory. Before contacting Fujitsu I would like to be 200% sure it's a hardware and not a driver issue.

The Hyper-V host hasn't rebooted, only this one VM. Of course, it's the biggest memory user so if one of the 4 sticks of RAM is going south, chances are high that the errors occur within this VM.

I hope that someone out there recognizes this behavior and is wanting to share his/her experience here.

Thanks!

Kristof

Hi,

I have a dell R720 configured as a VDI virtualization host, hosting approximately 30 Windows 10 VMs. Specs are as follows:

Intel Xeon CPU E5-2650 v2 @ 2.60Ghz

2 sockets

16 cores

32 logocial processors

192GB Ram

The problem I'm facing is when assigning 32 vCPU to some of the virtual machines, the VM is not detecting the second NUMA Node and socket, and just uses the one, resulting in performance issues with startup times and CPU usage. 

When a VM has 32 logical processors assigned and is using 1 NUMA nodes and socket, I receive event ID 3056:

'VM_NAME' is configured to have a different non-uniform memory architecture (NUMA) topology to that of the physical machine. Some virtual processors are accessing memory on a remote NUMA node. This configuration is not recommended because it will reduce performance

There are some virtual machines that are able to use all numa nodes and doesn't seem to generate the error, which is strange as the vms are using the same master image.

How can I set these vms to use all numa nodes and sockets when assigning 32 logical processors? I normally configure all vms with maximum virtual processors then adjust restriction in the VM settings.

Hi guys,

can someone please advise on how to configure hyper-v for replication between a cluster 2 nodes in mainsite and a standalone hyper-v in the DR site.

Regards,

I have a windows 10 enterprise x64 (10.0.15063) hyper-v host and a windows 10 pro x64 (10.0.18362) hyper-v guest. Both run windows defender and no other anti-virus. All has been working fine until today, I see this behaviour:

 - The guest starts up in the hyper-v console and I can connect using RDP or the built in window from the management console. All is good.

 - I can minimize the session and re-store, all is good.

 - If I close the remote desktop session then the vm is in a strange state. I cannot reconnect to it or shut it down, my only choice is to turn off the VM (!) then restart it and again and only then is it ok again. (I see this exact same behaviour when connecting directly from the hyper-v console).

The only thing that I am aware of that's changed is that a windows update ran on the guest (to get me to the version listed above).

Please let me know if this is a known issue or what I can check on the vm that could cause this?

Hi all,

Just set up a windows 7 vm on windows 10 host. I am wondering since enhanced session mode is not available for my VM, what are my options to connect to host drives so I can access host usb-ports and printers?

Hi All,

I've got a Hyper-V Node connected to a Windows FileServer (all Server 2016) via 2x10Gb on the node and 4x10Gb on the FileServer. 

During testing I've removed all NIC teaming with no effect but bot machines are doing multichannel as expected.

But when I now run CrystalDiskMark on the VM it shows up to 1800MB/S _read_ but only 150MB/R _write_.

This can be also watched on the network side as the FileServer NICs are not show much traffic during write.

Any Idea how I can track that down? Measured already Read/Write on the fileserver but this is good (RAID100 SSD with 6K MB read and 1k MB write).

If I just manually copy some files to the fileshare the transfer runs around 600MB/S.

BG

Daniel

After converting my main disk from MBR to GPT and in the process switching to UEFI, I can no longer run the virtual machines on my hyper-V server because the hypervisor is not running. 

My understanding is that the hypervisor is started during boot and that because I have switched boot method it is now not started. How do I fix this? 

Can someone please explain the correct activation and rules for the VM's.  With the standard edition you can not use AVMA, supposedly you use the same OEM key you activated on the host. When I try to use the key I got the 0xC004FD03 error. I used telephone activation for the 1st VM. When I activated the Host I answered 1 computer, apparently you answer 0 for the first activation, 1 for the second (VM1) and 2 for the third (VM3). When I use 2 I get the error. BTW I did try the AVMA keys first before I tried the OEM key. My OEM is useless and keeps telling me to use AVMA but I do not have  the datacenter version of windows server 2016 to use as the virtulization server. It seems that this problem is very common and Microsoft should clarify how to fix it! I have seen some vage info that the OEM should supply keys or authorize guests but I have not been able to get this done. 

Thanks in advance for any help. 

I have a cluster, 2 nodes. There are repeating events id 25, every few second:

The MAC address 00-15-5D-AE-25-19 has moved from port 53A6AB28-EBD3-4F72-B690-69406077B8A7 (Friendly Name: vSwitchFN_External) to port 6D5ABBEF-0A8E-4F5D-8CC2-0871846CEF82 (Friendly Name: BCK).

The MAC address 00-15-5D-AE-25-18 has moved from port 53A6AB28-EBD3-4F72-B690-69406077B8A7 (Friendly Name: vSwitchFN_External) to port 9A8780F9-D931-4F85-B087-397A2F718C01 (Friendly Name: HB).

Config:

No NLB (MAC spoofing) on VMs.

PS C:\Users\pero>  Get-NetLbfoTeam

Name                   : mgmt
Members                : {M2, M1}
TeamNics               : mgmt-624
TeamingMode            : Lacp
LoadBalancingAlgorithm : Dynamic
LacpTimer              : Fast
Status                 : Up

Name                   : vm
Members                : {K2L1, K1L1}
TeamNics               : vm
TeamingMode            : Lacp
LoadBalancingAlgorithm : Dynamic
LacpTimer              : Fast
Status                 : Up

PS C:\Users\pero> Get-VMNetworkAdapter -ManagementOS

Name          IsManagementOs VMName SwitchName MacAddress   Status IPAddresses
----          -------------- ------ ---------- ----------   ------ -----------
BCK           True                  vSwitchFN  00155DAE2519 {Ok}
HB            True                  vSwitchFN  00155DAE2518 {Ok}
SETswitchSMB2 True                  SETswitch  00155DAE250C {Ok}
SETswitchSMB1 True                  SETswitch  48DF371EFC2D {Ok}
LM            True                  SETswitch  00155DAE250D {Ok}

PS C:\Users\pero> Get-VMSwitch

Name      SwitchType NetAdapterInterfaceDescription
----      ---------- ------------------------------
vSwitchFN External   Microsoft Network Adapter Multiplexor Driver #2
SETswitch External   Teamed-Interface

PS C:\Users\pero> Get-VMNetworkAdapterVlan -ManagementOS

VMName VMNetworkAdapterName Mode   VlanList
------ -------------------- ----   --------
       SETswitchSMB1        Access 1980
       LM                   Access 1981
       HB                   Access 1044
       BCK                  Access 887
       SETswitchSMB2        Access 1979

PS C:\Users\pero> Get-NetAdapter

Name                      InterfaceDescription                    ifIndex Status       MacAddress             LinkSpeed
----                      --------------------                    ------- ------       ----------             ---------
mgmt-624                  Microsoft Network Adapter Multiplexo...       6 Up           F4-03-43-5C-75-91         2 Gbps
K1L2                      HPE Ethernet 10Gb 2-port 561T Adapter        32 Up           48-DF-37-1F-0F-09        10 Gbps
K1L1                      HPE Ethernet 10Gb 2-port 561T Adap...#4      30 Up           48-DF-37-1F-0F-08        10 Gbps
SMB2                      Hyper-V Virtual Ethernet Adapter #5          35 Up           00-15-5D-AE-25-0C        20 Gbps
SMB1                      Hyper-V Virtual Ethernet Adapter #4          11 Up           48-DF-37-1E-FC-2D        20 Gbps
LM1                       Hyper-V Virtual Ethernet Adapter #3          16 Up           00-15-5D-AE-25-0D        20 Gbps
BCK1                      Hyper-V Virtual Ethernet Adapter #2          27 Up           00-15-5D-AE-25-19        10 Gbps
HB1                       Hyper-V Virtual Ethernet Adapter             19 Up           00-15-5D-AE-25-18        10 Gbps
M2                        Broadcom NetXtreme Gigabit Ethernet #3       20 Up           F4-03-43-5C-75-91         1 Gbps
M1                        Broadcom NetXtreme Gigabit Ethernet          24 Up           F4-03-43-5C-75-90         1 Gbps
vm                        Microsoft Network Adapter Multiple...#2      17 Up           48-DF-37-1E-FC-2C        20 Gbps
K2L2                      HPE Ethernet 10Gb 2-port 561T Adap...#2      15 Up           48-DF-37-1E-FC-2D        10 Gbps
K2L1                      HPE Ethernet 10Gb 2-port 561T Adap...#3      18 Up           48-DF-37-1E-FC-2C        10 Gbps

The events intensively fill the log and sometimes network connectivity for these virtual adapters (BCK, HB) is temporary lost.

What is problem?

Hi everyone.

I have a task to migrate Virtual machines from one Hyper-V cluster(A) to another Hyper-V cluster(B) in another domain forest.

I prefer to do it by  sharing-nothing live migration with resource based Kerberos constrained delegation.

I have rise 2 AD forests with 2 Hyper-V Clusters in my lab. There are no any firewall enabled on hosts or ports blocked between networks A and B. Each infrastructure emulated on 1 physical host under Windows 2016 because it supports nested virtualization and gives opportunities to rise Hyper-V cluster undo 1 physical PC. Thus i have that:

Both Clusters runs at Windows 2012R2.  I have set transitive cross domain forests trusts between domains A and B and Kerberos constarined delegation between cluster hosts setup like described here  with little fixeds. Given example doesn't work because Get-ADComputer cant resolve computer object from trusted AD forest, thus command should contain - searchbase  and -server key to refer to trusted DC.

Get-ADComputer -Filter 'Name -eq "destHost"' -SearchBase "DC=A,DC=domain" -Server "a.domain" | Set-ADComputer -PrincipalsAllowedToDelegateToAccount (Get-ADComputer -Filter 'Name -eq "srvHost"' -SearchBase "DC=B,DC=domain" -Server "nt.lan")

After that the SIDs of delegated hosts appears in the PrincipalsAllowedToDelegateToAccount properties in host objects.

Also I set concrete IP addresses on Hyper-V manager / Live migration setting. I think better to do it, because once i have migrate virtual machines between two clusters in single domain and different networks - and moving fails somewhere over migration until i set concrete IPs for LM traffic. 

After that I tried to perform LM through Hyper-V manger GUI. 

Process begins, in the destination folder was created Planned Virtual machines folder with future VM configuration files within. 

After migration fails with error 0x80090303

In the Hyper-V branch logs got such events:

Source Host: 

Dest Host:

This tread  tells that LM in that case may be successful only through PS Move-VM comandlet.

I also tried such as shown here but get strange result.

[vins0010]: PS C:\Users\username\Documents>  Get-VM  | % { Move-VM -DestinationHost vins011.%dest.domain% -IncludeStorage -DestinationStoragePath "C:\ClusterStorage\Volume1\$($_.name)" -verbose }
cmdlet Move-VM at command pipeline position 1
Supply values for the following parameters:
Name: vins0004
VERBOSE: Move-VM will move the virtual machine "vins0004" to computer "Microsoft.Virtualization.Client.Management.Server".
Move-VM : Virtual machine migration operation failed at migration source.
Failed to create folder.
Virtual machine migration operation for 'vins0004' failed at migration source 'VINS0010'. (Virtual machine ID A70F0D0E-4183-4E5A-A34A-E74AFBE84BD8)
Migration did not succeed. Failed to create folder '\\VINS011\VINS0010.2159109741$\{f887c148-ea23-4ab4-b2a1-d9ed8813c5f5}\vins0004\Virtual Hard Disks': 'The network 
path was not found.'('0x80070035').

I cant understand why got so strange result path: \\VINS011\VINS0010.2159109741$\{f887c148-ea23-4ab4-b2a1-d9ed8813c5f5}\vins0004\Virtual Hard Disks

What does it mean VINS0010.2159109741$  ?

Does anyone migrate machines between domains and clusters at all?

We have a two clustered W2016 Hyper-V hosts, every couple of days one of the hosts gets stuck when the backup kicks off. In Hyper-V manager the VMs all say 'Creating checkpoint 9%' It's always the same percentage 9%. You can't cancel the operation and the VMMS service refuses to stop, the only way to get out of the mess is to shutdown the VMs, and hard reset the effected node. The backup works for a few days then it all starts again.

The only events I can see on the effected node is:

Event ID: 19060 source: Hyper-V-VMMS

'VMName' failed to perform the 'Creating Checkpoint' operation. The virtual machine is currently performing the following operation: 'Creating Checkpoint'.

Can anybody help please? Cluster validation is clean. Hosts and guests are patched up.

We have Hyper-V running on a 4 server cluster (using Failover Cluster Manager).   Lately we have had a weird problem where one of the virtual machines will start DRAGGING.

If we move the virtual machine to one of the other cluster hosts, the VM will immediately have improved performance.

Note that it doesn't matter which cluster machine we move the VM to.  In fact, we can move it to another cluster host, then move it back, and the performance remains high.  Which tells me, the problem isNOT with the cluster host.  

Suggestions?

Hello,

Im having trouble merging an AVHDX into the parent VHDX.  The AVHDX is only 21MB but after running the Merge-VHD command in powershell it gets to 99% and just stops or hangs there.  The IO on the CSV drops to nothing and it never actually finishes merging into the VHDX.

Is there anything else I should try here?  This is the last AVHDX file I needed to merge to recover the VM.  If I cancel the Merge-VHD powershell it locks the file and I have to reboot the node in order to try again.

Could another node in the cluster still be holding a lock on one of the files that is preventing the merge from finishing?

We have a 4 node Windows Server 2016 Hyper-V Cluster.  Over the weekend, one node of the cluster reported this in the System event log:

"Memory allocated for packets in a vRss queue (on CPU 14) on switch 620355C8-4D29-4D91-BE80-B840921EDC4A (Friendly Name: Team_Trunked) due to low resource on the physical NIC has increased to 256MB. Packets will be dropped once queue size reaches 512MB."

Seven seconds later it reported that it had increased to 512MB, then two seconds later the LiveMigration NIC reported it had begun resetting, then a few seconds later a reset was issued to \device\raidport1 (source: ql2300).  After two minutes of this and a few other repeats, I started getting warnings of CSVs no longer being able to access this cluster node and then the Cluster service shut down on this node and all VMs are shut down and migrated to other nodes in the Cluster.  

Our weekly DPM backups of the VMs had started about 1.5 hours before this occurred so there was some additional strain on the NICs at this time, though that should have gone through the NIC the OS is running on so I don't know why that would have affected the other NICs where the VMs general data was going through (Team_Trunked) and the LM NIC.

Does it make sense that the first warning about Event 252 would have caused all this or is there more to this?  

Hi,

I have a Hyper-v host running several VMs ...

I would like to automate the Network configuration on 1 or several VMs via PowerShell. 

When you use the cmdlet Get-VM, there is not the parameter -Name that I can use with the "Get-VM ... | New-NetIPAddress -IPAddress ..."

Do you have an idea please ?

Thank you.

Hi all,

Hoping someone may have ideas on how to resolve this one!  We have a 2-node cluster (HPE ProLiant DL360 Gen10) built on Windows 2019 Standard GUI, iSCSI to Nimble storage.  All validation tests pass with flying colors!  When we were testing, i.e. with very little load we had one instance of a 2019 VM (vm version 9) which got stuck in a 'stopping-critical' state when rebooting.  At this point, the host has gone 'awol' with cluster functions, from then on any cluster operation failed until the host is rebooted (draining would never finish, any other live migration times out and fails).  We found that we had a switch down so put it down to that (even though redundant paths were AOK).  Since then it has been working beautifully as expected, I've done plenty of migrations, draining of hosts without issue, until yesterday.  The client decided to patch a Window 7 VM, it rebooted and got stuck in the same fashion, Stopping-Critical.  I have tried the trick of killing the VM process, this does not work in this scenario.  There is nothing we can do except cold boot the server, forcing failover to the other host.  Obviously a bit of an issue as the 30 servers now running on there restart on the other host, interrupting file / print services etc.

I've read about disabling VMQ but I really don't want to go disabling the advanced performance features unless we absolutely have to.  I am using Dynamic memory on most of the machines, otherwise pretty standard configurations.

No issues with resource we are using under half memory on both hosts.

Any ideas?

Thanks,
Simon

We have VMs with snapshot what I generally like but if the host fails to boot and we need to restore the VHD files to a new machine we would have to merge with the snapshots.

Is there an easy way to recover a VM with snapshots or/and merge a snapshot with a VM without any down time?