I’m having an issue after Live Migration where VMs are intermittently losing their network connectivity for up to 90 seconds, then it returns without intervention. It doesn’t seem to matter which hosts are involved in the Live Migration, although moving from node 2 to node 3 seems to have less frequent issues.

4 node Hyper-V cluster running W2012 R2 Data Center on DL580 Gen8s. Fully patched and with recommended Hyper-V and Failover Clustering hotfixes. I’m aware of the issue with intermittent connectivity with Broadcom 1Gbps NICs, but believe it is resolved in the drivers we’re using (latest from Broadcom – 17.2.0.0). Additionally, VMQ confirmed as disabled on all NICs.

Network setup on hosts:

HV_HM – single NIC used by host only. Production network subnet

HV_CSV – 2 NICs teamed using LBFO. Set to LACP and Dynamic. Backend network on 192.168.5.0/24 subnet

HV_LM – 2 NICs teamed using LBFO. Set to LACP and Dynamic. Backend network on 192.168.6.0/24 subnet

Backup – single NIC with virtual switch BKUP-NET. Management OS allowed to use. Backend backup network on 192.168.1.0/24 subnet

vEthernet (BKUP-NET) – virtual NIC for backing up host. Backend backup network on 192.168.1.0/24 subnet

VM – 2 NICs teamed using LBFO. Set to Switch Independent and Dynamic. Virtual switch VM-NET with management OS not allowed to use. Production network subnet

Connections to the backend networks are 1Gbps into Cisco 3750 switches, which are managed by our own Networking team. These are stacked. Connections to the Production network are 100 Mbps into Cisco 3750 switches, which are managed by a 3^rd party. I don’t know if the 2 switches involved are configured as a stack or not.

For the VM team members on each host, nodes 1 & 2 have 1 team member on one switch and the other team member on another switch. For nodes 3 & 4, all VM team members are on the same switch.

If I move a VM that has a vNIC on the VM-NET virtual switch and a vNIC on the BKUP-NET virtual switch and have pings going to both of its IP addresses, the BKUP-NET IP consistently responds. The VM-NET IP is the one that drops after Live Migration; sometimes with a time out then a destination host unreachable, other times straight to destination host unreachable.

If I run an arp -a before the move I can see the MAC for the VM. When it has an issue after migration, the ARP cache on the host shows the MAC address while the pings are timing out, then when it changes to destination host unreachable, the IP and MAC address for the VM are not listed in the cache.

The only differences I can think of between the non-working VM-NET connections and the working BKUP-NET connections are the Teaming Mode of the NICs on the host; and potentially the port/switch configuration, which is a black box for me for the problematic VM-NET connections.

My feeling is that it could be a switch configuration issue, but I’m not sure what settings to have the 3^rd party check. Has anybody seen a similar issue or have any thoughts on how to progress please?

I am having really slow network performance using Hyper-V on Windows Server 2012 R2.

1) I am using Broadcom NetXtreme Gigabit network adapters.

2) I have five Virtual machines that are using a single virtual switch that is hooked up to a single Gigabit NIC and these machines are not generating that much traffic.

3) This was not a problem when I was using Windows 2008 R2, only a problem when I reformatted from scratch and installed Windows Server 2012 R2.

4) When I ping the host OS (hyperv3), I can getting lots of latency (200ms), but when I check the network traffic over that, I am only getting (2.4 megabits upload and very small download rates).  I am using Remote Desktop to connect, so that makes sense.

How can I troubleshoot this where this wasn't a problem before, but now it is?  The OS has changed, the network cables and ports have not changed.   Here is a screenshot showing my network info.

Can you help me troubleshoot the network performance issues?  Thank you guys :)

Vincent Sprague

This is the first time in some time I'm building a Hyper-V server from the ground up.. a few questions on the network side of things here..

I have teaming setup for the physical nics on this server.. I have basically 8 1GBit ports available on the server.. i've teamed up 2 at a time, giving me 4 Teams

Team1:  TeamManagement (static ip)

Team2:  TeamHV#1  (dhcp)

Team3:  TeamHV#2  (dhcp)

Team4:  TeamHV-EDGE (dhcp)

At this point i have a static ip for the management team.. but no static ip's for the other teams (which will be hyperv virtual switches) and i believe this is how i've done this in the past (dhcp basically)..

I then created 3 virtual switches that are associated with each "team" member with a similar name.

Is there any need to set those other 3 teams that will become virtual network switches to static ips before creating the virtual network switches for the VM's?

I do notice that if i ping the host server, it doesnt respond on the main static ip, but one of the dhcp ones..(3 dns entries)  I do have allow management operating system to share this network adapter set on all 3 Virtual Switches.

I looked on our other Hyper-V server and i noticed that the hyper-v virtual adapters created, you cant view the status and see that its a dhcp provided ip address, but this new box (same OS), you can see the status on the adapters and it shows the dhcp address... not sure why (they are set to "disabled" on the old hyper-v box but enabled on this new one with the same settings somehow)

***EDIT:  I see if i uncheck Allow management operating system to share on all 3 virtual switches, that the microsoft hyper-v adapters actually get removed from network connections and i can ping off the single main static ip address  (i'm still not clear why on the old box i had these boxes checked but yet the hyper-v virtual adapters became disabled and there werent multiple ip addresses on that machine, and here i have to uncheck the option, maybe team related?)

On the other side of things, i've always just defaulted to setting the virtual switches to External for internet access.. we really dont have a DMZ in our environment.. everything flows through a sonicwall gateway.

However, as of now, I have one of the teams setup to be in a future DMZ zone off a port on the sonicwall, mainly for a Skype for Business Edge service.

Any reason to avoid using the external setting on a virtual switch at this point, given our environment?

Thanks in advance

Tech, the Universe, Everything: http://tech-stew.com Just Plane Crazy http://flight-stew.com

I am receiving the following error when attempting to move a VM to a Hyper-V Server:

An error occurred when attempting to contact the Virtual Machine Management service on destination computer.  Verify that the service is running and that you are authorized to connect to the destination computer.  You do not have the required permission to complete this task....

I am logged on as the Domain Administrator and have configured Constrained Delegation.

What do I need to do to resolve this permission issue?

Hi,

Just a question in theory at the moment, but how unsupported is running Hyper-V within VMware ESXi 6?

I already know VMWares non-favourable verdict on this and I assume that MS would say the same?

Certainly, the internet is full of people who have done this and found many issues. But I can't find MS's official standpoint on running this.

Thanks

James

We are in the process off moving our 2008R2 VM's from the 2008R2 HyperV servers to new Server 2012R2 Hosts.

We shut down the VM's copy the files and VHD's to the new CSV's en import the VM in the Hyperv Manager. Then we make them high available in the Failover Cluster Manager (Configure role - Virtual machine). We mount the integration tools and update the VM to version 6.3.9600.16384

For a specific type of VM (mostly RDS Host servers) we always had Dynamic Memory configured (when they were hosted on de 2008R2 platform), so we are using the same settings on the 2012r2 platform. The memory settings were;

Startup memory: 1024 MB

Minimum memory: 1024 MB

Maximum memory: 12288 MB

These VM's reboot every morning, this is done for specific reasons. But now once in a while (once per week/2 weeks) we notice that the VM's are not using more memory then 1024 MB while the demand is much higher. Rebooting the server helps most of the times, live migrating to another host also helps. In the VM we see that memory usage in the taskmanager is 99-100%, and after the move it immediately starts using more than the minimum configured amount.

Until the failover the memory usage was 1024 MB and it did not get any higher.

This happened several times. Last week we changed the Memory configuration to:

Startup memory : 2048 MB

Minimum memory: 2048 MB

Maximum memory: 12288 MB

But this morning we had a call about the performance of one of the VM's, We saw that it was only using 2 GB memory while the demand was much higher. After live migrating it to another host it started using more memory immediately.

The 2012R2 hosts are not overcommited, there is a lot of memory still available for the VM's. Those VM's never had this problem on the 2008R2 Hyperv platform.

Any idea why this happens?

Peter Camps

I'm reading through this KB (https://support.microsoft.com/en-us/kb/2974384) since it describes my issue quite well.

The team in question consists of a quad port i350.

Question 1: Can I make any of the changes while the system is in production, or should I drain the node just to be safe?

Question 2: The cluster doesn't show any negative effects from this error [yet], Can I leave it alone?

Question 3: If Sum-queue is the underlying issue, is changing over to Min-queue mode feasible?

Name                                            : Team.VM2.4x1GbE
InterfaceDescription                            : Microsoft Network Adapter Multiplexor Driver #2
Enabled                                         : True
NumberOfReceiveQueues                           : 0
Profile                                         : NUMAStatic
BaseProcessor: [Group:Number]                   : 0:0
MaxProcessor: [Group:Number]                    : 0:38
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:2/0  0:4/0  0:6/0  0:8/0  0:10/0  0:12/0  0:14/0
                                                  0:16/0  0:18/0  0:20/0  0:22/0  0:24/0  0:26/0  0:28/0  0:30/0
                                                  0:32/0  0:34/0  0:36/0  0:38/0
IndirectionTable: [Group:Number]                :  :

Hi All,

So I am running a 2012 R2 Hyper-V Cluster.

The guest machine is also a 2012 R2 server. It has been allocated up to 4 gigs of dynamic memory.  When I go to the task manager to see how much memory is in use it is totally different than what I see in Hyper-V manager. In this case, the guest is showing that it has 4 gigs of memory with only 3.1 of it in use. In Hyper-V manager, I see that has assigned 2492 MB of memory and the memory demand is 2093 MB. 

So the question that I have is many different groups where I work (staff members) run guest servers on my Hyper-V cluster, how do they figure out what their server is using for memory resources if the guest OS is showing something totally different than the Hyper-V manager??  Who is right??

Everything I have read so far says that eventually task and Hyper-V Manager will settle down and be the same but I have yet to see this. I see that in all cases a huge discrepancy between the two.

Anyone??

Hi all, I'm looking for some advice on setting up CAU on a 2012r2 hyper-v cluster. Systems Center Configuration Manager is set up to deploy updates to all servers across the company every month on a week by week schedule depending on server function. For the hyper-v nodes, you can see the updates waiting to be applied in the server's Software Center and you can see that they will be applied at a certain time if you don't apply them manually. Well, I apply them manually to avoid the hyper-v nodes from restarting at the same time and causing mass vm outages. That's where I started looking into Cluster Aware Updating.  To test this, I made sure there were updates shown in Software center on each of the cluster nodes. Then I bring up the Cluster Aware Updating screen and click on "Preview updates for this cluster" and then "Generate Update Preview List".  No updates found. So, that's where my question originates from. How can I get CAU to see the updates that SCCM has set up to deploy on the hyper-v nodes? I'm not sure if you have to get a Windows Plug-In for CAU or configure SCCM a specific way for the hyper-v nodes. Any feedback would be much appreciated. Thanks.

Hey guys,

We're setting up a Hyper-V with internal virtual switch, now upon checking on the Microsoft guides, we should be able to make the host and guest vm to communicate with each other without making the guest vms visible to our entire network. Now everything is working fine except that the guest vms and host can't talk to each other.

Things I tried:

Disable all firewall

set the guest vms with static IP

Ping

Tons of searching around for the solution

Maybe anyone knows something here that I'm missing?

Thank you,

Good morning.

I have to live migrate VMs from “Hyper-v on Server 2012” to “Hyper-v Server 2012 R2”.
Well, at the same time I have to migrate all the storage (Config, Snapshots, Pagingfiles and VHDs).
Up to here everything is ok. 
At the same time without loose the VMs connectivity I have to change the vNetwork because the vNetworks on the new Hosts are different.

I will use Move-VM powershell cmdlet, example:

Move-VM $VM remoteServer –VirtualMachinePath $Path1 –SnapshotFilePath $Path1 –SmartPagingFilePath $Path1 –IncludeStorage –VHDs @(@{"SourceFilePath" = $OldVHDPath1; "DestinationFilePath" = $NewVHDPath1}, @{"SourceFilePath" = $OldVHDPath2;"DestinationFilePath" = $NewVHDPath2}) 

When I do it manually through the Hyper-v Console, it ask me to change it:

How can I change vSwitch at the same time that I Move the VM using a cmdlet without loosing connection?

Thanks in advance and have a nice week,
Angel Biurrun.

Hello, i am having a problem with one of my servers on Hyper-V. I have this setup:

- Hyper-V Server 2008r2 (24GB of ram- previosly 16GB)on this server i have 2 virtual machines one is SBS2011(10GB of ram Previously 7GB) and the other is server 2008r2 with SQL(10 GB of ram - previosly 5GB) on it.

The problem starts randomly, the virtual with the SQL shuts down or goes down (the SBS server went down just once or twice), and when i try to turn it on the server said that there is not enough resources to start this server. Then i set 7GB of ram to this SQL server and the server is starting again, then a few days or even weeks is OK, then randomly this thing happens. I then put 5GB to the server SQL and then its ok, and this goes on and on,... Al before that i had in the server just 16 GB of ram but when this problem occured i added aditional 8GB of ram into the server because i thought the server is low on ram.... But after upgrade the same thing is happening every few weeks... Could somebody helps me to find the solution.. In eventviewer under  system is just error of the server that there is not enogh resources avalible...I checked resource monitor when this happens and it shows 22GBof ram in use and around 1-2GB in standby, free memory is droping from 800MB to 2MB the same is when all runs smothly...

I have 5 or 6 same setups around the offices but this problem is just here...

Using HyperV Manager 6.2.900.16384 to connect to a 2012 R2 HyperV server over a 10MB MPLS WAN connection spikes the utilization over that connection. I think this is a bug that should be investigated. This is repeatable and can be seen using FLUKE monitoring. We didn't figure it out until late in the day.

Hi,

I'm setting up my 1st virtual server (lab environment before we go live) and I'm unsure what if any roles I should assign on the Hyper-V server.  

We are a small business with less then 50 users.  The physical server this is installed upon will eventually be our main file and AD server.  The only other server in our environment is a 2008r2 remote desktop server.  

This server is Server 2012 r2 on a new HP ProLiant DL380.  The Hyper V server will have it's own mirrored SAS 300gb hard drive.  There will only be 2 applications installed on this server and both use a SQL EXPRESS instance (NOT full blown SQL server). 

So I'm confused how to set up a virtual server.  I know to add the Hyper V role on the main server and I know how to add a new virtual server and how to redirect it's storage location.  Where I get confused is if I need to install any roles or simply attach it to our domain then install the SQL express, the apps and start mapping the 2 drive letters?

Thx,  :)

Di

Hello,

I am experiencing a strange problem on my VM's, when i copy a larger file (over 10GB) the file copy starts alright, and speed is good (60-100MB/S), but then suddenly, kind of random, the speed drops and the connections kind of spikes on the target VM.

I have tested my connection to my ISCSI storage, and if i try to copy directly to my CSV speed is good, around 200MB/S, so the problem must be either on the VM NIC or the VM network.

The problem is the same on either one of my cluster hosts (i have 3) , i use SCVMM, if that makes a difference in this case, even though i wouldn't think so.

VMQ is disabled - so that's not the problem.

Thanks! 

/Mick Negendahl

Dear reader,

We sometimes have a problem where our windows 2012 r2 RDS virtual servers, that reside on windows 2012r2 hyper-v hosts, loose their dynamic memory and only have their startup memory left to work with. Users start complaining that things are very slow etc.

If I check several screens (RDS Broker load balancing, hyper-v manager, cluster manager and the vm's task manager) it's clear that the vm only has its startup memory allocated. I'm not sure if this happens instantly or immidiatly after the nightly reboot. To resolve the problem we have to call all users on the vm where it happens and ask them to logoff (if they are even able to), and then we reboot the machine.

I have checked the logs from the machine where the VM resides on and the logs from the vm itself. But I cannot find anything. We also have alot of windows 2008r2 vm's with dynamivc memory, but none of those have ever had this problem.

Searched the internet, but so far it seems we are only.

Can anyone give me a lead to troubleshoot this?

Best regards,

Ruud Boersma

MCITP Enterprise administrator

Scenario

I have a 3 Windows 2012 R2 Hyper-V hosts which is participating in a cluster, all of the virtual machines are hosted on CSV's. We also use Dynamic Memory for most of the servers

We do regular reboots every week for our virtual machines, but we've found that sometimes, a virtual guest will suddenly stop requesting additional memory.  At the time of writing, I have identified 10 machines - on different hosts, and even different clusters which are exhibiting this behaviour.  They're also running OS's ranging from Windows 7 to Windows 2008 R2 and Windows 2012.

I have selected one to demonstrate below.

Here, you can see the virtual machine is configured to use Dynamic Memory

and here, you can see that it's not actually demanding more than the start memory, despite the fact that I've overloaded the servers virtual memory by using MSPAINT.

Workaround

Rebooting the virtual machine does not resolve the issue, but live migrating the virtual machine from the host it is on to another host suddenly allows it to demand more memory.  One can then live-migrate it back to the original host and the problem is no longer there.

Troubleshooting

We have performed the following troubleshooting steps:

• Verified that the Integration Services are up-to-date (currently running 6.3.9600.16384)
• Verified that the virtual server has up-to-date Windows Updates
• Checked the Hyper-V-Worker\Admin event log on the current host - no errors pertaining to this VM.
• Verified that the Hyper-V host has sufficient RAM (currently only using 542 out of 768 GB)
• Checked the System event log on the virtual machine - nothing out of the ordinary to report.
• Verified that all services which start with the word "Hyper-V" on the guest machine is running.

Using the following command, I can easily identify which machines are affected (this command applies to SCVMM only):

Get-SCVirtualMachine | where {$_.DynamicMemoryEnabled -eq $true -and $_.DynamicMemoryDemandMB -eq $_.Memory} | ft Name,HostName,OperatingSystem,Memory,DynamicMemoryDemandMB,DynamicMemoryMinimumMB,DynamicMemoryMaximumMB

I do not know any diagnostic program which can give me more information regarding the memory status of a server - specifically one which can tell me more information about Memory demand, and why it's not working. 

Considering that the problem is resolved when I live migrate the affected machine, it would suggest that the issue is with the host, but as live migrating it back no longer exhibits the issue - that seems to negate that theory.  Additionally, the other 103 virtual machines which run on this host seem to not show this issue.

It's almost as if the machine is being ran with static memory, and Hyper-V doesn't realise it's dynamic, but the moment it's live migrated, it suddenly realises this and enables dynamic memory.

Any thoughts?

Hello.

I've set up a NAS as an iSCSI storage unit (one LUN) and made it a CSV - it's 14.5 TB.
On the storage I've created a dynamic VHDX file at around 14.3 TB.
I've then attached the VHDX file to a VM that runs my backup software (Veeam).

When no data was on the disk the space available in Windows on the VM was almost the same as the available space on the iSCSI unit / CSV.

Recently I've been having problems with my VM pausing due to low disk space.

In Windows the drive has 8,41 TB free space of the 14,3 TB available.
In the Failover Cluster Manager the drive has 122 MB free space of the 14,3 TB available.
If I inspect the disk via the Failover Cluster Manager it says the current file size is 14,28 TB and the maximum disk size is 14,4 TB.
In explorer one of the Hyper-V hosts I can see the VHDX file is 14.2 TB.

It appears there's an inconsistency as to how much data and free space is available.
I expect it might be the VHDX file has actually been used to it's fullest potential and then the backup software has removed content according to it's retention policy leaving "space" inside the VHDX structure. Now when backing up the software writes to the disk but the OS thinks the disk is full and puts the VM on pause mode to prevent crash. If this is correct how do I prevent this from happening? I guess it's possible to use fixed size VHDX instead but are there any other options?