I’ve had one of those weeks when I remember why I work in IT. It was the inaugural super user group meeting for the VTUG, I’ve been deep diving into performance metrics and determining the best practices for Server 2008 R2 Guest VMs running on ESXi 5 connecting to an Isilon CIFS share.
First let’s talk about the Virtual Technology User Group. The VTUG has sprouted from the NEVMUG and you can learn more about it at the VTUG website. At the head of this brain child is Chris Harney @csharney. If you do not know Chris or haven’t met him before next time you are at VMWorld, or a NEVMUG take some time to talk to him. He’s one of the few genuine great guys left. I’m looking forward to seeing what events come out of this group as well as watching the online community grow.
I ran into an interesting issue this week, we’ve been preparing some Dell R810 Servers with Broadcom 57711 10Gbe NICs to go live with a data transfer intensive application. The application would connect to a CIFS share on a Isilon. Within the virtual machines we were seeing a spiking a high spike of CPU utilization when testing the CIFS share with robocopy. The issue was odd and left us scratching our heads. This lead down a spiral of performance documentation ranging from VMware to Microsoft and Dell; I could find all the documentation I was looking for in one place so I decided this blog would be the best location.
There is an amazing document that VMware published that details tuning for latency specific applications, in my situation the document was great to reference and brought about ideas for testing different configurations.
Within the document is this handy little chart that outlines the changes in the document,
I’m still testing the above configurations and will update when I do.
This lead me down the path of determining which drivers were being used for the BCM 57711 nics that I installed after installing ESXi. Come to find out I was using an outdated driver and updated the driver. I also found out that my BIOS version was outdated as well as the NIC firmware. Both were updated for the R810. To be honest I wish I could get on a Dell Mailing list that emails me every time there is an Urgent update to firmware or drivers for servers I own. I remember HP doing this but for some reason I’ve never got a message from Dell.
The BIOS, 2.7 release I installed fixed a “lock up” when L7 series processors were installed on the system. Of course I had L7 processors on this system.
The last piece was Server 2008 R2.
Which to me is the most complicated piece. I did a few things for Server 2008 R2. First I enabled File Services, second I wanted to learn more about Receive Side-Scaling, TCP Chimney Offload, and Network DMA. From what I understand TCO doesn’t work in VMs, but NDMA and RSS does.
I didn’t realize how large this post was getting so I’ve decided to split it and continue later on.
The next blog post will review how to enable NDMA, RSS and how to disable TCP Chimney Offload. It will also include the 2008 R2 Performance guide.