Note:  I’ve recently re-done my lab and found significant changes to this post, so I’m re-writing this post and splitting it into two parts.

• For reference:  My physical Lab setup
• Continue on with the following parts:
  • ESX/NSX nested lab – Part I    (Install ESXi as a VM)
  • ESX/NSX nested lab – Part II   (Setup Nested ESXi VM with NSX )
  • Adding vSAN to nested lab   ( Prerequisites for vSAN and turning it on )

Overview

This guide documents how I setup my working nested ESX/NSX lab after having numerous problems setting up the nested ESX 6.5 appliance with NSX  with existing documentation.  One issue is that most nested ESX/NSX documentation covers ESX5.5, and much has changed since then,  and it’s not immediately clear what parts to skip or use.

This guide covers items specific to nested ESXi & NSX.  Since this is a lab, only a single NSX controller is used to save resources,  but if you have the capacity all 3 can be installed.

Thanks to William Lam’s ESX appliance and blog entries below I was able to get started with nested ESXi, but I continued to have random issues with networking as there are numerous moving parts and most of the physical networking is not described.  My lab tries to stay close to production recommendations of keeping networking, storage, and vMotion on separate vLANS, and this lab maintains that separation as much as possible.

I’m  moving away from William Lam’s ‘vGhetto ESXi’ appliance, as most of the features added in that appliance are now handled directly by ESXi 6.5 and later such as:

• The vGhetto ESXi appliance has ESX Mac learn dvFilter installed, which conflicts with the newer method of using ESX-LearnSwitch on the physical host.
• ESXi 6.0+ installer now has vmtools pre-installed , so that feature of the appliance isn’t needed anymore.   When ESXi detects that is is running as a VM, the vmtools is activated automatically.
• The ESXi cloning referenced in link #3 had numerous issues with dropped packets on vMotion and NFS so I’m not recommending cloning for ESXi 6.5.  Creating new ESXi hosts with a fresh ISO install had no issues.  Cloning is untested for ESXi 6.7
• ESXi 6.7 and later have ESX-Learnswitch VIB pre-installed so skip the install on the physical host.

Continue reading “Definitive Nested ESXi 6.5/NSX guide”