This part continues to build out NSX on top of the Nested ESX cluster created in the last part ESX/NSX nested lab – Part I.
Part I included some basic NSX setup for convenience such as.
- NSX Manager installed from OVF, and linked to vCenter.
- Physical network ESXi uplinks included allowed vLAN 100 for vxlan.
- distributed switch set to allow Jumbo 9000 MTU, (> 1600 MTU require for NSX )
- Install NSX on hosts
- Create IP pools
Networking & Security – Groups – IP Pools - Controller-pool ( in management IP LAN )
10.24.10.1 – 10.24.10.5 /16 - vTEP-pool ( create new unused LAN subnet + vLAN for vxlan )
10.100.1.10 – 10.30.1.20 /24 vlan: 100, GW: 10.100.1.1, no DNS
The vxlan GW is not used here – this would typically be a hardware device that routes vxlan to vLAN.
- Controller-pool ( in management IP LAN )
- Setup new vLAN 100 for vxlan segment on physical switch
Important: Set vxlan ( vlan 100 ) mtu to 1600 on physical switch.
E.G. For Cisco 2970 switchconfig t vlan 100 name vxlan mtu 1600 exit exit
- Install one Controller ( into NSX-compute cluster )
For a lab, only a single controller needed – not three
Controllers can run on non NSX prepared hosts ( e.g in Management cluster ) - Install NSX to Compute cluster ( with 3 Nested ESX/NSX hosts )
- Prepare vxlan on hosts
- Setup Segment ID
- pool: 5000-5999 ( 999 segments )
- Multicast: 239.1.1.1-239.1.4.254 ( for optional multicast )
- Setup transport zone ‘Global’
- To include NSX-compute cluster, Unicast mode
- Create IP pools
- Test vTEP connectivity
After NSX install each host should have a vTEP ( with IP from vtep-pool )
connected to the autogenerated vxlan portgroup.
Login to one of the nested hosts and ping the vTEP on other hosts. Make sure at least one Nested host is on a different physical host to verify physical connetivity.- ssh esx-n1 ( host )
esxcfg-vmknic -l ( show vmkernal nics and IP’s ) Note vmk3 is used for vxlan.vmkping ++netstack=vxlan -s 1470 -d -I vmk3 10.100.1.10 vmkping ++netstack=vxlan -s 1570 -d -I vmk3 10.100.1.11
- The 2nd command should ping to show MTU 1600 is working. If only the 1st command pings , then you need to set mtu size on vlan 100. Ping to all combinations nested hosts vtep ip’s
- ssh esx-n1 ( host )
Links
- Nested ESXi 6.5 virtual appliance (vGhetto)
- Nested ESXi LearnSwitch (vGhetto)
- How to clone a nested ESXi VM (vGhetto)
- NSX issue on dvs in nested ESXi (telecomOccasionally)
- Nested Virtualization (Limitless) installing ESX, trunked portgroup
2 thoughts on “ESX/NSX nested lab – Part II”