References

• vSphere / ESXi homelab setup   ( Existing lab setup )
• Logical Switch setup on NSX   ( Earlier post on setting up Logical switches )

Intro

This post continues my studying of basic NSX concepts in preparation for my VCAP6-NV. I’ll try and cover some answers I’ve found to some of my mis-conceptions along the way, and finally run a few labs.

As a review ,  I have a 4 host Nested ESXi / NSX lab as described above.

After getting logical switching working,  one of the first interesting things to look at in the lab is providing Internet access.

To do this we can use an NSX Edge to our Internet provider in three ways:

• NSX Edge
• NSX Edge HA ( pair )
• Multiple Edges with ECMP routing ( Equal-Cost-Multi-path )

Using an HA Edge provides simple fail-over redundancy by using a 2nd edge in standby mode.  The fail-over for logical switch and firewall occurs after 15 seconds of missed heartbeats, which is often good enough for small clients.    Note that Firewall state and routing tables are synchronized between primary and standby,  so after fail-over traffic picks up without dropping the connection.   Note however  that Edge VPN and Load-balancer services do drop after an HA fail-over and require new connections.

Using multiple edges ( routing in parallel ) with ECMP  has the advantage that  single edge failure with N edges only affects 1/N of the traffic ( which will be re-routed in seconds over the remaining paths).   However the majority of traffic on un-affected edges experiences no problems at all.     Because ECMP steers traffic over N different networks depending on availability or load-balancing it require dynamic routing to handle those changes.

So how do we setup N/S routing in our lab ?

There are several options for N/S routing.  In fact in the vRealize Automation product ( not covered here ) there are exactly three options for connecting NSX to other networks.

• External
• Routed
• NATed

For further info see VMware Automation Fundamentals Handbook  p61

I found out I didn’t fully understand the basics of each of these network types, so lets discuss each of them further.

External Network

An External network is an existing network ( outside of NSX ) that has the same IP subnet as your NSX network.     So in the simple case we are just bridging from a vxlan network to a vLAN network.      For example:

NSX:   vxlan: 5050 ,  subnet 192.168.50.0/24

External:  vLAN 50,  submit 192.168.50.0/24

Bridging is typically done on a Logical distributed Router (LDR) with an associated  LDR control VM (edge) dedicated to the LDR.   The data-plane of the bridge is in the Kernel of the host that is running the LDR control VM.     The control VM does not handle actual traffic – it only handles the dynamic routing decisions in the control-plane.

Note that NSX VM’s on the External network will use the Internet access available there.   It could be a hardware gateway providing SNAT to the Internet and that will work for our VM’s as well.    I plan to cover bridging separately in a later article.

Routed Network

A routed network performs N/S routing  ( but not NAT ).   So if our internal IP are private (RFC 1918 ) we can certainly route them through an Edge up to a core router , and to a different lab/building ,  but we can’t actually get Internet access without NAT ( or a proxy ).

Alternatively we could assign public IP’s directly to our VM’s ,  then the Edge could provide Inbound/Outbound Internet access through it’s firewall.  However it is generally undesirable to assign Public IP this way.

A routed network could use a single Edge,  HA Edge pair.   Multiple Edges with ECMP can only be used if our ISP supports dynamic routing ( generally not the case for a lab )

NAT’ed Network

A NAT’ed network Performs  two types Network Address translation ( NAT )

• One-to-One NAT
  Each VM has internal IP and a corresponding public IP ( One pubic IP for each VM )
  Both SNAT and DNAT rules can be setup
• One-to-Many NAT
  Each VM gets SNAT to one external ‘shared’ IP
  DNAT is not possible !,   Incoming access is only available through Load-balancer.

Note that network “Port Translation” or PAT as typically done for mapping incoming requests on consumer routers is not supported.

Lab Options

There are a few options for upcoming labs.    ( To be announced )

• External:  Bridge vxlan Logical switch to a physical vLAN
  Optionally use physical router on vLAN for  SNAT/DNAT
• NAT:  Edge connected to ISP with SNAT/DNAT
  Requires a routed subnet from ISP ( with multiple static IP’s )
  We can  optionally use Edge Fail-over for reliability.
• Routed:
  1) Edge connected directly to ISP routed network.
  2) Edge connected to local EdgeRouter-X ( connected to ISP )
  which allows us to try dynamic routing protocols.
  The EdgeRouter-X is a low cost router supporting OSPF and will allow us to setup ECMP from 2 or more Edges to the ER-X.    The ER-X support both OSPF and BGP dynamic routing protocols.   Note that we would need to place public IP’s on the VM’s to test Internet access,  as we are not using NAT.