Big Switch – Cloud Fabric: External (Outside the PoD) Connectivity Options – Part 2

This post is a continuation of Part 1.

Layer 3 Connectivity out of Fabric:

Layer 3 handoff to external peers outside the fabric is the ideal option with BCF. At a high level, there are 3 options of achieving this:

1. Via Dedicated External Tenant.
   • All internal Tenants connect to the system Tenant (for inter-tenant & External Routing).
   • System Tenant connects to a dedicated “External” tenant which interfaces with external routers.
     • System tenant will have a default route to external tenant.
     • External tenant will have a default route to external routers, and subnet routes to system tenant.
2. Via Dedicated External Segment (Edge/Transit VLAN) WITHIN each tenant
   • A Tenant will have a dedicated “Edge/Transit” vlan/segment contained within, which interfaces with external routers.
     • This edge VLAN will have a local ip address and remote ip address. If applied to a remote  port-channel, you can have 2 remote IPs and ECMP.
     • The tenant logical router will have a default route using next hop address(es) at the remote end of Edge/Transit VLAN.
3. Via Dedicated Gateway, within Segments.

BCF makes use of Next-Hop groups to facilitate ECMP (Equal cost Multi Path) for PBR/Static Routes. Upto 16 next hops can be supported, & traffic is load balanced across all the configured paths.

Let us look at the aforementioned options in a little more detail:

Dedicated External Tenant

This option is depicted below:

 

In this example

• The Internal Tenant Red comprises two VLANs
• The internal VLANs (Red 10,Red 20) have respective ranges 10.0.10.0/24 and 10.0.20.0/24
• Tenant Red has a default route with next hop to system tenant Router
• Tenant System, in turn, has a default route with next hop assigned to BCF-Edge Tenant Router.
• BCF-Edge Tenant is depicted as using two transit VLANs to connect to Core, using ECMP. Other configurations including the use of a Port-Channel with the two links as members, are possible.
• Transit VLANs could for e.g. be built as 192.168.0.0/30 and 192.168.0.4/30. fabric VLAN 101 could own 192.168.0.1/30 local, with corresponding 192.168.0.2/30 as Remote end. Fabric VLAN 102 could own 192.168.0.5/30, with Remote end as 192.168.0.6/30.
• ECMP will assign the two remote next hops to a next hop group, with automatic load balancing.
• Core Routers will have (if using Staic Routes) routes pointing to subnets within BCF fabric, e.g. 10.0.10.0/24 via next hop interfaces of BCF-Edge VLAN 101 & 102.
• BCF-Edge router will have subnet routes for the internal VLANs 10 and 20, via next hop of System Router. e.g. 10.0.10.0/24 & 10.0.20.0/24 via next hop (tenant) system.
• System Tenant, which is used for Inter-Tenant Routing, witll have Tenant interfaces for both Tenant Red and Tenant BCF-Edge.

 

Dedicated External Segment contained within the Tenant. 

The second option does not use an Edge/External-facing Tenant for peering with the Core/Our-of-Fabric Network. instead, it creates a VLAN (segment) within the tenant that interfaces with the Core/External routers.

For comparison with a traditional NOS,

• Within a VRF, you have an “Edge” Transit VLAN.
• The VLAN is applied to a port-channel (there are other ways of configuring/achieving this as well)
• The local interface for the VLAN is the local L3 endpoint (ip address applied to vlan int)
• the port-channel termination at the remote end has Two L3 addresses assigned to the physical ports (ip address applied to physical ports)

For e.g. the config on a DNOS (Dell Networking OS) 9 CLI would look something like:

Interface vlan 101

Name “Edge Transit” VLAN

Ip address 10.3.1.1 /24

Untagged port-channel 10

Int port-channel 10

Description “port-channel to External Router”

channel-member TenGigE 1/20

channel-member TenGigE 2/20            ##Assuming VLT and port-channel split between two switches##

No shut

The option for using a dedicated external tenant for services such as External Connectivity, Firewall and inspection etc. is the cleaner, more scale-able option. The internal Tenants and VLANs can expand and contract, grow and change while having a linear handover of traffic (via System Router) to the dedicated services Tenant.

Note that Tenant logical router supports BGP & Static, while System Logical router supports only static (but can redistribute). System Router can export routes between User Tenants, whose respective routers will import the routes.

Automatic Import/Export of Tenant Routes between User/System Tenant, and External Routers, is possible. By default, the system tenant router will accept routes from internal tenant routers to which it is connected. However, it will not propagate imported routes between tenants. (So you can export to External Tenant). this can be altered so:

1. Internal Tenant => static route to system
2. System Tenant => Export-Route to External Tenant
3. External Tenant => Import-route from system
4. External Tenant => Redistribute-connected to BGP and outside.