Part II is a continuation of the previous post on SmartFabric Services. We will start with a discussion of Fabric Operations.

Operations

To understand SmartFabric Services, we will have a look below at both fabric formation as well as event handling.

Part 1: Fabric Formation

As SmartFabric Services matures over time, it will steadily add Solutions and associated automation profiles to its repertoire. Let us have a quick review of the formation of a fabric governed by SmartFabric Services.

Automation Profiles:

The Automation profiles (e.g. VxRail, Isilon etc.) are what enable the participating nodes, to align/converge as a fabric.  Once enabled on all switching nodes which are to be a part of the fabric, the groundwork is laid for fabric formation.

Infrastructure VLAN:

Following the activation of an automation profile (discussed under Operations), an “Infrastructure VLAN” is provisioned to facilitate communication between participating nodes. The VLAN ID is configurable, set by default to VLAN 1.  The requisite is for it to be identical on all nodes.

This VLAN is the used for Layer 2 Multicast Keep Alive messages, which are exchanged between the nodes – every 2 seconds by default. Cluster formation is based on IPv6 address.

Since the base requirement is only IP reachability between the nodes in fabric, it means that it is possible to have fragmented islands of this VLAN, connected via Layer 3 i.e. This VLAN is tunneled between two racks or even DCs, which have L3 connectivity in between. This opens further interesting possibilities around stretched fabric between racks/DCs, once we get the Generic LAN fabric available as a Solution.

Fabric ID

In addition to the infrastructure VLAN, which is the fabric communication channel, the fabric itself has an identifier, known as Fabric ID. Once again, this is user configurable. The intent is to enable multiple clusters if required, in parallel yet distinct.  The Fabric ID is what determines which automation profile or fabric instance, the particular node belongs to.

Fabric Master

The Fabric at this point, proceeds to elect a Master. However, note that this is not the same as a Virtual Chassis/Stack Master, nor does it have any connection with Master/Local Switching.

This assignment of a “Master”, though not entirely cosmetic, is still fairly low impact/emphasize. The Master is a master because it hosts the SmartFabric Service Master Module, which interfaces with the External Fabric Manger via the Rest provider it contains. Consequently, it anchors the API endpoint for the fabric itself, behaving as a proxy for External communication with the fabric. In case of failure, another node steps forward and assumes Master responsibility.

• Master “Failure” is decreed when 3 consecutive Keep Alive messages are missed from it.
• The criteria of election are the same as used for VRRP Master election.
• The election does not impact production traffic in any way.
• If a failed Master re-joins, there is no “Pre-empt”. The ownership stays with the existing Master.

Database Sync

Database information exchange or sync uses Point-to-Point Stunnels, between each switch & the master. It happens through the publish/subscribe mechanism. The subscription target is the Group CPS – Distributed Database held on the Master. The Fabric Agent module on each node is the initiator. The goal is to sync the local database on each node with the database held on the Master, so each has an identical copy.

Host Detection via LLDP

After the Activation of Automation Profile, the “trigger” which the nodes now await, is the detection of “Qualifying” end hosts. This detection is enabled via LLDP, i.e. the switches use LLDP to detect attached hosts that fit the activated profiles. As soon as that happens, the automatic assignment starts to execute [for the activated automation profile].

Assuming a Leaf/Spine fabric, a high-level walk through of a fabric coming to life, could be as follows

• Desired Automation Profile, activated.
• End Node sends LLDP, which is received by a Leaf switch.
• Leaf deduces it is leaf because it is receiving an LLDP packet from an end node
• Leaf then announces itself on all connections as Leaf. A leaf only connects to spines.
• When spine receives an LLDP from leaf, it deduces it must be a spine to be receiving an LLDP frame from a leaf.
• The spine sends out its own LLDP on its links.
• Any leaf switches receiving the frame deduce that since they are receiving an LLDP from spine, they must be leaf switches.
• All fabric switches, talk to each other. Multicast Keep Alive frames within VLAN 1, are seen by all. Each switch knows the topology, and availability of the other switches.
• Master election criteria. Same as VRRP: higher priority, then higher IP address.

Connectivity outside the Fabric

Full CLI/command set is still available on every switch. SmartFabric Services is only concerned with the particular Automation profile. Outside of the profile, the switches continue to execute their control plane protocols. SmartFabric Services is not aware of switch specific configuration.

The nodes can connect to switches outside the fabric as par usual, and make use of their traditional complement of control plane protocols.

Part 2: Event/Request Handling

We will have a look now at how the fabric handles requests, and the resulting events.

1. External Fabric Manager passes Configuration Change Request to SmartFabric Master, via Rest provider (Accessed via its Virtual IP).
2. The request passes down from the Rest provider, to the appropriate configuration module within the SmartFabric Master block, which anchors task specific modules for the fabric itself.
3. The configuration module handling the request, publishes the required change/update to the Master’s Group CPS – Distributed Database (Using publish/subscribe mechanism).
4. All fabric agents – local & foreign – subscribe to the Group Control Plane Service on the Master. When changes are made to Master Distributed Database, Control Plane Service informs all nodes of the change, via CPS Events Service. Its contexts include local node, specific node, and Group.
5. The fabric agents pick up the changes, over Point-to-Point STunnels, and pass these to their local CPS.
6. If action applies to that switch, it’s local CPS implements the actions on that node. if it does not apply, only the Database is updated.

(Publish/Subscribe):

• When configuration changes, Master updates its OWN copy of Distributed Database.
• CPS informs all nodes of the change via CPS Events Service.
• The nodes then pull the changes over P2P secure tunnels, and update their own copies of distributed DB.

Future:

As SmartFabric Services steadily expends the breadth of its supported solutions, we can look forward to Automation profiles for VMWare (ESX – VM Tracing, vMotion Port Profile Migration etc.), NSX VTEP, Nuage as well as Generic LAN Fabric. The near-term enablement comprises profiles for solutions like VxRail, VxRack, Isilon etc. I might do further posts on SmartFabric Services in the future, to cover the enhancements.