31 SDN Architecture and OpenFlow
Prof. Bhushan Trivedi
Introduction
In this module we will look at SDN architecture, how management, control, and data planes interact with each other, how northbound and southbound interfaces help them to communicate and how forwarding devices are controlled by the control plane. This module discusses the OpenFlow protocol which acts between the control plane and the data plane and helps controller managing network forwarding devices.
SDN architecture and terminology
Once we have learned about SDN, it is time to look at the complete architectural diagram of SDN depicted in figure 33.1
The first type of devices used with SDN is known as a forwarding device. The protocols like OpenFlow is used to define flow rules to take actions on incoming packets. The devices perform various forwarding routines, for example, forward incoming packets to specific output ports, or to the controller or modify the header before doing so etc.
The forwarding devices are networked with other devices in either wired or wireless fashion. The complete network of such forwarding devices is called the data plane. Thus the data plane is a collection of all forwarding devices. It is important to note that data plane is NOT the data network. The data network is what we all use, send and receive packets. Data plane helps data networks in forwarding packets in the right direction, with required quality.
Control plane contains the logic to manage the network and is situated in between forwarding devices and network applications. Control plane consists of one or more controllers.
When an SDN controller wants to communicate with upper (or higher) level, it uses a typical interface named as the northbound interface. Conversely, the southbound interface is one which allows the component to communicate with lower level component, the forwarding devices. Both interfaces are OPEN (vendor-neutral). Both are typical network layers as the southbound interface takes services from the lower part and northbound interface provides service to higher part.
The southbound protocol describes how the control plane talks to the forwarding device and northbound protocols describe how the control plane talks to the networking applications. When northbound interface connects the control plane with networking applications, the applications acts as management plane. The management plane contains applications like MAC Learning1, Routing, Firewalling, Intrusion detection, load balancing, monitoring traffic etc. These applications represent the network management and control logic that the administrators are interested in implementing.
The SDN controller provides northbound API for the applications and southbound API for the forwarding devices to communicate with it. That means, any switch vendor can design a switch to accept commands based on controller’s southbound API and that switch can work under that controller. Similarly, a network application designer can design any applications based on controller’s northbound API and it can now instruct the controller to do what they want.
1 Self-learning MAC switches have the ability to learn MAC addresses from the communication coming to it. The administrators do not need to feed MAC addresses for a given port. The switch learns this by simple logic. When the MAC address is not known, it sends it over to all ports, thus broadcasts. When a sender sends something and the switch receives it on a typical port, it understands the sender is connected to that port and enters that information in the table and won’t broadcast for that address thereafter. It also starts broadcasting if the same sender does not send data on that port for some length of time. This will help when the sender moves out of that network and join some other network.
In recent years, the SDN has broadened into other functionalities than those which are possible by decoupling data and control planes. One typical method is to use an additional abstraction over existing network. It is like an additional control layer on top of existing network with conventional switches having control and data part closely connected to each other. The network itself does not change a bit but the system which is laid on top of it will provide a virtual interface to applications. The APIs will offer a single point interface to applications and pass their commands to SDN controller, which is known as orchestration platform, acting as a broker in between applications and network devices. This approach is known as Broker SDN approach. This is useful for the case where the user wants to get SDN benefits with legacy networking infrastructure in place.
Another approach is to use overlay network to connect to SDN controller with the network it manages. The components of the network are connected by tunnels. The control plane is distributed and manage these networks which are not directly connected as if they are. This is known as Overlay SDN. When geographically distributed data centers are to be managed from a single location, such overlay SDNs are a quite attractive option.
Some standardization efforts in SDN
Open Network Foundation (ONF) has developed a protocol known as OpenFlow. OpenFlow is open source development process and contains protocols for control planes to talk to data planes. One SDN Research Group from IETF is also acting in the same field to find and address research challenges in SDN. Similarly, ITU-T also has a study group which is working on the standardization of SDN. IEEE has also started standardization process known as 802.1CF. Wired and wireless IEEE networks will have a new control interface to that standard. European Telecommunication Standards Institute also is working on Network Function Virtualization which is complementary in the sense that both SDN and NFV try to achieve better control over the network by adding programmability and software-based control.
Standards for communication
When we discuss communication between network devices, controllers, and management applications etc., the most critical part is, how do they do it? If ever the controller sends a packet to the switch which is destined for another controller, how will switch differentiate between a command for itself and this packet? Let us try to understand. If controller C sends a packet to S, it can be one of the two things, first, a specific command for S and other switches, and second, a packet for some other controller. How a switch differentiates between them? Another question is, how a switch from vendor 1 can communicate with a controller from vendor 2?
The answers to both of above questions clearly indicate that one needs some standard which defines a protocol for communication between a controller and devices it controls and controller to another controller2. There are many such attempts to design standards, one which is oldest and most used is OpenFlow. We will discuss OpenFlow in the following.
OpenFlow is an outcome of an academic experiment at Stanford University. Now, Open Network Foundation (ONF) manages it. The word open is significant. The idea is to make sure ANY switch can talk to ANY controller (and thus possible to be managed by ANY management application.) The protocol and the technology, therefore, use word OPEN. The open network foundation, which was created to manage this movement, is also using word open to indicate that the technology embraces all switches from all vendors (who would like to comply with this standard) and not confined to a specific vendor. OpenFlow is into its fifth version (1.5.1 is the latest update). Many vendors adopted OpenFlow including AT&T, Cisco, Samsung, Google, Intel, NEC, Huawei, and many others3.
The OpenFlow controller is designed to have most of the intelligence. The job of network devices is reduced as a forwarding device (if it is a switch or router) or receiving OpenFlow commands and execute them for the incoming data. For example, an IDS sensor, upon receiving a command from the controller, monitors packets with a specific pattern. A firewall, upon receiving a command from a controller, blocks packets coming from a typical IP address. Please take a note that which packet to block and how each packet is to be filtered is decided by the controller but executed by the firewall or IDS. Thus, the intelligence lies with the controller. The network devices which obeys the controller’s commands are only intermediaries and so-called forwarding devices now.
ONF is also working for NFV or network function virtualization. NFV is a concept of replacing network appliances like firewall or switches by software appliances. These software appliances run on servers and provide services from the cloud. This is a movement which standardizes how software and cloud-based network functions can be provided. One can easily understand that both SDN and NFV are quite close. You can now guess why ONF is working on both of them together. We will not be discussing NFV further in this course.
A switch which understands OpenFlow commands is called an OpenFlow switch. It can be configured to handle both types of traffic, conventional network packets, as well as protocol-related traffic (commands and their responses). An interesting capability of OpenFlow protocol is to allow generating and using exceptions. For example, if the conventional switch-like functionality suggests that a packet for destination IP address 1.2.3.4 is to be forwarded to 2.3.4.5, OpenFlow can indicate that if the sending IP address is 4.3.2.1, irrespective of the destination IP address (even if it is 1.2.3.4), the packet should be
2 One also needs a protocol for the management applications to communicate with SDN Controllers, which is usually managed by typical API provided by controller so we will not be discussing that.
3 The members of Open Network Foundation include 21 operators, 110+ innovators 70+ collaborators and 100s of volunteers as on 16th Feb 2017: source ONF website forwarded to 3.4.5.6 (and not 2.3.4.5). This is called an exception. For all network traffic to 1.2.3.4, the normal router is preferred but for this exceptional case, a separate route, as indicated by the OpenFlow controller, is chosen.
Scope of OpenFlow
The OpenFlow protocol has three main jobs.
1. It defines the exact format of messages that switch and controller used to communicate with each other.
2. It defines what exactly a controller can configure in a switch, list of items and tasks.
3. It defines all communication components between the switch and a controller. What can be communicated and what are possible responses to the commands etc. are defined by OpenFlow, a common element for any protocol.
The exact format of the message and the fields are not possible to be specified completely but we will discuss them in brief soon.
Basically, OpenFlow switch concentrates on two things, first, packet classification, and second, forwarding. A table that implements classification and forwarding logic is popularly known as flow table. The packet classification process is a basic rule of type LHS -> RHS. That means if LHS is true, RHS is going to be executed. An LHS is basically a pattern which may or may not is true for a given packet. The RHS is basically an action to be performed. The pattern matching part is usually implemented using Ternary CAM we studied during packet classification but OpenFlow allows the vendor to choose some other alternative if he prefers so. The RHS is popularly known as an Action. There are many actions defined (including forwarding, dropping or modifying a packet). Additionally, most administrators are also interested in learning about the frequency of the event (how many times you get a packet with a positive virus signature? Or How many packets routed on path A vs path B4). The flow tables also contain that statistics5.
As it is the case that the controller may not be directly connected with a switch physically but the command must reach the switch, OpenFlow allows a TCP connection from the controller to the switch for sending commands. It is recommended to use SSL or TLS over TCP (Secure Socket Layer and its later version Transport Layer Security provides authentication and encryption of the connection and the receiver can verify two things, who the sender is, and the content is the same as what is sent and additionally guaranteed that no intermediary has read the content) instead to provide a secured option.
4 This is a typical SDN statistics, otherwise impossible to get from conventional network switches as they cannot have information about packets flowing over one path vs. packets passing over another path.
5 It also stores information about the length of that packet and time
Matching
One of the critical parts of OpenFlow processing is the matching of an incoming packet with some rule. The fields of the incoming packets are used for matching. The OpenFlow 1.4 specification allows 41 fields of a packet for classification. It also allows some arbitrary portion of the packet to match. Interestingly that includes MPLS labels, traffic classes, and BoS (Bottom of Stack) bits. That means OpenFlow can work with existing MPLS network and use those fields for classification. So if one has deployed MPLS and would like to switch over to OpenFlow, it is a straight forward process. Matching rules that he has designed for specific MPLS packets can work with this OpenFlow rules with MPLS tags or traffic class values of BoS value as the case may be.
Ethernet is also standardized to provide backbone architecture. Whenever a service provider provides network services, he also needs to connect networks through a backbone. When Ethernet is used as a backbone, IEEE 802.1ah protocol is used for communicating over that backplane. The service provides use that protocol to label their customers’ traffic and route them in their network using their priorities based on subscription classes.
Let us take an example to understand. Consider a service provider has 3 subscribers, Net-1, Net-2, and Net-3. Traffic of all these three customers is passing through the service provider’s own network. The traffic of each subscriber is labeled as L-Net-1, L-Net-2, and L-Net-3 for example. The service provider’s network switches and routers route the packets based on these labels. It might provide first priority to L-Net-2 by providing a separate queue for the packets bearing that label and only when that queue is empty, packets from other queues are forwarded. 802.1ah is used for this job. Such backbone networks are popularly known as PBB (Provider Backbone Bridges). OpenFlow can also work with IEEE 802.1ah protocol values. That means, the SDN can work with PBB headers. Providers’ networks are separated and routed using 802.1ah protocol while using Ethernet as a backbone technology. This enables the ISP’s traffic to be labeled using 802.1ah and routed accordingly. That means if the provider uses 802.1ah, he can switch over to OpenFlow without much of a trouble. Figure 33.2 provides an example.
There are fields which describe fields of IPv4 or IPv6 packet headers, for example, port number, source, and destination address etc. One more noteworthy point is; it can classify packets based on Ethernet Type field as well. One can use unallocated type values for typical Ethernet packets to have specialized routing for them. For example, a typical real-time VOIP call is given a specific Ethernet Type value. It is possible to instruct an OpenFlow switch to route that traffic through a specific route (usually with minimum hops). Another field of interest is tunnel id, so OpenFlow can also be used in conjunction with virtual network tunnels. Based on typical tunnel id values, the traffic is forwarded or processed. An earlier example of having a network, entirely of accountants separated from others, using specific tunnel ids and routing them to the central office server irrespective of how other packets are being routed, can be easily implemented using OpenFlow and use this (tunnel id) field for routing.
Once the matching gets over, the next thing is action. If the match is found, that means the LHS value is true, an action specified in the RHS is executed.
Three basic actions on the packet are defined. First, is forwarding the packet to a typical port or a set of multiple ports. Second is to encapsulate the packet and send it to a typical controller for processing and third is to drop the packet without doing anything else. Let us delve deeper to understand.
The first basic action is a default action for any switch so OpenFlow, when start controlling the switch, needs only to decide exceptions (when the switch does not forward in a normal way). Sending it to multiple ports help to manage multicast or broadcast transmissions.
The second action is a case where there is no match with the flow table entry. In that case, it is possible to have an external controller to specify some default action, so what we do is, we forward that packet, in an encapsulated form (as it is not to be processed by any switch along the path), to that specific controller. The controller decides the path for this particular connection, decide the classification, send commands to configure that classification in this switch, send that packet back to the switch for further processing. In fact, this action also helps in the process of learning by the switch. For example, if a switch receives a TCP request for some destination. The packet (connection request packet) is sent to the external controller. Now the flow table has the entry for this packet and so the packet is forwarded on the specified route. All subsequent packets of the same connection will be forwarded by the switch following the same classification rule.
The third action is required when there is some problem. For example, when the source is compromised and sending packets containing malware, or some source started malfunctioning or application has some problem which resulted into unnecessary forwarding packets etc. In such cases, the controller instructs this ‘drop such packets’ action to the switch (or switches). Any packet coming from that source (or a typical connection from that source) can be dropped using this action.
15 different types of actions, other than the default, are possible. Sending output to a specific switch port is defined as 0th action. 1 to 10 actions is currently undefined. Action 11 and 12 copy time to live values from one layer to another. One such case is, time to live from IP header is copied to OpenFlow header so it can take action when it becomes zero. Another case is copying it from MPLS header. So if an MPLS packet is being forwarded, OpenFlow header TTL value is copied from that MPLS header. Actions 15 and 16 are related to MPLS time to live value, 17 and 18 are used for pushing and popping MPLS tags.
Action 21 can specify a typical queue when the packets are scheduled to exit from a port. We have already hinted about separate queues for different types of the traffic before. This action aims to implement it. When multiple queues are possible to attach to a port, priority-based queueing is possible and thus one type of traffic (for example VOIP), is queued in queue-1 and another type of traffic (for example, the rest of the traffic) is queued in another queue, queue-2. Now it is possible for the port to see that if there is any item in queue-1, it is sent to the other queue, thus giving higher priority to queue-1.
Action 22 is about groups, 23 and 24 deals with IP time to live values. A typical header field setting is done by 25, 26 and 27 are adding and removing tags of PBB. We will not elaborate their working further here. One can refer to the ONF website for detailed description of each of these actions.
Flow and Group Tables in switch
A flow table contains LHS-> RHS types of rules. LHS is a matching process, involving the fields that we have seen already. The RHS is one of the basic actions or other actions described above. A switch has an architecture described as in figure 35.3.
The switch can have multiple flow tables; each table can have multiple entries. The process always begins with the first flow table. If the entry matches, one of the possible action is to jump to some other table in forwarding direction. No jump back is allowed and thus there is no loop possible. The tables are processed in a sequential fashion as per jumps decided by the content of the packet and the matching rules. That mechanism is known as pipeline processing and the set of flow tables associated with it are known to be linked with each other. There is a concept of group table as well. Thus a flow table entry might point to a group table. A group table is a special table which contains common actions which can be applied to multiple flows. Thus the actions which need to be applied to a group of flows are listed in this table. One more point is worth mentioning. What if the switch has to handle both, normal network traffic, and control traffic together? One way to handle them is to have two different VLANs, first for the normal (production) network traffic and other for controller (OpenFlow) traffic. It is also possible to configure action which instructs the switch to forward the packet based on rules for those networks.
Interestingly, a later version of OpenFlow allows OXM (OpenFlow Extensible Match) that we have hinted while discussing the packet classification process. It can match an arbitrary bit pattern in the header or even payload. A mask value and length is provided in TLV (Type, length, value) format. The switch matches that OXM entry with a pattern can take any action like it does for a specific field.
Pipelining
As we discussed earlier, the processing of a packet entering the switch is based on a chain of flow tables. The process which links these flow tables is known is called pipelining.
When the packet enters the switch, the first table in the chain is looked for a match of a flow entry within that table. The packet is processed according to that flow entry. If the entry points to another table (it is like a linked list), the packet is sent to that table. The packet is again processed by this table and forwarded to another table if the entry indicates so. As we have seen before, the next table may be a group table.
Flow tables can also be synchronized. When they are synchronized, upon addition, modification or deletion of one entry in one table, the entries in the synchronized table is also changed accordingly. When the entries are synchronized, the actions and counters are not. Thus it is possible for a controller to affect multiple switches to sync on a specific entry but still inflict different actions based on the match. This synchronization is either single or bi-directional depending on the requirement. One such example of a flow table which matches with Ethernet source address and associated Ethernet port number. Another table which matches with Ethernet Destination Address and associated Ethernet port number. These set of synchronized table help the switches to learn about Ethernet addresses to be associated with a specific port from where the packet arrived6. If the tables are synchronized, whenever a sender sends a packet which arrives at this switch, the entry is made in the first flow table containing source addresses and associated networks. The second table, which is synchronized with this table, automatically gets the entry if it is not present, so the destination address is automatically added. Even when the source has changed its network and the second flow table has a stale entry, whenever that source sends a packet, and the first flow table gets an entry, the second flow table entry is automatically corrected. The flow table entries are purged after some time and thus won’t have stale entries usually.
When the tables are linked, it is sometimes needed to pass information from one table to another. An additional field, called metadata, is also used in a pipelining process. This metadata is an additional information for the next table. For example, when the OXM match is done, it can have a metadata field indicating the switch port on which the packet has arrived. When a logical port is used instead of a physical port, metadata indicates tunnel id which is represented by that logical port. If metadata match is not found, the table does not do anything with the metadata and pass the same to next table. Another example, at one instance working with a table, the next hop is determined (where to send the packet next is learned), but there are few more tables in the link are still to be processed, this information is passed as metadata.
6 When a self-learning switch receives a packet which is to be forwarded, whose source address is not found in the configuration table, it associates that packet and network that the packet is coming from to that port. This process is known as reverse path forwarding and is quite popular in self-learning switches.
This process continues till the flow entry does not point to next table. The actions that the flow tables matching entries indicate are not immediately executed. All such actions are accumulated. That means if packet matches with five different entries of five different flow tables, associated actions are accumulated while packet passing through all five tables. Each table may add or remove actions from that accumulated set. Thus when packet proceeds through the pipeline, a typical set of actions is accumulated. When the pipeline reaches to a stage where there is no jump to next table, it is time for all actions of this set to execute. When OpenFlow executes this set of actions, one of them must be able to inform OpenFlow where to forward this packet; i.e. specify the port (logical or physical) to send the packet over. That action is known as OUTPUT action.
Out of accumulated set of actions, some of them are compulsory and many are optional. It is possible for the flow to be processed based on TTL values. When local processing is required, OpenFlow can also pass the packet to local protocol stack like packet classification process. MPLS label addition, removal, and processing all are covered by actions. We have also seen that even priority queue related actions are also supported.
The group tables can provide a single entry for a group. For example, indicating the set of ports as a single entity for forwarding packets. Four different types of groups are possible. The group of all ports is used for multicasting and flooding. Select is another type. Here one of the members of the group (it selects one of the ports to belong to the group) is chosen. Some processing is done to choose the member. For example, a packet header is hashed to achieve at some value which can be used to choose the port in near random fashion. This process helps chooses any one of the possible ports randomly. What is the advantage? It offers automatic load balancing between those ports. If multiple ports offer paths to the same destination, packets belong to one connection traveling over one and another connection traveling over another avoid the situation where one port is loaded and its queue is full while another port is empty. This is also known as multipath routing. We have already seen that algorithms like LS and DV do not support multipath routing. The third type of group is known as an Indirect group which is used for indirection. Multiple flow entries forward to a single id, like IP forwarding where multiple entries are aggregated to one of the interfaces. The fourth type, fast failover, executes the first live port of group. When a switch fails, whoever another port in this group is live (that means running), is forwarded this packet. This helps in recovering from a switch or line failure. The switch changes to next available path without even consulting to the controller. Switches, in this case, must be capable of testing the liveliness of the ports and recording them. For example, if a switch finds that a typical port is not responding to commands, it might conclude that the port is not live. Whenever there is a failure, the switch looks at this group and check the liveliness of each member, and start routing the packets over next live member (a running port). This process is called fast, as otherwise the controller has to be informed and the controller has to calculate an alternate route and reconfigure the switch which might take much longer time. However, the controller can always control this process by providing the set of ports it wants to be used as alternate as a failover group.
One important idea is to have a set of modification for the switch in a form of a message known as a bundle. The idea is quite similar to transactions in databases. The switch retains all modifications ONLY IF all of them succeed and not otherwise.
OpenFlow Header
The OpenFlow header has four fields. First, indicates the version of the implemented OpenFlow protocol. One of the critical version is an experimental version. Type is a constant value indicating the type of message. Messages can be of varieties of types. Error messages indicate errors. Total Length is the length of the message including this header and Xid is called transaction id. This field is used to match the request with a response. Whenever a controller or a switch has sent a command with Xid value as 10, for example, and receives a response with Xid value 10, they understand that the response is the same command. .
An OpenFlow channel connects each OpenFlow switch to a controller. Using this interface, the messages are passed. The controller sends messages to conjure and manage switch (set typical flow for a typical case, instruct to add or remove a typical MPLS tag, asking for the typical set of features a switch might support etc.). Controller also receives packets related to events from switch (for example, port not working, new TCP connection is being established so flow is to be defined, a packet does not match with any entries etc.)
There are three different types of OpenFlow messages, controller-to-switch, asynchronous and asymmetric. All of them have the same header as specified above. Controller to switch messages are initiated by the controller. Asynchronous messages are initiated by the switch. Symmetric messages are sent by either of them without solicitation. Hello is one such message where one party is trying to see if the party at the other end of the port is alive. Echo is another such message. Symmetric messages can also help capability exchange where both sides learn about capabilities of another side (for example, handling messages of larger than normal sizes) and optimize their processing accordingly.
Software Defined Radios
The word software in SDN indicates a critical paradigm shift. The parameters of the switch which could only be set using proprietary hardware configuration are now possible to be done using the software. It is a software driven networking solution. The SDN has also inspired the wireless world to some extent. The wireless networking devices which communicate in a wireless manner, use some radio devices to send and receive the message. The radio devices use many built-in parameters which are hard coded in the device, for example, frequency to send and receive and the modulation type (QPSK or QAM-16 or QAM-256 etc.) When we use SDR (software defined radios), these things can be specified using software, even when the radio is on and working. The advantage is, the radio device can change its frequency and modulation type on the go. The configuration of the device can be changed easily, and the device can even work with multiple frequencies at the same point in time. It is also possible for the radio device to hop from one frequency to another when needed. When two different radio devices, communicating with each other, hop from one frequency to another, makes it hard for an attacker to either eavesdrop or block the transmission. Both communicating parties can also find out frequencies not used by other communicators in the vicinity and switch over to that frequency. They can choose the frequency to communicate with least interference and maximum throughput.
What is the advantage of software configurable radio device? One is mass production of the general device which can be set as per requirement for a specific product. For example, Remote controls for different types of toys does not need to be designed separately. They all can be designed using same SDR chips and at the time of installation, the vendor may fix those values needed.
It is also possible to write an application which runs on top of that radio chip and set the parameters according to the need of the hour. If the application can sense the frequencies being used in the surrounding, it can find the unused frequency and start using it. It can also assess if some other users start using the frequency it is currently using and again switch over to some other currently unused frequency. Such a design enables the whitespace movement. The whitespace movement is about abolishing all licensing and allow anybody to use any frequency which is currently free. One must pay as per the time he occupies a typical frequency or amount of data it sent over that frequency to the owner of that frequency.
Summary
In this module, we looked at the complete SDN architecture and described each component. We have also looked at OpenFlow architecture used for communication between controllers and switches. We have looked at OpenFlow rules, matches and actions. We have seen how flow tables and the group takes are organized and used. The pipeline processing, matching entries of each of the flow tables and generation of action list is also discussed. We have seen the OpenFlow header design and introduced SDR at the end of the module.
| you can view video on SDN Architecture and OpenFlow | |
References
1. Computer Networks by Bhushan Trivedi, Oxford University Press
2. Data Communication and Networking, Bhushan Trivedi, Oxford University Press
3. Internetworking with TCP/IP, Douglas Comer, Pearson
4. Software-Defined Networking: A Comprehensive Survey Diego Kreutz,Member,IEEE,Fernando M.