Michael Haugh, chair of the ONF Testing and Interoperability Working Group, provides a recap of the Fall ONF PlugFest hosted by Spirent.
Last week we conducted the semi-annual Open Networking Foundation (ONF) PlugFest, hosted by member company Spirent in Sunnyvale, California with support from Indiana University’s InCNTRE lab. As part of the charter of the Testing and Interoperability Working Group, we are looking to foster interoperability of ONF standards through test events called “PlugFests” that are open to all ONF member companies and academic associates. We have now completed four of these events – the first was held in March 2012 at Ixia, and the last two in October 2012 and June 2013 were hosted by InCNTRE. The format of previous events was to build a testbed of OpenFlow® switches, then provide each controller and application time for testing. Although this offered the ability to identify many issues, we needed to change things up to improve productivity, since only one controller can use the testbed (without slicing the network). We moved to a “speed dating” format for the event last week, where each controller and application could test in an isolated testbed with an individual switch for a set period of time. This improved the engagement of all participants, reduced troubleshooting, and provided excellent feedback to both the controller and the switch in each test. We moved to small topologies after the one-on-one testing, providing each controller and application with a network of three to five switches for testing.
We chose to focus on OpenFlow® v1.3 and OF-Config v.1.1.1 at this event. The applications tested were primarily L2/3 centric including L2 (MAC) learning, L3 (ARP) learning, L2 provisioned paths (both with MPLS and VLAN), ECMP/load balancing, path failover, and others. Fifteen ONF member companies and two of our three lab partners participated: Alcatel-Lucent, Beijing Internet Institute (BII), Ciena, Freescale, HP, IBM, InCNTRE, Infoblox, Ixia, Luxoft, NEC, Netronome, NoviFlow, NTT Communications, Spirent Communications, Tail-f Systems, and VMware.
Throughout the event, we would break each morning and at lunch to review test progress and have a technical discussion on “The Good, The Bad, and The Ugly.” Here is a high level summary of our discussions:
The Good – In general, there were very few issues with basic features like the OpenFlow® channel, stat request and reply, single-table processing, LLDP-based topology discovery, and basic match and action features. Controller maturity was much better, with sophisticated GUI interfaces and more robust application support, and testing maturity was also impressive, with two emulated controllers as well as the first OpenFlow® v1.3 switch emulation to test controllers. There was decent progress with conformance test frameworks, of which two were able to run early OpenFlow® v1.3 suites. We also had a chance to run some performance testing with some of the vendors, including one test going up to 1.1 million flow table entries on a software-based switch. We tested data-plane convergence as well as forwarding performance for most switches. On the controller side, we performed scale testing going up to 1,000 emulated switches. These performance metrics will help to move SDN and OpenFlow® toward being carrier grade for scale and reliability. In addition to OpenFlow, we were also able to test OF-Config v1.1.1 using one vendor as the framework, tested against one open-source switch and one commercial switch.
The Bad – There were a number of issues identified in both controller and switch implementations. At a conservative estimate of at least five issues per vendor, there were at least 75 issues identified at this event. These issues were bugs as well as implementation issues and limitations. Most switches could not support all match and actions, and many had issues with prerequisite checks. These implementation limitations sometimes led to improper error handling by the controller. When errors occurred on the switch or controller, we frequently observed behavior that did not gracefully handle the error condition. On the positive side, many vendors were fixing issues in real time and, in some cases, adding features.
The Ugly – The main issue we had with OpenFlow® v1.3 was with multi-table support and pipelining. Many vendors implemented in slightly different ways, causing many challenges for the controllers. The table capabilities in some cases were determined through table feature request and reply messages, and in some instances, this could not be handled on the fly – the controllers would need to implement fixes to work with each switch. This allowed them to progress in the one-on-one testing, but when it came to the small topologies, the controllers normally could not implement support for different implementations on each switch. We were able to progress and get good results out of the small topologies, but in most cases this meant pairing switches and controllers that worked well together.
To address these challenges, the main fix will be the implementation and adoption of Table Typing Patterns (TTPs) which the Forwarding Abstractions Working Group (FAWG) has defined. Curt Beckmann, chair of the FAWG, provided a lunch tutorial on TTPs at the event. We had a good discussion on how they would help address the issue of run-time negotiation and instead allow the switch and controller to agree on a pre-defined set of match or action and table pipeline processes.
In summary, the event was very successful. Many of the participants completed months of quality assurance testing in just a few days. We have made great progress on OpenFlow® v1.3 implementations, and the progression of TTPs will help address all the variables when it comes to controller and switch compatibility for multi-table processing. At the next PlugFest, expected to return to the InCNTRE lab in May 2014, we hope to see even more ONF members participating. Our Working Group will be producing a technical paper as feedback to the ONF Extensibility Working Group to address issues found during testing, and as OF-Config v1.2 is finalized, we expect to also see implementations which will support TTPs. We are also working to extend the ONF certification program to add OpenFlow® v1.3 switch certification in the first half of next year, so make sure that your company is planning to be certified, or encourage your suppliers to be certified. And don’t forget to participate in the next ONF PlugFest event!
- Michael Haugh, senior manager of market development at Ixia and chair of the ONF Testing and Interoperability Working Group
