A fresh look at the Open Infrastructure Foundation
February 25 2021
by Agatha Poon
Announced last October, the rebranding of the OpenStack Foundation to the Open Infrastructure Foundation (Openinfra Foundation) went into effect on January 1. The board of directors, which is tasked with providing strategic and financial oversight over the foundation's resources, has expanded to include representatives from newly added Platinum member Facebook and existing Platinum members Ant Group and Red Hat. Having more than 110,000 community members and 710 companies spanning 187 countries, OpenStack remains one of the top three most active open source projects with 15 million cores in production, the Openinfra Foundation says. Other open infrastructure projects with varying degrees of maturity include Airship, Kata Containers, Magma, StarlingX, Zuul and OpenInfra Labs. The foundation envisages a technology environment that is not only intrinsically open, but also diversified in terms of workload deployments – from the core to the edge.
The 451 Take
The idea of openness takes center stage with an increasing number of open source projects currently running in production environments. Early adopters such as CERN and Walmart are textbook examples of companies embracing open source in a meaningful way with hundreds of thousands of cores in production. Additionally, there is a growing interest in the use of open source technologies and tools for specific use cases among key vertical segments: telecommunications (AT&T and Verizon), finance (ANT Group and Société Générale), automotive (BMW and Volvo), retail (Walmart) and technology (IBM, Red Hat, Intel and Baidu). Accelerated infrastructure development is likely in the spirit of technology innovation. Looking into the future, deployment scenarios are going to be more complex and diversified than today. To sustain a thriving community, the foundation will have a bigger role to play in resource prioritization and skills development.
A year in review
It has been several years now since the foundation first extended its support to a handful of open source projects designed for use cases in the areas of CI/CD, container infrastructure, edge computing, and public, private and hybrid clouds. While projects like Kata Containers and Zuul have already made several releases in a year, OpenStack remains one of the top three (the others are Linux kernel and Chromium) most active open source projects in production. Accordingly, CERN's OpenStack footprint has exceeded 300,000 cores, with 7,000 nodes using OpenStack Ironic. Another superuser is Walmart, which has gone from 100,000 cores in 2015 to now 875,000 cores of OpenStack to power its e-commerce site. Meanwhile, Société Générale, an investment bank based in Paris, is deploying cloud services across multiple availability zones using OpenStack.
Early movers such as Dell, Huawei and Red Hat remain strong advocates for the commercial use of OpenStack with new features and enhanced functionality. Nevertheless, a new breed of technology vendors and enterprise developers emerges when open-source-based infrastructure projects evolve beyond OpenStack. There are use cases provided by Ant Group (workload isolation using Kata Containers), Facebook (initially developed Magma for automated network management and then released it under an open source license), Wind River (edge infrastructure enablement via StarlingX), and Volvo (autonomous driving and thermal control with Zuul). For Ant Group, an affiliate company of Alibaba Group, its active participation in the OpenInfra user community has created a ripple effect by which the engineers at Alibaba Group have expressed interest in working with the Openinfra community, although conversations are still in the early stages.
The rebranded Openinfra Foundation now has a total of 27 board members from 22 technology companies, including AT&T, City Network, Dell Technologies, FiberHome and Tencent. It is also supported by 60 corporate members, including nine platinum members, 18 gold members and 33 silver members.
OpenStack is in its 22nd iteration with the release of Victoria in October 2020. Having more than 20,000 codes changes by nearly 800 developers spanning 45 countries, OpenStack Victoria is touted to adapt to the needs of infrastructure providers by enabling better native integration with Kubernetes workloads, extending support for diverse architectures and standards such as direct programming of FPGAs, and providing enhanced networking capabilities. From a user perspective, OpenStack Ironic is well used by developers seeking to deploy bare-metal infrastructures.
Airship, led by AT&T, is to enable operators to provision and manage the lifecycle of the open infrastructure tools and underlying hardware. Airship version 2, which is scheduled to be available by Q1, will provide support for OpenStack Ironic. Additionally, Airship has a modular structure where users can pick and choose the components to be integrated into the infrastructure. Using Kubernetes Cluster API, it can deploy Kubernetes clusters across a host of public clouds, in addition to OpenStack clusters and bare-metal provisioning. Airship has been deployed in production environments by the likes of AT&T and SK Telecom for 5G use cases. There is a growing ecosystem of industry participants, including Dell EMC, Insight Global, Accenture, T&T, Ericsson and Mirantis.
StarlingX, which has been approved by the OpenInfra Foundation as a top-level project since June 2020, continues to garner interest from telecom operators, technology providers, open source startups and individual developers. Now in version 4.0, StarlingX has added new features and capabilities – providing support for Kata Containers as a container runtime, TSN support in Kata Containers and FPGA support for Kubernetes, just to name a few. It has also added support for OpenStack Ussuri and Kubernetes v1.18. Positioned as a cloud orchestration platform optimized for edge computing and IoT use cases, StarlingX has been deployed in production environments by T-Systems and Verizon for edge use cases. Following a cycle-trailing model relative to the OpenStack release cycle, StarlingX version 5.0 is slated for release by mid-year.
Zuul, a CI/CD platform, is widely supported both by technology providers such as Red Hat and Huawei and enterprise developers like BMW and Volvo. In 2020, the Zuul project was backed by nearly 100 contributors with more than 1,300 changes. Some of the new features include the integration with GitLab (a web-based DevOps lifecycle tool) as a code review platform, the addition of a Nodepool driver to provision test resources and an authenticated admin REST API. Zuul version 4.0 is ready for an upgrade but requires some deployment changes in order to ease a transition to the next iteration with minimal disruption. In v4, it will require a SQL database connection and network connectivity from all components of Zookeeper connection. The Zookeeper connection must be made with SSL/ TLS. As mentioned earlier, Volvo is a major user of Zuul in a production environment, having built numerous software components using Zuul. Accordingly, the Volvo Polestar 2 One Pedal Drive was built with Zuul.
OpenInfra Labs is a community practice for universities to run open source software in production environments and publish reproducible stacks of existing and emerging workloads to support the university community. The ultimate goal is to better understand how open infrastructure projects operate at scale in production. Boston University, Harvard University, MIT, Northeastern University and the University of Massachusetts are part of the university community.
Magma, which was first developed by Facebook to provide a software-centric distributed mobile packet core and tools for automating network management, will be managed under a neutral governance framework at the Linux Foundation.
Kata Containers 2.0, which was released last October, is touted to provide stronger workload isolation without the overhead of running it in VMs. At the core, Kata agent has been rewritten in Rust, which in turn leads to a tenfold improvement in the memory footprint – from 11MiB to 300KiB. This rewrite also replaced the gRPC agent protocol with ttRPC to minimize the complexity of network device management. Despite the Intel connection, Kata Containers is compatible with the Open Container Initiative specification and the Kubernetes container runtime interface. Using the new Virtual Machine Manager hypervisor, it provides users with a choice of virtualization stacks – the traditional cloud-based or serverless approach. As examples, companies in the fintech, internet services and technology arena are running production workloads using Kata Containers. Ant Group indicates that part of its deployment has been upgraded to version 2.0. In terms of scale, Kata Containers is running on thousands of nodes and over 10,000 CPU cores. Likewise, IBM Cloud provides a secure environment for its CD Pipeline service with Kata Containers. Finally, Baidu utilizes Kata Containers to deploy function as a service, container as a service and edge computing.
The IT infrastructure landscape has evolved rapidly in recent years, driven by the growing availability of technologies and services. Companies of all kinds have little choice but to keep pace with changing business requirements. For the most part, technology providers and enterprise IT are motivated to work collaboratively on technology innovation. Such collaboration has contributed to and will continue to shape the development agenda of open-source-based infrastructures. Key development imperatives include:
Hardware diversification is trending up. While x86 servers have been predominantly used for scale-out infrastructures, the low power approach of ARM architecture is starting to draw the attention of technology providers and internet heavyweights. With Nvidia's proposed acquisition of Arm, providers may want to evaluate their hardware strategy if they haven't done so. Along with the growing use of specialized accelerators like GPU and FPGA for workloads such as data modeling, hardware diversification is likely to intensify.
One size doesn't fit all when it comes to workload deployment. The advent of IoT is changing the way in which a data infrastructure is designed and deployed. Instead of bringing all workloads and applications to a massive centralized datacenter, edge cloud deployment is gaining ground for IoT use cases. From micro-scale to hyperscale computing, deployments will vary greatly in size and number, regardless of where they operate – on-premises datacenter, cloud or edge infrastructure.
Industry collaboration is inevitable for new workloads and expanded use cases. The recent announcement regarding the management of Magma using a neutral-governance framework to facilitate industry-wide collaboration seems to be a logical step in the right direction. Whether industry collaboration will accelerate the proliferation of production use cases remains to be seen, but it helps define a common framework for different open source communities such as OpenInfra Foundation and OpenAirInterface Software Alliance to work in concert.