Be prepared: the COVID-19 complexity storm is brewing
March 22 2021
by Owen Rogers, Jean Atelsek
As our way of life has changed as a result of the pandemic, cloud has provided us with at least some normality through online streaming, collaboration and education. But, as we discuss in our report, there is a risk that this rapid adoption and scaling of cloud could have negative effects. We urge enterprises to take stock of what they have achieved in the past year, and to focus now on what needs to be done to manage it.
The 451 Take
Because of a perfect storm of factors, there is a risk that cloud complexity may spiral out of control. The adoption of scalable cloud services as a result of COVID-19 has created larger IT estates with more interactions to manage, and the need to rapidly adapt to pandemic conditions has put implementation at the forefront, and optimization and ongoing management on the back burner. Unfortunately, challenging economic conditions will make cost savings a priority, and enterprises are likely to squeeze costs wherever possible on an ongoing basis. Custom silicon, being pushed by the hyperscalers, is one means of saving, as are ongoing price cuts, but enterprises will need to keep track and change as new capabilities are released. Hybrid and multicloud give enterprises more options, but also more workloads to manage, secure and optimize. Organizations need to make sure they are using the right cloud services, managed in the right way and at the right price, on an ongoing basis to prevent complexity spiraling out of control and impacting business objectives. Tools can automate optimization on an ongoing basis, but they can't help optimize based on business objectives – human expertise will always be needed to understand the context of IT use.
Although few positives emerged from the global struggles of 2020, one (relatively) bright spot was the real-world validation of the resilience of the cloud operating model for IT. As businesses shifted millions of employees to remote working to comply with COVID-19 stay-at-home orders, the hyperscalers and other cloud providers kept up a steady pace of service innovation and portfolio growth. In fact, cloud services in large part were enablers of this transition, with virtual desktop infrastructure (VDI), videoconferencing applications and streaming services ably weathering an unprecedented surge in traffic. Cloud's ability to flex usage in response to changes in volume served a dual role, accommodating surging demand for at-home services while conserving costs for industries (e.g., travel and hospitality) whose business activity plummeted. But although cloud has been an enabler for change, we urge enterprises to take stock of what they have achieved in the past year, and to focus now on what they need to do to manage it.
COVID-19 has stimulated increased complexity in two ways:
Adoption of cloud has increased as organizations take advantage of on-demand scaling to meet demand, to address changing business models, and to aid remote working. More than half (54%) of enterprises have increased their proportion of workloads in the cloud, according to a recent report.
The need to make changes to business models to capitalize and protect against the economic impact of COVID-19 has meant applications have been accelerated onto the cloud. Often, other plans have been deprioritized to move resources to more pressing projects, but sometimes this is at the detriment of proper planning for developments and optimization. One in five enterprises have canceled DevOps projects, for example, due to the pandemic, according to a 451 Research survey.
Drivers and Risks of Growing Cloud Complexity
451 Research 2020
But the cloud market was already facing complexity long before COVID-19 came about:
The desire to 'sweat the assets' of existing IT investments as a cost-cutting measure while using more nimble and scalable resources for new workloads means that hybrid deployments are now the rule: 59% of enterprises are pursuing a hybrid approach to IT, using both public and private clouds, according to our 2020 survey.
Add to this the continued rapid release of new services – ranging from raw infrastructure building blocks to fully managed SaaS and multicloud control planes – as well as hyperscaler regions, and businesses are faced with a huge number of decisions to make and choices to navigate.
The problem with complexity is that it doesn't resolve itself. The probability that the house of cards will eventually fall down, over a long enough time frame, is certain; the probability that a house of cards will spontaneously organize itself into a structure, over that same time frame, is near zero. Entropy is the property that describes this disorder as a system flows from a state of order into a state of disorder. In 2018, we identified three risks in the form of laws that describe how the scale of cloud makes it more susceptible to complexity due to its nature to flow into a state of disorder.
Law of Cloud Scalability: A cloud application left unmanaged will tend toward greater resource consumption over time.
Law of Cloud Entropy: Increasing scale left unmanaged tends toward increasing disorder, complexity and fragility.
Law of Cloud Complexity: The longer the period between resolution of disorder, the more effort required to resolve that disorder.
These laws don't just apply to compute infrastructure. Whether you're talking about objects in a storage bucket, records in a database, data in a machine learning model, or sensors in an IoT deployment, it is more likely that the number of things to keep track of will increase over time rather than decrease. The result is that disorder of cloud estates will be compounded by COVID-19, and costs and resource use are likely to spiral. This is unfortunate because the economic downturn precipitated by COVID-19 is likely to force cost savings upon enterprises. Already, 49% of IT decision-makers say belt-tightening has assumed higher priority than a year ago due to pandemic conditions. Technical complexity (39%), cost (33%) and conflicting processes or methodologies (32%) were named as top challenges by IT decision-makers in our 2020 survey.
But there are some inhibitors that might work in enterprises' favor to lower costs.
This acceleration in cloud adoption coincided with the sharpest annual drop in compute costs since the CPI started tracking VM prices in 2015. The CPI's general-purpose compute benchmark fell by 8% in the US East region, while the compute- and memory-optimized VM benchmarks recorded even steeper drops (15% and 10%, respectively). The primary driver of these declines was the rollout of processors with better price/performance ratios, enabling more computing to be done for less.
Cloud providers continue to cut prices across their portfolios, beyond compute. The CPI tracked over 4,000 price cuts of greater than 10% to non-compute services in 2020.
However, these inhibitors will likely have a minimal effect – for many businesses, these reductions will be swamped by increased usage.
This is a vicious cycle. Deployment of more resources and services leads to increased complexity and higher costs. The longer it takes to optimize this complexity to deliver business value, the more complex the IT estate becomes. And resolving this complexity isn't a one-time process – it requires constant reassessment and optimization. Tools, yes, but human expertise, too.
Enterprises have quite correctly taken advantage of cloud to address changing and tough times. But now they must check that they have implemented their changes in a way that is manageable in the longer term. The longer the period between resolution of disorder, the more effort required to resolve that disorder. In other words, they must adjust their roadmaps to ensure that the most pressing matters are dealt with first – actions taken now to apply discipline and governance to cloud deployment will help determine the shape and speed of a business's recovery. Those without the engineering resources to affect this transition internally can use professional and managed services to stay on the path to modern and automated IT transformation. This isn't just a matter of automating and optimizing, it's about building a process to ensure there are enough IT resources (and at a high enough level of innovation) for future needs of the business. The longer this is put off, the bigger the risk that complexity will impact cost, security, availability and, ultimately, business objectives.