Trends in Enterprise AI Adoption

By Nick Patience and Rachel Dunning

The adoption of AI is driving changes in the infrastructure that underpins it, both in terms of technical changes – from chips up to the cloud (and everything in between) – and in the way organizations plan and purchase such infrastructure. Our AI & Machine Learning, Infrastructure 2021 survey includes direct input from 700 AI decision-makers across the US and UK. Diving into this insight reveals many trends and developments in the market, including an ever-increasing adoption rate, a shift toward both the cloud and the edge, and several hurdles that need to be overcome to get AI projects from concept into production. As ever, all these factors vary by industry and, to a certain extent, by country. This report highlights some key findings ahead of our full Advisory Report.

Although much of the data that fuels machine learning comes from edge devices of various types, our data shows that ML is increasingly being done in the cloud, and attention is shifting toward tools that complement that scenario. AI infrastructure is seen as something distinct from overall IT infrastructure (be it cloud, on-premises or hybrid), and budgets are set differently and are expected to grow, having been accelerated by the COVID-19 pandemic. Production pipelines will become more efficient with time, but right now the disconnect between IT and business users may be slowing things down, as the latter have a harder time getting approval to start projects and play a smaller role in spending and approval decisions. Organizations are becoming more aware of ESG concerns regarding AI and ML, including whether AI is being used for immoral or unethical purposes, the environmental impact of AI, and government regulation of the sector – the last of which is higher in the US than the UK.

Key takeaways from the survey

Adoption demographics are gradually shifting from planning and proof-of-concept (POC) stages toward production. In 2020’s AI & Machine Learning, Infrastructure survey, 37% of adopters had AI in production, compared with 42% this year. This suggests the market is maturing.

AI infrastructure is planned differently than other IT infrastructure. Two-thirds (66%) of organizations say they have an AI infrastructure purchasing strategy that is separate from the rest of their infrastructure purchasing plans. This percentage is even higher for those that plan their AI infrastructure in advance (75%) versus those that scale on demand (56%). AI infrastructure is different from regular infrastructure in many ways, and therefore requires different kinds of planning and resource allocation.

Public cloud dominates AI workload venues. Last year’s trend toward public cloud has only been further solidified in this year’s survey. For respondents currently using AI/ML, public cloud is the most popular primary venue for storage (51%), training (39%) and inference (37%).

Along with the cloud, containers are growing in popularity. Sixty-eight percent (68%) of organizations are using containerized environments for ML production, compared with 55% last year. However, container usage among our respondent base remains significantly more popular in the US than the UK.

Demand is rising quickly – and spending with it. On average over the past 12 months, enterprises have spent about $1.5m on AI/ML infrastructure. When asked about future demand and spending, the large majority of enterprises (76%) anticipate demands on their infrastructure increasing over the next two years, so it comes as no surprise that, over the next 12 months, 82% of enterprises plan to increase their spending as well. Targets for that spending are led by accelerators (e.g., GPUs, TPUs), whether in the cloud or on-premises.

COVID-19 has played a role in accelerating this spending increase. Sixty-four percent (64%) of organizations have increased AI infrastructure spending in response to COVID-19. Spending on AI has and will increase regardless, as demand was rising before the pandemic; however, it has served as a catalyst for both spending and adoption.

The average organization takes three months to develop and deploy an ML model from conception to production. Our data suggests that IT-led projects gain approval significantly faster (nine weeks) than projects led by non-IT or business users (12 weeks).

Many ML projects never see the light of day. On average, nearly two in five (39%) ML projects in POC are abandoned. Of all industries, financial services and telecommunications have the longest approval timelines for starting a new ML project – both regulated industries.

There is no sign of slowdown in data volume growth. On average, organizations use a total of 82 petabytes of data to build and train their models collectively, and nearly 103 petabytes of data to make predictions (also known as inference) from the models they’ve created. Seventy-nine percent (79%) of organizations expect to see an increase in the amount of data used for building and training AI models over the next year, and 67% expect an increase in volume for predictions.

That data is being collected from a wide variety of sources, both core and edge. Datacenters are in the lead (40%), but close behind come edge data sources such as smartphones and tablets (35%), industry-specific equipment (29%), supply chain devices such as shipping containers (29%), and myriad others, from door locks to drones. Industry variants include retail, dominated by point-of-sale devices (55%), and manufacturing, with factory equipment (44%) in the lead.

Real-time inferencing is both the present and the future. Forty-five percent (45%) of organizations say they do at least some degree of real-time inference now, and 45% plan to do so in the future, with only 9% sticking exclusively with batch inferencing.

Growing trust or a blind eye? Thirty-seven percent (37%) of organizations are not very concerned or not concerned at all about AI being used for immoral or unethical purposes. Based on our data, it seems that users that have the most exposure to or experience with AI are the ones most wary of its ESG impacts.