Cloud Price Index: AWS, IBM and Microsoft all win in the battle for IoT cost-efficiency
June 30 2021
by Owen Rogers
There are many reasons for choosing an IoT platform beyond just cost, and IoT platforms vary in terms of features and capabilities. However, it would be naive to say prospective customers aren't concerned about costs. Buyers of cloud IoT platforms must wrestle with nine different factors to understand their total costs. In a recent report, we used machine learning techniques to build a decision tree that distills this complexity down to just three simple criteria. AWS, Microsoft and IBM can all be cheapest in different circumstances, depending on the size of the deployment and the amount of data.
The 451 Take
The decision tree provides a quick reference that is good for understanding cost in the IoT cloud platform market. By examining just three pricing metrics, an assessment of the cheapest provider can be made with 80% accuracy. Most establishments today are using a limited number of IoT devices, with limited data transfer from device to cloud and back again. Granular pricing models, such as those provided by AWS, appear to have a cost advantage here. For deployments with more devices and more data transferred, fixed capacity models, such as IBM's, have a cost advantage. In the middle ground, fixed models charged in more granular increments, such as Azure's model, can be the cheapest. Choosing the most effective provider for a specific scenario can help cut costs, and data transfer can be reduced to some degree by processing more data at the edge and only sending to the cloud messages that must be stored or acted upon centrally. A key consideration for enterprises should be their future plans. We've shown that a relatively small change in number of devices or data transferred can change the lowest-cost provider. Enterprises need to look at their plans today and evaluate how their needs may change in the future.
Cloud IoT platforms, although similar in capability, vary widely in terms of how they are priced and billed. For a cost comparison, nine metrics must be considered, which puts a huge challenge on enterprise decision-making when seeking best value. Although reliability is the key vendor consideration, value for money is a table-stakes criterion.
There are too many pricing parameters to perform a simple comparison or to define a standard 'basket of goods' benchmark per the Cloud Price Index. We decided to perform a Monte Carlo simulation to look at a wide range of possibilities, and then use a machine learning decision tree classifier to best unravel these possibilities. For each cloud provider, we coded a Python script that produced an output cost based on several input parameters related to the usage of the IoT platform. The simulation script determined the cost of 10,000,000 scenarios by randomly selected configurations of these nine pricing metrics, all of which have an impact on cost, and therefore impact which provider is cheapest. We considered only list pricing published to provider websites, our view being that this would be the daunting challenge an enterprise in early-stage procurement would be faced with. And if providers believe their pricing is good value, why not make it public?
The aim of the decision tree is to identify, in general, which pricing parameter has the biggest impact on price-competitiveness, and in which situations is one provider likely to be cheaper than another.
In the decision tree below, the grey boxes represent the questions regarding the scenario and the arrows represent the answers to those questions. The pie charts show what percentage of scenarios within those bounds are cheapest by provider. For example, if the number of messages per device per day is less than 10 messages per minute per device (13,000 per day), and the number of devices is less than 2,000, then AWS is cheapest in 76% of scenarios analyzed. The classifier has distilled all the complexity of the multiple pricing metrics into a two-step decision tree, which is correct in 80% of scenarios. Note that it is possible to add more steps to increase accuracy, and with five steps this accuracy increases to 93%. However, with additional steps, the complexity of the decision tree increases. We feel 80% accuracy is a good balance between accuracy and accessibility.
What does the tree tell us? For smaller deployments of less than 2,000 devices, each transferring less than 10 messages per minute, Amazon Web Services (AWS) is found to be cheapest in 76% of scenarios tested due to its granular pricing method.
For high data transfer requirements, IBM leads the pack. If the rate of messages is greater than 10 messages per minute, IBM is cheapest in 82% of scenarios due to its unlimited data transfer model. This increases to 89% if we consider cases where message sizes are greater than 8KB.
Microsoft Azure sits in the middle ground. Enterprises with requirements of more than 2,000 devices, but with higher message rates, might find Azure cheaper than AWS. Similarly, enterprises that are sending more than 10 messages a minute may find Microsoft Azure cheaper than IBM – but only when the message size is less than 8KB.
The decision tree shows us that, although there are nine pricing dimensions to be concerned with, the cheapest cloud provider for a given scenario is primarily determined by just three – the number of devices, the average message size and the number of messages. We can further consolidate our metrics to two by multiplying average message size by number of messages to get the average daily data transfer. In our simulation, we separated message size and number of messages, but we can multiply them together here to better visualize what is happening in practice.
In the figure below, each color represents the cheapest provider for a specific combination of number of devices and total data transfer per device per day.
Most industry enterprise IoT deployments are limited in size today. The average number of industrial devices per establishment today is just 218, with average transfer to the cloud at just 0.8GB per day, according to 451 Research. Looking at this point on the above figure suggests that the more granular pricing models provided by AWS and Azure may be less expensive for most companies that have limited requirements. However, for more widespread deployments, IBM may have the upper hand.
The differences in provider cost models aren't trivial. For instance, when AWS is cheapest, on average, the most expensive competitor is 20x the cost of AWS. When IBM is cheapest, on average, the most expensive competitor is 8x the cost of IBM.
To squeeze costs, it's difficult to reduce the number of sensors in a deployment without affecting the performance, resiliency and capability of the application. However, data transfer can be reduced to some degree by processing more data at the edge and only sending to the cloud messages that must be stored or acted upon centrally.
The decision tree developed shows accuracy of 80%. For 100% accuracy, enterprises must perform their own assessments for their specific use cases, and consider other factors such as discounts, capabilities and use of other cloud services beyond IoT. Buyers must also consider how their needs will change over time as different circumstances affect which provider is cheapest.