Omdia view
Summary
Arm has hit the financial news headlines ahead of its IPO, with the likes of Forbes and the Financial Times writing detailed analyses of the company’s recently published financials. As one would expect, some voices in the media highlight red flags while others highlight opportunity. What is not discussed in enough detail is Arm’s strengthening position in the data center. First, large cloud service providers are ramping up the use of Arm’s CPUs. Second, a huge upcoming wave of AI inference is highly likely to run on workload-optimized Arm-based co-processors. This is already underway at Amazon, where Arm’s technology is used to run the company’s software-defined network. Third, in China, Arm is on track to become the leading processor architecture in the data center as various geopolitical legislation limits access to Intel, AMD, and NVIDIA processors.
Arm has won a beachhead in the data center
In 2017, Omdia predicted that the establishment of interoperability and standardization of Arm processor design, coupled with a proliferating vendor ecosystem, would enable Arm to get a beachhead in the data center. Amazon’s Graviton and Ampere Computing’s CPUs are the manifestation of this prediction.
Our follow-up articles on the topic showcased specific domain expertise forming around Arm, with workload-optimized processors becoming commonplace. Examples of this trend include NVIDIA’s HPC and AI-optimized Grace processor and edge-optimized Jetson processor and Amazon’s networking and storage-optimized Nitro processor.
In 2021, we followed up our competitive analysis on Arm when it launched its v9 architecture at its Vision Day event. Better parallel computing performance for workloads like AI and HPC and improved security features stood out as the right development areas. We concluded that the tangible roadmap Arm presented at the launch of Arm v9 reassured key partners as they made future adoption decisions. For us, this event cemented the fact that Arm’s success in entering the data center is not a fluke but rather that it is here to stay.
The software ecosystem for Arm servers is surprisingly mature
The final key hurdle in Arm’s path of ascent in the data center was software ecosystem maturity, and Omdia believes this has been overcome. In 2022, we ran a series of end-user and software vendor interviews to better understand the maturity of the Arm software ecosystem and found that a stable and tested software ecosystem has been established. Of the 10 software developer companies we interviewed, only one indicated that it does not have a version of its software for Arm servers. A significant factor for this was a set of focused engagements between Amazon and software developers, where the former funded the resource overhead required to support two software binaries. This would explain why dozens of companies, including Netflix, have adopted cloud instances (infrastructure as a service) based on Amazon’s Graviton CPU.
Arm’s position in the data center will only grow stronger
We expect over a third of the servers shipping in 2027 to run Arm-based CPUs. All the hyperscale cloud service providers—Amazon, Microsoft, Google, and Meta—are using at scale or developing their own in-house Arm-based CPUs. Those cloud service providers without the capacity to develop Arm-based CPUs in-house (i.e., Oracle) have deployed Ampere Computing’s Arm-based CPUs at scale.
Simultaneously, the use of Arm-based programmable Ethernet adapters (that is, DPUs) to compute network and storage applications and security policies will become a de facto standard across the data center industry. They are already a key part of cloud service providers’ IT architecture.
A dramatic change in Arm’s fortunes could come if its architecture becomes the norm for AI inference-optimized processors. This is not an unlikely scenario. The rapid adoption of ChatGPT has already fueled an investment spree at cloud service providers and enterprises. Most of this centers on the buildout of AI training capacity. However, the ecosystem for large foundational models is fast maturing, with the fine-tuning of models on the agenda for many AI developers.
We can already see the next wave of AI investment coming, and it will target inference. The focus there will not only be on the speed of computing but also on cost and energy efficiency. Today, the cost per query/inference at ChatGPT is several times higher than the revenue from advertisers and subscribers. AI inference computation needs to follow in the footsteps of Google’s search engine, which runs on a highly optimized, workload-specific processor, with each query costing several times less than the revenue Google is able to recognize from its advertisers. Luckily for Arm, its architecture is well suited to workload optimization. It also already has a proven track record in this area, given its wide adoption for networking, security, and storage applications.
Appendix
Further reading
“Can ARM get a beachhead in the data center?” (April 2017)
“NVIDIA acquires ARM to bring AI to the edge” (September 2020)
“Arm v9: The next Arm offensive” (April 2021)
“An icy road for Intel amid Grace bombshell” (April 2021)
Author
Vlad Galabov, Research Director, Cloud and Data Center
