Omdia AI News: Right to be Disappointed in NVIDIA

It’s not hard to spot the biggest AI news story in the last seven days – an NVIDIA results week has become financial news up there with a Federal Reserve meeting or a big move in the oil price. NVIDIA crushed the quarter yet again, with $30bn in revenue, $26bn of that coming from data centers, and $22.6bn of that coming from its AI GPUs as opposed to networking products. Even if the ramp-up of Blackwell GPUs has been delayed, improvements through the supply chain (notably TSMC getting a second source for the silicon interposers used in CoWoS packaging) mean a wave of Hopper GPUs is going to hit the market through the rest of 2024. Lead times are down, and many of the companies who have been waiting for backorders are now getting their deliveries – which in turn implies a wave of AI products a few months further on, as around 60 per cent of them are used for model training projects.

So why did the shares sell off (well, briefly)? The trees don’t grow up to the sky. Although NVIDIA is still reporting astronomical year-on-year growth rates, the quarter-on-quarter clip has slowed significantly over the last year. The data center line grew 16% from 1Q 2024 to 2Q 2024, 22.5% from 4Q 2023 to 1Q 2024, and 26.8% from 3Q 2023 to 4Q 2023. Also, fairly soon, the first half of 2023 will drop out of year-on-year comparisons, reducing the frequency of three-digit growth numbers. That’s the sort of base effect you’d think the stock market would see through, but it might leave a mark anyway. More seriously, though, the headlong growth rates of 2023 were such that if they were somehow sustained, NVIDIA would fairly soon be bigger than the entire world semiconductor market, having in the meantime blasted through the total data center chip market.

The Omdia AI research team sees three forces in action that will eventually slow down demand for AI processors:

• Straightforward saturation - there is some evidence that the AI boom is increasing the size of the addressable market, but it’s growing as a subset of the market much faster than the market is growing, so saturation is eventually inevitable through classic innovation-diffusion logic.
• Smaller AI models becoming competitive with mega-scale general purpose models – Cohere for AI co-founder and Google Brain alumna Sara Hooker’s paper On the Limitations of Compute Thresholds as a Governance Strategy points out that the number of giant models that are outperformed by at least one <13B model is growing rapidly
• Year-on-year improvements in comparable AI models’ efficiency – looking at the models listed on the OpenLLM Leaderboard, the current generation of models is on average just under 7 per cent more efficient in terms of tokens/second for similar parameter counts and architectures.

That said, even a slowdown will leave GPUs especially and AI processors more generally as a massive share of the data center processor market. See the chart below, from the new edition of our AI Processors for Cloud and the Data Center:

Latest publications in Omdia AI

Fundraisings for quantum computing projects have exceeded the total for 2023 by mid-year, signalling a revival in the sector after what looked like something of a “quantum winter”, writes Sam Lucero in Investment in quantum computing returns to growth in 2024. Note, though, that nearly half the total came from just one deal – the Australian federal government’s $616m investment in PsiQuantum.

Chinese vendors are putting RISC-V chips into more and more Internet of Things modules in an effort to hedge against possible trade restrictions and stop sending as much of quite a low average selling price to Arm plc. Omdia expects RISC-V to account for 30% of IoT modules by 2028; shipments in the APAC region make up around half the current total. The utility sector is slowing down but industrial demand just keeps growing.  There’s more in Lian Jye Su, Shobhit Srivastava, and Andrew Brown’s IoT Modules in China.

The AI Processor M&A Tracker from Alex Harrowell and Xin Yu Lee says that AI-related mergers and acquisitions for the first half of 2024 look much like they did in 2023, with a similar pace of around a deal a month. Microsoft, so far a voracious buyer of AI startups, has stopped entirely, while another major acquirer, IBM, is keeping up buying about one software or services company a quarter.

In The art of unlocking value through smart data fabrics, Brad Shimmin looks into how InterSystems, one of those important companies you’ve not heard of, has responded to the AI revolution. Its business is running big analytical data warehouse systems, and as such you’d expect it to be vulnerable to a technology disruption here – until you find out it added “vector/tensor” as a column data type as far back as 2016. Omdia’s current AI Market Maturity survey wave finds that data collection, preparation, and governance are the top spending priorities for enterprise AI projects, rivalled only by GPUs, so you can see why we’re interested.

Curious and interesting

Training a diffusion model with a different optimizer can change its aesthetics: https://x.com/revhowardarson/status/1829978461806662066

Omdia’s AI team will soon be launching a newsletter that covers the most important AI news, the latest insights from our AI research, our featured AI research paper of the week, and a unique highlight from our collection of AI curiosities. Stay tuned!