Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!
November 27, 2023
This translates to $14.5 billion in data center revenue for the quarter, nearly quadrupling from the same quarter a year ago, wrote Vlad Galabov, director of Omdia's cloud and data center research practice, and Manoj Sukumaran, principal analyst, data center computing and networking, in their latest Market Snapshot Report.
Most of the Q3 GPU server shipments went to hyperscale cloud service providers. Meta was one of its largest clients while Microsoft also ordered massive numbers of H100 GPUs, likely to power their increasing roster of AI products or Copilots.
Google, Amazon, Oracle and Chinese tech giant Tencent were the next highest on the list, though the latter will not be able to obtain H100s under strict export restrictions imposed by the Biden administration.
The Omdia analysts expect Nvidia to cross the half-a-million mark in GPU shipments by Q4 of this year as the demand for hardware continues.
Meanwhile, server manufacturers such as Dell, Lenovo and HPE have not been able to get their H100 server orders filled yet, due to a lack of GPU allocation from Nvidia, with the wait time about 36 to 52 weeks.
Galabov and Sukumaran expect the server market to be worth $195.6 billion in 2027 – more than doubling from a decade ago.
Server processor and co-processor contributions are driving that growth as companies move towards hyper heterogenous computing - or application-optimized server configurations with many co-processors.
For servers running AI training and inference, the most popular server configurations for large language model training are the Nvidia DGX server, configured with eight H100/A100 GPUs, and Amazon’s servers for AI inference, with 16 custom-built co-processors (Inferentia 2).
For video transcoding servers with many custom-built co-processors, the most popular ones are Google's video transcoding server with 20 VCUs (video coding units) and Meta's video processing server with 12 Scalable Video Processors.
“We expect this trend to only grow as demand for some applications has matured to the scale that makes it cost-efficient to build an optimized custom processor,” the authors wrote. “Media and AI are the early benefactors of hyper heterogeneous computing, but we expect other workloads like databases and web services to see a similar optimization push.”
Omdia’s report states that the increase of highly configured servers for AI is driving momentum in data center physical infrastructure.
Rack power distribution revenue in the first half of the year was 17% ahead of last year, trending slightly ahead of Omdia’s forecasted 14% growth. Data cementer thermal management revenue is on track for 17% growth in 2023, fueled by higher rack densities requiring liquid cooling solutions.
“With a ramp of professional services for generative AI enabling broad enterprise adoption in 2024 and beyond, the only thing that can curb the current rate of AI deployment is power availability," the authors said. “Our forecast is unrestricted by financing availability, and it is worth noting that we’ve seen an interesting trend where companies can use sought-after GPUs as debt leverage.”
Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.
You May Also Like
Generative AI Journeys with CDW UK's Chief TechnologistFeb 28, 2024
Qantm AI CEO on AI Strategy, Governance and Avoiding PitfallsFeb 14, 2024
Deloitte AI Institute Head: 5 Steps to Prepare Enterprises for an AI FutureJan 31, 2024
Athenahealth's Data Science Architect on Benefits of AI in Health CareJan 19, 2024