Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!
September 18, 2023
Chipmaking giant Nvidia shipped a whopping 900 tons of its flagship H100 GPU in the second quarter due to a “bold acceleration of demand,” according to a new report from sister research firm Omdia.
Its latest “Cloud and Data Center Market Snapshot” report said "an avalanche" of GPUs is arriving at the data centers of hyperscalers – to the detriment of server shipments. Deliveries of servers are down by one million units or 17% from the prior year. In July, it fell for the first time in 15 years.
Instead, Omdia reports a "bold acceleration” towards AI GPUs, with the price tag of AI hardware driving up the average price of servers by over 30% year-over-year. This “dramatic shift” in the server mix towards highly configured servers led Omdia to forecast the market size to be $114 billion, up 8% year-over-year.
Nvidia is the company set to benefit the most, with Omdia reporting that 22% of its $13.5 billion second-quarter revenue was driven by a single customer, likely Meta. Omdia also expects Microsoft, Google and “a few other top 10 cloud service providers” to become “priority clients” of Nvidia.
Demand for AI hardware will continue until the first half of 2024, especially given that chip foundry TSMC is increasing the packaging capacity required to build the H100.
Demand for AI hardware also saw a vendor and even the U.S. government opt to ‘rent out’ their existing infrastructure – such as discounted access to the Perlmutter supercomputer or Hugging Face’s new Training Cluster as a Service.
However, demand for AI hardware does not mean AI usage has likewise seen rapid growth. "The rapid investment in AI training capabilities that we're currently experiencing should not be confused for rapid adoption," Omdia's report read.
Omdia expects the AI wave to continue until 2027. In five years, it further forecasts that the number of large language models will be as numerous as consumer-facing apps.
AI will turn to more specialized uses, with broader models taking a back seat. Industry-specific models akin to Med-PaLM, Bio-GPT, BioBert, and PubMedBert will be the focus but will be built by fine-tuning the broad, multimodal frontier models such as ChatGPT or Claude.
However, the need for AI servers should decrease over time even if AI adoption by enterprises and other organizations increase. That is because domain-specific AI models have “significantly fewer” parameters, dataset size, tokens and epochs, according to the research firm.
Read more about:ChatGPT / Generative AI
Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.
You May Also Like