Google Custom TPUs Power Faster AI Workloads, Google I/O 2024

Google’s new Trillium hardware boasts double memory bandwidth while Google Cloud support for Nvidia Blackwell starts in 2025

Ben Wodecki, Jr. Editor

May 15, 2024

2 Min Read

Google has unveiled the sixth generation of its custom hardware units, Trillium, at Google I/O 2024 this week.

Google’s Tensor Processing Units (TPUs) custom-built chips power the company’s enterprise-accessible cloud hypercomputers.

The sixth generation offers 4.7 times increased peak compute performance per chip compared to the previous TPU v5es.

Trillium TPUs boast double the High Bandwidth Memory (HBM) capacity, double the bandwidth and double the Interchip Interconnect (ICI) bandwidth over the previous TPU generation.

The chips also house the third generation of SparseCore, a specialized accelerator for processing ultra-large embeddings, commonly used to power ranking and recommendation workloads.

Google’s new custom chips are designed to power AI training workloads faster at reduced latency and lower cost. They’re also 67% more energy-efficient compared to the TPU v5e.

Trillium pods will contain up to 256 TPUs, with cloud customers able to scale their computing needs.

“Training state-of-the-art models requires a lot of computing power,” said Google CEO Sundar Pichai. “Industry demand for machine learning compute has grown by a factor of one million in the last six years and every year it increases tenfold. Google was built for this. For 25 years we've invested in world-class technical infrastructure from the cutting-edge hardware that powers search to our custom TPUs that power our AI advances.”

Related:Google Expands Gemini Lineup With Large, Small Model Updates at Google I/O 2024

Trillium will be available to Google Cloud customers in late 2024.

Dr. Andrew Rogoyski from the University of Surrey's Institute for People-Centred AI called Google’s announcement of its latest TPU a “critical enabler for offering affordable AI services in the future.”

“Increasingly AI firms are turning to development of AI hardware as the only way that they can offer sustainable AI services,” Rogoyski said. “Chip design and manufacturing is another field that only companies with the deepest pockets can afford to explore.”

Nvidia Blackwell Update

Google also provided an update on the timeline for the launch of its support for Nvidia’s new Blackwell hardware. Google Cloud will be supporting Blackwell starting in early 2025.

CEO Pichai claimed Google Cloud would be among the first cloud providers to offer Nvidia’s newest GPUs

“We are fortunate to have a long-standing partnership with Nvidia, [we’re] excited to bring Blackwell's capabilities to our customers,” Piachi said.

About the Author(s)

Ben Wodecki

Jr. Editor

Ben Wodecki is the Jr. Editor of AI Business, covering a wide range of AI content. Ben joined the team in March 2021 as assistant editor and was promoted to Jr. Editor. He has written for The New Statesman, Intellectual Property Magazine, and The Telegraph India, among others. He holds an MSc in Digital Journalism from Middlesex University.

Keep up with the ever-evolving AI landscape
Unlock exclusive AI content by subscribing to our newsletter!!

You May Also Like