Google Custom TPUs Power Faster AI Workloads, Google I/O 2024

Google has unveiled the sixth generation of its custom hardware units, Trillium, at Google I/O 2024 this week.

Google’s Tensor Processing Units (TPUs) custom-built chips power the company’s enterprise-accessible cloud hypercomputers.

The sixth generation offers 4.7 times increased peak compute performance per chip compared to the previous TPU v5es.

Trillium TPUs boast double the High Bandwidth Memory (HBM) capacity, double the bandwidth and double the Interchip Interconnect (ICI) bandwidth over the previous TPU generation.

The chips also house the third generation of SparseCore, a specialized accelerator for processing ultra-large embeddings, commonly used to power ranking and recommendation workloads.

Google’s new custom chips are designed to power AI training workloads faster at reduced latency and lower cost. They’re also 67% more energy-efficient compared to the TPU v5e.

Trillium pods will contain up to 256 TPUs, with cloud customers able to scale their computing needs.

“Training state-of-the-art models requires a lot of computing power,” said Google CEO Sundar Pichai. “Industry demand for machine learning compute has grown by a factor of one million in the last six years and every year it increases tenfold. Google was built for this. For 25 years we've invested in world-class technical infrastructure from the cutting-edge hardware that powers search to our custom TPUs that power our AI advances.”

Related:Google Expands Gemini Lineup With Large, Small Model Updates at Google I/O 2024

Trillium will be available to Google Cloud customers in late 2024.

Dr. Andrew Rogoyski from the University of Surrey's Institute for People-Centred AI called Google’s announcement of its latest TPU a “critical enabler for offering affordable AI services in the future.”

“Increasingly AI firms are turning to development of AI hardware as the only way that they can offer sustainable AI services,” Rogoyski said. “Chip design and manufacturing is another field that only companies with the deepest pockets can afford to explore.”

Nvidia Blackwell Update

Google also provided an update on the timeline for the launch of its support for Nvidia’s new Blackwell hardware. Google Cloud will be supporting Blackwell starting in early 2025.

CEO Pichai claimed Google Cloud would be among the first cloud providers to offer Nvidia’s newest GPUs

“We are fortunate to have a long-standing partnership with Nvidia, [we’re] excited to bring Blackwell's capabilities to our customers,” Piachi said.