SoC can run and train AI models faster than general CPU

IBM has unveiled a new AI chip that it said can run and train deep learning models faster than a general-purpose CPU.

The prototype chip, IBM Research Artificial Intelligence Unit or AIU, is its first complete system-on-chip specially built for deep learning.

It is an ASIC-designed semiconductor that can be programmed to run any type of deep learning task, including processing spoken language or words and images on a screen.

“The workhorse of traditional computing — standard chips known as CPUs, or central processing units — were designed before the revolution in deep learning, a form of machine learning that makes predictions based on statistical patterns in big data sets,” an IBM blog post reads.

“The flexibility and high precision of CPUs are well suited for general-purpose software applications. But those winning qualities put them at a disadvantage when it comes to training and running deep learning models which require massively parallel AI operations.”

The prototype AIU features 32 processing cores and 23 billion transistors – around the same as its z16 chip for AI-powered transaction processing.

The IBM AIU is also designed to be as simple as a graphics card, capable of being plugged into any computer or server with a PCIe slot.

The chip uses IBM’s approximate computing technique to cut the amount of crunching needed to train and run an AI model, without sacrificing accuracy. AIU uses smaller bit formats to run an AI model at a rate requiring considerably less memory.

The IBM AIU has also been designed to send data directly from one compute engine to the next, which the company says can create “enormous energy savings.”

“Deploying AI to classify cats and dogs in photos is a fun academic exercise. But it won’t solve the pressing problems we face today,” IBM claims.

“For AI to tackle the complexities of the real world — things like predicting the next Hurricane Ian, or whether we’re heading into a recession — we need enterprise-quality, industrial-scale hardware. Our AIU takes us one step closer.”

IBM launched its AI Hardware Center in 2019. The company’s goal is to train and run AI models one thousand times faster by 2029.

The company hopes to share the release date for the AIU chip soon.