DeepMind's AI system that can discover new algorithms

AlphaTensor is the first AI system for finding efficient algorithms used in ML applications.

Ben Wodecki

October 10, 2022

3 Min Read

AlphaTensor is the first AI system for finding efficient algorithms used in ML applications.

DeepMind, the Alphabet-owned company behind AlphaFold, has unveiled AlphaTensor, an AI system that can discover new matrix multiplication algorithms that are critical computationally to ML applications.

AlphaTensor is an agent-based system built off AlphaZero, the DeepMind AI capable of playing games such as chess and Go. The latest refinement of AlphaZero is designed with computational science in mind – built to discover new algorithms for tasks like matrix multiplication, according to a paper published in Nature.

Matrix multiplication is an algebraic operation often applied in ML use cases such as image processing and speech recognition. It’s used in several AI applications and neural networks.

"Companies around the world spend large amounts of time and money developing computing hardware to efficiently multiply matrices. So, even minor improvements to the efficiency of matrix multiplication can have a widespread impact," the company said in a blog.

Figure 1:

“AlphaTensor discovered algorithms that are more efficient than the state of the art for many matrix sizes” and its AI-designed algorithms "outperform human-designed ones, which is a major step forward in the field of algorithmic discovery."

The researchers made the process of finding algorithms into a single-player game where the board is a three-dimensional tensor or array of numbers. Through algorithm rules that specify allowed moves, the AI system figured out how to modify the tensor and "zero out its entries." The result is a "provably correct matrix multiplication algorithm."

Orders of magnitude more than Go

DeepMind said the game was "incredibly challenging," with the number of possible algorithms "much greater" than the number of atoms in the universe and the number of possible moves at each step of the game 30 orders of magnitude greater than Go.

Through reinforcement learning, AlphaTensor played the game without any knowledge about existing algorithms but improved over time until it surpassed current techniques for uncovering matrices.

"For example, if the traditional algorithm taught in school multiplies a 4x5 by 5x5 matrix using 100 multiplications, and this number was reduced to 80 with human ingenuity, AlphaTensor has found algorithms that do the same operation using just 76 multiplications,” DeepMind claimed.

The AI system was designed to work on various hardware, with DeepMind saying it can be used on Nvidia V100 GPUs and Google TPU v2s.

“These algorithms multiply large matrices 10% to 20% faster than the commonly used algorithms on the same hardware, which showcases AlphaTensor's flexibility in optimizing arbitrary objectives,” the Google-owned company said.

After focusing on matrix multiplication, DeepMind said it hopes to inspire others to use AI to guide algorithmic discovery for other fundamental computational tasks.

The release of AlphaTensor comes hot off the heels of DeepMind’s Sparrow, a dialogue agent designed to respond to users safely and sensibly.

About the Authors

Ben Wodecki

Assistant Editor

Get the newsletter
From automation advancements to policy announcements, stay ahead of the curve with the bi-weekly AI Business newsletter.