As part of our goal to build increasingly capable and general artificial intelligence (AI) systems, we are working to create AI tools with a broader understanding of the world. This can enable the transfer of useful knowledge across many different types of tasks.

Using reinforcement learning, our AI systems AlphaZero and MuZero have achieved superhuman feats in gaming. Since then, we’ve expanded their capabilities to help design better computer chips, as well as optimize data centers and video compression. And our specialized version of AlphaZero, called AlphaDev, has also discovered new algorithms for accelerating the software that powers our digital society.

Early results have shown the transformative potential of more general AI tools. Here, we explain how these developments are shaping the future of computing — and already helping billions of people and the planet.

Designing better computer chips

Specialized hardware is essential to ensure that today’s AI systems are resource-efficient for mass-users. But designing and manufacturing new computer chips can take years of work.

Our researchers have developed an AI-based approach to designing more powerful and efficient circuits. By treating a circuit as a neural network, we have found a way to accelerate chip design and take performance to new heights.

Neural networks are often designed to process user input and generate output, such as images, text, or video. Within the neural network, edges connect to nodes in a graph-like structure.

To create a circuit design, our team proposed circuit neural networks, a new type of neural network that turns edges into wires and nodes into logic gates and learns how to connect them together.

We optimized the learned circuit for computational speed, energy efficiency, and size, while maintaining its functionality. Using simulated annealing, a classical search technique that looks one step into the future, we also tested different options to find the optimal configuration.

With this technique we won the IWLS 2023 programming competition, with the best solution for 82% of the circuit design problems in the competition.

Our team also used AlphaZero, which can see far into the future, to improve the track design by treating the challenge as a game to be solved.

So far, our research combining circuit neural networks with the reward function of reinforcement learning has produced promising results for building even more advanced computer chips.

Optimizing data center resources

Data centers manage everything from delivering search results to processing datasets. Like a game of multi-dimensional Tetris, a system called Borg manages and optimizes workloads within Google’s vast data centers.

To schedule tasks, Borg relies on manually coded rules. But at Google’s scale, manually coded rules can’t cover the variety of ever-changing workload distributions. That’s why they’re designed as a one-size-fits-all approach.

Machine learning technologies such as AlphaZero are particularly useful here: they can work on a large scale and automatically create individual rules that are optimally tailored to different workload distributions.

During the training, AlphaZero learned to recognize patterns in tasks entering the data centers, how to best manage capacity, and how to make decisions that yield the best results in the long run.

When we applied AlphaZero to Borg in experimental trials, we found that we could reduce the percentage of underutilized hardware in the data center by as much as 19%.

Compress video efficiently

Video streaming makes up the majority of internet traffic, so finding ways to make streaming more efficient, no matter how big or small, will have a huge impact on the millions of people who watch video every day.

We worked with YouTube to compress and transmit video using MuZero’s problem-solving skills. By reducing bitrate by 4%, MuZero improved the overall YouTube experience — without compromising visual quality.

We initially applied MuZero to optimize the compression of each individual video frame. We have now extended this work to help make decisions about how frames are grouped and referenced during encoding, leading to greater bitrate savings.

The results of these first two steps show a promising impression of the potential of MuZero to grow into a more general tool that helps find optimal solutions throughout the entire video compression process.

Discovering faster algorithms

AlphaDev, a version of AlphaZero, made a new breakthrough in computer science when it discovered faster sorting and hashing algorithms. These fundamental processes are used trillions of times a day to sort, store, and retrieve data.

AlphaDev’s Sorting Algorithms

Sorting algorithms help digital devices process and display information, from ranking online search results and social posts to user recommendations.

AlphaDev discovered an algorithm that increases the efficiency of sorting short sequences of elements by 70%, and by about 1.7% for sequences containing more than 250,000 elements, compared to the algorithms in the C++ library. This means that results generated by user queries can be sorted much faster. When used on a large scale, this will save a huge amount of time and energy.

AlphaDev’s hashing algorithms

Hashing algorithms are often used for data storage and retrieval, such as in a customer database. They typically use a key (e.g., username “Jane Doe”) to generate a unique hash, which corresponds to the data values ​​to be retrieved (e.g., “order number 164335-87”).

Like a librarian using a classification system to quickly find a specific book, the computer with a hashing system already knows what it is looking for and where to find it. When applied to the 9-16 byte range of hashing functions in data centers, AlphaDev’s algorithm improved efficiency by 30%.

The impact of these algorithms

We added the sorting algorithms to the LLVM standard C++ library — replacing subroutines that have been in use for over a decade. And contributed AlphaDev’s hashing algorithms to the abseil library.

Since then, millions of developers and companies have started using them in industries ranging from cloud computing to online shopping and supply chain management.

Universal tools to power our digital future

Our AI tools are already saving billions of people time and energy. This is just the beginning. We envision a future where general-purpose AI tools can help optimize the global computing ecosystem.

We’re not there yet: we still need faster, more efficient and more sustainable digital infrastructure.

Many more theoretical and technological breakthroughs are needed to create fully generalized AI tools. But the potential of these tools — across technology, science, and medicine — makes us excited about what lies ahead.