OpenAI teaches AI to play Minecraft using video data

New skill can be used for real-world automation tasks, researchers say

OpenAI’s DALL·E 2 recently took the internet by storm with users all over the world by being able to create weird and wonderful ‘art.’

Now, the Microsoft-backed company has taught one of its AIs to undertake one of the internet’s most popular past times, playing the Minecraft sandbox video game.

Using unlabeled video data available online, OpenAI researchers created a novel semi-supervised imitation learning method called Video PreTraining (VPT).

Essentially, OpenAI used vast amounts of videos to train a neural network to play Minecraft. The model focuses on keypresses and mouse movements, which the network learns and mimics – proving able to perform in-game actions far faster than human players.

Video games have proven to be a fruitful option for AI developers in training systems to perform tasks. From Deep Blue defeating a chess grandmaster in the 1990s to DeepMind’s unbeatable StarCraft II system, the learnings show potential for transferring machine decision-making to the physical world.

Upon publishing a paper showcasing its work, OpenAI referenced this idea and suggested the research “represents a step towards general computer-using agents” given the model’s focus on general aspects such as mouse movements.

Figure 1:

Minecraft is a video game where players are given a vast sandbox world made up of blocks to explore. Created in Java by Markus ‘Notch’ Persson, the game, released in 2011, is one of the most popular video games globally. Microsoft acquired the developer Mojang three years later.

There’s a lot to Minecraft – such as creepers, mobs, the Enderman, the Nether and griefing and where players can do essentially anything. Players use resources in the world around them to craft things from shelters to better weapons and tools.

Turning trees into diamonds

OpenAI’s researchers said they chose the popular block-based game because it is one of the most actively played video games in the world – thereby providing a wealth of video data and because the game itself has many things players can do, similar to real-world applications such as computer usage.

OpenAI’s researchers collected 270,000 hours of Minecraft-related video data, which was edited down to around 70,000 hours and used to train its VPT model. The labeled video data was used to train an Inverse Dynamics Model (IDM), which learned from occurrences in the gameplay to predict keyboard and mouse inputs. The VPT model essentially learned to predict and clone human player behaviors.

Using that data, the system was able to achieve feats such as crafting items that require intensive work, or chopping down trees to collect logs and then using those logs to craft things like tables and shelters. It can even turn trees into diamonds.

The AI-powered Minecraft player was even able to respond to questions about how to build things as the system was trained on natural language as well as code.

According to OpenAI, VPT “paves the path toward allowing agents to learn to act by watching the vast numbers of videos on the internet.”

"Compared to generative video modeling or contrastive methods that would only yield representational priors, VPT offers the exciting possibility of directly learning large-scale behavioral priors in more domains than just language.

“While we only experiment in Minecraft, the game is very open-ended and the native human interface (mouse and keyboard) is very generic, so we believe our results bode well for other similar domains, e.g. computer usage.”

OpenAI’s study is not the first looking into using AI to play Minecraft. From 2019 onwards, the MineRL Challenge has sought to teach AI to play it, with research teams conducting various building and crafting tasks.