DARPA Gamebreaker Aims to Train Military AI Systems on Open World Video Games

The US defense research agency is looking to use video games to train an AI capable of “developing winning warfighting strategies.”

DARPA, best known for its involvement in the creation of the Internet, partners with private companies and research institutions to develop cutting-edge technologies to ensure the US stays ahead of rival superpowers.

Break the game

In a notice on the federal procurement site SAM.gov, DARPA detailed ‘Gamebreaker,’ an ambitious project aiming to develop artificial intelligence programs that are able to automate video game balance and work out how to destabilize the simulation.

During the development of open world games (where the player can move anywhere on the game map, rather than being forced to go down a prescribed ‘linear’ path), programmers spend a significant amount of time balancing countless variables. 

In, say, a warfighting game, effort is made to ensure that a single overpowered weapon can't completely destablize a match between two or more competitors. “Elements are buffed (performance increases) or nerfed (performance decreases) to achieve game balance,” DARPA explains in a Gamebreaker document. In a perfectly balanced game, either of the two equally skilled players would win approximately 50 percent of the time.

This careful fine-tuning is done during development, after initial 'beta' testing before the game releases, and via regular updates after the title has released, in a laborious and manual process.

With Gamebreaker, DARPA hopes to develop an AI system that can analyze existing open-world video games “to quantitatively assess game balance, identify parameters that significantly contribute to balance, and to explore new capabilities/tactics/rule modifications that are most destabilizing to the game or simulation.”

This, the document explains, could include the introduction of a new weapon, capability, or rule change, and the AI system must be able to predict the outcome of the change.

DARPA explains: “In future conflicts, DoD investment is designed to maximize imbalance to create an advantage or to seek equilibrium when an adversary is seeking an advantage. 

“New AI algorithms inspired by Gamebreaker could help develop winning warfighting strategies when the adversary’s objectives - i.e. the ‘rules of the game’ - are not clearly known. By exploiting game balance, Gamebreaker addresses an existing gap in AI and data analytics research as applied to current wargaming and simulation.”

Project teams seeking DARPA funding will get to choose the game they plan to work on, as well as a secondary title that the AI system must be compatible with, nine months later. The two games must be distinctly different, “for example, one might begin with a fantasy game and then apply the method to a sports game,” but the first game should be a real-time strategy title - with a preference for titles that are “representative of a campaign-level military engagement.”

To test the AI system further, applicants will be expected to “generate predictions to Government-provided independent perturbations to avoid trivial or absurd solutions and impose natural constraints to control scope.” Presumably, this stops the AI model simply inventing a doomsday weapon capable of immediately winning the match, and instead focuses on achievable innovations that could prove useful to DoD wargames.

The Mosaic strategy

In 2018, after wargames began to show Russian and Chinese forces exacting devastating losses on US troops (when fighting within Russian and Chinese spheres of influence), DARPA unveiled its new concept for modern warfare: Mosaic.

Mosaic aims to leverage the power of information networks, advanced processing, and disaggregated infrastructure to win conflicts with near-peer competitors. 

Current military approaches can be seen like a puzzle, DARPA argued, where forces need several interlinking technologies (troops, tanks, drones, etc.) to work together perfectly, in military maneuvers that are carefully planned, supported by extensive logistics networks, as part of a brittle tactical approach. 

"They are exquisitely engineered to fit into a certain part of the picture and one part only," Dr. Thomas J. Burns, former director of DARPA’s Strategic Technology Office, said in 2018. "You can’t pull it out and put in a different puzzle piece. It won’t fit.”

If just one of those puzzle pieces is not available, the whole thing risks falling apart. Instead, Mosaic sees the pieces as diverse and fluid, full of weapon and sensor platforms that a commander could instantly turn to, slotting one in when another is out of action.

Burns described a potential ground battle where a commander would send an unmanned aerial vehicle or a robot ahead of the main battle force. It might spot an enemy tank. The unmanned system would pass the coordinates back, which would be relayed to a non-line-of-sight strike system in the rear; it would launch its munitions and take out the target.

“It sounds like it should be something very doable, but it’s not right now,” Burns said. “The interfaces are not made to communicate that kind of information and the Army doesn’t have air and ground vehicles that it can send forward.”

Mosaic introduces significantly more complexity to the battlefield - a factor that DARPA hopes will overwhelm enemy strategies - but current wargames have shown the complexity is more likely to overwhelm the US commanders, forcing them to revert to traditional approaches. 

For Mosaic to prove successful, advances are required not just in the data that weapons systems collect, and in how they communicate, but also in how the data is processed and reacted to. Enter artificial intelligence.

“Complex, multi-domain modeling and simulation (M&S) environments currently under development by several DARPA programs aim to create a useful ‘Mosaic’ model within which to experiment on new warfighting constructs using distributed, adaptive, all-domain force composition, tactics, and strategies - yet these models do not currently exist,” DARPA says in its Gamebreaker document.

“It is reasonable to assume, however, that once these simulation environments reach maturity, an ‘AlphaMosaic’ equivalent will be capable of searching for optimal strategies and tactics in the same way AlphaGo and AlphaStar agents have already proven effective in exploiting their respective game environments.”

Google’s DeepMind research division developed AlphaGo, which was able to beat the world’s best Go player, and more recently, AlphaStar, which obtained the title of grandmaster by defeating humans in the popular real-time strategy video game StarCraft II. Both were heralded as major advances in AI, but an equivalent AlphaMosaic could prove a much greater challenge.

Neither of Google's project explored “account modifications to the construct of the game itself,” DARPA notes. “While AI techniques have demonstrated the ability to master games and models of increasing complexity, there has been almost no research in the application of AI to game modification.”

Work on Gamebreaker, which hopes to solve at least a fraction of this challenge, is expected to begin this month. The project is headed by Lt Col Dan 'Animal' Javorsek, a former experimental test pilot for stealth and combat fighter aircraft.

"Mosaic models that are being built and will soon exist ultimately will have algorithms that will be able to win them, and our goal is to make these winning AI algorithms useful beyond just finding strategies in that particular game," Javorsek said in an unlisted DARPA video.

"The benefit is that in the real world, we can in fact modify the game in many senses."

Update: Earlier this week, DARPA released a promotional video for Gamebraker: https://www.youtube.com/watch?v=tt5nxCi423U